Social
Conversational AI

A quick guide to Amazon’s papers at ACL 2024

Work on large language models predominates, with a particular focus on model evaluation.

By Staff writer
August 12, 2024
5 min read
Share

Like the field of conversational AI in general, Amazon’s papers at this year’s meeting of the Association for Computational Linguistics (ACL) are dominated by work on large language models (LLMs). The properties that make LLMs’ outputs so extraordinary — such as their linguistic fluency and semantic coherence — are also notoriously difficult to quantify; as such, model evaluation has emerged as a particular area of focus. But Amazon’s papers explore a wide range of LLM-related topics, from applications such as code synthesis and automatic speech recognition to problems of LLM training and deployment, such as continual pretraining and hallucination mitigation. Papers accepted to the recently inaugurated Proceedings of the ACL are marked with asterisks.

Code synthesis

Fine-tuning language models for joint rewriting and completion of code with potential bugs
Dingmin Wang, Jinman Zhao, Hengzhi Pei, Samson Tan, Sheng Zha

Bug injection.png
Obtaining buggy partial code via bug injection. From “Fine-tuning language models for joint rewriting and completion of code with potential bugs”.

Continual pretraining

Efficient continual pre-training for building domain specific large language models*
Yong Xie, Karan Aggarwal, Aitzaz Ahmad

Data quality

A shocking amount of the web is machine translated: Insights from multi-way parallelism*
Brian Thompson, Mehak Dhaliwal, Peter Frisch, Tobias Domhan, Marcello Federico

Document summarization

The power of summary-source alignments
Ori Ernst, Ori Shapira, Aviv Slobodkin, Sharon Adar, Mohit Bansal, Jacob Goldberger, Ran Levy, Ido Dagan

Hallucination mitigation

Learning to generate answers with citations via factual consistency models
Rami Aly, Zhiqiang Tang, Samson Tan, George Karypis

Intent classification

Can your model tell a negation from an implicature? Unravelling challenges with intent encoders
Yuwei Zhang, Siffi Singh, Sailik Sengupta, Igor Shalyminov, Hwanjun Song, Hang Su, Saab Mansour

Irony recognition

MultiPICo: Multilingual perspectivist irony corpus
Silvia Casola, Simona Frenda, Soda Marem Lo, Erhan Sezerer, Antonio Uva, Valerio Basile, Cristina Bosco, Alessandro Pedrani, Chiara Rubagotti, Viviana Patti, Davide Bernardi

Knowledge grounding

Graph chain-of-thought: Augmenting large language models by reasoning on graphs
Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Zheng Li, Ruirui Li, Xianfeng Tang, Suhang Wang, Yu Meng, Jiawei Han

MATTER: Memory-augmented transformer using heterogeneous knowledge sources*
Dongkyu Lee, Chandana Satya Prakash, Jack G. M. FitzGerald, Jens Lehmann

Tree-of-traversals: A zero-shot reasoning algorithm for augmenting black-box language models with knowledge graphs
Elan Markowitz, Anil Ramakrishna, Jwala Dhamala, Ninareh Mehrabi, Charith Peris, Rahul Gupta, Kai-Wei Chang, Aram Galstyan

Tree of traversals.png
An example of how the tree-of-traversals method uses a knowledge graph interface for the query “What actor played in both Inception and Interstellar?” From "Tree-of-traversals: A zero-shot reasoning algorithm for augmenting black-box language models with knowledge graphs".

LLM decoding

BASS: Batched attention-optimized speculative sampling*
Haifeng Qian, Sujan Gonugondla, Sungsoo Ha, Mingyue Shang, Sanjay Krishna Gouda, Ramesh Nallapati, Sudipta Sengupta, Anoop Deoras

Machine translation

Impacts of misspelled queries on translation and product search
Greg Hanneman, Natawut Monaikul, Taichi Nakatani

The fine-tuning paradox: Boosting translation quality without sacrificing LLM abilities
David Stap, Eva Hasler, Bill Byrne, Christof Monz, Ke Tran

Model editing

Propagation and pitfalls: Reasoning-based assessment of knowledge editing through counterfactual tasks
Wenyue Hua, Jiang Guo, Marvin Dong, Henghui Zhu, Patrick Ng, Zhiguo Wang

ReCoE construction.png
Demonstration of the process used to construct data for the reasoning-based counterfactual-editing (ReCoE) dataset. Straight lines represent data sourced from existing datasets; dashed lines denote data derived from LLM generation; zigzag lines denote data obtained through the corruption of other data. From "Propagation and pitfalls: Reasoning-based assessment of knowledge editing through counterfactual tasks".

Model evaluation

Bayesian prompt ensembles: Model uncertainty estimation for black-box large language models
Francesco Tonolini, Jordan Massiah, Nikolaos Aletras, Gabriella Kazai

ConSiDERS—the-human evaluation framework: Rethinking human evaluation for generative large language models
Aparna Elangovan, Ling Liu, Lei Xu, Sravan Bodapati, Dan Roth

Factual confidence of LLMs: On reliability and robustness of current estimators
Matéo Mahaut, Laura Aina, Paula Czarnowska, Momchil Hardalov, Thomas Müller, Lluís Marquez

Fine-tuned machine translation metrics struggle in unseen domains
Vilém Zouhar, Shuoyang Ding, Anna Currey, Tatyana Badeka, Jenyuan Wang, Brian Thompson

Measuring question answering difficulty for retrieval-augmented generation
Matteo Gabburo, Nicolaas Jedema, Siddhant Garg, Leonardo Ribeiro, Alessandro Moschitti

Model robustness

Extreme miscalibration and the illusion of adversarial robustness
Vyas Raina, Samson Tan, Volkan Cevher, Aditya Rawal, Sheng Zha, George Karypis

Multimodal models

CaMML: Context-aware multimodal learner for large models
Yixin Chen, Shuai Zhang, Boran Han, Tong He, Bo Li

CAMML.png
The CaMML framework, which consists of a retriever, a perceiver and a generator. After receiving user query q, the CaMML retriever identifies relevant multimodal contexts C from the data store. Then the CaMML perceiver seamlessly integrates data of various modalities, effectively encoding long-context information and injecting it into the CaMML generator. This enables the prediction of responses that are conditioned on both the context and the query. From "CaMML: Context-aware multimodal learner for large models".

Multi-modal retrieval for large language model based speech recognition
Jari Kolehmainen, Aditya Gourav, Prashanth Gurunath Shivakumar, Yi Gu, Ankur Gandhe, Ariya Rastrow, Grant Strimel, Ivan Bulyko

REFINESUMM: Self-refining MLLM for generating a multimodal summarization dataset
Vaidehi Patil, Leonardo Ribeiro, Mengwen Liu, Mohit Bansal, Markus Dreyer

Ordinal classification

Exploring ordinality in text classification: A comparative study of explicit and implicit techniques
Siva Rajesh Kasa, Aniket Goel, Sumegh Roychowdhury, Karan Gupta, Anish Bhanushali, Nikhil Pattisapu, Prasanna Srinivasa Murthy

Question answering

Beyond boundaries: A human-like approach for question answering over structured and unstructured information sources*
Jens Lehmann, Dhananjay Bhandiwad, Preetam Gattogi, Sahar Vahdati

MinPrompt: Graph-based minimal prompt data augmentation for few-shot question answering
Xiusi Chen, Jyun-Yu Jiang, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Wei Wang

Synthesizing conversations from unlabeled documents using automatic response segmentation
Fanyou Wu, Weijie Xu, Chandan Reddy, Srinivasan Sengamedu, "SHS"

Reasoning

Eliciting better multilingual structured reasoning from LLMs through code
Bryan Li, Tamer Alkhouli, Daniele Bonadiman, Nikolaos Pappas, Saab Mansour

II-MMR: Identifying and improving multi-modal multi-hop reasoning in visual question answering*
Jihyung Kil, Farideh Tavazoee, Dongyeop Kang, Joo-Kyung Kim

Recommender systems

Generative explore-exploit: Training-free optimization of generative recommender systems using LLM optimizers
Besnik Fetahu, Zhiyu Chen, Davis Yoshida, Giuseppe Castellucci, Nikhita Vedula, Jason Choi, Shervin Malmasi

Towards translating objective product attributes into customer language
Ram Yazdi, Oren Kalinsky, Alexander Libov, Dafna Shahaf

Responsible AI

SpeechGuard: Exploring the adversarial robustness of multimodal large language models
Raghuveer Peri, Sai Muralidhar Jayanthi, Srikanth Ronanki, Anshu Bhatia, Karel Mundnich, Saket Dingliwal, Nilaksh Das, Zejiang Hou, Goeric Huybrechts, Srikanth Vishnubhotla, Daniel Garcia-Romero, Sundararajan Srinivasan, Kyu Han, Katrin Kirchhoff

Text completion

Token alignment via character matching for subword completion*
Ben Athiwaratkun, Shiqi Wang, Mingyue Shang, Yuchen Tian, Zijian Wang, Sujan Gonugondla, Sanjay Krishna Gouda, Rob Kwiatkowski, Ramesh Nallapati, Bing Xiang

Token alignment.png
An illustration of token alignment process presented in "Token alignment via character matching for subword completion".

Research areas
Tags
About the Author
Staff writer

Related content

Work with us

See more jobs See more jobs
Applied Scientist II, CMT
IN, KA, Bengaluru
As a member of the CMT team, you'll play a key role in the evolution of our Competitive Monitoring systems to solve significantly complex and interesting technical challenges in machine learning, large language models in production, and recommender systems to name a few. The team's work directly impacts customer experience at a worldwide scale. Key job responsibilities Thought leader on the team and help set team directions Research multiple problem domains, suggest various approaches to try and be as hands-on as needed while providing more junior scientists with critical mentorship Collaborate with engineers to come up with the right LLD and HLD to solve key business problems Strong emphasis on communication via writing, internal and external talks, and being able to align with multiple stakeholders A day in the life As an Applied scientist II, a typical day will involve aligning with key product, engineering and business stakeholders ; advising junior scientists on the work they are doing ; reading current research papers and staying up-to-date on AI research ; diving deep as needed to improve CMT models and addressing stakeholders from the science perspective ; writing python code
Applied Scientist, CMT
IN, KA, Bengaluru
As a member of the CMT team, you'll play a key role in the evolution of our Competitive Monitoring systems to solve significantly complex and interesting technical challenges in machine learning, large language models in production, and recommender systems to name a few. The team's work directly impacts customer experience at a worldwide scale. Key job responsibilities 1. Research the problem domain and come up with various approaches to solve the problem. 2. Be willing to experiment quickly and fail fast. 3. Collaborate with engineers to come up with the right end to end solution to the business problems. 4. Ideate on future roadmap for science in CMT 5. Be willing to roll up your sleeves and learn core topics outside applied science, for example ML engineering A day in the life A typical day might involve (a) working on ideas for improving models around product similarity or price recommendations, (b) working closely with other scientists and our ML engineers to ensure that the best models are in production, (c) writing good maintainable code that can be reused and reproduced, (d) sharing your work across CMT and beyond via technical writings and presentations
Principal Applied Scientist, CX Science
US, CA, Santa Clara
We are looking for passionate, talented, and inventive Principal Applied Scientist with a strong machine learning background to help build industry-leading Conversational AI Systems. Our mission is to provide a delightful experience to Amazon’s customers by pushing the envelope in Natural Language Understanding (NLU), Dialog Systems including Generative AI with Large Language Models (LLMs) and Applied Machine Learning (ML). As part of our team, you will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in human language technology. Your work will directly impact millions of our customers in the form of products and services that make use language technology. You will gain hands on experience with Amazon’s heterogeneous text, structured data sources, and large-scale computing resources to accelerate advances in language understanding. We are hiring in all areas of human language technology: NLU, Dialog Management, Conversational AI, LLMs and Generative AI. A day in the life The team uses generative AI and foundation models to reimagine the experience of all customers on AWS. We explore new technologies and find creative solutions. Curiosity and an explorative mindset can find a place here to impact the life of engineers around the world. If you are excited about this space and want to enlighten your peers with new capabilities, this is the team for you.
Applied Scientist, Neuron ARG, Annapurna ML
US, WA, Seattle
The Automated Reasoning Group in the Amazon Neuron team is looking for an Applied Scientist to work on the intersection of Artificial Intelligence and program analysis to raise the code quality bar in our state-of-the-art deep learning compiler stack. This stack is designed to optimize application models across diverse domains, including Large Language and Vision, originating from leading frameworks such as PyTorch and JAX. Your role will involve working closely with our custom-built Machine Learning accelerator, Trainium, which represents the forefront of innovation for advanced ML capabilities, and is the underpinning of Generative AI. In this role as an Applied Scientist, you'll be instrumental in designing, developing, and deploying analyzers for ML compiler stages and compiler IRs. You will architect and implement business-critical tooling, publish research, and mentor a brilliant team of experienced scientists and engineers. You will need to be technically capable, credible, and curious in your own right as a trusted AWS Neuron engineer, innovating on behalf of our customers. Your responsibilities will involve tackling crucial challenges alongside a talented engineering team, contributing to leading-edge design and research in compiler technology and deep-learning systems software. Strong experience in programming languages, compilers, program analyzers, theorem provers, and program synthesis engines will be a benefit in this role. A background in machine learning and AI accelerators is preferred but not required.
Applied Scientist, Prime Video - Generative AI (Video)
US, CA, Sunnyvale
Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. Prime members can customize their viewing experience and find their favorite movies, series, documentaries, and live sports – including Amazon MGM Studios-produced series and movies; licensed fan favorites; and programming from Prime Video subscriptions such as Apple TV+, HBO Max, Peacock, Crunchyroll and MGM+. All customers, regardless of whether they have a Prime membership or not, can rent or buy titles via the Prime Video Store, and can enjoy even more content for free with ads. Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. As a Prime Video team member, you’ll have end-to-end ownership of the product, user experience, design, and technology required to deliver state-of-the-art experiences for our customers. You’ll get to work on projects that are fast-paced, challenging, and varied. You’ll also be able to experiment with new possibilities, take risks, and collaborate with remarkable people. We’ll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented technologists, you can decide where a career Prime Video Tech takes you! Prime Video is pioneering the use of Generative AI to empower the next generation of creatives. Our mission is to make world-class media creation accessible, scalable and efficient. We are seeking an Applied Scientist to advance the state of the art in Generative AI and to deliver these innovations as production-ready systems at Amazon scale. Your work will give creators unprecedented freedom and control while driving new efficiencies. Key job responsibilities As an Applied Scientist, you will have end-to-end ownership of the product, related research and experimentation. In addition, you will be applying advanced machine learning techniques in Computer Vision, Multimedia Understanding and Generative AI. We're building the foundational technology stack, spanning diffusion and flow-matching models, 3D/4D scene and character generation, motion and camera control, and post-training alignment. Other responsibilities include: - Research and develop generative models for controllable synthesis across images, video, vector graphics, and multimedia - Innovate in advanced diffusion and flow-based methods (e.g., inverse flow matching, parameter efficient training, guided sampling, test-time adaptation) to improve efficiency, controllability, and scalability - Advance visual grounding, depth and 3D estimation, segmentation, and matting for integration into pre-visualization, compositing, VFX, and post-production pipelines - Design multimodal GenAI workflows including visual-language model tooling, structured prompt orchestration, agentic pipelines
Principal Applied Scientist, Prime Video Personalization & Discovery , PV Personalization & Discovery, Principal Applied Scientist, PVPD
US, WA, Seattle
As a Principal Applied Scientist at Prime Video, you will be a technical and strategic leader responsible for inventing, developing, and deploying groundbreaking AI solutions that power personalized, relevant, and delightful experiences for millions of global customers. You will help shape the vision and direction of key ML systems that support Prime Video’s mission to deliver AI-powered customer experiences. This role demands a unique blend of deep technical expertise in machine learning and recommendation systems, industry leadership, and strong collaboration skills. You will guide the development of high-impact systems end-to-end - leading innovation from foundational research through production deployment - while mentoring scientists and influencing product and engineering roadmaps. We are looking for a thought leader who brings a strong track record of delivering ML innovations at scale, along with the curiosity and drive to push boundaries. This is a rare opportunity to drive meaningful impact at one of the largest streaming services in the world. Key job responsibilities - Invent, prototype, and productionize large-scale AI solutions across Prime Video’s personalization and discovery ecosystem using deep learning, generative AI, reinforcement learning, and optimization techniques; - Provide technical leadership and influence product vision by collaborating closely with engineers, product managers, and senior stakeholders; - Design and lead high-impact A/B tests and data analyses to validate hypotheses and guide product direction; - Drive technical bar-raising across science and engineering teams through mentorship, design reviews, and collaboration; - Stay ahead of industry trends and emerging research; leverage them to evolve long-term strategy and architecture; - Publish impactful research internally and externally (e.g. top-tier conferences and journals).
Applied Scientist II, Search Ranking, Search Ranking
US, CA, Palo Alto
The Amazon Search team creates customer-focused search solutions and technologies. Whenever a customer visits an Amazon site worldwide and types in a query or browses through product categories, Amazon Product Search services go to work. We design, develop, and deploy high-performance distributed search systems that rank a catalog of billions of products for hundreds of millions of shoppers. The Search Relevance team owns the ranking models that decide the order of results on every Amazon search page. In this role, you will design and post-train deep ranking models, including LLM-based rankers and multi-tower deep learning models, that jointly optimize purchase, relevance, and personalization. You will invent modeling and training techniques that push the Pareto frontier across multiple objectives, and take your work end to end from novel research prototype through offline evaluation to production online experimentation. Personalization is a first-class objective on this team. You will build models that reason over each customer's history, durable preferences, and query intent to decide which results best fit that specific customer, rather than optimizing a single population-level ranking. We treat search as an active research frontier and invest heavily in staying at the leading edge of ML. Beyond today's ranking stack, our current explorations include LLM agents that reason and plan across multi-step workflows, tool-augmented foundation models, and new paradigms that combine retrieval, reasoning, and personalization. You will help chart where search goes next, and see your ideas ship to real customers within weeks, not quarters. You will work in a dynamic, entrepreneurial team while leveraging the resources of Amazon.com, one of the world's leading technology companies. Please visit https://www.amazon.science for more information. Key job responsibilities Your responsibilities include but are not limited to: - Design, train, and deploy state-of-the-art ranking models that decide how results are ordered on Amazon search, spanning LLM-based rankers and multi-tower deep learning architectures that jointly model engagement, relevance, and personalization. - Post-train LLMs and ranking models with supervised fine-tuning, reinforcement learning (e.g. GRPO, DPO, RLHF), knowledge distillation, and listwise ranking losses (e.g. LambdaLoss, ListNet, ListMLE). - Compose multiple objectives (engagement, relevance, personalization) into a single ranking through principled multi-objective optimization at inference. - Design large-scale label pipelines, including LLM-as-teacher supervision, that turn customer signals and expert judgment into training and reward signals. - Optimize inference for production ranking models through quantization, quantization-aware training, teacher-student distillation, and serving-stack tuning. - Evaluate proposed solutions through offline benchmarks and online A/B tests, and drive the analysis that decides whether a change ships. - Publish and present your work at internal and external scientific venues in ML, NLP, and IR.
Applied Scientist II, Computational Modeling, OMHS SCS
US, MA, Boston
Are you excited about applying machine learning and applied mathematics to real-world systems at massive scale? As an Applied Scientist on this newly formed team, you will collaborate closely with scientists and engineers to bring research into production across a broad portfolio of problems — from computer vision perception platforms to building-wide optimization and orchestration. You will frame ambiguous business problems as tractable scientific challenges and implement novel machine learning (ML) systems, first-principles models, embedded systems prototypes, and performance optimizations in both prototype and production environments. This is a ground-floor opportunity to shape the scientific direction of a new organization, where your contributions will directly influence how Amazon's fulfillment network operates and evolves. Key job responsibilities - Design, develop, and deploy ML and scientific solutions spanning classical machine learning, statistical modeling, computer vision, optimization, and physics-informed modeling in production environments. - Rapidly ramp on unfamiliar problem domains, frame ambiguous business problems as tractable scientific challenges, and prototype solutions end to end. - Author or co-author research findings for internal or external peer-reviewed venues, and provide peer feedback on research procedures and results across teams. - Prototype and evaluate sensing hardware and lightweight, edge-deployable models that run on commodity compute under real-world constraints. - Collaborate across multiple science and engineering teams to integrate your solutions into deployment architecture, mentoring less experienced scientists along the way. A day in the life You might start your morning reviewing experiment results from an overnight model training run, then shift into a design discussion with engineers on how to deploy a new computer vision model to edge hardware in a fulfillment center. After lunch, you could be prototyping a physics-informed optimization approach, writing up findings for a research paper, or pairing with a teammate to debug a tricky data pipeline. As part of a new and growing organization, you will have a direct hand in shaping team practices, scientific roadmaps, and the tools you use every day. About the team Our team sits within Amazon's fulfillment technology organization and applies a range of scientific disciplines — including computer vision, optimization, reinforcement learning, and statistical modeling — to improve how goods move through Amazon's global fulfillment network. We build the models and systems that drive real-time orchestration, optimizing throughput, flow, and operational performance at scale. As a newly formed organization, we are building our culture and scientific agenda from the ground up. You will join a collaborative, inclusive group of scientists and engineers who value experimentation, rigorous research, and delivering measurable impact for customers.
Data Scientist, AI Program, CNGS NBS (New Business & New Seller) Amazon Business
CN, 31, Shanghai
Team & Project Overview The NBS Data Central team powers analytics, data science, and AI capabilities for Worldwide Global Selling (WWGS). We build scalable data products, and insight-generation systems that drive seller growth across 10+ marketplaces. Seller Intelligence is a P0 foundation theme at the Global Selling level, formed by merging "One Tagging" and "Good Contact" workstreams. It provides seller identity, segmentation, and contact-reach infrastructure that underpins all downstream seller-facing AI workflows — including intelligent outreach, personalized recommendations, and automated engagement. Scope of Impact Own the science pillar for Seller Intelligence within a cross-functional POD (PM + DE + DS + SDE) Directly impact seller engagement metrics across CN, IN, LATAM, and East-Asia expansion regions Models and data products consumed by 5+ downstream teams (ESM, NSR, MKT, NBS AI Ops, ROC) Influence $100M+ annual seller GMS through improved segmentation and contact optimization Key job responsibilities Design and deliver seller segmentation and propensity models at scale — incorporating GMS, category, growth trajectory, engagement signals, and lifecycle stage. Build contact quality scoring and lifecycle management systems (coverage optimization, dormancy detection, reactivation modeling). Define success metrics, experimentation frameworks (A/B, causal inference), and measurement methodology for seller engagement interventions. Productionize ML models and data products — partner with engineering to deploy seller scores, contact quality indices, and recommendation signals. Explore LLM/GenAI applications: automated insight generation from seller data, contact intent classification, and intelligent report synthesis. Serve as the science representative in bi-weekly NBS theme reviews; present findings and proposals to theme Bar Raisers and leadership. Collaborate with BIE team members to democratize analytical outputs via dashboards and self-serve tools. Contribute to cross-marketplace seller behavior analysis supporting Global Expansion strategy (IN, KR, VN, LATAM). Evaluate, integrate, and iterate on AI systems — assess new AI/ML tools, frameworks, and third-party models for applicability to seller intelligence use cases.
Applied Scientist, Prime Video Commerce Insights
GB, London
Come build the future of entertainment with us. Are you interested in shaping the future of movies and television? Do you want to define the next generation of how and what Amazon customers are watching? Prime Video is a premium streaming service that offers customers a vast collection of TV shows and movies — all with the ease of finding what they love to watch in one place. We offer customers thousands of popular movies and TV shows from Originals and Exclusive content to exciting live sports events. We also offer our members the opportunity to subscribe to add-on channels which they can cancel at anytime and to rent or buy new release movies and TV box sets on the Prime Video Store. Prime Video is a fast-paced, growth business — available in over 240 countries and territories worldwide. The team works in a dynamic environment where innovating on behalf of our customers is at the heart of everything we do. If this sounds exciting to you, please read on. Prime Video Commerce's mission is to present the right offer to the right customer at the right time — across subscriptions, channels, and transactional video, in every market and on every device. Our science team replaces static business rules with ML-driven decisions that personalise the entire commerce journey, from discovery through checkout and beyond. We operate at scale across hundreds of millions of customers, and we are expanding into new frontiers — combining the latest advances in agentic and generative AI, behavioural simulation, and causal inference to understand the impact of our decisions before they reach customers. We are looking for an Applied Scientist to join the Prime Video Commerce Insights team in London. You will develop and deploy customer-facing models, understand customer behaviour at scale, and explore emerging techniques that help us make better decisions faster. This is a delivery focused role within a high-visibility multidisciplinary group of engineers and scientists, focused on improving the customer experience for Prime Video. Key job responsibilities - Research, design, and implement machine learning approaches (e.g. reinforcement learning and recommendation systems) that personalise across different customer touch points. - Collaborate with engineers to deploy and integrate successful experiment results into large-scale, complex Amazon production systems with low latency. - Design and execute rigorous experiments to demonstrate the technical efficacy and business value of your methods. - Act as a subject-matter expert and help define the science roadmap and research agenda in line with organisational priorities and production constraints. - Provide machine learning thought leadership to technical and business leaders, thinking strategically about business, product, and technical challenges. - Work with technical product managers to work backwards from what matters to customers and deliver ML-backed solutions. - Share results with the team and wider scientific community through documents that are both statistically rigorous and compellingly relevant. A day in the life You will be a research leader and innovator within the Commerce Insights organisation. You will collaborate with talented engineers and senior leaders to solve problems that are uniquely challenging at Amazon's scale: personalising commerce decisions across multiple business lines, balancing competing objectives, and positively impacting hundreds of millions of customers worldwide. The problems here are technically deep — combining large-scale ML, causal reasoning, and behavioural modelling in a domain where every decision carries real revenue and customer-experience consequences. Your research will ship to production and move metrics that matter. About the team You will join a team of engineers and applied scientists with a proven track record of solving highly complex, ambiguous problems — work that has produced patents and publications at top-tier conferences. The team has direct visibility to senior Prime Video leadership and collaborates broadly across Commerce, Content, and Platform teams to shape how customers discover, subscribe to, and engage with video content. This is a team that operates at the intersection of rigorous research and real-world impact, where your ideas move from whiteboard to production for hundreds of millions of customers.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social