Social
Conversational AI

A quick guide to Amazon’s 50-plus papers at EMNLP 2024

Large language models predominate, both as a research subject themselves and as tools for researching topics of particular interest to Amazon, such as speech, recommendations, and information retrieval.

By Staff writer
November 14, 2024
7 min read
Share

Large language models (LLMs) have come to dominate the field of natural-language processing, so it’s no surprise that they also dominate the research that Amazon scientists are presenting at this year’s Conference on Empirical Methods in Natural-Language Processing (EMNLP). LLM training is the topic with the greatest number of Amazon papers, followed closely by strategies for mitigating misinformation in LLMs’ outputs — including but not limited to hallucinations. At the same time, a number of papers apply LLMs to topics of traditional interest at Amazon, such as speech, recommender systems, and information retrieval. (Papers marked with asterisks were accepted to Findings of EMNLP.)

AI agents

MARCO: Multi-agent real-time chat orchestration
Anubhav Shrimal, Shervin Malmasi, Kriti Biswas, Swarnalatha Raghuraman, Anish Nediyanchath, Yi Zhang, Promod Yenigalla

Code generation

CodeFort: Robust training for code generation models
Yuhao Zhang, Shiqi Wang, Haifeng Qian, Zijian Wang, Mingyue Shang, Linbo Liu, Sanjay Krishna Gouda, Baishakhi Ray, Murali Krishna Ramanathan, Xiaofei Ma, Anoop Deoras

Socratic human feedback (SoHF): Expert steering strategies for LLM code generation
Subramanian Chidambaram, Erran Li, Min Bai, Xiaopeng LI, Kaixiang Lin, Xiong Zhou, Alex C. Williams

Structured object language modeling (SoLM): Native structured objects generation conforming to complex schemas with self-supervised denoising
Amir Tavanaei, Kee Kiat Koo, Hayreddin Ceker, Shaobai Jiang, Qi Li, Julien Han, Karim Bouyarmane

Contrastive decoding

Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM
Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal, Anil Ramakrishna, Tagyoung Chung

Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM.png
Given a simple question with clues, contrastive decoding could have an “obvious blindness” (e.g., assigning higher probability to an uncommon answer, such as "invertebrate", than to the most obvious answer, "bees"). In contrast, the asymptotic probability decoding proposed in "Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM" correctly assigns the highest probability to "bees" by leveraging the probabilities from multiple LMs of different sizes.

Data integration

ASTRA: Automatic schema matching using machine translation
Tarang Chugh, Deepak Zambre

Learning from natural language explanations for generalizable entity matching
Somin Wadhwa, Adit Krishnan, Runhui Wang, Byron C. Wallace, Chris (Luyang) Kong

Pretraining and finetuning language models on geospatial networks for accurate address matching
Saket Maheshwary, Arpan Paul, Saurabh Sohoney

Retrieval augmented spelling correction for e-commerce applications
Xuan Guo, Rohit Patki, Dante Everaert, Christopher Potts

Dataset distillation

Textual dataset distillation via language model embedding
Yefan Tao, Chris (Luyang) Kong, Andrey Kan, Laurent Callot

Textual dataset distillation via language model embedding: DaLLME.png
The DaLLME framework proposed in "Textual dataset distillation via language model embedding" begins by using a language model to transform raw textual data into embedding vectors. A set of distilled vectors is then derived in the embedding space, through a process designed to encapsulate maximum informational content. Finally, the vec2text model translates these distilled vectors back into textual form.

Document understanding

DocKD: Knowledge distillation from LLMs for open-world document understanding models
Sungnyun Kim, Haofu Liao, Srikar Appalaraju, Peng Tang, Zhuowen Tu, Ravi Kumar Satzoda, R. Manmatha, Vijay Mahadevan, Stefano Soatto

Information retrieval

Evaluating D-MERIT of partial-annotation on information retrieval
Royi Rassin, Yaron Fairstein, Oren Kalinsky, Guy Kushilevitz, Nachshon Cohen, Alexander Libov, Yoav Goldberg

Identifying high consideration e-commerce search queries
Zhiyu Chen, Jason Choi, Besnik Fetahu, Shervin Malmasi

Learning when to retrieve, what to rewrite, and how to respond in conversational QA*
Nirmal Roy, Leonardo Ribeiro, Rexhina Blloshmi, Kevin Small

Natural-language understanding

Intent detection in the age of LLMs
Gaurav Arora, Shreya Jain, Srujana Merugu

Intent detection in the age of LLMs.png
"Intent detection in the age of LLMs" proposes a methodology for adaptive in-context learning and chain-of-thought-based intent detection using LLMs.

Predicting entity salience in extremely short documents
Ben Bullough, Harrison Lundberg, Chen Hu, Weihang Xiao

LLM evaluation

AXCEL: Automated eXplainable consistency evaluation using LLMs*
P Aditya Sreekar, Sahil Verma, Suransh Chopra, Sarik Ghazarian, Abhishek Persad, Narayanan Sadagopan

Precise model benchmarking with only a few observations
Riccardo Fogliato, Pratik Patil, Nil-Jana Akpinar, Mathew Monfort

LLM fine tuning

AdaZeta: Adaptive zeroth-order tensor-train adaption for memory-efficient large language models fine-tuning
Yifan Yang, Kai Zhen, Ershad Banijamali, Thanasis Mouchtaris, Zheng Zhang

RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning
Haoyu Wang, Tianci Liu, Ruirui Li, Monica Cheng, Tuo Zhao, Jing Gao

RoseLoRA.png
The row- and column-wise sparse low-rank adaptation (RoseLoRA) framework proposed in "RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning".

LLMs for speech

Speechworthy instruction-tuned language models
Hyundong Cho, Nicolaas Jedema, Leonardo Ribeiro, Karishma Sharma, Pedro Szekely, Alessandro Moschitti, Ruben Janssen, Jonathan May

LLM misinformation mitigation

ECON: On the detection and resolution of evidence conflicts
Cheng Jiayang, Chunkit Chan, Qianqian Zhuang, Lin Qiu, Tianhang Zhang, Tengxiao Liu, Yangqiu Song, Yue Zhang, Pengfei Liu, Zheng Zhang

Generative subgraph retrieval for knowledge graph–grounded dialog generation
Jinyoung Park, Minseok Joo, Joo-Kyung Kim, Hyunwoo J. Kim

HalluMeasure: Fine-grained hallucination measurement using chain-of-thought reasoning
Shayan Ali Akbar, Md Mosharaf Hossain, Tess Wood, Si-Chi Chin, Erica Salinas, Victor Alvarez, Erwin Cornejo

Knowledge-centric hallucination detection
Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Zheng Zhang, Yue Zhang

LLM reasoning

Auto-evolve: Enhancing large language model’s performance via self-reasoning framework*
Krishna Aswani, Alex Lu, Pranav Patankar, Priya Dhalwani, Iris Tan, Jayant Ganeshmohan, Simon Lacasse

LLM self-correction

LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints
Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, Nanyun Peng

DeCRIM.png
In the DeCRIM pipeline proposed in "LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints", an LLM first generates a response to a user request. The Decomposer then breaks down the request into granular constraints, and the Critic model gives feedback on whether the response meets those constraints. If it does, the response is output; if not, the LLM uses the feedback to refine the response.

LLM training

Dancing in chains: Reconciling instruction following and faithfulness in language models
Zhengxuan Wu, Yuhao Zhang, Peng Qi, Yumo Xu, Rujun Han, Yian Zhang, Jifan Chen, Bonan Min, Zhiheng Huang

DEM: Distribution edited model for training with mixed data distributions
Dhananjay Ram, Aditya Rawal, Momchil Hardalov, Nikolaos Pappas, Sheng Zha

DEM: Distribution Edited Model for Training with Mixed Data Distributions
The distribution-edited model (ΘD) described in "DEM: Distribution edited model for training with mixed data distributions" results from fine-tuning a pretrained model (Θ) on n individual data distributions (Di) and combining the resulting models with basic element-wise vector operations. Here, the extracted distribution vectors (∆ΘDi ) are multiplied by weight coefficients, and the weighted sum is added to the base model.

Evolutionary contrastive distillation for language model alignment
Julian Katz-Samuels, Zheng Li, Hyokun Yun, Priyanka Nigam, Yi Xu, Vaclav Petricek, Bing Yin, Trishul Chilimbi

Hop, skip, jump to convergence: Dynamics of learning rate transitions for improved training of large language models
Shreyas Subramanian, Vignesh Ganapathiraman, Corey Barrett

Learning from relevant subgoals in successful dialogs using iterative training for task-oriented dialog systems
Magdalena Kaiser, Patrick Ernst, Gyuri Szarvas

Quality matters: Evaluating synthetic data for tool-using LLMs
Shadi Iskander, Nachshon Cohen, Zohar Karnin, Ori Shapira, Sofia Tolmach

Query autocompletion

AmazonQAC: A large-scale, naturalistic query autocomplete dataset
Dante Everaert, Rohit Patki, Tianqi Zheng, Christopher Potts

DiAL: Diversity aware listwise ranking for query auto-complete
Sonali Singh, Sachin Farfade, Prakash Mandayam Comar

Question answering

RAG-QA arena: Evaluating domain robustness for long-form retrieval-augmented question answering
Rujun Han, Yuhao Zhang, Peng Qi, Yumo Xu, Jenyuan Wang, Lan Liu, William Yang Wang, Bonan Min, Vittorio Castelli

Retrieving contextual information for long-form question answering using weak supervision
Philipp Christmann, Svitlana Vakulenko, Ionut Teodor Sorodoc, Bill Byrne, Adrià de Gispert

Recommender systems

Efficient pointwise-pairwise learning-to-rank for news recommendation
Nithish Kannen Senthilkumar, Yao Ma, Gerrit van den Burg, Jean Baptiste Faddoul

Efficient pointwise-pairwise learning-to-rank for news recommendation.png
An illustration of the GLIMPSE framework proposed in "Efficient pointwise-pairwise learning-to-rank for news recommendation". GLIMPSE adopts a multitask approach in which a pretrained language model is fine-tuned on both the relevance prediction task and the pairwise-preference task. During inference, the relevance predictions are used to produce an initial pointwise ranking, which is subsequently improved by one or more right-to-left (RTL) passes using pairwise comparisons.

PEARL: Preference extraction with exemplar augmentation and retrieval with LLM agents
Vijit Malik, Akshay Jagatap, Vinayak Puranik, Anirban Majumder

Sequential LLM framework for fashion recommendation
Han Liu, Xianfeng Tang, Tianlang Chen, Jiapeng Liu, Indu Indu, Henry Peng Zou, Peng Dai, Roberto Fernandez Galan, Mike Porter, Dongmei Jia, Ning Zhang, Lian Xiong

Responsible AI

Attribute controlled fine-tuning for large language models: A case study on detoxification
Tao Meng, Ninareh Mehrabi, Palash Goyal, Anil Ramakrishna, Aram Galstyan, Richard Zemel, Kai-Wei Chang, Rahul Gupta, Charith Peris

FLIRT: Feedback loop in-context red teaming
Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta

Order of magnitude speedups for LLM membership inference
Rongting Zhang, Martin Bertran Lopez, Aaron Roth

Synthetic data generation

CorrSynth: A correlated sampling method for diverse dataset generation from LLMs
Suhas Kowshik, Abhishek Divekar, Vijit Malik

A Correlated Sampling Method for Diverse Dataset Generation from LLMs
"CorrSynth: A correlated sampling method for diverse dataset generation from LLMs" introduces a sampling method that uses anti-correlation between examples rather than few-shot generation.

DATA ADVISOR: Dynamic data curation for safety alignment of large language models
Fei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta, Kai-Wei Chang, Aram Galstyan

Evaluating differentially private synthetic data generation in high-stakes domains
Krithika Ramesh, Nupoor Gandhi, Pulkit Madaan, Lisa Bauer, Charith Peris, Anjalie Field

SYNTHESIZRR: Generating diverse datasets with retrieval augmentation
Abhishek Divekar, Greg Durrett

Abstract depiction of the SYNTHESIZRR procedure
Abstract depiction of the procedure proposed in "SYNTHESIZRR: Generating diverse datasets with retrieval augmentation". The content sourcing stage retrieves K unique documents {r1,...,rK} from a large corpus for each in-context covariate xICL. The task-inversion stage uses a parameterized context refinement prompt, Pτ, which takes parameters Rinv (inversion instruction), rk (a retrieved document), and V(yICL) (the verbalized target label). A generalist teacher LLM autoregressively generates a synthetic covariate. Each in-context example thus produces K unique synthetic examples {x̃1,..., x̃K}, which we include in the dataset with target yICL.

Text classification

Distance-aware calibration for pre-trained language models*
Alberto Gasparin, Gianluca Detommaso

Performance-guided LLM knowledge distillation for efficient text classification at scale

Flavio Di Palo, Prateek Singhi, Bilal Fadlallah

Prompt-tuned muti-task taxonomic transformer (PTMTTaxoFormer)
Rajashekar Vasantha, Nhan Nguyen, Yue Zhang

Text summarization

Salient information prompting to steer content in prompt-based abstractive summarization
Lei Xu, Asad Karim, Saket Dingliwal, Aparna Elangovan

Research areas
Tags
About the Author
Staff writer

Related content

Work with us

See more jobs See more jobs
Applied Scientist , Sponsored Products
US, NY, New York
The Sponsored Products and Brands team at Amazon Ads is re-imagining the advertising landscape through cutting-edge generative AI technologies, revolutionizing how millions of customers discover products and engage with brands across Amazon.com and beyond. We are at the forefront of re-inventing advertising experiences, bridging human creativity with artificial intelligence to transform every aspect of the advertising lifecycle from ad creation and optimization to performance analysis and customer insights. We are a passionate group of innovators dedicated to developing responsible and intelligent AI technologies that balance the needs of advertisers, enhance the shopping experience, and strengthen the marketplace. If you're energized by solving complex challenges and pushing the boundaries of what's possible with AI, join us in shaping the future of advertising. Key job responsibilities - Define and lead science initiatives from problem framing through production deployment in a high-ambiguity environment - Develop and deploy models spanning computer vision, language, and search and retrieval that operate on multimodal inputs at scale - Design and analyze large-scale online experiments to measure impact on shopper and advertiser outcomes - Collaborate with engineering, product, and design to ship science into production A day in the life As an Applied Scientist on the Sponsored Videos team, you will tackle problems at the intersection of computer vision, generative AI, search and retrieval, and personalization. You'll own the full science lifecycle from research and experimentation through online testing and production deployment, working closely with engineering, product, and design partners to bring ideas to market. You should be comfortable working with multimodal signals, building models that operate at scale, and measuring impact through rigorous experimentation. Your work will directly influence the experience of hundreds of millions of shoppers and the outcomes of tens of thousands of advertisers. About the team The Sponsored Videos team within Sponsored Products and Brands develops the science and systems behind video advertising experiences that connect advertisers and shoppers across Amazon. We are on a mission to make Amazon the best-in-class destination for shoppers to discover, engage with, and build affinity with brands through videos.
Research Scientist , WW Sustainability
US, WA, Seattle
Join us at the forefront of Amazon's sustainability initiatives to work on environmental and social advancements that support Amazon's long-term worldwide sustainability strategy. At Amazon, we're working to be the most customer-centric company on earth. To get there, we need exceptionally talented, bright, and driven people. We are looking for a Research Scientist to join our growing Sustainability team to drive the science behind value chain decarbonization. This role will establish Amazon's scientific methodologies for sector- and cross-sectoral decarbonization mechanisms, and establish benchmarks for automated validation and risk assessment. As a Research Scientist, you will be responsible for independently leading assessments of environmental issues across the full spectrum of Amazon businesses and evaluating sustainability impacts across the value chain. You will independently develop quality frameworks and methodologies that enable Amazon to scale procurement of high-quality environmental interventions while maintaining scientific rigor and environmental integrity. Key job responsibilities - Develop quality assessment frameworks for complex environmental interventions, baseline-setting approaches, and measurement methodologies - Build quantitative benchmark and statistical models that enable scalable evaluation across heterogeneous data sources - Create attribution methodologies for supply chain interventions across Amazon's diverse footprint - Develop social and environmental safeguard criteria that integrate community impact assessments - Collaborate with cross-functional teams including procurement, sustainability operations, and business units to translate scientific methodologies into operational requirements - Work under the direction of senior business leaders while acting as lead Subject Matter Expert for value chain decarbonization science, including designing and leading research, data collection, modeling, documentation, interpretation, and validation About the team Diverse Experiences: World Wide Sustainability values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Inclusive Team Culture: It’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (inclusive diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth: We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Structural Design Analysis Engineer, Structural Design Integrity
US, CA, Sunnyvale
Amazon Lab126 is an inventive research and development company that designs and engineers high-profile consumer electronics. Lab126 began in 2004 as a subsidiary of Amazon.com, Inc., originally creating the best-selling Kindle family of products. Since then, we have produced industry leading devices like Fire tablets, Fire TV and Amazon Echo. As a Design Analysis Engineer, you will be responsible for bringing new product designs through to manufacturing. Structural engineering contributes unique, in-depth technical knowledge to solve complex engineering problems in concert with multi-disciplinary teams including Industrial Design, Hardware Engineering, and Operations. Key job responsibilities You will work closely with multi-disciplinary groups including Product Design, Industrial Design, Hardware Engineering, and Operations, to drive key aspects of engineering of consumer electronics products. In this role, you will: · Perform analysis and testing of complex electronic assemblies using advanced simulation and experimentation tools and techniques · Develop, analyze and test thermal, acoustic and structural solutions; from concept design, feature development, product architecture, through system validation · Support creative developments through application of analysis and testing of complex electronic assemblies using advanced simulation and experimentation tools and techniques · Use simulation tools like Abaqus for analysis and design of products · Validate design modifications using simulation and actual prototypes · Use of programming languages like Python and Matlab for analytical/statistical analyses and automation · Establish noise thresholds for usability and compliance requirements · Determine and validate structural performance under use and test conditions · Have strong knowledge of various materials such as heat spreaders solutions to resolve thermal issues, damping materials for noise and vibration suppression · Use various data acquisition systems with thermocouples, accelerometers, strain gauges and IR cameras · Collaborate as part of the device team to iterate and optimize design parameters of enclosures and structural parts to establish and deliver project performance objectives · Design and execute tests using statistical tools to validate analytical models, identify risks and assess design margins · Create and present analytical and experimental results · Develop and apply design guidelines based on project results
Sr. Applied Scientist, Workforce Solutions
CA, BC, Vancouver
Success in any organization begins with its people and having a comprehensive understanding of our workforce and how we best utilize their unique skills and experience is paramount to our future success. WISE (Workforce Intelligence powered by Scientific Engineering) delivers the scientific and engineering foundation that powers Amazon's enterprise-wide workforce planning ecosystem. Addressing the critical need for precise workforce planning, WISE enables a closed-loop mechanism essential for ensuring Amazon has the right workforce composition, organizational structure, and geographical footprint to support long-term business needs with a sustainable cost structure. We are looking for a Sr. Applied Scientist to join our ML/AI team to work on Advanced Optimization and LLM solutions. You will partner with Software Engineers, Machine Learning Engineers, Data Engineers and other Scientists, TPMs, Product Managers and Senior Management to help create world-class solutions. We're looking for people who are passionate about innovating on behalf of customers, demonstrate a high degree of product ownership, and want to have fun while they make history. You will leverage your knowledge in machine learning, advanced analytics, metrics, reporting, and analytic tooling/languages to analyze and translate the data into meaningful insights. You will have end-to-end ownership of operational and technical aspects of the insights you are building for the business, and will play an integral role in strategic decision-making. Further, you will build solutions leveraging advanced analytics that enable stakeholders to manage the business and make effective decisions, partner with internal teams to identify process and system improvement opportunities. As a tech expert, you will be an advocate for compelling user experiences and will demonstrate the value of automation and data-driven planning tools in the People Experience and Technology space. Key job responsibilities * Engineering execution - drive crisp and timely execution of milestones, consider and advise on key design and technology trade-offs with engineering teams * Priority management - manage diverse requests and dependencies from teams * Process improvements – define, implement and continuously improve delivery and operational efficiency * Stakeholder management – interface with and influence your stakeholders, balancing business needs vs. technical constraints and driving clarity in ambiguous situations * Operational Excellence – monitor metrics and program health, anticipate and clear blockers, manage escalations To be successful on this journey, you love having high standards for yourself and everyone you work with, and always look for opportunities to make our services better.
Research Scientist, Fauna
US, NY, New York
We are seeking an Research Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers
Applied Scientist II, Amazon Search
IN, KA, Bengaluru
We are embarking on a multi-year journey to improve the shopping experience for customers globally. Amazon Search team creates customer-focused search solutions and technologies that make shopping delightful and effortless for our customers. Our goal is to understand what customers are looking for in whatever language happens to be their choice at the moment and help them find what they need in Amazon's vast catalog of billions of products — starting from the very first keystroke. As Amazon expands to new interfaces, we are faced with the unique challenge of maintaining the bar on Search Results Quality and Search Autocomplete. We are looking for a Applied Scientist II to work on improving search on Amazon using NLP, ML, and DL technology. As an Applied Scientist, you will lead our efforts in query understanding, semantic matching, and ranking. You will build systems that anticipate search query intent and surface the right results. As part of this role, you will develop high precision, high recall, and low latency solutions for search. Your solutions should work for all languages that Amazon supports and will be used in all Amazon locales world-wide. You will develop scalable science and engineering solutions that work successfully in production. Key job responsibilities As an Applied Scientist on the team, you will lead science innovation to improve the customer search experience through higher-quality search results. You will: - Develop and deploy ML models to produce relevant search results. - Design and train semantic matching models (bi-encoders, cross-encoders, and distillation from large foundation models) for ranking and relevance. - Develop reinforcement learning and reward-modeling approaches to continuously improve search results quality. - Train multi-objective ranking and scoring systems that balance suggestion diversity, specificity, and relevance. - Design and implement scalable model architectures optimized for strict latency constraints, including knowledge distillation, quantization, and efficient inference strategies for production deployment. - Lead end-to-end science projects from problem formulation through production launch, collaborating closely with engineers and scientists within and outside the team to deliver customer-facing impact.
Applied Scientist II, Amazon Search
IN, KA, Bengaluru
We are embarking on a multi-year journey to improve the shopping experience for customers globally. Amazon Search team creates customer-focused search solutions and technologies that make shopping delightful and effortless for our customers. Our goal is to understand what customers are looking for in whatever language happens to be their choice at the moment and help them find what they need in Amazon's vast catalog of billions of products — starting from the very first keystroke. As Amazon expands to new interfaces, we are faced with the unique challenge of maintaining the bar on Search Results Quality and Search Autocomplete. We are looking for a Applied Scientist II to work on improving search on Amazon using NLP, ML, and DL technology. As an Applied Scientist, you will lead our efforts in query understanding, semantic matching, and ranking. You will build systems that anticipate search query intent and surface the right results. As part of this role, you will develop high precision, high recall, and low latency solutions for search. Your solutions should work for all languages that Amazon supports and will be used in all Amazon locales world-wide. You will develop scalable science and engineering solutions that work successfully in production. Key job responsibilities As an Applied Scientist on the team, you will lead science innovation to improve the customer search experience through higher-quality search results. You will: - Develop and deploy ML models to produce relevant search results. - Design and train semantic matching models (bi-encoders, cross-encoders, and distillation from large foundation models) for ranking and relevance. - Develop reinforcement learning and reward-modeling approaches to continuously improve search results quality. - Train multi-objective ranking and scoring systems that balance suggestion diversity, specificity, and relevance. - Design and implement scalable model architectures optimized for strict latency constraints, including knowledge distillation, quantization, and efficient inference strategies for production deployment. - Lead end-to-end science projects from problem formulation through production launch, collaborating closely with engineers and scientists within and outside the team to deliver customer-facing impact.
Applied Science Manager, Prime Video Personalization & Discovery
US, CA, Sunnyvale
Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. Prime members can customize their viewing experience and find their favorite movies, series, documentaries, and live sports – including Amazon MGM Studios-produced series and movies; licensed fan favorites; and programming from Prime Video add-on subscriptions such as Apple TV+, Max, Crunchyroll and MGM+. All customers, regardless of whether they have a Prime membership or not, can rent or buy titles via the Prime Video Store, and can enjoy even more content for free with ads. Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. As a Prime Video technologist, you’ll have end-to-end ownership of the product, user experience, design, and technology required to deliver state-of-the-art experiences for our customers. You’ll get to work on projects that are fast-paced, challenging, and varied. You’ll also be able to experiment with new possibilities, take risks, and collaborate with remarkable people. We’ll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented technologists, you can decide where a career Prime Video Tech takes you! We are looking for a self-motivated, passionate and resourceful Applied Science Manager to bring diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. You will lead a strong science team and work closely with other science and engineering leaders, product and business partners together to build the best personalized customer experience for Prime Video. At the end of the day, you will have the reward of seeing your contributions benefit millions of Amazon.com customers worldwide. Key job responsibilities - Lead to develop AI solutions for various Prime Video recommendation and personalization systems using Deep learning, GenAI, Reinforcement Learning, recommendation system and optimization methods; - Work closely with engineers and product managers to design, implement and launch AI solutions end-to-end; - Effectively communicate technical and non-technical ideas with teammates and stakeholders; - Stay up-to-date with advancements and the latest modeling techniques in the field; - Hire and grow a science team working in this exciting video personalization domain. About the team Prime Video Recommendation Science team owns science solution to power recommendation and personalization experience on various devices. We work closely with the engineering teams to launch our solutions in production.
Applied Scientist II, Prime AI/ML Science
US, WA, Seattle
Interested in modeling and understanding customer behavior through machine learning, artificial intelligence, and data mining over TB scale data with huge business impact on millions of customers? Join our team of Scientists developing models to model customer behavior and optimize the customer experience with Amazon Prime. This includes understanding who our customers are, long-term value of the Prime membership program, and creating the right personalized framework for content and subscription optimization. As an AI/ML expert, you will partner directly with product owners to intake, build, and directly apply your modeling solutions. There are numerous scientific and technical challenges you will get to tackle in this role, such as optimizing/fine-tuning GenAI/LLM solutions for Prime personalization, building GenAI foundation models, global scalability of models, combinatorial optimization, cold start problem, accelerated experimentation, short/long term goals modeling, and multi-step optimization leading to reinforcement learning of the customer journey. We employ techniques from GenAI/LLMs, supervised/semi-supervised learning, deep learning, transformer architectures, using outcomes from causal Econometric modeling, and Reinforcement learning. As the central science team within Prime, our expertise gets routinely called upon to weigh in on a variety of topics. We also emphasize the need and value of scientific research and have developed a strong publication and patent record (internally/externally) which you will be a part of. You will also utilize and be exposed to the latest in ML technologies and infrastructure: AWS technologies (EMR/Spark, Sagemaker, DynamoDB, S3, ClaudeCode), various AI/ML algorithms and techniques (Deep Learning, GenAI/LLMs, transformers, supervised/unsupervised/semi-supervised/reinforcement learning), and statistical modeling techniques. - Stay abreast of current literature in the field and advance/build novel science solutions leveraging SoTA solutions. - Build and develop AI/ML models and supporting infrastructure at TB scale, in coordination with software engineering teams. - Leverage Deep Learning and GenAI solutions for building foundation models and personalized optimization solution. - Develop offline policy estimation tools and integrate with measurement systems/econometric models. - Establish scalable, efficient, automated processes for large scale data analyses, science development, science validation and model implementation. - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes. - Work closely with the business to understand their problem space, identify the opportunities and formulate the problems. - Use AI/machine learning, data mining, statistical techniques and others to create actionable, meaningful, and scalable solutions for the business problems. - Design, develop and evaluate highly innovative models and statistical approaches to understand and predict customer behavior and to solve business problems. Key job responsibilities - Stay abreast of current literature in the field and advance/build novel science solutions leveraging SoTA solutions. - Build and develop AI/ML models and supporting infrastructure at TB scale, in coordination with software engineering teams. - Leverage Deep Learning and GenAI solutions for building foundation models and personalized optimization solution. - Develop offline policy estimation tools and integrate with measurement systems/econometric models. - Establish scalable, efficient, automated processes for large scale data analyses, science development, science validation and model implementation. - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes. - Work closely with the business to understand their problem space, identify the opportunities and formulate the problems. - Use AI/machine learning, data mining, statistical techniques and others to create actionable, meaningful, and scalable solutions for the business problems. - Design, develop and evaluate highly innovative models and statistical approaches to understand and predict customer behavior and to solve business problems.
Applied Scientist, Amazon Leo Satellite Build Systems
US, WA, Bellevue
Build the scientific intelligence layer powering Amazon’s satellite manufacturing system. As an Applied Scientist, you will develop machine learning models that transform fragmented manufacturing, test, quality, and operational data into actionable intelligence that improves how satellites are built. You will tackle ambiguous, high-impact problems where data is incomplete, noisy, and distributed, and where model outputs influence real-world manufacturing decisions. Your work will power AI-enabled workflows such as non-conformance disposition, root-cause analysis, and predictive test optimization - reducing defects, accelerating production, and helping create more intelligent, data-driven manufacturing systems. Export Control Requirement: Due to applicable export control laws and regulations, candidates must be a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum. Key job responsibilities - Translate ambiguous manufacturing and operational problems into well-defined scientific problems, modeling approaches, and evaluation criteria - Design, train, and deploy machine learning models, including LLM-based systems, retrieval models, and task-specific models - Develop and evaluate models using large-scale, noisy, heterogeneous datasets with incomplete, delayed, or imperfect ground truth - Apply state-of-the-art techniques in areas such as anomaly detection, root-cause inference, multimodal learning, information retrieval, and generative AI, adapting or extending them to meet project requirements - Design experiments and evaluation frameworks that capture real-world failure modes, distribution shift, and decision risk - Make principled tradeoffs among model complexity, data quality, accuracy, latency, cost, and maintainability - Build production-quality scientific components with appropriate testing, documentation, monitoring, and operational mechanisms - Work with Manufacturing, Quality, Test, and engineering partners to understand customer needs and translate them into effective scientific solutions - Analyze model and system performance, identify gaps and root causes, and iteratively improve deployed solutions - Clearly document scientific approaches, experimental results, design decisions, and lessons learned so that others can understand and reproduce the work - Contribute to technical discussions, mentor less experienced teammates, and help advance scientific and engineering best practices within the team A day in the life You may start by partnering with Quality and Manufacturing teams to define a training dataset for a root-cause prediction model, including how historical cases should be labeled and evaluated. You then design experiments and train models, comparing approaches across architectures, features, and data slices. Later, you analyze benchmark results to identify failure modes, data-quality issues, and generalization gaps, and refine the evaluation set to better represent real-world cases. You work with engineers to integrate the model into a production workflow, adding testing, monitoring, and feedback mechanisms. Throughout the day, you balance scientific rigor with practical constraints such as data availability, latency, reliability, and operational cost. About the team Leo Satellite Build Systems is the centralized AI team within Leo Production Operations. We build shared capabilities for AI across Production Operations, including governed data assets, machine learning models, retrieval systems, evaluation frameworks, and knowledge services. We work on real-world systems where scientific decisions can influence physical outcomes. We value rigorous experimentation, strong data foundations, clear documentation, and production-ready engineering. Our team is helping enable AI-native manufacturing by turning fragmented operational knowledge and data into reliable intelligence that improves production outcomes.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social