Social
Conversational AI

A quick guide to Amazon’s 50-plus papers at EMNLP 2024

Large language models predominate, both as a research subject themselves and as tools for researching topics of particular interest to Amazon, such as speech, recommendations, and information retrieval.

By Staff writer
November 14, 2024
7 min read
Share

Large language models (LLMs) have come to dominate the field of natural-language processing, so it’s no surprise that they also dominate the research that Amazon scientists are presenting at this year’s Conference on Empirical Methods in Natural-Language Processing (EMNLP). LLM training is the topic with the greatest number of Amazon papers, followed closely by strategies for mitigating misinformation in LLMs’ outputs — including but not limited to hallucinations. At the same time, a number of papers apply LLMs to topics of traditional interest at Amazon, such as speech, recommender systems, and information retrieval. (Papers marked with asterisks were accepted to Findings of EMNLP.)

AI agents

MARCO: Multi-agent real-time chat orchestration
Anubhav Shrimal, Shervin Malmasi, Kriti Biswas, Swarnalatha Raghuraman, Anish Nediyanchath, Yi Zhang, Promod Yenigalla

Code generation

CodeFort: Robust training for code generation models
Yuhao Zhang, Shiqi Wang, Haifeng Qian, Zijian Wang, Mingyue Shang, Linbo Liu, Sanjay Krishna Gouda, Baishakhi Ray, Murali Krishna Ramanathan, Xiaofei Ma, Anoop Deoras

Socratic human feedback (SoHF): Expert steering strategies for LLM code generation
Subramanian Chidambaram, Erran Li, Min Bai, Xiaopeng LI, Kaixiang Lin, Xiong Zhou, Alex C. Williams

Structured object language modeling (SoLM): Native structured objects generation conforming to complex schemas with self-supervised denoising
Amir Tavanaei, Kee Kiat Koo, Hayreddin Ceker, Shaobai Jiang, Qi Li, Julien Han, Karim Bouyarmane

Contrastive decoding

Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM
Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal, Anil Ramakrishna, Tagyoung Chung

Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM.png
Given a simple question with clues, contrastive decoding could have an “obvious blindness” (e.g., assigning higher probability to an uncommon answer, such as "invertebrate", than to the most obvious answer, "bees"). In contrast, the asymptotic probability decoding proposed in "Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM" correctly assigns the highest probability to "bees" by leveraging the probabilities from multiple LMs of different sizes.

Data integration

ASTRA: Automatic schema matching using machine translation
Tarang Chugh, Deepak Zambre

Learning from natural language explanations for generalizable entity matching
Somin Wadhwa, Adit Krishnan, Runhui Wang, Byron C. Wallace, Chris (Luyang) Kong

Pretraining and finetuning language models on geospatial networks for accurate address matching
Saket Maheshwary, Arpan Paul, Saurabh Sohoney

Retrieval augmented spelling correction for e-commerce applications
Xuan Guo, Rohit Patki, Dante Everaert, Christopher Potts

Dataset distillation

Textual dataset distillation via language model embedding
Yefan Tao, Chris (Luyang) Kong, Andrey Kan, Laurent Callot

Textual dataset distillation via language model embedding: DaLLME.png
The DaLLME framework proposed in "Textual dataset distillation via language model embedding" begins by using a language model to transform raw textual data into embedding vectors. A set of distilled vectors is then derived in the embedding space, through a process designed to encapsulate maximum informational content. Finally, the vec2text model translates these distilled vectors back into textual form.

Document understanding

DocKD: Knowledge distillation from LLMs for open-world document understanding models
Sungnyun Kim, Haofu Liao, Srikar Appalaraju, Peng Tang, Zhuowen Tu, Ravi Kumar Satzoda, R. Manmatha, Vijay Mahadevan, Stefano Soatto

Information retrieval

Evaluating D-MERIT of partial-annotation on information retrieval
Royi Rassin, Yaron Fairstein, Oren Kalinsky, Guy Kushilevitz, Nachshon Cohen, Alexander Libov, Yoav Goldberg

Identifying high consideration e-commerce search queries
Zhiyu Chen, Jason Choi, Besnik Fetahu, Shervin Malmasi

Learning when to retrieve, what to rewrite, and how to respond in conversational QA*
Nirmal Roy, Leonardo Ribeiro, Rexhina Blloshmi, Kevin Small

Natural-language understanding

Intent detection in the age of LLMs
Gaurav Arora, Shreya Jain, Srujana Merugu

Intent detection in the age of LLMs.png
"Intent detection in the age of LLMs" proposes a methodology for adaptive in-context learning and chain-of-thought-based intent detection using LLMs.

Predicting entity salience in extremely short documents
Ben Bullough, Harrison Lundberg, Chen Hu, Weihang Xiao

LLM evaluation

AXCEL: Automated eXplainable consistency evaluation using LLMs*
P Aditya Sreekar, Sahil Verma, Suransh Chopra, Sarik Ghazarian, Abhishek Persad, Narayanan Sadagopan

Precise model benchmarking with only a few observations
Riccardo Fogliato, Pratik Patil, Nil-Jana Akpinar, Mathew Monfort

LLM fine tuning

AdaZeta: Adaptive zeroth-order tensor-train adaption for memory-efficient large language models fine-tuning
Yifan Yang, Kai Zhen, Ershad Banijamali, Thanasis Mouchtaris, Zheng Zhang

RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning
Haoyu Wang, Tianci Liu, Ruirui Li, Monica Cheng, Tuo Zhao, Jing Gao

RoseLoRA.png
The row- and column-wise sparse low-rank adaptation (RoseLoRA) framework proposed in "RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning".

LLMs for speech

Speechworthy instruction-tuned language models
Hyundong Cho, Nicolaas Jedema, Leonardo Ribeiro, Karishma Sharma, Pedro Szekely, Alessandro Moschitti, Ruben Janssen, Jonathan May

LLM misinformation mitigation

ECON: On the detection and resolution of evidence conflicts
Cheng Jiayang, Chunkit Chan, Qianqian Zhuang, Lin Qiu, Tianhang Zhang, Tengxiao Liu, Yangqiu Song, Yue Zhang, Pengfei Liu, Zheng Zhang

Generative subgraph retrieval for knowledge graph–grounded dialog generation
Jinyoung Park, Minseok Joo, Joo-Kyung Kim, Hyunwoo J. Kim

HalluMeasure: Fine-grained hallucination measurement using chain-of-thought reasoning
Shayan Ali Akbar, Md Mosharaf Hossain, Tess Wood, Si-Chi Chin, Erica Salinas, Victor Alvarez, Erwin Cornejo

Knowledge-centric hallucination detection
Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Zheng Zhang, Yue Zhang

LLM reasoning

Auto-evolve: Enhancing large language model’s performance via self-reasoning framework*
Krishna Aswani, Alex Lu, Pranav Patankar, Priya Dhalwani, Iris Tan, Jayant Ganeshmohan, Simon Lacasse

LLM self-correction

LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints
Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, Nanyun Peng

DeCRIM.png
In the DeCRIM pipeline proposed in "LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints", an LLM first generates a response to a user request. The Decomposer then breaks down the request into granular constraints, and the Critic model gives feedback on whether the response meets those constraints. If it does, the response is output; if not, the LLM uses the feedback to refine the response.

LLM training

Dancing in chains: Reconciling instruction following and faithfulness in language models
Zhengxuan Wu, Yuhao Zhang, Peng Qi, Yumo Xu, Rujun Han, Yian Zhang, Jifan Chen, Bonan Min, Zhiheng Huang

DEM: Distribution edited model for training with mixed data distributions
Dhananjay Ram, Aditya Rawal, Momchil Hardalov, Nikolaos Pappas, Sheng Zha

DEM: Distribution Edited Model for Training with Mixed Data Distributions
The distribution-edited model (ΘD) described in "DEM: Distribution edited model for training with mixed data distributions" results from fine-tuning a pretrained model (Θ) on n individual data distributions (Di) and combining the resulting models with basic element-wise vector operations. Here, the extracted distribution vectors (∆ΘDi ) are multiplied by weight coefficients, and the weighted sum is added to the base model.

Evolutionary contrastive distillation for language model alignment
Julian Katz-Samuels, Zheng Li, Hyokun Yun, Priyanka Nigam, Yi Xu, Vaclav Petricek, Bing Yin, Trishul Chilimbi

Hop, skip, jump to convergence: Dynamics of learning rate transitions for improved training of large language models
Shreyas Subramanian, Vignesh Ganapathiraman, Corey Barrett

Learning from relevant subgoals in successful dialogs using iterative training for task-oriented dialog systems
Magdalena Kaiser, Patrick Ernst, Gyuri Szarvas

Quality matters: Evaluating synthetic data for tool-using LLMs
Shadi Iskander, Nachshon Cohen, Zohar Karnin, Ori Shapira, Sofia Tolmach

Query autocompletion

AmazonQAC: A large-scale, naturalistic query autocomplete dataset
Dante Everaert, Rohit Patki, Tianqi Zheng, Christopher Potts

DiAL: Diversity aware listwise ranking for query auto-complete
Sonali Singh, Sachin Farfade, Prakash Mandayam Comar

Question answering

RAG-QA arena: Evaluating domain robustness for long-form retrieval-augmented question answering
Rujun Han, Yuhao Zhang, Peng Qi, Yumo Xu, Jenyuan Wang, Lan Liu, William Yang Wang, Bonan Min, Vittorio Castelli

Retrieving contextual information for long-form question answering using weak supervision
Philipp Christmann, Svitlana Vakulenko, Ionut Teodor Sorodoc, Bill Byrne, Adrià de Gispert

Recommender systems

Efficient pointwise-pairwise learning-to-rank for news recommendation
Nithish Kannen Senthilkumar, Yao Ma, Gerrit van den Burg, Jean Baptiste Faddoul

Efficient pointwise-pairwise learning-to-rank for news recommendation.png
An illustration of the GLIMPSE framework proposed in "Efficient pointwise-pairwise learning-to-rank for news recommendation". GLIMPSE adopts a multitask approach in which a pretrained language model is fine-tuned on both the relevance prediction task and the pairwise-preference task. During inference, the relevance predictions are used to produce an initial pointwise ranking, which is subsequently improved by one or more right-to-left (RTL) passes using pairwise comparisons.

PEARL: Preference extraction with exemplar augmentation and retrieval with LLM agents
Vijit Malik, Akshay Jagatap, Vinayak Puranik, Anirban Majumder

Sequential LLM framework for fashion recommendation
Han Liu, Xianfeng Tang, Tianlang Chen, Jiapeng Liu, Indu Indu, Henry Peng Zou, Peng Dai, Roberto Fernandez Galan, Mike Porter, Dongmei Jia, Ning Zhang, Lian Xiong

Responsible AI

Attribute controlled fine-tuning for large language models: A case study on detoxification
Tao Meng, Ninareh Mehrabi, Palash Goyal, Anil Ramakrishna, Aram Galstyan, Richard Zemel, Kai-Wei Chang, Rahul Gupta, Charith Peris

FLIRT: Feedback loop in-context red teaming
Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta

Order of magnitude speedups for LLM membership inference
Rongting Zhang, Martin Bertran Lopez, Aaron Roth

Synthetic data generation

CorrSynth: A correlated sampling method for diverse dataset generation from LLMs
Suhas Kowshik, Abhishek Divekar, Vijit Malik

A Correlated Sampling Method for Diverse Dataset Generation from LLMs
"CorrSynth: A correlated sampling method for diverse dataset generation from LLMs" introduces a sampling method that uses anti-correlation between examples rather than few-shot generation.

DATA ADVISOR: Dynamic data curation for safety alignment of large language models
Fei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta, Kai-Wei Chang, Aram Galstyan

Evaluating differentially private synthetic data generation in high-stakes domains
Krithika Ramesh, Nupoor Gandhi, Pulkit Madaan, Lisa Bauer, Charith Peris, Anjalie Field

SYNTHESIZRR: Generating diverse datasets with retrieval augmentation
Abhishek Divekar, Greg Durrett

Abstract depiction of the SYNTHESIZRR procedure
Abstract depiction of the procedure proposed in "SYNTHESIZRR: Generating diverse datasets with retrieval augmentation". The content sourcing stage retrieves K unique documents {r1,...,rK} from a large corpus for each in-context covariate xICL. The task-inversion stage uses a parameterized context refinement prompt, Pτ, which takes parameters Rinv (inversion instruction), rk (a retrieved document), and V(yICL) (the verbalized target label). A generalist teacher LLM autoregressively generates a synthetic covariate. Each in-context example thus produces K unique synthetic examples {x̃1,..., x̃K}, which we include in the dataset with target yICL.

Text classification

Distance-aware calibration for pre-trained language models*
Alberto Gasparin, Gianluca Detommaso

Performance-guided LLM knowledge distillation for efficient text classification at scale

Flavio Di Palo, Prateek Singhi, Bilal Fadlallah

Prompt-tuned muti-task taxonomic transformer (PTMTTaxoFormer)
Rajashekar Vasantha, Nhan Nguyen, Yue Zhang

Text summarization

Salient information prompting to steer content in prompt-based abstractive summarization
Lei Xu, Asad Karim, Saket Dingliwal, Aparna Elangovan

Research areas
Tags
About the Author
Staff writer

Related content

Work with us

See more jobs See more jobs
Data Scientist, Prime Video
US, WA, Seattle
Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. Prime members can customize their viewing experience and find their favorite movies, series, documentaries, and live sports – including Amazon MGM Studios-produced series and movies; licensed fan favorites; and exclusive access to coverage of live sports. All customers regardless of whether they have a Prime membership or not, can access programming from subscriptions such as Apple TV, Peacock Premium Plus, HBO Max, FOX One, Crunchyroll and MGM+, as well as more than 900 free ad-support (FAST) Channels, rent or buy titles, and enjoy even more content for free with ads. The Prime Video Personalization and Discovery team matches customers with the right content at the right time, at all touch points throughout the content discovery journey. We are looking for a customer-focused, solutions-oriented Data Scientist to help build new data-driven frameworks to understand what makes new personalization and content discovery innovations successful for users and the business. You'll be part of an embedded science team on projects that are fast-paced, challenging, and ultimately influence what millions of customers around the world see when the log into Prime Video. The ideal candidate brings strong problem-solving skills, stakeholder communication skills, and the ability to balance technical rigor with delivery speed and customer impact. You will build cross-functional support within Prime Video, assess business problems, define metrics, and support iterative scientific solutions that balance short-term delivery with long-term science roadmaps. Key job responsibilities - Use advanced statistical and machine learning techniques to extract insights from complex, large-scale data sets - Design and implement end-to-end data science workflows, from data acquisition and cleaning to model development, testing, and deployment - Support scalable, self-service data analyses by building datasets for analytics, reporting and ML use cases - Partner with product stakeholders and senior science peers to identify strategic data-driven opportunities to improve the customer experience - Communicate findings, conclusions, and recommendations to technical and non-technical stakeholders - Stay up-to-date on the latest data science tools, techniques, and best practices and help evangelize them across the organization
Applied Scientist, Amazon Music - Catalog Quality
IN, KA, Bengaluru
Amazon Music is an immersive audio entertainment service that deepens connections between fans, artists, and creators. From personalized music playlists to exclusive podcasts, concert livestreams to artist merch, Amazon Music is innovating at some of the most exciting intersections of music and culture. We offer experiences that serve all listeners with our different tiers of service: Prime members get access to all the music in shuffle mode, and top ad-free podcasts, included with their membership; customers can upgrade to Amazon Music Unlimited for unlimited, on-demand access to 100 million songs, including millions in HD, Ultra HD, and spatial audio; and anyone can listen for free by downloading the Amazon Music app or via Alexa-enabled devices. Join us for the opportunity to influence how Amazon Music engages fans, artists, and creators on a global scale. Learn more at https://www.amazon.com/music. The Music Catalog Quality team at Amazon Music serves a key role in developing solutions to ensure and improve the quality of catalog metadata and content across the music streaming experience. We create solutions that detect, measure, and remediate quality issues in music metadata - including artist information, track attributes, versions, content tags, and provide actionable insights that enable continuous improvement of the catalog. We leverage a host of scientific and engineering technologies to accomplish this mission, including Generative AI, classical ML, Natural Language Processing, Computer Vision, and automated data validation pipelines. Key job responsibilities As an Applied Scientist, you will own the design and development of end-to-end systems. You’ll have the opportunity to create technical roadmaps, and drive production level projects that will support Amazon Science. You will work closely with Amazon scientists, and other science interns to develop solutions and deploy them into production. The ideal scientist must have the ability to work with diverse groups of people and cross-functional teams to solve complex business problems. Other responsibilities include: - Collaborate with scientists, engineers, and product managers to define and frame business problems as ML or optimization tasks. - Use machine learning, deep learning, LLMs and Agentic AI techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon's data to help automate and optimize key processes - Design, development and evaluation of AI models for predictive learning - Research and implement novel machine learning and statistical approaches - Implement scalable data pipelines and model-serving systems. - Analyze experimental results, draw insights, and refine models to improve accuracy and robustness. - Communicate findings and recommendations to technical and non-technical audiences.
Applied Scientist, Amazon Music - Catalog Quality
IN, KA, Bengaluru
Amazon Music is an immersive audio entertainment service that deepens connections between fans, artists, and creators. From personalized music playlists to exclusive podcasts, concert livestreams to artist merch, Amazon Music is innovating at some of the most exciting intersections of music and culture. We offer experiences that serve all listeners with our different tiers of service: Prime members get access to all the music in shuffle mode, and top ad-free podcasts, included with their membership; customers can upgrade to Amazon Music Unlimited for unlimited, on-demand access to 100 million songs, including millions in HD, Ultra HD, and spatial audio; and anyone can listen for free by downloading the Amazon Music app or via Alexa-enabled devices. Join us for the opportunity to influence how Amazon Music engages fans, artists, and creators on a global scale. Learn more at https://www.amazon.com/music. The Music Catalog Quality team at Amazon Music serves a key role in developing solutions to ensure and improve the quality of catalog metadata and content across the music streaming experience. We create solutions that detect, measure, and remediate quality issues in music metadata - including artist information, track attributes, versions, content tags, and provide actionable insights that enable continuous improvement of the catalog. We leverage a host of scientific and engineering technologies to accomplish this mission, including Generative AI, classical ML, Natural Language Processing, Computer Vision, and automated data validation pipelines. Key job responsibilities As an Applied Scientist, you will own the design and development of end-to-end systems. You’ll have the opportunity to create technical roadmaps, and drive production level projects that will support Amazon Science. You will work closely with Amazon scientists, and other science interns to develop solutions and deploy them into production. The ideal scientist must have the ability to work with diverse groups of people and cross-functional teams to solve complex business problems. Other responsibilities include: - Collaborate with scientists, engineers, and product managers to define and frame business problems as ML or optimization tasks. - Use machine learning, deep learning, LLMs and Agentic AI techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon's data to help automate and optimize key processes - Design, development and evaluation of AI models for predictive learning - Research and implement novel machine learning and statistical approaches - Implement scalable data pipelines and model-serving systems. - Analyze experimental results, draw insights, and refine models to improve accuracy and robustness. - Communicate findings and recommendations to technical and non-technical audiences.
Applied Scientist, Amazon Music - Catalog Quality
IN, KA, Bengaluru
Amazon Music is an immersive audio entertainment service that deepens connections between fans, artists, and creators. From personalized music playlists to exclusive podcasts, concert livestreams to artist merch, Amazon Music is innovating at some of the most exciting intersections of music and culture. We offer experiences that serve all listeners with our different tiers of service: Prime members get access to all the music in shuffle mode, and top ad-free podcasts, included with their membership; customers can upgrade to Amazon Music Unlimited for unlimited, on-demand access to 100 million songs, including millions in HD, Ultra HD, and spatial audio; and anyone can listen for free by downloading the Amazon Music app or via Alexa-enabled devices. Join us for the opportunity to influence how Amazon Music engages fans, artists, and creators on a global scale. Learn more at https://www.amazon.com/music. The Music Catalog Quality team at Amazon Music serves a key role in developing solutions to ensure and improve the quality of catalog metadata and content across the music streaming experience. We create solutions that detect, measure, and remediate quality issues in music metadata - including artist information, track attributes, versions, content tags, and provide actionable insights that enable continuous improvement of the catalog. We leverage a host of scientific and engineering technologies to accomplish this mission, including Generative AI, classical ML, Natural Language Processing, Computer Vision, and automated data validation pipelines. Key job responsibilities As an Applied Scientist, you will own the design and development of end-to-end systems. You’ll have the opportunity to create technical roadmaps, and drive production level projects that will support Amazon Science. You will work closely with Amazon scientists, and other science interns to develop solutions and deploy them into production. The ideal scientist must have the ability to work with diverse groups of people and cross-functional teams to solve complex business problems. Other responsibilities include: - Collaborate with scientists, engineers, and product managers to define and frame business problems as ML or optimization tasks. - Use machine learning, deep learning, LLMs and Agentic AI techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon's data to help automate and optimize key processes - Design, development and evaluation of AI models for predictive learning - Research and implement novel machine learning and statistical approaches - Implement scalable data pipelines and model-serving systems. - Analyze experimental results, draw insights, and refine models to improve accuracy and robustness. - Communicate findings and recommendations to technical and non-technical audiences.
Applied Scientist (GenAI/LLM), Sandstone
US, CA, San Diego
Do you want to join an innovative team of scientists and engineers who use terabytes of data and create state-of-the-art Generative AI algorithms to push the boundaries of AI creativity? We are building foundational behavioral models for Amazon Stores using Generative AI, LLMs and Large Model training techniques that fuses general world knowledge, customer shopping behavior and Amazon e-commerce domain knowledge. We are looking for scientists who are passionate about technology, innovation, and customer experience, and are ready to make a lasting impact on the industry using intelligent and transformative AI applications. Working closely with cross-functional teams, you will be an essential part of every stage of AI development, from ideation and design to rigorous testing and successful deployment, ensuring our AI projects drive innovation and provide value for our customers. If you’re fired up about being part of a dynamic, driven team, then this is your moment to join us on this exciting journey! Key job responsibilities In this role you will leverage your background and expertise to lead developing foundational behavioral model for Amazon Stores using Generative AI, LLM and Large Model training techniques. On a day-to-day basis, you will: - Research and implement new algorithms and architectures for generative AI applications. - Optimize model performance and scalability for inference and deployment. - Collaborate with other talented applied scientists and engineers to gather and preprocess large datasets and develop an improved training infrastructure that accelerates innovation. - Experiment with SOTA methods to improve generative AI model quality. - Provide technical expertise and guidance to support the integration of generative AI solutions into various products and services.
Applied Science Manager , Amazon Music Catalog Quality
IN, KA, Bengaluru
The Music Catalog Quality team at Amazon Music serves a key role in developing solutions to ensure and improve the quality of catalog metadata and content across the music streaming experience. We create solutions that detect, measure, and remediate quality issues in music metadata - including artist information, track attributes, versions, content tags, and provide actionable insights that enable continuous improvement of the catalog. We leverage a host of scientific and engineering technologies to accomplish this mission, including Generative AI, classical ML, Natural Language Processing, Computer Vision, and automated data validation pipelines. As an Applied Science Manager on the team, you will lead a team of scientists to define and execute a transformative vision for holistic catalog quality measurement, metadata enrichment, and content integrity. Your team will own the science solutions for foundational quality detection frameworks, metadata validation and correction technologies, state-of-the-art algorithms to identify and resolve catalog anomalies (violative content, duplicative/low value content, misattributed tracks, incorrect metadata), and/or agentic AI solutions that help internal teams quickly surface and fix quality issues to ensure customers receive accurate, complete catalog experiences. Key job responsibilities You independently manage a team of scientists. You identify the needs of your team and effectively grow, hire, and promote scientists to maintain a high-performing team. You have a broad understanding of scientific techniques, several of which may fall out of your specific job function. You define the strategic vision for your team. You establish a roadmap and successfully deliver scientific solutions that innovate on catalog quality detection, metadata enrichment, and content integrity. You define clear goals for your team and effectively prioritize, balancing short-term quality improvements and long-term innovation in catalog intelligence. You establish clear and effective metrics and scientific process to enforce consistent, high-quality artifact delivery and measurable catalog quality improvements. You proactively identify risks and bring them to the attention of your manager, customers, and stakeholders with plans for mitigation before they become roadblocks. You know when to escalate. You communicate ideas effectively, both verbally and in writing, to all types of audiences. You author strategic documentation for your team. You communicate issues and options with leaders in such a way that facilitates understanding and that leads to a decision. You work successfully with customers, leaders, and engineering teams. You foster a constructive dialogue, harmonize discordant views, and lead the resolution of contentious issues. About the team We are a team of scientists and MLEs focused on music catalog quality and metadata intelligence. You will work with colleagues with deep expertise in ML, NLP, CV, Gen AI, and data quality systems with a diverse range of backgrounds. We partner closely with top-notch engineers, product managers, content operations teams, and other scientists with expertise in music metadata, content classification, and building scalable modeling and software solutions that keep the Amazon Music catalog accurate, complete, and trustworthy.
Applied Scientist (GenAI/LLM), Sandstone
US, CA, San Diego
Do you want to join an innovative team of scientists and engineers who use terabytes of data and create state-of-the-art Generative AI algorithms to push the boundaries of AI creativity? We are building foundational behavioral models for Amazon Stores using Generative AI, LLMs and Large Model training techniques that fuses general world knowledge, customer shopping behavior and Amazon e-commerce domain knowledge. We are looking for scientists who are passionate about technology, innovation, and customer experience, and are ready to make a lasting impact on the industry using intelligent and transformative AI applications. Working closely with cross-functional teams, you will be an essential part of every stage of AI development, from ideation and design to rigorous testing and successful deployment, ensuring our AI projects drive innovation and provide value for our customers. If you’re fired up about being part of a dynamic, driven team, then this is your moment to join us on this exciting journey! Key job responsibilities In this role you will leverage your background and expertise to lead developing foundational behavioral model for Amazon Stores using Generative AI, LLM and Large Model training techniques. On a day-to-day basis, you will: - Research and implement new algorithms and architectures for generative AI applications. - Optimize model performance and scalability for inference and deployment. - Collaborate with other talented applied scientists and engineers to gather and preprocess large datasets and develop an improved training infrastructure that accelerates innovation. - Experiment with SOTA methods to improve generative AI model quality. - Provide technical expertise and guidance to support the integration of generative AI solutions into various products and services.
Applied Scientist (GenAI/LLM), Sandstone
US, WA, Seattle
Do you want to join an innovative team of scientists and engineers who use terabytes of data and create state-of-the-art Generative AI algorithms to push the boundaries of AI creativity? We are building foundational behavioral models for Amazon Stores using Generative AI, LLMs and Large Model training techniques that fuses general world knowledge, customer shopping behavior and Amazon e-commerce domain knowledge. We are looking for scientists who are passionate about technology, innovation, and customer experience, and are ready to make a lasting impact on the industry using intelligent and transformative AI applications. Working closely with cross-functional teams, you will be an essential part of every stage of AI development, from ideation and design to rigorous testing and successful deployment, ensuring our AI projects drive innovation and provide value for our customers. If you’re fired up about being part of a dynamic, driven team, then this is your moment to join us on this exciting journey! Key job responsibilities In this role you will leverage your background and expertise to lead developing foundational behavioral model for Amazon Stores using Generative AI, LLM and Large Model training techniques. On a day-to-day basis, you will: - Research and implement new algorithms and architectures for generative AI applications. - Optimize model performance and scalability for inference and deployment. - Collaborate with other talented applied scientists and engineers to gather and preprocess large datasets and develop an improved training infrastructure that accelerates innovation. - Experiment with SOTA methods to improve generative AI model quality. - Provide technical expertise and guidance to support the integration of generative AI solutions into various products and services.
Applied Scientist III, Alexa Edge AI
US, MA, Boston
The Alexa Edge AI team is looking for a passionate, talented, and innovative senior applied scientist with a strong background in deep learning and speech processing techniques. You will have an enormous opportunity to impact the customer experience, design, architecture, and implementation of an industry-leading product used every day by people you know. As an Applied Scientist, you will leverage Amazon’s heterogeneous data sources and large-scale computing resources to develop novel machine learning algorithms to advance the state of the art in speech and audio processing. Key job responsibilities · Conduct applied research project(s) effectively and know when to ask for help and when to work independently · Engage with an experienced cross-disciplinary staff to conceive and design innovative solutions for consumer products. · Actively participate and contribute to research activities including publications and patents · Work closely with an internal inter-disciplinary team, and outside partners to drive key aspects of product definition, execution and test. · Be proactive, flexible and able to succeed within an open collaborative peer environment
Applied Scientist, Safe RL, Robotics, SAF Lab
US, CA, Pasadena
We are seeking an Applied Scientist to join the SAF Lab. In this role, you will lead the effort in safe reinforcement learning (RL) including the development of legged locomotion algorithms that internalize safety and are deployable on physical hardware—enabling highly dynamic robots to walk, run, avoid collisions and recover from disturbances with agility and robustness. You will develop RL architectures that interface with physics-based models (for dynamic retargeting and reward shaping), internalize safety constraints in training, sim-to-real transfer and interface with safety filters at run-time. Therefore, your work will sit at the intersection of safety-critical control and learning, and you will collaborate with others in the SAF Lab and Amazon working on perception, planning, whole-body and safety-critical control. This is an opportunity to shape the foundations of safe learning on emerging platforms that will remove bottlenecks to deployment and enable these robots to safely operate around humans. Key job responsibilities • Collaborate with product teams and science leaders to set a science roadmap (with eventual impact on real robots). • Design, train, and deploy reinforcement learning (RL) policies for dynamic legged locomotion including walking, running, stair climbing, and fall recovery on physical robots • Develop sim-to-real transfer pipelines that produce policies robust to the reality gap, including domain randomization, system identification, and adaptive strategies • Integrate control-based methods with RL, as inputs to the RL (dynamic retargeting and control-guided rewards), in training (internalizing safety constraints in training), and as the RL feeds into safety layers and whole-body control • Develop and maintain large-scale training infrastructure for locomotion policy learning, including physics simulation environments, domain randomization and GPU parallelization • Investigate the distillation of locomotion policies, integration with whole-body control, foundation models, VLAs, world models, perception and full-stack autonomy • Evaluate policy performance rigorously through simulation benchmarks, hardware experiments, and failure-mode analysis • Publish research at top-tier robotics and ML venues and contribute to Amazon's scientific reputation in advanced robotics • Collaborate with perception and planning teams to enable terrain-aware and goal-conditioned locomotion behaviors A day in the life Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! About the team Work with the inventor of control barrier functions in the Safe Autonomy Frontiers (SAF) Lab. The first industry research lab in safe autonomy, developing a universal safety layer for the next generation of robotic systems: mobile robots, manipulators, mobile manipulators, and future platforms with dynamic stability. You will push the frontiers of performant safety for highly dynamic robots: CBF theory integrated with perception and learning, evaluated on next-generation robots. Your work will underpin robots operating alongside people at Amazon's unprecedented scale.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social