Social
Conversational AI

A quick guide to Amazon’s 50-plus papers at EMNLP 2024

Large language models predominate, both as a research subject themselves and as tools for researching topics of particular interest to Amazon, such as speech, recommendations, and information retrieval.

By Staff writer
November 14, 2024
7 min read
Share

Large language models (LLMs) have come to dominate the field of natural-language processing, so it’s no surprise that they also dominate the research that Amazon scientists are presenting at this year’s Conference on Empirical Methods in Natural-Language Processing (EMNLP). LLM training is the topic with the greatest number of Amazon papers, followed closely by strategies for mitigating misinformation in LLMs’ outputs — including but not limited to hallucinations. At the same time, a number of papers apply LLMs to topics of traditional interest at Amazon, such as speech, recommender systems, and information retrieval. (Papers marked with asterisks were accepted to Findings of EMNLP.)

AI agents

MARCO: Multi-agent real-time chat orchestration
Anubhav Shrimal, Shervin Malmasi, Kriti Biswas, Swarnalatha Raghuraman, Anish Nediyanchath, Yi Zhang, Promod Yenigalla

Code generation

CodeFort: Robust training for code generation models
Yuhao Zhang, Shiqi Wang, Haifeng Qian, Zijian Wang, Mingyue Shang, Linbo Liu, Sanjay Krishna Gouda, Baishakhi Ray, Murali Krishna Ramanathan, Xiaofei Ma, Anoop Deoras

Socratic human feedback (SoHF): Expert steering strategies for LLM code generation
Subramanian Chidambaram, Erran Li, Min Bai, Xiaopeng LI, Kaixiang Lin, Xiong Zhou, Alex C. Williams

Structured object language modeling (SoLM): Native structured objects generation conforming to complex schemas with self-supervised denoising
Amir Tavanaei, Kee Kiat Koo, Hayreddin Ceker, Shaobai Jiang, Qi Li, Julien Han, Karim Bouyarmane

Contrastive decoding

Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM
Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal, Anil Ramakrishna, Tagyoung Chung

Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM.png
Given a simple question with clues, contrastive decoding could have an “obvious blindness” (e.g., assigning higher probability to an uncommon answer, such as "invertebrate", than to the most obvious answer, "bees"). In contrast, the asymptotic probability decoding proposed in "Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM" correctly assigns the highest probability to "bees" by leveraging the probabilities from multiple LMs of different sizes.

Data integration

ASTRA: Automatic schema matching using machine translation
Tarang Chugh, Deepak Zambre

Learning from natural language explanations for generalizable entity matching
Somin Wadhwa, Adit Krishnan, Runhui Wang, Byron C. Wallace, Chris (Luyang) Kong

Pretraining and finetuning language models on geospatial networks for accurate address matching
Saket Maheshwary, Arpan Paul, Saurabh Sohoney

Retrieval augmented spelling correction for e-commerce applications
Xuan Guo, Rohit Patki, Dante Everaert, Christopher Potts

Dataset distillation

Textual dataset distillation via language model embedding
Yefan Tao, Chris (Luyang) Kong, Andrey Kan, Laurent Callot

Textual dataset distillation via language model embedding: DaLLME.png
The DaLLME framework proposed in "Textual dataset distillation via language model embedding" begins by using a language model to transform raw textual data into embedding vectors. A set of distilled vectors is then derived in the embedding space, through a process designed to encapsulate maximum informational content. Finally, the vec2text model translates these distilled vectors back into textual form.

Document understanding

DocKD: Knowledge distillation from LLMs for open-world document understanding models
Sungnyun Kim, Haofu Liao, Srikar Appalaraju, Peng Tang, Zhuowen Tu, Ravi Kumar Satzoda, R. Manmatha, Vijay Mahadevan, Stefano Soatto

Information retrieval

Evaluating D-MERIT of partial-annotation on information retrieval
Royi Rassin, Yaron Fairstein, Oren Kalinsky, Guy Kushilevitz, Nachshon Cohen, Alexander Libov, Yoav Goldberg

Identifying high consideration e-commerce search queries
Zhiyu Chen, Jason Choi, Besnik Fetahu, Shervin Malmasi

Learning when to retrieve, what to rewrite, and how to respond in conversational QA*
Nirmal Roy, Leonardo Ribeiro, Rexhina Blloshmi, Kevin Small

Natural-language understanding

Intent detection in the age of LLMs
Gaurav Arora, Shreya Jain, Srujana Merugu

Intent detection in the age of LLMs.png
"Intent detection in the age of LLMs" proposes a methodology for adaptive in-context learning and chain-of-thought-based intent detection using LLMs.

Predicting entity salience in extremely short documents
Ben Bullough, Harrison Lundberg, Chen Hu, Weihang Xiao

LLM evaluation

AXCEL: Automated eXplainable consistency evaluation using LLMs*
P Aditya Sreekar, Sahil Verma, Suransh Chopra, Sarik Ghazarian, Abhishek Persad, Narayanan Sadagopan

Precise model benchmarking with only a few observations
Riccardo Fogliato, Pratik Patil, Nil-Jana Akpinar, Mathew Monfort

LLM fine tuning

AdaZeta: Adaptive zeroth-order tensor-train adaption for memory-efficient large language models fine-tuning
Yifan Yang, Kai Zhen, Ershad Banijamali, Thanasis Mouchtaris, Zheng Zhang

RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning
Haoyu Wang, Tianci Liu, Ruirui Li, Monica Cheng, Tuo Zhao, Jing Gao

RoseLoRA.png
The row- and column-wise sparse low-rank adaptation (RoseLoRA) framework proposed in "RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning".

LLMs for speech

Speechworthy instruction-tuned language models
Hyundong Cho, Nicolaas Jedema, Leonardo Ribeiro, Karishma Sharma, Pedro Szekely, Alessandro Moschitti, Ruben Janssen, Jonathan May

LLM misinformation mitigation

ECON: On the detection and resolution of evidence conflicts
Cheng Jiayang, Chunkit Chan, Qianqian Zhuang, Lin Qiu, Tianhang Zhang, Tengxiao Liu, Yangqiu Song, Yue Zhang, Pengfei Liu, Zheng Zhang

Generative subgraph retrieval for knowledge graph–grounded dialog generation
Jinyoung Park, Minseok Joo, Joo-Kyung Kim, Hyunwoo J. Kim

HalluMeasure: Fine-grained hallucination measurement using chain-of-thought reasoning
Shayan Ali Akbar, Md Mosharaf Hossain, Tess Wood, Si-Chi Chin, Erica Salinas, Victor Alvarez, Erwin Cornejo

Knowledge-centric hallucination detection
Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Zheng Zhang, Yue Zhang

LLM reasoning

Auto-evolve: Enhancing large language model’s performance via self-reasoning framework*
Krishna Aswani, Alex Lu, Pranav Patankar, Priya Dhalwani, Iris Tan, Jayant Ganeshmohan, Simon Lacasse

LLM self-correction

LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints
Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, Nanyun Peng

DeCRIM.png
In the DeCRIM pipeline proposed in "LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints", an LLM first generates a response to a user request. The Decomposer then breaks down the request into granular constraints, and the Critic model gives feedback on whether the response meets those constraints. If it does, the response is output; if not, the LLM uses the feedback to refine the response.

LLM training

Dancing in chains: Reconciling instruction following and faithfulness in language models
Zhengxuan Wu, Yuhao Zhang, Peng Qi, Yumo Xu, Rujun Han, Yian Zhang, Jifan Chen, Bonan Min, Zhiheng Huang

DEM: Distribution edited model for training with mixed data distributions
Dhananjay Ram, Aditya Rawal, Momchil Hardalov, Nikolaos Pappas, Sheng Zha

DEM: Distribution Edited Model for Training with Mixed Data Distributions
The distribution-edited model (ΘD) described in "DEM: Distribution edited model for training with mixed data distributions" results from fine-tuning a pretrained model (Θ) on n individual data distributions (Di) and combining the resulting models with basic element-wise vector operations. Here, the extracted distribution vectors (∆ΘDi ) are multiplied by weight coefficients, and the weighted sum is added to the base model.

Evolutionary contrastive distillation for language model alignment
Julian Katz-Samuels, Zheng Li, Hyokun Yun, Priyanka Nigam, Yi Xu, Vaclav Petricek, Bing Yin, Trishul Chilimbi

Hop, skip, jump to convergence: Dynamics of learning rate transitions for improved training of large language models
Shreyas Subramanian, Vignesh Ganapathiraman, Corey Barrett

Learning from relevant subgoals in successful dialogs using iterative training for task-oriented dialog systems
Magdalena Kaiser, Patrick Ernst, Gyuri Szarvas

Quality matters: Evaluating synthetic data for tool-using LLMs
Shadi Iskander, Nachshon Cohen, Zohar Karnin, Ori Shapira, Sofia Tolmach

Query autocompletion

AmazonQAC: A large-scale, naturalistic query autocomplete dataset
Dante Everaert, Rohit Patki, Tianqi Zheng, Christopher Potts

DiAL: Diversity aware listwise ranking for query auto-complete
Sonali Singh, Sachin Farfade, Prakash Mandayam Comar

Question answering

RAG-QA arena: Evaluating domain robustness for long-form retrieval-augmented question answering
Rujun Han, Yuhao Zhang, Peng Qi, Yumo Xu, Jenyuan Wang, Lan Liu, William Yang Wang, Bonan Min, Vittorio Castelli

Retrieving contextual information for long-form question answering using weak supervision
Philipp Christmann, Svitlana Vakulenko, Ionut Teodor Sorodoc, Bill Byrne, Adrià de Gispert

Recommender systems

Efficient pointwise-pairwise learning-to-rank for news recommendation
Nithish Kannen Senthilkumar, Yao Ma, Gerrit van den Burg, Jean Baptiste Faddoul

Efficient pointwise-pairwise learning-to-rank for news recommendation.png
An illustration of the GLIMPSE framework proposed in "Efficient pointwise-pairwise learning-to-rank for news recommendation". GLIMPSE adopts a multitask approach in which a pretrained language model is fine-tuned on both the relevance prediction task and the pairwise-preference task. During inference, the relevance predictions are used to produce an initial pointwise ranking, which is subsequently improved by one or more right-to-left (RTL) passes using pairwise comparisons.

PEARL: Preference extraction with exemplar augmentation and retrieval with LLM agents
Vijit Malik, Akshay Jagatap, Vinayak Puranik, Anirban Majumder

Sequential LLM framework for fashion recommendation
Han Liu, Xianfeng Tang, Tianlang Chen, Jiapeng Liu, Indu Indu, Henry Peng Zou, Peng Dai, Roberto Fernandez Galan, Mike Porter, Dongmei Jia, Ning Zhang, Lian Xiong

Responsible AI

Attribute controlled fine-tuning for large language models: A case study on detoxification
Tao Meng, Ninareh Mehrabi, Palash Goyal, Anil Ramakrishna, Aram Galstyan, Richard Zemel, Kai-Wei Chang, Rahul Gupta, Charith Peris

FLIRT: Feedback loop in-context red teaming
Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta

Order of magnitude speedups for LLM membership inference
Rongting Zhang, Martin Bertran Lopez, Aaron Roth

Synthetic data generation

CorrSynth: A correlated sampling method for diverse dataset generation from LLMs
Suhas Kowshik, Abhishek Divekar, Vijit Malik

A Correlated Sampling Method for Diverse Dataset Generation from LLMs
"CorrSynth: A correlated sampling method for diverse dataset generation from LLMs" introduces a sampling method that uses anti-correlation between examples rather than few-shot generation.

DATA ADVISOR: Dynamic data curation for safety alignment of large language models
Fei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta, Kai-Wei Chang, Aram Galstyan

Evaluating differentially private synthetic data generation in high-stakes domains
Krithika Ramesh, Nupoor Gandhi, Pulkit Madaan, Lisa Bauer, Charith Peris, Anjalie Field

SYNTHESIZRR: Generating diverse datasets with retrieval augmentation
Abhishek Divekar, Greg Durrett

Abstract depiction of the SYNTHESIZRR procedure
Abstract depiction of the procedure proposed in "SYNTHESIZRR: Generating diverse datasets with retrieval augmentation". The content sourcing stage retrieves K unique documents {r1,...,rK} from a large corpus for each in-context covariate xICL. The task-inversion stage uses a parameterized context refinement prompt, Pτ, which takes parameters Rinv (inversion instruction), rk (a retrieved document), and V(yICL) (the verbalized target label). A generalist teacher LLM autoregressively generates a synthetic covariate. Each in-context example thus produces K unique synthetic examples {x̃1,..., x̃K}, which we include in the dataset with target yICL.

Text classification

Distance-aware calibration for pre-trained language models*
Alberto Gasparin, Gianluca Detommaso

Performance-guided LLM knowledge distillation for efficient text classification at scale

Flavio Di Palo, Prateek Singhi, Bilal Fadlallah

Prompt-tuned muti-task taxonomic transformer (PTMTTaxoFormer)
Rajashekar Vasantha, Nhan Nguyen, Yue Zhang

Text summarization

Salient information prompting to steer content in prompt-based abstractive summarization
Lei Xu, Asad Karim, Saket Dingliwal, Aparna Elangovan

Research areas
Tags
About the Author
Staff writer

Related content

Work with us

See more jobs See more jobs
Data Scientist II, Last Mile Science
IN, KA, Bangalore
Does the thought of improving one of the world’s most complex logistic systems inspire you? Is your passion to sift through hundreds of systems, processes, and data sources to solve the puzzle and identify the next big opportunity? Are you a creative big thinker who is passionate about using data to direct decision making and solve complex and large-scale challenges? Are you fascinated by the interactions between operations and strategy? Do you feel like your skills uniquely qualify you to bridge communication between teams with competing priorities? If so, then this position is for you! Come help Amazon create state-of-the-art science-driven technologies for delivering packages to the doorstep of our customers! The Last Mile Routing & Planning organization builds the software, algorithms and tools that make the “magic” of home delivery happen: our flow, sort, dispatch and routing intelligence systems are responsible for the billions of daily decisions needed to plan and execute safe, efficient and frustration-free routes for drivers around the world. Our team supports deliveries (and pickups!) for Amazon Logistics, Same Day, Amazon Grocery, Lockers, and other new initiatives across the world. Key job responsibilities In this role, your main focus will be to apply algorithms, synthesize information, identify business opportunities, provide data-driven insights and communicate business and technical requirements within the team and across stakeholder groups. You will partner closely with other scientists and engineers in a collegial environment with a clear path to business impact. We have an exciting portfolio of research areas including vehicle routing, planning for electric and autonomous vehicles, district and stops planning, ultra-fast deliveries, fleet planning, and forecasting solutions for different delivery programs leveraging the latest OR, ML, and Generative AI methods, at a global scale. Successful candidates will have a deep knowledge of Operations Research and/or Machine/Deep Learning methods, experience in applying these methods to large-scale business problems, the ability to map models into production-worthy code in Python or Java, the communication skills necessary to explain complex technical approaches to a variety of stakeholders and customers, and the excitement to take iterative approaches to tackle big research challenges.
Applied Scientist, Fauna
US, NY, New York
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers About the team Fauna Robotics, an Amazon company, is building capable, safe, and genuinely delightful robots for everyday life. Our goal is simple: make robots people actually want to live and interact with in everyday human spaces. We believe that future won’t arrive until building for robotics becomes far more accessible. Today, too much effort is spent reinventing the fundamentals. We’re changing that by developing tightly integrated hardware and software systems that make it faster, safer, and more intuitive to create real-world robotic products. Our work spans the full stack: mechanical design, control systems, dynamic modeling, and intelligent software. The focus is not just functionality, but experience. We’re building robots that feel responsive, expressive, and genuinely useful. At Fauna, you’ll work at the frontier of this space, helping define how robots move, manipulate, and interact with people in natural environments. It’s an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you.
Applied Scientist II, Foundation Model
US, CA, Sunnyvale
Industrial is seeking exceptional talent to help develop the next generation of advanced robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine innovative AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic manipulation, locomotion, and human-robot interaction. This role presents an opportunity to shape the future of robotics through innovative applications of deep learning and large language models. We leverage advanced robotics, machine learning, and artificial intelligence to solve complex operational challenges at unprecedented scale. Our fleet of robots operates across hundreds of facilities worldwide, working in sophisticated coordination to fulfill our mission of customer excellence. We are pioneering the development of robotics foundation models that: - Enable unprecedented generalization across diverse tasks - Integrate multi-modal learning capabilities (visual, tactile, linguistic) - Accelerate skill acquisition through demonstration learning - Enhance robotic perception and environmental understanding - Streamline development processes through reusable capabilities The ideal candidate will contribute to research that bridges the gap between theoretical advancement and practical implementation in robotics. You will be part of a team that's revolutionizing how robots learn, adapt, and interact with their environment. Join us in building the next generation of intelligent robotics systems that will transform the future of automation and human-robot collaboration. As an Applied Scientist, you will develop and improve machine learning systems that help robots perceive, reason, and act in real-world environments. You will leverage state-of-the-art models (open source and internal research), evaluate them on representative tasks, and adapt/optimize them to meet robustness, safety, and performance needs. You will invent new algorithms where gaps exist. You’ll collaborate closely with research, controls, hardware, and product-facing teams, and your outputs will be used by downstream teams to further customize and deploy on specific robot embodiments. Key job responsibilities As an Applied Scientist in the Foundations Model team, you will: - Leverage state-of-the-art models for targeted tasks, environments, and robot embodiments through fine-tuning and optimization. - Execute rapid, rigorous experimentation with reproducible results and solid engineering practices, closing the gap between sim and real environments. - Build and run capability evaluations/benchmarks to clearly profile performance, generalization, and failure modes. - Contribute to the data and training workflow: collection/curation, dataset quality/provenance, and repeatable training recipes. - Write clean, maintainable, well commented and documented code, contribute to training infrastructure, create tools for model evaluation and testing, and implement necessary APIs - Stay current with latest developments in foundation models and robotics, assist in literature reviews and research documentation, prepare technical reports and presentations, and contribute to research discussions and brainstorming sessions. - Work closely with senior scientists, engineers, and leaders across multiple teams, participate in knowledge sharing, support integration efforts with robotics hardware teams, and help document best practices and methodologies.
Manager, Research Science, Special Projects
US, CA, San Francisco
Innovators wanted! Are you an entrepreneur? A builder? A dreamer? This role is part of an Amazon Special Projects team that takes the company’s Think Big leadership principle to the extreme. We focus on creating entirely new products and services with a goal of positively impacting the lives of our customers. No industries or subject areas are out of bounds. If you’re interested in innovating at scale to address big challenges in the world, this is the team for you. Here at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have thirteen employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We are constantly learning through programs that are local, regional, and global. Amazon’s culture of inclusion is reinforced within our 16 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Our team highly values work-life balance, mentorship and career growth. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We care about your career growth and strive to assign projects and offer training that will challenge you to become your best.
Applied Scientist, Sponsored Products and Brands
US, WA, Seattle
About Sponsored Products and Brands The Sponsored Products and Brands (SPB) team at Amazon Ads is re-imagining the advertising landscape through generative AI technologies, revolutionizing how millions of customers discover products and engage with brands across Amazon.com and beyond. We are at the forefront of re-inventing advertising experiences, bridging human creativity with artificial intelligence to transform every aspect of the advertising lifecycle from ad creation and optimization to performance analysis and customer insights. We are a passionate group of innovators dedicated to developing responsible and intelligent AI technologies that balance the needs of advertisers, enhance the shopping experience, and strengthen the marketplace. If you're energized by solving complex challenges and pushing the boundaries of what's possible with AI, join us in shaping the future of advertising. About our team The General Shopping Intelligence (GSI) team is a highly motivated, collaborative, and fun-loving group with a strong entrepreneurial spirit and bias for action. We provide advanced real-time machine learning services that connect shoppers with the right ads across all platforms and surfaces worldwide. Through deep understanding of both shoppers and products, we help shoppers discover new products they love, enable advertisers to reach their customers most efficiently, and help Amazon continuously innovate on behalf of all customers. We are seeking a motivated Applied Scientist who loves to innovate at the intersection of customer experience, deep learning, generative AI and high-scale machine learning systems. If you're energized by solving complex challenges and pushing the boundaries of what's possible with AI, join us in shaping the future of advertising. Key job responsibilities As an Applied Scientist, you will: * Leverage Generative AI and Large Language Models (LLMs) to mine complex behavioral data, deriving deep, actionable shopper insights that identify customer experience gaps and unlock new business opportunities. * Design and develop scalable machine learning and GenAI models focused on shopper intent and preference modeling, ensuring a rapid path from prototype to production. * Partner closely with engineering teams to architect and deploy end-to-end GenAI solutions into production, integrating advanced insights directly into real-time, customer-facing systems. * Drive the scalability, efficiency, and automation of large-scale model training and real-time inference systems, pioneering the LLM infrastructure required to support next-generation GenAI workloads at Amazon Ads scale. * Design and run rigorous A/B experiments to quantify the business and customer impact of GenAI-driven shopper insights, performing advanced statistical analysis to guide iterative production rollouts. * Conduct applied research in novel generative AI techniques (e.g., fine-tuning, RAG, agentic workflows) to optimize the shopper experience and drive performance across all aspects of the Sponsored Products and Brands business.
Applied Scientist, Agentic WorkSpaces (AAWS)
US, WA, Seattle
AWS Applied AI Solutions (AAIS) is where science meets customer obsession at scale. We build the intelligent systems that power AWS services used by millions, combining research in machine learning, agentic AI, and applied science with the operational rigor required to deliver enterprise grade experiences. Within AAIS, Amazon WorkSpaces is our cloud based virtual desktop service that delivers secure, managed computing to over one million daily users across the globe, enabling organizations to provision, manage, and scale desktops with the reliability and performance their workforce depends on. We are looking for an Applied Scientist to be part of the tiger team building the capacity modelling for Amazon WorkSpaces, owning and advancing the science behind it. You will design, build, and continuously improve the forecasting and optimization models that ensure the right compute, storage, and networking resources are available at the right time, in the right regions, at the lowest possible cost, without ever compromising the end user experience. This is a high impact individual contributor role for someone who thrives at the intersection of applied research and production systems. You will define the scientific roadmap for capacity intelligence, turning reactive provisioning into a predictive, self optimizing engine that anticipates demand before customers feel any constraint. Key job responsibilities Define and drive the scientific strategy for capacity modelling, establishing the research agenda that transforms how WorkSpaces forecasts demand, plans supply, and allocates resources across a globally distributed infrastructure. Build advanced demand forecasting models that predict workspace usage across multiple time horizons, from intraday spikes to long range growth trajectories, incorporating signals such as customer onboarding patterns, seasonal trends, regional expansion, and macroeconomic indicators. Design supply optimization frameworks that determine optimal resource placement, instance mix, and pre warming strategies, balancing availability, performance, and cost by reasoning over hardware constraints, pricing dynamics, and service level objectives. Develop causal and probabilistic models that move beyond trend extrapolation to true understanding of demand drivers, enabling the organization to distinguish organic growth from one time events, anticipate shifts in usage patterns, and quantify uncertainty in planning decisions. Architect simulation and scenario planning systems that allow business and engineering leaders to run what if analyses, stress test capacity plans against disruption scenarios, and evaluate trade offs between investment timing, risk tolerance, and customer experience. Pioneer the integration of machine learning with operations research, combining deep learning based forecasting with mathematical optimization to jointly solve the demand prediction and resource allocation problem in a way that neither discipline can achieve alone. Establish evaluation frameworks and monitoring systems that measure forecast accuracy, capacity utilization, and cost efficiency in production, creating tight feedback loops that drive continuous model improvement and build organizational trust in science driven planning. Influence the broader organization's capacity strategy by translating model outputs into actionable recommendations for leadership, identifying opportunities to extend capacity intelligence patterns to adjacent services, and mentoring scientists and engineers across the team. About the team As part of the AWS solutions organization, we have a vision to provide business applications, leveraging Amazon's unique experience and expertise, that are used by millions of companies worldwide to manage day-to-day operations. We will accomplish this by accelerating our customers' businesses through delivery of intuitive and differentiated technology solutions that solve enduring business challenges. we blend vision with curiosity and Amazon's real-world experience to build opinionated, turnkey solutions. Where customers prefer to buy over build, we become their trusted partner with solutions that are no-brainers to buy and easy to use. Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture AWS values curiosity and connection. Our employee-led and company-sponsored affinity groups promote inclusion and empower our people to take pride in what makes us unique. Our inclusion events foster stronger, more collaborative teams. Our continual innovation is fueled by the bold ideas, fresh perspectives, and passionate voices our teams bring to everything we do. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve.
Principal Applied Scientist - AI for Life Sciences, AWS Applied AI Solutions - Life Sciences
US, WA, Seattle
As part of the AWS Applied AI Solutions organization, we have a vision to provide end user applications, leveraging Amazon's unique experience and expertise, that are used by millions of companies worldwide to manage day-to-day operations. We will accomplish this by accelerating our customers' businesses through delivery of intuitive and differentiated technology solutions that solve enduring business challenges. We blend vision with curiosity and Amazon's real-world experience to build opinionated, turnkey solutions. Where customers prefer to buy over build, we become their trusted partner with solutions that are easy to adopt and easy to use. The Team Join the next science revolution at AWS Life Sciences Applied AI Solutions, where you'll work alongside world-class scientists to build AI that transforms how therapeutics are discovered, developed, and brought to patients. We're out to revolutionize how medicines are discovered, developed, and brought to patients, powered by a new generation of AI. Our team tackles some of the hardest open problems at the intersection of frontier AI and life sciences. We apply biological foundation models, large language models, and agentic reasoning systems to life sciences problems, then put them into the hands of pharma, biotech, and diagnostics customers as applications and managed services they can fine-tune, tailor, and deploy on their own data. The science challenges are deep: how do you design agentic systems that reason correctly over complex biological, regulatory, and clinical logic? How do you enable customers to tailor foundation models to their proprietary data and get better outputs with less effort? How do you adapt models to reason faithfully in high-stakes scientific and regulatory domains? Today we're focused on two areas. In drug design, our products (including Amazon Bio Discovery) accelerate discovery by giving bench scientists AI-guided protein engineering and antibody design capabilities. In clinical trials, we're building AI that automates and optimizes regulatory and clinical development workflows. We combine frontier research with production-scale delivery to put breakthrough science into the hands of customers solving humanity's hardest problems. We value scientific rigor, encourage publication, and support conference participation. If you want to do research that ships, this is the team. The Role We are seeking an exceptional Principal Applied Scientist to set the scientific direction for our life sciences AI portfolio. You will be the scientific leader who defines research agendas, architects novel approaches, and delivers models and methods that give our customers capabilities that did not previously exist. This is a rare role that combines deep expertise in LLM reasoning and agentic AI with applied impact in life sciences. You will innovate on how large language models reason, plan, and act in complex scientific domains, while applying domain knowledge in biology to ensure models produce scientifically valid outputs. The problems span multiple fronts: - How do you build LLM-based agentic systems that correctly reason over clinical protocols, regulatory standards, and complex multi-step scientific workflows? - How do you develop model customization and training methods that let customers get state-of-the-art results from foundation models? - How do you adapt and extend protein and antibody models so customers can fine-tune on proprietary sequence data and get therapeutically relevant outputs? You will work across drug discovery (protein engineering, antibody design) and clinical trial operations (agentic automation, structured reasoning, domain adaptation). You will own end-to-end scientific solutions from research through production, and your work will directly shape the tools that thousands of scientists use daily. Key job responsibilities - Set the scientific vision and research agenda for LLM reasoning, agentic AI, and biological model customization across the portfolio - Innovate on LLM reasoning, planning, and agentic approaches for complex scientific and regulatory workflows - Develop model customization methods (fine-tuning, RLHF, retrieval augmentation, domain adaptation) that enable customers to train better models on their own data with less effort - Advance methods to adapt and extend biological foundation models for customer-specific therapeutic applications - Solve open research problems in faithful reasoning, multi-step planning, and tool use in high-stakes scientific domains - Partner with Life Sciences domain experts and customers to understand their hardest scientific challenges and translate those into tractable research problems - Publish at top-tier venues and build the team's external scientific reputation - Mentor applied scientists across the team while maintaining significant personal research contribution - Collaborate with product and engineering to ensure research translates into shipped products that serve customers at scale - Influence multi-year research roadmaps through deep scientific expertise and customer understanding A day in the life - Push a new reasoning approach into production that measurably improves outputs for a pharma customer's workflow - Design and run experiments to validate a novel fine-tuning method, then ship it as a capability customers can use immediately - Unblock a delivery milestone by diagnosing why a model is failing on a new class of inputs and implementing a fix - Meet with a customer's scientific team to scope what the next model release needs to do for them - Review a teammate's experimental results, sharpen the approach, and help get it over the finish line - Publish results from shipped work at a top venue, closing the loop between research and impact - Prototype a new idea that could become the next major capability in the product
Principal Applied Scientist, WorkSpaces for AI Agents, Applied AI Solutions
US, WA, Seattle
We are seeking a Principal Applied Scientist to own the scientific vision across Agentic WorkSpaces. This is a foundational role spanning the full portfolio — Personal, Applications, and Core, and the agentic surfaces (WS4Builders and WorkSpaces for Agents). You will define how we measure, improve, and guarantee the performance of AI agents and human-AI teams. A core part of the role is defining the science agenda itself — identifying which problems are most worth solving and where the highest-leverage bets lie. Directions worth exploring might include Organizational Intelligence (turning institutional knowledge into agent-consumable skills), AI Agent Experience / AiAX (agent observability and autonomous remediation), and contextual, behavioral security that adapts enforcement in real time for human and agent sessions — but these are illustrative examples, not a fixed roadmap, and many other directions are possible. You will help define which ones we pursue. The problems you will solve do not have established industry patterns. You will set the direction for the science of how AI agents and people perceive, reason about, and act reliably within computing environments at enterprise scale. Key job responsibilities - Set the long-term scientific vision: Define what best-in-class agent performance, evaluation, and learning look like across Agentic WorkSpaces — for computer-using agents and human-AI teams alike. Identify the unsolved scientific problems, chart a multi-year research roadmap, and secure buy-in from VP-level leadership. - Solve highly ambiguous, novel problems: Independently frame and deliver solutions to foundational challenges in agent perception, reasoning, evaluation, reliability, and human-AI collaboration — problems where neither the approach nor the success criteria are pre-defined. - Own the evaluation and measurement foundation: Build the benchmarks, datasets, and metrics that quantify agent and team accuracy, cost, productivity, and safety across the portfolio and diverse enterprise workflows, and that gate what we ship. - Drive cross-organizational scientific alignment: Work across partner teams (AgentCore, Bedrock model teams, Identity, Security, the MCP ecosystem) and across the Applied AI Solutions product portfolio to shape how models and agent frameworks are applied, and ensure scientific decisions compose into a coherent system. - Deliver measurable business impact: Ensure research translates to customer outcomes: higher task accuracy, lower cost-per-action, faster time-to-production, measurable productivity for human-AI teams, and the trust that lets enterprises scale agent workflows. - Raise the scientific bar: Establish rigor in experimentation, evaluation, and reproducibility. Mentor and grow senior scientists and engineers. Set the standard for applied science quality across the organization. - Advance the state of the art: Contribute to the external technical community through publications, patents, and open-source contributions that position AWS as the leader in the science of secure agent-computer interaction and human-AI teamwork. About the team AWS Applied AI Solutions' (AAIS) vision is every business innovating with Amazon AI teammates. Our mission is to build delightful AI solutions that improve human capabilities and business outcomes. The Agentic WorkSpaces organization within AAIS envisions a world where people, teams, and AI collaborate securely from anywhere to create unprecedented value for every organization. We build lovable products that empower every business to unlock the full potential of human-AI teamwork, driving smarter decisions, greater creativity, more value, and faster innovation with confidence. Amazon Agentic WorkSpaces (AAWS) is building the world's most lovable, secure, and trusted always-on workspace where AI agents and humans work as partners behind enterprise-grade security. Our portfolio spans persistent desktops (Personal), application streaming (Applications), and Core, and is evolving into the governed operating environment for the hybrid workforce: humans get AI-native desktops for their role, and agents get governed desktops scoped to their task, with administrators managing both as one. This surface includes WS4Builders (an AI-native environment for builders) and WorkSpaces for Agents (W4A) — enabling AI agents to work the way humans do, with access to real applications, real interfaces, and real computing environments. Enterprises want to use AI agents for critical business workloads that touch legacy desktop applications and mainframes, yet 75% of organizations run legacy applications that lack modern APIs, and 90% of corporate data remains locked in systems never designed for agents. Agentic WorkSpaces solves this: it gives enterprises a secure, governed environment where agents and humans operate both legacy and modern applications directly, just as an employee would, without costly migrations.
Senior Manager, Applied Science
US, WA, Seattle
We are seeking a Senior Manager, Applied Science to build and lead the science organization across Agentic WorkSpaces. This is a foundational leadership role spanning the full portfolio — Personal, Applications, and Core, and the agentic surfaces (WS4Builders and WorkSpaces for Agents). You will hire, grow, and lead a team of applied scientists who define how we measure and improve the performance of AI agents and human-AI teams. A core part of the role is defining the science agenda itself — identifying which problems are most worth solving and where the highest-leverage bets lie. Directions worth exploring might include Organizational Intelligence (turning institutional knowledge into agent-consumable skills), AI Agent Experience / AiAX (agent observability and autonomous remediation), and contextual, behavioral security that adapts enforcement in real time for human and agent sessions — but these are illustrative examples, not a fixed roadmap, and many other directions are possible. You and your team will define which ones we pursue. The problems your team will solve do not have established industry patterns. You will set the scientific direction and build the team that determines how AI agents and people perceive, reason about, and act reliably within computing environments at enterprise scale. What You Will Do Build and lead the applied science team. Hire, develop, and retain a high-caliber team of applied scientists spanning the Agentic WorkSpaces portfolio. Set the bar for scientific talent, create the growth paths, and build the culture that makes AAWS a destination for the best agent and human-AI researchers. Own the science strategy across the portfolio. Direct the research agenda for how we measure and improve agents and human-AI teams: the benchmarks, task suites, and metrics (accuracy, cost-per-task, task completion, productivity) that turn subjective "it works" judgments into rigorous, reproducible measurement that gates what we ship. Define and drive high-leverage research directions. Work with your team to identify the problems most worth solving and shape the science agenda. Directions worth exploring might include how agents combine deterministic tool use (MCP) with visual reasoning from computer use; Organizational Intelligence and workflow learning (learning from expert recordings, voice annotations, and SOPs); and AI Agent Experience / AiAX (detecting when agents are stuck or degrading productivity and autonomously remediating) — these are illustrative starting points, and your team will weigh them against many other possibilities. Translate science into shipped product. Partner with engineering, product, and program leaders to move models, evaluation, and learning systems from prototype into a decade-old production service operating at massive scale, without compromising the reliability that customers depend on. Represent science in leadership and to customers. Be the scientific voice in org-level planning and roadmap decisions across AAWS, and engage directly with enterprise customers on how agent performance, safety, and human-AI productivity are measured and earned. Key job responsibilities Build and lead the applied science team. Hire, develop, and retain a high-caliber team of applied scientists spanning the Agentic WorkSpaces portfolio. Set the bar for scientific talent, create the growth paths, and build the culture that makes AAWS a destination for the best agent and human-AI researchers. Own the science strategy across the portfolio. Direct the research agenda for how we measure and improve agents and human-AI teams: the benchmarks, task suites, and metrics (accuracy, cost-per-task, task completion, productivity) that turn subjective "it works" judgments into rigorous, reproducible measurement that gates what we ship. Define and drive high-leverage research directions. Work with your team to identify the problems most worth solving and shape the science agenda. Directions worth exploring might include how agents combine deterministic tool use (MCP) with visual reasoning from computer use; Organizational Intelligence and workflow learning (learning from expert recordings, voice annotations, and SOPs); and AI Agent Experience / AiAX (detecting when agents are stuck or degrading productivity and autonomously remediating) — these are illustrative starting points, and your team will weigh them against many other possibilities. Translate science into shipped product. Partner with engineering, product, and program leaders to move models, evaluation, and learning systems from prototype into a decade-old production service operating at massive scale, without compromising the reliability that customers depend on. Represent science in leadership and to customers. Be the scientific voice in org-level planning and roadmap decisions across AAWS, and engage directly with enterprise customers on how agent performance, safety, and human-AI productivity are measured and earned. Set the long-term scientific vision and team strategy: Define what best-in-class agent performance, evaluation, and learning look like across Agentic WorkSpaces — for computer-using agents and human-AI teams alike. Chart a multi-year research roadmap, and build the team and plan to deliver it. Secure buy-in from VP-level leadership. Hire and grow scientific talent: Own recruiting, calibration, development, and retention for the science team. Mentor scientists toward senior and principal scope, and raise the scientific bar across the organization. Direct research on highly ambiguous, novel problems: Guide the team through foundational challenges in agent perception, reasoning, evaluation, reliability, and human-AI collaboration — problems where neither the approach nor the success criteria are pre-defined. Drive cross-organizational alignment: Work across partner teams (AgentCore, Bedrock model teams, Identity, Security, the MCP ecosystem) and across the Applied AI Solutions product portfolio, with product and engineering leadership, to ensure scientific decisions compose into a coherent product. Deliver measurable business impact: Ensure your team's research translates to customer outcomes: higher task accuracy, lower cost-per-action, faster time-to-production, measurable productivity for human-AI teams, and the trust that lets enterprises scale agent workflows. Establish scientific rigor and operational excellence: Set the standard for experimentation, evaluation, and reproducibility, and the mechanisms that keep the science organization productive and accountable. Advance the state of the art: Enable and champion contributions to the external technical community through publications, patents, and open-source work that position AWS as the leader in the science of secure agent-computer interaction and human-AI teamwork. About the team AWS Applied AI Solutions' (AAIS) vision is every business innovating with Amazon AI teammates. Our mission is to build delightful AI solutions that improve human capabilities and business outcomes. The Agentic WorkSpaces organization within AAIS envisions a world where people, teams, and AI collaborate securely from anywhere to create unprecedented value for every organization. We build lovable products that empower every business to unlock the full potential of human-AI teamwork, driving smarter decisions, greater creativity, more value, and faster innovation with confidence. Amazon Agentic WorkSpaces (AAWS) is building the world's most lovable, secure, and trusted always-on workspace where AI agents and humans work as partners behind enterprise-grade security. Our portfolio spans persistent desktops (Personal), application streaming (Applications), and Core, and is evolving into the governed operating environment for the hybrid workforce: humans get AI-native desktops for their role, and agents get governed desktops scoped to their task, with administrators managing both as one. This surface includes WS4Builders (an AI-native environment for builders) and WorkSpaces for Agents (W4A) — enabling AI agents to work the way humans do, with access to real applications, real interfaces, and real computing environments. Enterprises want to use AI agents for critical business workloads that touch legacy desktop applications and mainframes, yet 75% of organizations run legacy applications that lack modern APIs, and 90% of corporate data remains locked in systems never designed for agents. Agentic WorkSpaces solves this: it gives enterprises a secure, governed environment where agents and humans operate both legacy and modern applications directly, just as an employee would, without costly migrations.
Applied Scientist II, Amazon Shipping
IN, HR, Gurugram
Work on ML teams building large-scale forecasting and optimization systems that power Amazon’s global transportation network and directly impact customer experience and cost. As an Applied Scientist II, you will set scientific direction, mentor applied scientists, and partner with engineering and product leaders to deliver production-grade ML solutions at massive scale. Key job responsibilities 1. Lead and grow a high-performing team of Applied Scientists, providing technical guidance, mentorship, and career development. 2. Define and own the scientific vision and roadmap for ML solutions powering large-scale transportation planning and execution. 3. Guide model and system design across a range of techniques, including tree-based models, deep learning (LSTMs, transformers), LLMs, and reinforcement learning. 4. Ensure models are production-ready, scalable, and robust through close partnership with stakeholders. Partner with Product, Operations, and Engineering leaders to enable proactive decision-making and corrective actions. 5. Own end-to-end business metrics, directly influencing customer experience, cost optimization, and network reliability. 6. Help contribute to the broader ML community through publications, conference submissions, and internal knowledge sharing. A day in the life Your day includes reviewing model performance and business metrics, guiding technical design and experimentation, mentoring scientists, and driving roadmap execution. You’ll balance near-term delivery with long-term innovation while ensuring solutions are robust, interpretable, and scalable. Ultimately, your work helps improve delivery reliability, reduce costs, and enhance the customer experience at massive scale.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social