Social
Conversational AI

A quick guide to Amazon’s 50-plus papers at EMNLP 2024

Large language models predominate, both as a research subject themselves and as tools for researching topics of particular interest to Amazon, such as speech, recommendations, and information retrieval.

By Staff writer
November 14, 2024
7 min read
Share

Large language models (LLMs) have come to dominate the field of natural-language processing, so it’s no surprise that they also dominate the research that Amazon scientists are presenting at this year’s Conference on Empirical Methods in Natural-Language Processing (EMNLP). LLM training is the topic with the greatest number of Amazon papers, followed closely by strategies for mitigating misinformation in LLMs’ outputs — including but not limited to hallucinations. At the same time, a number of papers apply LLMs to topics of traditional interest at Amazon, such as speech, recommender systems, and information retrieval. (Papers marked with asterisks were accepted to Findings of EMNLP.)

AI agents

MARCO: Multi-agent real-time chat orchestration
Anubhav Shrimal, Shervin Malmasi, Kriti Biswas, Swarnalatha Raghuraman, Anish Nediyanchath, Yi Zhang, Promod Yenigalla

Code generation

CodeFort: Robust training for code generation models
Yuhao Zhang, Shiqi Wang, Haifeng Qian, Zijian Wang, Mingyue Shang, Linbo Liu, Sanjay Krishna Gouda, Baishakhi Ray, Murali Krishna Ramanathan, Xiaofei Ma, Anoop Deoras

Socratic human feedback (SoHF): Expert steering strategies for LLM code generation
Subramanian Chidambaram, Erran Li, Min Bai, Xiaopeng LI, Kaixiang Lin, Xiong Zhou, Alex C. Williams

Structured object language modeling (SoLM): Native structured objects generation conforming to complex schemas with self-supervised denoising
Amir Tavanaei, Kee Kiat Koo, Hayreddin Ceker, Shaobai Jiang, Qi Li, Julien Han, Karim Bouyarmane

Contrastive decoding

Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM
Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal, Anil Ramakrishna, Tagyoung Chung

Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM.png
Given a simple question with clues, contrastive decoding could have an “obvious blindness” (e.g., assigning higher probability to an uncommon answer, such as "invertebrate", than to the most obvious answer, "bees"). In contrast, the asymptotic probability decoding proposed in "Explaining and improving contrastive decoding by extrapolating the probabilities of a huge and hypothetical LM" correctly assigns the highest probability to "bees" by leveraging the probabilities from multiple LMs of different sizes.

Data integration

ASTRA: Automatic schema matching using machine translation
Tarang Chugh, Deepak Zambre

Learning from natural language explanations for generalizable entity matching
Somin Wadhwa, Adit Krishnan, Runhui Wang, Byron C. Wallace, Chris (Luyang) Kong

Pretraining and finetuning language models on geospatial networks for accurate address matching
Saket Maheshwary, Arpan Paul, Saurabh Sohoney

Retrieval augmented spelling correction for e-commerce applications
Xuan Guo, Rohit Patki, Dante Everaert, Christopher Potts

Dataset distillation

Textual dataset distillation via language model embedding
Yefan Tao, Chris (Luyang) Kong, Andrey Kan, Laurent Callot

Textual dataset distillation via language model embedding: DaLLME.png
The DaLLME framework proposed in "Textual dataset distillation via language model embedding" begins by using a language model to transform raw textual data into embedding vectors. A set of distilled vectors is then derived in the embedding space, through a process designed to encapsulate maximum informational content. Finally, the vec2text model translates these distilled vectors back into textual form.

Document understanding

DocKD: Knowledge distillation from LLMs for open-world document understanding models
Sungnyun Kim, Haofu Liao, Srikar Appalaraju, Peng Tang, Zhuowen Tu, Ravi Kumar Satzoda, R. Manmatha, Vijay Mahadevan, Stefano Soatto

Information retrieval

Evaluating D-MERIT of partial-annotation on information retrieval
Royi Rassin, Yaron Fairstein, Oren Kalinsky, Guy Kushilevitz, Nachshon Cohen, Alexander Libov, Yoav Goldberg

Identifying high consideration e-commerce search queries
Zhiyu Chen, Jason Choi, Besnik Fetahu, Shervin Malmasi

Learning when to retrieve, what to rewrite, and how to respond in conversational QA*
Nirmal Roy, Leonardo Ribeiro, Rexhina Blloshmi, Kevin Small

Natural-language understanding

Intent detection in the age of LLMs
Gaurav Arora, Shreya Jain, Srujana Merugu

Intent detection in the age of LLMs.png
"Intent detection in the age of LLMs" proposes a methodology for adaptive in-context learning and chain-of-thought-based intent detection using LLMs.

Predicting entity salience in extremely short documents
Ben Bullough, Harrison Lundberg, Chen Hu, Weihang Xiao

LLM evaluation

AXCEL: Automated eXplainable consistency evaluation using LLMs*
P Aditya Sreekar, Sahil Verma, Suransh Chopra, Sarik Ghazarian, Abhishek Persad, Narayanan Sadagopan

Precise model benchmarking with only a few observations
Riccardo Fogliato, Pratik Patil, Nil-Jana Akpinar, Mathew Monfort

LLM fine tuning

AdaZeta: Adaptive zeroth-order tensor-train adaption for memory-efficient large language models fine-tuning
Yifan Yang, Kai Zhen, Ershad Banijamali, Thanasis Mouchtaris, Zheng Zhang

RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning
Haoyu Wang, Tianci Liu, Ruirui Li, Monica Cheng, Tuo Zhao, Jing Gao

RoseLoRA.png
The row- and column-wise sparse low-rank adaptation (RoseLoRA) framework proposed in "RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning".

LLMs for speech

Speechworthy instruction-tuned language models
Hyundong Cho, Nicolaas Jedema, Leonardo Ribeiro, Karishma Sharma, Pedro Szekely, Alessandro Moschitti, Ruben Janssen, Jonathan May

LLM misinformation mitigation

ECON: On the detection and resolution of evidence conflicts
Cheng Jiayang, Chunkit Chan, Qianqian Zhuang, Lin Qiu, Tianhang Zhang, Tengxiao Liu, Yangqiu Song, Yue Zhang, Pengfei Liu, Zheng Zhang

Generative subgraph retrieval for knowledge graph–grounded dialog generation
Jinyoung Park, Minseok Joo, Joo-Kyung Kim, Hyunwoo J. Kim

HalluMeasure: Fine-grained hallucination measurement using chain-of-thought reasoning
Shayan Ali Akbar, Md Mosharaf Hossain, Tess Wood, Si-Chi Chin, Erica Salinas, Victor Alvarez, Erwin Cornejo

Knowledge-centric hallucination detection
Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Zheng Zhang, Yue Zhang

LLM reasoning

Auto-evolve: Enhancing large language model’s performance via self-reasoning framework*
Krishna Aswani, Alex Lu, Pranav Patankar, Priya Dhalwani, Iris Tan, Jayant Ganeshmohan, Simon Lacasse

LLM self-correction

LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints
Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, Nanyun Peng

DeCRIM.png
In the DeCRIM pipeline proposed in "LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints", an LLM first generates a response to a user request. The Decomposer then breaks down the request into granular constraints, and the Critic model gives feedback on whether the response meets those constraints. If it does, the response is output; if not, the LLM uses the feedback to refine the response.

LLM training

Dancing in chains: Reconciling instruction following and faithfulness in language models
Zhengxuan Wu, Yuhao Zhang, Peng Qi, Yumo Xu, Rujun Han, Yian Zhang, Jifan Chen, Bonan Min, Zhiheng Huang

DEM: Distribution edited model for training with mixed data distributions
Dhananjay Ram, Aditya Rawal, Momchil Hardalov, Nikolaos Pappas, Sheng Zha

DEM: Distribution Edited Model for Training with Mixed Data Distributions
The distribution-edited model (ΘD) described in "DEM: Distribution edited model for training with mixed data distributions" results from fine-tuning a pretrained model (Θ) on n individual data distributions (Di) and combining the resulting models with basic element-wise vector operations. Here, the extracted distribution vectors (∆ΘDi ) are multiplied by weight coefficients, and the weighted sum is added to the base model.

Evolutionary contrastive distillation for language model alignment
Julian Katz-Samuels, Zheng Li, Hyokun Yun, Priyanka Nigam, Yi Xu, Vaclav Petricek, Bing Yin, Trishul Chilimbi

Hop, skip, jump to convergence: Dynamics of learning rate transitions for improved training of large language models
Shreyas Subramanian, Vignesh Ganapathiraman, Corey Barrett

Learning from relevant subgoals in successful dialogs using iterative training for task-oriented dialog systems
Magdalena Kaiser, Patrick Ernst, Gyuri Szarvas

Quality matters: Evaluating synthetic data for tool-using LLMs
Shadi Iskander, Nachshon Cohen, Zohar Karnin, Ori Shapira, Sofia Tolmach

Query autocompletion

AmazonQAC: A large-scale, naturalistic query autocomplete dataset
Dante Everaert, Rohit Patki, Tianqi Zheng, Christopher Potts

DiAL: Diversity aware listwise ranking for query auto-complete
Sonali Singh, Sachin Farfade, Prakash Mandayam Comar

Question answering

RAG-QA arena: Evaluating domain robustness for long-form retrieval-augmented question answering
Rujun Han, Yuhao Zhang, Peng Qi, Yumo Xu, Jenyuan Wang, Lan Liu, William Yang Wang, Bonan Min, Vittorio Castelli

Retrieving contextual information for long-form question answering using weak supervision
Philipp Christmann, Svitlana Vakulenko, Ionut Teodor Sorodoc, Bill Byrne, Adrià de Gispert

Recommender systems

Efficient pointwise-pairwise learning-to-rank for news recommendation
Nithish Kannen Senthilkumar, Yao Ma, Gerrit van den Burg, Jean Baptiste Faddoul

Efficient pointwise-pairwise learning-to-rank for news recommendation.png
An illustration of the GLIMPSE framework proposed in "Efficient pointwise-pairwise learning-to-rank for news recommendation". GLIMPSE adopts a multitask approach in which a pretrained language model is fine-tuned on both the relevance prediction task and the pairwise-preference task. During inference, the relevance predictions are used to produce an initial pointwise ranking, which is subsequently improved by one or more right-to-left (RTL) passes using pairwise comparisons.

PEARL: Preference extraction with exemplar augmentation and retrieval with LLM agents
Vijit Malik, Akshay Jagatap, Vinayak Puranik, Anirban Majumder

Sequential LLM framework for fashion recommendation
Han Liu, Xianfeng Tang, Tianlang Chen, Jiapeng Liu, Indu Indu, Henry Peng Zou, Peng Dai, Roberto Fernandez Galan, Mike Porter, Dongmei Jia, Ning Zhang, Lian Xiong

Responsible AI

Attribute controlled fine-tuning for large language models: A case study on detoxification
Tao Meng, Ninareh Mehrabi, Palash Goyal, Anil Ramakrishna, Aram Galstyan, Richard Zemel, Kai-Wei Chang, Rahul Gupta, Charith Peris

FLIRT: Feedback loop in-context red teaming
Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta

Order of magnitude speedups for LLM membership inference
Rongting Zhang, Martin Bertran Lopez, Aaron Roth

Synthetic data generation

CorrSynth: A correlated sampling method for diverse dataset generation from LLMs
Suhas Kowshik, Abhishek Divekar, Vijit Malik

A Correlated Sampling Method for Diverse Dataset Generation from LLMs
"CorrSynth: A correlated sampling method for diverse dataset generation from LLMs" introduces a sampling method that uses anti-correlation between examples rather than few-shot generation.

DATA ADVISOR: Dynamic data curation for safety alignment of large language models
Fei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta, Kai-Wei Chang, Aram Galstyan

Evaluating differentially private synthetic data generation in high-stakes domains
Krithika Ramesh, Nupoor Gandhi, Pulkit Madaan, Lisa Bauer, Charith Peris, Anjalie Field

SYNTHESIZRR: Generating diverse datasets with retrieval augmentation
Abhishek Divekar, Greg Durrett

Abstract depiction of the SYNTHESIZRR procedure
Abstract depiction of the procedure proposed in "SYNTHESIZRR: Generating diverse datasets with retrieval augmentation". The content sourcing stage retrieves K unique documents {r1,...,rK} from a large corpus for each in-context covariate xICL. The task-inversion stage uses a parameterized context refinement prompt, Pτ, which takes parameters Rinv (inversion instruction), rk (a retrieved document), and V(yICL) (the verbalized target label). A generalist teacher LLM autoregressively generates a synthetic covariate. Each in-context example thus produces K unique synthetic examples {x̃1,..., x̃K}, which we include in the dataset with target yICL.

Text classification

Distance-aware calibration for pre-trained language models*
Alberto Gasparin, Gianluca Detommaso

Performance-guided LLM knowledge distillation for efficient text classification at scale

Flavio Di Palo, Prateek Singhi, Bilal Fadlallah

Prompt-tuned muti-task taxonomic transformer (PTMTTaxoFormer)
Rajashekar Vasantha, Nhan Nguyen, Yue Zhang

Text summarization

Salient information prompting to steer content in prompt-based abstractive summarization
Lei Xu, Asad Karim, Saket Dingliwal, Aparna Elangovan

Research areas
Tags
About the Author
Staff writer

Related content

Work with us

See more jobs See more jobs
Data Scientist , Leo Customer Terminal
US, WA, Redmond
At Amazon, we’re inventing on behalf of customers, and with Amazon Leo, we’re redefining what global connectivity looks like. Our mission is to deliver fast, affordable broadband to unserved and underserved communities around the world through a constellation of low Earth orbit (LEO) satellites. Every system we build helps connect people to education, healthcare, opportunity, and each other. As a Data Scientist, you will be responsible for developing advanced analytics and machine learning solutions for user terminals. You will develop predictive models to proactively identify possible user terminal failures in the field. You will work in a collaborative environment with a multi-disciplinary team, including constellation, RF, antenna, silicon, algorithm, and software engineers. Key job responsibilities As a Data Scientist, you will develop analytic tools for a team developing current and future user terminals. Your responsibilities include: • Develop statistical and analytical tool to enable the regression decision from on-orbit and lab measurement of user terminals • Publish documents and create compelling visualizations and presentations to communicate insights to stakeholders • Create and manage datasets for continued pre-training and supervised fine-tuning of LLMs • Develop scalable visualizations for analysis of user terminal performance • Work closely with constellation, RF, antenna, silicon, algorithm, and software engineers to root-cause the failures using data as the primary tool • Drive consensus on metrics and analysis approaches to support product development strategy Export Control Requirement: Due to applicable export control laws and regulations, candidates must be a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum. A day in the life As a Data Scientist in the LEO Customer Terminal Team, you will work daily with satellite constellation, algorithm, RF, antenna, silicon, hardware, and software teams in a collaborative environment. Your focus will be using data as an intelligence source to enable design decisions for the team. About the team The LEO Customer Terminal team is responsible for developing both outdoor and indoor devices that enable customers to access internet service via the LEO satellite network. We own the entire process from early prototypes through mass production, including requirements documentation, architecture definition, hardware development, algorithm development, and all integration and verification testing.
Senior Applied Scientist, Fauna
US, NY, New York
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers
Satellite GNC Engineer, Amazon Leo
US, WA, Redmond
Amazon Leo is building a constellation of thousands of low Earth orbit satellites to deliver fast, reliable internet beyond the reach of existing networks. As a Guidance, Navigation & Control engineer, you will design the algorithms, simulations, and onboard autonomy that keep every satellite precisely pointed and stable so the payload can serve customers below, and keep the fleet flying safely. This is satellite GNC with demanding pointing and performance requirements, and you will see your work go from concept to orbit on a constellation that is launching and growing today. Amazon Leo will delight customers with reliable connectivity regardless of where they are. In this role, you will: - Take your GNC algorithms from concept all the way to orbit, and stay close to them through integration, test, and on-orbit operations - Solve hard precise-pointing, payload-cueing, and fleet-autonomy problems against demanding performance requirements - Work across the full lifecycle — design, simulation, test, and flight operations — rather than a narrow slice - Join a small team of some of the top GNC engineers in the industry, building a product people will love to use This position is part of the Satellite Attitude Determination and Control team, where we own the full lifecycle of our work — from algorithm design and flight software, through lab integration, to flying our own satellites on orbit. You will design and analyze the control system and algorithms, support development of our flight hardware and software, help integrate the satellite in our labs, and operate the GNC system in flight as a constellation of satellites flows through the production line into orbit. **Export Control Requirement:** Due to applicable export control laws and regulations, candidates must be a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum. Key job responsibilities - Design and analyze algorithms for state estimation, attitude determination and control, precise pointing, and payload cueing against demanding performance requirements. - Write the production C++ flight code that runs these algorithms on the satellite. Our GNC engineers own their algorithms all the way into flight software, working alongside software development engineers rather than handing prototypes off to be reimplemented by another team. - Design concepts of operation that balance pointing and payload cueing, momentum management, power, thermal, and communications across all phases of the mission, including off-nominal and contingency scenarios. - Develop momentum management strategies using reaction wheels and magnetic torquers, and help size and lay out the GNC sensor and actuator suite — star trackers, magnetometers, sun sensors, IMUs, and GPS receivers — and the algorithms that consume them. - Develop onboard autonomy, including Fault Detection, Isolation, and Recovery, so the fleet operates safely at scale with minimal intervention. - Develop and validate models and simulations of the satellite and constellation across a range of fidelity levels, including 6-DOF time-domain simulation and Monte Carlo analysis, and run hardware-in-the-loop testing to verify closed-loop performance. - Support component-level environmental testing, functional and performance checkout, and subsystem and satellite integration in our labs. - Analyze fleet telemetry in bulk and build the metrics, dashboards, and tooling that let us monitor and continuously improve pointing performance and payload operations time as the constellation grows. - Operate the GNC system on orbit — reviewing fleet data, supporting maneuvers, and resolving anomalies when on-orbit behavior diverges from the ground model. - Contribute to architecture, design, and code reviews, and document your algorithms, analyses, and results clearly. A day in the life This role spans a diverse set of problems across the spacecraft and network system. You will derive and prototype estimation and control algorithms, turn them into production flight code, and prove them out through 6-DOF and Monte Carlo simulation and hardware-in-the-loop testing in the lab. You will support satellite integration, and when something on orbit does not behave the way the ground model predicted, you will dig into fleet telemetry to find out why and drive the fix. You will own problems end to end and have the focus to go deep on each one. Because GNC sits at the center of safe flight, you will work across the whole program. The payload, power, thermal, and operations teams all depend on GNC, so you will balance their needs against your pointing and momentum budgets. To get your algorithms running reliably on the spacecraft, you will partner closely with avionics, flight software, and our embedded software development engineers. You will help the mechanical teams shape the next generation of satellite hardware, from reaction wheels to star trackers. And as satellites move from design through manufacturing into orbit, you will work alongside systems engineering and Assembly, Integration & Test. The problems are genuinely hard — demanding pointing and performance requirements, a large constellation to keep flying safely, and the chance to invent advanced capabilities that have not been built before. The work is staffed with some of the strongest engineers in the industry, and the constant collaboration across teams makes it a place to keep learning and growing. About the team Our team brings deep experience across many satellite systems and other flight vehicles, with real bench strength in both our mission and the core GNC disciplines. We design, prototype, test, iterate, and learn together, and because GNC is central to safe flight, we tend to drive Concepts of Operation and many of the system-level analyses on the program. We also have a lot of freedom. We trust our engineers to pursue the problems they face in whatever way they find best, to choose their own tools and approaches, and to own the results. If you want the autonomy to solve hard problems your way, and a team of strong engineers to do it with, you will feel at home here.
Applied Scientist, Fauna
US, NY, New York
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers
Applied Scientist, Advertiser Growth Engine
US, NY, New York
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses, responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products are strategically important to our businesses driving long term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day! Advertiser Growth Engine (AGE) team owns and builds services and applications across Amazon World-Wide Advertising that make advertising across multi-marketplaces as easy as flipping on a switch. We are focused on: (1) expanding Amazon Ads advertiser base, and (2) eliminating localization, operational, and marketplace knowledge gap barrier for Advertisers who advertise across multiple marketplaces. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth globally. We're looking for an experienced Applied Scientist with exceptional technical, analytical, and innovative capabilities to research, design, and create elegant machine learning solutions. The solutions will help our advertisers with multi-media and multi-lingual advertising offerings. You will use ideas from various domains of machine learning, including supervised and unsupervised methods, Deep Neural Networks, Natural Language Processing (NLP), and Computer Vision (CV) to build ML models that localizes multi-media advertising contents, including text, images and videos. You will also identify opportunities to leverage ML beyond localization, including, international expansion and global campaigns. Your work will directly impact our customers in the form of products and services used directly by our advertisers as well as our third-party integrators. As an Applied Scientist on this team, you will: - Build and deliver end-to-end machine learning solutions; build ML models and perform data analysis to deliver scalable solutions to business problems. - Perform hands-on analysis and modeling with enormous data sets to develop insights that increase traffic monetization and merchandise sales without compromising shopper experience. - Work closely with software engineers on detailed requirements to productionize the ML models you build. - Run A/B experiments that affect hundreds of millions of customers, evaluate the impact of your optimizations and communicate your results to various business stakeholders. - Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving. - Research new innovate machine learning approaches. Why you will love this opportunity: Amazon is investing heavily in building a world-class advertising business. This team defines and delivers a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are a highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate. Impact and Career Growth: You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding. Team video https://youtu.be/zD_6Lzw8raE
Applied Science Manager, Personalization
IL, Tel Aviv
Are you a scientist interested in pushing the state of the art in Information Retrieval, Large Language Models and Recommendation Systems? Are you interested in innovating on behalf of millions of customers, helping them accomplish their every day goals? Do you wish you had access to large datasets and tremendous computational resources? Do you want to join a team of capable scientist and engineers, building the future of e-commerce? Answer yes to any of these questions, and you will be a great fit for our team at Amazon. Our team is part of Amazon’s Personalization organization, a high-performing group that leverages Amazon’s expertise in machine learning, generative AI, large-scale data systems, and user experience design to deliver the best shopping experiences for our customers. Our team is building next-generation personalization systems powered by Large Language Models. We are tackling novel research challenges to help customers discover products they'll love - at Amazon scale and latency requirements. We are a team uniquely placed within Amazon, to have a direct window of opportunity to influence how customers will think about their shopping journey in the future. As an Applied Science Manager, you will lead a team of scientists working at the frontier of LLM-based personalization. You will set the technical vision, drive the research agenda, and ensure your team delivers production-ready solutions. You will hire, mentor, and develop world-class scientists while fostering a culture of innovation and scientific rigor. You will partner closely with engineering and product teams to translate ambitious research into customer-facing impact, and represent your team's work to senior leadership. Please visit https://www.amazon.science for more information.
Sr. Applied Scientist, AWS Applied AI Solutions - Life Sciences
US, WA, Seattle
AWS Applied AI Solutions (AAIS) is building toward a future where every business innovates with Amazon AI teammates. To get there, we build AI solutions that improve human capabilities and transform entire business functions. We create end-to-end products that surprise and delight out-of-the-box, making complex things easy and hard things possible, with no cloud experience required. We start with customers who embrace the future and build bridges to meet the rest where they are. We pursue ambitious opportunities with conviction, and we are looking for builders who share that mindset. The Team Join the next science revolution at AWS Life Sciences Applied AI Solutions where you'll work alongside world-class scientists to build AI that transforms how therapeutics are discovered, developed, and brought to patients. We're out to revolutionize how medicines are discovered, developed, and brought to patients powered by a new generation of AI. Our team tackles some of the hardest open problems at the intersection of frontier AI and life sciences. We apply biological foundation models large language models and agentic reasoning systems to life sciences problems then put them into the hands of customers as applications and managed services they can fine-tune tailor and deploy on their own data. The science challenges are deep: how do you design agentic systems that reason correctly over complex biological regulatory and clinical logic? How do you enable customers to tailor foundation models to their proprietary data and get better outputs with less effort? How do you adapt models to reason faithfully in high-stakes scientific and regulatory domains? Today we're focused on two areas. In clinical trials we're building AI that automates and optimizes regulatory and clinical development workflows. In drug design our products (including Amazon Bio Discovery) accelerate discovery by giving bench scientists AI-guided protein engineering and antibody design capabilities. We combine frontier research with production-scale delivery to put breakthrough science into the hands of customers solving humanity's hardest problems. We value scientific rigor encourage publication and support conference participation. If you want to do research that ships this is the team. The Role We are seeking an Applied Scientist to build the models and methods behind our life sciences AI products with a primary focus on clinical trial operations and agentic reasoning. You will design train and evaluate systems that reason over complex clinical and operational logic and ship them into products customers use directly. You will work closely with senior and principal scientists on well-scoped research problems own your results end to end and see your work reach production. This role combines expertise in LLM reasoning and agentic AI with applied impact in life sciences. You will work on how large language models reason plan and act in complex scientific domains while applying domain knowledge to ensure models produce scientifically valid outputs. The problems span multiple fronts: • How do you build LLM-based agentic systems that correctly reason over clinical protocols regulatory standards and complex multi-step operational workflows? • How do you evaluate agent reliability and faithfulness rigorously enough to trust in high-stakes clinical settings? • How do you develop model customization methods (fine-tuning retrieval augmentation domain adaptation) that let customers get strong results from foundation models on their own data? You will focus on clinical trial operations (agentic automation structured reasoning evaluation domain adaptation) with opportunities to contribute across drug discovery (protein engineering antibody design) as the portfolio grows. You will own end-to-end scientific solutions from research through production and your work will directly shape the tools that scientists use daily. Key job responsibilities • Design train fine-tune and evaluate LLM-based agentic systems that reason over clinical protocols regulatory standards and operational workflows • Build rigorous evaluation harnesses and benchmarks to measure agent reliability faithfulness and failure modes in high-stakes domains • Develop model customization methods (fine-tuning RLHF retrieval augmentation domain adaptation) that help customers get better outputs on their own data with less effort • Contribute to graph-based and causal modeling approaches for clinical trial operations • Partner with Life Sciences domain experts product and engineering to translate scientific challenges into shipped capabilities • Own experiments end to end: problem framing implementation evaluation iteration and hand-off to production • Publish at top-tier venues where the work supports it • Contribute to drug discovery efforts (protein engineering antibody design) as opportunities arise A day in the life • Design and run an experiment to validate a new agentic reasoning or fine-tuning method then ship it as a capability customers can use • Diagnose why a model is failing on a new class of inputs and implement a fix to unblock a delivery milestone • Build or extend an evaluation benchmark to measure how faithfully an agent reasons over clinical logic • Meet with domain experts to scope what the next model release needs to do • Review results with a senior scientist sharpen the approach and get it over the finish line • Prototype a new idea that could become the next capability in the product About the team Amazon values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn't followed a traditional path, or includes alternative experiences, don't let it stop you from applying. Amazon Web Services (AWS) is the world's most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that's why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there's nothing we can't achieve in the cloud. Here at AWS, it's in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity and AmazeCon conferences, inspire us to never stop embracing our uniqueness. We're continuously raising our performance bar as we strive to become Earth's Best Employer. That's why you'll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Data Scientist III, Amazon 1P Credito, Payments
BR, SP, Sao Paulo
Do you feel the challenge and the adrenaline kick when a huge data-set stares you in the face and you know that somewhere inside are hidden very important business insights that can fundamentally alter the way top business leaders think and act? Do you enjoy presenting strong data backed insights to business leaders; insights that can topple their long held beliefs and compel them to change their direction completely? If yes, then you are the one we are looking for. We are looking to invite passionate leaders, with expertise in generate power business insights from very large datasets, on a journey where the primary aim would be to enable needle moving business impacts through statistical analysis. We are looking for leaders who can envision the design and development of analytical infrastructure which can support strategic and tactical decision-making. Those who join this high visibility team would have to navigate through significant ambiguity in defining business problems and converting them to analytical problems. This role requires additional exposure and experience to Machine Learning. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems • Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes • Design, development, evaluate and deploy innovative and highly scalable ml models such as risk scorecards, income models, fraud models for predictive learning in credit risk applications • Research and implement novel machine learning and statistical approaches • Work closely with software engineering teams to drive real-time model implementations and new feature creations • Work closely with business owners and operations staff to optimize various business operations • Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation • Mentor other scientists and engineers in the use of ML techniques • Innovate with the latest GenAI technology to build highly automated solutions for efficient customer promotions • Design, develop and deploy end-to-end machine learning solutions in the Amazon production environment to delight Amazon customers • Collaborate with cross-functional teams to develop comprehensive ML/statistical models that can scale to millions of customers to multiple countries Understand the credit risk data and evaluate the best ml model/ solution for dynamic business problems. About the team Brazil Payments is part of the International Emerging Stores Payments team and focuses on supporting the launch of new payment and financial products to our customers in Brazil.
Data Scientist II, Amazon 1P Credito, Payments
BR, SP, Sao Paulo
Do you feel the challenge and the adrenaline kick when a huge data-set stares you in the face and you know that somewhere inside are hidden very important business insights that can fundamentally alter the way top business leaders think and act? Do you enjoy presenting strong data backed insights to business leaders; insights that can topple their long held beliefs and compel them to change their direction completely? If yes, then you are the one we are looking for. We are looking to invite passionate leaders, with expertise in generate power business insights from very large datasets, on a journey where the primary aim would be to enable needle moving business impacts through statistical analysis. We are looking for leaders who can envision the design and development of analytical infrastructure which can support strategic and tactical decision-making. Those who join this high visibility team would have to navigate through significant ambiguity in defining business problems and converting them to analytical problems. This role requires additional exposure and experience to Machine Learning. Key job responsibilities • Use machine learning and analytical techniques to create scalable solutions for business problems • Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes • Design, development, evaluate and deploy innovative and highly scalable models for predictive learning • Research and implement novel machine learning and statistical approaches • Work closely with software engineering teams to drive real-time model implementations and new feature creations • Work closely with business owners and operations staff to optimize various business operations • Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation • Mentor other scientists and engineers in the use of ML techniques • Innovate with the latest GenAI technology to build highly automated solutions for efficient customer promotions • Design, develop and deploy end-to-end machine learning solutions in the Amazon production environment to delight Amazon customers • Collaborate with cross-functional teams to develop comprehensive ML/statistical models that can scale to millions of customers to multiple countries About the team Brazil Payments is part of the International Emerging Stores Payments team and focuses on supporting the launch of new payment and financial products to our customers in Brazil.
Data Scientist III, Amazon 1P Credito, Payments
BR, SP, Sao Paulo
Do you feel the challenge and the adrenaline kick when a huge data-set stares you in the face and you know that somewhere inside are hidden very important business insights that can fundamentally alter the way top business leaders think and act? Do you enjoy presenting strong data backed insights to business leaders; insights that can topple their long held beliefs and compel them to change their direction completely? If yes, then you are the one we are looking for. We are looking to invite passionate leaders, with expertise in generate power business insights from very large datasets, on a journey where the primary aim would be to enable needle moving business impacts through statistical analysis. We are looking for leaders who can envision the design and development of analytical infrastructure which can support strategic and tactical decision-making. Those who join this high visibility team would have to navigate through significant ambiguity in defining business problems and converting them to analytical problems. This role requires additional exposure and experience to Machine Learning. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems • Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes • Design, development, evaluate and deploy innovative and highly scalable ml models such as risk scorecards, income models, fraud models for predictive learning in credit risk applications • Research and implement novel machine learning and statistical approaches • Work closely with software engineering teams to drive real-time model implementations and new feature creations • Work closely with business owners and operations staff to optimize various business operations • Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation • Mentor other scientists and engineers in the use of ML techniques • Innovate with the latest GenAI technology to build highly automated solutions for efficient customer promotions • Design, develop and deploy end-to-end machine learning solutions in the Amazon production environment to delight Amazon customers • Collaborate with cross-functional teams to develop comprehensive ML/statistical models that can scale to millions of customers to multiple countries Understand the credit risk data and evaluate the best ml model/ solution for dynamic business problems. About the team Brazil Payments is part of the International Emerging Stores Payments team and focuses on supporting the launch of new payment and financial products to our customers in Brazil.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social