Social
Conversational AI

A quick guide to Amazon’s papers at ACL 2024

Work on large language models predominates, with a particular focus on model evaluation.

By Staff writer
August 12, 2024
5 min read
Share

Like the field of conversational AI in general, Amazon’s papers at this year’s meeting of the Association for Computational Linguistics (ACL) are dominated by work on large language models (LLMs). The properties that make LLMs’ outputs so extraordinary — such as their linguistic fluency and semantic coherence — are also notoriously difficult to quantify; as such, model evaluation has emerged as a particular area of focus. But Amazon’s papers explore a wide range of LLM-related topics, from applications such as code synthesis and automatic speech recognition to problems of LLM training and deployment, such as continual pretraining and hallucination mitigation. Papers accepted to the recently inaugurated Proceedings of the ACL are marked with asterisks.

Code synthesis

Fine-tuning language models for joint rewriting and completion of code with potential bugs
Dingmin Wang, Jinman Zhao, Hengzhi Pei, Samson Tan, Sheng Zha

Bug injection.png
Obtaining buggy partial code via bug injection. From “Fine-tuning language models for joint rewriting and completion of code with potential bugs”.

Continual pretraining

Efficient continual pre-training for building domain specific large language models*
Yong Xie, Karan Aggarwal, Aitzaz Ahmad

Data quality

A shocking amount of the web is machine translated: Insights from multi-way parallelism*
Brian Thompson, Mehak Dhaliwal, Peter Frisch, Tobias Domhan, Marcello Federico

Document summarization

The power of summary-source alignments
Ori Ernst, Ori Shapira, Aviv Slobodkin, Sharon Adar, Mohit Bansal, Jacob Goldberger, Ran Levy, Ido Dagan

Hallucination mitigation

Learning to generate answers with citations via factual consistency models
Rami Aly, Zhiqiang Tang, Samson Tan, George Karypis

Intent classification

Can your model tell a negation from an implicature? Unravelling challenges with intent encoders
Yuwei Zhang, Siffi Singh, Sailik Sengupta, Igor Shalyminov, Hwanjun Song, Hang Su, Saab Mansour

Irony recognition

MultiPICo: Multilingual perspectivist irony corpus
Silvia Casola, Simona Frenda, Soda Marem Lo, Erhan Sezerer, Antonio Uva, Valerio Basile, Cristina Bosco, Alessandro Pedrani, Chiara Rubagotti, Viviana Patti, Davide Bernardi

Knowledge grounding

Graph chain-of-thought: Augmenting large language models by reasoning on graphs
Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Zheng Li, Ruirui Li, Xianfeng Tang, Suhang Wang, Yu Meng, Jiawei Han

MATTER: Memory-augmented transformer using heterogeneous knowledge sources*
Dongkyu Lee, Chandana Satya Prakash, Jack G. M. FitzGerald, Jens Lehmann

Tree-of-traversals: A zero-shot reasoning algorithm for augmenting black-box language models with knowledge graphs
Elan Markowitz, Anil Ramakrishna, Jwala Dhamala, Ninareh Mehrabi, Charith Peris, Rahul Gupta, Kai-Wei Chang, Aram Galstyan

Tree of traversals.png
An example of how the tree-of-traversals method uses a knowledge graph interface for the query “What actor played in both Inception and Interstellar?” From "Tree-of-traversals: A zero-shot reasoning algorithm for augmenting black-box language models with knowledge graphs".

LLM decoding

BASS: Batched attention-optimized speculative sampling*
Haifeng Qian, Sujan Gonugondla, Sungsoo Ha, Mingyue Shang, Sanjay Krishna Gouda, Ramesh Nallapati, Sudipta Sengupta, Anoop Deoras

Machine translation

Impacts of misspelled queries on translation and product search
Greg Hanneman, Natawut Monaikul, Taichi Nakatani

The fine-tuning paradox: Boosting translation quality without sacrificing LLM abilities
David Stap, Eva Hasler, Bill Byrne, Christof Monz, Ke Tran

Model editing

Propagation and pitfalls: Reasoning-based assessment of knowledge editing through counterfactual tasks
Wenyue Hua, Jiang Guo, Marvin Dong, Henghui Zhu, Patrick Ng, Zhiguo Wang

ReCoE construction.png
Demonstration of the process used to construct data for the reasoning-based counterfactual-editing (ReCoE) dataset. Straight lines represent data sourced from existing datasets; dashed lines denote data derived from LLM generation; zigzag lines denote data obtained through the corruption of other data. From "Propagation and pitfalls: Reasoning-based assessment of knowledge editing through counterfactual tasks".

Model evaluation

Bayesian prompt ensembles: Model uncertainty estimation for black-box large language models
Francesco Tonolini, Jordan Massiah, Nikolaos Aletras, Gabriella Kazai

ConSiDERS—the-human evaluation framework: Rethinking human evaluation for generative large language models
Aparna Elangovan, Ling Liu, Lei Xu, Sravan Bodapati, Dan Roth

Factual confidence of LLMs: On reliability and robustness of current estimators
Matéo Mahaut, Laura Aina, Paula Czarnowska, Momchil Hardalov, Thomas Müller, Lluís Marquez

Fine-tuned machine translation metrics struggle in unseen domains
Vilém Zouhar, Shuoyang Ding, Anna Currey, Tatyana Badeka, Jenyuan Wang, Brian Thompson

Measuring question answering difficulty for retrieval-augmented generation
Matteo Gabburo, Nicolaas Jedema, Siddhant Garg, Leonardo Ribeiro, Alessandro Moschitti

Model robustness

Extreme miscalibration and the illusion of adversarial robustness
Vyas Raina, Samson Tan, Volkan Cevher, Aditya Rawal, Sheng Zha, George Karypis

Multimodal models

CaMML: Context-aware multimodal learner for large models
Yixin Chen, Shuai Zhang, Boran Han, Tong He, Bo Li

CAMML.png
The CaMML framework, which consists of a retriever, a perceiver and a generator. After receiving user query q, the CaMML retriever identifies relevant multimodal contexts C from the data store. Then the CaMML perceiver seamlessly integrates data of various modalities, effectively encoding long-context information and injecting it into the CaMML generator. This enables the prediction of responses that are conditioned on both the context and the query. From "CaMML: Context-aware multimodal learner for large models".

Multi-modal retrieval for large language model based speech recognition
Jari Kolehmainen, Aditya Gourav, Prashanth Gurunath Shivakumar, Yi Gu, Ankur Gandhe, Ariya Rastrow, Grant Strimel, Ivan Bulyko

REFINESUMM: Self-refining MLLM for generating a multimodal summarization dataset
Vaidehi Patil, Leonardo Ribeiro, Mengwen Liu, Mohit Bansal, Markus Dreyer

Ordinal classification

Exploring ordinality in text classification: A comparative study of explicit and implicit techniques
Siva Rajesh Kasa, Aniket Goel, Sumegh Roychowdhury, Karan Gupta, Anish Bhanushali, Nikhil Pattisapu, Prasanna Srinivasa Murthy

Question answering

Beyond boundaries: A human-like approach for question answering over structured and unstructured information sources*
Jens Lehmann, Dhananjay Bhandiwad, Preetam Gattogi, Sahar Vahdati

MinPrompt: Graph-based minimal prompt data augmentation for few-shot question answering
Xiusi Chen, Jyun-Yu Jiang, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Wei Wang

Synthesizing conversations from unlabeled documents using automatic response segmentation
Fanyou Wu, Weijie Xu, Chandan Reddy, Srinivasan Sengamedu, "SHS"

Reasoning

Eliciting better multilingual structured reasoning from LLMs through code
Bryan Li, Tamer Alkhouli, Daniele Bonadiman, Nikolaos Pappas, Saab Mansour

II-MMR: Identifying and improving multi-modal multi-hop reasoning in visual question answering*
Jihyung Kil, Farideh Tavazoee, Dongyeop Kang, Joo-Kyung Kim

Recommender systems

Generative explore-exploit: Training-free optimization of generative recommender systems using LLM optimizers
Besnik Fetahu, Zhiyu Chen, Davis Yoshida, Giuseppe Castellucci, Nikhita Vedula, Jason Choi, Shervin Malmasi

Towards translating objective product attributes into customer language
Ram Yazdi, Oren Kalinsky, Alexander Libov, Dafna Shahaf

Responsible AI

SpeechGuard: Exploring the adversarial robustness of multimodal large language models
Raghuveer Peri, Sai Muralidhar Jayanthi, Srikanth Ronanki, Anshu Bhatia, Karel Mundnich, Saket Dingliwal, Nilaksh Das, Zejiang Hou, Goeric Huybrechts, Srikanth Vishnubhotla, Daniel Garcia-Romero, Sundararajan Srinivasan, Kyu Han, Katrin Kirchhoff

Text completion

Token alignment via character matching for subword completion*
Ben Athiwaratkun, Shiqi Wang, Mingyue Shang, Yuchen Tian, Zijian Wang, Sujan Gonugondla, Sanjay Krishna Gouda, Rob Kwiatkowski, Ramesh Nallapati, Bing Xiang

Token alignment.png
An illustration of token alignment process presented in "Token alignment via character matching for subword completion".

Research areas
Tags
About the Author
Staff writer

Related content

Work with us

See more jobs See more jobs
Applied Scientist II, Amazon Connect
US, CA, Mountain View
Do you want to join a team of innovative scientists to research and develop generative AI technology that would disrupt the industry? Do you enjoy dealing with ambiguity and working on hard problems in a fast-paced environment? Amazon Connect is a highly disruptive cloud-based contact center from AWS that enables businesses to deliver intelligent, engaging, dynamic, and personalized customer service experiences. The Agentic Customer Experience (ACX) org is responsible for weaving native-AI across the Connect application experiences delivered to end-customers, agents, and managers/supervisors. The Interactive AI Science team, serves as the cornerstone for AI innovation across Amazon Connect, functioning as the sole science team support high impact product including Amazon Q in Connect, Contact Lens and other key initiatives. As an Applied Scientist on our team, you will work closely with senior technical and business leaders from within the team and across AWS. You distill insight from huge data sets, conduct cutting edge research, foster ML models from conception to deployment. You have deep expertise in machine learning and deep learning broadly, and extensive domain knowledge in natural language processing, generative AI and LLM Agents evaluation and optimization, etc. You are comfortable with quickly prototyping and iterating your ideas to build robust ML models using technology such as PyTorch, Tensorflow and AWS Sagemaker. The ideal candidate has the ability to understand, implement, innovate on the state-of-the-art Agentic AI based systems. We have a rapidly growing customer base and an exciting charter in front of us that includes solving highly complex engineering and scientific problems. We are looking for passionate, talented, and experienced people to join us to innovate on modern contact centers in the cloud. The position represents a rare opportunity to be a part of a fast-growing business soon after launch, and help shape the technology and product as we grow. You will be playing a crucial role in developing the next generation contact center, and get the opportunity to design and deliver scalable, resilient systems while maintaining a constant customer focus. About the team Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
Sr Research Scientist, Amazon Shipping
IN, HR, Gurugram
Lead ML teams building large-scale forecasting and optimization systems that power Amazon’s global transportation network and directly impact customer experience and cost. As an Sr Research Scientist, you will set scientific direction, mentor applied scientists, and partner with engineering and product leaders to deliver production-grade ML solutions at massive scale. Key job responsibilities 1. Lead and grow a high-performing team of Applied Scientists, providing technical guidance, mentorship, and career development. 2. Define and own the scientific vision and roadmap for ML solutions powering large-scale transportation planning and execution. 3. Guide model and system design across a range of techniques, including tree-based models, deep learning (LSTMs, transformers), LLMs, and reinforcement learning. 4. Ensure models are production-ready, scalable, and robust through close partnership with stakeholders. Partner with Product, Operations, and Engineering leaders to enable proactive decision-making and corrective actions. 5. Own end-to-end business metrics, directly influencing customer experience, cost optimization, and network reliability. 6. Help contribute to the broader ML community through publications, conference submissions, and internal knowledge sharing. A day in the life Your day includes reviewing model performance and business metrics, guiding technical design and experimentation, mentoring scientists, and driving roadmap execution. You’ll balance near-term delivery with long-term innovation while ensuring solutions are robust, interpretable, and scalable. Ultimately, your work helps improve delivery reliability, reduce costs, and enhance the customer experience at massive scale.
Member of Technical Staff, Engineering Manager , Amazon AGI Lab
US, MA, Boston
Amazon launched the AGI Lab to develop foundational capabilities for useful AI agents. We built Nova Act - a new AI model trained to perform actions within a web browser. The team builds AI/ML infrastructure that powers our production systems to run performantly at high scale. We’re also enabling practical AI to make our customers more productive, empowered, and fulfilled. In particular, our work combines large language models (LLMs) with reinforcement learning (RL) to solve reasoning, planning, and world modeling in both virtual and physical environments. Our lab is a small, talent-dense team with the resources and scale of Amazon. Each team in the lab has the autonomy to move fast and the long-term commitment to pursue high-risk, high-payoff research. We’re entering an exciting new era where agents can redefine what AI makes possible. We’d love for you to join our lab and build it from the ground up! Key job responsibilities This role will lead a team of SDEs building AI agents infrastructure from launch to scale. The role requires the ability to span across ML/AI system architecture and infrastructure. You will work closely with application developers and scientists to have a impact on the Agentic AI industry. We're looking for a Software Development Manager who is energized by building high performance systems, making an impact and thrives in fast-paced, collaborative environments. About the team Check out the Nova Act tools our team built on on nova.amazon.com/act
Instrument Control Engineer, Software, Center for Quantum Computing
US, CA, Pasadena
The Amazon Center for Quantum Computing (CQC) is a multi-disciplinary team of scientists, engineers, and technicians, on a mission to develop a fault-tolerant quantum computer. We are looking to hire an Instrument Control Engineer to join our growing software team. You will work closely with our experimental physics and control hardware development teams to enable their work characterizing, calibrating, and operating novel quantum devices. The ideal candidate should be able to translate high-level science requirements into software implementations (e.g. Python APIs/frameworks, compiler passes, embedded SW, instrument drivers) that are performant, scalable, and intuitive. This requires someone who (1) has a strong desire to work within a team of scientists and engineers, and (2) demonstrates ownership in initiating and driving projects to completion. This role has a particular emphasis on working directly with our control hardware designers and vendors to develop instrument software for test and measurement. Inclusive Team Culture Here at Amazon, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon conferences, inspire us to never stop embracing our uniqueness. Diverse Experiences Amazon values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Export Control Requirement Due to applicable export control laws and regulations, candidates must be either a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum, or be able to obtain a US export license. If you are unsure if you meet these requirements, please apply and Amazon will review your application for eligibility. Key job responsibilities - Work with control hardware developers, as a “subject matter expert” on the software interfaces around our control hardware - Collaborate with external control hardware vendors to understand and refine integration strategies - Implement instrument drivers and control logic in Python and/or a low-level languages, including C++ or Rust - Contribute to our compiler backend to enable the efficient execution of OpenQASM-based experiments on our next-generation control hardware - Benchmark system performance and help define key performance metrics - Ensure new features are successfully integrated into our Python-based experimental software stack - Partner with scientists to actively contribute to the codebase through mentorship and documentation We are looking for candidates with strong engineering principles, a bias for action, superior problem-solving, and excellent communication skills. Working effectively within a team environment is essential. As an Instrument Control Engineer embedded in a broader science organization, you will have the opportunity to work on new ideas and stay abreast of the field of experimental quantum computation. A day in the life Your time will be spent on projects that extend functional capabilities or performance of our internal research software stack. This requires working backwards from the needs of science staff in the context of our larger experimental roadmap. You will translate science and software requirements into design proposals balancing implementation complexity against time-to-delivery. Once a design proposal has been reviewed and accepted, you’ll drive implementation and coordinate with internal stakeholders to ensure a smooth roll out. Because many high-level experimental goals have cross-cutting requirements, you’ll often work closely with other engineers or scientists or on the team. About the team You will be joining the Software group within the Amazon Center of Quantum Computing. Our team is comprised of scientists and software engineers who are building scalable software that enables quantum computing technologies.
Applied Scientist, Workplace Health & Safety
US, WA, Bellevue
Are you driven by the challenge of solving complex problems that directly impact the safety and well-being of millions of Amazon Associates worldwide? Do you want to push the boundaries of AI to build innovative solutions that make workplaces safer and more efficient? If so, we invite you to join our WHS DataTech team as an Applied Scientist and take your career to the next level! At WHS DataTech, we leverage Large Language Models (LLMs), Computer Vision, and AI-driven innovations to develop industry-leading solutions that proactively enhance workplace safety. Our work spans real-time risk assessment, predictive analytics, and AI-powered insights, all aimed at creating a safer work environment at scale. As an Applied Scientist specializing in LLMs and Computer Vision, you will play a pivotal role in shaping our next-generation safety solutions. You’ll be at the forefront of innovation, designing and implementing AI-powered features that redefine workplace safety. Your work will drive strategic decisions, optimize system architecture, and influence best practices, ensuring our technology remains industry-leading. Key job responsibilities - Apply LLM model to analyze complex unstructured datasets and extract meaningful insights. - Collaborate with software engineers to implement and deploy machine learning (LLM or CV) solutions. - Conduct experiments and evaluate model performance, iterating and improving as needed. - Stay up-to-date with the latest advancements in machine learning and related fields. - Collaborate with cross-functional teams to understand business needs and identify areas for application of machine learning. - Present findings and recommendations to stakeholders and contribute to the overall research and development strategy. A day in the life Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! About the team WHS DataTech is a multidisciplinary team of scientists and engineers dedicated to building AI-powered solutions that improve workplace safety across Amazon. We work at the intersection of large-scale data, advanced machine learning, and computer vision, delivering innovations that enhance decision-making, streamline operations, and protect millions of associates worldwide. Our collaborative culture emphasizes scientific rigor, engineering excellence, and a strong mission focus on creating safer, more efficient workplaces.
Applied Scientist II, AERO Agentic AI Team
IN, KA, Bengaluru
Amazon is seeking passionate, talented and inventive Applied Scientists with a strong machine learning background to help build Agentic AI solutions for Selling Partners and Retail users. Our mission is to elevate their experience through intelligent, collaborative Agentic AI solution powered by specialized agents that deliver seamless, personalized support at scale. Key job responsibilities As part of our AERO team in Amazon PGX, you will work alongside internationally recognized experts to develop novel solutions and modeling techniques to advance the state-of-the-art in Generative and Agentic AI. Your work will directly impact millions of our users in the form of products and services that improves their experience. You will gain hands on experience with Amazon’s native/partnered Large Language Models (LLMs), technologies like AgentCore, AgentMemory, Strands and Model Context Protocol (MCPs). We are also looking for talents with experiences/expertise in building large-scale and high-performing systems.
Machine Learning Scientist / Applied Scientist, EU Prime and Marketing Analytics & Science (PRIMAS)
ES, B, Barcelona
Are you interested in building the measurement foundation that proves whether targeted, cohort-based marketing actually changes customer behavior at Amazon scale? We are seeking an Applied Scientist to own measurement and experimentation for our Lifecycle Marketing Experimentation roadmap within the PRIMAS (Prime & Marketing Analytics and Science) team. In this role, you will design and execute rigorous experiments that measure the effectiveness of audience-based marketing campaigns across multiple channels, providing the evidence that guides marketing strategy and investment decisions. This is a high-impact role where you will build measurement frameworks from scratch, design experiments that isolate causal effects, and establish the experimental standards for lifecycle marketing across EU. You will work closely with business leaders and the senior science lead to answer critical questions: does targeting specific cohorts (Bargain hunters, Young adults) improve efficiency vs. broad campaigns? Which creative strategies drive behavior change? How should we optimize marketing spend across channels? Key job responsibilities Measurement & Experimentation Ownership: 1. Own measurement end-to-end for lifecycle marketing campaigns – design experiments (RCTs, geo-tests, audience holdouts) that measure campaign effectiveness across marketing channels 2. Build measurement frameworks and experimental best practices that work across different activation platforms and can scale to multiple campaigns 3. Establish experimental standards and tooling for lifecycle marketing, ensuring statistical rigor while balancing business constraints Causal Inference & Analysis: 1. Apply causal inference methods to measure incremental impact of marketing campaigns vs. counterfactual 2. Navigate measurement challenges across different platforms (Meta attribution, LiveRamp, clean rooms, onsite tracking) 3. Analyze experiment results and provide optimization recommendations based on statistical evidence 4. Establish guardrails and success criteria for campaign evaluation About the team The PRIMAS team, is part of a larger tech tech team of 100+ people called WIMSI (WW Integrated Marketing Systems and Intelligence). WIMSI core mission is to accelerate marketing technology capabilities that enable de-averaged customer experiences across the marketing funnel: awareness, consideration, and conversion.
Principal Applied Scientist, Amazon - Sponsored Products and Brands
US, WA, Seattle
The Sponsored Products and Brands (SPB) team at Amazon Ads is re-imagining the advertising landscape through generative and agentic AI technologies, revolutionizing how millions of customers discover products and engage with brands across Amazon.com and beyond. We are at the forefront of re-inventing advertising experiences to transform every aspect of the advertising lifecycle; from ad creation, delivery, optimization, performance management, and beyond. We are a passionate group of innovators dedicated to developing state-of-the-art AI technologies that balance the needs of advertisers and enhance the shopping experience. Within SPB, the SPB Offsite (SPBO) team builds solutions to extend campaigns to reach customers off the store and extend shopping experiences on third-party sites where shoppers search and discover products. We use industry-leading machine learning, high-scale low-latency systems, and gen AI technologies to create better sponsored customer experiences off the store. The Principal Applied Scientist for SPBO leads the technical vision and scientific strategy for extending Amazon Advertising's sponsored experiences to the broader web—meeting shoppers wherever they search, browse, and discover products. This is a multi-disciplinary scientific space spanning machine learning, large-scale optimization, causal inference, NLP, information retrieval, and generative AI. You will define and drive the science roadmap for how Amazon connects advertisers with high-intent customers across third-party environments at massive scale and with low latency. As a GenAI-first organization, we build foundational and agentic models that power advertiser use cases across Ads, while empowering our Applied Scientists to directly build and ship products. You will be a hands-on technical leader who architects novel solutions end-to-end—from research through production—while mentoring a team of scientists across diverse domains. The problems you will tackle are among the hardest in ad tech. You will develop models that leverage Amazon's first-party shopping signals to reach high-value audiences in third-party environments where signal density differs fundamentally from on-Amazon contexts. You will innovate on real-time bidding, auction dynamics, and ranking models across heterogeneous supply sources with distinct inventory characteristics, latency constraints, and auction mechanics. You will design ML approaches that maintain effectiveness amid an evolving privacy landscape—turning constraints from cookie deprecation, regulation, and platform restrictions into innovation opportunities. You will influence attribution models that capture the incremental value of offsite advertising on shopping outcomes, bridging measurement gaps between offsite touchpoints and on-Amazon conversions. You will pioneer generative and agentic AI to personalize ad creatives and shopping experiences for offsite contexts, and develop scientific frameworks to optimize spend allocation across supply partners and channels. You will partner with engineering, product, and business leaders as well as external partners to shape product strategy with scientific insight and drive results at scale. You will represent Amazon Advertising's offsite science externally through patents and industry engagement. Key job responsibilities - Driving the scientific vision of the teams in your organization and advising and influencing its technical leadership on ad serving, bidding, ranking, and offsite advertising models and products. - Identifying, tackling, and proposing innovative solutions to intrinsically hard, previously unsolved problems in offsite ad tech. - Bringing clarity to complex problems, probing assumptions, illuminating pitfalls, fostering shared understanding, and guiding towards effective solutions. - Serving and being recognized by internal and external peers as a thought leader in offsite advertising science, including real-time bidding, personalization, privacy-preserving ML, and generative AI for ad experiences. - Influencing your team's science and business strategy by driving one or more team roadmaps contributing to the organization's roadmap and taking responsibility for some organizational goals. You drive multiple new product features from inception to production launch. - Guiding the career development of others, actively mentoring and educating the larger applied science community on trends, technologies, and best practices.
Applied Scientist II, Industrial Robotics Group
US, MA, N.reading
Industrial Robotics Group is seeking exceptional talent to help develop the next generation of advanced robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine cutting-edge AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at an unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic manipulation, locomotion, and human-robot interaction. We're seeking an Applied Scientist to join our Robotics team. This role focuses on developing innovative machine learning solutions that enable robots to perform complex manipulation tasks in real-world environments. You will work on creating adaptive learning approaches that combine traditional robotics with modern ML techniques to improve robot performance and reliability. In this role, you will collaborate with multidisciplinary teams to advance the state-of-the-art in robotic manipulation, contributing to the development of next-generation autonomous systems that can operate safely and efficiently within Amazon fulfillment centers. Key job responsibilities - Lead design, adapt, and implement novel machine learning solutions for manipulation robots - Create hybrid approaches combining classical methods with learning-based solutions - Design learning algorithms for automated parameter tuning and adaptation - Develop data collection pipelines and methodologies for capturing high-quality demonstrations of dexterous tasks - Build and test prototype robotic workcell setups to validate the performance of the solution - Partner with cross-functional teams to rapidly create new concepts and prototypes - Work with Amazon's robotics engineering and operations teams to grasp their requirements and develop tailored solutions - Document the architecture, performance, and validation of the final system
Applied Scientist, EU INTech Consumer Selection, Tamale
ES, M, Madrid
At Amazon, we are committed to being the Earth's most customer-centric company. The European International Technology group (EU INTech) owns the enhancement and delivery of Amazon's engineering to all the varied customers and cultures of the world. We do this through a combination of partnerships with other Amazon technical teams and our own innovative new projects. You will be joining the Tamale team to work on Haul. As part of EU INTech and Haul, Tamale strives to create a discovery-driven shopping experience using challenging machine learning and ranking solutions. You will be exposed to large-scale recommendation systems, multi-objective optimization, and state-of-the-art deep learning architectures, and you'll be part of a key effort to improve our customers' browsing experience by building next-generation ranking models for Amazon Haul's endless scroll experience. We are looking for a passionate, talented, and inventive Scientist with a strong machine learning background to help build industry-leading ranking solutions. We strongly value your hard work and obsession to solve complex problems on behalf of Amazon customers. Key job responsibilities We look for applied scientists who possess a wide variety of skills. As the successful applicant for this role, you will work closely with your business partners to identify opportunities for innovation. You will apply machine learning solutions to optimize multi-objective ranking, improve discovery engagement through contextual signals, and scale ranking systems across multiple marketplaces. You will work with business leaders, scientists, and product managers to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed ranking services. You will be part of a team of scientists and engineers working on solving ranking and personalization challenges at scale. You will be able to influence the scientific roadmap of the team, setting the standards for scientific excellence. You will be working with state-of-the-art architectures and real-time feature serving systems. Your work will improve the experience of millions of daily customers using Amazon Haul worldwide. You will have the chance to have great customer impact and continue growing in one of the most innovative companies in the world. You will learn a huge amount - and have a lot of fun - in the process!
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our monthly newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social