Social
Conversational AI

A quick guide to Amazon's 65-plus papers at this year's ACL

Familiar topics such as question answering and natural-language understanding remain well represented, but a new concentration on language modeling and multimodal models reflect the spread of generative AI.

By Staff writer
July 10, 2023
10 min read
Share

Between the main conference and the recently inaugurated ACL Proceedings, Amazon researchers have more than 65 papers at this year's meeting of the Association for Computational Linguistics (ACL).

Automatic speech recognition

Masked audio text encoders are effective multi-modal rescorers*
Jason Cai, Monica Sunkara, Xilai Li, Anshu Bhatia, Xiao Pan, Sravan Bodapati

Code generation

A static evaluation of code completion by large language models
Hantian Ding, Varun Kumar, Yuchen Tian, Zijian Wang, Rob Kwiatkowski, Xiaopeng LI, Murali Krishna Ramanathan, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta, Dan Roth, Bing Xiang

Multitask pretraining with structured knowledge for text-to-SQL generation
Robert Giaquinto, Dejiao Zhang, Benjamin Kleiner, Yang Li, Ming Tan, Parminder Bhatia, Ramesh Nallapati, Xiaofei Ma

Code switching

Code-switched text synthesis in unseen language pairs*
I-Hung Hsu, Avik Ray, Shubham Garg, Nanyun Peng, Jing Huang

Code-switched text synthesis.png
From "Code-switched text synthesis in unseen language pairs".

CoMix: Guide transformers to code-mix using POS structure and phonetics*
Gaurav Arora, Srujana Merugu, Vivek Sembium

Continual learning

Characterizing and measuring linguistic dataset drift
Tyler A. Chang, Kishaloy Halder, Neha Anna John, Yogarshi Vyas, Yassine Benajiba, Miguel Ballesteros, Dan Roth

Data-/table-to-text applications

An inner table retriever for robust table question answering
Weizhe Lin, Rexhina Blloshmi, Bill Byrne, Adrià de Gispert, Gonzalo Iglesias

Few-shot data-to-text generation via unified representation and multi-source learning
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, JIE MA, Patrick Ng, Zhiguo Wang, Bonan Min, William Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang

Improving cross-task generalization of unified table-to-text models with compositional task configurations*
Jifan Chen, Yuhao Zhang, Lan Liu, Rui Dong, Xinchi Chen, Patrick Ng, William Wang, Zhiheng Huang

LI-RAGE: Late interaction retrieval augmented generation with explicit signals for open-domain table question answering
Weizhe Lin, Rexhina Blloshmi, Bill Byrne, Adrià de Gispert, Gonzalo Iglesias

Dialogue

Diable: Efficient dialogue state tracking as operations on tables*
Pietro Lesci, Yoshinari Fujinuma, Momchil Hardalov, Chao Shang, Lluis Marquez

NatCS: Eliciting natural customer support dialogues
James Gung, Emily Moeng, Wesley Rose, Arshit Gupta, Yi Zhang, Saab Mansour

Schema-guided user satisfaction modeling for task-oriented dialogues
Yue Feng, Yunlong Jiao, Animesh Prasad, Nikolaos Aletras, Emine Yilmaz, Gabriella Kazai

Toward more accurate and generalizable evaluation metrics for task-oriented dialogs
Abi Komma, Nagesh Panyam, Timothy Leffel, Anuj Goyal, Angeliki Metallinou, Spyros Matsoukas, Aram Galstyan

Explainable AI

Efficient Shapley values estimation by amortization for text classification
Alan Yang, Fan Yin, He He, Kai-Wei Chang, Xiaofei Ma, Bing Xiang

Efficient Shapley values.png
From "Efficient Shapley values estimation by amortization for text classification".

Few shot rationale generation using self-training with dual teachers*
Aditya Srikanth Veerubhotla, Lahari Poddar, Jun Yin, Gyuri Szarvas, Sharanya Eswaran

Information extraction

An AMR-based link prediction approach for document-level event argument extraction
Yuqing Yang, Qipeng Guo, Xiangkun Hu, Yue Zhang, Qipeng Guo, Zheng Zhang

AVEN-GR: Attribute value extraction and normalization using product graphs
Donato Crisostomi, Thomas Ricatte

Large scale generative multimodal attribute extraction for e-commerce attributes
Anant Khandelwal, Happy Mittal, Shreyas Sunil Kulkarni, Deepak Gupta

ParaAMR: A large-scale syntactically diverse paraphrase dataset by AMR back-translation
Kuan-Hao Huang, Varun Iyer, I-Hung Hsu, Anoop Kumar, Kai-Wei Chang, Aram Galstyan

ParaAMR.png
From "ParaAMR: A large-scale syntactically diverse paraphrase dataset by AMR back-translation".

Weakly supervised hierarchical multi-task classification of customer questions
Jitenkumar Rana, Promod Yenigalla, Chetan Aggarwal, Sandeep Mukku, Manan Soni, Rashmi Patange

WebIE: Faithful and robust information extraction on the web
Chenxi Whitehouse, Clara Vania, Alham Fikri Aji, Christos Christodoulopoulos, Andrea Pierleoni

Information retrieval

CUPID: Curriculum learning based real-time prediction using distillation
Arindam Bhattacharya, Ankith M S, Ankit Gandhi, Vijay Huddar, Atul Saroop, Rahul Bhagat

Direct fact retrieval from knowledge graphs without entity linking
Jinheon Baek, Alham Fikri Aji, Jens Lehmann, Sung Ju Hwang

Language modeling

Adaptation approaches for nearest neighbor language models*
Rishabh Bhardwaj, George Polovets, Monica Sunkara

CONTRACLM: Contrastive learning for causal language model
Nihal Jain, Dejiao Zhang, Wasi Ahmad, Zijian Wang, Feng Nan, Xiaopeng LI, Ming Tan, Baishakhi Ray, Parminder Bhatia, Xiaofei Ma, Ramesh Nallapati, Bing Xiang

Controlled text generation with hidden representation transformations*
Vaibhav Kumar, Hana Koorehdavoudi, Masud Moshtaghi, Amita Misra, Ankit Chadha, Emilio Ferrara

KILM: Knowledge injection into encoder-decoder language models
Yan XU, Mahdi Namazifar, Devamanyu Hazarika, Aishwarya Padmakumar, Yang Liu, Dilek Hakkani-Tür

KILM.png
From "KILM: Knowledge injection into encoder-decoder language models".

ReAugKD: Retrieval-augmented knowledge distillation for pre-trained language models
Jianyi Zhang, Aashiq Muhamed, Aditya Anantharaman, Guoyin Wang, Changyou Chen, Kai Zhong, Qingjun Cui, Yi Xu, Belinda Zeng, Trishul Chilimbi, Yiran Chen

Recipes for sequential pre-training of multilingual encoder and seq2seq models*
Saleh Soltan, Andy Rosenbaum, Tobias Falke, Qin Lu, Anna Rumshisky, Wael Hamza

Rethinking the role of scale for in-context learning: An interpretability-based case study at 66 billion scale
Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff, Dan Roth

Machine learning

Mitigating the burden of redundant datasets via batch-wise unique samples and frequency-aware losses
Donato Crisostomi, Andrea Caciolai, Alessandro Pedrani, Alessandro Manzotti, Enrico Palumbo, Kay Rottmann, Davide Bernardi

Machine translation

RAMP: Retrieval and attribute-marking enhanced prompting for attribute-controlled translation
Gabriele Sarti, Phu Mon Htut, Xing Niu, Benjamin Hsu, Anna Currey, Georgiana Dinu, Maria Nădejde

Multimodal models

Benchmarking diverse-modal entity linking with generative models*
Sijia Wang, Alexander Li, Henry Zhu, Sheng Zhang, Pramuditha Perera, Chung-Wei Hang, JIE MA, William Wang, Zhiguo Wang, Vittorio Castelli, Bing Xiang, Patrick Ng

Generate then select: Open-ended visual question answering guided by world knowledge*
Xingyu Fu, Sheng Zhang, Gukyeong Kwon, Pramuditha Perera, Henry Zhu, Yuhao Zhang, Alexander Hanbo Li, William Wang, Zhiguo Wang, Vittorio Castelli, Patrick Ng, Dan Roth, Bing Xiang

KG-FLIP: Knowledge-guided fashion-domain language-image pre-training for e-commerce
Qinjin Jia, Yang Liu, Shaoyuan Xu, Huidong Liu, Daoping Wu, Jinmiao Fu, Roland Vollgraf, Bryan Wang

Resolving ambiguities in text-to-image generative models
Ninareh Mehrabi, Palash Goyal, Apurv Verma, Jwala Dhamala, Varun Kumar, Qian Hu, Kai-Wei Chang, Richard Zemel, Aram Galstyan, Rahul Gupta

Translation-enhanced multilingual text-to-image generation
Yaoyiran Li, Ching-Yun (Frannie) Chang, Stephen Rawls, Ivan Vulić, Anna Korhonen

Unsupervised melody-to-lyric generation
Yufei Tian, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Chenyang Tao, Gunnar Sigurdsson, Wenbo Zhao, Tagyoung Chung, Jing Huang, Violet Peng

Unsupervised melody-to-lyric.png
From "Unsupervised melody-to-lyric generation".

Natural-language processing

Multi-VALUE: A framework for cross-dialectal English NLP
Caleb Ziems, William Held, Jingfeng Yang, Jwala Dhamala, Rahul Gupta, Diyi Yang

vONTSS: vMF based semi-supervised neural topic modeling with optimal transport*
Weijie Xu, Xiaoyu Jiang, Srinivasan Sengamedu, "SHS", Francis Iannacci, Jinjin Zhao

Natural-language understanding

ECG-QALM: Entity-controlled synthetic text generation using contextual Q&A for NER*
Karan Aggarwal, Henry Jin, Aitzaz Ahmad

Entity contrastive learning in a large-scale virtual assistant system
Jonathan Rubin, Jason Crowley, George Leung, Morteza Ziyadi, Maria Minakova

EPIC: Multi-perspective annotation of a corpus of irony
Simona Frenda, Alessandro Pedrani, Valerio Basile, Soda Marem Lo, Alessandra Teresa Cignarella, Raffaella Panizzon, Cristina Marco, Bianca Scarlini, Viviana Patti, Cristina Bosco, Davide Bernardi

Measuring and mitigating local instability in deep neural networks*
Arghya Datta, Subhrangshu Nandi, Jingcheng Xu, Greg Ver Steeg, He Xie, Anoop Kumar, Aram Galstyan

Reducing cohort bias in natural language understanding systems with targeted self-training scheme
Thu Le, Gabriela Cortes Hernandez, Bei Chen, Melanie Bradford

Privacy

Controlling the extraction of memorized data from large language models via prompt-tuning
Mustafa Ozdayi, Charith Peris, Jack G. M. FitzGerald, Christophe Dupuy, Jimit Majmudar, Haidar Khan, Rahil Parikh, Rahul Gupta

Query rewriting

Context-aware query rewriting for improving users’ search experience on e-commerce websites
Simiao Zuo, Qingyu Yin, Haoming Jiang, Shaohui Xi, Bing Yin, Chao Zhang, Tuo Zhao

Unified contextual query rewriting
Yingxue Zhou, Jie Hao, Mukund Rungta, Yang Liu, Eunah Cho, Xing Fan, Yanbin Lu, Vishal Vasudevan, Kellen Gillespie, Zeynab Raeesy, Sawyer Shen, Edward Guo, Gokhan Tur

Question answering

Accurate training of web-based question answering systems with feedback from ranked users
Liang Wang, Ivano Lauriola, Alessandro Moschitti

Context-aware transformer pre-training for answer sentence selection
Luca Di Liello, Siddhant Garg, Alessandro Moschitti

Cross-Lingual Knowledge Distillation for answer sentence selection in low-resource languages*
Shivanshu Gupta, Yoshitomo Matsubara, Ankit Chadha, Alessandro Moschitti

Exploiting abstract meaning representation for open-domain question answering*
Cunxiang Wang, Zhikun Xu, Qipeng Guo, Xiangkun Hu, Xuefeng Bai, Zheng Zhang, Yue Zhang

Exploiting abstract meaning representation.png
From "Exploiting abstract meaning representation for open-domain question answering".

Hybrid hierarchical retrieval for open-domain question answering*
Manoj Ghuhan Arivazhagan, Lan Liu, Peng Qi, Xinchi Chen, William Wang, Zhiheng Huang

Learning answer generation using supervision from automatic question answering evaluators
Matteo Gabburo, Siddhant Garg, Rik Koncel-Kedziorski, Alessandro Moschitti

RobustQA: Benchmarking the robustness of domain adaptation for open-domain question answering*
Rujun Han, Peng Qi, Yuhao Zhang, Lan Liu, Juliette Burger, William Wang, Zhiheng Huang, Bing Xiang, Dan Roth

Reasoning

FolkScope: Intention knowledge graph construction for e-commerce commonsense discovery*
Changlong Yu, Weiqi Wang, Xin Liu, Jiaxin Bai, Yangqiu Song, Zheng Li, Yifan Gao, Tianyu Cao, Bing Yin

SCOTT: Self-consistent chain-of-thought distillation
Peifeng Wang, Zhengyang Wang, Zheng Li, Yifan Gao, Bing Yin, Xiang Ren

Self-learning

Constrained policy optimization for controlled self-learning in conversational AI systems
Mohammad Kachuee, Sungjin Lee

Scalable and safe remediation of defective actions in self-learning conversational systems
Sarthak Ahuja, Mohammad Kachuee, Fateme Sheikholeslami, Weiqing Liu, Jae Do

Semantic parsing

An empirical analysis of leveraging knowledge for low-resource task-oriented semantic parsing*
Mayank Kulkarni, Aoxiao Zhong, Nicolas Guenon Des Mesnards, Sahar Movaghati, Mukund Harakere, He Xie, Jianhua Lu

XSEMPLR: Cross-lingual semantic parsing in multiple natural languages and meaning representations
Yusen Zhang, Jun Wang, Zhiguo Wang, Rui Zhang

XSemPLR.png
From "XSEMPLR: Cross-lingual semantic parsing in multiple natural languages and meaning representations".

Spoken-language understanding

Regression-free model updates for spoken language understanding
Andrea Caciolai, Verena Weber, Tobias Falke, Alessandro Pedrani, Davide Bernardi

Sharing encoder representations across languages, domains and tasks in large-scale spoken language understanding
Jonathan Hueser, Judith Gaspers, Thomas Gueudre, Chandana Satya Prakash, Jin Cao, Daniil Sorokin, Quynh Do, Nicolas Anastassacos, Tobias Falke, Turan Gojayev, Mariusz Momotko, Denis Romasanta Rodriguez, Austin Doolittle, Kartik Balasubramaniam, Wael Hamza, Fabian Triefenbach, Patrick Lehnen

Toxic-language classification

QCon at SemEval-2023 Task 10: Data augmentation and model ensembling for detection of online sexism
Wes Feely, Prabhakar Gupta, Manas Mohanty, Tim Chon, Tuhin Kundu, Vijit Singh, Sandeep Atluri, Tanya Roosta, Viviane Ghaderi, Peter Schulam, Heba Elfardy

Towards building a robust toxicity predictor
Dmitriy Bespalov, Sourav Bhabesh, Yi Xiang, Yanjun (Jane) Qi

*Accepted to ACL Findings

Research areas
Tags
About the Author
Staff writer

Related content

Work with us

See more jobs See more jobs
Head of Flight Dynamics, Amazon Leo
US, WA, Redmond
The Head of Flight Dynamics is the single-threaded leader responsible for orbital maneuvering, orbital safety, and orbital performance across Amazon Leo's current and future satellite constellations. This role directly impacts service reliability for millions of customers worldwide through safe, precise on-orbit operations and industry-leading space safety practices. You will lead an organization of engineers, research scientists, and software developers building the algorithms, end-to-end ground system products, analysis tools, and infrastructure that enable safe and precise operation of Leo satellites at constellation scale. You will also lead special Flight Dynamics programs including GNSS independence, space traffic coordination services, and space situational awareness. The ideal candidate thrives in ambiguity, operates autonomously while building alignment across organizations, and has a proven track record of delivering complex, cross-organizational technical programs that solve critical technical and business challenges. You will partner directly with engineering leaders, product managers, legal and regulatory teams, and senior Leo leadership to drive technical clarity, architect solutions, make strategic trade-offs, and deliver these enabling capabilities for Amazon Leo. You will embody Amazon's Leadership Principles — particularly Ownership, Dive Deep, Are Right A Lot, and Hire and Develop the Best. Export Control Requirement: Due to applicable export control laws and regulations, candidates must be a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum. Key job responsibilities - Own the Flight Dynamics roadmap and technical strategy, including forward-looking initiatives that enable Amazon Leo to deliver a best-in-class customer experience and leadership in orbital safety. - Build, lead, and develop a high-performing organization of engineering managers and senior individual contributors; hire and grow top talent and foster a culture of innovation, operational excellence, and customer obsession. - Own and lead the architecture, development, and deployment of resilient ground software services and infrastructure that manage safe orbital maneuver operations of Amazon Leo satellites — ascent, station keeping, collision avoidance, and end of life descent for re-entry - Own and lead the development of algorithms for orbit determination, trajectory planning and guidance, and high-fidelity orbit modeling and prediction. - Collaborate closely with GN&C and Mission Operations to enable safe, autonomous on-orbit satellite operations that integrate seamlessly with Flight Dynamics ground systems. - Deliver the products needed by stakeholders for spacecraft design, constellation planning, and operations — including launch targets and launch-window assessments for launch and mission management teams - Provide leadership oversight and direction to the GNSS independence and space situational awareness programs. - Lead the development of metrics, monitoring, and response systems that leverage physical modeling and ML/AI to automate analysis of data from thousands of satellites for closed-loop assessment and improvement of on-orbit performance. - Initiate and lead operator-to-operator collaboration for orbital safety, and partner with regulatory and legal teams to promote adoption of best practices in space safety across the industry. - Own Flight Dynamics system risks and drive mitigations to minimize impact; manage program schedules and drive on-time delivery with support from technical program managers, balancing operational support against development efforts. - As a key voice in on-orbit operations, work with Mission Operations and peer leaders to influence efficiency, safety, performance, and simplicity for customers.
Applied Scientist, Modeling and Optimization, Modeling and Optimization
US, WA, Bellevue
Amazon's Modeling and Optimization (MOP) team seeks motivated individual with strong analytical and algorithmic skills to optimize the global logistics network and its operations. Key job responsibilities - Enhance global logistics network efficiency through data-driven optimization - Reduce variable costs by improving network design, inventory placement, process and operational planning, and resource allocation - Optimize capital investment through strategic fixed asset deployment planning - Develop metrics to quantify business impact of implemented solutions A day in the life - Lead development of production-ready algorithms and scientific tools for under-the-roof (UTR) and network process analysis and optimization - Drive planning and execution decisions on operation timing and resource allocation to improve capacity, cost, and speed. - Manage customer interactions, promote science-based processes, and incorporate customer needs into tool improvements. - Partner with team members and customers to exercise judgment on appropriate analysis methods for various business requests. - Interact with and influence adjacent systems and tools, including those for long-term operating policies and daily capacity planning. - Blend scientific expertise with business acumen to deliver impactful solutions across the organization.
Applied Scientist - Amazon Robotics, Amazon Robotics - Vulcan Stow
US, WA, Seattle
Our team in Amazon Robotics builds robotic systems that perform contact-rich manipulation tasks safely and reliably in complex, unstructured environments — at Amazon scale. Our scientists and engineers push the boundaries of robotic manipulation to handle enormous object diversity, bringing deep expertise across planning, control, perception, and machine learning. We learn from real-world data at a scale that few teams in robotics can access. We are seeking an Applied Scientist to join our Motion Behaviors team. You will drive the development of learned controllers and manipulation behaviors, applying techniques like reinforcement learning and behavior cloning to robots operating in Amazon fulfillment centers. These problems remain unsolved at scale: our robots must improve continuously in environments where simulation alone is insufficient. You will make principled decisions about when learned approaches should replace engineered solutions, and how to select behaviors based on estimated risk. You will collaborate across disciplines and leverage rich operational data to continuously improve system performance. Key job responsibilities • Develop learned controllers and manipulation behaviors, from research prototyping through deployment on production robots. • Research, design, and implement motion planning, control, and decision-making algorithms that improve the performance of deployed systems. • Design and deploy learning pipelines that take policies from simulation training to reliable, real-time execution on physical robots. • Develop models that predict manipulation outcomes and inform behavior selection under uncertainty. • Leverage operational data from deployed systems to systematically identify failure modes and drive policy improvements. • Represent Amazon in academia through publications and scientific presentations. A day in the life Amazon offers a full range of benefits that assist you and eligible family members, including domestic partners. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply!
Sr. Data Scientist, Amazon Pay Data Products
IN, KA, Bengaluru
Amazon Pay strives to be Earth’s most customer-centric payments service. Our mission is to serve customers and merchant partners with the most trusted, friction-less and rewarding payment solutions for their needs on and off Amazon. We are seeking an exceptional Data Scientist III to drive innovation in machine learning and artificial intelligence solutions while leading high-impact initiatives across the organization. Key job responsibilities Technical Excellence Lead end-to-end machine learning projects using PyTorch, AWS SageMaker, and other leading ML frameworks Design and implement complex statistical models and deep learning solutions Develop and optimize MLOps pipelines for model training, evaluation, and deployment Experience with modern LLM frameworks and Generative AI applications Expertise in Python, R, and related data science libraries MLOps & Development Build automated ML pipelines using AWS services (CodePipeline, Lambda, Step Functions) Implement CI/CD practices for ML model deployment and monitoring Create containerized solutions using Docker for scalable model deployment Experience with model optimization and hyperparameter tuning using tools like Optuna Integrate ML solutions with monitoring tools like MLflow Business Impact & Leadership Partner with stakeholders to translate business problems into technical solutions Design and develop business intelligence applications for real-time insights Lead technical initiatives and mentor junior data scientists Drive cross-functional collaboration to deliver innovative solutions Communicate complex technical concepts to non-technical audiences About the team The Amazon Pay Data Products team is a central unit that builds and maintains data products supporting Amazon Pay's growth across multiple markets. We operate at scale, processing 150M+ monthly transactions and managing 12 PB of data infrastructure. Our team consists of Business Intelligence Engineers, Data Engineers, and Product Managers who develop and maintain standardized reporting, data marts, and self-service analytics tools. Our expanded capabilities cover data science and Gen AI wherein we have built our first suite of multi-agent systems.
Principal Applied Scientist, AAIS
US, WA, Seattle
AI assistants are getting genuinely good at remembering individuals: your preferences, your projects, the thread you left open last week. But that memory stops at the edge of one person's usage. It doesn't reach the level at which real work happens, where the knowledge that matters is spread across many people, where one person's decision changes what everyone else should do next, and where nobody has the full picture. We're building AI that operates at that level: a durable, accurate understanding of how a team works, used to make that team measurably faster. We are looking for a Principal Applied Scientist to own the scientific direction of that work. This is a broad, ambiguous, high-leverage charter. The problems span knowledge representation, temporal reasoning, retrieval, agentic behavior, and the measurement science needed to know whether any of it is working. You will not be handed a well-posed problem. You will decide which problems are worth posing. This is a science leadership role, not a solo research role. You will set direction and raise the scientific bar across a team of applied scientists and MLEs, while staying deep enough in the work to prototype an idea yourself and prove it on real data. Key job responsibilities Own the scientific strategy for how organizational knowledge is represented, kept current, and retrieved: extraction, entity resolution, deduplication, graph structure, and retrieval that unifies graph, semantic, keyword, and temporal search. Advance temporal reasoning. Knowledge changes: facts are revised, decisions are reversed, priorities move. Representing what superseded what and when, and preserving the provenance to distinguish confirmed information from inferred information, is among the hardest open problems in this space. Define the science of proactive behavior. When is it right for an AI system to interrupt a human? These are precision-critical problems where a false positive costs far more than a miss, and where the right threshold varies by team and by individual. Lead our measurement science. Build evaluation for completeness and correctness across a multi-component agentic system, converging on a small number of trustworthy primary metrics rather than a sprawl of component scores. Judge honestly when an offline gain is real and when it is an artifact of a sparse dataset. Build the data that doesn't exist. The most valuable phenomena in this domain are also the rarest, which makes naturally occurring examples too scarce to learn from. Design synthetic and simulated data pipelines that generate controlled, realistic scenarios so these capabilities can be developed and tested at all. Own the learning loop. Turn human interaction into usable training signal, and set the direction for how the system improves from explicit feedback in the near term and from passive observation over the longer term. Make the efficiency calls. Decide where frontier models are required and where a smaller domain-tuned model is sufficient, and build the cost and capacity measurement that makes it a data-driven decision rather than an opinion. Raise the bar across the team. Mentor scientists, review designs, publish where the work merits it, and represent the science externally to customers and to the research community. A day in the life You might spend the morning in a design review arguing that a proposed approach won't survive contact with real data, the afternoon writing a prototype yourself to demonstrate the alternative, and the end of the day convincing an engineer that the capability is worth a sprint. Our sequencing is deliberate: try the idea on intuition, validate it on real data by inspection, then measure it, then operationalize it. Scientists here are expected to identify a problem, justify it, recruit others to it, and drive it into production, across whatever parts of the system that requires. Ownership follows the problem, not the org chart. About the team We are a combined science, product, and engineering team building one product together. Scientists own capabilities end to end rather than individual components, because these problems don't decompose cleanly: a single improvement typically touches extraction, storage, and retrieval at once. We invest in the tooling that makes that practical: local full-stack environments and sandboxed realistic data, so a scientist can go from idea to result in seconds rather than waiting on a deployment or on engineering support. The work is grounded in real usage rather than benchmarks alone, which is a rare combination for science this early: real users, real data, real feedback, and a genuinely unsolved research agenda.
2027 Applied Science Internship - Automated Reasoning - United States, PhD Student Science Recruiting, Automated Reasoning
US, WA, Seattle
This role sits within Amazon's Automated Reasoning and Formal Verification research horizon. Shape the Future of Cloud Computing. Are you a graduate student passionate about Automated Reasoning and its real-world applications? Join our team of innovators and embark on a journey to revolutionize cloud computing through innovative automated reasoning techniques. Our tools are called billions of times daily, powering the backbone of Amazon's products and services. We are changing the way computer systems are developed and operated, raising the bar for security, durability, availability, and quality. Applied Scientists in Automated Reasoning develop and apply formal methods, automated reasoning techniques, and neurosymbolic approaches to ensure the security, reliability, and correctness of Amazon and AWS services and customer applications. Application areas span cloud infrastructure verification, cryptographic assurance, AI safety, and formal guarantees for generative AI systems. Methods range from interactive theorem proving and constraint solving to neuro-inspired proof search. As an Applied Science Intern, you will have the opportunity to work alongside our scientists and contribute to projects. From distributed proof search and SAT/SMT solvers to program analysis, synthesis, and verification, you will tackle complex challenges at the intersection of theory and practice. Amazon has positions available for Automated Reasoning Applied Science Internships in, but not limited to, Arlington, VA; Boston, MA; New York, NY; Portland, OR; Santa Clara, CA; Seattle, WA; Austin, TX; Cambridge, UK. Key job responsibilities We are particularly interested in candidates with expertise in: Theorem Proving, Boolean Satisfiability Solvers, Bounded Model Checking, Deductive Verification, Programming/Scripting Languages, Abstract Interpretation, Automated Reasoning, Static/Program Analysis, Program Synthesis. Contribute to the design and implementation of algorithms and formal methods for automated reasoning, including constraint solving, model checking, static analysis, theorem proving, and program synthesis, within a guided research framework. Explore and apply generative AI and machine learning techniques to enhance automated reasoning, including learning-based heuristics for search, neural approaches to symbolic reasoning, and methods for verifying the correctness of AI-generated code. Contribute to automated reasoning techniques for generative AI and agentic coding systems, including methods that apply formal guarantees to large language model outputs. Contribute to the scientific community through publications at peer-reviewed conferences and journals. Leverage AI-powered tools where applicable to accelerate research, experimentation, and prototyping. Critically review and validate outputs from AI tools and automated systems. The ideal intern must have the ability to communicate research findings clearly to diverse audiences.
Applied Scientist (Speech AI), Content Reasoning, Enrichment and Localization
US, WA, Seattle
Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. Prime members can customize their viewing experience and find their favorite movies, series, documentaries, and live sports – including Amazon MGM Studios-produced series and movies; licensed fan favorites; and programming from Prime Video subscriptions such as Apple TV+, HBO Max, Peacock, Crunchyroll and MGM+. All customers, regardless of whether they have a Prime membership or not, can rent or buy titles via the Prime Video Store, and can enjoy even more content for free with ads. Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. As a Prime Video team member, you’ll have end-to-end ownership of the product, user experience, design, and technology required to deliver state-of-the-art experiences for our customers. You’ll get to work on projects that are fast-paced, challenging, and varied. You’ll also be able to experiment with new possibilities, take risks, and collaborate with remarkable people. We’ll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented technologists, you can decide where a career Prime Video Tech takes you! We are looking for passionate, hard-working, and talented individuals to help us push the envelope of content localization. We are seeking scientists with experience in audio processing, speech/voice AI and machine learning. We work on a broad array of research areas and applications, including but not limited to multimodal machine translation, speech synthesis, speech analysis, and asset quality assessment. Candidates should be prepared to help drive innovation in one or more areas of machine learning, audio processing, and natural language understanding. If you have experience with speech synthesis and foundational models, then that's a huge plus! Key job responsibilities As an Applied Scientist, you should be a strong communicator, able to describe scientifically rigorous work to business stakeholders of varying levels of technical sophistication. You will closely partner with the solution development teams, and should be intensely curious about how the research is moving the needle for business. Strong inter-personal and mentoring skills to develop applied science talent in the team is another important requirement. - Lead research and development of speech and audio generation technology and end-to-end speech-to-speech architecture - Develop audio processing solutions for production environments, including source separation, enhancement, and mixing - Define the research roadmap for your area, identify high-impact problems, and communicate technical direction to senior leadership - Publish research, contribute to the broader scientific community, and bring external advances into production systems A day in the life You might start your morning reviewing experimental results and refining a model architecture before syncing with your engineering partners on integration plans. After lunch, you could be whiteboarding a new approach to a problem your team recently identified, then writing up findings for an internal science review. You will regularly present your work to peers and stakeholders, participate in code and design reviews, and explore emerging research that could unlock new possibilities for your team. About the team Our team is driven by a shared commitment to applying science in ways that create meaningful impact for customers. We value rigorous research, collaborative problem-solving, and a willingness to experiment with new ideas. You will work alongside talented scientists and engineers in an inclusive environment where your contributions shape the direction of our work. We are focused on building solutions that matter at scale, and we are looking for teammates who are energized by that challenge.
Sr. Applied Scientist, Pricing Science
US, WA, Seattle
We are looking for a talented, organized, and customer-focused applied researcher to join our Pricing Optimization science group, with a charter to measure, refine, and launch customer-obsessed improvements to our algorithmic pricing and promotion models across all products listed on Amazon. This role requires an individual with exceptional machine learning modeling and architecture expertise — particularly in deep learning, neural networks, and transformer-based architectures applied to price prediction and forecasting problems. Equally important is deep expertise in causal machine learning — including causal inference, treatment-effect estimation, and experimentation methods (e.g., uplift modeling, double/debiased machine learning, instrumental variables, and A/B and quasi-experimental design) — to isolate the true impact of pricing and promotion decisions on customer behavior and business outcomes. The ideal candidate brings a strong foundation in applied statistics and probabilistic modeling, excellent cross-functional collaboration skills, business acumen, and an entrepreneurial spirit. We are looking for an experienced innovator who is a self-starter, comfortable with ambiguity, demonstrates strong attention to detail, and has the ability to work in a fast-paced and ever-changing environment. Key job responsibilities See the big picture. Understand and influence the long-term vision for Amazon's science-based competitive, perception-preserving pricing techniques. Develop and advance price prediction models leveraging deep learning frameworks, transformer architectures, and advanced statistical methods to drive pricing accuracy at scale. Build strong collaborations. Partner with product, engineering, and science teams within Pricing & Promotions to deploy machine learning price estimation and error correction solutions at Amazon scale. Design and implement neural network-based architectures — including sequence models and transformers — for large-scale price prediction and optimization. Stay informed. Establish mechanisms to stay up to date on the latest scientific advancements in deep learning, transformer architectures, applied statistics, neural network design, probabilistic forecasting, and multi-objective optimization techniques. Identify opportunities to apply them to relevant Pricing & Promotions business problems. Keep innovating for our customers. Foster an environment that promotes rapid experimentation, continuous learning, and incremental value delivery. Leverage statistical rigor and modern deep learning approaches to validate hypotheses and drive measurable pricing improvements. Successfully execute & deliver. Apply your exceptional technical machine learning expertise — including deep neural networks, attention-based models, and applied statistical analysis — to incrementally move the needle on some of our hardest pricing problems. A day in the life We are hiring a Sr. Applied Scientist to drive our pricing optimization initiatives. We drive cross-domain and cross-system improvements through: * shape and extend our RL optimization platform - a pricing centric tool that automates the optimization of various system parameters and price inputs. * Error detection and price quality guardrails at scale. * Identifying opportunities to optimally price across systems and contexts (marketplaces, request types, event periods) Price is a highly relevant input into Stores architectures; this role creates the opportunity to drive extremely large impact (measured in Bs not Ms), but demands careful thought and clear communication. About the team The Pricing Optimization science group builds and refines Amazon's algorithmic pricing and promotion models at scale. Our team combines expertise in deep learning, transformer architectures, applied statistics, and probabilistic forecasting to develop price prediction systems that directly impact the customer experience. The team also brings hands-on experience with causal modeling and inference — including uplift modeling and treatment effect estimation — to rigorously measure the impact of pricing decisions on customer behavior and business outcomes. We partner closely with product, engineering, and business teams to take solutions from research through production deployment.
Senior Applied Scientist, Fauna
US, NY, New York
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers About the team Fauna Robotics, an Amazon company, is building capable, safe, and genuinely delightful robots for everyday life. Our goal is simple: make robots people actually want to live and interact with in everyday human spaces. We believe that future won’t arrive until building for robotics becomes far more accessible. Today, too much effort is spent reinventing the fundamentals. We’re changing that by developing tightly integrated hardware and software systems that make it faster, safer, and more intuitive to create real-world robotic products. Our work spans the full stack: mechanical design, control systems, dynamic modeling, and intelligent software. The focus is not just functionality, but experience. We’re building robots that feel responsive, expressive, and genuinely useful. At Fauna, you’ll work at the frontier of this space, helping define how robots move, manipulate, and interact with people in natural environments. It’s an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you.
Sr. Applied Scientist, Last Mile Science
IN, KA, Bangalore
Have you ever ordered a product on Amazon and when that box with the smile arrived you wondered how it got to you so fast? Have you wondered where it came from and how much it cost Amazon to deliver it to you? If so, the WW Amazon Logistics, Business Analytics team is for you. We manage the delivery of tens of millions of products every week to Amazon’s customers, achieving on-time delivery in a cost-effective manner. We are looking for an enthusiastic, customer obsessed, Sr. Applied Scientist with good analytical skills to help manage projects and operations, implement scheduling solutions, improve metrics, and develop scalable processes and tools. The primary role of an Operations Research Scientist within Amazon is to address business challenges through building a compelling case, and using data to influence change across the organization. This individual will be given responsibility on their first day to own those business challenges and the autonomy to think strategically and make data driven decisions. Decisions and tools made in this role will have significant impact to the customer experience, as it will have a major impact on how the final phase of delivery is done at Amazon. Ideal candidates will be a high potential, strategic and analytic graduate with a PhD in (Operations Research, Statistics, Engineering, and Supply Chain) ready for challenging opportunities in the core of our world class operations space. Great candidates have a history of operations research, and the ability to use data and research to make changes. This role requires robust program management skills and research science skills in order to act on research outcomes. This individual will need to be able to work with a team, but also be comfortable making decisions independently, in what is often times an ambiguous environment. Responsibilities may include: - Develop input and assumptions based preexisting models to estimate the costs and savings opportunities associated with varying levels of network growth and operations - Creating metrics to measure business performance, identify root causes and trends, and prescribe action plans - Managing multiple projects simultaneously - Working with technology teams and product managers to develop new tools and systems to support the growth of the business - Communicating with and supporting various internal stakeholders and external audiences
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social