2020 Amazon Research Awards recipients announced

ARA funds nearly twice as many awards as in previous year; 100 award recipients represent 59 universities in 13 countries.

In March 2021, Amazon notified applicants that they were recipients of the 2020 Amazon Research Awards, a program that provides unrestricted funds and AWS Promotional Credits to academic researchers investigating research topics across a number of disciplines.

Today, we’re publicly announcing the 100 award recipients who represent 59 universities in 13 countries. This round, ARA received a record number of submissions and funded nearly twice as many awards as the previous year. Each award is intended to support the work of one to two graduate students or postdoctoral students for one year, under the supervision of a faculty member.

ARA is funding awards under five call for proposals: AI for Information Security, Alexa Fairness in AI, AWS AI, AWS Automated Reasoning, and Robotics. Proposals were reviewed for the quality of their scientific content, their creativity, and their potential to impact both the research community, and society more generally. Theoretical advances, creative new ideas, and practical applications were all considered.

Recipients have access to more than 200 Amazon public datasets, and can utilize AWS AI/ML services and tools through their AWS Promotional Credits. Recipients also are assigned an Amazon research contact who offers consultation and advice along with opportunities to participate in Amazon events and training sessions.

Additionally, Amazon encourages the publication of research results, presentations of research at Amazon offices worldwide, and the release of related code under open-source licenses.

“The 2020 Amazon Research Awards recipients represent a distinguished array of academic researchers who are pursuing research across areas such as ML algorithms and theory, fairness in AI, computer vision, natural language processing, edge computing, and medical research,” said Bratin Saha, vice president of AWS Machine Learning Services. “We are excited by the depth and breadth of their proposals, as well as the opportunity to advance the science through strengthened connections among academic researchers, their institutions, and our research teams.”

“As we enter into this golden age of robotics, we do so with our university partners. Not only are they shaping what is possible in robotics, they are inspiring many next- generation roboticists with their incredible creations and front-line teachings,” said Tye Brady, chief technologist for Amazon Robotics. “Our grant recipients are not only pursuing cutting-edge research that will benefit society, but perhaps more importantly are helping students from across the globe pursue a career in science and engineering.”

ARA funds proposals up to four times a year in a variety of research areas. Applicants are encouraged to visit the ARA call for proposals page for more information or send an email to be notified of future open calls.

Below is the list of 2020 award recipients, presented in alphabetical order.

Recipient

University

Research title

Vikram Adve

University of Illinois Urbana-Champaign

Extending the LLVM compiler infrastructure for tensor architectures

Pulkit Agrawal

Massachusetts Institute of Technology

A framework for multi-step planning for manipulating rigid objects

Ron Alterovitz

University of North Carolina at Chapel Hill

Cloud-based motion planning: an enabling technology for next-generation autonomous robots

Jimmy Ba

University of Toronto

Model-based reinforcement learning with causal world models

Saurabh Bagchi

Purdue University—West Lafayette

Content and contention-aware approximate streaming video analytics for edge devices

David Baker Effendi

Stellenbosch University

Dataflow analysis using code property graphs, graph databases and synchronized pushdown systems

Sivaraman Balakrishnan

Carnegie Mellon University

Foundations of robust machine learning: from principled approaches to practice

Elias Bareinboim

Columbia University

Off-policy evaluation through causal modeling

Clark Barrett

Stanford University

Model-based testing of SMT solvers

Lars Birkedal

Aarhus University

Modular reasoning about distributed systems: higher-order distributed separation logic

David Blei

Columbia University

New directions in observational causal inference

Eric Bodden

Paderborn University

HybridCG — dynamically-enriched call-Graph generation of Java enterprise applications

Legand Burge

Howard University

Voice-FAQ: artificial intelligence for triaging cognitive decline through modeling vocal prosody and facial expressions

James Caverlee

Texas A&M University, College Station

Fairness in recommendation without demographics

Changyou Chen

University at Buffalo

Scaling up human-action analysis systems

Danqi Chen

Princeton University

Building broad-coverage, structured dense knowledge bases for natural language processing tasks

Helen Chen

University of Waterloo

Optimizing pretrained clinical embeddings for automatic COVID-related ICD coding

Yiran Chen

Duke University

Privacy-preserving representation learning on graphs — a mutual information perspective

Margarita Chli

ETH Zurich

Vision-based emergency landing in urban environments using reinforcement learning and deep learning

Kyunghyun Cho

New York University

Independently controllable attributes for controllable neural text generation

Carlo Ciliberto

University College London

Optimal transport for meta-learning

Loris D’Antoni

University of Wisconsin–Madison

Correct-by-construction IAM policies

David Danks

Carnegie Mellon University

An integrated framework for understanding human-AI hybrid decision-making

Suhas Diggavi

University of California, Los Angeles

Compressed private and secure distributed edge learning

Greg Durrett

University of Texas At Austin

Making conditional text generation fair and factual

Sergio Escalera

Universitat de Barcelona and Computer Vision Center

Portable virtual try-on for smart devices

Jan Faigl

Czech Technical University in Prague

Communication maps building in subterranean environments

Pietro Ferrara

Ca’ Foscari University of Venice

IAM access control policies verification and inference

Katerina Fragkiadaki

Carnegie Mellon University

Generalizing manipulation across objects, configurations and views using a visually-grounded library of behaviors

Guillermo Gallego

Technical University of Berlin

Online in-hand object tracking and grasp failure detection with an event-based camera

Grace Gao

Stanford University

Trustworthy autonomous vehicle localization using a joint model-driven and data-driven approach

Stephanie Gil

Harvard University

Enabling the next generation of coordinated robots: scalable real-time decision making

Luca Giuggioli

University of Bristol

Multi-robot online exploration in extreme unbounded environments through adaptive socio-spatial ordering

Jorge Goncalves

University of Melbourne

Integrated qualification test framework to measure crowd worker quality and assign or recommend heterogeneous tasks

Ananth Grama

Purdue University—West Lafayette

Scaling causal inference to explainable clinical recommendations

Grace Gu

University of California, Berkeley

Surrogate machine learning model and quasi-static simulation of pneumatically actuated robotic devices

Ronghui Gu

Columbia University

Microverification of the Linux KVM hypervisor: proving VM confidentiality and integrity

Aarti Gupta

Princeton University

Learning abstract specifications from distributed program implementations

Saurabh Gupta

University of Illinois Urbana-Champaign

Self-supervised discovery of object states and transitions from unlabeled videos

Daniel Harabor

Monash University

Anytime constraint-based multi-agent pathfinding

Hynek Hermansky

Johns Hopkins University

Multistream lifelong federated learning for machine recognition of speech

Bin Hu

University of Illinois Urbana-Champaign

Provably robust adversarial reinforcement learning for sequential decision making in safety-critical environments

Lifu Huang

Virginia Tech

Event-centric temporal and causal knowledge acquisition and generalization for natural language understanding

Dinesh Jayaraman

University of Pennsylvania

Learning modular dynamics models for plug-and-play visual control

Sven Koenig

University of Southern California

Improving planning and plan execution for warehouse automation

Laura Kovacs

TU Wien

FOREST: first-order reasoning for ensuring system security

Arun Kumar

University of California, San Diego

Improving automated feature type inference for AutoML on tabular data

Himabindu Lakkaraju

Harvard University

Towards reliable and robust model explanations

Kevin Leyton-Brown

University of British Columbia

Automated machine learning for tabular datasets using hyperband embedded reinforcement learning

Bo Li

University of Illinois Urbana-Champaign

Machine learning evaluation as a service for robustness, fairness, and privacy utilities

Ke Li

University of Exeter

Many hands make work light: multi-task deep semantic learning for testing web application firewalls

Zhiqiang Lin

Ohio State University

Type-aware recovery of symbol names in binary code: a machine learning based approach

Jeffrey Liu

Massachusetts Institute of Technology

Integrating the low altitude disaster imagery (LADI) dataset into the MIT Beaver Works curriculum

Michael Mahoney

University of California, Berkeley

Systematic methods for efficient inference and training of neural networks

Radu Marculescu

University of Texas

New directions for 3D object detection: distributed inference on edge devices using knowledge distillation

Ruben Martins

Carnegie Mellon University

Improving performance and trust of MaxSAT solvers

Jiri Matas

Czech Technical University in Prague

Training neural networks on non-differentiable losses

Michael Milford

Queensland University of Technology

Complementarity-aware multi-process fusion for long term localization

Heather Miller

Carnegie Mellon University

Directed automated explicit-state model checking for distributed applications

Ndapa Nakashole

University of California, San Diego

Learning representations for voice-based conversational agents for older adults

Shrikanth Narayanan

University of Southern California

Toward inclusive human-AI conversational experiences for children

Lerrel Pinto

New York University

Learning to manipulate deformable objects through robust simulations

Ravi Ramamoorthi

University of California, San Diego

Sparse multi-view object acquisition using learned volumetric representations

Philip Resnik

University of Maryland, College Park

Advanced topic modeling to support the understanding of COVID-19 and its effects

Daniela Rus

Massachusetts Institute of Technology

Learning to plan through imagined self-play for multi-agent system

Supreeth Shashikumar

University of California, San Diego

Privacy preserving continual learning with applications to critical care

Robert Shepherd

Cornell University

Enduring and adaptive robots via electrochemical blood

Cong Shi

University of Michigan, Ann Arbor

Machine learning for personalized assortment optimization

Florian Shkurti

University of Toronto

Generating physically realizable adversarial driving scenarios via differentiable physics and rendering simulators

Abhinav Shrivastava

University of Maryland, College Park

The pursuit of knowledge: discovering and localizing new concepts using dual memory

Roland Siegwart

ETH Zurich

Safe self-calibration of hybrid aerial vehicles

Sameer Singh

University of California, Irvine

Detecting and fixing vulnerabilities in NLP models via semantic perturbations and tracing data influence

Noah Smith

University of Washington - Seattle

Language model customization

Mahdi Soltanolkotabi

University of Southern California

Artificial intelligence for fast and portable medical imaging (with limited training data)

Seung Woo Son

University of Massachusetts Lowell

Reliable and accurate anomaly detection in edge nodes using sparsity profile

Dawn Song

University of California, Berkeley

Knowledge-enhanced cyber threat hunting

Dezhen Song

Texas A&M University, College Station

Optoacoustic material and structure pretouch sensing at robot fingertip

Shuran Song

Columbia University

Dexterity through diversity:learning a generalizable grasping policy for diverse end-effectors

Yizhou Sun

University of California, Los Angeles

Accelerating graph neural network training

Russ Tedrake

Massachusetts Institute of Technology

Intuitive physics for manipulation

James Tompkin

Brown University

Real-time multi-camera fusion for unoccluded VR robot teleoperation

Emina Torlak

University of Washington - Seattle

Automated verification of JIT compilers for BPF

Marynel Vazquez

Yale University

Evaluating social robot navigation via online human-driven simulations

Nisheeth Vishnoi

Yale University

Fair and error-resilient algorithms for AI and ML

Gang Wang

University of Illinois at Urbana–Champaign

Combating concept drift in security applications via proactive data synthesis

Hao Wang

Rutgers University-New Brunswick

Structured domain adaptation with applications to personalization and forecasting

James Wang

Pennsylvania State University

Affective and social interaction between human and intelligent machine

Gloria Washington

Howard University

Towards identification of uncomfortable speech in conversations

Chuan Wu

The University of Hong Kong

Compilation optimization in distributed DNN training: joining OP and tensor fusion/partition

Eugene Wu

Columbia University

Human-in-the-loop data debugging for ML-oriented analytics

Jiajun Wu

Stanford University

Implicit dynamic scene representation learning for robotics

Ming-Ru Wu

Dana-Farber Cancer Institute

From bench to clinic – machine-learning based cancer immunotherapy design

Diyi Yang

Georgia Institute of Technology

Abstractive conversation summarization at scale

Sixian You

Massachusetts Institute of Technology

AI-driven label-free histology for cancer diagnosis

Jingjin Yu

Rutgers University-New Brunswick

Pushing the limits of efficient and optimal multi-agent path finding through exploring space utilization optimization and adaptive planning horizon heuristics

Rui Zhang

Pennsylvania State University

Building robust conversational question answering systems over databases of tabular data

Yu Zhang

University of South Florida

Design of an automated advanced air mobility flight planning system (AAFPS)

Yuke Zhu

University of Texas at Austin

Learning implicit shape affordance for grasping and manipulation

Marinka Zitnik

Harvard University

Actionable graph learning for finding cures for emerging diseases

James Zou

Stanford University

How to make AI forget you? Efficiently removing individuals’ data from machine learning models

Related content

ES, B, Barcelona
Are you interested in defining the science strategy that enables Amazon to market to millions of customers based on their lifecycle needs rather than one-size-fits-all campaigns? We are seeking a Applied Scientist to lead the science strategy for our Lifecycle Marketing Experimentation roadmap within the PRIMAS (Prime & Marketing analytics and science) team. The position is open to candidates in Amsterdam and Barcelona. In this role, you will own the end-to-end science approach that enables EU marketing to shift from broad, generic campaigns to targeted, cohort-based marketing that changes customer behavior. This is a high-ambiguity, high-impact role where you will define what problems are worth solving, build the science foundation from scratch, and influence senior business leaders on marketing strategy. You will work directly with Business Directors and channel leaders to solve critical business problems: how do we win back customers lost to competitors, convert Young Adults to Prime, and optimize marketing spend by de-averaging across customer cohorts. Key job responsibilities Science Strategy & Leadership: 1. Own the end-to-end science strategy for lifecycle marketing, defining the roadmap across audience targeting, behavioral modeling, and measurement 2. Navigate high ambiguity in defining customer journey frameworks and behavioral models – our most challenging science problem with no established playbook 3. Lead strategic discussions with business leaders translating business needs into science solutions and building trust across business and tech partners 4. Mentor and guide a team of 2-3 scientists and BIEs on technical execution while contributing hands-on to the hardest problems Advanced Customer Behavior Modeling: 1. Build sophisticated propensity models identifying customer cohorts based on lifecycle stage and complex behavioral patterns (e.g., Bargain hunters, Young adults Prime prospects) 2. Define customer journey frameworks using advanced techniques (Hidden Markov Models, sequential decision-making) to model how customers transition across lifecycle stages 3. Identify which customer behaviors and triggers drive lifecycle progression and what messaging/levers are most effective for each cohort 4. Integrate 1P behavioral data with 2P survey insights to create rich, actionable audience definitions Measurement & Cross-Workstream Integration: 1. Partner with measurement scientist to design experiments (RCTs) that isolate audience targeting effects from creative effects 2. Ensure audience definitions, journey models, and measurement frameworks work coherently across Meta, LiveRamp, and owned channels 3. Establish feedback loops connecting measurement insights back to model improvements About the team The PRIMAS (Prime & Marketing Analytics and Science) is the team that support the science & analytics needs of the EU Prime and Marketing organization, an org that supports the Prime and Marketing programs in European marketplaces and comprises 250-300 employees. The PRIMAS team, is part of a larger tech tech team of 100+ people called WIMSI (WW Integrated Marketing Systems and Intelligence). WIMSI core mission is to accelerate marketing technology capabilities that enable de-averaged customer experiences across the marketing funnel: awareness, consideration, and conversion.
IN, KA, Bengaluru
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you have an entrepreneurial spirit, know how to deliver, love to work with data, are deeply technical, highly innovative and long for the opportunity to build solutions to challenging problems that directly impact the company's bottom-line, we want to talk to you. Major responsibilities - Use machine learning and analytical techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes - Design, development, evaluate and deploy innovative and highly scalable models for predictive learning - Research and implement novel machine learning and statistical approaches - Work closely with software engineering teams to drive real-time model implementations and new feature creations - Work closely with business owners and operations staff to optimize various business operations - Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation - Mentor other scientists and engineers in the use of ML techniques
ES, M, Madrid
At Amazon, we are committed to being the Earth's most customer-centric company. The European International Technology group (EU INTech) owns the enhancement and delivery of Amazon's engineering to all the varied customers and cultures of the world. We do this through a combination of partnerships with other Amazon technical teams and our own innovative new projects. You will be joining the Tamale team to work on Haul. As part of EU INTech and Haul, Tamale strives to create a discovery-driven shopping experience using challenging machine learning and ranking solutions. You will be exposed to large-scale recommendation systems, multi-objective optimization, and state-of-the-art deep learning architectures, and you'll be part of a key effort to improve our customers' browsing experience by building next-generation ranking models for Amazon Haul's endless scroll experience. We are looking for a passionate, talented, and inventive Scientist with a strong machine learning background to help build industry-leading ranking solutions. We strongly value your hard work and obsession to solve complex problems on behalf of Amazon customers. Key job responsibilities We look for applied scientists who possess a wide variety of skills. As the successful applicant for this role, you will work closely with your business partners to identify opportunities for innovation. You will apply machine learning solutions to optimize multi-objective ranking, improve discovery engagement through contextual signals, and scale ranking systems across multiple marketplaces. You will work with business leaders, scientists, and product managers to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed ranking services. You will be part of a team of scientists and engineers working on solving ranking and personalization challenges at scale. You will be able to influence the scientific roadmap of the team, setting the standards for scientific excellence. You will be working with state-of-the-art architectures and real-time feature serving systems. Your work will improve the experience of millions of daily customers using Amazon Haul worldwide. You will have the chance to have great customer impact and continue growing in one of the most innovative companies in the world. You will learn a huge amount - and have a lot of fun - in the process!
IN, HR, Gurugram
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced ML systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real-world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning team for International Emerging Stores (IES). Machine Learning, Big Data and related quantitative sciences have been strategic to Amazon from the early years. Amazon has been a pioneer in areas such as recommendation engines, ecommerce fraud detection and large-scale optimization of fulfillment center operations. As Amazon has rapidly grown and diversified, the opportunity for applying machine learning has exploded. We have a very broad collection of practical problems where machine learning systems can dramatically improve the customer experience, reduce cost, and drive speed and automation. These include product bundle recommendations for millions of products, safeguarding financial transactions across by building the risk models, improving catalog quality via extracting product attribute values from structured/unstructured data for millions of products, enhancing address quality by powering customer suggestions We are developing state-of-the-art machine learning solutions to accelerate the Amazon India growth story. Amazon is an exciting place to be at for a machine learning practitioner. We have the eagerness of a fresh startup to absorb machine learning solutions, and the scale of a mature firm to help support their development at the same time. As part of the International Machine Learning team, you will get to work alongside brilliant minds motivated to solve real-world machine learning problems that make a difference to millions of our customers. We encourage thought leadership and blue ocean thinking in ML. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes Design, develop, evaluate and deploy, innovative and highly scalable ML models Work closely with software engineering teams to drive real-time model implementations Work closely with business partners to identify problems and propose machine learning solutions Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model maintenance Work proactively with engineering teams and product managers to evangelize new algorithms and drive the implementation of large-scale complex ML models in production Leading projects and mentoring other scientists, engineers in the use of ML techniques About the team International Machine Learning Team is responsible for building novel ML solutions across International Emerging Store (India, MENA, Far-East, LatAm) problems and impact the bottom-line and top-line of India business. Learn more about our team from https://www.amazon.science/working-at-amazon/how-rajeev-rastogis-machine-learning-team-in-india-develops-innovations-for-customers-worldwide
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Bellevue
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.