Amazon’s quantum computing papers at QIP 2023

Research on “super-Grover” optimization, quantum algorithms for topological data analysis, and simulation of physical systems displays the range of Amazon’s interests in quantum computing.

At this year’s Quantum Information Processing Conference (QIP), members of Amazon Web Services' Quantum Technologies group are coauthors on three papers, which indicate the breadth of the group’s research interests.

In “Mind the gap: Achieving a super-Grover quantum speedup by jumping to the end”, Amazon research scientist Alexander Dalzell, Amazon quantum research scientist Nicola Pancotti, Earl Campbell of the University of Sheffield and Riverlane, and I present a quantum algorithm that improves on the efficiency of Grover’s algorithm, one of the few quantum algorithms to offer provable speedups relative to conventional algorithms. Although the improvement on Grover’s algorithm is small, it breaks a performance barrier that hadn’t previously been broken, and it points to a methodology that could enable still greater improvements.

Related content
As the major quantum computing conference celebrates its anniversary, we ask the conference chair and the head of Amazon’s quantum computing program to take stock.

In “A streamlined quantum algorithm for topological data analysis with exponentially fewer qubits”, Amazon research scientist Sam McArdle, Mario Berta of Aachen University, and András Gilyén of the Alfréd Rényi Institute of Mathematics in Budapest consider topological data analysis, a technique for analyzing big data. They present a new quantum algorithm for topological data analysis that, compared to the existing quantum algorithm, enables a quadratic speedup and an exponentially more efficient use of quantum memory.

For “Sparse random Hamiltonians are quantumly easy”, Chi-Fang (Anthony) Chen, a Caltech graduate student who was an Amazon intern when the work was done, won the conference's best-student-paper award. He's joined on the paper by Alex Dalzell and me, Mario Berta, and Caltech's Joel Tropp. The paper investigates the use of quantum computers to simulate physical properties of quantum systems. We prove that a particular model of physical systems — specifically, sparse, random Hamiltonians — can, with high probability, be efficiently simulated on a quantum computer.

Super-Grover quantum speedup

Grover’s algorithm is one of the few quantum algorithms that are known to provide speedups relative to classical computing. For instance, for the 3-SAT problem, which involves finding values for N variables that satisfy the constraints of an expression in formal logic, the running time of a brute-force classical algorithm is proportional to 2N; the running time of Grover’s algorithm is proportional to 2N/2.

Related content
Watch as the panel talks about everything from what got them interested in quantum research to where they see the field headed in the future.

Adiabatic quantum computing is an approach to quantum computing in which a quantum system is prepared so that, in its lowest-energy state (the “ground state”), it encodes the solution to a relatively simple problem. Then, some parameter of the system — say, the strength of a magnetic field — is gradually changed, so that the system encodes a more complex problem. If the system stays in its ground state through those changes, it will end up encoding the solution to the complex problem.

As the parameter is changed, however, the gaps between the system’s ground state and its first excited states vary, sometimes becoming infinitesimally small. If the parameter changes too quickly, the system may leap into one of its excited states, ruining the computation.

Hamiltonian energies.jpg
In adiabatic quantum computing, as the parameters (b) of a quantum system change, the gap between the system’s ground energy and its first excited state may vary.

In “Mind the gap: Achieving a super-Grover quantum speedup by jumping to the end”, we show that for an important class of optimization problems, it’s possible to compute an initial jump in the parameter setting that runs no risk of kicking the system into a higher energy state. Then, a second jump takes the parameter all the way to its maximum value.

Most of the time this will fail, but every once in a while, it will work: the system will stay in its ground state, solving the problem. The larger the initial jump, the greater the increase in success rate.

Super-Grover leap.gif
An initial, risk-free jump in the quantum system’s parameter setting (b) decreases the chances that jumping to the final setting will kick the system into an excited energy state.

Our paper proves that the algorithm has an infinitesimal but quantifiable advantage over Grover’s algorithm, and it reports a set of numerical experiments to determine the practicality of the approach. Those experiments suggest that the method, in fact, increases efficiency more than could be mathematically proven, although still too little to yield large practical benefits. The hope is that the method may lead to further improvements that could make a practical difference to quantum computers of the future.

Topological data analysis

Topology is a branch of mathematics that treats geometry at a high level of abstraction: on a topological description, any two objects with the same number of holes in them (say, a coffee cup and a donut) are identical.

Related content
New phase estimation technique reduces qubit count, while learning framework enables characterization of noisy quantum systems.

Mapping big data to a topological object — or manifold — can enable analyses that are difficult at lower levels of abstraction. Because topological descriptions are invariant to shape transformations, for instance, they are robust against noise in the data.

Topological data analysis often involves the computation of persistent Betti numbers, which characterize the number of holes in the manifold, a property that can carry important implications about the underlying data. In “A streamlined quantum algorithm for topological data analysis with exponentially fewer qubits”, the authors propose a new quantum algorithm for computing persistent Betti numbers. It offers a quadratic speedup relative to classical algorithms and uses quantum memory exponentially more efficiently than existing quantum algorithms.

Topological mapping.png
Connecting points in a data cloud produces closed surfaces (or “simplices”, such as the triangle ABC) that can be mapped to the surface of a topological object, such as a toroid (donut shape).

Data can be represented as points in a multidimensional space, and topological mapping can be thought of as drawing line segments between points in order to produce a surface, much the way animators create mesh outlines of 3-D objects. The maximum length of the lines defines the length scale of the mapping.

At short enough length scales, the data would be mapped to a large number of triangles, tetrahedra, and their higher-dimensional analogues, which are known as simplices. As the length scale increases, simplices link up to form larger complexes, and holes in the resulting manifold gradually disappear. The persistent Betti number is the number of holes that persist across a range of longer length scales.

Related content
Researchers affiliated with Amazon Web Services' Center for Quantum Computing are presenting their work this week at the Conference on Quantum Information Processing.

The researchers’ chief insight is, though the dimension of the representational space may be high, in most practical cases, the dimension of the holes is much lower. The researchers define a set of boundary operators, which find the boundaries (e.g., the surfaces of 3-D shapes) of complexes (combinations of simplices) in the representational space. In turn, the boundary operators (or more precisely, their eigenvectors) provide a new geometric description of the space, in which regions of the space are classified as holes or not-holes.

Since the holes are typically low dimensional, so is the space, which enables the researchers to introduce an exponentially more compact mapping of simplices to qubits, dramatically reducing the spatial resources required for the algorithm.

Sparse random Hamiltonians

The range of problems on which quantum computing might enable useful speedups, compared to classical computing, is still unclear. But one area where quantum computing is likely to offer advantages is in the simulation of quantum systems, such as molecules. Such simulations could yield insights in biochemistry and materials science, among other things.

Related content
New approach reduces the number of ancillary qubits required to implement the crucial T gate by at least an order of magnitude.

Often, in quantum simulation, we're interested in quantum systems' low-energy properties. But in general, it’s difficult to prove that a given quantum algorithm can prepare a quantum system in a low-energy state.

The energy of a quantum system is defined by its Hamiltonian, which can be represented as a matrix. In “Sparse random Hamiltonians are quantumly easy”, we show that for almost any Hamiltonian matrix that is sparse — meaning it has few nonzero entries — and random — meaning the locations of the nonzero entries are randomly assigned — it is possible to prepare a low-energy state.

Moreover, we show that the way to prepare such a state is simply to initialize the quantum memory that stores the model to a random state (known as preparing a maximally mixed state).

Semicircular distribution.png
The semicircular distribution of eigenvalues for a particular quantum system, the Pauli string ensemble.

The key to our proof is to generalize a well-known result for dense matrices — Wigner's semicircle distribution for Gaussian unitary ensembles (GUEs) — to sparse matrices. Computing the energy level of a quantum system from its Hamiltonian involves calculating the eigenvalues of the Hamiltonian matrix, a standard operation in linear algebra. Wigner showed that the eigenvalues of random dense matrices form a semicircular distribution. That is, the possible eigenvalues of random matrices don’t trail off to infinity in a long tail; instead, they have sharp demarcation points. There are no possible values above and below some clearly defined thresholds.

Related content
The noted physicist answers 3 questions about the challenges of quantum computing and why he’s excited to be part of a technology development project.

Dense Hamiltonians, however, are rare in nature. The Hamiltonians describing most of the physical systems that physicists and chemists care about are sparse. By showing that sparse Hamiltonians conform to the same semicircular distribution that dense Hamiltonians do, we prove that the number of experiments required to measure a low-energy state of a quantum simulation will not proliferate exponentially.

In the paper, we also show that any low-energy state must have non-negligible quantum circuit complexity, suggesting that it could not be computed efficiently by a classical computer — an argument for the necessity of using quantum computers to simulate quantum systems.

Research areas

Related content

ES, B, Barcelona
Are you interested in defining the science strategy that enables Amazon to market to millions of customers based on their lifecycle needs rather than one-size-fits-all campaigns? We are seeking a Applied Scientist to lead the science strategy for our Lifecycle Marketing Experimentation roadmap within the PRIMAS (Prime & Marketing analytics and science) team. The position is open to candidates in Amsterdam and Barcelona. In this role, you will own the end-to-end science approach that enables EU marketing to shift from broad, generic campaigns to targeted, cohort-based marketing that changes customer behavior. This is a high-ambiguity, high-impact role where you will define what problems are worth solving, build the science foundation from scratch, and influence senior business leaders on marketing strategy. You will work directly with Business Directors and channel leaders to solve critical business problems: how do we win back customers lost to competitors, convert Young Adults to Prime, and optimize marketing spend by de-averaging across customer cohorts. Key job responsibilities Science Strategy & Leadership: 1. Own the end-to-end science strategy for lifecycle marketing, defining the roadmap across audience targeting, behavioral modeling, and measurement 2. Navigate high ambiguity in defining customer journey frameworks and behavioral models – our most challenging science problem with no established playbook 3. Lead strategic discussions with business leaders translating business needs into science solutions and building trust across business and tech partners 4. Mentor and guide a team of 2-3 scientists and BIEs on technical execution while contributing hands-on to the hardest problems Advanced Customer Behavior Modeling: 1. Build sophisticated propensity models identifying customer cohorts based on lifecycle stage and complex behavioral patterns (e.g., Bargain hunters, Young adults Prime prospects) 2. Define customer journey frameworks using advanced techniques (Hidden Markov Models, sequential decision-making) to model how customers transition across lifecycle stages 3. Identify which customer behaviors and triggers drive lifecycle progression and what messaging/levers are most effective for each cohort 4. Integrate 1P behavioral data with 2P survey insights to create rich, actionable audience definitions Measurement & Cross-Workstream Integration: 1. Partner with measurement scientist to design experiments (RCTs) that isolate audience targeting effects from creative effects 2. Ensure audience definitions, journey models, and measurement frameworks work coherently across Meta, LiveRamp, and owned channels 3. Establish feedback loops connecting measurement insights back to model improvements About the team The PRIMAS (Prime & Marketing Analytics and Science) is the team that support the science & analytics needs of the EU Prime and Marketing organization, an org that supports the Prime and Marketing programs in European marketplaces and comprises 250-300 employees. The PRIMAS team, is part of a larger tech tech team of 100+ people called WIMSI (WW Integrated Marketing Systems and Intelligence). WIMSI core mission is to accelerate marketing technology capabilities that enable de-averaged customer experiences across the marketing funnel: awareness, consideration, and conversion.
IN, KA, Bengaluru
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you have an entrepreneurial spirit, know how to deliver, love to work with data, are deeply technical, highly innovative and long for the opportunity to build solutions to challenging problems that directly impact the company's bottom-line, we want to talk to you. Major responsibilities - Use machine learning and analytical techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes - Design, development, evaluate and deploy innovative and highly scalable models for predictive learning - Research and implement novel machine learning and statistical approaches - Work closely with software engineering teams to drive real-time model implementations and new feature creations - Work closely with business owners and operations staff to optimize various business operations - Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation - Mentor other scientists and engineers in the use of ML techniques
ES, M, Madrid
At Amazon, we are committed to being the Earth's most customer-centric company. The European International Technology group (EU INTech) owns the enhancement and delivery of Amazon's engineering to all the varied customers and cultures of the world. We do this through a combination of partnerships with other Amazon technical teams and our own innovative new projects. You will be joining the Tamale team to work on Haul. As part of EU INTech and Haul, Tamale strives to create a discovery-driven shopping experience using challenging machine learning and ranking solutions. You will be exposed to large-scale recommendation systems, multi-objective optimization, and state-of-the-art deep learning architectures, and you'll be part of a key effort to improve our customers' browsing experience by building next-generation ranking models for Amazon Haul's endless scroll experience. We are looking for a passionate, talented, and inventive Scientist with a strong machine learning background to help build industry-leading ranking solutions. We strongly value your hard work and obsession to solve complex problems on behalf of Amazon customers. Key job responsibilities We look for applied scientists who possess a wide variety of skills. As the successful applicant for this role, you will work closely with your business partners to identify opportunities for innovation. You will apply machine learning solutions to optimize multi-objective ranking, improve discovery engagement through contextual signals, and scale ranking systems across multiple marketplaces. You will work with business leaders, scientists, and product managers to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed ranking services. You will be part of a team of scientists and engineers working on solving ranking and personalization challenges at scale. You will be able to influence the scientific roadmap of the team, setting the standards for scientific excellence. You will be working with state-of-the-art architectures and real-time feature serving systems. Your work will improve the experience of millions of daily customers using Amazon Haul worldwide. You will have the chance to have great customer impact and continue growing in one of the most innovative companies in the world. You will learn a huge amount - and have a lot of fun - in the process!
IN, HR, Gurugram
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced ML systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real-world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning team for International Emerging Stores (IES). Machine Learning, Big Data and related quantitative sciences have been strategic to Amazon from the early years. Amazon has been a pioneer in areas such as recommendation engines, ecommerce fraud detection and large-scale optimization of fulfillment center operations. As Amazon has rapidly grown and diversified, the opportunity for applying machine learning has exploded. We have a very broad collection of practical problems where machine learning systems can dramatically improve the customer experience, reduce cost, and drive speed and automation. These include product bundle recommendations for millions of products, safeguarding financial transactions across by building the risk models, improving catalog quality via extracting product attribute values from structured/unstructured data for millions of products, enhancing address quality by powering customer suggestions We are developing state-of-the-art machine learning solutions to accelerate the Amazon India growth story. Amazon is an exciting place to be at for a machine learning practitioner. We have the eagerness of a fresh startup to absorb machine learning solutions, and the scale of a mature firm to help support their development at the same time. As part of the International Machine Learning team, you will get to work alongside brilliant minds motivated to solve real-world machine learning problems that make a difference to millions of our customers. We encourage thought leadership and blue ocean thinking in ML. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes Design, develop, evaluate and deploy, innovative and highly scalable ML models Work closely with software engineering teams to drive real-time model implementations Work closely with business partners to identify problems and propose machine learning solutions Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model maintenance Work proactively with engineering teams and product managers to evangelize new algorithms and drive the implementation of large-scale complex ML models in production Leading projects and mentoring other scientists, engineers in the use of ML techniques About the team International Machine Learning Team is responsible for building novel ML solutions across International Emerging Store (India, MENA, Far-East, LatAm) problems and impact the bottom-line and top-line of India business. Learn more about our team from https://www.amazon.science/working-at-amazon/how-rajeev-rastogis-machine-learning-team-in-india-develops-innovations-for-customers-worldwide
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Bellevue
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.