Amazon and UCLA announce fellowship recipients

The Amazon Fellows fulfill the Science Hub for Humanity and Artificial Intelligence's mission of researching the societal impact of artificial intelligence

The Science Hub for Humanity and Artificial Intelligence, launched in October 2021 to facilitate collaboration between academic researchers and Amazon scientists, today announced the second cohort of Amazon Fellows. The fellowships are aimed at graduate students pursuing research into artificial intelligence and its impact on society.

Related content
The UCLA Science Hub seeks to address challenges to humanity through research using artificial intelligence, bringing together academic and industry scientists.

The fellowships provide PhD students at UCLA Samueli School of Engineering with up to two quarters of funding during the academic year to pursue independent research projects. The Amazon Fellows study within the departments of computer science, electrical and computer engineering, bioengineering, and mechanical and aerospace engineering. In addition to project funding, they will be invited to apply to intern at Amazon.

Top row, left to right, Sanae Amani Geshnigani, Kewei Cheng, Zi-Yi Dou, Kai Fukami, and Luzhe Huang; second row, left to right, Alexander Johnson, Tung Nguyen, Alexander Schperberg, and Zhouxing Shi; and bottom row, left to right, Zhaoqiang Wang, Yu Yang, Da Yin, and Zhe Zeng. The UCLA logo is on the bottom right.
The Science Hub for Humanity and Artificial Intelligence's second cohort of Amazon Fellows are: top row, left to right, Sanae Amani Geshnigani, Kewei Cheng, Zi-Yi Dou, Kai Fukami, and Luzhe Huang; second row, left to right, Alexander Johnson, Tung Nguyen, Alexander Schperberg, and Zhouxing Shi; and bottom row, left to right, Zhaoqiang Wang, Yu Yang, Da Yin, and Zhe Zeng.

What follows is the list of fellows, their areas of research, and their UCLA faculty advisors:

Sanae Amani Geshnigani is pursuing a PhD in electrical and computer engineering; her advisor is Lin Yang, assistant professor of electrical and computer engineering.

“My research goal is to expand the applicability of bandit and reinforcement learning algorithms to new application domains: specifically, safety-critical and distributed physical systems, such as robotics, wireless networks, the power grid and medical trials.”

Kewei Cheng, is pursuing a PhD in computer science; her advisor is Yizhou Sun, professor of computer science.

“My research interests mainly focus on knowledge graph reasoning with a specific concentration on neural-symbolic reasoning, and more generally in machine learning and network science.”

Zi-Yi Dou is pursuing a PhD in computer science; his advisor is Nanyun Peng, assistant professor of computer science.

“My research has been centered around advancing the field of artificial intelligence with an aim of helping people around the globe by allowing computers to interact with them through natural language and help them accomplish tasks. State-of-the-art models still struggle with gathering information from diverse modalities and languages, and generalizing well to novel scenarios. To overcome these limitations, my current research goal is to build robust multimodal and multilingual AI models and comprehensively evaluate them along multiple dimensions and domains.”

Related content
Models that map spoken language to objects in an image would make it easier for customers to communicate with multimodal devices.

Kai Fukami is pursuing a PhD in mechanical and aerospace engineering; his advisor is Kunihiko Taira, professor, computer science.

“My academic interest belongs to fluid dynamics which is a discipline to study flows around us such as air and water. In particular, I am working on the design of artificial-intelligent techniques and machine-learning methods to understand and control turbulent flows from limited sensor measurements.”

Luzhe Huang is pursuing a PhD in electrical and computer engineering; his advisor is Aydogan Ozcan, Chancellor's Professor and the Volgenau Chair for Engineering Innovation.

“In the past decade, AI has revolutionized many fields, including robotics, computer vision, and natural language processing, and greatly improved our daily life. When it comes to microscopy imaging, despite some researches exploring the integration of AI and microscopy imaging, critical challenges remain for real-world applications and prevent the advance of AI to benefit a broad group of users in biology, pathology and medical science. I am fortunate to be studying on this frontier of human’s knowledge and develop technologies to conquer these challenges using my interdisciplinary knowledge in both AI and optics.”

Alexander Johnson, is pursuing a PhD in electrical and computer engineering; his advisor is Abeer Alwan, professor of electrical and computer engineering.

“My research focuses on improving speech technology performance for children’s speech and African American English (AAE) speech in order to provide more equitable outcomes in early education. Speech technologies perform well for certain demographics (ie. able-bodied, adult, first-language speakers of mainstream dialects). However, they perform much worse for underrepresented groups (eg. young children, speakers of non-mainstream dialects, people with speech-related disabilities, etc.). Child speakers of AAE often show poorer reading and oral language performance than their white counterparts as a result of the orthographic mismatch between their spoken dialect and mainstream American English (MAE) taught in their classrooms. ASR systems trained to recognize AAE could give these students additional teaching support and help bridge this performance gap. However, this is a difficult low-resource problem given the small number of publicly available, labeled datasets for AAE speech in comparison to those for MAE speech. Thus, novel methods for low-resource dialects are needed in order to bring ASR systems for AAE-speaking children to the level of current data-driven ASR approaches for MAE.”

Tung Nguyen is pursuing a PhD in computer science; his advisor is Aditya Grover, assistant professor of computer science.

“Deep learning has grown rapidly in both scale and generalizability over the past decade. However, the majority of the real-world advances are made in the field of vision or language, while sequential decision-making paradigms such as reinforcement learning (RL) have lagged behind and only showed limited successes for controlled domains such as games. Sequential decision making in the real world is more challenging, because 1) the inputs are high-dimensional with long-range spatiotemporal dependencies; 2) agents need to quantify uncertainty to balance exploration and exploitation; and 3) active online interactions with the environment can be very expensive or even infeasible in high-stakes applications. My research goal is to address these challenges, and thereby enable robust sequential decision making for real-world applications. I outline my past research and future plans below.”

Alexander Schperberg is pursuing a PhD in mechanical and aerospace engineering; his advisor is Dennis Hong, professor of mechanical and aerospace engineering.

Related content
Teaching robots to stow items presents a challenge so large it was previously considered impossible — until now.

“My goal is to facilitate the dream of one day seeing diverse sets of wheeled, aerial, legged, and underwater robots being used ubiquitously towards reducing the burdens of society. Robotics and AI technology have the enormous potential to support humanity by performing tasks too dangerous for human workers, or through human-robot interactions. Unfortunately, while the potential use of robotics is an exciting prospect, they are still not commonly used due to a justified concern for both their safety and cost. For example, to make robots safer typically demands high-fidelity sensor and computer components. Thus, these robots are very expensive and are still seen as a luxury item rather than a product for everyday use. More troubling is that those from economically challenged and/or underprivileged groups may not have access and potentially cannot reap the benefit from this technology. Ideally, creating new robots using off-the-shelve or inexpensive components would greatly expand the robotic field and rapidly benefit society for all.”

Zhouxing Shi is pursuing a PhD in computer science; his advisor is Cho-Jui Hsieh, associate professor of computer science.

“My research interest is trustworthy machine learning and responsible AI, and I am currently working on the formally verifiable robustness of machine learning models especially neural networks.”

Zhaoqiang Wang is pursuing a PhD in bioengineering; his advisor is Liang Gao, assistant professor of bioengineering.

“Cardiovascular diseases (CVDs) are the leading cause of death globally, taking an estimated 17.9 million lives each year. In the United States, it is reported that approximately 82.6 million people currently live with at least one type of CVD, which contributes to a significant healthcare burden. To elucidate the underlying mechanism, researchers replicate the cardiac disease model in well-established genetic systems such as mouse and zebrafish. These model animals possess the essential common physiology as humans, but intelligent microscopy is critically necessary to reveal their heart morphology and dynamics.”

Yu Yang is pursuing a PhD in computer science; her advisor is Baharan Mirzasoleiman, assistant professor of computer science.

“My research contributes to the foundations of large-scale machine learning. Learning from massive datasets is financially and environmentally expensive. Moreover, large real-world data are usually biased toward large sub-populations, and often contain noisy or malicious examples that harm the generalization performance of the trained models. To address these problems, my research primarily focuses on understanding and improving the training data or learning objectives for resource-efficient and accountable learning.”

Da Yin is pursuing a PhD in computer science; his advisor is Kai-Wei Chang, associate professor computer science.

“I propose to utilize external knowledge to promote the effectiveness and inclusivity of neural models. Specifically, the framework of building models enhanced with external knowledge is usually separated into three important stages: 1) understanding what knowledge is not well learned by neural models; 2) acquiring knowledge necessary for specified domains; and 3) injecting knowledge to strengthen model’s capability.”

Zhe Zeng is pursuing a PhD in computer science; her advisor is Guy Van den Broeck, associate professor of computer science.

“How can we build artificial intelligence systems that are able to make efficient and re-liable inference under complex, noisy and highly structured real-world scenarios? One primary challenge to tackle this question is that probabilistic inference in such systems is, in general, computationally intractable. While current machine learning techniques heavily emphasize on scaling up probabilistic inference, they are at the cost of harming inference reliability. One promising direction is to combine probabilistic machine learning techniques and the formal verification techniques. My research interests primarily lie in bridging between AI and formal methods for such purposes.”

Related content

ES, B, Barcelona
Are you interested in defining the science strategy that enables Amazon to market to millions of customers based on their lifecycle needs rather than one-size-fits-all campaigns? We are seeking a Applied Scientist to lead the science strategy for our Lifecycle Marketing Experimentation roadmap within the PRIMAS (Prime & Marketing analytics and science) team. The position is open to candidates in Amsterdam and Barcelona. In this role, you will own the end-to-end science approach that enables EU marketing to shift from broad, generic campaigns to targeted, cohort-based marketing that changes customer behavior. This is a high-ambiguity, high-impact role where you will define what problems are worth solving, build the science foundation from scratch, and influence senior business leaders on marketing strategy. You will work directly with Business Directors and channel leaders to solve critical business problems: how do we win back customers lost to competitors, convert Young Adults to Prime, and optimize marketing spend by de-averaging across customer cohorts. Key job responsibilities Science Strategy & Leadership: 1. Own the end-to-end science strategy for lifecycle marketing, defining the roadmap across audience targeting, behavioral modeling, and measurement 2. Navigate high ambiguity in defining customer journey frameworks and behavioral models – our most challenging science problem with no established playbook 3. Lead strategic discussions with business leaders translating business needs into science solutions and building trust across business and tech partners 4. Mentor and guide a team of 2-3 scientists and BIEs on technical execution while contributing hands-on to the hardest problems Advanced Customer Behavior Modeling: 1. Build sophisticated propensity models identifying customer cohorts based on lifecycle stage and complex behavioral patterns (e.g., Bargain hunters, Young adults Prime prospects) 2. Define customer journey frameworks using advanced techniques (Hidden Markov Models, sequential decision-making) to model how customers transition across lifecycle stages 3. Identify which customer behaviors and triggers drive lifecycle progression and what messaging/levers are most effective for each cohort 4. Integrate 1P behavioral data with 2P survey insights to create rich, actionable audience definitions Measurement & Cross-Workstream Integration: 1. Partner with measurement scientist to design experiments (RCTs) that isolate audience targeting effects from creative effects 2. Ensure audience definitions, journey models, and measurement frameworks work coherently across Meta, LiveRamp, and owned channels 3. Establish feedback loops connecting measurement insights back to model improvements About the team The PRIMAS (Prime & Marketing Analytics and Science) is the team that support the science & analytics needs of the EU Prime and Marketing organization, an org that supports the Prime and Marketing programs in European marketplaces and comprises 250-300 employees. The PRIMAS team, is part of a larger tech tech team of 100+ people called WIMSI (WW Integrated Marketing Systems and Intelligence). WIMSI core mission is to accelerate marketing technology capabilities that enable de-averaged customer experiences across the marketing funnel: awareness, consideration, and conversion.
IN, KA, Bengaluru
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you have an entrepreneurial spirit, know how to deliver, love to work with data, are deeply technical, highly innovative and long for the opportunity to build solutions to challenging problems that directly impact the company's bottom-line, we want to talk to you. Major responsibilities - Use machine learning and analytical techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes - Design, development, evaluate and deploy innovative and highly scalable models for predictive learning - Research and implement novel machine learning and statistical approaches - Work closely with software engineering teams to drive real-time model implementations and new feature creations - Work closely with business owners and operations staff to optimize various business operations - Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation - Mentor other scientists and engineers in the use of ML techniques
ES, M, Madrid
At Amazon, we are committed to being the Earth's most customer-centric company. The European International Technology group (EU INTech) owns the enhancement and delivery of Amazon's engineering to all the varied customers and cultures of the world. We do this through a combination of partnerships with other Amazon technical teams and our own innovative new projects. You will be joining the Tamale team to work on Haul. As part of EU INTech and Haul, Tamale strives to create a discovery-driven shopping experience using challenging machine learning and ranking solutions. You will be exposed to large-scale recommendation systems, multi-objective optimization, and state-of-the-art deep learning architectures, and you'll be part of a key effort to improve our customers' browsing experience by building next-generation ranking models for Amazon Haul's endless scroll experience. We are looking for a passionate, talented, and inventive Scientist with a strong machine learning background to help build industry-leading ranking solutions. We strongly value your hard work and obsession to solve complex problems on behalf of Amazon customers. Key job responsibilities We look for applied scientists who possess a wide variety of skills. As the successful applicant for this role, you will work closely with your business partners to identify opportunities for innovation. You will apply machine learning solutions to optimize multi-objective ranking, improve discovery engagement through contextual signals, and scale ranking systems across multiple marketplaces. You will work with business leaders, scientists, and product managers to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed ranking services. You will be part of a team of scientists and engineers working on solving ranking and personalization challenges at scale. You will be able to influence the scientific roadmap of the team, setting the standards for scientific excellence. You will be working with state-of-the-art architectures and real-time feature serving systems. Your work will improve the experience of millions of daily customers using Amazon Haul worldwide. You will have the chance to have great customer impact and continue growing in one of the most innovative companies in the world. You will learn a huge amount - and have a lot of fun - in the process!
IN, HR, Gurugram
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced ML systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real-world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning team for International Emerging Stores (IES). Machine Learning, Big Data and related quantitative sciences have been strategic to Amazon from the early years. Amazon has been a pioneer in areas such as recommendation engines, ecommerce fraud detection and large-scale optimization of fulfillment center operations. As Amazon has rapidly grown and diversified, the opportunity for applying machine learning has exploded. We have a very broad collection of practical problems where machine learning systems can dramatically improve the customer experience, reduce cost, and drive speed and automation. These include product bundle recommendations for millions of products, safeguarding financial transactions across by building the risk models, improving catalog quality via extracting product attribute values from structured/unstructured data for millions of products, enhancing address quality by powering customer suggestions We are developing state-of-the-art machine learning solutions to accelerate the Amazon India growth story. Amazon is an exciting place to be at for a machine learning practitioner. We have the eagerness of a fresh startup to absorb machine learning solutions, and the scale of a mature firm to help support their development at the same time. As part of the International Machine Learning team, you will get to work alongside brilliant minds motivated to solve real-world machine learning problems that make a difference to millions of our customers. We encourage thought leadership and blue ocean thinking in ML. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes Design, develop, evaluate and deploy, innovative and highly scalable ML models Work closely with software engineering teams to drive real-time model implementations Work closely with business partners to identify problems and propose machine learning solutions Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model maintenance Work proactively with engineering teams and product managers to evangelize new algorithms and drive the implementation of large-scale complex ML models in production Leading projects and mentoring other scientists, engineers in the use of ML techniques About the team International Machine Learning Team is responsible for building novel ML solutions across International Emerging Store (India, MENA, Far-East, LatAm) problems and impact the bottom-line and top-line of India business. Learn more about our team from https://www.amazon.science/working-at-amazon/how-rajeev-rastogis-machine-learning-team-in-india-develops-innovations-for-customers-worldwide
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Bellevue
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.