Founding members of Black in Robotics Monroe Kennedy III and Ayanna Howard with a robot.
Ayanna Howard, chair of the School of Interactive Computing with the College of Computing at Georgia Tech, and Monroe Kennedy III, an assistant professor of mechanical engineering at Stanford University, are founding members of Black in Robotics (BiR), and members of the organization's leadership team. On Nov. 14, BiR and Amazon Robotics will announce they're collaborating to create a regional BiR chapter in Boston. It is the first of what BiR hopes will be many chapters established across the US.
Credit: Glynnis Condon, Georgia Tech, Stanford

Amazon Robotics is primary sponsor of new Black in Robotics Boston chapter

Recently formed organization advocates for more diversity, inclusion, and equity within robotics field.

At a virtual student event connected to the International Conference on Intelligent Robots and Systems (IROS), Amazon Robotics and the recently established Black in Robotics (BiR) organization will announce tomorrow they’re collaborating to create BiR’s first regional chapter, the first of what BiR hopes will be many chapters established across the US.  

Amazon Robotics’ sponsorship includes financial support for events and other activities in the Boston area, as well as providing meeting and event space in Amazon's offices in Cambridge, Massachusetts.

The Boston chapter will help create opportunities for up-and-coming roboticists, and help enrich the company’s talent pipeline, says Tye Brady, Amazon Robotics’ chief technology officer. Amazon Robotics’ headquarters is located in North Reading, Massachusetts, just north of Boston.  The Boston chapter will help Black engineering and science students at Boston-area colleges and universities such as MIT, Harvard, Boston University, Boston College, Northeastern, the University of Massachusetts, Tufts, Worcester Polytechnic Institute, Olin College, and Brown University, network and obtain mentoring and internship  opportunities from area academic and industry professionals.

“Black in Robotics is really about building community and advocating for diversity, and encouraging accountability,” said Ayanna Howard, chair of the School of Interactive Computing within the College of Computing at Georgia Tech, one of BiR’s founders, and a member of its leadership team. “We feel it makes sense to start in Boston because of the large robotics community that exists there, and because there are a large number of students there attending regional universities. It’s an ideal location to start because we can reach out to the students and reach engineers there, and establish a presence, and a blueprint for how we can build communities elsewhere.”

“If you’re an underrepresented minority student at one of the area colleges and universities, you might be only one of two or three people who look like you in your classes, and if you’re a professional working in industry, you might also be only one of two or three roboticists who look like you in your organization,” Howard added. “So this chapter is about building community, and helping students and professionals develop a network, and avoid the isolation they might otherwise feel.”

“We are really driven by the fact that we need to be the change we want to see,” added Monroe Kennedy III, an assistant professor of mechanical engineering at Stanford University who is also a founding member of BiR, and a member of its leadership team. “African Americans especially, but minorities in general, are underrepresented in STEM, and in robotics. As members of the robotics community, we recognize how valuable being a member of the community is to every aspect of our lives. So our objective is to make that opportunity available to more underrepresented minorities.”

Issues of diversity are on the agenda at AfroTech World

We asked some Black scientists to discuss the issues limiting underrepresented minorities’ involvement in the technology industry, and to share their own experiences. See what they had to say.

“The current and pervasive lack of racial, ethnic, and gender diversity in the technology ecosystem presents a significant national challenge,” says a report on women and girls of color in computing from the Kapor Center, and other contributors.

Brady says Amazon Robotics is excited to support BiR’s mission of bringing together researchers, industry professionals, and students in robotics to support one another, and help navigate academic, industry, and entrepreneurial paths. When Howard talked with Brady recently about how to address systemic inequities in robotics and the role Amazon Robotics could play in supporting the Boston BiR chapter, he didn’t hesitate.

“Ayanna is an amazing individual with deep technical chops.  Whenever she gets behind something, I pay attention,” Brady said. “I’m excited to support her and the organization’s mission.”

Brady believes it’s important to define the future of robotics with a mindset of diversity and inclusion.

Tye Brady
Tye Brady, Amazon Robotics' chief technology officer, says the Boston chapter of Black in Robotics will help create opportunities for up-and-coming roboticists, and help enrich the company's talent pipeline.

“Why do I say that?  Three reasons,” Brady said. “First, we become more creative with diverse viewpoints and perspectives, which ultimately leads to better robotics. Second, diversity is the answer to adversity. By that I mean, teams become stronger when they’re diverse; they bring more opportunities, more experience, and more viewpoints to the challenge of tackling hard problems. And third, we can source talent from a much larger pool of talent. We’re constantly seeking the best and the brightest to help us reimagine the future, and that means including others than yourself. We really need to grow the talent pipeline in computer engineering generally, and robotics more specifically, and this is a great opportunity to do that.”

The importance of mentoring

Kennedy says the new BiR Boston chapter represents the deep commitment industry and academia have to increasing the number of underrepresented minorities within STEM professions.  He’s excited about the forthcoming opportunities for students, faculty, and industry professionals to learn from each other, and collaborate on projects together.

“This is really a great opportunity for academia and industry to come together, put everything aside, and really come together as roboticists who care about each other, and want to help one another.  This first chapter can really facilitate that vision,” Kennedy said.

The mentorship component of BiR’s vision resonated with Brady.

“I just think about my own experiences when it comes to having mentors, having people help you when you have an interest in something, but no experience,” Brady said. “I’m thankful to those people who extended a helping hand to me. I’m thankful to those companies who said, ‘I can help you; I can give you a chance.’ I’ve lived that, and it’s important to pass that experience on to others. This initiative provides a great opportunity to do that.  Not just for myself, but for our entire Amazon Robotics team.”

Kennedy says he, too, has benefited from many mentors in his career journey, and he’s pulled together different attributes from each to help guide his career and lifestyle choices.

“One thing that was lacking is there wasn’t necessarily one place I could go where these examples came together in a nice, cohesive way,” Kennedy says. “This is something Black in Robotics has to offer. We want to bring these individuals, these examples, together in a cohesive way, so students don’t have to look all over the place to get exposed to these positive examples, to find their heroes.”

Mentorship is a critical component of Howard’s role as a professor at Georgia Tech.

“My group at Tech is very diverse; it’s representative of the world and I hope all of them pursue a career in robotics,” Howard said. “I mentor them, not just academically, but socially as well. I introduce them to people, and I work to push them out front and promote them. And we just need more of that. That’s what resonates for me with this Boston chapter; it’s that ability to network with others that doesn’t happen naturally without this kind of organization. We’re creating the water cooler where people can come together, and networking will be a natural byproduct of that.”

Education and community outreach

Education and community outreach is another component of BiR’s mission.  This, too, speaks to Brady, who’s concerned not only by the lack of underrepresented minorities within the tech industry overall, and the robotics field more specifically, but with declining college enrollments in STEM-related courses.

“We have to get to the K through 12 kids sooner, and engage them in computer science, and science-related topics sooner,” Brady said. “That’s one of the things we hope to enable with this chapter. We want to create robotics ambassadors who can get engaged with K through 12 kids, and share their experiences and their passion for robotics. “Our entire community needs to get involved because we need more college graduates with computer science degrees, engineering degrees, and graduates who are STEM minded.  So that’s another aspect of this initiative that we’re really excited about.”

"Outreach at every level of a student’s education is required"

Four economists from diverse backgrounds shared how economics can address its diversity problem and talked about how their lives have shaped their work as economists.

Howard agrees, though she emphasizes that the burden shouldn’t solely rest with Black roboticists to help drive change within the field.

“There has been a lot of progress; the needle has moved,” she says. “It just hasn’t moved at the rate is has in other fields like medicine or law. The way we’ve established this organization from the outset is that allies are a pillar. Our mission isn’t just to connect Black roboticists with other Black roboticists. We don’t believe that will move the needle. People like myself, who have been in this field for a while, shouldn’t bear the burden of fixing robotics.  That’s why our approach has been to bring in allies from the beginning; we’ve created an ally steering committee because this isn’t a Black or Brown problem, this is a robotics problem.”

Those of us who have more senior roles in our field have helped knock down some of the walls, but we need to make sure that even though we’ve knocked down some walls, the road for future roboticists is more comfortable to travel on.
Ayanna Howard

“Some of this is also about calling ourselves out,” Howard continues, “to say ‘Look, this is as much our fault as it is society’s fault, and therefore we need to play a role in fixing it. Those of us who have more senior roles in our field have helped knock down some of the walls, but we need to make sure that even though we’ve knocked down some walls, the road for future roboticists is more comfortable to travel on.”

In addition to Howard and Kennedy, other members of BiR’s leadership team include Carlotta Berry, a professor of electrical and computer engineering at the Rose-Hulman Institute of Technology; Edward  Tunstel, group leader, and associate director of robotics at Raytheon Technologies’ Research Center; Quincy Kissiedu-Brown, cofounder of blackcomputerHER; Maynard Holliday, senior engineer, Rand Corporation; Yves Nazon II, a graduate research assistant at the University of Michigan; and Kwesi Rutledge, a PhD candidate in electrical engineering at the University of Michigan.

Howard says the Boston chapter’s first event is scheduled for January. Individuals interested in becoming a BiR member can visit the organization’s website.

Research areas

Related content

  • Staff writer
    December 29, 2025
    From foundation model safety frameworks and formal verification at cloud scale to advanced robotics and multimodal AI reasoning, these are the most viewed publications from Amazon scientists and collaborators in 2025.
  • Staff writer
    December 29, 2025
    From quantum computing breakthroughs and foundation models for robotics to the evolution of Amazon Aurora and advances in agentic AI, these are the posts that captured readers' attention in 2025.
  • Staff writer
    October 21, 2025
    Initiative will fund over 100 doctoral students researching machine learning, computer vision, and natural-language processing at nine universities.
ES, B, Barcelona
Are you interested in defining the science strategy that enables Amazon to market to millions of customers based on their lifecycle needs rather than one-size-fits-all campaigns? We are seeking a Applied Scientist to lead the science strategy for our Lifecycle Marketing Experimentation roadmap within the PRIMAS (Prime & Marketing analytics and science) team. The position is open to candidates in Amsterdam and Barcelona. In this role, you will own the end-to-end science approach that enables EU marketing to shift from broad, generic campaigns to targeted, cohort-based marketing that changes customer behavior. This is a high-ambiguity, high-impact role where you will define what problems are worth solving, build the science foundation from scratch, and influence senior business leaders on marketing strategy. You will work directly with Business Directors and channel leaders to solve critical business problems: how do we win back customers lost to competitors, convert Young Adults to Prime, and optimize marketing spend by de-averaging across customer cohorts. Key job responsibilities Science Strategy & Leadership: 1. Own the end-to-end science strategy for lifecycle marketing, defining the roadmap across audience targeting, behavioral modeling, and measurement 2. Navigate high ambiguity in defining customer journey frameworks and behavioral models – our most challenging science problem with no established playbook 3. Lead strategic discussions with business leaders translating business needs into science solutions and building trust across business and tech partners 4. Mentor and guide a team of 2-3 scientists and BIEs on technical execution while contributing hands-on to the hardest problems Advanced Customer Behavior Modeling: 1. Build sophisticated propensity models identifying customer cohorts based on lifecycle stage and complex behavioral patterns (e.g., Bargain hunters, Young adults Prime prospects) 2. Define customer journey frameworks using advanced techniques (Hidden Markov Models, sequential decision-making) to model how customers transition across lifecycle stages 3. Identify which customer behaviors and triggers drive lifecycle progression and what messaging/levers are most effective for each cohort 4. Integrate 1P behavioral data with 2P survey insights to create rich, actionable audience definitions Measurement & Cross-Workstream Integration: 1. Partner with measurement scientist to design experiments (RCTs) that isolate audience targeting effects from creative effects 2. Ensure audience definitions, journey models, and measurement frameworks work coherently across Meta, LiveRamp, and owned channels 3. Establish feedback loops connecting measurement insights back to model improvements About the team The PRIMAS (Prime & Marketing Analytics and Science) is the team that support the science & analytics needs of the EU Prime and Marketing organization, an org that supports the Prime and Marketing programs in European marketplaces and comprises 250-300 employees. The PRIMAS team, is part of a larger tech tech team of 100+ people called WIMSI (WW Integrated Marketing Systems and Intelligence). WIMSI core mission is to accelerate marketing technology capabilities that enable de-averaged customer experiences across the marketing funnel: awareness, consideration, and conversion.
IN, KA, Bengaluru
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you have an entrepreneurial spirit, know how to deliver, love to work with data, are deeply technical, highly innovative and long for the opportunity to build solutions to challenging problems that directly impact the company's bottom-line, we want to talk to you. Major responsibilities - Use machine learning and analytical techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes - Design, development, evaluate and deploy innovative and highly scalable models for predictive learning - Research and implement novel machine learning and statistical approaches - Work closely with software engineering teams to drive real-time model implementations and new feature creations - Work closely with business owners and operations staff to optimize various business operations - Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation - Mentor other scientists and engineers in the use of ML techniques
ES, M, Madrid
At Amazon, we are committed to being the Earth's most customer-centric company. The European International Technology group (EU INTech) owns the enhancement and delivery of Amazon's engineering to all the varied customers and cultures of the world. We do this through a combination of partnerships with other Amazon technical teams and our own innovative new projects. You will be joining the Tamale team to work on Haul. As part of EU INTech and Haul, Tamale strives to create a discovery-driven shopping experience using challenging machine learning and ranking solutions. You will be exposed to large-scale recommendation systems, multi-objective optimization, and state-of-the-art deep learning architectures, and you'll be part of a key effort to improve our customers' browsing experience by building next-generation ranking models for Amazon Haul's endless scroll experience. We are looking for a passionate, talented, and inventive Scientist with a strong machine learning background to help build industry-leading ranking solutions. We strongly value your hard work and obsession to solve complex problems on behalf of Amazon customers. Key job responsibilities We look for applied scientists who possess a wide variety of skills. As the successful applicant for this role, you will work closely with your business partners to identify opportunities for innovation. You will apply machine learning solutions to optimize multi-objective ranking, improve discovery engagement through contextual signals, and scale ranking systems across multiple marketplaces. You will work with business leaders, scientists, and product managers to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed ranking services. You will be part of a team of scientists and engineers working on solving ranking and personalization challenges at scale. You will be able to influence the scientific roadmap of the team, setting the standards for scientific excellence. You will be working with state-of-the-art architectures and real-time feature serving systems. Your work will improve the experience of millions of daily customers using Amazon Haul worldwide. You will have the chance to have great customer impact and continue growing in one of the most innovative companies in the world. You will learn a huge amount - and have a lot of fun - in the process!
IN, HR, Gurugram
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced ML systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real-world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning team for International Emerging Stores (IES). Machine Learning, Big Data and related quantitative sciences have been strategic to Amazon from the early years. Amazon has been a pioneer in areas such as recommendation engines, ecommerce fraud detection and large-scale optimization of fulfillment center operations. As Amazon has rapidly grown and diversified, the opportunity for applying machine learning has exploded. We have a very broad collection of practical problems where machine learning systems can dramatically improve the customer experience, reduce cost, and drive speed and automation. These include product bundle recommendations for millions of products, safeguarding financial transactions across by building the risk models, improving catalog quality via extracting product attribute values from structured/unstructured data for millions of products, enhancing address quality by powering customer suggestions We are developing state-of-the-art machine learning solutions to accelerate the Amazon India growth story. Amazon is an exciting place to be at for a machine learning practitioner. We have the eagerness of a fresh startup to absorb machine learning solutions, and the scale of a mature firm to help support their development at the same time. As part of the International Machine Learning team, you will get to work alongside brilliant minds motivated to solve real-world machine learning problems that make a difference to millions of our customers. We encourage thought leadership and blue ocean thinking in ML. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes Design, develop, evaluate and deploy, innovative and highly scalable ML models Work closely with software engineering teams to drive real-time model implementations Work closely with business partners to identify problems and propose machine learning solutions Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model maintenance Work proactively with engineering teams and product managers to evangelize new algorithms and drive the implementation of large-scale complex ML models in production Leading projects and mentoring other scientists, engineers in the use of ML techniques About the team International Machine Learning Team is responsible for building novel ML solutions across International Emerging Store (India, MENA, Far-East, LatAm) problems and impact the bottom-line and top-line of India business. Learn more about our team from https://www.amazon.science/working-at-amazon/how-rajeev-rastogis-machine-learning-team-in-india-develops-innovations-for-customers-worldwide
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Bellevue
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.