Howard University's Founders Library is seen in the distance.
Howard University's Founders Library is seen in the distance. Howard is hosting AEASP “in support of increasing the pipeline of underrepresented minority economists.”
Oscar Merrida IV

Amazon to sponsor Howard University summer program aimed at increasing pipeline of minority economists

Howard University is the first Black college to host the American Economic Association Summer Training and Scholarship Program.

Howard University recently announced that it will host the American Economic Association Summer Training and Scholarship Program (AEASP) “in support of increasing the pipeline of underrepresented minority economists.” The program will be hosted at Howard for the next five years, and Amazon is sponsoring next summer’s program. Amazon first began discussions with Howard University about sponsoring AEASP about two years ago. The program, which aims to prepare “talented undergraduates for doctoral programs in economics and related disciplines,” will celebrate its 50th anniversary in 2024 at Howard.

"The lack of diversity in economics becomes self-reinforcing"

Four economists from diverse backgrounds shared how economics can address its diversity problem and talked about how their lives have shaped their work as economists.

That Howard, an historically Black college and university (HBCU) which produces more Black economics undergrads than any other institution, is hosting AEASP for the first time serves as a reminder of the progress the economics profession still must make.

The Caucus of Black Economists (later called the National Economics Association) first began exploring the issues of underrepresentation of minorities within the economics field in 1969. More than 50 years later, the economics profession is still grappling with structural issues. In fact, last January’s AEA conference in San Diego featured a panel titled, “How Can Economics Solve Its Race Problem.”

Rhonda Vonshay Sharpe and Omari H. Swinton standing in front of Howard University.
Rhonda Vonshay Sharpe, left, and Omari H. Swinton, right, are seen posing on the campus of Howard University. They discussed why economics still struggles with diversity.
Oscar Merrida IV

Omari H. Swinton, the chair of Howard University’s Department of Economics, who is both an alumni and the current director of the AEA summer program, as well as the past president of the National Economics Association, has observed that, “The vast majority of institutions in the US have never had a Black economist on staff, and the vast majority of schools have never graduated a Black PhD economist.”

Rhonda Vonshay Sharpe, the founder and president of the Women's Institute for Science, Equity, and Race (WISER), which is also a partner in next summer’s AEASP program, authored a research paper in 2019 that found that from 1966 to 2015, “the number of undergraduate economics degrees conferred to Black women was stagnant, and there was a decrease in the number of doctorates conferred to Black men.”

So why does the economics field still have such a massive disparity in minority representation? What needs to happen for systemic progress to be made? Amazon Science sat down with Sharpe and Swinton to ask those questions, as well as why Howard hosting the summer program is so significant, and what advice they would give to students considering economics as a major or profession. We also talked with Amazon chief economist Pat Bajari to find out why Amazon is sponsoring next summer’s AEASP program, and why he thinks diversity within the economics profession is essential.

A Howard University sign on the Howard campus
The AEASP will celebrate its 50th anniversary in 2024 at Howard University.
Oscar Merrida IV

Why does economics still have such a significant diversity problem?

Omari H. Swinton: I don't know that economics, as a profession, has really agreed that there's a problem. I think that's one of the big issues—we’ll say there's a problem, but nothing ever changes. You oftentimes hear people say things like, ‘We want to increase diversity’ but don't actually make any changes. They just say that that's something that they want to do.

It’s not as if these things haven't been out there. There are people out there who have dedicated their lives to bringing these types of issues to the forefront. I go back to Sandy Darity as an example. If you read from his earlier work, he's talking about these issues. Gregory Price has chronicled which institutions have Black economists in them. Rhonda has been looking at these issues for years.

Whether the economics profession is really ready to change is the issue. There have been a lot of people who have been talking about these issues for years. Others have come out and mentioned these problems more recently, but they ignore the fact that people have been talking about issues of underrepresentation for years.

Rhonda Vonshay Sharpe on the campus of Howard University
Rhonda Vonshay Sharpe says economics needs to define what diversity means. "If you don't define it, you can't measure it, or hold folks accountable."
Oscar Merrida IV

Rhonda Vonshay Sharpe: I narrow the problem down to be three things: 1) Economics has never defined what diversity means, and if you don't define it, you can't measure it, or hold folks accountable; 2) We don't have accurate data to track progress. We need to collect data that can be disaggregated by characteristics that have been used to limit participation in the profession. For example, when you talk about women, that usually means white women, and when we talk in terms of race, then you're really talking about men, and both of those descriptors are biased; and 3) As Omari said, there's enormous erasure happening. People have been doing this a long time, yet newcomers who have recently gotten tenure suddenly feel bad. They are handed a mic as if they are now the authorities. They don’t bother to understand whose shoulders they're standing on.

What needs to happen to address this problem? What role can academic institutions and companies like Amazon play?

Sharpe: I don't think the answer is to hire more Black economists. I really don't. And here is why: Because I think that when people say, ‘hire more Black economists’, people do just that, they hire Black economists. They do not think about whether or not those Black economists are bringing lived experiences that are going to help you craft policies to better interact with your customers.

One of the things I've been saying to folks recently is we need to talk more about structural classism and the ways in which we treat folks who are poor. So, it's not just about hiring Black economists, it's not about hiring Hispanic economists. It's about hiring folks who have lived experience in the US that will get at the inequality and related issues. That's not going to be solved just by hiring an economist because they are non-white.

Omari H. Swinton, the chair of Howard University's Department of Economics, on Howard's campus.
Omari H. Swinton says the AEASP program coming to Howard "is important because this is what our program is designed to do: increase minority participation in the economics profession."
Oscar Merrida IV

Swinton: If you say you want to diversify the profession, then stop looking at things that are not really problems. For example, there's not really a pipeline problem. You can ask almost any economics professor who teaches Principles of Economics, and most will tell you that is probably one of the worst classes to use if you want somebody to be interested in economics as a profession. But it really hasn't changed in years.

One change that we're making in the summer program is the experiential internship, or experiential learning. We’re going to place students with think tanks and corporations to actually see what an economist outside of the academy does. Everybody that gets a PhD in economics isn't going to be able to get a job as a professor. What does it look like to be an economist at Amazon? What does it look like to be an economist at the Census Bureau or at Brookings? Those are entirely different experiences. We’re trying to partner with as many different organizations as possible.

Hopefully we'll see change at those institutions, because students will come to the summer program, have that experience, and want to go back to those institutions. Rather than wanting to be a professor, they will, for example, say, ‘I want to be an economist at the Census Bureau, because I believe this research is important.’ It’s important for organizations, public and private, to be available to students, so they can see the type of experiences they can have if they work for you.

Pat Bajari
Pat Bajari, Amazon vice president and chief economist
Carl Clark, Amazon Imaging Studio

Pat Bajari: As an economist, I have always thought of this is in terms of diminishing returns. If you always have the same type of viewpoint, and keep hiring replicas of that viewpoint, the returns you get from that eventually decrease. Having different viewpoints allows you to do better work. And because we serve a large and diverse base of customers, we have a large and diverse base of problems. We want to take a leading role in supporting a new generation of economists from underrepresented minorities—it is not only the right thing to do, but it will also help bring strong and diverse voices that will create an even more inclusive customer experience.

When individuals come from different backgrounds, they bring different perspectives to the table. You do better work when you have different perspectives.
Pat Bajari

Swinton: One thing organizations can do is find programs that are actually successful at achieving the types of goals they’re pursuing. For example, some of the research done by Becker et al. shows that about 20 percent of Blacks that have PhDs in economics have attended the AEASP program. By helping support Howard in hosting AEASP in this first year, Amazon is doing that. Without Amazon’s support, Howard wouldn't be able to host the AEA summer program at all. We certainly hope others will follow Amazon’s lead.

What is the significance of the summer program coming to Howard?

Swinton: The summer program is extremely important in my path as an economist. My first cohort of economists were the people that I met through the summer program. Howard is the number one producer as an undergraduate feeder of Blacks who go on to get PhDs in economics. This is our mission and one of our goals as an institution and as a department, and I think the AEA summer program coming to Howard is important because this is what our program is designed to do: increase minority participation in the economics profession.

The National Economics Association summer program came out of Marcus Alexis’ mind as a program to help get minorities interested in economics. For the AEASP program to come to Howard at this point in time is a great honor. It’s an honor to be the first HBCU to host the summer program.

Sharpe: I'm excited to see a program that's going to be led by Blacks, which I think is incredibly important, as the program will celebrate 50 years while it's at Howard in 2024. It just feels full circle in terms of thinking about Marcus Alexis, who was a Black economist, and then having the program 50 years later be at an institution that is the number one producer of Black economists. That's incredibly exciting.

Finally, what advice would you give to someone considering whether to pursue a degree in economics? Why is economics such an important field?

Bajari: A lot of economics is understanding people's material wellbeing. Who has low wages? Who has high wages? If you take a given policy, whether that's central bank policy or interventions into labor markets, etcetera, these things deeply, deeply, deeply affect people's lives, people's material outcomes. What they can purchase and where they can live and where they can send their kids to school and so forth. It's an important set of questions, and they range from micro things about what happens to the individual, to macro things, such as how the whole world is evolving and changing in response to things like COVID-19.

Howard University's Founders Library
Howard University's Founders Library is seen here. Howard is the first Black college to host AEASP.
Oscar Merrida IV

If we change policy or somebody goes to college versus doesn't go to college, what are the implications of those economic variables? I know this is what attracted me to economics. As a young person, growing up pretty poor in rural Minnesota, I was interested in the world and how it worked. And I liked economics because it brought math and data and scientific formalism to those questions. That's not the only way you can look at those questions, or the only way you should look at them, but it’s one way that's highly useful.

Sharpe: For students pursuing a PhD in economics, my main advice is to pick a PhD program that's a good fit for you. Many students think that if you don't go to a top program, you can't have a successful career. That’s not true. I went to Claremont Graduate University, not highly ranked, but I had an amazing time as a graduate student. I loved it. My mentee when I was in graduate school was Olugbenga Ajilore who’s at CAP (Center for American Progress) now, who is a rock star right now in terms of being in the news and asking people to think about rural communities. He and I didn't go to top economics departments, but we went to places that were good fits for us, and that's incredibly important.

Bajari: “Technology economics” is a booming field. The largest conference held by the National Association of Business Economists is now the tech economics conference. It’s larger than their annual conference now, because it's been an explosive area of job growth for young people. We are one of the larger private sector employers of economists. When you're in that role, you have an obligation to demonstrate leadership. We saw sponsorship of AEASP as an opportunity to expose young PhDs to this emerging field. I thought Howard was very thoughtful about their proposal, and I'm hoping AEASP can help us establish a pipeline of highly qualified candidates.

Swinton: I talk to students about this all the time. You want to make a change, and you want to be a policy maker? Be an economist. You want to go into business and work on Wall Street, make a lot of money? Be an economist. Economics is one of the most useful majors because it allows you the opportunity you to go out and do a wide variety of things based on the basic training you obtain.

Applications for the summer program are open and the deadline to apply is January 31, 2021. To apply, visit economics.howard.edu/aeasp. The program will be held May 27 to July 25, 2021, and be offered in Washington, D.C., contingent upon COVID-19 restrictions.

Research areas

Related content

US, WA, Seattle
WW Amazon Stores Finance Science (ASFS) works to leverage science and economics to drive improved financial results, foster data backed decisions, and embed science within Finance. ASFS is focused on developing products that empower controllership, improve business decisions and financial planning by understanding financial drivers, and innovate science capabilities for efficiency and scale. We are looking for a data scientist to lead high visibility initiatives for forecasting Amazon Stores' financials. You will develop new science-based forecasting methodologies and build scalable models to improve financial decision making and planning for senior leadership up to VP and SVP level. You will build new ML and statistical models from the ground up that aim to transform financial planning for Amazon Stores. We prize creative problem solvers with the ability to draw on an expansive methodological toolkit to transform financial decision-making with science. The ideal candidate combines data-science acumen with strong business judgment. You have versatile modeling skills and are comfortable owning and extracting insights from data. You are excited to learn from and alongside seasoned scientists, engineers, and business leaders. You are an excellent communicator and effectively translate technical findings into business action. Key job responsibilities Demonstrating thorough technical knowledge, effective exploratory data analysis, and model building using industry standard ML models Working with technical and non-technical stakeholders across every step of science project life cycle Collaborating with finance, product, data engineering, and software engineering teams to create production implementations for large-scale ML models Innovating by adapting new modeling techniques and procedures Presenting research results to our internal research community
US, WA, Seattle
This role will contribute to developing the Economics and Science products and services in the Fee domain, with specialization in supply chain systems and fees. Through the lens of economics, you will develop causal links for how Amazon, Sellers and Customers interact. You will be a key and senior scientist, advising Amazon leaders how to price our services. You will work on developing frameworks and scalable, repeatable models supporting optimal pricing and policy in the two-sided marketplace that is central to Amazon's business. The pricing for Amazon services is complex. You will partner with science and technology teams across Amazon including Advertising, Supply Chain, Operations, Prime, Consumer Pricing, and Finance. We are looking for an experienced Economist to improve our understanding of seller Economics, enhance our ability to estimate the causal impact of fees, and work with partner teams to design pricing policy changes. In this role, you will provide guidance to scientists to develop econometric models to influence our fee pricing worldwide. You will lead the development of causal models to help isolate the impact of fee and policy changes from other business actions, using experiments when possible, or observational data when not. Key job responsibilities The ideal candidate will have extensive Economics knowledge, demonstrated strength in practical and policy relevant structural econometrics, strong collaboration skills, proven ability to lead highly ambiguous and large projects, and a drive to deliver results. They will work closely with Economists, Data / Applied Scientists, Strategy Analysts, Data Engineers, and Product leads to integrate economic insights into policy and systems production. Familiarity with systems and services that constitute seller supply chains is a plus but not required. About the team The Stores Economics and Sciences team is a central science team that supports Amazon's Retail and Supply Chain leadership. We tackle some of Amazon's most challenging economics and machine learning problems, where our mandate is to impact the business on massive scale.
US, WA, Bellevue
We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to apply their causal inference and/or structural econometrics skillsets to solve real world problems. The intern will work in the area of Economics Intelligence in Amazon Returns and Recommerce Technology and Innovation and develop new, data-driven solutions to support the most critical components of this rapidly scaling team. Our PhD Economist Internship Program offers hands-on experience in applied economics, supported by mentorship, structured feedback, and professional development. Interns work on real business and research problems, building skills that prepare them for full-time economist roles at Amazon and beyond. You will learn how to build data sets and perform applied econometric analysis collaborating with economists, scientists, and product managers. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement. These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. About the team The WWRR Economics Intelligence (RREI) team brings together Economists, Data Scientists, and Business Intelligence Engineers experts to delivers economic solutions focused on forecasting, causality, attribution, customer behavior for returns, recommerce, and sustainability domains.
US, WA, Bellevue
We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to apply their causal inference and/or structural econometrics skillsets to solve real world problems. The intern will work in the area of Economics Intelligence in Amazon Returns and Recommerce Technology and Innovation and develop new, data-driven solutions to support the most critical components of this rapidly scaling team. Our PhD Economist Internship Program offers hands-on experience in applied economics, supported by mentorship, structured feedback, and professional development. Interns work on real business and research problems, building skills that prepare them for full-time economist roles at Amazon and beyond. You will learn how to build data sets and perform applied econometric analysis collaborating with economists, scientists, and product managers. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement. These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. About the team The WWRR Economics Intelligence (RREI) team brings together Economists, Data Scientists, and Business Intelligence Engineers experts to delivers economic solutions focused on forecasting, causality, attribution, customer behavior for returns, recommerce, and sustainability domains.
US, WA, Seattle
Innovators wanted! Are you an entrepreneur? A builder? A dreamer? This role is part of an Amazon Special Projects team that takes the company’s Think Big leadership principle to the next level. We focus on creating entirely new products and services with a goal of positively impacting the lives of our customers. No industries or subject areas are out of bounds. If you’re interested in innovating at scale to address big challenges in the world, this is the team for you. As a Research Scientist, you will work with a unique and gifted team developing exciting products for consumers and collaborate with cross-functional teams. Our team rewards intellectual curiosity while maintaining a laser-focus in bringing products to market. Competitive candidates are responsive, flexible, and able to succeed within an open, collaborative, entrepreneurial, startup-like environment. At the intersection of both academic and applied research in this product area, you have the opportunity to work together with some of the most talented scientists, engineers, and product managers. Here at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have thirteen employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We are constantly learning through programs that are local, regional, and global. Amazon’s culture of inclusion is reinforced within our 16 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Our team highly values work-life balance, mentorship and career growth. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We care about your career growth and strive to assign projects and offer training that will challenge you to become your best.
US, WA, Seattle
Amazon has co-founded and signed The Climate Pledge, a commitment to reach net zero carbon by 2040. As a team, we leverage GenAI, sensors, smart home devices, cloud services, material science, and Alexa to build products that have a meaningful impact for customers and the climate. In alignment with this bold corporate goal, the Amazon Devices & Services organization is looking for a passionate, talented, and inventive Senior Applied Scientist to help build revolutionary products with potential for major societal impact. Great candidates for this position will have expertise in the areas of agentic AI applications, deep learning, time series analysis, LLMs, and multimodal systems. This includes experience designing autonomous AI agents that can reason, plan, and execute multi-step tasks, building tool-augmented LLM systems with access to external APIs and data sources, implementing multi-agent orchestration, and developing RAG architectures that combine LLMs with domain-specific knowledge bases. You will strive for simplicity and creativity, demonstrating high judgment backed by statistical proof. Key job responsibilities As a Senior Applied Scientist on the Energy Science team, you'll design and deploy agentic AI systems that autonomously analyze data, plan solutions, and execute recommendations. You'll build multi-agent architectures where specialized AI agents coordinate to solve complex optimization problems, and develop tool-augmented LLM applications that integrate with external data sources and APIs to deliver context-aware insights. Your work involves creating multimodal AI systems that synthesize diverse data streams, while implementing RAG pipelines that ground large language models in domain-specific knowledge bases. You'll apply advanced machine learning and deep learning techniques to time series analysis, forecasting, and pattern recognition. Beyond technical innovation, you'll drive end-to-end product development from research through production deployment, collaborating with cross-functional teams to translate AI capabilities into customer experiences. You'll establish rigorous experimentation frameworks to validate model performance and measure business impact, building AI-driven products with potential for major societal impact.
US, CA, San Francisco
Amazon launched the AGI Lab to develop foundational capabilities for useful AI agents. We built Nova Act - a new AI model trained to perform actions within a web browser. The team builds AI/ML infrastructure that powers our production systems to run performantly at high scale. We’re also enabling practical AI to make our customers more productive, empowered, and fulfilled. In particular, our work combines large language models (LLMs) with reinforcement learning (RL) to solve reasoning, planning, and world modeling in both virtual and physical environments. Our lab is a small, talent-dense team with the resources and scale of Amazon. Each team in the lab has the autonomy to move fast and the long-term commitment to pursue high-risk, high-payoff research. We’re entering an exciting new era where agents can redefine what AI makes possible. We’d love for you to join our lab and build it from the ground up! Key job responsibilities This role will lead a team of SDEs building AI agents infrastructure from launch to scale. The role requires the ability to span across ML/AI system architecture and infrastructure. You will work closely with application developers and scientists to have a impact on the Agentic AI industry. We're looking for a Software Development Manager who is energized by building high performance systems, making an impact and thrives in fast-paced, collaborative environments. About the team Check out the Nova Act tools our team built on on nova.amazon.com/act
US, WA, Seattle
MULTIPLE POSITIONS AVAILABLE Employer: AMAZON WEB SERVICES, INC. Offered Position: Applied Scientist III Job Location: Seattle, Washington Job Number: AMZ9674037 Position Responsibilities: Participate in the design, development, evaluation, deployment and updating of data-driven models and analytical solutions for machine learning (ML) and/or natural language (NL) applications. Develop and/or apply statistical modeling techniques (e.g. Bayesian models and deep neural networks), optimization methods, and other ML techniques to different applications in business and engineering. Routinely build and deploy ML models on available data, and run and analyze experiments in a production environment. Identify new opportunities for research in order to meet business goals. Research and implement novel ML and statistical approaches to add value to the business. Mentor junior engineers and scientists. Position Requirements: Master’s degree or foreign equivalent degree in Computer Science, Machine Learning, Engineering, or a related field and two years of research or work experience in the job offered, or as a Research Scientist, Research Assistant, Software Engineer, or a related occupation. Employer will accept a Bachelor’s degree or foreign equivalent degree in Computer Science, Machine Learning, Engineering, or a related field and five years of progressive post-baccalaureate research or work experience in the job offered or a related occupation as equivalent to the Master’s degree and two years of research or work experience. Must have one year of research or work experience in the following skill(s): (1) programming in Java, C++, Python, or equivalent programming language; and (2) conducting the analysis and development of various supervised and unsupervised machine learning models for moderately complex projects in business, science, or engineering. Amazon.com is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation. 40 hours / week, 8:00am-5:00pm, Salary Range $167,100/year to $226,100/year. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, visit: https://www.aboutamazon.com/workplace/employee-benefits.#0000
IN, KA, Bengaluru
Amazon Health Services (One Medical) About Us: At Health AI, we're revolutionizing healthcare delivery through innovative AI-enabled solutions. As part of Amazon Health Services and One Medical, we're on a mission to make quality healthcare more accessible while improving patient outcomes. Our work directly impacts millions of lives by empowering patients and enabling healthcare providers to deliver more meaningful care. Role Overview: We're seeking an Applied Scientist to join our dynamic team in building state of the art AI/ML solutions for healthcare. This role offers a unique opportunity to work at the intersection of artificial intelligence and healthcare, developing solutions that will shape the future of medical services delivery. Key job responsibilities • Lead end-to-end development of AI/ML solutions for Amazon Health organization, including Amazon Pharmacy and One Medical • Research, design, and implement state-of-the-art machine learning models, with a focus on Large Language Models (LLMs) and Visual Language Models (VLMs) • Optimize and fine-tune models for production deployment, including model distillation for improved latency • Drive scientific innovation while maintaining a strong focus on practical business outcomes • Collaborate with cross-functional teams to translate complex technical solutions into tangible customer benefits • Contribute to the broader Amazon Health scientific community and help shape our technical roadmap
US, CA, Santa Clara
Amazon Quick Suite is an enterprise AI platform that transforms how organizations work with their data and knowledge. Combining generative AI-powered search, deep research capabilities, intelligent agents and automations, and comprehensive business intelligence, Quick Suite serves tens of thousands of users. Our platform processes thousands of queries monthly, helping teams make faster, data-driven decisions while maintaining enterprise-grade security and governance. From natural language interactions with complex datasets to automated workflows and custom AI agents, Quick Suite is redefining workplace productivity at unprecedented scale. We are seeking a Data Scientist II to join our Quick Data team, focusing on evaluation and benchmarking data development for Quick Suite features, with particular emphasis on Research and other generative AI capabilities. Our mission is to engineer high-quality datasets that are essential to the success of Amazon Quick Suite. From human evaluations and Responsible AI safeguards to Retrieval-Augmented Generation and beyond, our work ensures that Generative AI is enterprise-ready, safe, and effective for users at scale. As part of our diverse team—including data scientists, engineers, language engineers, linguists, and program managers—you will collaborate closely with science, engineering, and product teams. We are driven by customer obsession and a commitment to excellence. Key job responsibilities In this role, you will leverage data-centric AI principles to assess the impact of data on model performance and the broader machine learning pipeline. You will apply Generative AI techniques to evaluate how well our data represents human language and conduct experiments to measure downstream interactions. Specific responsibilities include: * Design and develop comprehensive evaluation and benchmarking datasets for Quick Suite AI-powered features * Leverage LLMs for synthetic data corpora generation; data evaluation and quality assessment using LLM-as-a-judge settings * Create ground truth datasets with high-quality question-answer pairs across diverse domains and use cases * Lead human annotation initiatives and model evaluation audits to ensure data quality and relevance * Develop and refine annotation guidelines and quality frameworks for evaluation tasks * Conduct statistical analysis to measure model performance, identify failure patterns, and guide improvement strategies * Collaborate with ML scientists and engineers to translate evaluation insights into actionable product improvements * Build scalable data pipelines and tools to support continuous evaluation and benchmarking efforts * Contribute to Responsible AI initiatives by developing safety and fairness evaluation datasets About the team Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Hybrid Work We value innovation and recognize this sometimes requires uninterrupted time to focus on a build. We also value in-person collaboration and time spent face-to-face. Our team affords employees options to work in the office every day or in a flexible, hybrid work model near one of our U.S. Amazon offices.