Howard University's Founders Library is seen in the distance.
Howard University's Founders Library is seen in the distance. Howard is hosting AEASP “in support of increasing the pipeline of underrepresented minority economists.”
Oscar Merrida IV

Amazon to sponsor Howard University summer program aimed at increasing pipeline of minority economists

Howard University is the first Black college to host the American Economic Association Summer Training and Scholarship Program.

Howard University recently announced that it will host the American Economic Association Summer Training and Scholarship Program (AEASP) “in support of increasing the pipeline of underrepresented minority economists.” The program will be hosted at Howard for the next five years, and Amazon is sponsoring next summer’s program. Amazon first began discussions with Howard University about sponsoring AEASP about two years ago. The program, which aims to prepare “talented undergraduates for doctoral programs in economics and related disciplines,” will celebrate its 50th anniversary in 2024 at Howard.

"The lack of diversity in economics becomes self-reinforcing"

Four economists from diverse backgrounds shared how economics can address its diversity problem and talked about how their lives have shaped their work as economists.

That Howard, an historically Black college and university (HBCU) which produces more Black economics undergrads than any other institution, is hosting AEASP for the first time serves as a reminder of the progress the economics profession still must make.

The Caucus of Black Economists (later called the National Economics Association) first began exploring the issues of underrepresentation of minorities within the economics field in 1969. More than 50 years later, the economics profession is still grappling with structural issues. In fact, last January’s AEA conference in San Diego featured a panel titled, “How Can Economics Solve Its Race Problem.”

Rhonda Vonshay Sharpe and Omari H. Swinton standing in front of Howard University.
Rhonda Vonshay Sharpe, left, and Omari H. Swinton, right, are seen posing on the campus of Howard University. They discussed why economics still struggles with diversity.
Oscar Merrida IV

Omari H. Swinton, the chair of Howard University’s Department of Economics, who is both an alumni and the current director of the AEA summer program, as well as the past president of the National Economics Association, has observed that, “The vast majority of institutions in the US have never had a Black economist on staff, and the vast majority of schools have never graduated a Black PhD economist.”

Rhonda Vonshay Sharpe, the founder and president of the Women's Institute for Science, Equity, and Race (WISER), which is also a partner in next summer’s AEASP program, authored a research paper in 2019 that found that from 1966 to 2015, “the number of undergraduate economics degrees conferred to Black women was stagnant, and there was a decrease in the number of doctorates conferred to Black men.”

So why does the economics field still have such a massive disparity in minority representation? What needs to happen for systemic progress to be made? Amazon Science sat down with Sharpe and Swinton to ask those questions, as well as why Howard hosting the summer program is so significant, and what advice they would give to students considering economics as a major or profession. We also talked with Amazon chief economist Pat Bajari to find out why Amazon is sponsoring next summer’s AEASP program, and why he thinks diversity within the economics profession is essential.

A Howard University sign on the Howard campus
The AEASP will celebrate its 50th anniversary in 2024 at Howard University.
Oscar Merrida IV

Why does economics still have such a significant diversity problem?

Omari H. Swinton: I don't know that economics, as a profession, has really agreed that there's a problem. I think that's one of the big issues—we’ll say there's a problem, but nothing ever changes. You oftentimes hear people say things like, ‘We want to increase diversity’ but don't actually make any changes. They just say that that's something that they want to do.

It’s not as if these things haven't been out there. There are people out there who have dedicated their lives to bringing these types of issues to the forefront. I go back to Sandy Darity as an example. If you read from his earlier work, he's talking about these issues. Gregory Price has chronicled which institutions have Black economists in them. Rhonda has been looking at these issues for years.

Whether the economics profession is really ready to change is the issue. There have been a lot of people who have been talking about these issues for years. Others have come out and mentioned these problems more recently, but they ignore the fact that people have been talking about issues of underrepresentation for years.

Rhonda Vonshay Sharpe on the campus of Howard University
Rhonda Vonshay Sharpe says economics needs to define what diversity means. "If you don't define it, you can't measure it, or hold folks accountable."
Oscar Merrida IV

Rhonda Vonshay Sharpe: I narrow the problem down to be three things: 1) Economics has never defined what diversity means, and if you don't define it, you can't measure it, or hold folks accountable; 2) We don't have accurate data to track progress. We need to collect data that can be disaggregated by characteristics that have been used to limit participation in the profession. For example, when you talk about women, that usually means white women, and when we talk in terms of race, then you're really talking about men, and both of those descriptors are biased; and 3) As Omari said, there's enormous erasure happening. People have been doing this a long time, yet newcomers who have recently gotten tenure suddenly feel bad. They are handed a mic as if they are now the authorities. They don’t bother to understand whose shoulders they're standing on.

What needs to happen to address this problem? What role can academic institutions and companies like Amazon play?

Sharpe: I don't think the answer is to hire more Black economists. I really don't. And here is why: Because I think that when people say, ‘hire more Black economists’, people do just that, they hire Black economists. They do not think about whether or not those Black economists are bringing lived experiences that are going to help you craft policies to better interact with your customers.

One of the things I've been saying to folks recently is we need to talk more about structural classism and the ways in which we treat folks who are poor. So, it's not just about hiring Black economists, it's not about hiring Hispanic economists. It's about hiring folks who have lived experience in the US that will get at the inequality and related issues. That's not going to be solved just by hiring an economist because they are non-white.

Omari H. Swinton, the chair of Howard University's Department of Economics, on Howard's campus.
Omari H. Swinton says the AEASP program coming to Howard "is important because this is what our program is designed to do: increase minority participation in the economics profession."
Oscar Merrida IV

Swinton: If you say you want to diversify the profession, then stop looking at things that are not really problems. For example, there's not really a pipeline problem. You can ask almost any economics professor who teaches Principles of Economics, and most will tell you that is probably one of the worst classes to use if you want somebody to be interested in economics as a profession. But it really hasn't changed in years.

One change that we're making in the summer program is the experiential internship, or experiential learning. We’re going to place students with think tanks and corporations to actually see what an economist outside of the academy does. Everybody that gets a PhD in economics isn't going to be able to get a job as a professor. What does it look like to be an economist at Amazon? What does it look like to be an economist at the Census Bureau or at Brookings? Those are entirely different experiences. We’re trying to partner with as many different organizations as possible.

Hopefully we'll see change at those institutions, because students will come to the summer program, have that experience, and want to go back to those institutions. Rather than wanting to be a professor, they will, for example, say, ‘I want to be an economist at the Census Bureau, because I believe this research is important.’ It’s important for organizations, public and private, to be available to students, so they can see the type of experiences they can have if they work for you.

Pat Bajari
Pat Bajari, Amazon vice president and chief economist
Carl Clark, Amazon Imaging Studio

Pat Bajari: As an economist, I have always thought of this is in terms of diminishing returns. If you always have the same type of viewpoint, and keep hiring replicas of that viewpoint, the returns you get from that eventually decrease. Having different viewpoints allows you to do better work. And because we serve a large and diverse base of customers, we have a large and diverse base of problems. We want to take a leading role in supporting a new generation of economists from underrepresented minorities—it is not only the right thing to do, but it will also help bring strong and diverse voices that will create an even more inclusive customer experience.

When individuals come from different backgrounds, they bring different perspectives to the table. You do better work when you have different perspectives.
Pat Bajari

Swinton: One thing organizations can do is find programs that are actually successful at achieving the types of goals they’re pursuing. For example, some of the research done by Becker et al. shows that about 20 percent of Blacks that have PhDs in economics have attended the AEASP program. By helping support Howard in hosting AEASP in this first year, Amazon is doing that. Without Amazon’s support, Howard wouldn't be able to host the AEA summer program at all. We certainly hope others will follow Amazon’s lead.

What is the significance of the summer program coming to Howard?

Swinton: The summer program is extremely important in my path as an economist. My first cohort of economists were the people that I met through the summer program. Howard is the number one producer as an undergraduate feeder of Blacks who go on to get PhDs in economics. This is our mission and one of our goals as an institution and as a department, and I think the AEA summer program coming to Howard is important because this is what our program is designed to do: increase minority participation in the economics profession.

The National Economics Association summer program came out of Marcus Alexis’ mind as a program to help get minorities interested in economics. For the AEASP program to come to Howard at this point in time is a great honor. It’s an honor to be the first HBCU to host the summer program.

Sharpe: I'm excited to see a program that's going to be led by Blacks, which I think is incredibly important, as the program will celebrate 50 years while it's at Howard in 2024. It just feels full circle in terms of thinking about Marcus Alexis, who was a Black economist, and then having the program 50 years later be at an institution that is the number one producer of Black economists. That's incredibly exciting.

Finally, what advice would you give to someone considering whether to pursue a degree in economics? Why is economics such an important field?

Bajari: A lot of economics is understanding people's material wellbeing. Who has low wages? Who has high wages? If you take a given policy, whether that's central bank policy or interventions into labor markets, etcetera, these things deeply, deeply, deeply affect people's lives, people's material outcomes. What they can purchase and where they can live and where they can send their kids to school and so forth. It's an important set of questions, and they range from micro things about what happens to the individual, to macro things, such as how the whole world is evolving and changing in response to things like COVID-19.

Howard University's Founders Library
Howard University's Founders Library is seen here. Howard is the first Black college to host AEASP.
Oscar Merrida IV

If we change policy or somebody goes to college versus doesn't go to college, what are the implications of those economic variables? I know this is what attracted me to economics. As a young person, growing up pretty poor in rural Minnesota, I was interested in the world and how it worked. And I liked economics because it brought math and data and scientific formalism to those questions. That's not the only way you can look at those questions, or the only way you should look at them, but it’s one way that's highly useful.

Sharpe: For students pursuing a PhD in economics, my main advice is to pick a PhD program that's a good fit for you. Many students think that if you don't go to a top program, you can't have a successful career. That’s not true. I went to Claremont Graduate University, not highly ranked, but I had an amazing time as a graduate student. I loved it. My mentee when I was in graduate school was Olugbenga Ajilore who’s at CAP (Center for American Progress) now, who is a rock star right now in terms of being in the news and asking people to think about rural communities. He and I didn't go to top economics departments, but we went to places that were good fits for us, and that's incredibly important.

Bajari: “Technology economics” is a booming field. The largest conference held by the National Association of Business Economists is now the tech economics conference. It’s larger than their annual conference now, because it's been an explosive area of job growth for young people. We are one of the larger private sector employers of economists. When you're in that role, you have an obligation to demonstrate leadership. We saw sponsorship of AEASP as an opportunity to expose young PhDs to this emerging field. I thought Howard was very thoughtful about their proposal, and I'm hoping AEASP can help us establish a pipeline of highly qualified candidates.

Swinton: I talk to students about this all the time. You want to make a change, and you want to be a policy maker? Be an economist. You want to go into business and work on Wall Street, make a lot of money? Be an economist. Economics is one of the most useful majors because it allows you the opportunity you to go out and do a wide variety of things based on the basic training you obtain.

Applications for the summer program are open and the deadline to apply is January 31, 2021. To apply, visit economics.howard.edu/aeasp. The program will be held May 27 to July 25, 2021, and be offered in Washington, D.C., contingent upon COVID-19 restrictions.

Research areas

Related content

US, VA, Arlington
We are seeking an exceptional Data Scientist to join our team in PXT Central Science. The ideal candidate will thrive in a dynamic, multifaceted role where you'll translate complex business challenges into rigorous quantitative frameworks, extract actionable insights from structured and unstructured datasets, and architect science-backed, scalable solutions that elevate the experience of our 1 million+ employees worldwide. If you're energized by the opportunity to apply data science to our mission of making Amazon Earth's Best Employer, we want to hear from you. Key job responsibilities • Own the design, development, and maintenance of scalable models and prototypes leveraging statistical, machine learning, or GenAI methodologies to enhance employee experience. • Partner with scientists, engineers, and product leaders to solve for employee experience defects using scientific approaches, building new services and tools that deliverable measurable impact. • Author and maintain detailed technical documentation related to the projects you drive. • Communicate results to diverse audiences of varying technical background with effective writing, visualizations, and presentations • Stay current with emerging methods and technologies, and implement them strategically to amplify the team’s impact. About the team The Central Science Team within Amazon’s People Experience and Technology org (PXTCS) uses economics, behavioral science, statistics, machine learning, and Generative AI to proactively identify mechanisms and process improvements which simultaneously improve Amazon and the lives, well-being, and the value of work to Amazonians. We are an interdisciplinary team, which combines the talents of science, engineering, and UX to develop and deliver solutions that measurably achieve this goal.
US, WA, Bellevue
The Amazon Fulfillment Technologies (AFT) Science team is looking for an exceptional Applied Scientist, with strong optimization and analytical skills, to develop production solutions for one of the most complex systems in the world: Amazon’s Fulfillment Network. At AFT Science, we design, build and deploy optimization, simulation, and machine learning solutions to power the production systems running at world wide Amazon Fulfillment Centers. We solve a wide range of problems that are encountered in the network, including labor planning and staffing, demand prioritization, pick assignment and scheduling, and flow process optimization. We are tasked to develop innovative, scalable, and reliable science-driven solutions that are beyond the published state of art in order to run frequently (ranging from every few minutes to every few hours per use case) and continuously in our large scale network. Key job responsibilities As an Applied Scientist, you will work with other scientists, software engineers, product managers, and operations leaders to develop scientific solutions and analytics using a variety of tools and observe direct impact to process efficiency and associate experience in the fulfillment network. Key responsibilities include: * Develop an understanding and domain knowledge of operational processes, system architecture and functions, and business requirements * Deep dive into data and code to identify opportunities for continuous improvement and/or disruptive new approach * Develop scalable mathematical models for production systems to derive optimal or near-optimal solutions for existing and new challenges * Create prototypes and simulations for agile experimentation of devised solutions * Advocate technical solutions to business stakeholders, engineering teams, and senior leadership * Partner with engineers to integrate prototypes into production systems * Design experiment to test new or incremental solutions launched in production and build metrics to track performance About the team Amazon Fulfillment Technology (AFT) designs, develops and operates the end-to-end fulfillment technology solutions for all Amazon Fulfillment Centers (FC). We harmonize the physical and virtual world so Amazon customers can get what they want, when they want it. The AFT Science team has expertise in operations research, optimization, scheduling, planning, simulation, and machine learning. We also have domain expertise in the operational processes within the FCs and their defects. We prioritize advancements that support AFT tech teams and focus areas rather than specific fields of research or individual business partners. We influence each stage of innovation from inception to deployment which includes both developing novel solutions or improving existing approaches. Resulting production systems rely on a diverse set of technologies, our teams therefore invest in multiple specialties as the needs of each focus area evolves.
US, WA, Seattle
We are seeking an exceptional Data Scientist to join our team in PXT Central Science. The ideal candidate will thrive in a dynamic, multifaceted role where you'll translate complex business challenges into rigorous quantitative frameworks, extract actionable insights from structured and unstructured datasets, and architect science-backed, scalable solutions that elevate the experience of our 1 million+ employees worldwide. If you're energized by the opportunity to apply data science to our mission of making Amazon Earth's Best Employer, we want to hear from you. Key job responsibilities • Own the design, development, and maintenance of scalable models and prototypes leveraging statistical, machine learning, or GenAI methodologies to enhance employee experience. • Partner with scientists, engineers, and product leaders to solve for employee experience defects using scientific approaches, building new services and tools that deliverable measurable impact. • Author and maintain detailed technical documentation related to the projects you drive. • Communicate results to diverse audiences of varying technical background with effective writing, visualizations, and presentations • Stay current with emerging methods and technologies, and implement them strategically to amplify the team’s impact. About the team The Central Science Team within Amazon’s People Experience and Technology org (PXTCS) uses economics, behavioral science, statistics, machine learning, and Generative AI to proactively identify mechanisms and process improvements which simultaneously improve Amazon and the lives, well-being, and the value of work to Amazonians. We are an interdisciplinary team, which combines the talents of science, engineering, and UX to develop and deliver solutions that measurably achieve this goal.
US, WA, Bellevue
Alexa International is looking for a passionate, talented, and inventive Applied Scientist to help build industry-leading technology with Large Language Models (LLMs) and multimodal systems, requiring strong deep learning and generative models knowledge. You will contribute to developing novel solutions and deliver high-quality results that impact Alexa's international products and services. Key job responsibilities As an Applied Scientist with the Alexa International team, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art with LLMs. Your work will directly impact our international customers in the form of products and services that make use of digital assistant technology. You will leverage Amazon's heterogeneous data sources, unique and diverse international customer nuances and large-scale computing resources to accelerate advances in text, voice, and vision domains in a multimodal setup. The ideal candidate possesses a solid understanding of machine learning, natural language understanding, modern LLM architectures, LLM evaluation & tooling, and a passion for pushing boundaries in this vast and quickly evolving field. They thrive in fast-paced environments to tackle complex challenges, excel at swiftly delivering impactful solutions while iterating based on user feedback, and collaborate effectively with cross-functional teams. A day in the life * Analyze, understand, and model customer behavior and the customer experience based on large-scale data. * Build novel online & offline evaluation metrics and methodologies for multimodal personal digital assistants. * Fine-tune/post-train LLMs using techniques like SFT, DPO, RLHF, and RLAIF. * Set up experimentation frameworks for agile model analysis and A/B testing. * Collaborate with partner teams on LLM evaluation frameworks and post-training methodologies. * Contribute to end-to-end delivery of solutions from research to production, including reusable science components. * Communicate solutions clearly to partners and stakeholders. * Contribute to the scientific community through publications and community engagement.
US, WA, Bellevue
Amazon’s Last Mile Team is looking for a passionate individual with strong optimization and analytical skills to join its Last Mile Science team in the endeavor of designing and improving the most complex planning of delivery network in the world. Last Mile builds global solutions that enable Amazon to attract an elastic supply of drivers, companies, and assets needed to deliver Amazon's and other shippers' volumes at the lowest cost and with the best customer delivery experience. Last Mile Science team owns the core decision models in the space of jurisdiction planning, delivery channel and modes network design, capacity planning for on the road and at delivery stations, routing inputs estimation and optimization. Our research has direct impact on customer experience, driver and station associate experience, Delivery Service Partner (DSP)’s success and the sustainable growth of Amazon. Optimizing the last mile delivery requires deep understanding of transportation, supply chain management, pricing strategies and forecasting. Only through innovative and strategic thinking, we will make the right capital investments in technology, assets and infrastructures that allows for long-term success. Our team members have an opportunity to be on the forefront of supply chain thought leadership by working on some of the most difficult problems in the industry with some of the best product managers, scientists, and software engineers in the industry. Key job responsibilities Candidates will be responsible for developing solutions to better manage and optimize delivery capacity in the last mile network. The successful candidate should have solid research experience in one or more technical areas of Operations Research or Machine Learning. These positions will focus on identifying and analyzing opportunities to improve existing algorithms and also on optimizing the system policies across the management of external delivery service providers and internal planning strategies. They require superior logical thinkers who are able to quickly approach large ambiguous problems, turn high-level business requirements into mathematical models, identify the right solution approach, and contribute to the software development for production systems. To support their proposals, candidates should be able to independently mine and analyze data, and be able to use any necessary programming and statistical analysis software to do so. Successful candidates must thrive in fast-paced environments, which encourage collaborative and creative problem solving, be able to measure and estimate risks, constructively critique peer research, and align research focuses with the Amazon's strategic needs.
US, WA, Bellevue
Alexa International is looking for a passionate, talented, and inventive Applied Scientist to help build industry-leading technology with Large Language Models (LLMs) and multimodal systems, requiring strong deep learning and generative models knowledge. You will contribute to developing novel solutions and deliver high-quality results that impact Alexa's international products and services. Key job responsibilities As an Applied Scientist with the Alexa International team, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art with LLMs. Your work will directly impact our international customers in the form of products and services that make use of digital assistant technology. You will leverage Amazon's heterogeneous data sources, unique and diverse international customer nuances and large-scale computing resources to accelerate advances in text, voice, and vision domains in a multimodal setup. The ideal candidate possesses a solid understanding of machine learning, natural language understanding, modern LLM architectures, LLM evaluation & tooling, and a passion for pushing boundaries in this vast and quickly evolving field. They thrive in fast-paced environments to tackle complex challenges, excel at swiftly delivering impactful solutions while iterating based on user feedback, and collaborate effectively with cross-functional teams. A day in the life * Analyze, understand, and model customer behavior and the customer experience based on large-scale data. * Build novel online & offline evaluation metrics and methodologies for multimodal personal digital assistants. * Fine-tune/post-train LLMs using techniques like SFT, DPO, RLHF, and RLAIF. * Set up experimentation frameworks for agile model analysis and A/B testing. * Collaborate with partner teams on LLM evaluation frameworks and post-training methodologies. * Contribute to end-to-end delivery of solutions from research to production, including reusable science components. * Communicate solutions clearly to partners and stakeholders. * Contribute to the scientific community through publications and community engagement.
US, CA, Pasadena
The Amazon Web Services (AWS) Center for Quantum Computing (CQC) is a multi-disciplinary team of theoretical and experimental physicists, materials scientists, and hardware and software engineers on a mission to develop a fault-tolerant quantum computer. Throughout your internship journey, you'll have access to unparalleled resources, including state-of-the-art computing infrastructure, cutting-edge research papers, and mentorship from industry luminaries. This immersive experience will not only sharpen your technical skills but also cultivate your ability to think critically, communicate effectively, and thrive in a fast-paced, innovative environment where bold ideas are celebrated. Join us at the forefront of applied science, where your contributions will shape the future of Quantum Computing and propel humanity forward. Seize this extraordinary opportunity to learn, grow, and leave an indelible mark on the world of technology. Amazon has positions available for Quantum Research Science and Applied Science Internships in Santa Clara, CA and Pasadena, CA. We are particularly interested in candidates with expertise in any of the following areas: superconducting qubits, cavity/circuit QED, quantum optics, open quantum systems, superconductivity, electromagnetic simulations of superconducting circuits, microwave engineering, benchmarking, quantum error correction, fabrication, etc. Key job responsibilities In this role, you will work alongside global experts to develop and implement novel, scalable solutions that advance the state-of-the-art in the areas of quantum computing. You will tackle challenging, groundbreaking research problems, work with leading edge technology, focus on highly targeted customer use-cases, and launch products that solve problems for Amazon customers. The ideal candidate should possess the ability to work collaboratively with diverse groups and cross-functional teams to solve complex business problems. A successful candidate will be a self-starter, comfortable with ambiguity, with strong attention to detail and the ability to thrive in a fast-paced, ever-changing environment. About the team Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Hybrid Work We value innovation and recognize this sometimes requires uninterrupted time to focus on a build. We also value in-person collaboration and time spent face-to-face. Our team affords employees options to work in the office every day or in a flexible, hybrid work model near one of our U.S. Amazon offices.
US, WA, Bellevue
Alexa International Science team is looking for a passionate, talented, and inventive Senior Applied Scientist to help build industry-leading technology with Large Language Models (LLMs) and multimodal systems, requiring strong deep learning and generative models knowledge. At this level, you will drive cross-team scientific strategy, influence partner teams, and deliver solutions that have broad impact across Alexa's international products and services. Key job responsibilities As a Senior Applied Scientist with the Alexa International team, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art with LLMs, particularly delivering industry-leading scientific research and applied AI for multi-lingual applications — a challenging area for the industry globally. Your work will directly impact our global customers in the form of products and services that support Alexa+. You will leverage Amazon's heterogeneous data sources and large-scale computing resources to accelerate advances in text, speech, and vision domains. The ideal candidate possesses a solid understanding of machine learning, speech and/or natural language processing, modern LLM architectures, LLM evaluation & tooling, and a passion for pushing boundaries in this vast and quickly evolving field. They thrive in fast-paced environment, like to tackle complex challenges, excel at swiftly delivering impactful solutions while iterating based on user feedback, and are able to influence and align multiple teams around a shared scientific vision.
US, WA, Bellevue
Amazon is seeking a Language Data Scientist to join the Alexa International science team as domain expert. This role focuses on expanding analysis and evaluation of conversational interaction data deliverables. The Language Data Scientist is an expert in conversation assessment processes, working closely with a team of skilled machine learning scientists and engineers, and is a key member in developing new conventions for relevant annotation workflows. The Language Data Scientist will be own unique data analysis and research requests that support the training and evaluation of LLMs and machine learning models, and the overall processing of a data collection. Key job responsibilities To be successful in this role, you must have a passion for data, efficiency, and accuracy. Specifically, you will: - Own data analyses for customer-facing features, including launch go/no-go metrics for new features and accuracy metrics for existing features - Handle unique data analysis requests from a range of stakeholders, including quantitative and qualitative analyses to elevate customer experience with speech interfaces - Lead and evaluate changing dialog evaluation conventions, test tooling developments, and pilot processes to support expansion to new data areas - Continuously evaluate workflow tools and processes and offer solutions to ensure they are efficient, high quality, and scalable - Provide expert support for a large and growing team of data analysts - Provide support for ongoing and new data collection efforts as a subject matter expert on conventions and use of the data - Conduct research studies to understand speech and customer-Alexa interactions - Collaborate with scientists and product managers, and other stakeholders in defining and validating customer experience metrics
US, WA, Bellevue
Alexa International is looking for a passionate, talented, and inventive Applied Scientist to help build industry-leading technology with Large Language Models (LLMs) and multimodal systems, requiring strong deep learning and generative models knowledge. You will contribute to developing novel solutions and deliver high-quality results that impact Alexa's international products and services. Key job responsibilities As an Applied Scientist with the Alexa International team, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art with LLMs. Your work will directly impact our international customers in the form of products and services that make use of digital assistant technology. You will leverage Amazon's heterogeneous data sources, unique and diverse international customer nuances and large-scale computing resources to accelerate advances in text, voice, and vision domains in a multimodal setup. The ideal candidate possesses a solid understanding of machine learning, natural language understanding, modern LLM architectures, LLM evaluation & tooling, and a passion for pushing boundaries in this vast and quickly evolving field. They thrive in fast-paced environments to tackle complex challenges, excel at swiftly delivering impactful solutions while iterating based on user feedback, and collaborate effectively with cross-functional teams. A day in the life * Analyze, understand, and model customer behavior and the customer experience based on large-scale data. * Build novel online & offline evaluation metrics and methodologies for multimodal personal digital assistants. * Fine-tune/post-train LLMs using techniques like SFT, DPO, RLHF, and RLAIF. * Set up experimentation frameworks for agile model analysis and A/B testing. * Collaborate with partner teams on LLM evaluation frameworks and post-training methodologies. * Contribute to end-to-end delivery of solutions from research to production, including reusable science components. * Communicate solutions clearly to partners and stakeholders. * Contribute to the scientific community through publications and community engagement.