Social
Conversational AI

The life of a prescription at Amazon Pharmacy

From pricing estimation and regulatory compliance to inventory management and chatbot assistants, machine learning models help Amazon Pharmacy customers stay healthy and save time and money.

By Alexandre Alves, Anita Vila
September 30, 2024
8 min read
Share

Pharmacies play a vital role in ensuring patients’ health, but the process of dispensing medications is far more complex than it may appear. At Amazon Pharmacy, we are using artificial intelligence (AI) and cutting-edge technologies to remove this complexity and improve patients’ experiences.

The pharmacy challenge

When a prescription arrives at a pharmacy, its details must be entered into the pharmacy's software system. Then, a licensed pharmacist reviews the prescription to verify the patient's information, check for potential drug interactions or allergies, and confirm that the prescribed medication, dosage, and instructions are appropriate and accurate.

This process is susceptible to errors — even if the prescription arrives electronically. A U.S. study estimated that there are approximately 51.5 million dispensing errors annually in community pharmacies, with a meta-analysis supporting an error rate of around 1.5%.

Sunita Mishra, left, chief medical officer at Amazon Health Services, listens as Katrina Armstrong, right, dean of Columbia's Faculties of Health Sciences and the Vagelos College of Physicians and Surgeons, speaks at an October 2023 symposium hosted by Columbia University. Both are sitting, Katrina is holding a mic and gesturing with her left hand, there is also a person somewhat visible in the foreground.
Related content
How Amazon and Columbia University are collaborating to advance AI in healthcare
Amazon Health Services' Sunita Mishra and Columbia University’s Katrina Armstrong discuss technology's potential role in medical settings.

Pharmacies must also handle billing and insurance claims for their patients. These are involved calculations based on patients’ specific insurance policies, their copay responsibilities, and the rates that the pharmacy has negotiated with different insurance providers. In fact, just identifying a patient's insurance can be challenging. The result is that patients often do not know the prices they will pay for their medications until the end of the process, when they’re picking up their prescriptions at a retail pharmacy or checking out online.

After a prescription has been validated and the purchase completed, the pharmacy staff must locate the specific medication in their inventory. However, it's possible that at this stage, the prescribed medication, the required strength, or the preferred brand may no longer be available. Providing a substitute could necessitate contacting the prescribing physician again for approval. Additionally, depending on the substitution, the billing and insurance process may need to be re-initiated to account for any changes in pricing or coverage.

Once the medication is ready for dispensing, the pharmacist provides the patient with detailed instructions on how to properly take it. Patients may also have questions regarding their insurance coverage or costs. However, these conversations often take place in public areas, which can be uncomfortable for patients who have personal or sensitive questions.

Finally, patients need convenient access to pharmacists at any time, day or night. This allows patients to report how they are feeling while taking their medications, which can help pharmacists provide better guidance and support throughout the treatment process.

The AI-powered pharmacy

Amazon Pharmacy uses large language models (LLMs) to enhance the accuracy, safety, and speed of prescription processing. First, we use LLMs to transcribe raw prescription data into structured, standardized formats that are seamlessly processed by software and more easily understood by patients. For example, medical abbreviations like "PRN" and "QID" are transformed into their full-text equivalents, such as "take as needed" and "take four times a day," respectively.

After standardizing the prescription data, the system performs a validation step that includes checking the medication names, dosage forms, strengths, and directions for use against an industry database. After validation, all prescriptions are still carefully reviewed and verified by licensed pharmacists. By leveraging this automated process, Amazon Pharmacy has reduced the number of near-miss events (potential medication errors) by 50% and improved processing speed by up to 90%. This allows our pharmacists to focus their time and attention on critical tasks, such as providing personalized care and addressing complex medication-related issues.

Pharmacy.png
The AI workflow at Amazon Pharmacy.

Amazon Pharmacy understands the importance of price transparency for customers. When a patient is using insurance to cover the cost of medication, Amazon Pharmacy will first try to obtain the exact price directly from the insurance provider. However, if this real-time pricing information is not available, Amazon Pharmacy will provide an estimated out-of-pocket cost for the patient's copay, without requiring the customer to go through the entire checkout process first.

To generate accurate price estimates, Amazon Pharmacy uses an ensemble of decision-tree-based models. These models take into account factors such as historical claims data (time series features) and static information such as the specific medication, the number-of-days' supply, and the quantity prescribed. By providing upfront pricing information, either the exact cost or a reliable estimate, Amazon Pharmacy aims to increase transparency and help patients understand their out-of-pocket expenses before committing to purchases. Additionally, Amazon Pharmacy searches for applicable industry coupons and automatically applies them to orders. We also use ML to validate the patient's insurance registration and claim requests to insurance providers.

Amazon is known for its extensive logistics and fulfillment capabilities, and Amazon Pharmacy takes advantage of Amazon's vast network of same-day and local delivery facilities, as well as innovative transportation methods like Prime Air drones. Additionally, Amazon Pharmacy employs specialized automation technologies, such as robotic vial-filling systems, to streamline the medication-dispensing process, enabling prompt delivery of medications to patients across the nation.

Beyond the physical logistics infrastructure, Amazon Pharmacy has developed its own order fulfillment system to handle complex medication-routing and -dispensing logic, while ensuring compliance with over 160 different pharmacy regulatory bodies across the United States. For example, if a medication for an order is no longer available at the closest fulfillment center, Amazon Pharmacy can identify the next best eligible facility to fulfill the order, even if it's in a different state, provided that the relevant state regulations allow for such cross-state fulfillment.

Marinka Zitnik.jpg
Related content
Cracking the code of how diseases affect the body
ARA recipient Marinka Zitnik is focused on how machine learning can enable accurate diagnoses and the development of new treatments and therapies.

Moreover, Amazon Pharmacy's order fulfillment algorithm takes into account regional variations in insurance eligibility. In such cases, Amazon Pharmacy will first validate that there are no changes to the patient's copay. If changes are required, Amazon Pharmacy will work with the insurance providers to clarify the benefits. To accomplish all of this, Amazon Pharmacy's order fulfillment solution employs a combination of operations research techniques, such as optimization solvers, and deep-learning models such as variational autoencoders and diffusion models. These models help simulate different scenarios and optimize the fulfillment process to ensure efficient and compliant delivery of medications to patients.

Amazon Pharmacy also introduced personalized AI-powered chatbots to assist users. These virtual assistants can answer frequently asked questions about Amazon Pharmacy, such as how to enroll in the service. In a first for the industry, Amazon Pharmacy's chatbot also provides personalized support, allowing patients to ask questions about their medication orders, delivery status, prescription transfers, and inventory availability. If a patient prefers, there is always 24/7 access to direct pharmacist support and the customer care team.

Implementing a personalized AI chatbot in the healthcare setting is a complex task. It's crucial to safeguard patients' privacy and ensure the highest level of accuracy, avoiding LLM hallucinations. To address these challenges, Amazon Pharmacy has enhanced the typical retrieval-augmented generation (RAG) approach used for LLM chatbots. The enhancements include input and output guardrails, the use of ensembles of specialized (mini) AI models, and a continuous model improvement process through reinforcement learning using human feedback (RLHF).

The digital pharmacy counter

Amazon Pharmacy is leveraging ML and optimization algorithms to streamline the complex process of dispensing medications. By addressing long-standing challenges such as data entry errors, lack of price transparency, intelligent nationwide medication fulfillment, and personalized-AI-based experiences, Amazon Pharmacy enables patients to save time, save money, and stay healthy.

Gari Clifford is the tenured chair of the Department of Biomedical Informatics at Emory University and a professor of biomedical engineering at Georgia Institute of Technology. Clifford, an Amazon Research Award recipient, is seen here speaking at Emory University.
Related content
Using machine learning to reduce costs, increase accuracy, and improve access in healthcare
Gari Clifford, the chair of the Department of Biomedical Informatics at Emory University and an Amazon Research Award recipient, wants to transform healthcare.

For example, it is not uncommon for medications to go into short-term back order (STBO), especially new medications entering the market, as was recently the case with the GLP-1 line of medications used for diabetes and weight loss. Amazon Pharmacy’s intelligent-fulfillment solution has enabled an 85% decrease in delivery estimate misses for unforeseen reasons, including STBOs.

The AI-powered Amazon Pharmacy assistant helps customers navigate the complexities of the pharmacy industry, providing 24/7 assistance on topics like prescription tracking, insurance coverage, medication availability, and cost-saving strategies. Half of the customers who interact with the assistant don't require additional human support, which saves them time and effort. (For customers who still need assistance, Amazon Pharmacy likewise provides 24/7 access to pharmacist support.) Additionally, the assistant provides real-time medication transfer or shipment status updates in response to patient queries, handling follow-up questions to recommend next steps.

For all the success of our AI-based systems, however, Amazon Pharmacy’s research and engineering teams remain hard at work. We will continue pushing the envelope in scaling medication dispensing, improving personalized AI-based chatbots and assistants, and transitioning toward a longitudinal pharmacy that is proactively looking out for patients.

Research areas
Tags
About the Author
Alexandre Alves
Alexandre Alves is a senior principal engineer with Amazon Pharmacy.
Anita Vila
Anita Vila is a senior manager, product, with Amazon Pharmacy.

Related content

Work with us

See more jobs See more jobs
Senior Applied Scientist, Annapurna ML
US, WA, Seattle
The Annapurna ML team is looking for a Senior Applied Scientist to work on the intersection of Artificial Intelligence and program analysis to raise the code quality bar in our state-of-the-art deep learning compiler stack. This stack is designed to optimize application models across diverse domains, including Large Language and Vision, originating from leading frameworks such as PyTorch, TensorFlow, and JAX. Your role will involve working closely with our custom-built Machine Learning accelerators, Inferentia and Trainium, which represent the forefront of Annapurna innovation for advanced ML capabilities, and is the underpinning of Generative AI. As a Senior Applied Scientist, you'll be instrumental in designing, developing, and deploying analyzers for ML compiler stages and compiler IRs. You will architect and implement business-critical tooling, publish research, and mentor a brilliant team of experienced scientists and engineers. You will need to be technically capable, credible, and curious in your own right as a trusted scientist, innovating on behalf of our customers. Your responsibilities will involve tackling crucial challenges alongside a talented engineering team, contributing to leading-edge design and research in compiler technology and deep-learning systems software. Strong experience in programming languages, compilers, program analyzers, and program synthesis engines will be a benefit in this role. A background in machine learning and AI accelerators is preferred but not required. A day in the life Diverse Experiences Amazon values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why Amazon? We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at Amazon, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
Senior Applied Scientist, Perimeter Protection Applied Science
US, WA, Seattle
Join the AWS Perimeter Protection team as a Senior Applied Scientist, where you will bring your deep ML engineering expertise to design, build, and scale AI-driven security solutions that protect AWS customers worldwide. This role is ideal for someone who has already built and shipped production ML systems at industry scale and is looking to apply that experience to high-impact security challenges. You will own the full ML lifecycle — from research and prototyping to production deployment and optimization — powering services including Web Application Firewall, DDoS Protection, Bot Management, and Infrastructure Protection. With services spanning all AWS regions and handling trillions of requests per week, you will solve complex engineering and science problems where model performance, system reliability, and low-latency inference are critical. Key job responsibilities - Design, build, and deploy production-grade ML models and systems for real-time threat detection, mitigation, and protection against evolving cyber threats at cloud scale. - Own the full ML lifecycle end-to-end — from problem formulation, data engineering, and model development through to production deployment, monitoring, and continuous improvement. - Architect and optimize ML pipelines, training infrastructure, and serving systems to meet strict latency, throughput, and reliability requirements at AWS scale. - Bridge the gap between research and production by translating novel ML approaches into robust, scalable, and maintainable systems that operate in real-time security environments. - Design and implement feature engineering workflows and large-scale data processing pipelines to support rapid experimentation and reliable model iteration. - Collaborate closely with software engineering teams to integrate ML models into distributed, low-latency security services, driving engineering decisions around model serving, infrastructure, and system design. - Analyze large-scale production data to identify patterns, anomalies, and emerging threat vectors, and translate findings into measurable improvements to detection and mitigation capabilities. - Establish and improve best practices for ML system design, model evaluation, A/B testing, and production monitoring across the team. - Mentor junior scientists and engineers, raising the bar on both scientific rigor and engineering quality.
Applied Scientist, Neuron ARG, Annapurna ML
US, WA, Seattle
The Automated Reasoning Group in the Amazon Neuron team is looking for an Applied Scientist to work on the intersection of Artificial Intelligence and program analysis to raise the code quality bar in our state-of-the-art deep learning compiler stack. This stack is designed to optimize application models across diverse domains, including Large Language and Vision, originating from leading frameworks such as PyTorch and JAX. Your role will involve working closely with our custom-built Machine Learning accelerator, Trainium, which represents the forefront of innovation for advanced ML capabilities, and is the underpinning of Generative AI. In this role as an Applied Scientist, you'll be instrumental in designing, developing, and deploying analyzers for ML compiler stages and compiler IRs. You will architect and implement business-critical tooling, publish research, and mentor a brilliant team of experienced scientists and engineers. You will need to be technically capable, credible, and curious in your own right as a trusted AWS Neuron engineer, innovating on behalf of our customers. Your responsibilities will involve tackling crucial challenges alongside a talented engineering team, contributing to leading-edge design and research in compiler technology and deep-learning systems software. Strong experience in programming languages, compilers, program analyzers, theorem provers, and program synthesis engines will be a benefit in this role. A background in machine learning and AI accelerators is preferred but not required.
Applied Scientist, Delivery Networks and Tech
IN, KA, Bengaluru
Customer addresses, Geospatial information and Road-network play a crucial role in Amazon Logistics' Delivery Planning systems. We own exciting science problems in the areas of Address Normalization, Geocode learning, Maps learning, Time estimations including route-time, delivery-time, transit-time predictions which are key inputs in delivery planning. As part of the Science team within DNT (Last Mile), you will partner closely with other scientists and engineers in a collegial environment to develop enterprise ML solutions with a clear path to business impact. The setting also gives you an opportunity to think about a complex large-scale problem for multiple years and building increasingly sophisticated solutions year over year. In the process there will be opportunity to innovate, explore SOTA and publish the research in internal and external ML conferences. Successful candidates will have deep knowledge of competing machine learning methods for large scale predictive modelling, natural language processing, semi-supervised & graph based learning. We also look for the experience to graduate prototype models to production and the communication skills to explain complex technical approaches to the stakeholders of varied technical expertise. Here is a glimpse of the problem spaces and technologies that we deal with on a regular basis: 1. De-duping and organizing addresses into hierarchy while handling noisy, inconsistent, localized and multi-lingual user inputs. We do this at the scale of millions of customers for established (US, EU) as well as emerging geographies (IN, MX). We make use of technologies like LLMs, Weak supervision, Graph-based clustering & Entity matching. We also use additional modalities such as building outlines in maps, street view images and 3P datasets, gazetteers. 2. Build a generic ML framework which leverages relationship between places to improve delivery experience by learning precise delivery locations and propagating attributes, such as business hours and safe places. 3. (Work done in sister teams) Developing systems to consume inputs from areal imagery and optimize our maps to enable efficient delivery planning. Also building models to estimate travel time, turn costs, optimal route and defect propensities. Key job responsibilities As an Applied Scientist I, your responsibility will be to deliver on a well defined but complex business problem, explore SOTA technologies including GenAI and customize the large models as suitable for the application. Your job will be to work on a end-to-end business problem from design to experimentation and implementation. There is also an opportunity to work on open ended ML directions within the space and publish the work in prestigious ML conferences. About the team Last Mile Address Intelligence (LMAI) team owns WW charter for address and location learning solutions which are crucial for efficient Last Mile delivery planning. The team works out of Hyderabad and Bangalore offices in India. LMAI is a part of the DNT Science team, which also owns problems in the space of maps learning and travel time estimations. The rest of the DNT Science team and senior leadership of Last Mile org works out of Bellevue office.
Research Science Manager, Safe Autonomy, SAF Lab
US, CA, Pasadena
We are seeking a Research Science Manager to join the SAF Lab. In this role, you will lead a team of world-class applied scientists, engineers, post-docs and interns developing the next generation of safe autonomy on highly dynamic robotic systems. You will drive technical vision, research strategy, and ensure your team's innovations translate into production systems that operate at Amazon scale. You will interface with top academic researchers at the forefront of safe autonomy, and interdisciplinary teams across Amazon working on autonomous mobile robots, mobile manipulators, and dynamically stable robots. You will bridge academic research with real-world deployable safety layers that enable robots to safely operate around humans. Key job responsibilities • Manage a team of scientists and engineers developing a universal safety layer for robotic systems, with a focus on the next generation of robots • Work with leadership to design and execute multi-year research roadmaps for the development of safe autonomy that spans all types of current and emerging robotic platforms • Lead Scientists, Engineers, post-docs and interns to realize research and development goals and provide evidence of these results in a variety of formats • Drive integration of control barrier functions with planning, perception and learning while maintaining guarantees of safe high-performance robot behavior • Collaborate with product teams and science leaders to set a science roadmap (with eventual impact on real robots). • Publish research findings at top-tier conferences and contribute to the broader robotics community • Build relationships with academic and industry partners to stay at the forefront of safe autonomy A day in the life Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! About the team Work with the inventor of control barrier functions in the Safe Autonomy Frontiers (SAF) Lab. The first industry research lab in safe autonomy, developing a universal safety layer for the next generation of robotic systems: mobile robots, manipulators, mobile manipulators, and future platforms with dynamic stability. You will push the frontiers of performant safety for highly dynamic robots: CBF theory integrated with perception and learning, evaluated on next-generation robots. Your work will underpin robots operating alongside people at Amazon's unprecedented scale.
Research Scientist II, Global Logistics, Applied Science Team
US, WA, Bellevue
Transform global logistics by developing innovative technological solutions that optimize complex international shipping networks. Join a dynamic team where your advanced scientific expertise will directly impact millions of packages moving across borders, solving intricate transportation challenges through applied science. Key job responsibilities - Develop advanced deep learning and generative AI applications for sophisticated product classification systems - Lead complex modeling analyses to support strategic business decision-making and policy development - Partner with engineering and operations teams to design transformative machine learning solutions for transportation challenges - Design and document technical strategies and applied science roadmaps - Innovate and provide technological direction in international imports and exports logistics A day in the life Your workday will be an exciting blend of data exploration, innovative problem-solving, and collaborative technology development. You'll dive into complex datasets, design sophisticated machine learning models, engage with cross-functional teams, and create technological solutions that reshape international shipping capabilities. About the team We are a passionate group of scientists and engineers dedicated to building comprehensive transportation technologies. Our mission is to create highly scalable, intelligent systems that enable seamless international package movement. By pushing technological boundaries, we're reimagining global logistics through innovative applied science. Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan Learn more about our benefits here: https://www.amazon.jobs/en/internal/benefits/us-benefits-and-stock
Applied Scientist II, Special Projects
US, WA, Seattle
Innovators wanted! Are you an entrepreneur? A builder? A dreamer? This role is part of an Amazon Special Projects team that takes the company’s Think Big leadership principle to the limits. If you’re interested in innovating at scale to address big challenges in the world, this is the team for you. As an Applied Scientist on our team, you will focus on building state-of-the-art ML models for healthcare. Our team rewards curiosity while maintaining a laser-focus in bringing products to market. Competitive candidates are responsive, flexible, and able to succeed within an open, collaborative, entrepreneurial, startup-like environment. At the forefront of both academic and applied research in this product area, you have the opportunity to work together with a diverse and talented team of scientists, engineers, and product managers and collaborate with other teams. This role offers a unique opportunity to work on projects that could fundamentally transform healthcare outcomes. Key job responsibilities In this role, you will: • Design and implement novel AI/ML solutions for complex healthcare challenges • Drive advancements in machine learning and data science • Balance theoretical knowledge with practical implementation • Work closely with customers and partners to understand their requirements • Navigate ambiguity and create clarity in early-stage product development • Collaborate with cross-functional teams while fostering innovation in a collaborative work environment to deliver impactful solutions • Establish best practices for ML experimentation, evaluation, development and deployment • Partner with leadership to define roadmap and strategic initiatives You’ll need a strong background in AI/ML, proven leadership skills, and the ability to translate complex concepts into actionable plans. You’ll also need to effectively translate research findings into practical solutions. A day in the life You will solve real-world problems by getting and analyzing large amounts of data, generate insights and opportunities, design simulations and experiments, and develop statistical and ML models. The team is driven by business needs, which requires collaboration with other Scientists, Engineers, and Product Managers across the Special Projects organization. You will prepare written and verbal presentations to share insights to audiences of varying levels of technical sophistication. About the team We represent Amazon's ambitious vision to solve the world's most pressing challenges. We are exploring new approaches to enhance research practices in the healthcare space, leveraging Amazon's scale and technological expertise. We operate with the agility of a startup while backed by Amazon's resources and operational excellence. We're looking for builders who are excited about working on ambitious, undefined problems and are comfortable with ambiguity.
Sr. Comm Systems Engineer, Wireless Systems
US, TX, Austin
Amazon Leo is an initiative to launch a constellation of Low Earth Orbit satellites that will provide low-latency, high-speed broadband connectivity to unserved and underserved communities around the world. As a Systems Engineer, this role is primarily responsible for the design, development and integration of communication payload and customer terminal systems. The Role: Be part of the team defining the overall communication system and architecture of Amazon Leo’s broadband wireless network. This is a unique opportunity to innovate and define groundbreaking wireless technology at global scale. The team develops and designs the communication system for project Leo and analyzes its overall system level performance such as for overall throughput, latency, system availability, packet loss etc. This role in particular will be responsible for leading the effort in designing and developing advanced technology and solutions for communication system. This role will also be responsible developing advanced physical layer + protocol stacks systems as proof of concept and reference implementation to improve the performance and reliability of the LEO network. In particular this role will be responsible for using concepts from digital signal processing, information theory, wireless communications to develop novel solutions for achieving ultra-high performance LEO network. This role will also be part of a team and develop simulation tools with particular emphasis on modeling the physical layer aspects such as advanced receiver modeling and abstraction, interference cancellation techniques, FEC abstraction models etc. This role will also play a critical role in the integration and verification of various HW and SW sub-systems as a part of system integration and link bring-up and verification. Export Control Requirement: Due to applicable export control laws and regulations, candidates must be a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum.
Senior Applied Scientist, Real-Time Conversational AI , AGI
US, CA, Sunnyvale
We are looking for a Senior Applied Scientist to help drive the research and development of real-time multimodal conversational AI. You will contribute across two focus areas: advancing foundation models for speech and audio, and building the post-training systems (reward modeling, reinforcement learning) that shape natural, human-like conversational behavior. You will own a significant research area and contribute across the full model lifecycle — from pre-training and architecture design through post-training alignment and real-time deployment. You will work at the frontier of what’s possible in conversational AI, with the compute, data, and runway to pursue problems that few teams in the world have the resources to tackle. As a Senior Scientist, you will drive the technical execution of your research area, contribute to the team’s roadmap, and work closely with inference engineers to ensure your models are designed for real-time production deployment. Key job responsibilities What You’ll Do Foundation Model Scaling - Help build and train large-scale multimodal foundation models for real-time speech and audio generation, from architecture design through production-scale training - Advance the scaling and efficiency of conversational models, including the relationship between data, model size, and real-time performance - Design model architectures informed by hardware constraints and inference requirements, working with inference engineers to ensure models are servable from inception - Develop training methodologies for multimodal models that jointly process and generate speech, language, and audio in real-time streaming contexts - Contribute to the state of the art on efficient architectures and training methods for conversational AI at scale Post-Training & Reinforcement Learning - Design and build reward models and reward functions for speech systems — capturing naturalness, fluency, conversational quality, and real-time responsiveness - Develop and apply reinforcement learning methods to shape conversational behavior — teaching models natural timing, responsiveness, and fluid interaction - Build parts of the post-training pipeline from SFT through RL alignment, optimized for real-time multimodal outputs rather than text-only generation - Design evaluation frameworks that capture the quality dimensions unique to real-time conversation (latency sensitivity, audio quality, prosody, interaction naturalness) Real-Time Perception & Generation - Advance the team’s capabilities in real-time perception — the ability of the model to process incoming audio/speech while simultaneously generating responses - Develop techniques for natural interactive systems where the model handles concurrent input and output with human-like timing - Work at the intersection of model architecture and production constraints to ensure multimodal capabilities function within hard real-time latency budgets
Senior Applied Scientist, Fauna Robotics
US, WA, Bellevue
Robots are about to step out of the movies and into our homes. If you're excited about this robotics revolution, come join Amazon's Fauna Robotics. As an Applied Scientist in this org, you'll push the state-of-the-art techniques in Robotics. You'll drive technical excellence in areas such as perception, manipulation, sim2real transfer, reinforcement learning and multi-task learning, designing novel algorithms that bridge the gap between research and real-world deployment. In this role, you will integrate hands-on technical expertise with scientific leadership, ensuring your team delivers robust solutions for dynamic real-world environments. While staying updated with the latest advancements in robotics research, you will leverage your experience and expertise to guide the team towards the most promising direction. Collaborating with a talented group of software engineers and scientists, you will bring your ideas to fruition. Deploying robots in home environments requires you to anticipate challenges not encountered in structured settings like warehouses. Additionally, consumer-grade sensors and actuators necessitate working around their limitations, requiring out-of-the-box innovative ideas. As such, you are someone who truly enjoys thinking about problems using first principles at a holistic and systems level to address the aforementioned challenges. Key job responsibilities - Lead technical initiatives in areas such as robotics foundation models, reinforcement learning and manipulation - - Design experiments to identify the limitations of current state-of-the-art models and developing new models or techniques that can surpass these models - Design and implement novel deep learning architectures that push the boundaries of what robots can understand and accomplish - Mentor fellow scientists while maintaining strong individual technical contributions - Collaborate with engineering teams to optimize and scale models for real-world applications - Influence technical decisions and implementation strategies within your area of focus - Experienced in communicating complex technical work to a non-technical audience - The ability to work with minimal guidance, be proactive and to handle ambiguity and the challenge of quickly evolving goals A day in the life - Train ML models for deployment in simulation and real-world robots, identify and document their limitations post-deployment - Drive technical discussions within your team and with key stakeholders to develop innovative solutions to address identified limitations - Guide fellow scientists in solving complex technical challenges, from sim2real transfer to training RL policies - Mentor team members while maintaining significant hands-on contribution to technical solutions About the team Fauna Robotics, an Amazon company, is building capable, safe, and genuinely delightful robots for everyday life. Our goal is simple: make robots people actually want to live and interact with in everyday human spaces. We believe that future won’t arrive until building for robotics becomes far more accessible. Today, too much effort is spent reinventing the fundamentals. We’re changing that by developing tightly integrated hardware and software systems that make it faster, safer, and more intuitive to create real-world robotic products. Our work spans the full stack: mechanical design, control systems, dynamic modeling, and intelligent software. The focus is not just functionality, but experience. We’re building robots that feel responsive, expressive, and genuinely useful. At Fauna, you’ll work at the frontier of this space, helping define how robots move, manipulate, and interact with people in natural environments. It’s an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social