An Amazon employee is seen loading boxes into a delivery vehicle
To make package deliveries more effective, Amazon researchers drew from the vehicle routing problem, which builds upon the traveling salesman problem, one of the most popular combinatorial problems among the operations research community.

The science behind grouping package deliveries

How Customer Order and Network Density OptimizeR (CONDOR) has led to improved delivery routes.

In 2018, Rohit Malshe, an Amazon principal scientist, looked across the street from his home in Portland, Oregon, and spotted two trucks delivering Amazon packages to neighboring houses.

Malshe and his colleagues within Amazon Logistics (AMZL) Research Science and Amazon’s Supply Chain Optimization Technologies (SCOT) organizations had already started developing new methods to synchronize deliveries to neighboring locations. The goal: to avoid situations similar to the one he just observed — multiple trucks visiting the same area on a given day — to improve deliveries and reduce costs.

Related content
How the Amazon Logistics Research Science team guides important decisions related to last-mile delivery.

Important advances had been made by that time. Around 2017, scientists developed a method to group packages delivered to the same address on the same day. When that succeeded, the algorithm expanded to whole buildings.

“Those local optimizations were tackled first, and we solved them rather quickly,” says Malshe.

The next step: develop the concept of a stop consolidation — a chain of addresses linked by a small road segment. For example, a small group of houses in a cul-de-sac may be grouped together and, if two of them order items that will be delivered on the same day, the system will attempt to combine these deliveries into a single truck.

Focusing on the route

Following these initial advancements, AMZL and SCOT scientists turned their attention to optimizing shipment assignment and delivery systems by focusing on the routes.

“Instead of considering one address or building at a time, one should consider all demand assigned to a route, the time spent delivering, and the geographical blocks assigned to a route,” says Andrea Qualizza, a SCOT senior principal scientist.

The researchers drew from the prize-collecting vehicle routing problem (PCVRP), which builds upon the traveling salesman problem (TSP), one of the most popular combinatorial problems among the theoretical computer science and operations research communities.

Related content
Team Passing Through, with three academics working together, earns $100,000 first prize.

“In the TSP, you just want to find the optimal way — or the shortest possible route — to traverse a certain number of locations,” says Weihong Hu, a SCOT senior applied scientist. But as the number of destinations rises, the number of possible routes surpasses the capabilities of even the fastest computers.

For example, in a scenario with 10 destinations, there can be more than 300,000 possible routes. If the number of destinations increases to 15, the number of possible routes expands to more than 87 billion.

The PCVRP is a more challenging variant of the TSP. Imagine a scenario where there are packages sitting at each customer’s door and you send a driver to pick up all the packages they can within a certain time interval. Every package collected adds a “prize” that is equivalent to the cost of assigning that package to a third-party carrier.

“For every package assigned to AMZL, you collect a reward, but you are also spending your time and truck space to do so,” says Malshe.

Deploying CONDOR

The team iterated on this approach over the past couple of years, developing a new algorithm: Customer Order and Network Density OptimizeR (CONDOR).

Qualizza notes that CONDOR solves a problem even harder than the PCVRP: simultaneously determining how orders are split into shipments and the sourcing fulfillment center for each shipment.

“This project uses a combination of various techniques,” says Qualizza. “There is mathematical optimization, local search, capacitated vehicle routing problem solvers — all of that came together because these techniques considered various aspects of the problem and linked them very naturally with the way our systems work."

The breakthrough of CONDOR, Qualizza notes, is its ability to determine the right tradeoff between the levels of complexity and optimality.

While the problem involves a high number of possible decisions per geographical block, the program reduces that number to less than 10 per block.

“You're losing some optimality, but at the same time, you make the problem tractable,” he says. “The key is squeezing enough bits of utility from the current data when it is processed and considering it all together to achieve meaningful improvements.”

Once confident with the prototype, Malshe says, the scientists tested it in production, working side-by-side with engineers to make that happen. In one of the tests performed — a crossover experiment — a few selected cities were divided into two groups: one of them used the new program, and the other didn’t.

That experiment determined that use of CONDOR resulted in roughly 0.5% fewer routing resources required.

“That means that if the whole network needs 50,000 routes without CONDOR, with CONDOR we should need just 49,750 routes,” says Malshe. Applied over Amazon’s extensive trucking network, the potential gains could be enormous.

CONDOR started running in a few delivery stations in January, and expanded to more locations this spring. Now it is deployed across the entire United States, and there are plans to deploy it in other countries within the next few months. The research behind the new program was presented at the 2021 INFORMS Annual Meeting.

How customers benefit

When a customer places an order on the Amazon store, Amazon immediately reserves inventory for it. Within minutes, Amazon’s logistics models evaluate thousands of options for fulfilling the order. Fulfillment location is one variable: a single warehouse may assemble the order if it has all the items; if not, the order can be split into multiple shipments. Amazon also reserves capacity within its transportation network to ensure the order can be delivered on time.

An Amazon employee is seen scanning packages, there are pallets of boxes behind him
Customer Order and Network Density OptimizeR (CONDOR) has led to improved delivery routes by assessing and reassessing customer orders, before they leave a fulfillment center, to identify effective delivery options.

At that point, a decision is made about how the order will be fulfilled, but it won’t necessarily be the final one. There will be opportunities to revisit the decision before the shipment process starts. If a neighbor places an order later that day, for example, the logistics plan may be updated so that the same carrier delivers both orders.

Different programs may work together to make all of that happen. One program will consolidate orders by the customer’s address, for example, so that a carrier visits the address only once if possible. Another will consolidate the orders by buildings and another by groups of buildings.

Related content
Pioneering web-based PackOpt tool has resulted in an annual reduction in cardboard waste of 7% to 10% in North America, saving roughly 60,000 tons of cardboard annually.

CONDOR takes that to the next level by looking at the entire geographic area that a delivery station serves. Every once in a while, the decisions are reevaluated to ensure routes are optimized. From the time a customer places an order to the time it starts being processed by a fulfillment center, it could be five or six hours. During that time span, the CONDOR code might reassess the order three to four times, providing many opportunities for further optimization.

The team estimates that this year alone, CONDOR will help avoid millions of miles driven, boosting sustainability.

“We can enable carriers to deliver more packages to more customers on time, while reducing miles driven and carbon emissions from fuel,” Qualizza says. “That is the essence of CONDOR; it revisits all those decisions and finds those opportunities for us to further delight customers.”

Related content

IN, HR, Gurugram
Building large-scale forecasting and optimization systems that power Amazon’s global transportation network and directly impact customer experience and cost. Key job responsibilities 1. Guide model and system design across a range of techniques, including tree-based models, deep learning (LSTMs, transformers), LLMs, and reinforcement learning. 2. Ensure models are production-ready, scalable, and robust through close partnership with stakeholders. 3. Partner with Product, Operations, and Engineering leaders to enable proactive decision-making and corrective actions. 4 Own end-to-end business metrics, directly influencing customer experience, cost optimization, and network reliability. 5. Help contribute to the broader ML community through publications, conference submissions, and internal knowledge sharing.
US, WA, Seattle
Estimating the demand response of a pricing decision is genuinely hard. The causal effects are delayed, noisy, and confounded by factors that standard experiment analysis wasn't designed to handle. Most pricing teams default to heuristics not because they don't care about customer responses, but because measuring them rigorously is an unsolved problem. P2OS is building the science to solve it. We're hiring an Economist to own that work — defining how we estimate digital demand response in a pricing context, building the identification strategies that make those estimates credible, and translating outputs into something pricing teams can use to make better decisions. The role sits at the intersection of econometric methodology and production-quality analysis, and requires someone who can operate independently in both. As science lead, you'll own the digital pricing methodology domain, and be the internal authority on causal inference for pricing across P2OS and partner teams. Key job responsibilities * Own the end-to-end digital pricing methodology for pricing — identification strategy, modeling choices, validation approach, and business use cases — and drive adoption across pricing contexts * Deliver high-stakes analyses connecting digital pricing estimates to a concrete pricing decision and strategy change at VP+ level * Apply advanced causal methods to live pricing problems; document approaches so the team can build on and extend them. * Provide causal inference guidance on pricing experiment questions as they arise — being the methodology resource when experiments generate relevant questions * Serve as cross-team economic advisor to Digital Finance, Customer Behavior, and Demand Science on assumptions and causal identification * Actively mentor junior scientists, earn trust of cross-functional tech and product partners. A day in the life In a typical day, you'll move between methodology work and stakeholder-facing analysis. - On the science side, that means reviewing identification assumptions with the Causal AS, validating estimation choices for the LTV framework, and documenting methodology decisions in ways that non-economists can act on. - On the applied side, you'll be in rooms with Finance, Pricing PMs, and other science teams: aligning on LTV definitions, resolving disagreements between competing metrics, and translating causal findings into recommendations that land in strategy reviews. - As tech lead, you need to work to develop the economists and scientists on your scrum: structured reviews, identification strategy feedback, and raising the quality of analyses before they reach stakeholders. The mix shifts, but the through-line is to progress the LTV methodology from open questions to shipped frameworks, and making sure the team's causal work is rigorous enough to hold up when it counts. About the team P2Optimization Science (P2OS) is responsible for the ML models and analytical frameworks that drive pricing decisions at scale. The team spans demand lift modeling, pricing error detection, customer lifetime value, and experimentation. Our small team of specialized applied scientists and economists works closely alongside engineers, and pricing product managers.
US, WA, Seattle
We’re working to improve shopping on Amazon using the conversational capabilities of large language models, and are searching for pioneers who are passionate about technology, innovation, and customer experience, and are ready to make a lasting impact on the industry. You'll be working with talented scientists, engineers, and technical program managers (TPM) to innovate on behalf of our customers. If you're fired up about being part of a dynamic, driven team, then this is your moment to join us on this exciting journey!
US, MA, Boston
Are you interested in how to build AI reasoning systems that give provably correct answers? Are you excited by science at the interface of classical AI reasoning and Large Language Models (LLMs)? Would you like to apply your technology to serve operations customers better? Amazon Robotics is looking for a talented Applied Scientist in Neurosymbolic AI. You will innovate on combining language models (LMs) with classical AI reasoning. You will work with a team of scientists and engineers to achieve this. You will publish your results in papers at leading venues in AI. You will be part of a larger team and have the opportunity to work on problems such as: using LMs to generate plans, using AI reasoning to verify plan correctness, learning efficient reasoning strategies, self-improving models. You will work on basic science and on business problems in robotics, automation and fulfillment across our operations. Key job responsibilities In this role you will: • Work closely with other scientists and engineers, and be part of Amazon’s diverse global science community. • Publish your research in top-tier academic venues and hone your presentation skills. • Be inspired by challenges and opportunities to invent new techniques in your area(s) of expertise. A day in the life You'll meet regularly with your technical lead and your team on your ideas, get guidance and feedback, work together on architectures and algorithms, author papers, build AI systems, all with the aim of delivering results for your operations customers. You'll work closely with other scientists to review your plans and results. You'll meet with engineers to implement your ideas at scale. About the team The Veritas team is a science team working at the boundary between language models and classical AI reasoning. We work across on customer problems in fulfillment, automation and robotics. We focus on high quality research science informed by practical problems.
US, WA, Seattle
Economists in this role partner with business stakeholders to distill complex problems into testable economic questions and generate actionable insights. They collaborate with engineers and scientists to estimate models on large-scale data, design pilots, measure impact, and scale successful prototypes into improved policies and programs. They leverage AI tools to scale economic study for broader business impact. They communicate findings to business leaders, incorporate feedback, and deliver customer-centric solutions at scale.
CA, BC, Vancouver
The Alexa Daily Essentials team delivers experiences critical to how customers interact with Alexa as part of daily life. Alexa users engage with our products across experiences connected to Timers, Alarms, Calendars, Food, and News. Our experiences include critical time saving techniques, ad-supported news audio and video, and in-depth kitchen guidance aimed at serving the needs of the family from sunset to sundown. As a Data Scientist on our team, you'll work with complex data, develop statistical methodologies, and provide critical product insights that shape how we build and optimize our solutions. You will work closely with your Analytics and Applied Science teammates. You will build frameworks and mechanisms to scale data solutions across our organization. If you are passionate about redefining how AI can improves everyone's daily life, we’d love to hear from you. Key job responsibilities Problem-Solving - Analyze complex data to identify patterns, inform product decisions, and understand root causes of anomalies. - Develop analysis and modeling approaches to drive product and engineering actions to identify patterns, insights, and understand root causes of anomalies. Your solutions directly improve the customer experience. - Independently work with product partners to identify problems and opportunities. Apply a range of data science techniques and tools to solve these problems. Use data driven insights to inform product development. Work with cross-disciplinary teams to mechanize your solution into scalable and automated frameworks. Data Infrastructure - Build data pipelines, and identify novel data sources to leverage in analytical work - both from within Alexa and from cross Amazon - Acquire data by building the necessary SQL / ETL queries Communication - Excel at communicating complex ideas to technical and non-technical audiences. - Build relationships with stakeholders and counterparts. Work with stakeholders to translate causal insights into actionable recommendations - Force multiply the work of the team with data visualizations, presentations, and/or dashboards to drive awareness and adoption of data assets and product insights - Collaborate with cross-functional teams. Mentor teammates to foster a culture of continuous learning and development
US, NY, New York
The Ads Measurement Science team in the Measurement, Ad Tech, and Data Science (MADS) team of Amazon Ads serves a centralized role developing solutions for a multitude of performance measurement products. We create solutions which measure the comprehensive impact of advertiser's ad spend, including sales impacts both online and offline and across timescales, and provide actionable insights that enable our advertisers to optimize their media portfolios. We also own the science solutions for AI tools that unlock new insights and automate high-effort customer workflows, such as custom query and report generation based on natural language user requests. We leverage a host of scientific technologies to accomplish this mission, including Generative AI, classical ML, Causal Inference, Natural Language Processing, and Computer Vision. As a Senior Applied Scientist on the team, you will be at the forefront of innovation, developing measurement solutions end-to-end from inception to production. You will set the technical vision and innovate on behalf of our customers. You will propose, design, analyze, and productionize models to provide novel measurement insights to our customers. You will partner with engineering to deploy these solutions into production. You will work with key stakeholders from various business teams to enable advertisers to act upon those metrics. Key job responsibilities * Lead the development of ad measurement models and solutions that address the full spectrum of an advertiser's investment, focusing on scalable and efficient methodologies. * Collaborate closely with cross-functional teams including engineering, product management, and business teams to define and implement measurement solutions. * Use state-of-the-art scientific technologies including Generative AI, Classical Machine Learning, Causal Inference, Natural Language Processing, and Computer Vision to develop state of the art models that measure the impact of ad spend across multiple platforms and timescales. * Drive experimentation and the continuous improvement of ML models through iterative development, testing, and optimization. * Translate complex scientific challenges into clear and impactful solutions for business stakeholders. * Mentor and guide junior scientists, fostering a collaborative and high-performing team culture. * Foster collaborations between scientists to move faster, with broader impact. * Regularly engage with the broader scientific community with presentations, publications, and patents. A day in the life You will solve real-world problems by getting and analyzing large amounts of data, generate business insights and opportunities, design simulations and experiments, and develop statistical and ML models. The team is driven by business needs, which requires collaboration with other Scientists, Engineers, and Product Managers across the advertising organization. You will prepare written and verbal presentations to share insights to audiences of varying levels of technical sophistication. Team video https://advertising.amazon.com/help/G4LNN5YWHP6SM9TJ About the team We are a team of scientists across Applied, Research, Data Science and Economist disciplines. You will work with colleagues with deep expertise in ML, NLP, CV, Gen AI, and Causal Inference with a diverse range of backgrounds. We partner closely with top-notch engineers, product managers, sales leaders, and other scientists with expertise in the ads industry and on building scalable modeling and software solutions.
US, NY, New York
The Ads Measurement Science team in the Measurement, Ad Tech, and Data Science (MADS) team of Amazon Ads serves a centralized role developing solutions for a multitude of performance measurement products. We create solutions which measure the comprehensive impact of advertiser's ad spend, including sales impacts both online and offline and across timescales, and provide actionable insights that enable our advertisers to optimize their media portfolios. We also own the science solutions for AI tools that unlock new insights and automate high-effort customer workflows, such as custom query and report generation based on natural language user requests. We leverage a host of scientific technologies to accomplish this mission, including Generative AI, classical ML, Causal Inference, Natural Language Processing, and Computer Vision. As an Applied Scientist on the team, you will lead measurement solutions end-to-end from inception to production. You will propose, design, analyze, and productionize models to provide novel measurement insights to our customers. Key job responsibilities Leverage deep expertise in one or more scientific disciplines to invent solutions to ambiguous ads measurement problems Disambiguate problems to propose clear evaluation frameworks and success criteria Work autonomously and write high quality technical documents Implement a significant portion of critical-path code, and partner with engineers to directly carry solutions into production Partner closely with other scientists to deliver large, multi-faceted technical projects Share and publish works with the broader scientific community through meetings and conferences Communicate clearly to both technical and non-technical audiences Contribute new ideas that shape the direction of the team's work Mentor more junior scientists and participate in the hiring process About the team We are a team of scientists across Applied, Research, Data Science and Economist disciplines. You will work with colleagues with deep expertise in ML, NLP, CV, Gen AI, and Causal Inference with a diverse range of backgrounds. We partner closely with top-notch engineers, product managers, sales leaders, and other scientists with expertise in the ads industry and on building scalable modeling and software solutions.
US, WA, Seattle
Are you interested in shaping the future of Advertising and B2B Sales? We are a growing team with an exciting AI-first charter and need your passion, innovative thinking, and creativity to help take our products to new heights. Amazon Advertising is one of Amazon's fastest growing and most profitable businesses, responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products are strategically important to our businesses driving long term growth. We break fresh ground in product and technical innovations every day! Within the Advertising Sales organization, we are building a central AI/ML team and are seeking top Applied Science talent to help us build new, science-backed services that drive success for our customers. Our goal is to transform the way account teams operate by creating actionable insights and recommendations they can share with their advertising accounts, and ingesting Generative AI throughout their end-to-end workflows to improve their work efficiency. As an Applied Scientist on the team, you will bring deep expertise in modeling dynamic systems using statistical methods and deep learning, and in optimizing those systems using reinforcement learning and operations research. You have the scientific and technical skills to build and refine models that can be implemented in production, and you leverage natural language processing and generative AI to enhance their explainability. You will chart new courses with our ad sales support technologies, and you have the communication skills necessary to explain complex technical approaches to a variety of stakeholders and customers. You will be part of a team of fellow scientists and engineers taking iterative approaches to tackle big, long-term problems. You are fluently able to leverage the latest generative AI systems and services to accelerate and improve your work while maintaining high quality in your outputs. Key job responsibilities Scientific Modeling - Conceptualize and lead state-of-the-art research on new Machine Learning and Generative Artificial Intelligence solutions to optimize all aspects of the Ad Sales business - Lead the technical approach for the design and implementation of successful models and algorithms in support of expert cross-functional teams delivering on demanding projects - Run regular A/B experiments, gather data, and perform statistical analysis - Improve the scalability, efficiency and automation of large-scale data analytics, model training, deployment and serving - Publish scientific findings in reports and papers that can be shared internally and externally Product Development Support - Partner with software engineering and product management teams to support product and service development, define success metrics and measurement approaches, and help drive adoption of innovative new features for our services. - Lead requirements gathering sessions with product teams and business stakeholders - Maintain scientific documentation and knowledge for product initiatives Collaboration & Communication - Work closely with software engineers to deliver end-to-end solutions into production - Translate complex scientific findings into actionable business recommendations for stakeholders and senior management - Provide clear, compelling reports and presentations on a regular basis with respect to your models and services - Communicate with internal teams to showcase results and identify best practices. About the team Sales AI is a central science and engineering organization within Amazon Advertising Sales that powers selling motions and account team workflows via state-of-the-art of AI/ML services. Sales AI is investing in a range of sales intelligence models, including the development of advertiser insights, recommendations and Generative AI-powered applications throughout account team workflows.
US, NY, New York
About Sponsored Products and Brands The Sponsored Products and Brands team at Amazon Ads is re-imagining the advertising landscape through generative AI technologies, revolutionizing how millions of customers discover products and engage with brands across Amazon.com and beyond. We are at the forefront of re-inventing advertising experiences, bridging human creativity with artificial intelligence to transform every aspect of the advertising lifecycle from ad creation and optimization to performance analysis and customer insights. We are a passionate group of innovators dedicated to developing responsible and intelligent AI technologies that balance the needs of advertisers, enhance the shopping experience, and strengthen the marketplace. If you're energized by solving complex challenges and pushing the boundaries of what's possible with AI, join us in shaping the future of advertising. Key job responsibilities As a Machine Learning Applied Scientist, you will: * Conduct deep data analysis to derive insights to the business, and identify gaps and new opportunities * Develop scalable and effective machine-learning models and optimization strategies to solve business problems * Run regular A/B experiments, gather data, and perform statistical analysis * Work closely with software engineers to deliver end-to-end solutions into production * Improve the scalability, efficiency and automation of large-scale data analytics, model training, deployment and serving * Conduct research on new machine-learning modeling and Generative AI solutions to optimize all aspects of Sponsored Products and Brands business About the team The Ad Response Prediction team within Sponsored Products and Brands (SPB) drives personalized shopping experiences for SPB Ads across placements, pages, and devices worldwide. We achieve this through ML and GenAI solutions that include customized shopper response prediction and session-level understanding to optimize every stage of the ad-serving process, from sourcing and bidding to widget discovery and auctions. Our responsibilities include advancing response prediction through model and feature innovations and extending prediction beyond the auction stage to areas such as targeting, sourcing, and bidding.