AWS team wins best-paper award for work on automated reasoning

SOSP paper describes lightweight formal methods for validating new S3 data storage service.

At last week’s ACM Symposium on Operating Systems Principles (SOSP), my colleagues at Amazon Web Services and I won a best-paper award for our work using automated reasoning to validate that ShardStore — our new S3 storage node microservice — will do what it’s supposed to. 

Amazon Simple Storage Service (S3) is our fundamental object storage service — fast, cheap, and reliable. ShardStore is the service we run on our storage hardware, responsible for durably storing S3 object data. It’s a ground-up re-thinking of how we store and access data at the lowest level of S3. Because ShardStore is essential for the reliability of S3, it’s critical that it is free from bugs.

Formal verification involves mathematically specifying the important properties of our software and formally proving that our systems never violate those specifications — in other words, mathematically proving the absence of bugs. Automated reasoning is a way to find those proofs automatically.

ResetOperations_Animation.gif
An example of the ShardStore deletion procedure. Deleting the second data chunk in extent 18 (grey box) requires copying the other three chunks to different extents (extents 19 and 20) and resetting the write pointer for extent 18. The log-structured merge-tree itself is also stored on disk (in this case, in extent 17). See below for details.

Traditionally, formal verification comes with high overhead, requiring up to 10 times as much effort as building the system being verified. That’s just not practical for a system as large as S3.

For ShardStore, we instead developed a new lightweight automated-reasoning approach that gives us nearly all of the benefits of traditional formal proofs but with far lower overhead. 

Our methods found 16 bugs in the ShardStore code that would have required time-consuming and labor-intensive testing to find otherwise — if they could have been found at all. And with our method, specifying the software properties to be verified increased the ShardStore codebase by only about 14% — versus the two- to tenfold increases typical of other formal-verification approaches.

Our method also allows the specifications to be written in the same language as the code — in this case, Rust. That allows developers to write new specifications themselves whenever they extend the functionality of the code. Initially, experts in formal verification wrote the specifications for ShardStore. But as the project has progressed, software engineers have taken over that responsibility. At this point, 18% of the ShardStore specifications have been written by developers.

Reference models

One of the central concepts in our approach is that of reference models, simplified instantiations of program components that can be used to track program state under different input conditions.

For instance, storage systems often use log-structured merge-trees (LSMTs), a sophisticated data structure designed to apportion data between memory and different tiers of storage, with protocols for transferring data that take advantage of the different storage media to maximize efficiency.

The state of an LSMT, however — data locations and the record of data access patterns — can be modeled using a simple hash table. A hash table can thus serve as a reference model for the tree.

In our approach, reference models are specified using executable code. Code verification is then a matter of ensuring that the state of a component instantiated in the code matches that of the reference model, for arbitrary inputs. In practice, we found that specifying reference models required, on average, about 1% as much code as the actual component implementations.

Dependency tracking

ShardStore uses LSMTs to track and update data locations. Each object stored by ShardStore is divided into chunks, and the chunks are written to extents, which are contiguous regions of physical storage on a disk. A typical disk has tens of thousands of extents. Writes within each extent are sequential, tracked by a write pointer that defines the next valid write position.

The simplicity of this model makes data writes very efficient. But it does mean that data chunks within an extent can’t be deleted individually. Deleting a chunk from an extent requires transferring all the other chunks in the extent elsewhere and then moving the write pointer back to the beginning of the extent.

The sequence of procedures required to write a single chunk of data using ShardStore — the updating of the merge-tree, the writing of the chunk, the incrementation of the write pointer, and so on — create sets of dependencies between successive write operations. For instance, the position of the write pointer within an extent depends on the last write performed within that extent.

Dependency graph.png
The dependency graph for a sequence of S3 PUT (write) operations, together with the state of the LSM tree and the locations of the data on-disk after the operations have executed.

Our approach requires that we track dependencies across successive operations, which we do by constructing a dependency graph on the fly. ShardStore uses the dependency graph to decide how to most efficiently write data to disk while still remaining consistent when recovering from crashes. We use formal verification to check that the system always constructs these graphs correctly and so always remains consistent.

Test procedures

In our paper, we describe a range of tests, beyond crash consistency, that our method enables, such as concurrent-execution tests and tests of the serializers that map the separate elements of a data structure to sequential locations in memory or storage.

We also describe some of our optimizations to ensure that our verification is thorough. For instance, our method generates random sequences of inputs to test for specification violations. If a violation is detected, the method systematically pares down the input sequence to identify which specific input or inputs caused the error.

We also bias the random-input selector so that it selects inputs that target the same storage pathways, to maximize the likelihood of detecting an error. If each input read from or wrote to a different object, for instance, there would be no risk of encountering a data inconsistency.

We use our lightweight automated-reasoning techniques to validate every single deployment of ShardStore. Before any change reaches production, we check its behavior in hundreds of millions of scenarios by running our automated tools using AWS Batch

To support this type of scalable checking, we developed and open-sourced the new Shuttle model checker for Rust code, which we use to validate concurrency properties of ShardStore. Together, these approaches provide a continuous and automated correctness mechanism for one of S3’s most important microservices.

Research areas

Related content

US, CA, San Francisco
The AWS Center for Quantum Computing is a multi-disciplinary team of scientists, engineers, and technicians, all working to innovate in quantum computing for the benefit of our customers. We are looking to hire a Research Scientist to design and model novel superconducting quantum devices, including qubits, readout and control schemes, and advanced quantum processors. Candidates with a track record of original scientific contributions and/or software development experience will be preferred. We are looking for candidates with strong engineering principles and resourcefulness. Organization and communication skills are essential. About the team AWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon’s Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS’s services and features apart in the industry. As a member of the UC organization, you’ll support the development and management of Compute, Database, Storage, Internet of Things (Iot), Platform, and Productivity Apps services in AWS, including support for customers who require specialized security solutions for their cloud services. Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
AU, NSW, Sydney
AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success. AWS Support also partners with a global list of customers that are building mission-critical applications on top of AWS services. The Generative Artificial Intelligence (AI) Innovation Center team at AWS provides opportunities to innovate in a fast-paced organization that contributes to game-changing projects and technologies leveraging cutting-edge generative AI algorithms. As an Applied Scientist, you'll partner with technology and business teams to build solutions that surprise and delight our customers. We’re looking for Applied Scientists capable of using generative AI and other ML techniques to design, evangelize, and implement state-of-the-art solutions for never-before-solved problems. Key job responsibilities - Collaborate with scientists and engineers to research, design and develop cutting-edge generative AI algorithms to address real-world challenges - Work across customer engagement to understand what adoption patterns for generative AI are working and rapidly share them across teams and leadership - Interact with customers directly to understand the business problem, help and aid them in implementation of generative AI solutions, deliver briefing and deep dive sessions to customers and guide customer on adoption patterns and paths for generative AI - Create and deliver best practice recommendations, tutorials, blog posts, sample code, and presentations adapted to technical, business, and executive stakeholder - Provide customer and market feedback to Product and Engineering teams to help define product direction. A day in the life Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 16 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. About the team Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. What if I don’t meet all the requirements? That’s okay! We hire people who have a passion for learning and are curious. You will be supported in your career development here at AWS. You will have plenty of opportunities to build your technical, leadership, business and consulting skills. Your onboarding will set you up for success, including a combination of formal and informal training. You’ll also have a chance to gain AWS certifications and access mentorship programs. You will learn from and collaborate with some of the brightest technical minds in the industry today.
US, WA, Seattle
Amazon Prime is looking for an ambitious Economist to help create econometric insights for world-wide Prime. Prime is Amazon's premiere membership program, with over 200M members world-wide. This role is at the center of many major company decisions that impact Amazon's customers. These decisions span a variety of industries, each reflecting the diversity of Prime benefits. These range from fast-free e-commerce shipping, digital content (e.g., exclusive streaming video, music, gaming, photos), and grocery offerings. Prime Science creates insights that power these decisions. As an economist in this role, you will create statistical tools that embed causal interpretations. You will utilize massive data, state-of-the-art scientific computing, econometrics (causal, counterfactual/structural, time-series forecasting, experimentation), and machine-learning, to do so. Some of the science you create will be publishable in internal or external scientific journals and conferences. You will work closely with a team of economists, applied scientists, data professionals (business analysts, business intelligence engineers), product managers, and software engineers. You will create insights from descriptive statistics, as well as from novel statistical and econometric models. You will create internal-to-Amazon-facing automated scientific data products to power company decisions. You will write strategic documents explaining how senior company leaders should utilize these insights to create sustainable value for customers. These leaders will often include the senior-most leaders at Amazon. The team is unique in its exposure to company-wide strategies as well as senior leadership. It operates at the cutting-edge of utilizing data, econometrics, artificial intelligence, and machine-learning to form business strategies. A successful candidate will have demonstrated a capacity for building, estimating, and defending statistical models (e.g., causal, counterfactual, time-series, machine-learning) using software such as R, Python, or STATA. They will have a willingness to learn and apply a broad set of statistical and computational techniques to supplement deep-training in one area of econometrics. For example, many applications on the team use structural econometrics, machine-learning, and time-series forecasting. They rely on building scalable production software, which involves a broad set of world-class software-building skills often learned on-the-job. As a consequence, already-obtained knowledge of SQL, machine learning, and large-scale scientific computing using distributed computing infrastructures such as Spark-Scala or PySpark would be a plus. Additionally, this candidate will show a track-record of delivering projects well and on-time, preferably in collaboration with other team members (e.g. co-authors). Candidates must have very strong writing and emotional intelligence skills (for collaborative teamwork, often with colleagues in different functional roles), a growth mindset, and a capacity for dealing with a high-level of ambiguity. Endowed with these traits and on-the-job-growth, the role will provide the opportunity to have a large strategic, world-wide impact on the customer experiences of Prime members.
US, WA, Seattle
The CATALST NLP Services team within the Selling Partner Services (SPS) Core Services organization is responsible for simplifying multi-lingual experiences for our customers. We build and leverage various AI services to eliminate language barriers at scale through Machine Translation for Amazon Customers and Sellers WW across 30+ programs within SPS and customers outside of SPS. We leverage state-of-the-art NLP solutions including Large Language Models, Machine Translation, Language Detection, and OCR to provide a full suite of content analysis capabilities. Our customers include Selling Partners, Buyers, Amazon Associates, Amazon Investigators, and various Science teams. In this role, you will be a key owner within our cross-disciplinary team that includes Product Managers, Software Engineers, and Applied Scientists and execute on our 3 Year Plan. You will pioneer new technologies in NLP, machine translation, and machine learning. You will have ownership of the end-to-end development of solutions to complex problems from design to implementation and you will play an integral role in strategic decision-making. You will also work closely with other stakeholders such as engineers, operations teams and product owners to build ML pipelines, platforms and solutions that solve business problems. Key job responsibilities * Participate in the design, development, evaluation, deployment and updating of automated and scalable machine learning models, with a focus on machine translation * Develop and/or apply statistics, NLP and machine learning experiments and methodologies to different applications * Work closely with Scientists and Software Engineers on experimentations, evaluation and implementation * Work closely with business partners to understand the goals and develop solutions to achieve such goals
US, CA, Santa Clara
AWS AI/ML is looking for world class scientists and engineers to join its AI Research and Education group working on foundation models, large-scale representation learning, and distributed learning methods and systems. At AWS AI/ML you will invent, implement, and deploy state of the art machine learning algorithms and systems. You will build prototypes and innovate on new representation learning solutions. You will interact closely with our customers and with the academic and research communities. You will be at the heart of a growing and exciting focus area for AWS and work with other acclaimed engineers and world famous scientists. Large-scale foundation models have been the powerhouse in many of the recent advancements in computer vision, natural language processing, automatic speech recognition, recommendation systems, and time series modeling. Developing such models requires not only skillful modeling in individual modalities, but also understanding of how to synergistically combine them, and how to scale the modeling methods to learn with huge models and on large datasets. Join us to work as an integral part of a team that has diverse experiences in this space. We actively work on these areas: * Hardware-informed efficient model architecture, training objective and curriculum design * Distributed training, accelerated optimization methods * Continual learning, multi-task/meta learning * Reasoning, interactive learning, reinforcement learning * Robustness, privacy, model watermarking * Model compression, distillation, pruning, sparsification, quantization About the team Why AWS Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Utility Computing (UC) AWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon’s Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS’s services and features apart in the industry. As a member of the UC organization, you’ll support the development and management of Compute, Database, Storage, Internet of Things (IoT), Platform, and Productivity Apps services in AWS, including support for customers who require specialized security solutions for their cloud services. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Mentorship and Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Diverse Experiences Amazon values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.
US, NY, New York
The NA AMZL Supply Chain organization leads the innovation of Amazon’s Last Mile. We are an Operations org that hires and manages associates to deliver packages next day and sub-same day. The Execution and Planning Science (EPS) team sits within NA AMZL Supply Chain with the mission to build world-class automated Science-Tech products that enable ultra-fast delivery speeds for Amazon customers and job market opportunities for Amazon associates. Our key vision is to transform the online experience. We’re growing in scale and volume, by orders of magnitude. You will develop Science-Tech solutions to craft business strategy and roadmap to enable some of Amazon’s biggest brands to delight millions of customers. To succeed, we need senior technical leaders to forge a path into the future by building innovative, maintainable, and scalable systems. At Amazon, we are constantly inventing and re-inventing to be the most associate-centric company in the world. To get there, we need exceptionally talented, bright, and driven people. Amazon is one of the most recognizable brand names in the world and we distribute millions of products each year to our loyal customers. We are looking for an Applied Scientist who will be the science lead for all key optimization initiatives, responsible for building models and prototypes for labor planning systems, and will require close collaboration with other scientists on the team that are developing state-of-the-art optimization algorithms to scale. This role spans the innovation pipeline - from identifying business needs, to developing new optimization and prediction techniques, to prototyping and implementation by working closely with colleagues in engineering, product management, operations, retail and finance Key job responsibilities As a member of the scientist team, you will play an integral part on our Operations org with the following technical and leadership responsibilities: - Help the team define the forward-looking Science roadmap and vision by helping to identify, disambiguate and seek out new opportunities - Interact with engineering, operations, science and business teams to develop an understanding and domain knowledge of processes, system structures, and business requirements - Apply domain knowledge and business judgment to identify opportunities and quantify the impact aligning research direction to business requirements and make the right judgment on research project prioritization - Develop scalable models to derive optimal or near-optimal solutions to existing and new scheduling challenges - Create prototypes and simulations to test devised solutions - Advocate technical solutions to business stakeholders, engineering teams, as well as executive-level decision makers - Work closely with engineers to integrate prototypes into production system - Create policy evaluation methods to track the actual performance of devised solutions in production systems, identify areas with potential for improvement and work with internal teams to improve the solution with new features - Mentor and supervise the work of junior scientists on the team for technical development and their career development and growth - Present business cases and document models, analyses, and their results in order to influence important decisions
US, WA, Seattle
We are seeking a Senior Research Scientist who is curious, driven, and passionate about data insights and analytics. You will design, execute, and analyze user experiments to guide the development of Amazon FinTech products from a research prototype to launch. This role is on the FinTech UX Design and Research team with the impact, autonomy, flexibility, and speed of innovation of a high-growth startup, powered by the resources of Amazon. You will drive a program that oversees customer sentiment and attitudes at scale, using your unique research skills to uncover customer insights that help drive tactical and strategic product decisions across the organization. You will work alongside UX Designers and Researchers to advocate for our customers through quantitative and qualitative storytelling. Here at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/life Balance Our team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives. Mentorship & Career Growth Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship all while having fun. Our senior members enjoy one-on-one mentoring and thorough, but kind, design reviews. Key job responsibilities In this role, you will: • Own the development and establishment of a research program focused on measuring customer experience outcomes (CXO) or Voice of the Customer, which provides thought leadership to product teams and the broader organization. • Design and execute empirical research of medium to large scale that measures customer behavior and sentiment. Own the end-to-end analysis process including defining key business questions, recommending measures, data sources, methodology and study design, dataset creation, analysis execution, interpretation and presentation and publication of results. • Triangulate insights with key top-of-funnel metrics (e.g. usage, customer engagement) and other datasets to develop regular ways to report on drivers of movement with recommended action items aligned to business goals. • Own all communications to project members and stakeholders on progress, issues, and risks, provide pre and post launch communications to impacted teams. Communicate insights to non-scientific audiences in a compelling way. • Work with UX Design and Product partners to establish Voice of the Customer success metrics and tools for tracking and developing ongoing program improvement plans to meet relevant business needs. • Create long-term research roadmaps and opportunity frameworks that align to and inform product strategy and goals. • Collaborate with UX designers, product managers, and engineering to evaluate prototypes, functional mockups, usability testing, A/B testing, etc. • Build durable, scalable points-of-view and knowledge management tools that are leveraged within the Amazon FinTech Team and by other organizations across Amazon. • Advocate for the right insights and prioritization of research – ensure the right research questions are being asked; educate the organization on opportunities revealed by your own research and that of others; create and execute against long-term research roadmaps and opportunity frameworks that align to and inform high-level business strategy and goals.
US, WA, Bellevue
Conversational AI ModEling and Learning (CAMEL) team is part of Amazon Devices organization where our mission is to build a best-in-class Conversational AI that is intuitive, intelligent, and responsive, by developing superior Large Language Models (LLM) solutions and services which increase the capabilities built into the model and which enable utilizing thousands of APIs and external knowledge sources to provide the best experience for each request across millions of customers and endpoints. We are looking for a passionate, talented, and resourceful Applied Scientist in the field of LLM, Artificial Intelligence (AI), Natural Language Processing (NLP), Recommender Systems and/or Information Retrieval, to invent and build scalable solutions for a state-of-the-art context-aware conversational AI. A successful candidate will have strong machine learning background and a desire to push the envelope in one or more of the above areas. The ideal candidate would also have hands-on experiences in building Generative AI solutions with LLMs, enjoy operating in dynamic environments, be self-motivated to take on challenging problems to deliver big customer impact, moving fast to ship solutions and then iterating on user feedback and interactions. Key job responsibilities As an Applied Scientist, you will leverage your technical expertise and experience to collaborate with other talented applied scientists and engineers to research and develop novel algorithms and modeling techniques to reduce friction and enable natural and contextual conversations. You will analyze, understand and improve user experiences by leveraging Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in artificial intelligence. You will work on core LLM technologies, including Prompt Engineering and Optimization, Supervised Fine-Tuning, Learning from Human Feedback, Evaluation, Self-Learning, etc. Your work will directly impact our customers in the form of novel products and services.
US, NY, New York
Amazon is investing heavily in building a world class advertising business and developing a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses for driving long-term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities. We are seeking a technical leader for our Supply Science team. This team is within the Sponsored Product team, and works on complex engineering, optimization, econometric, and user-experience problems in order to deliver relevant product ads on Amazon search and detail pages world-wide. The team operates with the dual objective of enhancing the experience of Amazon shoppers and enabling the monetization of our online and mobile page properties. Our work spans ML and Data science across predictive modeling, reinforcement learning (Bandits), adaptive experimentation, causal inference, data engineering. Key job responsibilities Search Supply and Experiences, within Sponsored Products, is seeking an Applied Scientist to join a fast growing team with the mandate of creating new ads experience that elevates the shopping experience for our hundreds of millions customers worldwide. We are looking for a top analytical mind capable of understanding our complex ecosystem of advertisers participating in a pay-per-click model– and leveraging this knowledge to help turn the flywheel of the business. As an Applied Scientist on this team you will: --Build machine learning models, perform proof-of-concept, experiment, optimize, and deploy your models into production. --Run A/B experiments, gather data, and perform statistical analysis. --Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving. --Work closely with software engineers to assist in productionizing your ML models. --Research new machine learning approaches. A day in the life The successful candidate will be a self-starter comfortable with ambiguity, with strong attention to detail, and with an ability to work in a fast-paced, high-energy and ever-changing environment. The drive and capability to shape the direction is a must. About the team We are a customer-obsessed team of engineers, technologists, product leaders, and scientists. We are focused on continuous exploration of contexts and creatives where advertising delivers value to customers and advertisers. We specifically work on new ads experiences globally with the goal of helping shoppers make the most informed purchase decision. We obsess about our customers and we are continuously innovating on their behalf to enrich their shopping experience on Amazon
US, CA, San Diego
Amazon Private Brands is looking for a Data Scientist to join our Private Brand Intelligence (PBI) Sourcing Guidance team. PBI applies Machine Learning, Statistics, and Econometrics/Economics to derive actionable insights about the complex economy of Amazon’s retail business and develop statistical models and algorithms to drive strategic business decisions and improve operations. About the team We are an interdisciplinary team of Scientists, Economists, and Engineers, incubating and building day one solutions using cutting-edge technology, to solve some of the toughest business problems at Amazon. About the role You will work with business leaders, PMs, scientists, and economists to deep dive existing business problems, translate them into business and functional requirements and design concrete deliverables. These deliverables can include the design, development, testing of new in-house statistical models/ML models/Optimization engines, etc. and/or partnering with our sister teams to develop an improved version of an existing model/system. This is a unique, high visibility opportunity for someone who wants to have business impact, dive deep into large-scale economic problems and enable measurable actions on the consumer economy. We are particularly interested in candidates with experience applying stat, ML and OR concepts to business problems. To learn more about Amazon Science, please visit https://www.amazon.science (https://www.amazon.science/).