How we built Cedar with automated reasoning and differential testing

The new development process behind Amazon Web Services’ Cedar authorization-policy language.

Cedar is a new authorization-policy language used by the Amazon Verified Permissions and AWS Verified Access managed services, and we recently released it publicly. Using Cedar, developers can write policies that specify fine-grained permissions for their applications. The applications then authorize access requests by calling Cedar’s authorization engine. Because Cedar policies are separate from application code, they can be independently authored, updated, analyzed, and audited. 

Related content
CAV keynote lecture by the director of applied science for AWS Identity explains how AWS is making the power of automated reasoning available to all customers.

We want to assure developers that Cedar’s authorization decisions will be correct. To provide that assurance, we follow a two-part process we call verification-guided development when we’re working on Cedar. First, we use automated reasoning to prove important correctness properties about formal models of Cedar’s components. Second, we use differential random testing to show that the models match the production code. In this blog post we present an overview of verification-guided development for Cedar.

A primer on Cedar

Cedar is a language for writing and enforcing authorization policies for custom applications. Cedar policies are expressed in syntax resembling natural language. They define who (the principal) can do what (the action) on what target (the resource) under which conditions (when)?

To see how Cedar works, consider a simple application, TinyTodo, designed for managing task lists. TinyTodo uses Cedar to control who can do what. Here is one of TinyTodo’s policies:

// policy 1
permit(principal, action, resource)
when {
	resource has owner && resource.owner == principal
};

This policy states that any principal (a TinyTodo User) can perform any action on any resource (a TinyTodo List) as long as the resource’s creator, defined by its owner attribute, matches the requesting principal. Here’s another TinyTodo Cedar policy:

// policy 2
permit (
	principal,
	action == Action::"GetList",
	resource
)
when {
	principal in resource.editors || principal in resource.readers
};

This policy states that any principal can read the contents of a task list (Action::"GetList") if that principal is in either the list’s readers group or its editors group. Here is a third policy:

// policy 3
forbid (
	principal in Team::"interns",
	action == Action::"CreateList",
	resource == Application::"TinyTodo"
);

This policy states that any principal who is an intern (in Team::"interns") is forbidden from creating a new task list (Action::"CreateList") using TinyTodo (Application::"TinyTodo").

Related content
Meet Amazon Science’s newest research area.

When the application needs to enforce access, as when a user of TinyTodo issues a command, it only needs to make a corresponding request to the Cedar authorization engine. The authorization engine evaluates the request in light of the Cedar policies and relevant application data. If it returns decision Allow, TinyTodo can proceed with the command. If it returns decision Deny, TinyTodo can report that the command is not permitted.

How do we build Cedar to be trustworthy?

Our work on Cedar uses a process we call verification-guided development to ensure that Cedar’s authorization engine makes the correct decisions. The process has two parts. First, we model Cedar’s authorization engine and validator in the Dafny verification-aware programming language. With Dafny, you can write code, and you can specify properties about what the code is meant to do under all circumstances. Using Dafny’s built-in automated-reasoning capabilities we have proved that the code satisfies a variety of safety and security properties.

Second, we use differential random testing (DRT) to confirm that Cedar’s production implementation, written in Rust, matches the Dafny model’s behavior. We generate millions of diverse inputs and feed them to both the Dafny model and the production code. If both versions always produce the same output, we have a high degree of confidence that the implementation matches the model.

Cedar figure.png
Building Cedar using automated reasoning and differential testing.

Proving properties about Cedar authorization

 Cedar’s authorization algorithm was designed to be secure by default, as exemplified by the following two properties:

  • explicit permit — permission is granted only by individual permit policies and is not gained by error or default;
  • forbid overrides permit — any applicable forbid policy always denies access, even if there is a permit policy that allows it.

With these properties, sets of policies are easier to understand. Policy authors know that permit policies are the only way access is granted, and forbid policies decline access regardless of whether it is explicitly permitted.

Related content
Distributing proof search, reasoning about distributed systems, and automating regulatory compliance are just three fruitful research areas.

Given an authorization request, the Cedar authorization engine takes each Cedar policy and evaluates it after substituting the application request parameters into the principal, action and resource variables. For example, for the request principal= User::”Alice”, action=Action::”GetList”, and resource=List::”AliceList”, substituting for the variables in policy 1 would produce the expression List::”AliceList” has owner && List::”AliceList”.owner == User::”Alice”. If this expression evaluates to true, we say the request satisfies the policy. The authorization engine collects the satisfied forbid and permit policies into distinct sets and then makes its decision.

We model the authorization engine as a Dafny function and use Dafny’s automated-reasoning capabilities to state and prove the explicit-permit and forbid-overrides-permit properties. To see how this helps uncover mistakes, let’s consider a buggy version of the authorization engine:

function method isAuthorized(): Response { // BUGGY VERSION
	var f := forbids();
	var p := permits();
	if f != {} then
		Response(Deny, f)
	else
		Response(Allow, p)
}

The logic states that if any forbid policy is applicable (set f is not the empty set {}), the result should be Deny, thus overriding any applicable permit policies (in set p). Otherwise, the result is Allow. While this logic correctly reflects the desired forbid-overrides-permit property, it does not correctly capture explicit permit. Just because there are no applicable forbid policies doesn’t mean there are any applicable permit policies. We can see this by specifying and attempting to prove explicit permit in Dafny:

// A request is explicitly permitted when a permit policy is satisfied
predicate IsExplicitlyPermitted(request: Request, store: Store) {
	exists p ::
		p in store.policies.policies.Keys &&
		store.policies.policies[p].effect == Permit &&
		Authorizer(request, store).satisfied(p)
}
lemma AllowedIfExplicitlyPermitted(request: Request, store: Store)
ensures // A request is allowed if it is explicitly permitted
	(Authorizer(request, store).isAuthorized().decision == Allow) ==>
	IsExplicitlyPermitted(request, store)
{ ... }

A Dafny predicate is a function that takes arguments and returns a logical condition, and a Dafny lemma is a property to be proved. The IsExplicitlyPermitted predicate defines the condition that there is an applicable permit policy for the given request. The AllowedIfExplicitlyPermitted lemma states that a decision of Allow necessarily means the request was explicitly permitted. This lemma does not hold for the isAuthorized definition above; Dafny complains that A postcondition might not hold on this return path and points to the ensures clause.

Here is the corrected code:

function method isAuthorized(): Response {
	var f := forbids();
	var p := permits();
	if f == {} && p != {} then
		Response(Allow, p)
	else
		Response(Deny, f)
}

Now a response is Allow only if there are no applicable forbid policies, and there is at least one applicable permit policy. With this change, Dafny automatically proves AllowedIfExplicitlyPermitted. It also proves forbid overrides permit (not shown).

Related content
To mark the occasion of the eighth Federated Logic Conference (FloC), Amazon’s Byron Cook, Daniel Kröning, and Marijn Heule discussed automated reasoning’s prospects.

We have used the Cedar Dafny models to prove a variety of properties. Our most significant proof is that the Cedar validator, which confirms that Cedar policies are consistent with the application’s data model, is sound: if the validator accepts a policy, evaluating the policy should never result in certain classes of error. When carrying out this proof in Dafny, we found a number of subtle bugs in the validator’s design that we were able to correct.

We note that Dafny models are useful not just for automated reasoning but for manual reasoning, too. The Dafny code is much easier to read than the Rust implementation. As one measure of this, at the time of this writing the Dafny model for the authorizer has about one-sixth as many lines of code as the production code. Both Cedar users and tool implementers can refer to the Dafny models to quickly understand precise details about how Cedar works.

Differential random testing

Once we have proved properties about the Cedar Dafny model, we want to provide evidence that they hold for the production code, too, which we can do by using DRT to show that the model and the production code behave the same. Using the cargo fuzz random-testing framework, we generate millions of inputs — access requests, accompanying data, and policies — and send them to both the Dafny model engine and the Rust production engine. If the two versions agree on the decision, then all is well. If they disagree, then we have found a bug.

The main challenge with using DRT effectively is to ensure the necessary code coverage by generating useful and diverse inputs. Randomly generated policies are unlikely to mention the same groups and attributes chosen in randomly generated requests and data. As a result, pure random generation will miss a lot of core evaluation logic and overindex on error-handling code. To resolve this, we wrote several input generators, including ones that take care to generate policies, data, and requests that are consistent with one another, while also producing policies that use Cedar’s key language constructs. As of this writing, we run DRT for six hours nightly and execute on the order of 100 million total tests.

Related content
Rungta had a promising career with NASA, but decided the stars aligned for her at Amazon.

The use of DRT during Cedar’s development has discovered corner cases where there were discrepancies between the model and the production code, making it an important tool in our toolkit. For example, there was a bug in a Rust package we were using for IP address operations; the Dafny model exposed an issue in how the package was parsing IP addresses. Since the bug is in an external package, we fixed the problem within our code while we wait for the upstream fix. We also found subtle bugs in the Cedar policy parser, in how the authorizer handles missing application data, and how namespace prefixes on application data (e.g., TinyTodo::List::”AliceList”) are interpreted.

Learn more

In this post we have discussed the verification-guided development process we have followed for the Cedar authorization policy language. In this process, we model Cedar language components in the Dafny programming language and use Dafny’s automated-reasoning capabilities to prove properties about them. We check that the Cedar production code matches the Dafny model through differential random testing. This process has revealed several interesting bugs during development and has given us greater confidence that Cedar’s authorization engine makes correct decisions.

To learn more, you can check out the Cedar Dafny models and differential-testing code on GitHub. You can also learn more about Dafny on the Dafny website and the Cedar service on the Cedar website.

Related content

IN, KA, Bengaluru
Alexa International is looking for passionate, talented, and inventive Senior Applied Scientists to help build industry-leading technology with Large Language Models (LLMs) and multimodal systems, requiring strong deep learning and generative models knowledge. Senior applied scientists will drive cross-team scientific strategy, influence partner teams, and deliver solutions that have broad impact across Alexa's international products and services. Key job responsibilities As a Applied Scientist with II the Alexa International team, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art with LLMs, particularly delivering industry-leading scientific research and applied AI for multi-lingual applications — a challenging area for the industry globally. Your work will directly impact our global customers in the form of products and services that support Alexa+. You will leverage Amazon's heterogeneous data sources and large-scale computing resources to accelerate advances in text, speech, and vision domains. The ideal candidate possesses a solid understanding of machine learning, speech and/or natural language processing, modern LLM architectures, LLM evaluation & tooling, and a passion for pushing boundaries in this vast and quickly evolving field. They thrive in fast-paced environment, like to tackle complex challenges, excel at swiftly delivering impactful solutions while iterating based on user feedback, and are able to influence and align multiple teams around a shared scientific vision. A day in the life * Analyze, understand, and model customer behavior and the customer experience based on large-scale data. * Build novel online & offline evaluation metrics and methodologies for multimodal personal digital assistants. * Fine-tune/post-train LLMs using advanced and innovative techniques like SFT, DPO, Reinforcement Learning (RLHF and RLAIF) for supporting model performance specific to a customer’s location and language. * Quickly experiment and set up experimentation framework for agile model and data analysis or A/B testing. * Contribute through industry-first research to drive innovation forward. * Drive cross-team scientific strategy and influence partner teams on LLM evaluation frameworks, post-training methodologies, and best practices for international speech and language systems. * Lead end-to-end delivery of scientifically complex solutions from research to production, including reusable science components and services that resolve architecture deficiencies across teams. * Serve as a scientific thought leader, communicating solutions clearly to partners, stakeholders, and senior leadership. * Actively mentor junior scientists and contribute to the broader internal and external scientific community through publications and community engagement.
US, NY, New York
About the Role In this role, you will own the science strategy and technical vision for this intelligence layer, leading a team of applied scientists working across GenAI and predictive modeling. You will shape how heterogeneous signals — text, behavioral, network, temporal — come together to power talent applications at Amazon scale, from workforce forecasting to personalized development to compensation strategy. You will identify opportunities where science investment can have material impact on long-term objectives or annual goals and build consensus around needed investments, working comfortably across different modeling paradigms and data modalities to guide principal and senior scientists in their most challenging and strategic decisions while serving as the strategic science advisor to PXT leaders operating at the Director, VP, and SVP levels. As a hands-on leader, you will personally own development and delivery of the most complex science problems at the intersection of multiple ML disciplines, stay current with emergent AI/ML science and engineering trends to influence focus areas in a rapidly evolving landscape, and participate in organizational planning, hiring, mentorship, and leadership development. Key job responsibilities • Lead technical initiatives in people science models, driving breakthrough approaches through hands-on research and development in areas like foundation models for predictive modeling, efficient multi-modal LLMs, and zero-shot learning • Design and implement novel ML architectures that push the boundaries of how workforce signals are represented, fused, and predicted at scale • Guide technical direction for research initiatives across the team, ensuring robust performance in production environments serving hundreds of thousands of employees • Mentor and develop senior scientists while maintaining strong individual technical contributions on the most complex cross-domain problems • Collaborate with engineering teams to optimize and scale models for real-world talent applications • Influence technical decisions and implementation strategies across teams, shaping the long-term platform architecture About the team The People eXperience and Technology (PXT) Core Science Team uses science, engineering, and customer-obsessed problem solving to proactively identify mechanisms, process improvements, and products that simultaneously improve Amazon and Amazonians' lives, wellbeing, and value of work. As an interdisciplinary team combining talents from machine learning, statistics, economics, behavioral science, engineering, and product development, the Core Science team develops and delivers measurable solutions through innovation and rapid prototyping to accelerate informed, accurate, and reliable decision-making backed by science and data.
US, MA, N.reading
Amazon is seeking exceptional talent to help develop the next generation of advanced robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine cutting-edge AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at an unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic dexterous manipulation, locomotion, and human-robot interaction. This role presents an opportunity to shape the future of robotics through innovative applications of deep learning and large language models. At Amazon we leverage advanced robotics, machine learning, and artificial intelligence to solve complex operational challenges at an unprecedented scale. Our fleet of robots operates across hundreds of facilities worldwide, working in sophisticated coordination to fulfill our mission of customer excellence. The ideal candidate will contribute to research that bridges the gap between theoretical advancement and practical implementation in robotics. You will be part of a team that's revolutionizing how robots learn, adapt, and interact with their environment. Join us in building the next generation of intelligent robotics systems that will transform the future of automation and human-robot collaboration. Key job responsibilities - Design and implement whole body control methods for balance, locomotion, and dexterous manipulation - Utilize state-of-the-art in methods in learned and model-based control - Create robust and safe behaviors for different terrains and tasks - Implement real-time controllers with stability guarantees - Collaborate effectively with multi-disciplinary teams to co-design hardware and algorithms for loco-manipulation - Mentor junior engineer and scientists
IN, KA, Bengaluru
Have you ever ordered a product on Amazon and when that box with the smile arrived you wondered how it got to you so fast? Have you wondered where it came from and how much it cost Amazon to deliver it to you? If so, the WW Amazon Logistics, Business Analytics team is for you. We manage the delivery of tens of millions of products every week to Amazon’s customers, achieving on-time delivery in a cost-effective manner. We are looking for an enthusiastic, customer obsessed, Applied Scientist with good analytical skills to help manage projects and operations, implement scheduling solutions, improve metrics, and develop scalable processes and tools. The primary role of an Operations Research Scientist within Amazon is to address business challenges through building a compelling case, and using data to influence change across the organization. This individual will be given responsibility on their first day to own those business challenges and the autonomy to think strategically and make data driven decisions. Decisions and tools made in this role will have significant impact to the customer experience, as it will have a major impact on how the final phase of delivery is done at Amazon. Ideal candidates will be a high potential, strategic and analytic graduate with a PhD in (Operations Research, Statistics, Engineering, and Supply Chain) ready for challenging opportunities in the core of our world class operations space. Great candidates have a history of operations research, and the ability to use data and research to make changes. This role requires robust program management skills and research science skills in order to act on research outcomes. This individual will need to be able to work with a team, but also be comfortable making decisions independently, in what is often times an ambiguous environment. Responsibilities may include: - Develop input and assumptions based preexisting models to estimate the costs and savings opportunities associated with varying levels of network growth and operations - Creating metrics to measure business performance, identify root causes and trends, and prescribe action plans - Managing multiple projects simultaneously - Working with technology teams and product managers to develop new tools and systems to support the growth of the business - Communicating with and supporting various internal stakeholders and external audiences
GB, London
Come build the future of entertainment with us. Are you interested in shaping the future of movies and television? Do you want to define the next generation of how and what Amazon customers are watching? Prime Video is a premium streaming service that offers customers a vast collection of TV shows and movies - all with the ease of finding what they love to watch in one place. We offer customers thousands of popular movies and TV shows including Amazon Originals and exclusive licensed content to exciting live sports events. We also offer our members the opportunity to subscribe to add-on channels which they can cancel at anytime and to rent or buy new release movies and TV box sets on the Prime Video Store. Prime Video is a fast-paced, growth business - available in over 200 countries and territories worldwide. The team works in a dynamic environment where innovating on behalf of our customers is at the heart of everything we do. If this sounds exciting to you, please read on. The Insights team is looking for an Applied Scientist for our London office experienced in generative AI and large models. This is a wide impact role working with development teams across the UK, India, and the US. This greenfield project will deliver features that reduce the operational load for internal Prime Video builders and for this, you will need to develop personalized recommendations for their services. You will have strong technical ability, excellent teamwork and communication skills, and a strong motivation to deliver customer value from your research. Our position offers opportunities to grow your technical and non-technical skills and make a global impact immediately. Key job responsibilities - Develop machine learning algorithms for high-scale recommendations problems - Rapidly design, prototype and test many possible hypotheses in a high-ambiguity environment, making use of both quantitative analysis and business judgement - Collaborate with software engineers to integrate successful experimental results into Prime Video wide processes - Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports A day in the life You will lead the design of machine learning models that scale to very large quantities of data across multiple dimensions. You will embody scientific rigor, designing and executing experiments to demonstrate the technical effectiveness and business value of your methods. You will work alongside other scientists and engineering teams to deliver your research into production systems. About the team Our team owns Prime Video observability features for development teams. We consume PBs of data daily which feed into multiple observability features focussed on reducing the customer impact time.
CN, 31, Shanghai
You will be working with a unique and gifted team developing exciting products for consumers. The team is a multidisciplinary group of engineers and scientists engaged in a fast paced mission to deliver new products. The team faces a challenging task of balancing cost, schedule, and performance requirements. You should be comfortable collaborating in a fast-paced and often uncertain environment, and contributing to innovative solutions, while demonstrating leadership, technical competence, and meticulousness. Your deliverables will include development of thermal solutions, concept design, feature development, product architecture and system validation through to manufacturing release. You will support creative developments through application of analysis and testing of complex electronic assemblies using advanced simulation and experimentation tools and techniques. Key job responsibilities * Evaluate and optimize thermal solution requirements of consumer electronic products * Use simulation tools like Star-CCM+ or FloTherm XT/EFD for analysis and design of products * Validate design modifications for thermal concerns using simulation and actual prototypes * Establish temperature thresholds for user comfort level and component level considering reliability requirements * Have intimate knowledge of various materials and heat spreaders solutions to resolve thermal issues * Use of programming languages like Python and Matlab for analytical/statistical analyses and automation * Collaborate as part of device team to iterate and optimize design parameters of enclosures and structural parts to establish and deliver project performance objectives * Design and execute of tests using statistical tools to validate analytical models, identify risks and assess design margins * Create and present analytical and experimental results * Develop and apply design guidelines based on project learnings
US, CA, San Francisco
MULTIPLE POSITIONS AVAILABLE Employer: AMAZON DEVELOPMENT CENTER U.S., INC., Offered Position: Research Scientist II Job Location: San Francisco, California Job Number: AMZ9674001 Position Responsibilities: Design research studies to obtain scientific information. Develop theories or models of physical phenomena encountered in quantum computing, superconducting qubit device physics, materials or process development and characterization. Collaborate with others to determine design specifications, including of superconducting quantum processor chips, microwave chip packages, and associated electrical and mechanical components. Develop scientific or mathematical models to predict physical device behavior and performance, and verify the implementation of computational models. Apply mathematical principles or statistical approaches to solve problems, for example to validate modeling predictions under experimental uncertainty using statistical methods. Operate laboratory or field equipment and scientific instrumentation for device fabrication, device characterization, or advanced materials research. Develop new algorithms or methods for designing, simulating, or measuring quantum computers. Develop performance metrics or standards related to quantum information technology. Recommend technical design or process changes to improve quality or performance of superconducting quantum processors and efficiency of their design, manufacture, and testing. Collaborate on research activities with scientists or technical specialists. Prepare scientific or technical reports or presentations and present research results to others. 40 hours / week, 8:00am-5:00pm, Salary Range $168,126/year to $212,800/year. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, visit: https://www.aboutamazon.com/workplace/employee-benefits. Amazon.com is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation.#0000
US, WA, Seattle
This role leads the science function in WW Stores Finance as part of the IPAT organization (Insights, Planning, Analytics and Technology), driving transformative innovations in financial analytics through AI and machine learning across the global Stores finance organization. The successful candidate builds and directs a multidisciplinary team of data scientists, applied scientists, economists, and product managers to deliver scalable solutions that fundamentally change how finance teams generate insights, automate workflows, and make decisions. As part of the WW Stores Finance leadership team, this leader partners with engineering, product, and finance stakeholders to translate emerging AI capabilities into production systems that deliver measurable improvements in speed, accuracy, and efficiency. The role's outputs directly inform VP/SVP/CFO/CEO leadership decisions and drive impact across the entire Stores P&L. Success requires translating complex technical concepts for finance domain experts and business leaders while maintaining deep technical credibility with science and engineering teams. The role demands both strategic vision—identifying high-impact opportunities where AI can transform finance operations—and execution excellence in coordinating project planning, resource allocation, and delivery across multiple concurrent initiatives. This leader establishes methodologies and models that enable Amazon finance to achieve step-change improvements in both the speed and quality of business insights, directly supporting critical processes including month-end reporting, quarterly guidance, annual planning cycles, and financial controllership. Key job responsibilities Transformation of Finance Workflows — Lead development of agentic AI solutions that automate routine finance tasks and transform how teams communicate business insights. Deploy these solutions across financial analysis, narrative generation, and dynamic table creation for month-end reporting and planning cycles. Partner with engineering and product teams to integrate these capabilities into production systems that directly support Stores Finance and FGBS automation goals, delivering measurable reductions in manual effort and cycle time. Science-Based Forecasting — Develop and deploy machine learning forecasts that integrate into existing planning processes including OP1, OP2, and quarterly guidance cycles. Partner with finance teams across WW Stores to iterate on forecast accuracy, applying these models either as alternative viewpoints to complement bottoms-up forecasts or as hands-off replacements for manual forecasting processes. Establish evaluation frameworks that demonstrate forecast performance against business benchmarks and drive adoption across critical planning workflows. Financial Controllership — Scale AI capabilities across controllership workstreams to improve reporting accuracy and automate manual processes. Leverage generative AI to identify financial risk through systematic pattern recognition in transaction data, account reconciliations, and variance analysis. Develop production systems that enhance decision-making speed and quality in financial close, audit preparation, and compliance reporting, delivering quantifiable improvements in error detection rates and process efficiency. About the team IPAT (Insights, Planning, Analytics, and Technology) is a team in the Worldwide Amazon Stores Finance organization composed of leaders across engineering, finance, product, and science. Our mission is to reimagine finance using technology and science to provide fast, efficient, and accurate insights that drive business decisions and strengthen governance. We are dedicated to improving financial operations through innovative applications of technology and science. Our work focuses on developing adaptive solutions for diverse financial use cases, applying AI to solve complex financial challenges, and conducting financial data analysis. Operating globally, we strive to develop adaptable solutions for diverse markets. We aim to advance financial science, continually improving accuracy, efficiency, and insight generation in support of Amazon's mission to be Earth's most customer-centric company.
US, NY, New York
Do you want to lead the Ads industry and redefine how we measure the effectiveness of Amazon Ads business? Are you passionate about causal inference, Deep Learning/DNN, raising the science bar, and connecting leading-edge science research to Amazon-scale implementation? If so, come join Amazon Ads to be an Economist leader within our Advertising Incrementality Measurement science team! Our work builds the foundations for providing customer-facing experimentation tools, furthering internal research & development on Econometrics, and building out Amazon's advertising measurement offerings. Incrementality is a lynchpin for the next generation of Amazon Advertising measurement solutions and this role will play a key role in the release and expansion of these offerings. Key job responsibilities As an Economist leader within the Advertising Incrementality Measurement (AIM) science team, you are responsible for defining and executing on key workstreams within our overall causal measurement science vision. In particular, you can lead the development of experimental methodologies to measure ad effectiveness, and also build observational models that lay the foundations for understanding the impact of individual ad touchpoints for billions of daily ad interactions. You will work on a team of Applied Scientists, Economists, and Data Scientists, alongside a dedicated Engineering team, to work backwards from customer needs and translate product ideas into concrete science deliverables. You will be a thought leader for inventing scalable causal measurement solutions that support highly accurate and actionable insights--from defining and executing hundreds of thousands of RCTs, to developing an exciting science R&D agenda. You will be working with massive data and industry-leading partner scientists, while also interfacing with leadership to define our future vision. Your work will help shape the future of Amazon Advertising. About the team AIM is a cross disciplinary team of engineers, product managers, economists, data scientists, and applied scientists with a charter to build scientifically-rigorous causal inference methodologies at scale. Our job is to help customers cut through the noise of the modern advertising landscape and understand what actions, behaviors, and strategies actually have a real, measurable impact on key outcomes. The data we produce becomes the effective ground truth for advertisers and partners making decisions affecting millions in advertising spend.
US, NY, New York
The Measurement Intelligence Science Team (MIST) in the Measurement, Ad Tech, and Data Science (MADS) organization of Amazon Ads serves a centralized role developing solutions for a multitude of performance measurement products. We create solutions which measure the comprehensive impact of their ad spend, including sales impacts both online and offline and across timescales, and provide actionable insights that enable our advertisers to optimize their media portfolios. We leverage a host of scientific technologies to accomplish this mission, including Generative AI, classical ML, Causal Inference, Natural Language Processing, and Computer Vision. As an Applied Science Manager on the team, you will lead a team of scientists to define and execute a transformative vision for holistic measurement and reporting insights for ad effectiveness. Your team will own the science solutions for foundational experimentation platforms, foundational customer journey understanding technologies, state of the art attribution algorithms to measure the role of advertising in driving observed retail outcomes, and/or agentic AI solutions that help advertisers get quick access to custom insights that inform how to get the most out of their ad spend. Key job responsibilities You independently manage a team of scientists. You identify the needs of your team and effectively grow, hire, and promote scientists to maintain a high-performing team. You have a broad understanding of scientific techniques, several of which may fall out of your specific job function. You define the strategic vision for your team. You establish a roadmap and successfully deliver scientific solutions that execute that vision. You define clear goals for your team and effectively prioritize, balancing short-term needs and long-term value. You establish clear and effective metrics and scientific process to enforce consistent, high-quality artifact delivery. You proactively identify risks and bring them to the attention of your manager, customers, and stakeholders with plans for mitigation before they become roadblocks. You know when to escalate. You communicate ideas effectively, both verbally and in writing, to all types of audiences. You author strategic documentation for your team. You communicate issues and options with leaders in such a way that facilitates understanding and that leads to a decision. You work successfully with customers, leaders, and engineering teams. You foster a constructive dialogue, harmonize discordant views, and lead the resolution of contentious issues. About the team We are a team of scientists across Applied, Research, Data Science and Economist disciplines. You will work with colleagues with deep expertise in ML, NLP, CV, Gen AI, and Causal Inference with a diverse range of backgrounds. We partner closely with top-notch engineers, product managers, sales leaders, and other scientists with expertise in the ads industry and on building scalable modeling and software solutions.