Social
Machine learning

Can we change an AI model’s architecture without retraining it from scratch?

“Priming” reuses pretrained Transformer weights to build models with more efficient architectures at less than 1% of the original training cost and with throughputs up to twice as high.

By Luca Zancato
October 12, 2026
14 min read
Share
Key takeaways
  • A new AWS paper introduces priming, a method that reduces pretraining costs for hybrid architectures by over 99% by leveraging pretrained Transformer weights, saving trillions of tokens and hundreds of thousands of GPU-hours.
  • A 32-billion-parameter Gated KalmaNet (GKA)-based hybrid model matches its source Transformer on reasoning benchmarks while decoding up to twice as quickly with 128,000 tokens of context.
  • State space models consistently outperform sliding-window attention (SWA) even when SWA receives twice the memory budget, and GKA's variable-iteration mechanism enables 10% faster decoding with only ~1% accuracy loss.
Was this answer helpful?

AI agents are rapidly scaling: longer sessions, deeper reasoning chains, more autonomy. But the Transformer architecture underneath them still pays a memory tax that grows with the number of tokens it processes. As agents become more capable, that cost scales with them.

Now, there’s a proliferation of hybrid architectures that use some combination of methods to achieve a better balance of memory, speed, and quality. Those methods include sparse attention, which selects a subset of past tokens to consider when selecting the next token; sliding-window attention, which preserves only the most recent tokens; and state space models, which compress information from the full history into a fixed-size state.

These hybrid architectures can be understood as points in a sequence-model design space: to differing degrees, they mix information while traversing a sequence and passing through different states. What distinguishes them from each other is how they retain and use the past.

Until now, exploring this design space has required training every new architecture from scratch, using trillions of tokens and hundreds of thousands of GPU-hours. This has made controlled comparisons of hybrid architectures extremely expensive.

In a new paper, “Priming: Hybrid state space models from pre-trained Transformers,” our team at Amazon Web Services addresses this problem. We introduce priming, a method that leverages pretrained Transformer weights as a starting point to reduce the pretraining cost of hybrid architectures by more than 99%.

With priming, model developers can ask questions that were previously too expensive to answer at scale: Which attention layers can be replaced by more-efficient layers? How many attention layers can be replaced without sacrificing quality? Which state space layer works best? And what does the trade-off between speed and accuracy look like?

In our paper, we compare hybrid models that use three different types of state space models (Gated KalmaNet, Gated DeltaNet, and Mamba-2). Some of our findings:

  • Not all state space models are equal. Under controlled conditions, Gated KalmaNet (GKA) performs best, followed by Gated DeltaNet (GDN) and then Mamba-2. GKA uniquely offers variable-length inference computation without retraining and gives the user direct control over its position on the speed-accuracy frontier.
    Clock time vs. accuracy graph.png
    A 32-billion-parameter hybrid model reaches each accuracy threshold faster than an equivalently sized Transformer, converging to similar final accuracy in less wall clock time. The dataset is the AIME 2025 math competition benchmark (hard subset). Accuracy at time t is the fraction of rollouts for which a correct final answer has been produced by time t.
  • A good lossy summary of everything beats a perfect recording of the most recent past. State space models consistently outperform sliding-window attention (SWA), even when SWA gets twice the memory budget.
  • Same quality, twice the throughput. A 32-billion-parameter GKA-based hybrid model matches its source Transformer on reasoning benchmarks while decoding up to twice as quickly with 128,000 tokens of context and reaching correct answers in as little as two-thirds of the wall-clock time.

To foster further research on hybrid architectures, we have released a zoo of primed hybrid models at the eight-billion-parameter and 32-billion-parameter scales for long-context reasoning and instruction following, together with training code, optimized kernels, and a vLLM-serving plugin under the Apache 2.0 License.

The attention mechanism

A Transformer’s attention mechanism — which determines how relevant previously seen tokens are to the token currently being processed — relies on three different vectors, which take their names from database design: the query, the key, and the value. For each token, the query captures what that token is “looking for,” the key captures what it “offers,” and the value carries the actual information it will pass along. Relevance is computed by comparing the current token’s query against every other token’s key; the closer the match, the more that token’s value contributes to the output of attention.

Each of these vectors is produced by multiplying the token by a learned weight projection matrix. The Transformer outputs one token at a time, and each output token becomes a new input, so the context grows one token at a time as well. To save computation, the Transformer thus stores the key and value projections of each new input in a “KV cache”.

Two kinds of memory layers: Eidetic versus fading

The attention mechanism gives Transformers what we call eidetic memory: verbatim recall of every past token. The possibility of recalling over an LLM’s entire context window is what made in-context learning, long-document reasoning, and retrieval-augmented generation possible. But it comes at a cost that scales with the number of input tokens.

However, not all attention layers do the same job. Some perform long-range retrieval across tens of thousands of tokens. Others operate more locally, tracking the current clause, copying nearby tokens, and the like. The local ones don't need a growing cache: a compact, fixed-size summary of the past — what we call fading memory — suffices.

Hybrid models mix both: attention layers (eidetic, growing cache) for the layers that need global recall and state space layers (fading, fixed size) for the rest. The result is a model with a reduced memory footprint that still has access to verbatim recall where it matters.

Priming: Knowledge isn't locked to the architecture

Exploring hybrids’ design space has been impractical because each hybrid configuration would normally require pretraining from scratch. Priming sidesteps this by reusing the stored knowledge in a pretrained Transformer's weights.

Priming is possible because a Transformer is itself a special case of a state space realization. The outputs of both attention and SSM layers can be expressed as the result of multiplying the value sequence by a “mixing” matrix that decides how much each past token contributes to each current token. Attention computes the mixing matrix by multiplying the query and key matrices; an SSM produces a similar output through a recurrent state of fixed dimension that all past information must pass through.

Priming takes advantage of this structural similarity: if an SSM with a fixed-dimensional state can produce a mixing matrix close enough to the attention layer's, the query, key, and value projections from the source Transformer can be reused as the SSM's own input projections, with only the SSM-specific parameters needing to be learned.

Whether the approximation holds depends on whether the attention layer performs complex, long-range retrievals or simple, local operations. We distinguish between the two using a cheap empirical proxy: replace each layer with an SWA — which preserves only a fraction of the context — and measure the degradation.

Layers that tolerate this substitution are natural candidates for SSM conversion: their mixing matrices don’t need to capture long-range relationships, so the state of an SSM can faithfully represent what they do. This proxy connects to a formal quantity from dynamical-systems theory, the Hankel rank of the layer's mixing matrix, which measures how much information must pass through the state bottleneck and therefore the minimum SSM state dimension required to realize the layer exactly.

Priming proceeds in three stages:

  1. Layer selection: identify which layers can be faithfully compressed into a fixed-size state while retaining the rest as attention;
  2. Weight transfer and alignment: initialize SSM layers from the corresponding attention weights, then run a short distillation process aligning each SSM to the behavior of the layer it replaces;
  3. End-to-end adaptation: fine-tune the full hybrid model on next-token prediction, letting all layers coadapt.
Workflow.png
Priming begins with the use of a simple proxy (a sliding-window attention) to identify attention layers that largely operate locally. The weights of those layers are distilled into state-space-model (SSM) layers, which are then aligned with the behaviors of the layers they replace. Finally, the full hybrid model is fine-tuned on next-token prediction.

The entire pipeline consumes less than 0.5% of the source model's pretraining token budget, and it works across Transformer families (Qwen, Llama, Mistral), model classes (dense or MoE), and scales (eight billion to 32 billion and more).

With priming we can now explore the design space, the layer type, the hybrid ratio, and layer placement cheaply and systematically, because every variant starts from the same strong initialization and requires minimal training, compared to pretraining.

A controlled comparison of fading-memory layers

For instance, consider this question: given the same data, the same training budget, and the same ratio of eidetic to fading layers, which SSM layer works best in a hybrid model?

In our paper, we evaluate three families of state space models, each representing a different approach to using a fixed-size state:

  • Mamba-2 applies a uniform scalar decay to the entire state at each step, a running average that forgets everything at the same rate. When a new token arrives, it is written into the state, while all previous entries are uniformly attenuated. Mamba-2 is simple and efficient but indiscriminate.
  • Gated DeltaNet (GDN) adds targeted updates on top of uniform decay. Rather than decaying everything uniformly, it can selectively erase the value previously associated with a specific key before writing the new one (the delta rule). This makes it more expressive than Mamba-2: it can revise a specific association without uniformly decaying the rest.
  • Gated KalmaNet (GKA) goes further. Its erase direction depends not only on the current key but on statistics of all past keys, a history-aware update that accounts for the cumulative structure of everything the model has seen.
Qwen accuracy on four datasets.png
The results of hybrid models using Mamba-2, GDN, and GKA state space models on four tasks. GKA generally outperforms GDN, which always outperforms Mamba-2, in line with theoretical predictions.

The theoretical expressiveness hierarchy predicts the empirical one: GKA > GDN > Mamba-2. The gap widens on long-context reasoning, where compression quality matters most, and narrows on short-context tasks. Furthermore, at the 32-billion-parameter (32B) scale, hybrid GKA is within 1% of its source Transformer trained on the same data, despite requiring half the KV cache at inference.

Fading memory beats verbatim windowed memory

An interesting finding in our study comes from a direct comparison of hybrid models using SSM layers and those using sliding-window attention (SWA), which retain a fixed window of recent tokens verbatim rather than compressing the past into a state. One might expect that verbatim recall, even over a limited window, would outperform lossy compression. After all, no information is lost within the sliding window’s range. And stacking multiple SWA layers increases the “receptive field” of the end-to-end model.

Instead, every SSM-based hybrid we tested outperformed the corresponding SWA hybrid on long-context benchmarks, even when the window gets two to four times as much memory as the state.

Qwen vs. hybrid.png
Hybrid models using lossy context summaries (orange) consistently outperformed hybrid models using sliding context windows (teal).

A runtime knob for speed versus accuracy: Variable computation at inference time

GKA has a property that neither Mamba-2 nor GDN offers: variable test-time computation.

Mamba-2 and GDN have the same computation budget for every input token, regardless of its content. GKA, by contrast, solves a regression problem iteratively, and the iteration count is adjustable at serving time without retraining.

Long context vs. speedup.png
The Gated KalmaNet (GKA) accuracy-speed frontier, at four different context lengths. Geometric shapes indicate different numbers of iterations on a regression problem.

In practice, serving GKA at 10 iterations instead of its usual 30 sacrifices roughly 1% of its accuracy on AIME 2025 while delivering 10% faster decoding. A single trained model can serve multiple points on the speed-accuracy frontier, so model providers can offer different latency tiers without maintaining separate model variants.

This connects to a principle we have discussed in earlier work: intelligence is the ability to allocate effort commensurate with difficulty. GKA's variable-computation mechanism is an architectural embodiment of this principle: a deployment that demands speed can run fewer iterations, while one that demands accuracy can run more.

Faster reasoning, not just faster tokens

Raw per-token throughput tells only part of the story. A model that is cheaper per token can still be slower end to end if its reasoning is weaker and it needs more tokens to compensate. The deployment-relevant question is how much more rapidly, in wall clock terms, the hybrid reaches a target accuracy.

We measured this on AIME 2025, a challenging math competition benchmark, comparing our GKA-Primed-HQwen3-32B-Reasoner against the source Qwen3-32B trained on the same reasoning recipe. Both models generate similar-length thinking traces (a median of about 14,000 tokens on the full set, 28,000 on the hardest problems) — that is, they produce equally in-depth responses — but the hybrid has a much higher throughput.

At realistic concurrency levels, where the Transformer's KV cache saturates GPU memory, the hybrid reaches target accuracy thresholds 1.2 to 1.4 times as quickly on the full benchmark and up to 1.6 times as quickly on the hardest problems. This is a consequence of the SSM layers' fixed-size state, which roughly halves the per-sequence memory footprint, enabling more concurrent operations on the same hardware.

Decode throughput vs context length.png
Decode throughput speedup of GKA-Primed-HQwen3-32B over the source Transformer as a function of context length. Replacing half the attention layers with a fixed-size SSM state roughly halves the size of the KV cache, yielding up to 2.06 times the throughput at a context length of 128,000 tokens. Lowering the number of GKA solver iterations from 30 to 10 at serving time (no retraining necessary) trades a small accuracy margin for additional throughput.

The throughput advantage grows with context length: 1.50× at 16K tokens, scaling to 2.06× at 128K for the 32B model (GKA at 10 iterations). With the 8B parameter models, the gains are even greater: 1.94× at 16K to 2.29× at 128K. At long context, KV cache reads dominate the forward pass through the Transformer, and halving the cache roughly doubles the effective batch size at memory saturation.

This has implications beyond inference serving. In reinforcement learning (RL) post-training pipelines, throughput gates how rapidly the policy generates correct trajectories. The hybrid's advantage translates directly to a 20% to 100% increase in RL training data rate at a fixed GPU budget.

Comparison of state space models.png
Reasoning accuracy of GKA-primed hybrid models versus source Qwen3 Transformer models across five benchmarks at eight-billion-parameter and 32-billion-parameter scale. Primed hybrid models match their source Transformers in quality at half the memory, and they decode faster. At eight billion parameters, the GKA hybrid uniformly outperforms its GDN counterpart, consistent with the theoretical expressiveness hierarchy GKA > GDN.

Beyond the training window: Context extension with state composition

Our primed hybrids natively support 128,000 tokens. But agent sessions, codebases, and long legal documents routinely exceed that. Extending context further would normally require another round of continued pretraining on curated data. Can the hybrid architecture itself generalize to longer contexts without additional training?

Existing training-free methods for expanding the context window fall into two classes: those for pure Transformers (positional-embedding rescaling schemes like YaRN) and those for pure SSMs, such as our PICASO, which we presented last year at the International Conference on Learning Representations (ICLR). Neither addresses the hybrid setting, where KV caches and recurrent states must be reconciled jointly.

In our new arXiv paper, we describe a method of extending hybrid models’ context beyond their native limits without training. The procedure is simple:

  • partition a long input into chunks shorter than the native context length and prepend a shared prefix to each chunk;
  • process all chunks independently in parallel;
  • concatenate the KV caches across chunks while merging the SSM states via the weighted-averaging scheme we introduced in our PICASO paper.
State composition for hybrid models.png
State composition for hybrid models. Long inputs are partitioned into chunks at the native context length and processed independently. KV caches are concatenated, while SSM states are merged via the weighted-averaging scheme we proposed in our PICASO paper.

This approach enables us to extend the context length of our primed hybrid models from 128K to 256K tokens while preserving substantial accuracy on both retrieval (RULER NIAH) and reasoning (BABILong) tasks at lengths the model was never trained on. In our experiments we show that hybrid state composition preserves the SSM expressivity hierarchy (GKA > GDN > Mamba-2).

The hybrid design space is now open

All artifacts discussed in this blog are released under the Apache 2.0 License in our Hybrid Model Factory on GitHub and our model zoo on HuggingFace:

  • A collection of primed hybrids for long-context reasoning and instruction following (8B and 32B);
  • The priming training code;
  • Point-to-point sequence parallelism algorithms for SSM layers (up to 2.3 times as fast as all-to-all-based alternatives at multinode scale);
  • Universal sequence parallelism for prototyping novel layers on long sequences without custom kernels or distributed-training expertise;
  • Optimized Triton kernels for GKA decoding and training;
  • A vLLM serving plugin supporting primed hybrid architectures with continuous batching, chunked prefill, and tensor parallelism for inference deployment at scale.

If you serve long-context workloads, the 32B GKA reasoning model with the vLLM plugin gives you higher throughput with no quality loss on long context reasoning tasks.

If you're scaling RL post-training, the hybrid's throughput advantage means more correct rollouts per unit time at a fixed GPU budget, for faster policy iteration without changing your training recipe.

If you're researching new architectures, priming lets you evaluate any layer type under controlled conditions at a fraction of the pretraining cost while starting from a strong initialization. The universal SP module means you don't need custom distributed kernels to train at long context.

Priming is not tied to a specific Transformer family or model class. Extending it to Llama, Mistral, or MoE models requires no algorithmic changes. We hope these tools enable the community to explore architectures beyond what we've tested and to develop new algorithms and training recipes for building scalable next-generation architectures for AI agents.

For full details, see our paper, "Priming: Hybrid state space models from pre-trained Transformers". Code, models, and serving plugin are available on GitHub, and the models are also on Hugging Face.

Research areas
Tags
About the Author
Luca Zancato
Luca Zancato is a senior applied scientist in Amazon Web Services' Fundamental Research organization.

Related content

Work with us

See more jobs See more jobs
Applied Scientist, Mobile Manipulation Robotics (I/O)
US, MA, North Reading
Amazon Robotics is transforming warehouse automation through edge AI and machine learning applied to real-world robotics challenges. We're seeking a Applied Scientist to advance our mobile manipulation capabilities by developing learning-based approaches that enable robots to navigate and manipulate objects in dynamic fulfillment environments. This role offers the opportunity to apply state-of-the-art research to production systems operating at Amazon's unprecedented scale. Key job responsibilities - Model Development and Training: Designing and implementing the model architectures, training and fine tuning the models using various datasets, and optimize the model performance through iterative experiments - Data Management: Process and prepare training data, including data governance, provenance tracking, data quality checks and creating reusable data pipelines. - Experimentation and Validation: Design and execute experiments to test model capabilities on the simulator and on the embodiment, validate performance across different scenarios, create a baseline and iteratively improve model performance. - Code Development: Write clean, maintainable, well commented and documented code, contribute to training infrastructure, create tools for model evaluation and testing, and implement necessary APIs - Research: Stay current with latest developments in foundation models and robotics, assist in literature reviews and research documentation, prepare technical reports and presentations, and contribute to research discussions and brainstorming sessions. - Collaboration: Work closely with senior scientists, engineers, and leaders across multiple teams, participate in knowledge sharing, support integration efforts with robotics hardware teams, and help document best practices and methodologies. A day in the life Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! About the team Are you inspired by invention? Is problem solving through teamwork in your DNA? Do you like the idea of seeing how your work impacts the bigger picture? Answer yes to any of these and you’ll fit right in here at Amazon Robotics. We are a smart, collaborative team of enthusiastic doers that work passionately to apply innovative advances in robotics and software to solve real-world challenges that will transform our customers’ experiences in ways we can’t even image yet. We invent new improvements every day. We are Amazon Robotics and we will give you the tools and support you need to invent with us in ways that are rewarding, fulfilling and fun!
Principal Applied Scientist, Items and Relationships Platform
US, WA, Seattle
Interested in setting the scientific vision for how every product at Amazon is understood, identified, and connected, with the opportunity to impact every Amazon customer and every selling partner? The Items and Relationships Platform is the foundation that establishes what every product in Amazon’s catalog is and how products relate to one another. We are rebuilding the platform to establish identity and relationships from meaning and context, using multimodal large language models and generative AI. This is a foundational problem: understanding products deeply enough to identify them, relate them, and keep the catalog trustworthy as it scales to the full breadth of Amazon’s global contribution traffic. A Principal Applied Scientist on the Items and Relationships Platform owns the science direction for product identity and product relationships. It spans how contributions are resolved into a committed product identity, how products are connected into structural relationships (variations, multipacks, newer versions, lines of products, bundles), how confident those decisions are, and how their quality is measured. You will own the strategy for multimodal understanding, representation learning, relationship inference, calibration, and evaluation, so that science capability compounds across identity and relationships rather than being built one use case at a time. This role lives at the intersection of research and production. Product identity and relationships are fundamentally an entity resolution problem: deciding what is the same product, and how different products relate, across diverse sources, languages, and formats. You will make the hard modeling calls on where sameness ends and relatedness begins, how models express and calibrate confidence, and how the platform measures quality when there is no external ground truth. Successful candidates are passionate about translating ambiguous business goals into a coherent science roadmap that enables many businesses at once. You are comfortable being the final scientific arbiter on hard cross-team decisions, and working alongside top-notch applied scientists and software engineers to turn advanced research into production systems. Key job responsibilities As a Principal Applied Scientist on the Items and Relationships Platform, you will: - Own the science direction for product identity: define how products are semantically understood, resolved, and committed as coherent identities, navigating the precision/recall and friction/selection trade-offs inherent in every decision - Build representations that decide both sameness and typed relatedness, and drive relationship inference from semantic identity rather than one pipeline per relationship type - Own structural relationship science across variations, multipacks and base products, newer versions, lines of products, and bundles, and discover new relationship types from the gap between defined policy and what identity reveals - Own calibration and evaluation science: define how models emit calibrated confidence, and build the LLM judge methodology and Catalog Identity Quality metric that measure identity and relationship quality at catalog scale - Partner with Principal Engineers on cost-aware modeling (cascades, distillation, routing) and close the loop from production signals and corrections back into training - Set science direction partnering with ACCS Applied Science teams, mentoring senior and principal-track scientists, and driving the documentation and publication of scientific innovation A day in the life On a typical day, you move between setting science direction and going deep on hard problems: reviewing a multimodal identity model or a relationship inference experiment with scientists, deciding how a borderline case (a multipack, a newer version, a near-identical variant) should be modeled, and pressure-testing whether a quality gain is real by examining the judge and its calibration. You also spend time zoomed out, shaping the longer-term science roadmap, aligning with Principal Engineers and partner teams on the identity and relationship signals they consume, and mentoring scientists as they take on the ambiguous, high-impact problems that define the platform’s future. About the team The Items and Relationships Platform advances state-of-the-art GenAI to deeply understand, uniquely identify, and meaningfully connect every product at Amazon scale. We build the canonical product identity, matching, and relationship systems that power Amazon’s universal catalog: eliminating duplicates, preventing abuse, keeping product families coherent, and integrating the world’s product data to power agentic shopping and trusted discovery for customers worldwide. We solve foundational challenges including schema-agnostic multimodal understanding, relationship inference, evaluation without ground truth, and efficient inference at reduced cost.
Senior Data Scientist, Amazon Leo
US, WA, Redmond
Amazon Leo is seeking a Senior Data Scientist to transform complex satellite, network, and customer terminal telemetry into actionable insights that drive system performance, reliability, and engineering decisions. In this role, you will work with large-scale datasets generated across satellite payloads, ground infrastructure, customer terminals, and end-to-end laboratory and on-orbit testing environments. You will develop analytical models, statistical methods, and automated data processing solutions to identify performance trends, detect anomalies, isolate issues, and support system validation and operational readiness. You will partner closely with systems, hardware, software, RF, and test engineering teams to establish meaningful performance metrics, correlate telemetry across multiple system components, and translate complex technical data into clear recommendations. Your work will directly influence engineering priorities, test strategies, system improvements, and launch readiness decisions. The ideal candidate combines strong statistical and data science expertise with practical experience analyzing complex engineering systems, processing high-volume time-series data, and developing scalable analytical solutions. Experience in satellite communications, wireless networks, or telecommunications is highly desirable. Export Control Requirement: Due to applicable export control laws and regulations, candidates must be a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum. Key job responsibilities - Develop and maintain scalable data processing and analytical solutions for satellite payload, ground network, customer terminal, and laboratory test telemetry. - Analyze large volumes of structured and unstructured data to identify system performance trends, anomalies, degradation, and potential failure conditions. - Correlate telemetry across satellite payloads, RF systems, networking infrastructure, customer terminals, and application-level measurements to identify relationships and isolate root causes. - Design statistical models, machine learning techniques, and predictive analytics to detect anomalies, characterize system behavior, and identify emerging reliability or performance issues. - Establish performance baselines, key performance indicators (KPIs), and statistical methodologies to evaluate system functionality, reliability, and performance across laboratory and on-orbit environments. - Analyze automated test results, test campaign data, and regression trends to identify software and hardware performance changes, recurring failures, and gaps in test coverage. - Develop dashboards, automated reports, and visualization tools that enable engineering teams and leadership to understand system health, performance, and readiness. - Apply statistical analysis and experimental design to compare software releases, hardware configurations, and system changes, quantifying their impact on end-to-end performance. - Translate complex telemetry and analytical findings into actionable engineering recommendations that influence test priorities, troubleshooting, product improvements, and operational decisions. - Collaborate across multiple engineering organizations to define data requirements, improve telemetry quality, and develop consistent analytical methodologies. - Lead the development of reusable analytical frameworks and scalable solutions that improve engineering efficiency, reduce manual investigation, and accelerate decision-making. - Mentor engineers and data scientists in statistical methods, telemetry analysis, data interpretation, and data-driven problem-solving.
Applied Scientist, PXT Central Science
US, WA, Seattle
The People eXperience Technology (PXT) Central Science (PXTCS)'s mission is to make PXT the most scientific, technologically proficient, and inclusive HR organization in the world. PXTCS does this by accelerating scientific rigor in business-led initiatives, working backwards from employee experience, and delivering science-driven products that improve the well-being of and value of work for Amazonian worldwide. We are seeking an Applied Scientist to build production machine learning systems that turn Amazon's employee listening signals into insight at scale. You will design, develop, and deploy ML solutions that transform large volumes of employee feedback — survey responses, free-text comments, and other listening channels — into signals that drive strategic decision-making across the organization. In this role, you will work on challenging problems spanning natural language processing, large language models, predictive modeling, and generative AI applications applied to employee experience. You will collaborate with cross-functional teams to translate ambiguous business challenges into rigorous technical solutions. Key job responsibilities - Apply and adapt state-of-the-art NLP and machine learning techniques to extract structured, reliable signals from employee listening data (surveys, free-text feedback, and related channels), using reasonable assumptions, data, and customer requirements. - Design, develop, and implement small-to-medium ML components on top of the employee listening system — such as theme and sentiment extraction, text classification, and aggregated signal generation — with input and guidance from senior scientists, taking ownership of the code in your components. - Write secure, stable, testable, maintainable, well-reviewed code (at the SDE I bar) to deliver solutions into production that benefit customers and the business. - Design features and signals responsibly, applying privacy-preserving practices appropriate to sensitive employee data. - Collaborate with cross-functional partners to understand business context and impact, and help mentor interns. - Communicate complex technical concepts to diverse audiences, from technical peers to senior leadership. About the team The People eXperience and Technology Central Science Team (PXTCS) uses economics, applied science, statistics, and machine learning to proactively identify mechanisms and process improvements which simultaneously improve Amazon and the lives, wellbeing, and the value of work to Amazonians. We build and apply science on top of Amazon's employee listening systems — turning what Amazonians tell us into products and insights that improve their experience. We are an interdisciplinary team that combines the talents of science and engineering to develop and deliver solutions that measurably achieve this goal.
Principal Applied Scientist, Frontier AI Assets , Frontier AI Assets
US, CA, Sunnyvale
The Frontier AI Assets (FAI) team is looking for a passionate, talented, and inventive Principal Applied Scientist with a strong deep learning and reinforcement learning background to build the training and evaluation assets that advance frontier Generative AI (GenAI) with Large Language Models (LLMs). AI models become more capable largely because of the learning environments, the RL gyms, in which they are trained. FAI builds those gyms and the platform that certifies them, so that model training teams across Amazon can rely on high-quality, graded assets. As a Principal on this team, you will set the technical direction for how these gyms and the tooling around them are built, evaluated, and scaled. Key job responsibilities As a Principal Applied Scientist with FAI, you will: •Set technical direction across data collection, sampling, and evaluation for RL gyms focused on agentic tool use across a range of real-world domains. •Design and lead experiments that produce reliable, efficient signal on what drives model improvement. •Build gym-building tooling and automated quality control that scale with demand. •Advance the science of how gyms are evaluated and validated before they're used in model training. •Act as a hands-on technical leader: produce exemplary, high-quality assets yourself, and audit and support more junior scientists and engineers. •Partner closely with domain experts and cross-functional teams, turning real-world problems into challenging, high-value tasks. •Lead scientific reviews, scrutinize experimental design and verification, probe assumptions, and align teams toward coherent strategies. Contribute to hiring, mentorship, and the development of other scientists. About the team The FAI team has the mission to push the envelope in GenAI with LLMs, in order to provide the best-possible experience for our customers. We build frontier training and evaluation assets, RL gyms and the certification platform behind them, that directly shape how Amazon's models are trained and improved. We value scientific depth, hands-on building, and turning research into measurable impact at scale.
Applied Scientist, Amazon Robotics
US, CA, Sunnyvale
Amazon Robotics is seeking an exceptional Applied Scientist to join our Foundation Models and Agentic AI team. This role presents an opportunity to shape the future of robotics through innovative applications of large vision-language-action (VLA) models, reasoning and task planning models, and reinforcement learning. At Amazon Robotics, we leverage advanced robotics, machine learning, and artificial intelligence to solve complex operational challenges at unprecedented scale. Our fleet of robots operates across hundreds of facilities worldwide, working in sophisticated coordination to fulfill our mission of customer excellence. We are pioneering the development of robotics foundation models that: - Enable unprecedented generalization across diverse tasks - Integrate multi-modal learning capabilities (visual, depth, force, tactile, linguistic) - Accelerate policy skill acquisition through imitation learning and reinforcement learning - Enhance robotic perception and environmental understanding - Streamline development processes through reusable capabilities The ideal candidate will contribute to research that bridges the gap between theoretical advancement and practical implementation in robotics. You will be part of a team that's revolutionizing how robots learn, adapt, and interact with their environment. Join us in building the next generation of intelligent robotics systems that will transform the future of automation and human-robot collaboration. Key job responsibilities As an Applied Scientist in the Applied AI and Robotics Foundations Model team, you will: - Model Development and Training: Designing and implementing the model architectures, training and fine tuning the foundation models using various datasets, and optimize the model performance through iterative experiments - Data Management: Process and prepare training data, including data governance, provenance tracking, data quality checks and creating reusable data pipelines. - Experimentation and Validation: Design and execute experiments to test model capabilities on the simulator and on the embodiment, validate performance across different scenarios, create a baseline and iteratively improve model performance. - Code Development: Write clean, maintainable, well commented and documented code, contribute to training infrastructure, create tools for model evaluation and testing, and implement necessary APIs - Research: Stay current with latest developments in foundation models and robotics, assist in literature reviews and research documentation, prepare technical reports and presentations, and contribute to research discussions and brainstorming sessions. - Collaboration: Work closely with senior scientists, engineers, and leaders across multiple teams, participate in knowledge sharing, support integration efforts with robotics hardware teams, and help document best practices and methodologies. A day in the life Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: - Medical, Dental, and Vision Coverage - Maternity and Parental Leave Options - Paid Time Off (PTO) - 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! About the team Are you inspired by invention? Is problem solving through teamwork in your DNA? Do you like the idea of seeing how your work impacts the bigger picture? Answer yes to any of these and you’ll fit right in here at Amazon Robotics. We are a smart, collaborative team of enthusiastic doers that work passionately to apply innovative advances in robotics and software to solve real-world challenges that will transform our customers’ experiences in ways we can’t even image yet. We invent new improvements every day. We are Amazon Robotics and we will give you the tools and support you need to invent with us in ways that are rewarding, fulfilling and fun.
Senior Research Scientist, RBS Tech
IN, KA, Bengaluru
RBS (Retail Business Services) Tech team works towards enhancing the customer experience (CX) and their trust in product data by providing technologies to find and fix Amazon CX defects at scale. Our platforms help in improving the CX in all phases of customer journey, including selection, discoverability & fulfilment, buying experience and post-buying experience (product quality and customer returns). The team also develops GenAI platforms for automation of Amazon Stores Operations. As a Sciences team in RBS Tech, we focus on foundational ML research and develop scalable state-of-the-art ML solutions to solve the problems covering customer experience (CX) and Selling partner experience (SPX). We work to solve problems related to multi-modal understanding (text and images), task automation through multi-modal LLM Agents, supervised and unsupervised techniques, multi-task learning, multi-label classification, aspect and topic extraction for Customer Anecdote Mining, image and text similarity and retrieval using NLP and Computer Vision for product groupings and identifying duplicate listings in product search results Key job responsibilities As a Senior Research Scientist, you will be responsible to design and deploy scalable GenAI, NLP and Computer Vision solutions that will impact the content visible to millions of customer and solve key customer experience issues. You will develop novel LLM, deep learning and statistical techniques for task automation, text processing, image processing, pattern recognition, and anomaly detection problems. You will define the research and experiments strategy with an iterative execution approach to develop AI/ML models and progressively improve the results over time. You will partner with business and engineering teams to identify and solve large and significantly complex problems that require scientific innovation. You will help the team leverage your expertise, by coaching and mentoring. You will contribute to the professional development of colleagues, improving their technical knowledge and the engineering practices. You will independently as well as guide team to file for patents and/or publish research work where opportunities arise. The RBS org deals with problems that are directly related to the selling partners and end customers and the ML team drives resolution to organization level problems. Therefore, the Senior Research Scientist role will impact the large product strategy, identifies new business opportunities and provides strategic direction which is very exciting.
Sr Applied Scientist, Amazon Shipping
IN, HR, Gurugram
Lead ML teams building large-scale forecasting and optimization systems that power Amazon’s global transportation network and directly impact customer experience and cost. As an Sr Applied Scientist, you will set scientific direction, mentor applied scientists, and partner with engineering and product leaders to deliver production-grade ML solutions at massive scale. Key job responsibilities 1. Lead and grow a high-performing team of Applied Scientists, providing technical guidance, mentorship, and career development. 2. Define and own the scientific vision and roadmap for ML solutions powering large-scale transportation planning and execution. 3. Guide model and system design across a range of techniques, including tree-based models, deep learning (LSTMs, transformers), LLMs, and reinforcement learning. 4. Ensure models are production-ready, scalable, and robust through close partnership with stakeholders. Partner with Product, Operations, and Engineering leaders to enable proactive decision-making and corrective actions. 5. Own end-to-end business metrics, directly influencing customer experience, cost optimization, and network reliability. 6. Help contribute to the broader ML community through publications, conference submissions, and internal knowledge sharing. A day in the life Your day includes reviewing model performance and business metrics, guiding technical design and experimentation, mentoring scientists, and driving roadmap execution. You’ll balance near-term delivery with long-term innovation while ensuring solutions are robust, interpretable, and scalable. Ultimately, your work helps improve delivery reliability, reduce costs, and enhance the customer experience at massive scale.
Applied Scientist, North America Sort Centers, Amazon Transportation Services, North America Sort Centers, Amazon Transportation Services
US, WA, Bellevue
Have you ever ordered a product on Amazon and when that box with the smile arrived you wondered how it got to you so fast? Have you wondered where it came from and how much it cost Amazon to deliver it to you? Have you also wondered what are different ways that the transportation assets can be used to delight the customer even more. If so, the Amazon transportation Services, North America Sort Center Science team is for you . We manage the delivery of tens of millions of products every week to Amazon’s customers, achieving on-time delivery in a cost-effective manner. We are looking for an enthusiastic, customer obsessed Applied Scientist with strong scientific thinking, good software and statistics experience, skills to help manage projects and operations, improve metrics, and develop scalable processes and tools. The primary role of an Applied Scientist within Amazon is to address business challenges through building a compelling case, and using data to influence change across the organization. This individual will be given responsibility on their first day to own those business challenges and the autonomy to think strategically and make data driven decisions. Decisions and tools made in this role will have significant impact to the customer experience, as it will have a major impact on how we operate the middle mile network. Ideal candidates will be a high potential, strategic and analytic graduate with a PhD in (Operations Research, Statistics, Engineering, and Supply Chain) ready for challenging opportunities in the core of our world class operations space. Great candidates have a history of operations research, machine learning , and the ability to use data and research to make changes. This role requires robust skills in research and implementation of scalable products and models . This individual will need to be able to work with a team, but also be comfortable making decisions independently, in what is often times an ambiguous environment. Key job responsibilities Key job responsibilities - Design and develop advanced mathematical optimization and machine learning solutions in the domains of network design and optimization. - Research, prototype, simulate, and experiment with these models using programming languages such as Java and Python; participate in the production level deployment. - Create, enhance, and maintain technical documentation and science designs. - Present to other Scientists, Product, and Software Engineering teams, as well as Stakeholders. - Lead project plans from a scientific perspective by managing product features, technical risks, milestones and launch plans.
Applied Scientist II, Prime Video - Ads Science
US, WA, Seattle
Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. Prime members can customize their viewing experience and find their favorite movies, series, documentaries, and live sports – including Amazon MGM Studios-produced series and movies; licensed fan favorites; and programming from Prime Video subscriptions such as Apple TV+, HBO Max, Peacock, Crunchyroll and MGM+. All customers, regardless of whether they have a Prime membership or not, can rent or buy titles via the Prime Video Store, and can enjoy even more content for free with ads. Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. As a Prime Video team member, you’ll have end-to-end ownership of the product, user experience, design, and technology required to deliver state-of-the-art experiences for our customers. You’ll get to work on projects that are fast-paced, challenging, and varied. You’ll also be able to experiment with new possibilities, take risks, and collaborate with remarkable people. We’ll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented technologists, you can decide where a career Prime Video Tech takes you! Key job responsibilities The science challenges include: - Heterogeneous customer responses. The same customer responds differently to ads depending on what they’re watching, how engaged they are, and their broader streaming context. We need to understand and predict this variation to make better advertising decisions in real time. - Signal sparsity in streaming. Unlike search or retail advertising, streaming offers no clicks, no conversions, and limited direct feedback. We must develop creative approaches to infer customer preferences, intent, and tolerance from indirect behavioral signals. - Personalization at scale. A one-size-fits-all ad experience leaves value on the table for both customers and the business. We build systems that adapt ad load, placement, and content to individual viewers across 100M+ customers and 100k+ titles. - Small effects, large variance. Ad interventions typically produce 0.1-2% shifts in engagement, effects easily overwhelmed by natural behavioral variance. Measuring, attributing, and optimizing these small signals requires rigorous experimental design and causal methodologies. - Competing objectives. Revenue, customer engagement, long-term retention, and advertiser value are in tension. We develop principled frameworks to navigate these tradeoffs and optimize for sustainable outcomes rather than any single metric. A day in the life - Contribute to the research and development of ML models that personalize advertising decisions for 100M+ customers across 100k+ titles, with production deployment in mind - Develop deep learning architectures (multi-task learning, embedding-based representations) for customer behavior prediction at scale - Design and analyze large-scale A/B experiments, applying causal inference techniques to measure and optimize the impact of ad strategies on customer engagement and monetization - Partner with engineering to ensure models meet production latency and scalability requirements - Collaborate with product managers to frame business problems as tractable ML problems and translate findings into product decisions - Shape the team’s scientific roadmap, identifying high-impact research directions About the team Prime Video Ads is pioneering the use of Generative AI, Deep Learning and Causal ML to empower the next generation of Ads experience in Prime Video. Our vision is to build the world’s most valuable and differentiated ad-supported streaming platform: one where every ad is relevant to the viewer, and every dollar spent by an advertiser delivers measurable results. We are seeking lead applied scientists who have demonstrated experience in advancing state of the art ML algorithms, particularly in Personalization, Relevance, Casual Inference and Gen-AI areas. Your role will be to deliver these innovations as production-ready systems at Amazon scale.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social