Social
Publication

Domain-compatible synthetic data generation for emergency vehicle detection

By Dipika Khullar, Negin Sokhandan, Ninad Kulkarni, Yash Shah, Suchitra Sathyanarayana
2023
Download Copy BibTeX
Share
Download
Copy BibTeX
Share
Recent advancements in generative models have led to significant improvements in the quality of generated images, making them virtually indistinguishable from real ones. However, using AI generated images for training robust computer vision models for real-world applications, especially object detection in road scene perception, is still a challenge. AI generated images usually lack the required diversity and scene complexity where specific objects appear with critically low frequency in the available real datasets.

An example of such applications is the detection of emergency vehicles like police cars, fire trucks, and ambulances in road scenes. These vehicles appear with drastically low frequencies in available datasets. Successfully generating synthetic images of road scenes that include these types of vehicles and using them in training downstream models would prove useful for autonomous driving vehicles, mitigating safety concerns on the road.

To address this, this paper proposes a new approach for synthetically generating diverse, complex, and domain-compatible images of emergency vehicles in road scenes by employing a diffusion-based generative model pretrained on a generic dataset. We investigate the impact of using generated synthetic images in the performance of downstream object detection models. Finally, we thoroughly discuss challenges of generating synthetic datasets with the proposed approach.
Research areas
Tags

Latest news

Work with us

See more jobs See more jobs
Data Scientist, Traffic Quality
IN, KA, Bengaluru
Amazon Ads is a multi-billion dollar global business that delivers advertising experiences across Amazon's owned-and-operated properties (including Prime Video, Twitch, Fire TV, and Amazon.com), third-party publisher networks, and emerging channels like generative AI-powered shopping experiences. As one of the fastest-growing segments of Amazon, we operate at unprecedented scale across desktop, mobile, connected TV, and emerging surfaces. Within Amazon Ads, Traffic Quality is a critical pillar of advertiser trust and marketplace integrity. Our mission is to build advanced capabilities that work at petabyte scale to detect sophisticated invalid traffic (IVT) which includes sophisticated non-human traffic, bot networks, and fraudulent engagement patterns across programmatic advertising. We are on a journey to establish Amazon Ads as an industry leader in traffic quality standards and transparency. Our research agenda focuses on staying ahead of adversarial actors through continuous innovation in detection methodologies, leveraging state-of-the-art techniques in deep learning and generative modeling, user behavior and multi-modal representation learning, anomaly detection, time-series analysis, and sparse labeling methods. We process billions of ad events daily, developing novel algorithms that balance precision and recall while operating under strict latency constraints. Our work directly protects hundreds of millions of dollars in advertiser spend annually while maintaining a seamless user experience. Key job responsibilities As a Data Scientist II in Traffic Quality, you will solve inherently hard problems in advertising fraud detection by applying advanced statistical techniques and machine learning. You'll work on systems that process billions of ad impressions and clicks per day, using Amazon's cloud services including EC2, S3, EMR, Sagemaker, and RedShift. - Define and frame new research problems in fraud detection where neither problem nor solution is well-defined. - Apply new machine learning approaches, models, and algorithms to detect sophisticated invalid traffic. - Apply domain knowledge to perform broad data analysis as a precursor to modeling and build business insights. - Work with unstructured and massive datasets to deliver results. - Produce research reports meeting top-tier external publication standards. - Mentor and develop junior scientists on the team. About the team Here are a few papers published by the team: 1/ [Scaling Generative Pre-training for User Ad Activity Sequences. AdKDD 2023.](https://assets.amazon.science/b7/42/03be071743d5a57cb1656e6caa34/scaling-generative-pre-training-for-user-ad-activity-sequences.pdf) 2/ [SLIDR: Real-time Robot Detection On Online Ads, IAAI 2023, Deployed Highly Innovative Applications of AI Track (AAAI 2023)](https://assets.amazon.science/75/2f/3b7106b143f38f7f4d2806388ace/real-time-detection-of-robotic-traffic-in-online-advertising.pdf) 3/ [Self-supervised Representation Learning Across Sequential and Tabular Features Using Transformers, NeurIPS 2022, First Table Representation Learning Workshop](https://openreview.net/forum?id=wIIJlmr1Dsk)
Applied Scientist, Fauna
US, CA, Sunnyvale
We are seeking an Applied Scientist to focus on Robot Navigation. In this role, you'll research and develop advanced navigation systems that enable robots to move reliably and safely through complex, dynamic environments. You'll work across a broad spectrum of navigation approaches—from classical methods to learning-based techniques and foundation models—to build robust solutions for autonomous robot navigation. Key job responsibilities - Develop and implement robust navigation systems that enable reliable autonomous operation in complex, dynamic indoor environments with static and dynamic obstacles - Build simulation-based and on-device evaluation frameworks with comprehensive benchmarks and metrics for systematic comparison of navigation methods - Conduct sim-to-real transfer experiments, analyzing performance gaps and developing techniques to ensure reliable real-world navigation performance - Collaborate with world model, manipulation, and other teams to ensure seamless integration of navigation capabilities into the full robot system - Stay current with the latest advances in robot navigation, spatial reasoning, and related fields, and apply relevant findings to improve system performance - Mentor fellow scientists and engineers while maintaining strong individual technical contributions About the team Fauna Robotics, an Amazon company, is building capable, safe, and genuinely delightful robots for everyday life. Our goal is simple: make robots people actually want to live and interact with in everyday human spaces. We believe that future won’t arrive until building for robotics becomes far more accessible. Today, too much effort is spent reinventing the fundamentals. We’re changing that by developing tightly integrated hardware and software systems that make it faster, safer, and more intuitive to create real-world robotic products.
Applied Scientist, Pricing Science
US, WA, Seattle
Pricing is one of the most consequential decisions Amazon makes — and the science behind it needs to be causally rigorous, not just predictive. The P2 Optimization Science (P2OS) team builds the machine learning systems that power Amazon's pricing decisions at scale: demand lift models, customer lifetime value frameworks, and the experimentation infrastructure that validates whether our pricing changes actually work. We're hiring an Applied Scientist to own causal inference at the intersection of ML and pricing experimentation. This role exists because our team has identified a real gap: the methodological bridge between econometric analysis (owned by our economists) and production-scale ML pipelines (owned by our engineers) needs a practitioner who lives in both worlds. You'll build CATE estimation models, design analysis workflows for pricing weblabs, and develop the reusable causal ML infrastructure that the broader team — including non-ML scientists — can rely on. This is not a research role. The bias here is toward shipping production-quality causal pipelines with real downstream business impact. You'll measure success by what changes in LTV estimates, what pricing errors your models help avoid, and whether the economists on your team can actually use what you build. If you're a scientist who wants to work on hard causal identification problems in a high-stakes production environment — and who finds satisfaction in making rigorous methods accessible to a broader team — this role is for you. Key job responsibilities * Build causal ML pipelines for pricing — Design, train, evaluate, and deploy end-to-end causal estimation models for pricing use cases. * Own the science on heterogeneous treatment effects — Be the team SME on causal ML methodology: identification strategies, model selection, evaluation standards, and the tradeoffs between econometric and ML approaches to causal estimation. * Support pricing experiment analysis — Contribute causal analysis methodology to pricing weblab and A/B test post-analysis; build reusable tooling that economists can use without requiring ML expertise * Connect model outputs to business outcomes — Define, before writing code, what business metric each model moves; deliver model evaluation reports framed around pricing errors avoided and LTV estimate changes. * Evaluate and adopt novel techniques — Assess applicability of emerging causal inference methods (synthetic DiD, generalized random forests, causal representation learning) to Amazon's pricing context; write internal methodology proposals for adoption * Write internal documentation and methodology papers — Produce at least one internal write-up per half that connects a causal ML technique to a concrete pricing use case; make pipelines extensible and well-documented so other scientists can build on them. * Collaborate across disciplines — Partner closely with the Sr. Economist on identification strategy and causal assumptions; work with SDE and DE partners on production deployment; align with PMs on experiment design requirements A day in the life As an Applied Scientist on the P2OS team, your work directly shapes the prices customers see on hundreds of millions of Amazon products. In a given workweek, you might: * Investigate an optimization anomaly in simulation and trace it back to a model input gap or an unmodeled market dynamic * Design an offline evaluation framework to benchmark competing optimization approaches before committing to online testing * Collaborate with Sr. Economists on the identification strategy for the model you're building for a pricing lab * Present a science proposal for incorporating a new competitiveness or inventory signal into an optimization system * Work cross-team with the experimentation platform team on randomization design. * Develop and write up a novel scientific finding — preparing a paper or technical report for submission to a top-tier venue such as KDD, NeurIPS, or the ACM Conference on Economics and Computation
Applied Scientist II, Text-to-Speech on Device team
PL, Gdansk
Have you ever wondered how we give voice to devices — even when they're offline? The Text-to-Speech on Device team at Amazon builds AI-powered voice models that run locally on hardware with limited resources, serving customers across Alexa, automotive, and accessibility experiences for visually impaired users. We sit at the intersection of speech generation, generative AI, and on-device machine learning, and we're looking for a curious, collaborative Applied Scientist to help us push what's possible. In this role, you will research and develop production-ready speech generation models optimized for constrained environments. You will work across the full model lifecycle — from early experimentation and prototyping through to integration on real devices. If you're excited about solving hard scientific problems that directly improve how millions of people interact with technology, we'd love to hear from you. Key job responsibilities - Design and develop end-to-end machine learning models for on-device speech generation, from early research and experimentation through production-ready deployment. - Research and apply advanced techniques in generative AI, model compression, and knowledge distillation to deliver high-quality voice models within tight hardware constraints. - Propose and validate novel scientific approaches by authoring detailed technical specifications and contributing to peer-reviewed publications when appropriate. - Evaluate model performance rigorously, identify improvement opportunities, and iterate on training and inference pipelines to optimize quality and efficiency. - Collaborate with science and engineering teams across cloud and device platforms to bring speech generation capabilities from research prototypes to integrated product experiences. About the team The Text-to-Speech on Device team builds low-footprint AI models for speech generation that run locally on devices such as Android and FireOS platforms. Our models require significantly less computation than cloud-hosted alternatives, enabling offline voice experiences for Alexa, automotive partners, and accessibility solutions. We work closely with device engineering teams and cloud-based speech science teams to deliver the best possible experience for our customers. Our focus in the coming years is expanding the range of voices and languages we support while continuing to improve naturalness and efficiency on constrained hardware.
Applied Scientist, Navigation
US, CA, Sunnyvale
Amazon is on a mission to redefine the future of automation — and we're looking for exceptional talent to help lead the way. We are building the next generation of advanced robotic systems that seamlessly blend cutting-edge AI, sophisticated control systems, and novel mechanical design to create adaptable, intelligent automation solutions capable of operating safely alongside humans in dynamic, real-world environments. At Amazon, we leverage the power of machine learning, artificial intelligence, and advanced robotics to solve some of the most complex operational challenges at a scale unlike anywhere else in the world. Our fleet of robots spans hundreds of facilities globally, working in sophisticated coordination to deliver on our promise of customer excellence — and we're just getting started. As a Sr. Scientist in Robot Navigation, you will be at the forefront of this transformation — architecting and delivering navigation systems that are intelligent, safe, and scalable. You will bring deep expertise in learning-based planning and control, a strong understanding of foundation models and their application to embodied agents, and as well as have in-depth understanding of control-theoretic approaches such as model predictive control (MPC)-based trajectory planning. You will develop navigation solutions that seamlessly blend data-driven intelligence with principled control-theoretic guarantees. Our vision is bold: to build navigation systems that allow robots to move fluidly and safely through dynamic environments — understanding context, anticipating change, and adapting in real time. You will lead research that bridges the gap between cutting-edge academic advances and production grade deployment, collaborating with world-class teams pushing the boundaries of robotic autonomy, manipulation, and human-robot interaction. Join us in building the next generation of intelligent navigation systems that will define the future of autonomous robotics at scale. Key job responsibilities - Design, develop, and deploy perception algorithms for robotics systems, including object detection, segmentation, tracking, depth estimation, and scene understanding - Lead research initiatives in computer vision, sensor fusion and 3D perception - Collaborate with cross-functional teams including robotics engineers, software engineers, and product managers to define and deliver perception capabilities - Drive end-to-end ownership of ML models — from data collection and labeling strategy to training, evaluation, and deployment - Mentor junior scientists and engineers; contribute to a culture of technical excellence - Define and track key metrics to measure perception system performance in real-world environments - Publish research findings in top-tier venues (CVPR, ICCV, ECCV, ICRA, NeurIPS, etc.) and contribute to patents A day in the life - Train ML models for deployment in simulation and real-world robots, identify and document their limitations post-deployment - Drive technical discussions within your team and with key stakeholders to develop innovative solutions to address identified limitations - Actively contribute to brainstorming sessions on adjacent topics, bringing fresh perspectives that help peers grow and succeed — and in doing so, build lasting trust across the team - Mentor team members while maintaining significant hands-on contribution to technical solutions About the team Our team is a group is a diverse group of scientists and engineers passionate about building intelligent machines. We value curiosity, rigor, and a bias for action. We believe in learning from failure and iterating quickly toward solutions that matter.
Applied Scientist II, Translation Services
IN, TS, Hyderabad
Have you ever wondered how Amazon launches and maintains a consistent customer experience across hundreds of countries and languages it serves its customers? Are you passionate about data and mathematics, and hope to impact the experience of millions of customers? Are you obsessed with designing simple algorithmic solutions to very challenging problems? If so, we look forward to hearing from you! At Amazon, we strive to be Earth's most customer-centric company, where both internal and external customers can find and discover anything they want in their own language of preference. Our Translations Services (TS) team plays a pivotal role in expanding the reach of our marketplace worldwide and enables thousands of developers and other stakeholders (Product Managers, Program Managers, Linguists) in developing locale specific solutions. Amazon Translations Services (TS) is seeking an Applied Scientist to be based in our Hyderabad office. As a key member of the Science and Engineering team of TS, this person will be responsible for designing algorithmic solutions based on data and mathematics for translating billions of words annually across 130+ and expanding set of locales. The successful applicant will ensure that there is minimal human touch involved in any language translation and accurate translated text is available to our worldwide customers in a streamlined and optimized manner. With access to vast amounts of data, technology, and a diverse community of talented individuals, you will have the opportunity to make a meaningful impact on the way customers and stakeholders engage with Amazon and our platform worldwide. Together, we will drive innovation, solve complex problems, and shape the future of e-commerce. Key job responsibilities * Apply your expertise in LLM models to design, develop, and implement scalable machine learning solutions that address complex language translation-related challenges in the eCommerce space. * Collaborate with cross-functional teams, including software engineers, data scientists, and product managers, to define project requirements, establish success metrics, and deliver high-quality solutions. * Conduct thorough data analysis to gain insights, identify patterns, and drive actionable recommendations that enhance seller performance and customer experiences across various international marketplaces. * Continuously explore and evaluate state-of-the-art modeling techniques and methodologies to improve the accuracy and efficiency of language translation-related systems. * Communicate complex technical concepts effectively to both technical and non-technical stakeholders, providing clear explanations and guidance on proposed solutions and their potential impact. About the team We are a start-up mindset team. As the long-term technical strategy is still taking shape, there is a lot of opportunity for this fresh Science team to innovate by leveraging Gen AI technoligies to build scalable solutions from scratch. Our Vision: Language will not stand in the way of anyone on earth using Amazon products and services. Our Mission: We are the enablers and guardians of translation for Amazon's customers. We do this by offering hands-off-the-wheel service to all Amazon teams, optimizing translation quality and speed at the lowest cost possible.
Applied Scientist II, CMT
IN, KA, Bengaluru
As a member of the CMT team, you'll play a key role in the evolution of our Competitive Monitoring systems to solve significantly complex and interesting technical challenges in machine learning, large language models in production, and recommender systems to name a few. The team's work directly impacts customer experience at a worldwide scale. Key job responsibilities Thought leader on the team and help set team directions Research multiple problem domains, suggest various approaches to try and be as hands-on as needed while providing more junior scientists with critical mentorship Collaborate with engineers to come up with the right LLD and HLD to solve key business problems Strong emphasis on communication via writing, internal and external talks, and being able to align with multiple stakeholders A day in the life As an Applied scientist II, a typical day will involve aligning with key product, engineering and business stakeholders ; advising junior scientists on the work they are doing ; reading current research papers and staying up-to-date on AI research ; diving deep as needed to improve CMT models and addressing stakeholders from the science perspective ; writing python code
Applied Scientist, CMT
IN, KA, Bengaluru
As a member of the CMT team, you'll play a key role in the evolution of our Competitive Monitoring systems to solve significantly complex and interesting technical challenges in machine learning, large language models in production, and recommender systems to name a few. The team's work directly impacts customer experience at a worldwide scale. Key job responsibilities 1. Research the problem domain and come up with various approaches to solve the problem. 2. Be willing to experiment quickly and fail fast. 3. Collaborate with engineers to come up with the right end to end solution to the business problems. 4. Ideate on future roadmap for science in CMT 5. Be willing to roll up your sleeves and learn core topics outside applied science, for example ML engineering A day in the life A typical day might involve (a) working on ideas for improving models around product similarity or price recommendations, (b) working closely with other scientists and our ML engineers to ensure that the best models are in production, (c) writing good maintainable code that can be reused and reproduced, (d) sharing your work across CMT and beyond via technical writings and presentations
Senior Data Scientist, Customer Experience and Business Trends
US, WA, Seattle
Amazon's Customer Experience and Business Trends (CXBT) organization is hiring a Senior Data Scientist for its Benchmarking, Economics, Analytics and Measurement (BEAM) team. BEAM's mission is to improve customer experience across every Amazon business by turning fragmented internal and external data into decision-grade intelligence for senior leaders. This role anchors our Topline View of Retail initiative: an always-on competitive intelligence capability that fuses third-party transaction panels with Amazon's internal signals to answer how Amazon is winning or losing share of wallet, across segments, categories, and time. You will not just analyze metrics — you will decide which metrics should exist, then build the statistical and machine learning models that generate, forecast, and explain them. Given a new business question and a pile of raw panel and internal data, you are the person who defines what "competitiveness" means, translates it into a defensible, reproducible metric and model, and defends the methodology to skeptical senior stakeholders. We are looking for someone with exceptional business judgment and metric intuition, paired with deep hands-on modeling skills: a strong point of view on what to measure, and the technical ability to build the models that measure it at scale. You should be equally comfortable pressure-testing a model's assumptions and explaining its "so what" to a VP in two sentences. Key job responsibilities - Own the definition, methodology, and evolution of BEAM's competitiveness metrics (e.g., share-of-wallet, segment-level penetration, price/selection/fulfillment competitiveness), including the trade-offs behind each definition. - Design, build, and validate statistical and machine learning models — forecasting, causal inference, segmentation, and anomaly detection — that power those metrics and surface competitive shifts. - Pull together disparate third-party transaction-panel data and internal Amazon signals into coherent, reproducible modeling pipelines that leadership can trust for recurring reporting. - Translate ambiguous business questions ("are we losing ground in grocery?") into the right metric, the right model, the right cut of data, and a clear, defensible answer. - Productionize models and analyses so they run reliably as recurring mechanisms, partnering with data engineers to scale and automate. - Produce monthly flash reports and deep-dive narratives that turn models and metrics into decisions for CXBT and business-line leadership. - Set the standard for analytical and modeling rigor on the team — methodology reviews, documentation, and guardrails against misleading cuts. - Mentor and technically uplift other scientists and BIEs on the team — reviewing modeling approaches and raising the bar on rigor (e.g., partnering with our Data Scientist on the MatchIQ model to strengthen methodology and validation). About the team CXBT is a group of diverse functions dedicated to understanding, improving, and influencing customer experience globally, across all of Amazon. We are builders who develop products, services, and data-driven approaches that shape offerings for nearly every Amazon business and customer type — consumers, developers, sellers, brands, employees, investors, streamers, and gamers. We work backwards from customer needs, combine technical and non-technical methods, and track industry and business trends. Our roles span Product Managers, Data Scientists, Economists, Business Intelligence Engineers, Data Engine, Applied Scentist, and Product Manager.
Applied Scientist II, Reinforcement Learning
US, MA, N.reading
Amazon is seeking exceptional talent to help develop the next generation of advanced robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine cutting-edge AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at an unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic dexterous manipulation, locomotion, and human-robot interaction. This role presents an opportunity to shape the future of robotics through innovative applications of deep learning and large language models. At Amazon we leverage advanced robotics, machine learning, and artificial intelligence to solve complex operational challenges at an unprecedented scale. Our fleet of robots operates across hundreds of facilities worldwide, working in sophisticated coordination to fulfill our mission of customer excellence. The ideal candidate will contribute to research that bridges the gap between theoretical advancement and practical implementation in robotics. You will be part of a team that's revolutionizing how robots learn, adapt, and interact with their environment. Join us in building the next generation of intelligent robotics systems that will transform the future of automation and human-robot collaboration. Key job responsibilities - Design and implement whole body control methods for balance, locomotion, and dexterous manipulation - Utilize state-of-the-art in methods in learned and model-based control - Create robust and safe behaviors for different terrains and tasks - Implement real-time controllers with stability guarantees - Collaborate effectively with multi-disciplinary teams to co-design hardware and algorithms for loco-manipulation - Mentor junior engineer and scientists
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social