Data augmentation is a necessity to enhance data efficiency in deep learning. For vision-language pre-training, data is only augmented either for images or for text in previous works. In this paper, we present MixGen: a joint data augmentation for vision-language representation learning to further improve data efficiency. It generates new image-text pairs with semantic relationships preserved by interpolating images and concatenating text. It's simple, and can be plug-and-played into existing pipelines. We evaluate MixGen on four architectures, including CLIP, ViLT, ALBEF and TCL, across five downstream vision-language tasks to show its versatility and effectiveness. For example, adding MixGen in ALBEF pre-training leads to absolute performance improvements on downstream tasks: image-text retrieval (+6.2% on COCO fine-tuned and +5.3% on Flicker30K zero-shot), visual grounding (+0.9% on RefCOCO+), visual reasoning (+0.9% on NLVR2), visual question answering (+0.3% on VQA2.0), and visual entailment (+0.4% on SNLI-VE).

Research areas

Related content

See more See more
US, NY, New York
We are seeking a Human-Robot Interaction (HRI) Applied Scientist to develop cutting-edge interactions that make robots feel alive, personal, and fun. In this role, you will focus on verbal and non-verbal conversational systems, social dynamics, memory, and long-term relationship formation between robots, their environments, and the people they interact with. Your contributions will be essential in advancing robotics by enabling expressive, socially intelligent, and trustworthy interactions between robots and humans. Key job responsibilities - Develop interactive systems that leverage large language models, multimodal inputs and outputs, reinforcement learning from human feedback, or other advanced techniques to achieve fluid, engaging, and socially appropriate robot behavior - Design and implement intelligent conversational systems that handle turn-taking, grounding, interruption, and incorporates context drawn from a robot's physical environment and shared history with a user - Integrate perceptual sensor streams including gaze, facial expression, gesture, posture, and more to understand social context and produce coherent, lifelike interactions. - Develop memory and personalization systems that allow robots to form lasting relationships with individual users, learn their environments, and adapt their behavior over weeks and months - Stay updated on advancements in HRI, NLP, multimodal AI, and cognitive and social science to apply cutting-edge techniques to robot interaction challenges - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers - Bridge research initiatives with practical engineering implementation
US, CA, Sunnyvale
Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. Prime members can customize their viewing experience and find their favorite movies, series, documentaries, and live sports – including Amazon MGM Studios-produced series and movies; licensed fan favorites; and programming from Prime Video add-on subscriptions such as Apple TV+, Max, Crunchyroll and MGM+. All customers, regardless of whether they have a Prime membership or not, can rent or buy titles via the Prime Video Store, and can enjoy even more content for free with ads. Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. As a Prime Video technologist, you’ll have end-to-end ownership of the product, user experience, design, and technology required to deliver state-of-the-art experiences for our customers. You’ll get to work on projects that are fast-paced, challenging, and varied. You’ll also be able to experiment with new possibilities, take risks, and collaborate with remarkable people. We’ll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented technologists, you can decide where a career Prime Video Tech takes you! Key job responsibilities Develop foundation models for content understanding using state-of-the-art deep learning and multimodal learning techniques to analyze video and text Build time sequence foundation models to understand and predict customer behavior patterns and viewing trajectories Work closely with engineers and product managers to design, implement and launch solutions end-to-end across various Prime Video experiences Design and conduct offline and online (A/B) experiments to evaluate proposed solutions based on in-depth data analyses Effectively communicate technical and non-technical ideas with teammates and stakeholders Stay up-to-date with advancements and the latest modeling techniques in foundation models, multimodal learning, and time series analysis Publish your research findings in top conferences and journals A day in the life We're using advanced approaches such as foundation models to connect information about our videos and customers from a variety of information sources, acquiring and processing data sets on a scale that only a few companies in the world can match. This will enable us to recommend titles effectively, even when we don't have a large behavioral signal (to tackle the cold-start title problem). It will also allow us to find our customer's niche interests, helping them discover groups of titles that they didn't even know existed. We are looking for creative & customer obsessed machine learning scientists who can apply the latest research, state of the art algorithms and ML to build highly scalable page personalization solutions. You'll be a research leader in the space and a hands-on ML practitioner, guiding and collaborating with talented teams of engineers and scientists and senior leaders in the Prime Video organization. You will also have the opportunity to publish your research at internal and external conferences. About the team Prime Video Recommendation Science team owns science solution to power recommendation and personalization experience on various Prime Video surfaces and devices. We work closely with the engineering teams to launch our solutions in production.
US, CA, Sunnyvale
Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. Prime members can customize their viewing experience and find their favorite movies, series, documentaries, and live sports – including Amazon MGM Studios-produced series and movies; licensed fan favorites; and programming from Prime Video add-on subscriptions such as Apple TV+, Max, Crunchyroll and MGM+. All customers, regardless of whether they have a Prime membership or not, can rent or buy titles via the Prime Video Store, and can enjoy even more content for free with ads. Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. As a Prime Video technologist, you’ll have end-to-end ownership of the product, user experience, design, and technology required to deliver state-of-the-art experiences for our customers. You’ll get to work on projects that are fast-paced, challenging, and varied. You’ll also be able to experiment with new possibilities, take risks, and collaborate with remarkable people. We’ll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented technologists, you can decide where a career Prime Video Tech takes you! We are looking for a self-motivated, passionate and resourceful Applied Scientist to bring diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. You will spend your time as a hands-on machine learning practitioner and a research leader. You will play a key role on the team, building and guiding machine learning models from the ground up. At the end of the day, you will have the reward of seeing your contributions benefit millions of Amazon.com customers worldwide. Key job responsibilities Develop foundation models for content understanding using state-of-the-art deep learning and multimodal learning techniques to analyze video, audio, and text. Build time sequence foundation models to understand and predict customer behavior patterns and viewing trajectories. Work closely with engineers and product managers to design, implement and launch solutions end-to-end across various Prime Video experiences. Design and conduct offline and online (A/B) experiments to evaluate proposed solutions based on in-depth data analyses. Effectively communicate technical and non-technical ideas with teammates and stakeholders. Stay up-to-date with advancements and the latest modeling techniques in foundation models, multimodal learning, and time series analysis. Publish your research findings in top conferences and journals. About the team Prime Video Recommendation Science team owns science solution to power recommendation and personalization experience on various Prime Video surfaces and devices. We work closely with the engineering teams to launch our solutions in production.
US, NY, New York
We are seeking an Research Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers
US, WA, Seattle
Innovators wanted! Are you an entrepreneur? A builder? A dreamer? This role is part of an Amazon Special Projects team that takes the company’s Think Big leadership principle to the nextlevel. We focus on creating entirely new products and services with a goal of positively impacting the lives of our customers. No industries or subject areas are out of bounds. If you’re interested in innovating at scale to address big challenges in the world, this is the team for you. As a Research Scientist, you will work with a unique and gifted team developing exciting products for consumers and collaborate with cross-functional teams. Our team rewards intellectual curiosity while maintaining a laser-focus in bringing products to market. At the intersession of both academic and applied research in this product area, you have the opportunity to work together with some of the most talented scientists, engineers, and product managers. Here at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have thirteen employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We are constantly learning through programs that are local, regional, and global. Amazon’s culture of inclusion is reinforced within our 16 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Our team highly values work-life balance, mentorship and career growth. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We care about your career growth and strive to assign projects and offer training that will challenge you to become your best.
US, WA, Seattle
The Data Intelligence team is a new function within Amazon Customer Service (CS). We own the end-to-end process of defining, building, implementing, and monitoring a comprehensive data strategy. We also develop and apply Generative Artificial Intelligence (GenAI), Machine Learning (ML), Ontology, and Natural Language Processing (NLP) to enhance customer service associate and customer experiences. As an Applied Scientist, you'll own the definition and implementation of customer-focused, AI-driven innovation in Amazon Customer Service globally, leveraging GenAI, ML, and/or NLP to transform complex business requirements and customer needs into innovative technology solutions. Your expertise will be key in shaping data-driven strategies and addressing complex data challenges. With your expertise in AI, text analysis, embeddings, language modeling, and generation, you'll design and develop scalable AI-powered technology solutions, prioritize initiatives, drive data-driven insights, and deliver business impact. This position will advance applied science best practices, leverage data and AI to drive customer experience improvements, and set new global standards for customer experience. This role requires you to work with a cross-functional team, including scientists, engineers, and product managers, to develop scalable and maintainable AI solutions for both structured and unstructured data. The ideal candidate has strong technical skills in AI techniques (e.g., automated reasoning, reasoning, planning, knowledge representation), excellent written documentation skills, and experience with big data technologies. Success in this role requires combining deep business knowledge with hands-on technical skills to solve customer problems and address complex technical challenges. Key job responsibilities - Develop innovative solutions to complex problems (e.g., Automated Reasoning for Trusted AI-Enabled Customer Service). - Apply technical expertise to implement novel algorithms and modeling solutions, in collaboration with other scientists and engineers. - Analyze data and define metrics to identify actionable insights and measure improvements in customer experience. - Communicate results and insights to both technical and non-technical audiences through written reports, presentations, and internal/external publications. - Collaborate with product management and engineering teams to integrate and optimize models in production systems. A day in the life A typical day as an Applied Scientist in the Data Intelligence team involves combining business expertise with hands-on problem-solving in ML and AI. The role encompasses tackling complex data initiatives, ensuring alignment with customer needs and business objectives, and translating business requirements into practical AI-driven solutions. Working collaboratively with cross-functional teams, this position involves designing and enhancing AI models, focusing on efficiency, precision, and scalability. Daily activities include ensuring data quality, monitoring model performance, and generating actionable insights from vast amounts of information. Each day presents opportunities to resolve complex technical challenges, advance important AI projects, and conceive innovative ways to leverage data in transforming the customer experience. About the team The Data Intelligence team is a new function within Amazon Customer Service. We develop and apply Generative Artificial Intelligence (GenAI), Machine Learning (ML), and Natural Language Processing (NLP) techniques to enhance customer service associate and customer experiences.
US, MA, N.reading
Amazon is seeking exceptional talent to help develop the next generation of advanced robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine cutting-edge AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at an unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic dexterous manipulation, locomotion, and human-robot interaction. This role presents an opportunity to shape the future of robotics through innovative applications of deep learning and large language models. At Amazonwe leverage advanced robotics, machine learning, and artificial intelligence to solve complex operational challenges at an unprecedented scale. Our fleet of robots operates across hundreds of facilities worldwide, working in sophisticated coordination to fulfill our mission of customer excellence. The ideal candidate will contribute to research that bridges the gap between theoretical advancement and practical implementation in robotics. You will be part of a team that's revolutionizing how robots learn, adapt, and interact with their environment. Join us in building the next generation of intelligent robotics systems that will transform the future of automation and human-robot collaboration. Key job responsibilities - Collaborate with simulation and robotics experts to translate physical modeling needs into robust, scalable, and maintainable simulation solutions. - Design and implement high-performance simulation modeling and tools for rigid and deformable body simulation. - Identify and optimize performance bottlenecks in simulation pipelines to support real-time and batch simulation workflows. - Help build validation and unit testing pipelines to ensure correctness and physical fidelity of simulation results. - Identify potential sources of sim-to-real gaps and propose modeling and numerical approximations to reduce them. - Stay current with the latest advances in numerical methods, parallel computing, and GPU architectures, and incorporate them into our tools.
US, MA, North Reading
robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine cutting-edge AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic manipulation, locomotion, and human-robot interaction. This role presents an opportunity to shape the future of robotics through innovative applications of deep learning and large language models. At Amazon Industrial Robotics we leverage advanced robotics, machine learning, and artificial intelligence to solve complex operational challenges at unprecedented scale. Our fleet of robots operates across hundreds of facilities worldwide, working in sophisticated coordination to fulfill our mission of customer excellence. We are pioneering the development of robotics foundation models that: Enable unprecedented generalization across diverse tasks Enable unprecedented robustness and reliability, industry-ready Integrate multi-modal learning capabilities (visual, tactile, linguistic) Accelerate skill acquisition through demonstration learning Enhance robotic perception and environmental understanding Streamline development processes through reusable capabilities The ideal candidate will contribute to research that bridges the gap between theoretical advancement and practical implementation in robotics. You will be part of a team that's revolutionizing how robots learn, adapt, and interact with their environment. Join us in building the next generation of intelligent robotics systems that will transform the future of automation and human-robot collaboration. As an Applied Science Manager in the Foundation Model team, you will build and lead a team that develops and improves machine learning systems that help robots perceive, reason, and act in real-world environments. You will set the technical direction for leveraging state-of-the-art models (open source and internal research), evaluating them on representative tasks, and adapting/optimizing them to meet robustness, safety, and performance needs. You will drive the capability roadmap and the evaluation strategy that defines “what the robot brain can do,” and you will sponsor targeted innovation when gaps remain. You’ll collaborate closely with research, controls, hardware, and product teams, and ensure the team’s outputs can be further customized and deployed by downstream teams on specific robot embodiments.
US, WA, Seattle
Are you ready to revolutionize the future of aerial delivery? Join our pioneering team working on autonomous drone technology that will transform how Amazon delivers to customers. As a Manager III, Applied Science for our Perception team, you'll lead scientists and engineers building world-class machine learning models and computer vision and new frontier model based systems that enable drones to safely navigate complex environments. In this high-impact role, you'll guide the development of perception systems combining radar, machine learning, and distributed computing to solve unprecedented challenges in autonomous flight. Your work will directly shape the future of Amazon's drone delivery service, bringing innovative solutions to customers while pushing the boundaries of what's possible in autonomous systems technology. Key job responsibilities Lead and mentor a team of scientists and software engineers developing advanced perception systems that integrate computer vision, machine learning algorithms, and detect-and-avoid capabilities for autonomous drones Define and drive the technical vision and roadmap for perception technology, making strategic architectural decisions that balance current delivery needs with long-term innovation goals Collaborate with cross-functional teams including hardware engineers, autonomy specialists, and product stakeholders to deliver scalable, distributed systems that meet rigorous safety and performance requirements Establish audit mechanisms and metrics to track system performance, model accuracy, and team progress against goals, using data-driven insights to continuously improve customer experience Build and grow an inclusive, high-performing team through effective hiring, mentoring, and career development while fostering a culture of innovation and technical excellence About the team The Perception team is at the heart of Amazon's autonomous drone delivery initiative, developing the advanced computer vision and machine learning systems that enable safe, reliable flight operations. Our team tackles fascinating challenges at the intersection of radar technology, distributed computing, and real-time detection algorithms, creating solutions that have never been built before. We work in an innovative, collaborative environment where every scientists and engineer's contribution directly impacts the future of delivery technology. As we scale Amazon's drone delivery to new locations, you'll help shape both the technical direction and team culture that will define this transformative service for years to come.
US, MA, North Reading
How should a robot dig a single item out of a cluttered bin, feel when it has made contact, and adjust in real time without crushing what it touches? Single- and dual-arm contact-rich manipulation at production scale is still being solved. We are looking for a Principal Applied Scientist to define the control architectures that our next-generation grasping systems will be built on. The Manipulation Robotics team develops robotic workcells to pick, grasp, and move millions of items and packages across Amazon's fulfillment network. Reaching into clutter and grasping deformable and diverse items depends on solving contact: force-aware control, compliant behaviors, and the layering of classical controllers and learned policies. You will set the technical direction for how the team approaches contact-rich manipulation, working with leaders across the organization to establish this foundational capability. Key job responsibilities - Own the technical direction and contact-control strategy for contact-rich manipulation, including the boundary between low-level control and learned policy, balancing tradeoffs among speed, performance, quality, cost, complexity, and adaptability. - Identify the open scientific problems that gate contact-rich manipulation at production scale, and invent the methods that solve them. - Architect contact control built on force and tactile sensing, covering compliant-contact behaviors, hybrid position and force control, and the safe interaction envelope. Stay hands-on and personally write critical-path code. - Partner with scientists, hardware designers, and systems engineers to treat the platform, arm, and end-effector as one coherent hierarchical system, including co-designing compliant end-effectors as part of the control strategy. - Create mechanisms to learn from fielded production, reasoning rigorously about failure modes and improving recovery from unexpected contact. - Drive how simulation, analytical models, demonstrations, and real-robot data are used together to turn lab results into scaled warehouse performance. - Establish the reference implementations, evaluation standards, and design review practices that let others build on the architecture and scale your impact without your continuous involvement. - Develop other scientists and engineers through mentorship, technical review, and a leading role in hiring. A day in the life Amazon offers a full range of benefits that support you and eligible family members, including domestic partners. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! About the team We are a small, high-ownership team building robotic workcells, from early-stage R&D through products fielded 24/7 in Amazon fulfillment centers. We work at the hard edge of grasping: contact-rich tasks, deformable and variable items, and mechanisms that must survive millions of cycles at fleet scale. Our people own problems end to end, and scientists work shoulder to shoulder with mechanical, electrical, software, and controls engineers in a fast-moving, highly empowered environment where research ideas turn into shipped systems.