Feedback
Follow Us
Conversational AI

Introducing the Amazon Trusted AI Challenge

University students will compete for cash prizes in a competition to securely advance LLMs that code.

July 08, 2024
Share

Today, Amazon is announcing the Amazon Trusted AI Challenge, a global university competition to drive secure innovation in generative AI technology. This year’s challenge focuses on responsible AI and specifically on large language model (LLM) coding security.

“We are focusing on advancing the capabilities of coding LLMs, exploring new techniques to automatically identify possible vulnerabilities and effectively secure these models,” said Rohit Prasad, senior vice president and head scientist, Amazon AGI. “The goal of the Amazon Trusted AI Challenge is to see how students' innovations can help forge a future where generative AI is consistently developed in a way that maintains trust, while highlighting effective methods for safeguarding LLMs against misuse to enhance their security.”

University students will compete in a tournament-style challenge as either model developer teams or red teams to enhance the AI user experience, prevent misuse, and enable users to build more secure code. Model developer teams will build security features into code-generating models, while red teams will develop automated techniques to test these models. Each round will allow teams to refine their models and techniques based on multi-turn interactions, identifying strengths and weaknesses.

Amazon will select up to 10 teams for the competition starting November 2024, which will run through the academic year. Each of the 10 selected teams will receive $250,000 in sponsorship along with monthly AWS credits, and winning teams have a chance to win an additional $700,000 in cash prizes.

Advancements and opportunities in AI-assisted software development

The Amazon Trusted AI Challenge aims to enhance the safety, reliability, and trustworthiness of LLMs powering AI-assisted software development tools. With the rise of generative AI coding assistants, these technologies demonstrate unprecedented innovative capabilities and offer exciting opportunities to ensure responsible and reliable use. This challenge looks to inspire developers, scientists and researchers to create solutions that enhance AI-assisted coding tools' ability to protect users and systems.

Tournament structure

Through four tournaments and a live finals event, red teams will test model developer teams' AI models to uncover vulnerabilities and improve their security. Red teams will be ranked on their success in forcing models to breach their policies through automated conversational red-teaming. Model developer teams will create code-generating models to enhance security, identify threats, and prevent unintended behavior. They will be ranked on their ability to build and reinforce successful defenses through techniques such as fine-tuning and alignment. The goal is to discover innovative ways for LLM creators to mitigate risks and implement effective safety measures.

The top model developer team wins $250,000, with $100,000 for second place. The red team demonstrating the most effective vulnerability identification also wins $250,000, with $100,000 for second place.

For more information about the challenge, including rules and frequently asked questions, visit the Amazon Trusted AI Challenge landing page.

Amazon Trusted AI Challenge square.png
Start your application
Create an account on the YouNoodle website and submit your application before the deadline on September 1, 2024.
Apply now

Research areas
Tags

Related content

Work with us

See more jobs See more jobs
Research Scientist II, Worldwide Defect Elimination
US, WA, Seattle
The Worldwide Defect Elimination (WWDE) Science team in Amazon Customer Service builds state-of-the-art Artificial Intelligence (AI) models to enable defect-free shopping experiences for Amazon customers. We develop technology and mechanisms to discover, root cause, measure, and escalate defects for resolution before they impact a broader range of customers. We are looking for a creative problem solver and technically-skilled Research Scientist able and interested in building AI solutions to address customer issues at scale. The ideal candidate will lead the development of innovative solutions that identify, root cause, attribute, and summarize problems embedded in large volumes of customer feedback in different modalities. They will also utilize the latest advances in GenAI technology to explore billions of customer contacts and automate defect resolution workflows. As a part of this role, this candidate will collaborate with a large team of experts in the field and move the state of defect elimination research forward. This candidate should have a knack for leveraging AI to translate complex data insights into actionable strategies and can communicate these effectively to both technical and non-technical audiences. Key job responsibilities - Apply science models to extract actionable information from large volumes and varying modalities of customer feedback - Leverage GenAI/Large Language Model (LLM) technology for scaling and automating defect elimination workflows - Design and implement metrics to evaluate the effectiveness of AI models - Present deep dives and analysis to both technical and non-technical stakeholders, ensuring clarity and understanding and influencing business partners - Perform statistical analysis and statistical tests including hypothesis testing and A/B testing - Recognize and adopt best practices in reporting and analysis: data integrity, test design, analysis, validation, and documentation A day in the life If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan About the team The Worldwide Defect Elimination (WWDE) team's mission is to understand and resolve all issues impacting customers at scale. The WWDE Science team is a force multiplier within this group, helping to to apply science solutions to eliminate defects and enhance customer experience.
Applied Scientist II, Alexa Devices, GENIE Science
US, WA, Bellevue
We are a part of Amazon Alexa Devices organization with the mission “delight customers through contextual and personalized proactive experiences that keep customers informed, engaged, and productive without cognitive burden”. We are developing an advanced system using Large Language Model (LLM) technologies to deliver engaging, intuitive, and adaptive content recommendations across all Amazon surfaces. We aim to facilitate seamless reasoning and customer experiences, surpassing the capabilities of previous machine learning models. We are looking for a passionate, talented, and resourceful Applied Scientist in the field of Natural Language Processing (NLP), Recommender Systems and/or Information Retrieval, to invent and build scalable solutions for a state-of-the-art context-aware speech assistant. A successful candidate will have strong machine learning background and a desire to push the envelope in one or more of the above areas. The ideal candidate would also enjoy operating in dynamic environments, be self-motivated to take on challenging problems to deliver big customer impact, shipping solutions via rapid experimentation and then iterating on user feedback and interactions. Key job responsibilities As an Applied Scientist on the team, you will collaborate with other applied scientists and engineers to develop novel algorithms to enable timely, relevant and delightful recommendations and conversations. Your work will directly impact our customers in the form of products and services that make use of various machine learning, deep learning and language model technologies. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in the state of art.
Applied Scientist II, Amazon
US, CA, Palo Alto
The Amazon Search Mission Understanding (SMU) team is at the forefront of revolutionizing the online shopping experience through the Amazon search page. Our ambition extends beyond facilitating a seamless shopping journey; we are committed to creating the next generation of intelligent shopping assistants. Leveraging cutting-edge Large Language Models (LLMs), we aim to redefine navigation and decision-making in e-commerce by deeply understanding our users' shopping missions, preferences, and goals. By developing responsive and scalable solutions, we not only accomplish the shopping mission but also foster unparalleled trust among our customers. Through our advanced technology, we generate valuable insights, providing a guided navigation system into various search missions, ensuring a comprehensive and holistic shopping experience. Our dedication to continuous improvement through constant measurement and enhancement of the shopper experience is crucial, as we strategically navigate the balance between immediate results and long-term business growth. We are seeking an Applied Scientist who is not just adept in the theoretical aspects of Machine Learning (ML), Artificial Intelligence (AI), and Large Language Models (LLMs) but also possesses a pragmatic, hands-on approach to navigating the complexities of innovation. The ideal candidate will have a profound expertise in developing, deploying, and contributing to the next-generation shopping search engine, including but not limited to Retrieval-Augmented Generation (RAG) models, specifically tailored towards enhancing the Rufus application—an integral part of our mission to revolutionize shopping assistance. You will take the lead in conceptualizing, building, and launching groundbreaking models that significantly improve our understanding of and capabilities in enhancing the search experience. A successful applicant will display a comprehensive skill set across machine learning model development, implementation, and optimization. This includes a strong foundation in data management, software engineering best practices, and a keen awareness of the latest developments in distributed systems technology. We are looking for individuals who are determined, analytically rigorous, passionate about applied sciences, creative, and possess strong logical reasoning abilities. Join the Search Mission Understanding team, a group of pioneering ML scientists and engineers dedicated to building core ML models and developing the infrastructure for model innovation. As part of Amazon Search, you will experience the dynamic, innovative culture of a startup, backed by the extensive resources of Amazon.com (AMZN), a global leader in internet services. Our collaborative, customer-centric work environment spans across our offices in Palo Alto, CA, and Seattle, WA, offering a unique blend of opportunities for professional growth and innovation. Key job responsibilities Collaborate with cross-functional teams to identify requirements for ML model development, focusing on enhancing mission understanding through innovative AI techniques, including retrieval-Augmented Generation or LLM in general. Design and implement scalable ML models capable of processing and analyzing large datasets to improve search and shopping experiences. Must have a strong background in machine learning, AI, or computational sciences. Lead the management and experiments of ML models at scale, applying advanced ML techniques to optimize science solution. Serve as a technical lead and liaison for ML projects, facilitating collaboration across teams and addressing technical challenges. Requires strong leadership and communication skills, with a PhD in Computer Science, Machine Learning, or a related field.
Applied Scientist, Shanghai AI Lab
CN, 31, Shanghai
The AWS Shanghai AI Lab is looking for a passionate, talented, and inventive staff in all AI domains with a strong machine learning background as an Applied Scientist. Founded in 2018, the Shanghai Lab has been an innovation center of for long-term research projects across domains as machine learning, computer vision, natural language processing, and open-source AI system. Meanwhile, these incubated projects power products across various AWS services. As part of the lablet, you will take a leadership role and join a vibrant team with a diverse set of expertise in both machine learning and applicational domains. You will work on state-of-the-art solutions on fundamental research problems with other world-class scientists and engineers in AWS around the globe and across the boarders. You will have the responsibility to design and innovate solutions to our customers. You will build models to tame large amount of data, achieve industry-level scalability and efficiency, and along the way rapidly grow and build the team.
Applied Scientist I, ACT (Alexa Customers & Technology) CAS
US, WA, Bellevue
As part of Alexa CAS team, our mission is to create a best-in-class Conversational AI that is intuitive, intelligent, and responsive, by developing superior Large Language Models (LLM) solutions and services which increase the capabilities built into the model and which enable utilizing thousands of APIs and external knowledge sources to provide the best experience for each request across millions of customers and endpoints. We are looking for a passionate, talented, and resourceful Applied Scientist in the field of LLM, Artificial Intelligence (AI), Natural Language Processing (NLP), Recommender Systems and/or Information Retrieval, to invent and build scalable solutions for a state-of-the-art context-aware conversational AI. A successful candidate will have strong machine learning background and a desire to push the envelope in one or more of the above areas. The ideal candidate would also have hands-on experiences in building Generative AI solutions with LLMs, enjoy operating in dynamic environments, be self-motivated to take on challenging problems to deliver big customer impact, moving fast to ship solutions and then iterating on user feedback and interactions. Key job responsibilities As an Applied Scientist, you will leverage your technical expertise and experience to collaborate with other talented applied scientists and engineers to research and develop novel algorithms and modeling techniques to reduce friction and enable natural and contextual conversations. You will analyze, understand and improve user experiences by leveraging Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in artificial intelligence. You will work on core LLM technologies, including Supervised Fine-Tuning (SFT), In-Context Learning (ICL), Learning from Human Feedback (LHF), etc. Your work will directly impact our customers in the form of novel products and services . Key job responsibilities . You will analyze, understand and improve user experiences by leveraging Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in artificial intelligence. . You will work on core LLM technologies, including developing best-in-class modeling, prompt optimization algorithms to enable Conversation AI use cases · Build and measure novel online & offline metrics for personal digital assistants and customer scenarios, on diverse devices and endpoints · Create, innovate and deliver deep learning, policy-based learning, and/or machine learning based algorithms to deliver customer-impacting results · Perform model/data analysis and monitor metrics through online A/B testing · Research and implement novel machine learning and deep learning algorithms and models.
Applied Scientist, Prime Tech
US, WA, Seattle
Interested in helping build Prime's content and offer personalization system to drive huge business impact on millions of customers? Join our team of Scientists and Engineers developing algorithms to adaptively generate, optimize, and personalize the customer experience with Amazon Prime. This includes identifying who our customers are and providing them with personalized relevant content. As an ML scientist, you will partner directly with product owners to intake, build, and directly apply your modeling solutions. There are numerous scientific and technical challenges you will get to tackle in this role, such as deep learning and reinforcement learning, and their application to various types of contextual, multi-step optimization of the customer journey. We employ techniques from supervised learning, multi-armed bandits, optimization, and RL - while this role is focused on the space of discriminative and generative recommender systems. As the central science team within Prime, our expertise gets routinely called upon to weigh in on a variety of topics. We also emphasize the need and value of scientific research and have developed a strong publication and patent record (internally/externally) which you will be a part of. You will also utilize and be exposed to the latest in ML technologies and infrastructure: AWS technologies (EMR/Spark, Redshift, Sagemaker, DynamoDB, S3, ...), various ML algorithms and techniques (Random Forests, Neural Networks, supervised/unsupervised/semi-supervised/reinforcement learning, LLMs), and statistical modeling techniques. Major responsibilities - Build and develop machine learning models and supporting infrastructure at TB scale, in coordination with software engineering teams. - Leverage Bandits, Supervised Learning, and Reinforcement Learning for Contextual Recommendation and Optimization Systems. - Develop offline policy estimation tools and integrate with reporting systems. - Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation. - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes. - Work closely with the business to understand their problem space, identify the opportunities and formulate the problems. - Use machine learning, data mining, statistical techniques and others to create actionable, meaningful, and scalable solutions for the business problems. - Design, develop and evaluate highly innovative models and statistical approaches to understand and predict customer behavior and to solve business problems.
Principal Reliability Scientist
US, WA, Bellevue
We are designing the future. If you are in quest of an iterative fast-paced environment, where you can drive innovation through scientific inquiry, and provide tangible benefit to hundreds of thousands of our associates worldwide, this is your opportunity. Come work on the Amazon Worldwide Fulfillment Design & Engineering Team! We are looking for an exceptional Principal Reliability Engineer, someone that is excited to work on complex real-world challenges for which a comprehensive scientific approach is necessary to drive solutions. Your investigations will include identifying and mitigating reliability risks resulting in design and implementation of safe and reliable fulfillment center designs. We are seeking a Principal Reliability Scientist who will be responsible to lead reserach specific to reliability for Amazon Fulfillment Facilities. Key job responsibilities a. Employing a Failure based approach to develop and implement both analytical and empirical methods for identifying and evaluating reliability risks at the stages of facility design, MHE selection, and deployment. b. Lead research activities related Design for Reliability (DfR) initiatives for a warehouse, Develop predictive modelling techniques, and test plans. c. Lead AI-powered reliability solutions that combine data, machine learning, and domain knowledge about asset operations to help anticipate reliability risks, increase uptime, manage CapEx and OpEx costs, and improve productivity. d. Identifying gaps in current Reliability standards and guidelines, and lead comprehensive research to redefine “industry best practices” based on solid scientific foundations. e. Continuously strive to gain in-depth knowledge of your profession, as well as branch out to learn about intersecting fields, such as robotics and mechatronics. f. Travelling to our various sites to perform thorough assessments and gain in-depth operational feedback, approximately 25%-50% of the time. g. Regular communication with Executive management regarding status, risks and operational program metrics.
Applied Scientist II, Amazon
US, CA, Palo Alto
The Amazon Search Mission Understanding (SMU) team is at the forefront of revolutionizing the online shopping experience through the Amazon search page. Our ambition extends beyond facilitating a seamless shopping journey; we are committed to creating the next generation of intelligent shopping assistants. Leveraging cutting-edge Large Language Models (LLMs), we aim to redefine navigation and decision-making in e-commerce by deeply understanding our users' shopping missions, preferences, and goals. By developing responsive and scalable solutions, we not only accomplish the shopping mission but also foster unparalleled trust among our customers. Through our advanced technology, we generate valuable insights, providing a guided navigation system into various search missions, ensuring a comprehensive and holistic shopping experience. Our dedication to continuous improvement through constant measurement and enhancement of the shopper experience is crucial, as we strategically navigate the balance between immediate results and long-term business growth. We are seeking an Applied Scientist who is not just adept in the theoretical aspects of Machine Learning (ML), Artificial Intelligence (AI), and Large Language Models (LLMs) but also possesses a pragmatic, hands-on approach to navigating the complexities of innovation. The ideal candidate will have a profound expertise in developing, deploying, and contributing to the next-generation shopping search engine, including but not limited to Retrieval-Augmented Generation (RAG) models, specifically tailored towards enhancing the Rufus application—an integral part of our mission to revolutionize shopping assistance. You will take the lead in conceptualizing, building, and launching groundbreaking models that significantly improve our understanding of and capabilities in enhancing the search experience. A successful applicant will display a comprehensive skill set across machine learning model development, implementation, and optimization. This includes a strong foundation in data management, software engineering best practices, and a keen awareness of the latest developments in distributed systems technology. We are looking for individuals who are determined, analytically rigorous, passionate about applied sciences, creative, and possess strong logical reasoning abilities. Join the Search Mission Understanding team, a group of pioneering ML scientists and engineers dedicated to building core ML models and developing the infrastructure for model innovation. As part of Amazon Search, you will experience the dynamic, innovative culture of a startup, backed by the extensive resources of Amazon.com (AMZN), a global leader in internet services. Our collaborative, customer-centric work environment spans across our offices in Palo Alto, CA, and Seattle, WA, offering a unique blend of opportunities for professional growth and innovation. Key job responsibilities Collaborate with cross-functional teams to identify requirements for ML model development, focusing on enhancing mission understanding through innovative AI techniques, including retrieval-Augmented Generation or LLM in general. Design and implement scalable ML models capable of processing and analyzing large datasets to improve search and shopping experiences. Must have a strong background in machine learning, AI, or computational sciences. Lead the management and experiments of ML models at scale, applying advanced ML techniques to optimize science solution. Serve as a technical lead and liaison for ML projects, facilitating collaboration across teams and addressing technical challenges. Requires strong leadership and communication skills, with a PhD in Computer Science, Machine Learning, or a related field.
Applied Scientist, Sponsored Products Marketplace Intelligence
US, CA, Santa Monica
We are the Sponsored Products - Marketplace Intelligence (MI) team. We are looking for a Data Scientist to help build production ML and bandit solutions to customize the search experience. We determine which ads to show in Amazon search, where to place them, how many ads to place, and to which customers. This helps shoppers discover new products while helping advertisers put their products in front of the right customers, aligning shoppers’, advertisers’, and Amazon’s interests. To do this, we apply a broad range of machine learning, causal inference, and optimization techniques to continuously explore, learn, and optimize the allocation and ranking of ads on the search page. We are an interdisciplinary team with a focus on customer obsession and inventing and simplifying. Our primary focus is on improving the SP experience in search by gaining a deep understanding of shopper pain points and developing new innovative solutions to address them. You will be on the Interleaving team. Our mission is to personalize and contextualize SP ad allocation on the search page. We do this by modeling shopper responses to the number, placement, and quality of ads. We are a data- and hypothesis-driven organization that uses online experimentation, simulation, causal modeling, and online feedback to place ads where they’re useful to shoppers and provide improved discoverability and sales for advertisers. You’ll lead experimentation efforts based on data deep dives to support these solutions. This is a unique opportunity for someone who wants to have broad business impact, a direct impact on customers and the search experience, and get broad exposure to a wide range of scientific techniques (machine learning, bandit learning, optimization, LLMs). We are looking for an Applied Scientist to join Interleaving team in Marketplace Intelligence with a broad mandate to experiment and innovate to grow Sponsored Products. If you thrive in a product-focussed and data-driven environment, then this role is for you. As an Applied Scientist in this team, you will help to identify unique opportunities to create customized and delightful shopping experience for our growing marketplaces worldwide. Your job will be to identify big opportunities for the team that can help to grow Sponsored Products business working with retail partner teams, product managers, software engineers and TPMs. You will have opportunity to design, run and analyze / experiments to improve the experience of millions of Amazon shoppers while driving quantifiable revenue impact. More importantly, you will have the opportunity to broaden your technical skills in an environment that thrives on creativity, experimentation, and product innovation. Key job responsibilities * Tackle and solve challenging science and business problems that balance the interests of advertisers, shoppers, and Amazon. * Develop real-time machine learning algorithms to allocate billions of ads per day in advertising auctions. * Develop efficient algorithms for multi-objective optimization and AI control methods to find operating points for the ad marketplace then evolve them * Be an expert at designing and implementing solutions that use a range of data science methodologies to automate data analysis or to solve complex business problems. * Perform hands-on analysis and modeling of enormous data sets to develop insights that improve shopper experience, without compromising Ad revenue in addition to designing metrics for complex systems. * Drive end-to-end machine learning projects that have a high degree of ambiguity, scale, complexity. * Run A/B experiments, gather data, and perform statistical analysis.
Research Scientist , Amazon Stores Economics & Science (SEAS)
US, WA, Seattle
The Stores Economics and Science (SEAS) team uses Economics, Statistics, and Machine Learning to understand and design the complex economy of Amazon’s network of buyers and sellers. We are an interdisciplinary team, committed to use of cutting edge technology and leveraging the strengths of engineers and scientists to build solutions for some of the toughest business problems at Amazon. We are looking for an outstanding RSI who is able to provide structure around complex business problems, design and implement large scale sequential optimization algorithms, and validate their models through simulation and real world data. We are looking for creative thinkers who can combine a strong optimization toolbox with a desire to learn from others, and who know how to execute and deliver on big ideas. Experience in Operations Research, Optimization, Control Theory is essential, and you should be familiar with modern tools for data science and business analysis. Key job responsibilities * Develop scalable mathematical models to derive optimal or near-optimal solutions to existing and new business challenges. * Build prototype and simulation models to test devised solutions. * Advocate technical solutions to business stakeholders, engineering teams, as well as executive-level decision makers. * Create policy evaluation methods to track the actual performance of devised solutions in production systems, identify areas with potential for improvement and work with internal teams to improve the solution with new features. * Mentor team members for their career development and growth. * Present business cases and document models, analyses, and their results in order to influence important decisions. * Work comfortably in a multi-disciplinary setting.
View from space of a connected network around planet Earth representing the Internet of Things.
Have feedback?
Let us know by taking this short survey
Get started
Amazon.com | Conditions of Use | Privacy | © 1996-2024 Amazon.com, Inc. or its affiliates
Follow Us