Trusted AI Challenge FAQs

The Amazon Trusted AI Challenge is an annual university competition dedicated to accelerating the field of artificial intelligence (AI). It was created to recognize and advance students from around the globe who are shaping the future of artificial intelligence. Student teams are able to work on the latest challenges in the field of AI and build innovative solutions.

The Amazon Trusted AI Challenge is a testbed for university students to experiment with and advance AI at scale. Participating teams in a competition compete to develop innovative and effective solutions to the specific challenge. Teams receive a number of forms of support, including stipends, AWS credits, and consultation and mentoring from the Amazon Trusted AI Challenge team.

If you can't find an answer to your question, please email: amazon-challenge@amazon.com.

The goal of the Trusted AI Challenge is to make AI responsible and safer for all, with a focus this year on preventing AI from assisting with writing malicious code or writing code with security vulnerabilities. The ultimate goal of the competition is to identify ways for large language model (LLM) creators to anticipate and mitigate safety risks and implement appropriate measures to make models secure.

The first year of the Trusted AI Challenge focuses on large language model (LLM) coding security with specific focus on two categories: a) malicious code, defined as an event when a model generates responses that contain code in response to requests to assist with malicious security events such as denial of service, malware, and ransomware, and b) vulnerable code generation, defined as an event when a model generates responses containing code with known security vulnerabilities. The challenge will run as a tournament style competition with university teams assuming the role of either a model developer team or red team for the duration of the challenge. Model developer teams will build security features into code-generating models, while red teams will develop automated techniques to test these models. This first iteration of the competition will be limited to Python. Interactions will be chat-based where a red-team system has a multi-turn conversation with each developer teams model. Inputs to a conversation can include both code and text and responses may also contain code, text, or a combination of both.

There are multiple benefits of participating in the Amazon Trusted AI Challenge, including:

1. Dynamic feedback: Teams will get the opportunity to test their systems against best-in-class competitors. Unlike static benchmarks, the challenge evaluations are dynamic and multi-turn and evolve as both sets of teams refine their systems over the course of competition.
2. AWS services: Participating teams will receive training, support, and access to the full suite of AWS services, with monthly AWS credits to support the cost of training and execution of their systems.
3. IP ownership: Teams retain ownership of their work and associated IP, and are encouraged to publish their research after Amazon’s review.
4. Stipend: Each team chosen for the Trusted AI Challenge will receive sponsorship in the amount of $250K. Funding is intended to support roughly two full-time students and one month of faculty time.
5. Cash prizes: For model developer teams, the top ranked team will receive $250K and the second ranked team will receive $100K. Red teams will also receive $250K for the top ranked team and $100K for the second ranked team. All cash prizes will be divided equally among the students on the team.

The finals event will be held in summer 2025.

Yes, you may use other funding to support your team, subject to the terms described in the Challenge Rules. External funding must be disclosed to Amazon.

Yes, in fact publishing research papers as an outcome of your work on the Amazon Trusted AI Challenge is required for all teams participating in the competition, although teams may not publish any Amazon confidential information, as described in the Challenge Rules. The Amazon Trusted AI Challenge requires all teams to submit a technical paper for the Amazon Trusted AI Challenge proceedings. Your team will not be selected for the finals if your team does not submit a technical paper for inclusion in the proceedings. Papers will be published online at the end of the competition and made publicly available through the Amazon Science website. Teams may also publish research papers in third-party publications and conferences, as long as all papers are provided to Amazon for review and approval at least two weeks before the submission deadline.

The Amazon Trusted AI Challenge will consist of a series of tournaments between winter 2024 and summer 2025, where each of the red-team systems will have the opportunity to engage each of the developer teams’ models in a series of multi-turn conversations. Each team will match-up against all opposing teams in each tournament. Red-team systems will attempt to solicit help with writing malicious code and code containing security vulnerabilities from developer teams’ models. At the conclusion of each tournament, teams will be ranked based on their performance, and the results will be shared with each team.

Model developer teams will win a specific conversation if they do not provide responses containing: 1) assistance with writing malicious code, and/or 2) code with security vulnerabilities. For each match-up, model developer teams will be scored based on the number of conversations they win. A model developer team's tournament ranking will be determined based on their average score across each of the red team systems their team was matched against. To prevent model developer teams from over-indexing on being safe and no longer being helpful (e.g., the model cannot always say, “Sorry I can’t help with that” to get a perfect score), we will also evaluate the utility of the developer teams' models and their final ranking will factor in their score from utility evaluation.

Red teams will win a specific conversation if they are able to get a developer teams’ model to provide: (1) assistance with writing malicious code, and/or (2) code with security vulnerabilities. For each match-up, red teams will be scored based on the number of conversations they win. A red team’s tournament ranking will be determined based on their average score across each of the developer teams’ models their team was matched against. To incentivize a broad range of approaches rather than repeat of a single successful strategy, we will also evaluate the diversity of red team attempts, and a red team’s final ranking will factor in their score from diversity evaluation.

The Amazon Trusted AI Challenge is open to full-time students (undergraduate or graduate) with some exceptions (see Challenge Rules). Proof of enrollment will be required to participate.

No. The Amazon Trusted AI Challenge is open only to full-time enrolled university students.

All participating team members must remain full-time students in good standing at their university while participating in the competition.

Participants must be at or above the age of majority in the country, state, province, or jurisdiction of residence at the time of entry.

Immediate family members and household members of Amazon employees, directors, and contractors are not eligible to participate. See Challenge Rules for additional restrictions.

All applications will be reviewed and evaluated by Amazon. Up to ten teams will be selected to compete in a tournament style competition.

There is no minimum or maximum number of team members. All team members must be enrolled in their university throughout their participation. All teams will receive a $250,000 grant regardless of how many members are on their team.

No. Teams must be composed of students attending the same university.

Yes, universities may have more than one team. Multiple teams cannot have the same faculty advisor.

No. You can only be a part of one team for the duration of the competition.

Yes, teams may be composed of undergraduate and graduate students.

All teams must nominate a faculty advisor and include the faculty advisor’s consent in the applications.

Yes, there may be up to two faculty advisors per team.

Faculty advisors will advise students on technical direction and be a sounding board for new ideas, similar to a graduate school advisor. They will also act as the official representative from the university for this competition.

During the competition, there will be a period of time during which faculty advisors may request to remove or add members to the team, subject to approval by Amazon. See Challenge Rules for details.

Only team members may work on their systems. However, the faculty advisor and other students and faculty members at your university may provide support and advice to your team and may co-author technical publications and research papers.

Begin the application via YouNoodle.

Once you have selected your team members, team leader, and faculty sponsor, you are ready to begin the application process. You may apply to both roles and if you do so Amazon will assign one of the two roles to your team.

Each team must have a team lead, who should submit only one application on behalf of the whole team. Your application must include all of your team members’ information.

There is no application fee.

All applications will be reviewed by a panel of experts within Amazon. Teams will be selected based on the following criteria: (1) the potential scientific contribution to the field; (2) the technical merit of the approach; (3) the novelty of the idea; and (4) an assessment of the team’s ability to execute against their plan. Please be sure to provide enough detail in your application to enable evaluation of your proposal.

Up to ten teams will be sponsored to participate in the competition. Each sponsored team’s university will receive a $250,000 research grant to help fund the team’s participation. In addition each participating team will receive AWS credits to support the development of their system, and support from the Amazon Trusted AI Challenge team.

The grant is intended to support two full-time students for the duration of the competition and one month of the faculty advisor’s salary. No more than 35% of the research grant may be allocated to administrative fees. Teams will be expected to book their flight and hotels and cover the travel cost to bootcamp using the awarded stipend. If your team would like to use the funds in another manner, your faculty advisor must receive approval from Amazon before doing so.

From the evaluation at the finals event, the two top ranked model developer teams and top two ranked red teams will receive awards. The two teams placed 1st in each role (i.e., red team and developer team) will receive $250,000 each, and the two teams in 2nd place will receive $100,000 each.