Lessons from Defending Gemini Against Indirect Prompt Injections

Chongyang Shi, Sharon Lin, Shuang Song, Jamie Hayes, Ilia Shumailov, Itay Yona, Juliette Pluto, Aneesh Pappu, Christopher A. Choquette-Choo, Milad Nasr, Chawin Sitawarin, Gena Gibson, Andreas Terzis, John “Four” Flynn · 2025

Read Lessons from Defending Gemini Against Indirect Prompt Injections by Chongyang Shi, Sharon Lin, Shuang Song, Jamie Hayes, Ilia Shumailov, Itay Yona, Juliette Pluto, Aneesh Pappu, Christopher A. Choquette-Choo, Milad Nasr, Chawin Sitawarin, Gena Gibson, Andreas Terzis, John “Four” Flynn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14534.txt.

Description

Shi and co-authors report lessons from defending the Gemini AI models against hidden instruction attacks in a 2025 paper. Attackers plant commands in data the model reads, hoping it will stray from the user's wishes. Tool use raises the stakes, since the model can touch user data and rights. The Google DeepMind team built a standing test arena for such attacks.

Adaptive attack suites run against old, current, and future model versions without pause. The report shares what the team learned about where models bend and which guards hold. Findings feed straight back into training and product rules. The authors argue that steady attack testing, not one-off checks, is what hardens a tool-using model.

Catalog record

CREATORChongyang Shi, Sharon Lin, Shuang Song, Jamie Hayes, Ilia Shumailov, Itay Yona, Juliette Pluto, Aneesh Pappu, Christopher A. Choquette-Choo, Milad Nasr, Chawin Sitawarin, Gena Gibson, Andreas Terzis, John “Four” Flynn
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE140 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-14534

Browse the full library · About the Tower of Babel