Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software

Tomer Kordonsky*, Maayan Yamin*, Noam Benzimra*, Amit LeVi*, Avi Mendelson · 2026

Read Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software by Tomer Kordonsky*, Maayan Yamin*, Noam Benzimra*, Amit LeVi*, Avi Mendelson (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04894.txt.

Description

Software written by large language models tends to follow recurring templates, and its security flaws may repeat in the same predictable ways. The work is by Tomer Kordonsky and 4 co-authors. The team asks whether an attacker who can only see a program's public face can guess its hidden weaknesses.

They build a tool called FSTab that links visible front-end features to likely back-end flaws, using knowledge of which model wrote the code. Tested on leading code models including GPT-5.2 and Claude, the attack reached up to 94 percent success on internal tools, even for app types it was never trained on. A second part of the tool measures how steadily each model repeats the same flaws. The findings warn that generated code carries a shared, learnable attack surface.

Catalog record

CREATORTomer Kordonsky*, Maayan Yamin*, Noam Benzimra*, Amit LeVi*, Avi Mendelson
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-04894

Browse the full library · About the Tower of Babel