CIPHER: Cryptographic Insecurity Profiling via Hybrid Evaluation of Responses

Max Manolov, Tony Gao, Siddharth Shukla, Cheng-Ting Chou, Ryan Lagasse · 2026

Read CIPHER: Cryptographic Insecurity Profiling via Hybrid Evaluation of Responses by Max Manolov, Tony Gao, Siddharth Shukla, Cheng-Ting Chou, Ryan Lagasse (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01438.txt.

Description

Max Manolov and co-authors introduce CIPHER, a benchmark for crypto flaws in model-generated Python code. Language models increasingly help developers, yet their crypto code often carries flaws. Small choices, like static initialization vectors or missing authentication, can quietly void guarantees. CIPHER tests each task under insecure, neutral, and secure prompts, using a crypto flaw taxonomy and line-level attribution. Across many models, secure prompting fixes some targeted issues but fails to reliably remove crypto flaws overall. The benchmark and scoring pipeline will be shared publicly after publication.

Catalog record

CREATORMax Manolov, Tony Gao, Siddharth Shukla, Cheng-Ting Chou, Ryan Lagasse
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46931
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.01438

Browse the full library · About the Tower of Babel