The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation

Akanksha Narula, Mofasshara Binte Rafique, Laurent Bindschaedler · 2026

Read The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation by Akanksha Narula, Mofasshara Binte Rafique, Laurent Bindschaedler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-04057.txt.

Description

Narula, Rafique, and Bindschaedler study hidden cues that steer code generation by language models. Models now write much production code. Many tasks allow more than one valid algorithm. Words or metadata outside the task spec can shift that choice. All outputs may still pass the same tests.

They ran 46,535 controlled experiments on 11 tasks. Nineteen cue types and 15 model setups were covered. Shifts in algorithm-family shares were large and systematic, up to 100 points. The shifts tracked cue meaning, even in applied tasks like rate limiting. Naming the algorithm directly was the steadiest fix they tested. Random context thus forms an invisible lottery over speed, safety, and upkeep.

Catalog record

CREATORAkanksha Narula, Mofasshara Binte Rafique, Laurent Bindschaedler
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.04057

Browse the full library · About the Tower of Babel