On the Role of Pre-trained Embeddings in Binary Code Analysis
Alwin Maier, Felix Weißberg, Konrad Rieck · 2025
Read On the Role of Pre-trained Embeddings in Binary Code Analysis by Alwin Maier, Felix Weißberg, Konrad Rieck (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08682.txt.
Description
Machine learning tools that read compiled program code often start from pre-trained embeddings, vector summaries learned from unlabeled assembly code. Alwin Maier, Felix Weißberg, and Konrad Rieck question that habit in a 2025 paper on binary code analysis. Labels for many code tasks, they note, are cheap to generate from compiler debug data, unlike labels for human language.
The team tested recent embeddings on five tasks using 1.2 million functions drawn from the Debian software archive. With plenty of labeled data, the embeddings performed much alike, and training a model end to end without pre-training did best on average. Reported gaps between embeddings largely vanished. By shrinking the labeled data step by step, the authors derive guidance on when embeddings still earn their keep.
Catalog record
| CREATOR | Alwin Maier, Felix Weißberg, Konrad Rieck |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-08682