On the Role of Pre-trained Embeddings in Binary Code Analysis

Alwin Maier, Felix Weißberg, Konrad Rieck · 2025

Read On the Role of Pre-trained Embeddings in Binary Code Analysis by Alwin Maier, Felix Weißberg, Konrad Rieck (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08682.txt.

Description

Machine learning tools that read compiled program code often start from pre-trained embeddings, vector summaries learned from unlabeled assembly code. Alwin Maier, Felix Weißberg, and Konrad Rieck question that habit in a 2025 paper on binary code analysis. Labels for many code tasks, they note, are cheap to generate from compiler debug data, unlike labels for human language.

The team tested recent embeddings on five tasks using 1.2 million functions drawn from the Debian software archive. With plenty of labeled data, the embeddings performed much alike, and training a model end to end without pre-training did best on average. Reported gaps between embeddings largely vanished. By shrinking the labeled data step by step, the authors derive guidance on when embeddings still earn their keep.

Catalog record

CREATORAlwin Maier, Felix Weißberg, Konrad Rieck
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-08682

Browse the full library · About the Tower of Babel