Trim My View: An LLM-Based Code Query System for Module Retrieval in Robotic Firmware

Sima Arasteh 1 , Pegah Jandaghi 2 , Nicolaas Weideman 2 , Dennis Perepech 1 , Mukund Raghothaman 1 , Christophe Hauser 3 and Luis Garcia 4 · 2025

Read Trim My View: An LLM-Based Code Query System for Module Retrieval in Robotic Firmware by Sima Arasteh 1 , Pegah Jandaghi 2 , Nicolaas Weideman 2 , Dennis Perepech 1 , Mukund Raghothaman 1 , Christophe Hauser 3 and Luis Garcia 4 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03969.txt.

Introduction

Sima Arasteh and co-authors build a search engine that explains stripped robot firmware in plain words, in a 2025 paper. Compiling software scrambles its design, and shipped binaries often lose their labels, so analysts struggle to find each module and guess its job. Their system, Trim My View, chains binary splitting, decompilation, and language-model summaries of each function.

They tried three open language models as the summarizer on 467 modules from four devices running the ArduPilot suite. The best, CodeStral, reached an F1 score of 0.68 and answered in seconds. The authors present the pipeline as a cheap way to map unknown binaries, with robotics firmware as the proving ground.

Catalog record

CREATORSima Arasteh 1 , Pegah Jandaghi 2 , Nicolaas Weideman 2 , Dennis Perepech 1 , Mukund Raghothaman 1 , Christophe Hauser 3 and Luis Garcia 4
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-03969

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Trim My View: An LLM-Based Code Query System for Module Retrieval in Robotic Firmware PubID: pubid: Workshop on Binary Analysis Research (BAR) 2025 28 February 2025, San Diego, CA, USA ISBN 979-8-9919276-4-2 https://dx.doi.org/10.14722/bar.2025.23xxx www.ndss-symposium.org Sima Arasteh 1 , Pegah Jandaghi 2 , Nicolaas Weideman 2 , Dennis Perepech 1 , Mukund Raghothaman 1 , Christophe Hauser 3 and Luis Garcia 4 Abstract The software compilation process has a tendency to obscure the original design of the system and makes it difficult both to identify individual components and discern their purpose simply by examining the resulting binary code. Although decompilation techniques attempt to recover higher-level source code from the machine code in question, they are not fully able to restore the semantics of the original functions. Furthermore, binaries are often stripped of metadata, and this makes it challenging to reverse engineer complex binary software. In this paper we show how a combination of binary decomposition techniques, decompilation passes, and LLM-powered function summarization can be used to build an economical engine to identify modules in stripped binaries and associate them with high-level natural language descriptions. We instantiated this technique with three underlying open-source LLMs—CodeQwen, DeepSeek-Coder and CodeStral—and measured its effectiveness in identifying modules in robotics firmware. This experimental evaluation involved 467 modules from four devices from the ArduPilot software suite, and showed that CodeStral, the best-performing backend LLM, achieves an average F1-score of 0.68 with an online running time of just a handful of seconds. I Introduction Knowledge of the high-level organization of software—its functions, files and modules—and how its functionality is divided among these units is crucial to productive programming, reverse engineering, and other software engineering practices. While at least some of this information is evident while examining the source code , it is much more challenging to recover when working with binaries . Crucially, the compilation process tends to obscure the organizational units of software, including information about its modules, comments, data types, and identifier names (when working with stripped binaries). Although this is sometimes desirable (for example, to protect intellectual property), it severely hinders the understandability of the final binary. Furthermore, despite impressive advances in decompilation techniques [ 1 ] , they are unable to completely recover semantic details of the original source code [ 2 ] . Recent breakthroughs in the development of large language models (LLMs) have prompted researchers to investigate their application in various problems related to the semantic understanding of (binary) code: i.e., binary code summarization [ 3 , 4 ] , source-to-binary matching [ 5 ] , and function identification [ 6 ] . Several studies have also evaluated their effectiveness for function summarization tasks [ 3 , 4 , 7 , 8 , 9 ] . Despite their impressive accuracy, we argue that function summarization is insufficient for a number of problems related to understanding the high-level structure of programs. For example, engineers may wish to identify portions of the codebase responsible for implementing communication protocols such as the serial peripheral interface (SPI), or for managing the universal asynchronous receiver/transmitter (UART). These functionalities may be deeply embedded in the code and/or divided among multiple functions. This limits the effectiveness of function summarization tools for many reverse engineering activities, which instead require identifying and navigating between larger units of the programs, such as its modules. Another challenge that arises while using LLMs is economic in nature: Well-known commercial LLMs such as GPT-X [ 10 ] , Copilot, 1 1 1 https://copilot.github.com/ and Gemini [ 11 ] charge money when used on a large scale. As an example, the authors of BinSum [ 3 ] report incurring an expense of $11,000 when evaluating GPT-4. One could use open-source LLMs instead [ 12 , 13 ] . However, GPUs with large amounts of memory are prohibitively expensive, and this would restrict us to use mid-size language models. This raises an additional worry that these smaller locally-run models would be less accurate or effective at discharging their tasks. In this paper, we describe a system—called ChatCPS—in which the user requests parts of the code related to some high-level functionality of interest. The code takes the form of stripped binaries. In response, the system automatically identifies modules/groups of related functions related to the desired functionality and presents them to the user for examination. As indicated by the name ( \cdots -CPS ), our primary focus is on reverse engineering firmware for robot controllers. Our system relies on a combination of binary decomposition techniques, function decompilation, and multiple levels of LLM application. In the first LLM pass, we obtain summaries of individual functions with the binary. In parallel, we apply BCD [ 14 ] , an algorithm for decomposing binary code into components/modules. (Because the source code of the original BCD implementation was unavailable, and because we were working with ARM rather than x86-64 binaries, we had to reimplement this algorithm). We then group together summaries for related functions, and use a second pass with the LLM to obtain a categorization of the module as a whole. We summarize this workflow in Figure 1 . We focus on four categories in which CPS experts frequently express interest: data transfer, navigation, controllers, and safety checks. We describe these categories in more detail in Section III-C . Fig. 1: The Workflow of the ChatCPS system. A) ChatCPS decomposes the binary firmware into its modules. B) Shows the decompilation process. F_{n} indicates binary functions, while F_{n}^{*} indicates the corresponding decompiled functions. C) ChatCPS summarizes each function in a module using three open-source LLMs (CodeQwen, CodeStral, DeepSeek-Coder) and D) categorizes modules based on function descriptions. In our experience, this two-pass approach—first summarize individual decompiled functions, and then summarize groups of related function summaries—improves classification accuracy when compared to a hypothetical single-pass procedure (which would presumably attempt to directly summarize entire modules that were previously identified by BCD). This improvement in accuracy allows us to use less expensive (i.e., smaller and open-source) LLMs, rather than expensive commercial offerings. We instantiated our system with three mid-sized LLMs drawn from the BigCodeBench leaderboard [ 15 , 16 ] : CodeQwen [ 17 ] , DeepSeek-Coder [ 13 ] and CodeStral [ 18 ] . Our experimental evaluation with 467 modules from the ArduPilot software suite [ 19 ] indicates that CodeStral provides the most accurate high-level categorization, with an average F1-score of 0.68. As a secondary measurement, we also separately calculated the accuracy of the first LLM pass, i.e., we measured the accuracy of the generated function summaries. This measurement required us to build a source code parser that normalized function bodies by removing comments, variable and function names. We describe our experimental setup and our results in Sections IV and V respectively. Our hope is that ChatCPS would streamline the reverse engineering process and enable users to efficiently identify and distinguish parts of the codebase related to different functionalities. We also hope that module categorization would enhance the user’s understanding of the binary’s underlying structure. Finally, we expect that this top-level categorization would form a basis for more fine-grained categorization of the functions within each module. This would help to distinguish different kinds of algorithms, including Kalman filters [ 20 ] from PID controllers [

Read the complete text