Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures
Nicolas Lacroix , Frédéric Precioso, Mireille Blay-Fornarino, Sébastien Mosser · 2026
Read Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures by Nicolas Lacroix , Frédéric Precioso, Mireille Blay-Fornarino, Sébastien Mosser (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03988.txt.
Introduction
Nicolas Lacroix and three co-authors, Mireille Blay-Fornarino, Sébastien Mosser, and Frederic Precioso, ask whether small language models can recover the steps inside machine learning code. Knowing those steps helps researchers see how data work is really done. Fast change in tools and libraries makes the task hard.
The group first compares several small models with a Cochran Q test. They then test the best model against two earlier reference studies with McNemar tests. Further checks vary how step labels are defined and compare the pattern of findings with past work. The study is built as a careful check of earlier claims, not just a new tool demo. Results bear on how far small models can read real code.
Catalog record
| CREATOR | Nicolas Lacroix , Frédéric Precioso, Mireille Blay-Fornarino, Sébastien Mosser |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03988
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures Nicolas Lacroix , Frédéric Precioso, Mireille Blay-Fornarino, Sébastien Mosser Abstract Background : Extracting the stages that structure Machine Learning (ML) pipelines from source code is key for gaining a deeper understanding of data science practices. However, the diversity caused by the constant evolution of the ML ecosystem ( e.g. , algorithms, libraries, datasets) makes this task challenging. Existing approaches either depend on non-scalable, manual labeling or on ML classifiers that do not properly support the diversity of the domain. These limitations highlight the need for more flexible and reliable solutions. Objective : We evaluate whether Small Language Models (SLMs) can leverage their code understanding and classification abilities to address these limitations, and subsequently, how they can advance our understanding of data science practices. Method : We conduct a confirmatory study based on two reference works selected for their relevance regarding current state-of-the-art’s limitations. First, we compare several SLMs using Cochran’s Q test. The best-performing model is then evaluated against the reference studies using two distinct McNemar’s tests. We further analyze how variations in taxonomy definitions affect performance through an additional Cochran’s Q test. Finally, a goodness-of-fit analysis is conducted using Pearson’s chi-squared tests to compare our insights on data science practices with those from prior studies. I Introduction Understanding how data scientists structure their Machine Learning (ML) pipelines is key to better supporting the quality and evolution of these pipelines. Studying them at an abstract level enables the analysis of how workflows are organized, iterated, and reused. This higher-level perspective is therefore crucial to characterize current practices empirically. Recently, studies have examined pipeline quality [ 32 , 12 ] , identified recurring patterns [ 4 ] , and supported practitioners through pipeline reuse [ 21 ] . All these efforts rely on accurately extracting the underlying structure of ML pipelines from source code. However, this task is often achieved manually because of the high diversity of the ML domain. Indeed, the exploratory nature [ 15 ] of ML associated with the rapid evolution of new data processing algorithms and ML libraries has led to increasingly heterogeneous ML pipelines [ 3 , 20 ] . From the reliance on static mappings between specific libraries and pipeline stages [ 13 , 3 , 28 ] , efforts have recently moved toward automated approaches with rule-based or supervised classifiers [ 31 , 34 , 19 ] . Despite their promising results, they are still constrained in performance or in scope ( e.g. , specific libraries or pipeline patterns). As a result, scale and reliability of current analyses of data science practices remain limited. Simultaneously, language models have shown strong capabilities in code understanding and classification tasks [ 11 ] . In addition, Small Language Models (SLMs) have drawn interest from the scientific community, looking for more sustainable and reproducible practices. Yet, their potential for extracting ML pipeline structures has still to be evaluated. In this confirmatory study, we will investigate whether SLMs can be used as a classification component in a reverse-engineering process to efficiently extract ML pipeline structures. Our contribution is threefold: 1. An assessment of the performance gain of five SLMs, pre-selected based on their HumanEval performance [ 8 ] , compared to existing works 2. A statistical evaluation of the potential impact of taxonomy definition on classification results, using a mutation process 3. A goodness-of-fit (Pearson’s chi-squared) test to challenge the current understanding of data scientists’ practices reported in existing literature Results of our work could open new perspectives about reliable ML pipelines characterization ( e.g. , anti-pattern identification [ 9 , 4 ] ), ML transparency ( e.g. , data provenance enrichment [ 16 ] ), and data scientists’ guidance ( e.g. , visualizations and recommendation systems [ 21 ] ). II Background and Related Work II-A Reverse-Engineering and Language Models While code summarizing and documentation generation remain very common applications [ 11 ] , researchers have recently become interested in the ability of language models to extract structured information from code bases. More specifically, Boronat et al. [ 5 ] have proposed a tool, MDRE-LLM , that integrates LLMs with traditional Model-Driven Reverse Engineering (MDRE) techniques for extracting high-level abstractions from source code. Additionally, Siala et al. [ 25 ] studied the use of LLMs for reverse engineering Python and Java code to the OCL formal language. Similarly, Campanello et al. [ 6 ] have shown promising results in the use of LLM (GPT-4) for reverse engineering class (as PlantUML) diagrams. It is also crucial to adopt a rigorous methodology to properly support the probabilistic nature of language models. To that end, Wagner et al. [ 29 ] have proposed important preliminary guidelines. Most notably, open models, transparent prompts and models configuration are described as key factors in achieving reliable and reproducible results. II-B Reverse-Engineering Applied to ML Pipelines Extraction Software engineering researchers have a long-standing interest in analyzing scientific workflows [ 26 , 33 ] and managing computational processes [ 9 , 14 , 18 ] , with a growing focus on extracting ML pipeline structures from source code for understanding and evaluating practices. Static mapping or rule-based approaches identify stages by matching code to predefined functions or libraries. For instance, initial work by Venkatesh et al. [ 28 ] identified stages based on a static mapping of 13 libraries, integrating these as documentation. Similarly, Jiang et al. [ 13 ] proposed around 700 hard-coded rules derived from a few selected ML libraries and have shown that one cell can contain code instructions related to multiple stages. Biswas et al. [ 3 ] used a static mapping composed of 403 specific functions to explore data scientists’ practices, giving first insights about stages frequency and ordering in Kaggle notebooks. Works leveraging supervised classifiers have tried to mitigate these static limitations. For example, Zhang et al.’s CORAL solution [ 34 ] uses an ML classifier trained on a custom taxonomy but is restricted to seven Data Science libraries (pandas, statsmodels, etc.). More recently, Venkatesh et al. has extended their initial work and proposed a supervised classifier [ 27 ] trained on their previous mapping of 13 libraries. Similarly, Perez et al. [ 19 ] tried to improve this approach by combining a decision tree model with a rule-based model. In their comprehensive empirical study, Ramasamy et al. [ 20 ] also developed a classifier and explored data scientists’ practices. They confirmed the imbalanced stage frequencies and transition probabilities, but showed mixed results in their classifier evaluation (e.g., poor F 1 score of 0.29 for prediction), highlighting the need for more flexible reverse-engineering solutions. Overall, these approaches have demonstrated promising results within specific contexts. However, their reliance on predefined rules or narrowly trained datasets limits their scalability and generalization to the highly variable ML ecosystem. III Research Questions and Hypotheses Our confirmatory study targets two specific research goals: • Research Goal 1 : Evaluate the applicability of SLMs for extracting ML pipelines structures • Research Goal 2 : Challenge our current understanding of data scientists’ practices The first goal will be reached by answering the two research questions RQ1 and RQ2 while answering RQ3 will contribute to the second goal.