Control Models for In-IDE Code Completion Conference: 3rd International Workshop on Integrated Development Environments ; April 12–18, 2026; Rio de Janeiro, Brazil 3rd International Workshop on Integrated Development Environments (IDE ’26), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3786

Aral de Moor, Yana Hrynevich, Hleb Badzeika, Vladyslav Furda, Marko Kojic, Artem Savelev, Kostadin Cvejoski, Darya Rovdo, Ekaterina Garanina · 2026

Read Control Models for In-IDE Code Completion Conference: 3rd International Workshop on Integrated Development Environments ; April 12–18, 2026; Rio de Janeiro, Brazil 3rd International Workshop on Integrated Development Environments (IDE ’26), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3786 by Aral de Moor, Yana Hrynevich, Hleb Badzeika, Vladyslav Furda, Marko Kojic, Artem Savelev, Kostadin Cvejoski, Darya Rovdo, Ekaterina Garanina (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20223.txt.

Introduction

Aral de Moor and eight colleagues introduce control models for AI code completion inside JetBrains development tools. These are classifiers that decide when to trigger a completion and which suggestions to show. The goal is to match suggestions to what a user wants and to cut needless model calls.

The authors test boosting and transformer designs on real completion data from dozens of users. Each wasted call costs money and can annoy the user with a poor suggestion. Filters of this kind sit between the user and the model, and they shape how helpful code completion feels in daily work.

Catalog record

CREATORAral de Moor, Yana Hrynevich, Hleb Badzeika, Vladyslav Furda, Marko Kojic, Artem Savelev, Kostadin Cvejoski, Darya Rovdo, Ekaterina Garanina
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20223

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Control Models for In-IDE Code Completion Conference: 3rd International Workshop on Integrated Development Environments ; April 12–18, 2026; Rio de Janeiro, Brazil 3rd International Workshop on Integrated Development Environments (IDE ’26), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3786151.3788608 ISBN: 979-8-4007-2384-1/2026/04 CCS: Human-centered computing Human computer interaction (HCI) CCS: Computing methodologies Supervised learning by classification Aral de Moor, Yana Hrynevich, Hleb Badzeika, Vladyslav Furda, Marko Kojic, Artem Savelev, Kostadin Cvejoski, Darya Rovdo, Ekaterina Garanina Abstract . We introduce control models for LLM-powered code completion in JetBrains IDEs: ML classifiers which trigger inference and filter the generated suggestions to better align them with users and reduce unnecessary requests. To this end, we evaluate boosting- and transformer-based architectures on an offline dataset of real code completions with n = 98 n=98 users. We further evaluate the offline classification performance of our boosting-based approach on a range of syntactically diverse languages; and perform an A/B study in a production environment where they improve completion efficiency and quality metrics. With this study, we hope to demonstrate the potential in using auxiliary models for smarter in-IDE integration of LLM-driven features, highlight fruitful future directions, and open problems. Saving Inference Costs While Improving Completion Quality Metrics † † cc-license: by Figure 1 . Control Models Problem Setting: Completion Flow with Positive (Accept) and Negative (Reject) Targets. Control Models Problem Setting After an editor action, the trigger model is invoked. If the trigger model passes the event, a completion is generated, which is then passed to the filter model. Only when the filter model passes the completion, is it shown to the user. This provides our labels for the dataset, an event is positive if it is accepted by the user, and negative otherwise. 1. Introduction When it comes to code completion powered by Large Language Models (LLMs), suggestions with incorrect code, poor context-awareness, or bad timing can be distracting to the developer ( 20 ) . Furthermore, misalignment with their state of mind ( 1 ) or experience level ( 18 ) can be disruptive to their flow. At best, this results in wasteful LLM inference ( 32 ; 25 ; 16 ) , but often leads to counterproductive interactions ( 28 ; 18 ; 1 ; 24 ) . As LLMs are becoming more integrated into developer workflows, it is necessary to think beyond aligning content itself – and also consider how users interact with it. We introduce the term Control Models , which modify human-AI interactions with the goal of better aligning LLMs with developers’ flow . To this end, control models leverage explicit in-IDE information that is not available to the language model, namely in-IDE telemetry such as previous actions, typing speed and caret scope information, as well as the outputs of static analyses (e.g. ( 16 ; 6 ; 4 ; 19 ) ). We intentionally settle on a definition that does not mention a specific IDE feature, as we believe that the advancements in LLM use-cases come hand-in-hand with an increased potential to disrupt user flows; and expect control models to be generally relevant. In the context of this paper though, we consider the application of control models to inline “grey-text” completion ( 17 ; 21 ) . This involves limiting the “suggestion space” of inline completions to align better with developer workflows, using, specifically, smarter triggers for generation ( 6 ; 25 ; 16 ) , and smarter filters thereafter ( 16 ) . This is motivated by only 31% of inferences materialising into shown completions, and only 31% of those shown completions being accepted, as well as by user reports of distraction ( 18 ; 20 ) and unnecessary context-switches ( 1 ; 28 ) . See Figure 1 for a diagram of the problem setting. To summarise, our contributions are: • An offline analysis and online AB study of lightweight, boosting-based trigger and filter classifier models, demonstrating that they can save around 20% of inference requests while improving completion quality metrics in JetBrains IDEs. • Large-scale offline empirical evidence highlighting the potential of transformer models in this area, which successfully combine contextual code understanding with additional in-IDE feature modalities; but also the obstacles regarding latency, privacy, and integration we faced. • A discussion underscoring the most fruitful directions for future work on similar control models. This includes our development of evaluation metrics towards ones that measure the proclaimed productivity gains over a longer term, an exploration of alternative model architectures, and observations that highlight the need to design for user-personalisation. 2. Related Work 2.1. Productive Human-AI Interaction AI Assistance is commonplace in IDEs, powering widely used features such as code completion ( 17 ; 21 ) . Yet, its alignment with developer flow is often overlooked at first ( 20 ; 1 ) . The LLMs powering AI features have increasingly better understanding of the task at hand through scaled training ( 14 ) , cleverer context-collection strategies ( 22 ; 27 ) , and advancements in fine-tuning processes. This leads to increasingly autonomous and proactive uses such as next-edit suggestions ( 23 ) and agents ( 30 ; 13 ) . However, this focus on extending LLM-based functionality has inadvertently overshadowed a vital aspect of the user experience: the interaction dynamics between developers and their AI tools ( 24 ) . At the same time, macro-scale repository studies indicate rising duplication and code churn correlated with AI-assisted workflows ( 9 ) , highlighting potential maintainability risks that are orthogonal to raw output volume. Evaluations of experienced developers show misalignment between perceived (+20%) and actual productivity (-20%) ( 2 ) . A large-scale survey primarily finds AI in software development to be an “amplifier”, both of software delivery throughput (+5%) and of software delivery instability (+10%) ( 5 ) , underscoring the need to carefully calibrate AI-assisted tools to where they can be useful over the long term. 2.2. Inline Code Completion We limit the scope of this paper to LLM-driven code completion in JetBrains IDEs. Our IDEs support both local-inferenced completions which work fully offline, through an on-device small LM designed for the in-editor language ( 21 ) ; and cloud-inferenced completions through larger LMs that can handle groups of semantically and syntactically similar languages ( 17 ) . While this study is likely relevant to any similar tool, our focus is on the cloud-based code completion provided by JetBrains . Our cloud-based code completion has about a 30% accept-per-show rate, which is similar to GitHub Copilot’s 27.5% in 2022 ( 32 ) , indicating we are likely able to reduce two-thirds of generated completions without affecting the user experience drastically. Moreover, these shown completions constitute only one-third of completions that still match the code context (characters before the cursor) by the time they are generated. This implies we are theoretically able to save two-thirds of inference requests without affecting the user experience at all, and close to 90% of completions without affecting the number of accepted completions. There exist several studies investigating the useful and counter-productive interactions between developers and inline completion ( 1 ; 32 ; 15 ; 18 ; 28 ) . To summarise, 1 find that useful developer interactions are mainly bimodal:

Read the complete text