SCALAR: A Part-of-speech Tagger for Identifiers

Christian D. Newman 1 , Brandon Scholten 2 , Sophia Testa 2 , Joshua A. C. Behler 2 , Syreen Banabilah 2 , Michael L. Collard 3 , Michael J. Decker 4 , Mohamed Wiem Mkaouer 5 , Marcos Zampieri 6 , Eman Abdullah AlOmar 7 Reem Alsuhaibani 8 Anthony Peruma 9 Jonathan I. Maletic 2 · 2025

Read SCALAR: A Part-of-speech Tagger for Identifiers by Christian D. Newman 1 , Brandon Scholten 2 , Sophia Testa 2 , Joshua A. C. Behler 2 , Syreen Banabilah 2 , Michael L. Collard 3 , Michael J. Decker 4 , Mohamed Wiem Mkaouer 5 , Marcos Zampieri 6 , Eman Abdullah AlOmar 7 Reem Alsuhaibani 8 Anthony Peruma 9 Jonathan I. Maletic 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17038.txt.

Introduction

Christian Newman and a large team present SCALAR, a tool that tags the grammar of names in source code. Their twenty twenty-five paper deals with identifiers: function names, variable names, and the like. Such names are tiny phrases in a developer's own dialect, and standard grammar taggers read them poorly.

SCALAR is trained on a hand-built oracle of names and their grammar patterns, using a gradient boosting classifier. It maps each name to its part-of-speech sequence, the pattern a name follows, like verb plus noun. Tests show it beats both an older version of the tagger and a modern general tagger on this niche task. The code is on GitHub.

Catalog record

CREATORChristian D. Newman 1 , Brandon Scholten 2 , Sophia Testa 2 , Joshua A. C. Behler 2 , Syreen Banabilah 2 , Michael L. Collard 3 , Michael J. Decker 4 , Mohamed Wiem Mkaouer 5 , Marcos Zampieri 6 , Eman Abdullah AlOmar 7 Reem Alsuhaibani 8 Anthony Peruma 9 Jonathan I. Maletic 2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-17038

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SCALAR: A Part-of-speech Tagger for Identifiers Christian D. Newman 1 , Brandon Scholten 2 , Sophia Testa 2 , Joshua A. C. Behler 2 , Syreen Banabilah 2 , Michael L. Collard 3 , Michael J. Decker 4 , Mohamed Wiem Mkaouer 5 , Marcos Zampieri 6 , Eman Abdullah AlOmar 7 Reem Alsuhaibani 8 Anthony Peruma 9 Jonathan I. Maletic 2 Abstract The paper presents the Source Code Analysis and Lexical Annotation Runtime (SCALAR), a tool specialized for mapping (annotating) source code identifier names to their corresponding part-of-speech tag sequence (grammar pattern). SCALAR’s internal model is trained using scikit-learn’s GradientBoostingClassifier in conjunction with a manually-curated oracle of identifier names and their grammar patterns. This specializes the tagger to recognize the unique structure of the natural language used by developers to create all types of identifiers (e.g., function names, variable names etc.). SCALAR’s output is compared with a previous version of the tagger, as well as a modern off-the-shelf part-of-speech tagger to show how it improves upon other taggers’ output for annotating identifiers. The code is available on Github 1 1 1 https://github.com/SCANL/scanl_tagger I Introduction The identifiers developers create represent a significant amount of the information other developers must use to understand related code. Given that identifiers represent, on average, 70% of the characters in a code base [ 1 ] , and developers spend more time reading code than writing [ 2 , 3 ] , it is important for researchers to better understand of how identifiers convey information, and how they can be improved to increase developer reading efficiency. This problem is complicated by the fact that there are multiple comprehension styles [ 4 , 5 ] , and the influence of identifier naming at varying levels of experience is currently understudied, especially in education contexts [ 6 , 7 ] . Further, while there are many studies that correlate the influence of identifiers (and various related characteristics) on comprehension [ 8 , 9 , 10 , 11 ] , research has not found formal rules or unification among the outcomes of these studies due to the complexity of the problem, and the need for further research. Thus, research is needed to improve identifier naming practices. Many approaches try to improve identifiers through predicting words that should appear in a name [ 12 , 13 ] or analyzing/normalizing identifiers [ 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ] to understand how well they fit within a coding context, or make them easier to process. However, a significant challenge in identifier naming research lies in measuring the semantics of identifier names, then using that information to critique/generate better names. Measurement of identifier name semantics requires mapping between the terms and their meaning as an identifier (i.e., a sequence of terms). The ability to do this in a formal way allows us to recommend sequences of terms based on an understanding of the semantics they typically convey together. It also allows us to determine when the terms used in a sequence are inappropriate, since certain sequences are uncommon or even known to be anti-patterns. This is a challenge, as there are many ways to cluster identifiers using the terms used to construct them. Prior research shows that using grammar patterns to cluster/group identifiers with similar part-of-speech (PoS) sequences is an effective way to study how different types of identifiers convey meaning [ 24 , 25 , 26 ] , and has the advantage that every term that is part of a natural language can be associated with a PoS tag. In addition, PoS tags are a known and well-supported means to formally define the function of a term within a sequence. Thus, we can group identifiers by grammar patterns (i.e., their PoS sequence) to measure the information they convey, and what patterns are most commonly used to convey different types of information. This is in contrast, and potentially complementary, to other approaches that cluster raw terms, or use vector representations. Thus, we present SCALAR: A part-of-speech tagging approach specialized for source code identifiers. SCALAR is explicitly designed to support the generation of grammar pattern sequences to support future research and development of techniques that leverage grammar patterns. The goal of this paper is to show the effectiveness of SCALAR for generating grammar patterns. SCALAR can be used in the future to improve techniques to analyze, recommend, and critique identifiers, by generating grammar patterns that can be used to understand identifier name meanings [ 24 , 25 , 27 , 26 ] . SCALAR is still very much in development as we apply it in our research and modify it to assist. However, it is a working tool that will be useful for others in identifier-oriented work. TABLE I: Part-of-speech categories in dataset and supported by SCALAR Abbreviation Expanded Form Examples N noun stack, function, language DT determiner the, this, that, these, those, which CJ conjunction and, for, nor, but, or, yet, so P preposition behind, in front of, at, under, beside, above, beneath, despite NPL noun plural strings, identifiers, classes NM noun modifier ( noun adjunct , adjective) employee Name, token Parser, dynamic V verb run, execute, implement, develop VM verb modifier (adverb) quickly, safely, eventually PR pronoun she, he, her, him, it, we, us, they, them, I, me, you D digit 1, 2, 10, 4.12, 0xAF PRE preamble* Gimp, GLEW, GL, G II Related Work POSSE [ 19 ] and SWUM [ 28 ] , and SCANL tagger [ 29 ] are part-of-speech taggers created specifically to be run on software identifiers; they are trained to deal with the specialized context in which identifiers appear. Both POSSE and SWUM take advantage of static analysis to provide annotations. For example, they will look at the return type of a function to determine whether the word set is a noun or a verb. Additionally, they are both aware of common naming structures in identifier names. For example, methods are more likely to contain a verb in certain positions within their name (e.g., at the beginning) [ 19 , 28 ] . They leverage this information to help determine what POS to assign different words. Olney et al. [ 30 ] compared taggers for accuracy on 200+ identifiers, but only on Java method names. They found that SWUM and POSSE were the most accurate taggers for source code at the time of publication. Newman et al. [ 24 ] compared the same taggers but on a larger dataset (1,335 identifiers) and five identifier categories: function, class, attribute, parameter, and declaration statement. They found that SWUM was the most accurate overall, with an average accuracy around 59.4% at the identifier level. Later, Newman et al. created a new tagger and compared with SWUM, POSSE, and Stanford [ 31 ] , finding that their new tagger exceeded the others’ performance metrics on identifier names [ 29 ] . III Methodology The core of SCALAR is a GradientBoostingClassifier that is trained using the combination of two manually-curated data sets of identifiers and their corresponding grammar patterns. The first data set is called the General Grammar Dataset (GGD). The second dataset is called the

Read the complete text