NeoN : A Tool for Automated Detection, Linguistic and LLM - Driven Analysis of Neologisms in Polish

Aleksandra Tomaszewska, Dariusz Czerski, Bartosz Żuk, Maciej Ogrodniczuk · 2025

Read NeoN : A Tool for Automated Detection, Linguistic and LLM - Driven Analysis of Neologisms in Polish by Aleksandra Tomaszewska, Dariusz Czerski, Bartosz Żuk, Maciej Ogrodniczuk (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15426.txt.

Introduction

Aleksandra Tomaszewska and three co-authors present NeoN, a tool that finds and analyzes brand-new words in Polish. The tool appeared in twenty twenty-five. Dictionaries update slowly, and hand-checking candidate words eats researcher time.

NeoN layers several filters in a pipeline. It compares daily news feeds against reference corpora to spot fresh candidates. Polish-specific rules clean the list, and a language model filter sharpens it further. The system merges spelling variants and counts how often each form appears. Researchers review candidates in a visual interface. A built-in module then drafts definitions and tags each word by field and tone. Tests show high accuracy with far less manual effort than older methods.

Catalog record

CREATORAleksandra Tomaszewska, Dariusz Czerski, Bartosz Żuk, Maciej Ogrodniczuk
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-15426

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NeoN : A Tool for Automated Detection, Linguistic and LLM - Driven Analysis of Neologisms in Polish Aleksandra Tomaszewska, Dariusz Czerski, Bartosz Żuk, Maciej Ogrodniczuk Abstract We introduce NeoN , a tool for detecting and analyzing Polish neologisms. Unlike traditional dictionary-based methods requiring extensive manual review, NeoN combines reference corpora, Polish-specific linguistic filters, an LLM-driven precision-boosting filter, and daily RSS monitoring in a multi-layered pipeline. The system uses context-aware lemmatization, frequency analysis, and orthographic normalization to extract candidate neologisms while consolidating inflectional variants. Researchers can verify candidates through an intuitive interface with visualizations and filtering controls. An integrated LLM module automatically generates definitions and categorizes neologisms by domain and sentiment. Evaluations show NeoN maintains high accuracy while significantly reducing manual effort, providing an accessible solution for tracking lexical innovation in Polish. 1 1 1 The prompt templates and a number of NeoN interface screenshots are available at https://drive.google.com/file/d/1THipys62nlU7panPnIVdAUKzIk0QGlUx/view 1 Introduction Neologism detection in Polish remains challenging due to the heavy reliance on dictionary-based methods and manual curation. Traditional approaches, while informative, are time-consuming and prone to human bias [ 11 , 5 , 41 ] . Semi-automatic systems like Neoveille [ 3 ] and NeoCrawler [ 11 ] provide interactive interfaces for candidate review; however, their dependence on static lexicons and basic filtering mechanisms limits their ability to capture the full spectrum of emerging lexical phenomena in Polish. In addition, although recent advances in Large Language Models (LLMs) have transformed many areas of natural language processing, no existing tool has yet used LLMs for the semantic categorization and analysis of new words in Polish. In this paper, we present NeoN – an automated tool designed for Polish neologism detection, monitoring, and analysis. Figure 1: NeoN : overview of the monitoring interface. Rather than relying solely on dictionary lookups, NeoN continuously processes RSS feeds through a multi-layered filtering pipeline. It uses four high-quality Polish corpora: the National Corpus of Polish [ 7 ] , the Corpus of Contemporary Polish [ 4 ] , Polish Wikipedia dump, and a web corpus NEKST [ 19 ] . In the tool, we focus exclusively on neologisms that have entered usage after 2020 to monitor ongoing changes in the language rather than older forms. Custom filters tailored for Polish help extract candidate neologisms and consolidate variants through frequency analysis, structural constraints, context-aware lemmatization, and the final LLM-driven precision-boosting filter. NeoN offers a minimalistic interface accessible to users without programming expertise, displaying candidates in context with visualizations and metrics. Its integrated LLM module automatically generates definitions and categorizes neologisms by domain and sentiment, reducing manual annotation while facilitating detection and preliminary analysis. By combining corpus-based filtering with LLM-driven analysis, NeoN provides a scalable framework for tracking lexical innovation in Polish. 2 Related Work Earlier studies, such as those by Paryzek [ 41 ] , employed discriminant-based approaches to identify textual markers for flagging potential neologisms. In the Polish context, the Dictionary of Polish Neologisms, developed by the Language Observatory of the University of Warsaw [ 5 ] , serves as a recognized but manually curated resource; its reliance on diagnostic dictionaries and reference corpora underscores both its utility and limitations. Over the past two decades, semi-automated systems like NeoCrawler [ 11 ] and Neoveille [ 3 ] have emerged, using dictionaries and rule-based filters to extract new words from sources such as websites, blogs, and press releases. Although these tools reduce manual workload, they still require expert oversight to eliminate non-neologisms, errors, and non-lexical noise, as well as to incorporate metadata (e.g., definitions and categories). Moreover, their candidate lists are typically generated through dictionary comparisons rather than by capturing neologisms as they naturally emerge, and their architectures are seldom tailored to the nuances of the Polish language. Simultaneously, statistical and machine-learning approaches have advanced the field. For instance, Falk et al. [ 31 ] proposed a framework integrating form-related, morphological, and thematic features to detect neologisms in French newspapers, while McCrae [ 27 ] expanded on this by using pre-trained language models to differentiate compositional from non-compositional multiword expressions, thereby enhancing detection precision. More recent efforts have incorporated LLMs for additional tasks, including definition generation and translation [ 6 , 10 ] , as well as unsupervised techniques that normalize variant forms via embedding-space mapping [ 28 ] . The dynamic nature of language on social media has also prompted research into network-based methods. Yin and Cheng [ 37 ] applied overlapping community detection on Weibo data to track the propagation of neologisms, and Wang and Wu [ 34 ] adopted similar approaches to extract new lexical items in Chinese texts. Studies of French and Italian tweets by Tarrade et al. [ 22 ] and Spina et al. [ 21 ] , respectively, have further demonstrated the effectiveness of these methods in capturing both the morphological evolution and semantic shifts of neologisms over time. Character-based pattern mining offers an alternative methodology, as exemplified by Lejeune and Cartier [ 35 ] , who combined unsupervised data mining with supervised learning at the subword level to detect neologisms in multilingual datasets. Another nascent line of research is whether large language models can substitute for human participants in forced elicitation tasks. One study [ 42 ] replicated two experiments using GPT4o-mini, whose zero-shot performance exceeded that of human informants and was further enhanced through Chain-of-Thought prompting, albeit in a limited-scale study. 3 Functionalities Our system processes daily RSS feed data through a unified pipeline integrating candidate extraction, variant grouping, and multi-layered filtering mechanism for removing noise candidates. All NeoN functionalities are integrated into an intuitive web-based interface allowing users to customize filter settings, review, adjust candidate lists, and export the resulting data in CSV format. Users can easily generate definitions and categories on demand using real-life usage examples extracted through our multi-layered pipeline. A screenshot showcasing the NeoN interface can be seen in Figure 1 . Subsection 3.1 details the various filters applied during the selection of potential neologism candidates. In Subsection 3.2 , we describe the methods we use to address spelling variations and typographical errors. Additionally, given Polish’s rich inflectional morphology, we consolidate various forms of the same lexical item through a dedicated lemmatization module. After testing several tools, we selected the best-performing Hydra [ 12 ] lemmatizer. Subsections 3.3 and 3.4 describe experiments on definition generation and multidimensional categorization using Llama-3.3-70B-Instruct and DeepSeek-R1 models. Ultimately, we integrated Llama-70B into NeoN as it performed comparably on definition generation and better on sentiment and domain categorization while being smaller and more cost-effective. 3.1 Form Filtering In neologism detection, various filters are applied to determine the likelihood of a word being new or non-standard. The key filters in NeoN , categorized according to their respective purposes, are: (1) Frequency and occurrence : Document frequency, Term frequency, Unique domain frequency, Domain distribution; (2) Structural constraints : Word length constraints, Invalid character check, Presence of digits, Triple repeated characters, (3) Lexical validation : Common Polish corpus check, English loanword detection; (4) Spelling and typographical errors : Polish word matching, Edit distance with diacritics, Adjacent character swap detection; (5) Contextual analysis : English context detection, Name Entity heuristic, Capitalization pattern; (6) Other : Compound word detection, Filtering using a few-shot LLM prompt.

Read the complete text