Parsing the Switch: LLM-Based UD Annotation for Complex Code-Switched and Low-Resource Languages
Olga Kellert, Nemika Tyagi, Muhammad Imran Nelvin Licona-Guevara, Carlos Gómez-Rodríguez · 2025
Read Parsing the Switch: LLM-Based UD Annotation for Complex Code-Switched and Low-Resource Languages by Olga Kellert, Nemika Tyagi, Muhammad Imran Nelvin Licona-Guevara, Carlos Gómez-Rodríguez (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07274.txt.
Introduction
Olga Kellert and co-authors present a parser for code-switched text in low-resource languages, in a research paper from 2025. Code-switching mixes languages inside one text, which breaks parsers trained on single-language data. Labeled data for such mixed text are scarce, above all for pairs that include a minority language.
Their pipeline uses large language models with guided prompts to draft Universal Dependencies annotations. Expert review then corrects the drafts. Two data sets are released, including the first parsed corpus for Spanish and Guaraní. Analysis of switch points follows. Scores after expert revision beat prior baselines and standard multilingual parsers by a clear margin.
Catalog record
| CREATOR | Olga Kellert, Nemika Tyagi, Muhammad Imran Nelvin Licona-Guevara, Carlos Gómez-Rodríguez |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-07274
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Parsing the Switch: LLM-Based UD Annotation for Complex Code-Switched and Low-Resource Languages Olga Kellert, Nemika Tyagi, Muhammad Imran Nelvin Licona-Guevara, Carlos Gómez-Rodríguez Abstract Code-switching presents a complex challenge for syntactic analysis, especially in low-resource language settings where annotated data is scarce. While recent work has explored the use of large language models (LLMs) for sequence-level tagging, few approaches systematically investigate how well these models capture syntactic structure in code-switched contexts. Moreover, existing parsers trained on monolingual treebanks often fail to generalize to multilingual and mixed-language input. To address this gap, we introduce the BiLingua Parser , an LLM-based annotation pipeline designed to produce Universal Dependencies (UD) annotations for code-switched text. First, we develop a prompt-based framework for Spanish-English and Spanish-Guaraní data, combining few-shot LLM prompting with expert review. Second, we release two annotated datasets, including the first Spanish-Guaraní UD-parsed corpus. Third, we conduct a detailed syntactic analysis of switch points across language pairs and communicative contexts. Experimental results show that BiLingua Parser achieves up to 95.29% LAS after expert revision, significantly outperforming prior baselines and multilingual parsers. These results show that LLMs, when carefully guided, can serve as practical tools for bootstrapping syntactic resources in under-resourced, code-switched environments 1 1 1 Data and source code are available at https://github.com/N3mika/ParsingProject . . 1 Introduction Code-switching (CSW) is a widespread linguistic phenomenon observed in multilingual communities around the world. Despite its prevalence in spoken and informal digital communication, it remains a complex challenge for natural language processing (NLP), particularly for syntactic parsing. One of the central issues is that most state-of-the-art parsing models are trained on monolingual treebanks and thus lack robustness when applied to mixed-language data ( Özateş et al. 2022 ) . Previous works of Özateş et al. 2022 ; Rijhwani et al. 2017 ; Bhat et al. 2018 took an important step toward addressing this gap by proposing, for instance, a semi-supervised dependency parsing framework that augments training with auxiliary sequence labeling tasks ( Özateş et al. 2022 ) . Their model improved parsing accuracy on Turkish-German spoken corpus by learning better representations of syntactic structure in a multilingual setting. However, even with such enhancements, existing models often rely on large amounts of annotated data, which is particularly limiting for under-resourced language pairs. Figure 1: Comparison of dependency relation predictions (DepREL) for a Spanish-English CSW sentence across three parsers. The English-only and Spanish-only models misassign key relations due to monolingual bias. In contrast, the BiLingua Parser correctly analyzes the full structure across the language boundary. Figure 2: Overview of BiLingua Parser pipeline for Spanish-English code-switching. Left: Conversation transcripts from the Miami-Bilingual Corpus are processed through token-level segmentation, language ID tagging, and filtering of intra-sentential code-switches. Center: The BiLingua Parser assigns UD tags to CSW sentences, handling contractions, repetitions, ellipsis, and clausal structure. Right: The resulting annotated dataset is reviewed by linguistic experts and enables downstream tasks such as POS/DEPREL analysis and extension to low-resource settings, including Spanish-Guaraní. Motivated by this lack of resources, we introduce BiLingua Parser, a bilingual syntactic parser based on large language models (LLMs), specifically the GPT-4.1 model, to generate syntactically annotated CSW datasets. Figure 1 illustrates how current monolingual parsers perform significantly worse than BiLingua Parser on a widely studied CSW language pair. Next, we tackle this issue for two language pairs in particular: Spanish-English (a relatively well-resourced code-switching language pair) and Spanish-Guarani (a low-resource language pair for which most linguistic tools are largely unavailable) Chiruzzo et al. 2023 . To our knowledge, these are the first datasets for Spanish-English and Spanish–Guaraní code-switching with UD-based syntactic annotations reviewed by native speakers. The entire pipeline for creating and using BiLingua Parser is shown in Figure 2 . In addition to developing BiLingua parser, we also examine the limitations of current syntactic parsing evaluation metrics. To this end, we introduce additional methods for assessing the performance of our parser with the help of linguistic experts. Moreover, the annotated datasets we create also support a wide range of linguistic analyses involving bilingual speakers, including understanding of fine-grained switch-point behavior. While most previous structural studies in NLP on code-switching have focused on part-of-speech (POS) tags Martínez 2020 ; Rijhwani and Solorio 2016 ; Solorio and Liu 2008 , our analysis using dependency parsing shows that syntactic subjects (nsubj) are among the most frequent switch points in both language pairs and that Spanish-Guarani code-switching exhibits higher variation in switch points. This finding underscores the value of dependency parsing for analyzing switch points across languages. The main contributions of our work are as follows: • We introduce a method for generating UD-style syntactic annotations using LLM-based prompting, and compare it against baselines from previous work on dependency parsing of code-switched texts, as well as a parser trained on a synthetic combination of monolingual treebanks. • We release two code-switched datasets, with POS and dependency annotations, reviewed by native speakers, including a new resource for Spanish- Guaraní. • We conduct a linguistic case study of common syntactic structures at code-switch boundaries, revealing cross-linguistic switching patterns. Overall, we believe our findings and released resources will support future work in both computational modeling and linguistic analysis of code-switching, especially for under-resourced and typologically diverse language pairs. 2 Related Work Parsing code-switched text is considerably more challenging than monolingual parsing due to structural variability, mixed grammar rules, and limited annotated corpora. Prior work ( Rijhwani et al. 2017 ; Solorio and Liu 2008 ; Solorio et al. 2014 ; Özateş et al. 2022 ) has demonstrated that models trained exclusively on monolingual data perform poorly on CSW without specific adaptation. Özateş et al. 2022 addressed this gap by proposing a semi-supervised parsing framework that incorporates auxiliary sequence labeling tasks. Their approach improved parsing performance on Turkish-German CSW with labeled attachment scores (LAS) reaching up to 73%. Similarly, Bhat et al. 2018 and Rijhwani et al. 2017 reported LAS scores in the 70–72% range for Hindi-English data using adapted models (Table 1 ). Study Language Pair POS Accuracy LAS (Parsing) Solorio et al. 2014 Spanish-English 85.1% – Solorio et al. 2014 Hindi-English 83.3% – Rijhwani et al. 2017 Spanish-English 80.0% – Özateş et al. 2022 Hindi-English – 71.93% Özateş et al. 2022 Turkish-German – 73.0% Bhat et al. 2018 Hindi-English – 71.03% Table 1: POS tagging and dependency parsing performance on code-switched datasets in prior work. Most prior research focuses on part-of-speech tagging rather than full syntactic parsing, and available resources remain limited to a few language pairs. A draft UD treebank exists for Spanish-English code-switching, but it is not publicly released, further illustrating the scarcity of syntactically annotated CSW data. Efforts to increase parsing speed, such as recasting dependency parsing as a sequence labeling task ( Strzyz et al. 2019 ; Roca et al. 2023 ) , have improved runtime, but still depend heavily on monolingual training data. Our work builds on these foundations but shifts toward LLM-based annotation. Large language models like OpenAI GPT can be prompted with examples and linguistic rules to produce annotations without requiring extensive supervised training. We apply this technique to generate and evaluate new CSW datasets, including one for Spanish-Guaraní, thus expanding the reach of syntactic tools to underserved language communities. 3 Dataset and Experiments Datasets Statistics Spanish-English Spanish-Guaraní