TalkTag: Fine-Grained Morphosyntactic Error Annotation for Transcribed Speech

Shamira Venturini, Oliver Hennhöfer, Steffen Kinkel, Jannik Strötgen · 2026

Read TalkTag: Fine-Grained Morphosyntactic Error Annotation for Transcribed Speech by Shamira Venturini, Oliver Hennhöfer, Steffen Kinkel, Jannik Strötgen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01820.txt.

Introduction

Venturini and co-authors build a tool to mark speech errors in fine detail. Such marks aid study of child speech and clinic work. Hand marking is slow and needs expert skill. It is hard to scale to large sets of speech records. Their TalkTag tool uses a tuned chat model for the task.

TalkTag marks errors in the CHAT style used by experts. It was built with very scarce training data. Child story speech served as the base case. Tests show the marks are quite precise, the authors report. The tool also flags spots where speech is truly unclear. It offers a scalable stand-in for hand marking.

Catalog record

CREATORShamira Venturini, Oliver Hennhöfer, Steffen Kinkel, Jannik Strötgen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.01820

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TalkTag: Fine-Grained Morphosyntactic Error Annotation for Transcribed Speech Shamira Venturini, Oliver Hennhöfer, Steffen Kinkel, Jannik Strötgen Abstract Fine-grained morphosyntactic error annotation is important in clinical and developmental language research, yet it is labour-intensive, expert-dependent, and difficult to scale. We present TalkTag , an LLM-based lightweight tool fine-tuned to automate CHAT-style error annotation in spoken-language transcripts. Developed under conditions of extreme data scarcity using children’s narrative data, the system shows the feasibility of linguistic analysis in low-resource settings. Our evaluation demonstrates that TalkTag produces encouragingly precise annotation while effectively identifying instances where linguistic ambiguity makes automated tagging genuinely complex. In summary, with TalkTag , we provide a scalable alternative to manual error annotation and practically viable support for morphosyntactic error annotation. 1 Introduction Language Sample Analysis has become an increasingly important method in clinical linguistics and developmental psycholinguistics ( 22 ) . Drawing on naturalistic spoken interaction data, it supports the study of language development and impairment in context. During the past decades, TalkBank 1 1 1 http://talkbank.org ( 21 ) has substantially advanced the infrastructure for this line of research through large open spoken corpora, the CHAT transcription format, and the CLAN analysis tools ( 23 ) . CHAT provides a standard way to represent spoken-language transcripts, while CLAN (Computerized Language ANalysis) provides analysis programs for CHAT-formatted files. In this setting, morphosyntactic error codes are written inline on the transcript line, immediately after the form they describe, so that the annotation preserves both the child’s production and the analyst’s interpretation of the intended target. Although TalkBank includes tools for some aspects of automatic transcription and morphosyntactic annotation, to the best of our knowledge, it currently does not support automatic annotation of morphosyntactic errors. However, fine-grained morphosyntactic error annotation is important because it provides evidence of grammatical development, impairment, and variation in typically developing ( 30 ) , as well as populations such as children with developmental language disorders ( 19 ; 8 ; 40 ) , Down syndrome ( 46 ; 17 ; 38 ) , deaf and hard-of-hearing children with cochlear implant ( 2 ; 12 ) , and autism spectrum disorder ( 16 ) . Manually annotating these phenomena remains slow, labour-intensive, and dependent on expert knowledge. Unlike ordinary morphosyntactic tagging, this task often requires identifying a structured divergence between an erroneous production and an intended target, or recovering morphology that is absent from the surface string but obligatory in context. It therefore goes beyond simple sequence labelling, requiring structural linguistic reasoning rather than surface-level pattern matching. Large Language Models (LLMs) offer a promising solution for this setup by integrating contextual modelling with schema-constrained generation 7 . Unlike assigning isolated tags token by token, LLMs can, in principle, evaluate an entire utterance holistically. This allows for the generation of well-formed inline annotations that capture syntactic dependencies and the underlying structural nature of the error. At the same time, annotated clinical and developmental language data are often scarce, access-restricted, and difficult to use for large-scale model development ( 1 ; 10 ) . We therefore introduce TalkTag 2 2 2 http://github.com/OliverHennhoefer/talk-tag , a tool for fine-grained morphosyntactic error annotation that follows the CHAT guidelines for word-level error coding. The tool was developed under extremely low-resource conditions, with very limited annotated data 15 . To address this, we employed synthetic data augmentation to fine-tune a lightweight, open-weight LLM, effectively expanding the model’s exposure to rare error patterns. In this initial prototype, we focus on children’s narrative data, a domain selected not only for its availability but for its high density of developmentally salient morphosyntactic phenomena. These narratives provide a benchmark for evaluating fine-grained annotation capabilities in a complex, real-world linguistic context. The main contribution of this paper is a lightweight tool for fine-grained morphosyntactic error annotation in clinical and developmental language research, together with its accompanying Python package, TalkTag . More specifically, we formulate CHAT morphosyntactic error coding as a constrained structured-generation task, adapt a small instruction-tuned model to this setting under severe data sparsity, and evaluate the resulting system using automatic scoring, blinded post-hoc adjudication, and human review on unseen corpus material. The remainder of the paper is structured as follows. Section 2 defines the target annotation scheme and the subset of CHAT morphosyntactic error labels modelled in this study. Section 3 then situates the work in relation to prior research on clinical language annotation, error coding, and automatic linguistic analysis. Section 4 describes the model, training setup, data, and evaluation design. Finally, Section 5 reports the automatic and human-reviewed results. Section 6 discusses the main linguistic error patterns, the implications of the findings for annotation practice, and the limitations of the current system. INPUT CHAT input Transcript utterance plus error scheme ADAPT TalkTag model CHAT-token components and LoRA fine-tuning GENERATE Candidate tags Preserve utterance and insert inline tags USE Review Automatic scoring plus human review Figure 1: The TalkTag Workflow Pipeline. 2 The Annotation Language Within the TalkBank ecosystem, which provides infrastructure for the transcription and analysis of spoken interaction data, the MOR and GRASP programs ( 24 ) support automatic morphosyntactic analysis of CHAT transcripts. MOR is a morphological analyser that assigns lexical and grammatical information to each token, producing the %mor tier with lemma, part-of-speech, and inflectional information. Building on this output, GRASP derives syntactic structure by assigning grammatical relations and dependency-based representations across the utterance on the %gra tier. Together, these tools make it possible to move from raw transcript text to a linguistically enriched representation of children’s speech. However, their purpose is to recover morphological and syntactic structure rather than to identify, classify, or encode morphosyntactic errors explicitly. The CHAT Transcription Guidelines provide a general system for marking word-level errors ( 23 ; 25 ) . At the level of string form, the annotation tags have a simple and regular surface structure: first, they consist of a fixed bracketed frame following the relevant error, marked with an * . Next, a flat sequence of colon-separated fields indicates i) the error domain (phonological, semantic, neologistic, dysfluency, and morphological), ii) the error pattern (e.g., missing, superfluous, over-regularised, double-marked morphemes, unknown/known target, etc.) and iii) the morpheme or part-of-speech involved (e.g., past tense, perfective, plural, or third-person singular agreement morphemes; pronouns, prepositions, determiners as parts-of-speech). When the intended target is known, it can also be provided in brackets next to the error using the format [: target] . This is used when CLAN’s MOR morphological analyser should analyse the target form instead of the produced one, whereas [:: target] 3 3 3 Since the time of this study, TalkBank has updated the CHAT manual, replacing the [:: target] syntax with [= target] . While the model described here was trained on the earlier convention, the associated Python package includes post-processing to ensure compliance with the latest standard and offers options to toggle the visibility of reconstructions depending on user preference. can be used to preserve analysis of the produced real-word form while still recording the intended target. The surface syntax of the labels is therefore relatively simple, yet their assignment is highly challenging: it often depends on contextual linguistic interpretation and, in many cases, on reconstructing an intended target form. In this work, we focus on treating

Read the complete text