Large Language Model Data Generation for Enhanced Intent Recognition in German Speech
Theresa Pekarek Rosin, Burak Can Kaplan, Stefan Wermter · 2025
Read Large Language Model Data Generation for Enhanced Intent Recognition in German Speech by Theresa Pekarek Rosin, Burak Can Kaplan, Stefan Wermter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06277.txt.
Introduction
Theresa Rosin, Burak Kaplan, and Stefan Wermter boost German speech intent recognition in a 2025 paper. Assistants must grasp what a spoken command intends, but most work targets short English commands. Their focus is elderly German speakers. They fine-tuned a speech recognizer on that group's speech and trained language models on synthetic text made by three large language models.
Synthetic data clearly raised accuracy and robustness to new speaking styles and unseen words. A smaller German-focused model produced better training data than a much larger general one. Tests used synthetic speech across datasets to stress the system. The authors document their process in detail for reproducibility.
Catalog record
| CREATOR | Theresa Pekarek Rosin, Burak Can Kaplan, Stefan Wermter |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-06277
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Large Language Model Data Generation for Enhanced Intent Recognition in German Speech Theresa Pekarek Rosin, Burak Can Kaplan, Stefan Wermter Abstract Intent recognition (IR) for speech commands is essential for artificial intelligence (AI) assistant systems; however, most existing approaches are limited to short commands and are predominantly developed for English. This paper addresses these limitations by focusing on IR from speech by elderly German speakers. We propose a novel approach that combines an adapted Whisper ASR model, fine-tuned on elderly German speech (SVC-de), with Transformer-based language models trained on synthetic text datasets generated by three well-known large language models (LLMs): LeoLM, Llama3, and ChatGPT. To evaluate the robustness of our approach, we generate synthetic speech with a text-to-speech model and conduct extensive cross-dataset testing. Our results show that synthetic LLM-generated data significantly boosts classification performance and robustness to different speaking styles and unseen vocabulary. Notably, we find that LeoLM , a smaller, domain-specific 13B LLM, surpasses the much larger ChatGPT (175B) in dataset quality for German intent recognition. Our approach demonstrates that generative AI can effectively bridge data gaps in low-resource domains. We provide detailed documentation of our data generation and training process to ensure transparency and reproducibility. 1 Introduction Speech command recognition is essential for natural interaction with artificial agents in everyday life, especially for elderly or handicapped users ( Fronemann et al. 2021 ) . Commercial solutions have seen vast improvements and increased usage over the last few years. However, their cloud-based models not only struggle with speech from user groups that would benefit from a voice assistant the most ( Moro-Velazquez et al. 2019 ; Ngueajio and Washington 2022 ) , but they also introduce privacy issues because the processing of speech input usually does not happen locally. Additionally, traditional speech command recognition is usually restricted to short phrases and words ( Warden 2018 ) , which requires the user to adapt to the system and change their way of speaking. That is usually not as intuitive for older people who might not have the same experience with recent technologies as younger users ( Pekarek Rosin et al. 2025 ) . Therefore, we argue that to create a more natural interaction with speech-based assistant systems, we need to move away from one-word command recognition to unconstrained utterance-based intent recognition. Figure 1: An overview of our model setup (left) and a traditional baseline (right). The speech is transcribed with a Whisper ASR model, the transcript is then classified with a transformer-based LM (BERT, DistilBERT, Electra) trained on a generated text dataset and a one-layer classification network (CN1). For the baseline, we use the output of the Whisper encoder to classify the intent with a two-layer classification network (CN2). However, the lack of large-scale datasets for languages besides English makes it difficult to implement such a change by retraining open-source speech models for intent recognition on different domains (e.g., elderly voices). Collecting new speech data on a larger scale for any speech recognition or classification task is not only an incredible effort for researchers and participants but also ultimately inefficient, since the process would need to be repeated every time new functionalities become available. This issue is amplified by the amount of training data that Transformer-based, state-of-the-art models require to be trained in their entirety. We suggest two potential solutions to these issues: 1) through the use of layer-specific fine-tuning ( Shor et al. 2019 ; Pekarek Rosin and Wermter 2023 ) pretrained foundation models can be adapted to other domains with small amounts of speech data while preserving existing knowledge, 2) through the generation of additional domain-focused data with large language models (LLMs), we can increase the generalization abilities of the model to different linguistic patterns. We combine a state-of-the-art automatic speech recognition (ASR) model, Whisper ( Radford et al. 2023 ) , with three transformer-based pretrained language models, BERT ( Devlin et al. 2019 ) , DistilBERT ( Sanh et al. 2019 ) , and Electra ( Clark et al. 2020 ) for German speech intent recognition. We utilize three LLMs to generate additional training data for these language models, and we synthesize speech samples based on this data with a text-to-speech model to evaluate our approach. 2 Related Work Speech command recognition typically involves accurately identifying words or short phrases as commands linked to functionalities in an underlying system. This classification can be performed on either the audio feature representations or the transcript produced by the ASR model. In contrast, intent recognition requires a larger context, such as complete sentences, to discern user intent. For English-language tasks, the Speech Command Dataset ( Warden 2018 ) is a commonly used resource for training and evaluating models in speech command recognition or keyword spotting. Diverse strategies have been proposed for speech command recognition, with a recent approach by Sadovsky et al. 2023 exploring the use of spiking neural networks for this task, and achieving a maximum accuracy of 72% on the Speech Command Dataset. Other baselines mentioned in their paper reach 79% accuracy using Gated Recurrent Units. Berg et al. 2021 propose the Keyword Transformer model. They train their model on the Speech Command Dataset for approximately 100 episodes and achieve between 97.49% and 98.56% accuracy using knowledge distillation with only minimal improvements over the baseline. In low-resource settings, Kumar Nayak et al. 2023 explore speech command recognition in the Kui language using a small dataset of 7,090 utterances covering just seven words. Their convolutional neural network (CNN) baseline was the only model to exceed 90% accuracy after 300 training episodes, highlighting the challenges of limited data. Similarly, Hernández et al. 2024 investigate intent recognition in Spanish and Nahuatl by training models on a manually collected dataset of 383 natural language navigation commands. Their results show that transformer-based models outperform traditional baselines, especially for longer and more complex utterances. Despite these advances, most approaches often rely on extensive training, large-scale datasets, or costly real-world data collection. Notably, the lack of recent work examining this research topic for German speech suggests that the absence of datasets deters research in that direction. LLMs have emerged as powerful tools for synthetic dataset generation ( Kaplan et al. 2025 ) , enabling the creation of task-specific training data and offering a solution to data scarcity. Additionally, their capabilities in common-sense reasoning Li et al. 2022 and their strong performance in natural language understanding make them well-suited for tasks such as intent recognition. Transformer-based language models have shown value in both data augmentation for intent classification ( Kumar et al. 2020 ) and in intent classification itself ( Chen et al. 2019 ) . Additionally, the use of more powerful LLMs for intent recognition has increased significantly ( Dighe et al. 2024 ; Dzeparoska et al. 2024 ; Wang et al. 2024 ) , and LLMs have been widely studied for their ability to assist people through multimodal applications with speech ( Wagner et al. 2024 ; Padmanabha et al. 2024 ) . However, the majority of the work has been conducted in English, leaving considerable room for exploration in other languages. While some recent work explores German-language applications of LLMs ( Irrgang et al. 2024 ; Volk et al. 2024 ) ,