Context-aware child-directed speech detection from long-form recordings
Théo, Tarek, Kaveri K, Alejandrina, Marvin · 2026
Read Context-aware child-directed speech detection from long-form recordings by Théo, Tarek, Kaveri K, Alejandrina, Marvin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01134.txt.
Introduction
Charlot and co-authors teach machines to hear when speech is aimed at a child in this 2026 paper. Long recordings of home life hold the data of language growth research. Sorting child directed speech from adult directed speech by hand does not scale. Past tools judge each utterance alone and were tested mostly on English.
The team fine tunes six self supervised models on recordings of 182 children, in many languages. Models pretrained on child centered audio beat those trained on adult speech by a wide mark. Context helps a lot: adding the speech around each utterance lifts the average F1 score by 13.8 points in absolute terms across the tests. A full pipeline test, from finding adult speech to classifying its target, still beats a rule based baseline. Scores drop under automatic segmentation, they note. The work opens large, multilingual studies of what children hear.
Catalog record
| CREATOR | Théo, Tarek, Kaveri K, Alejandrina, Marvin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.01134
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Context-aware child-directed speech detection from long-form recordings Théo, Tarek, Kaveri K, Alejandrina, Marvin Abstract Automatically distinguishing child-directed speech from adult-directed speech in long-form recordings is key to scalable analyses of children’s language environments. Existing approaches process utterances in isolation and have been evaluated primarily on English. We address these gaps along three dimensions. First, we fine-tune and evaluate six-self supervised models on a multilingual dataset of 182 children, showing that in-domain pre-training on child-centered recordings substantially outperforms models trained on adult speech. Second, we demonstrate that incorporating surrounding context substantially improves classification, with an absolute gain of 13.8% in average F1-score. Third, we evaluate our model in a realistic end-to-end pipeline, from adult speech detection to addressee classification, showing that performance drops under automatic segmentation but still consistently outperforms a rule-based baseline. Charlot Kunze Sheth Cristia Lavechin keywords child-directed speech detection, long-form recordings, addressee classification, language experience † † address: 1 LSCP, DEC, ENS, EHESS, CNRS, PSL University, France 2 Laboratoire d’Informatique et Systèmes, Université Aix-Marseille, CNRS, France † † email: [email protected] 1 Introduction † † footnotetext: Code available at: https://github.com/LAAC-LSCP/addressee Children’s environments are complex, and the language input they receive is no exception. Among the sources of this complexity is the distinction between child-directed speech (CDS), the register adults typically adopt when speaking to young children, and adult-directed speech (ADS). CDS is characterized by features such as higher pitch, stretched vowels, and exaggerated intonation, as in ”Look at the doggy!” , features hypothesized to scaffold language development [ 32 , 18 , 23 ] Indeed, the exaggerated pitch contours of CDS may help infants segment the speech stream and identify word boundaries [ 39 ] , its acoustic saliency has been shown to preferentially capture infants’ attention compared to ADS [ 12 ] , and the use of diminutives and simplified vocabulary may help children map words onto referents more easily [ 43 ] . Understanding the role of CDS in children’s language development is therefore of considerable interest, particularly given that both the amount and nature of CDS vary substantially across cultures and socioeconomic contexts [ 14 , 35 ] . Child-worn microphones that record audio throughout the day have become a powerful tool for capturing naturalistic language input. However, researchers typically rely on manual annotation to distinguish between CDS and ADS, which is time and labor intensive, limiting the scale at which children’s linguistic environments can be studied. Machine learning approaches could offer a scalable solution, enabling researchers to analyze large corpora that would otherwise be infeasible to annotate by hand. The task of automatically distinguishing CDS from adult-directed speech ADS has drawn the attention of the speech processing community. It has been proposed in one of the tracks of the Interspeech 2017 Computational Paralinguistics Challenge (ComParE) [ 34 ] . The challenge organizers established a baseline using a late fusion of three systems: an end-to-end convolutional neural network with long short-term memory trained on raw waveforms, hand-crafted acoustic features, and bag-of-audio-words features, all paired with linear support vector machines, achieving a UAR of 70.2 [ 34 ] . Despite several submissions [ 19 , 22 ] , no team was able to surpass this baseline, and no open-source classifier emerged from the challenge, limiting the task’s broader impact on the research community. More recently, Bang et al. (2022) trained an XGBoost classifier on LENA recordings to predict whether a 5-minute segment was predominantly CDS or ADS, achieving an AUC of 0.72 [ 3 ] . However, within such segments, CDS and ADS utterances may be interleaved, making it a coarser tool than utterance-level classification. Al Futaisi et al. (2023) tackled the task at the utterance level by training deep neural networks on hand-crafted acoustic features using multi-task and adversarial learning strategies [ 1 ] . Their best model achieved a UAR of 67.6% on the test set, falling short of the ComParE baseline of 70.2%. Despite these efforts, several limitations remain. First, most existing models have been trained predominantly on English data, representing only a narrow slice of the world’s linguistic and cultural diversity [ 20 ] , a particularly consequential limitation given that the acoustic and linguistic properties of CDS are known to vary substantially across cultures and languages [ 15 ] . Second, all existing approaches train on utterances in isolation, discarding surrouding conversational context that human annotators naturally rely on when making addressee judgments. This is compounded by the fact that utterances are particularly short, with average durations of 1.5 seconds for CDS and 1.7 seconds for ADS in the ComParE training set, leaving little acoustic information for the model to exploit [ 34 ] . One may rightfully hypothesize that surrounding context (such as the presence of child vocalizations) carries useful information for addressee classification, and that models trained on isolated utterances may therefore be at a disadvantage. Third, none of the existing systems have been evaluated in a realistic setting, where speaker diarization and utterance detection must first be performed automatically on raw audio, meaning their practical utility for analyzing daylong recordings remains untested. In this paper, we make three contributions. First, we benchmark multiple self-supervised models for CDS/ADS classification, including models classically trained on adult clean speech and two in-domain models trained on child-centered long-forms recordings, demonstrating that in-domain pretraing yields substantially better performance. Second, we show that providing the model with contextual audio windows improves classification, challenging the common practice of processing utterances in isolation and suggesting that context should be a key consideration in future model design. Third, we evaluate our approach in a fully automatic pipeline, where adult speech utterances are first detected from raw audio using the Voice Type Classifier 2.0 [ 9 ] , before being passed to the classifier, demonstrating feasibility in a realistic deployment scenario. 2 Methods We introduce the corpora used in this study. We then formalize the addressee classification problem before introducing the self-supervised models we considered, and the context-aware fine-tuning strategy we implemented. We present the baseline against which our best model is compared and the evaluation metric. We conclude this section by providing implementation details. 2.1 Dataset Corpora. We gathered multiple corpora across languages and sociocultural contexts. The resulting dataset comprises a total of 22 hours of audio across 6 languages and 182 children. Table 1: Dataset used to train and evaluate our addressee classification system. Corpora were obtained through scientific archives such as Homebank* [ 40 ] , the Language Archive** [ 36 ] , and CHILDES*** [ 28 ] or accessed through direct data sharing agreements with research groups. Summary statistics are presented in Table 1 . Dataset Language(s) Number of children Total adult speech dur. Lyon* [ 5 ] French 16 5h22m Cougar* [ 41 ] English 46 3h32m Bergelson* [ 4 ] English 20 2h13m Png* [ 13 ] Mostly Yeli Dnye 10 2h07m Lucid** [ 31 ] English 20 2h01m Warlaumont* [ 42 ] Mostly English 15 1h38m Soderstrom** [ 37 ] English 2 1h17m Tsimane [ 33 ] Tsimane 24 0h36m PhonSES [ 17 ] French 6 0h08m Thomas*** [ 27 ] English 1 0h07m Vanuatu [ 16 ] Bislama 1 0h01m Tseltal* [ 6 ] Mayan 10 1h37m Winnipeg* [ 38 ]