Elementary, My Dear Watson: Non-Invasive Neural Keyword Spotting in the LibriBrain Dataset

Gereon Elvers, Gilad Landau, Oiwi Parker Jones · 2025

Read Elementary, My Dear Watson: Non-Invasive Neural Keyword Spotting in the LibriBrain Dataset by Gereon Elvers, Gilad Landau, Oiwi Parker Jones (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21038.txt.

Introduction

Gereon Elvers leads two co-authors on non invasive neural keyword spotting in the LibriBrain data set, a brain computer interface task pitched between basic detection and full brain to text. Keyword spotting is practical and privacy aware, since only chosen words trigger action rather than open decoding.

The fifty two hour within subject corpus gains standardized splits for reproducible benchmarking, with an evaluation protocol built for extreme class imbalance. Precision recall area serves as the main score, suited to rare keyword events. Solid keyword results would mark a step toward usable thought driven controls that respect user privacy by design.

Catalog record

CREATORGereon Elvers, Gilad Landau, Oiwi Parker Jones
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21038

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Elementary, My Dear Watson: Non-Invasive Neural Keyword Spotting in the LibriBrain Dataset Gereon Elvers, Gilad Landau, Oiwi Parker Jones Abstract Non-invasive brain-computer interfaces (BCIs) are beginning to benefit from large, public benchmarks. However, current benchmarks target relatively simple, foundational tasks like Speech Detection and Phoneme Classification, while application-ready results on tasks like Brain-to-Text remain elusive. We propose Keyword Spotting (KWS) as a practically applicable, privacy-aware intermediate task. Using the deep 52-hour, within-subject LibriBrain corpus, we provide standardized train/validation/test splits for reproducible benchmarking, and adopt an evaluation protocol tailored to extreme class imbalance. Concretely, we use area under the precision-recall curve (AUPRC) as a robust evaluation metric, complemented by false alarms per hour (FA/h) at fixed recall to capture user-facing trade-offs. To simplify deployment and further experimentation within the research community, we are releasing an updated version of the pnpl library with word-level dataloaders and Colab-ready tutorials. As an initial reference model, we present a compact 1-D Conv/ResNet baseline with focal loss and top- k k pooling that is trainable on a single consumer-class GPU. The reference model achieves ∼ \sim 13 × 13\times the permutation-baseline AUPRC on held-out sessions, demonstrating the viability of the task. Exploratory analyses reveal: (i) predictable within-subject scaling—performance improves log-linearly with more training hours—and (ii) the existence of word-level factors (frequency and duration) that systematically modulate detectability. \workshoptitle Data on the Brain & Mind Figure 1: KWS task setup. The participant listens to a Sherlock Holmes audiobook while the model spots a chosen keyword (e.g., “Watson”) from MEG signals. 1 Introduction The arrival of large and readily available datasets has begun to supply non-invasive brain-computer-interface (BCI) research with the kind of “common yard-stick” that ImageNet ( Russakovsky et al. 2015 ) provided for computer vision. Among current non-invasive datasets for decoding speech, LibriBrain ( Özdogan et al. 2025 ) is the deepest (i.e., largest within-subject ) with 52 hours of magnetoencephalography (MEG) recorded from a single participant. This dataset forms the foundation for the 2025 PNPL Competition ( Landau et al. 2025 ) , an open machine-learning competition that has catalyzed progress on two foundational decoding tasks: Speech Detection and Phoneme Classification. Progress on these tasks can be seen by looking at the online leaderboards ( https://libribrain.com/ ). For example, in just two months F1 macro scores on the Speech Detection task advanced rapidly from about 68% to a new state-of-the-art on the (extended) public leaderboard of 96%. Our aim in this paper is to build on this success by introducing a new standardized task: Word Detection (a.k.a. Keyword Spotting). Substantively, we provide the same kinds of supporting infrastructure for this task (e.g., data loader, reference model, reproducible metrics, public leaderboard) which led directly to accelerated improvements in Speech Detection and Phoneme Classification. As Landau et al. 2025 explain, the two tasks in the 2025 PNPL Competition were selected for their simplicity. Over time, the idea was to increase the complexity, and utility, of benchmark decoding tasks. Keyword Spotting (KWS) is an exciting landmark as it represents the first decoding task on this curriculum with practical utility for BCIs. In the established domain of voice computing, Keyword Spotting is commonly used to detect “wake words” (e.g., “Hey Siri”, “Alexa”, “OK Google”). Wake words like these can be used to indicate that subsequent speech should be interpreted as a command, or they can be used themselves as commands. In the emerging domain of brain computing, even a single wake word (e.g., “help”) could be profoundly meaningful to someone with severe paralysis. Only slightly further along the curriculum, a small set of working keywords (e.g., “hungry”, “tired”, “thirsty”, “toilet”, “pain”) would transform their quality of life. The benchmark task established in this paper is intended, ultimately, to lead to such an outcome. For clarity, and to contrast the use of acoustic inputs, we use the term Neural Keyword Spotting to denote Keyword Spotting from continuous brain data, a task represented schematically in Figure 1 . 2 Related Work 2.1 Existing Tasks for Non-Invasive Speech Decoding As mentioned in the introduction, the introduction of a rigorous benchmark a recent development in non-invasive speech decoding. Here a benchmark is a set of resources used by the community to measure progress. Benchmarks go beyond typical published work by including the following to support the measurement of progress. • Standardized data : Publicly available, with well-defined train and holdout splits. • Evaluation metrics : Agreed measure of success with public leaderboard to track progress. • Reference model : Reproducible implementation with open weights and training code. Prior to the release of LibriBrain ( Özdogan et al. 2025 ) , together with a standard Python library ( pnpl ) for loading predefined data splits ( Landau et al. 2025 ) , a number of open datasets ( Schoffelen et al. 2019 ; Nastase et al. 2022 ; Gwilliams et al. 2023 , e.g.,) were starting to reappear across large-scale studies ( Défossez et al. 2023 ; d’Ascoli et al. 2024 ; Ridge and Parker Jones 2024 ; Jayalath et al. 2025b , e.g.,) . However, data splits were not generally replicated making it difficult to compare methods. The same model architectures and weights were neither generally shared nor used as baselines and there were no public leaderboards. A rich set of speech decoding tasks have nonetheless emerged. These include the following tasks. Brain-to-Text (B2T) takes variable-length neural sequences (e.g., EEG, MEG, fMRI) as input and outputs text transcripts, typically evaluated with word error rate (WER) or semantic similarity metrics such as the BERTScore ( Zhang et al. 2020 ) . B2T is the analogue of ASR and represents a long-term goal, though its difficulty has motivated the development of what we call intermediate tasks, which lie between more foundational tasks like Speech Detection and full B2T. Non-invasive B2T has been explored with EEG and MEG ( Duan et al. 2023 ; Jo et al. 2024 ; Yang et al. 2024b ; Yang et al. 2024c ; Yang et al. 2024a ) , with semantic metrics often reported in place of WER, although recent work shows that competitive WERs are beginning to be achievable non-invasively ( Jayalath et al. 2025a ) . In fMRI, the coarse temporal resolution makes word-level alignment unlikely, though remarkable paraphrases have been produced which retain some semantic similarities to the ground truth speech ( Tang et al. 2023 , e.g.,) . Word Classification uses fixed-length neural segments aligned to individual words, producing categorical labels from a closed vocabulary. A number of recent works have focused on vocabularies of 250 words ( d’Ascoli et al. 2024 ; Özdogan et al. 2025 ; Jayalath et al. 2025a ) , though recent models can also impute out-of-vocabulary items with an external LLM ( Anthropic 2025 , e.g.,) if the predicted word is “unknown” ( Jayalath et al. 2025a ) . Phoneme Classification operates on shorter neural segments aligned to phonemes, predicting categorical labels over the phoneme inventory ( Özdogan et al. 2025 ; Landau et al. 2025 ) . Relatedly, Phonetic Feature Classification outputs binary labels for broader phonological features such as voicing. Phonetic features group together multiple phoneme classes (e.g., voiced /b, v, z/ vs. unvoiced /p, f, s/) and can

Read the complete text