Keyword Spotting with Hyper-Matched Filters for Small Footprint Devices
Yael Segal-Feldman 1, Ann R. Bradlow 2, Matthew Goldrick 2, Joseph Keshet 1 · 2025
Read Keyword Spotting with Hyper-Matched Filters for Small Footprint Devices by Yael Segal-Feldman 1, Ann R. Bradlow 2, Matthew Goldrick 2, Joseph Keshet 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-04857.txt.
Introduction
Yael Segal-Feldman, Ann Bradlow, Matthew Goldrick, and Joseph Keshet built an open-vocabulary keyword spotter in 2025 for small devices. Keyword spotting listens for a target word in speech, and the open form must find words it was never trained on. Their model targets the tight memory and power limits of small-footprint hardware.
Three parts make up the system: a speech encoder, in tiny Whisper or Conformer form; a keyword encoder built as a hyper-network; and a detection network. The keyword encoder turns the target word, given as letters, into weights for a filter matched to that word. That filter guides attention over the speech to judge if the word occurs. The catalog title appends a preprint note; the clean title governs. Readers get a design that can learn new wake words on demand.
Catalog record
| CREATOR | Yael Segal-Feldman 1, Ann R. Bradlow 2, Matthew Goldrick 2, Joseph Keshet 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-04857
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Keyword Spotting with Hyper-Matched Filters for Small Footprint Devices Thanks: The corresponding author is Yael Segal-Feldman (email: [email protected]). This document is a pre-print version of the paper. Yael Segal-Feldman 1, Ann R. Bradlow 2, Matthew Goldrick 2, Joseph Keshet 1 Abstract Open-vocabulary keyword spotting (KWS) refers to the task of detecting words or terms within speech recordings, regardless of whether they were included in the training data. This paper introduces an open-vocabulary keyword spotting model with state-of-the-art detection accuracy for small-footprint devices. The model is composed of a speech encoder, a target keyword encoder, and a detection network. The speech encoder is either a tiny Whisper or a tiny Conformer. The target keyword encoder is implemented as a hyper-network that takes the desired keyword as a character string and generates a unique set of weights for a convolutional layer, which can be considered as a keyword-specific matched filter. The detection network uses the matched-filter weights to perform a keyword-specific convolution, which guides the cross-attention mechanism of a Perceiver module in determining whether the target term appears in the recording. The results indicate that our system achieves state-of-the-art detection performance and generalizes effectively to out-of-domain conditions, including second-language (L2) speech. Notably, our smallest model, with just 4.2 million parameters, matches or outperforms models that are several times larger, demonstrating both efficiency and robustness. I Introduction Keyword Spotting (KWS) is the task of identifying a predefined set of target keywords. It is an essential task in speech recognition systems, particularly crucial for voice assistants on small-footprint devices , such as smartphones and small speakers [ 1 ] . Throughout the years, various techniques have been explored for this task, beginning with basic deep neural networks based on Convolutional Neural Networks (CNNs) [ 2 , 3 ] to Recurrent Neural Networks (RNNs) as audio encoders [ 4 , 5 ] . As research has advanced, attention has increasingly shifted toward open vocabulary KWS, which focuses on detecting keywords, terms, or phrases in speech recordings — even when these targets were not seen during training 1 1 1 Throughout this paper, we use the term target keyword to refer to a word, term, or phrase of interest. . A central challenge in this task is achieving the flexibility required to accurately detect previously unseen keywords within spoken utterances. Various approaches have been proposed to address this challenge. One widely adopted strategy is query-by-example (QbE), which leverages enrolled audio samples of the target keyword to facilitate detection [ 6 , 7 , 8 , 9 , 10 ] . In a QbE approach, the system builds a reference model using enrolled audio examples of the target keyword. This model is then used to match incoming audio, identifying instances of the target keyword based on similarity to the stored examples. An alternative QbE method leverages Text-to-Speech (TTS) technology to synthesize audio for new keywords [ 11 ] , which serves as the reference for detecting those keywords in future recordings. However, the performance of QbE approaches is highly sensitive to variations in recording conditions, making consistency in the audio environment crucial for reliable detection. To overcome this problem, several studies have focused on deep acoustic models trained with Connectionist Temporal Classification (CTC) loss [ 12 ] and techniques for detecting the target keywords within speech [ 13 , 14 , 15 , 16 ] . Other research approaches involve presenting the target keyword in phonetic or graphemic form and using the resulting keyword embedding as input to the acoustic model [ 17 , 18 , 19 , 20 , 21 , 22 , 23 ] . This method translates the target keyword into a representation that the acoustic model can process, allowing the system to recognize the keyword based on its phonetic or graphemic characteristics. Additionally, some methods utilize hyper-networks, where a primary model receives the target keyword in either phonetic or graphemic form and subsequently generates weights for another network. This approach effectively creates a tailored set of parameters for the secondary network based on the input keyword. For example, [ 24 ] employs this technique to produce weights specifically for a single CNN layer, enhancing its ability to detect the target keyword. On the other hand, [ 25 ] generates weights for the normalization layer of a Transformer [ 26 ] , adjusting the model’s internal parameters to better accommodate the target keyword. Despite these efforts, the mentioned works still face a major challenge: their performance dramatically degrades when converted to work on small-footprint devices. The models of [ 24 , 20 , 21 , 22 ] are designed for small-footprint devices but exhibit reduced performance on challenging target keywords. On the other hand, [ 25 ] demonstrates excellent performance, but their model is too big for small-footprint devices. This paper presents an open-vocabulary KWS model suitable for small-footprint devices with a state-of-the-art detection rate. Inspired by the works of [ 25 ] and [ 24 ] , the proposed system is composed of a speech encoder, target keyword encoder, and detection network. The speech encoder provides an effective speech representation. As part of our effort to address the small-footprint challenge, the encoder is implemented using either tiny Whisper [ 27 ] or as tiny Conformer [ 21 ] . The target keyword encoder learns to represent an unseen keyword. It functions as a hyper-network, and rather than representing the keyword as an embedding vector, it generates a unique set of weights for a convolution layer of the detection network. The detection network leverages these weights to perform keyword-specific detection. It is implemented with a Perceiver architecture [ 28 ] that allows the use of Transformer layers with fewer parameters, making it suitable for small-footprint devices. Specifically, the keyword-specific encoder’s weights are used in the convolutional operation to guide the cross-attention weights within a Perceiver module. The overall detection network can be considered as a matched filter that highlights keyword locations. As a result, the audio encoder output is directed towards the target keyword-specific, enabling the detection of whether the keyword-specific was uttered in the recording. Our implementation and trained models are available on GitHub 2 2 2 https://github.com/YaelSegal/HyperSpotter . We performed an empirical evaluation of the model across various model sizes and multiple datasets. The evaluation was performed for in-domain scenarios, where both the training and testing data originate from the same source. Furthermore, we assessed the model’s performance in out-of-domain generalization scenarios, including out-of-domain datasets, languages unseen during training, and non-native English speakers. The results indicate that our system achieves a state-of-the-art detection rate and generalizes effectively to out-of-domain scenarios. Additionally, our smallest network, with 4.2M parameters, performs comparably to a model with 10M parameters in certain scenarios. The contributions of our work are as follows: (1) We propose a novel, state-of-the-art open-vocabulary KWS model with an architecture suitable for small-footprint devices; (2) We evaluate several implementations of the proposed architecture using various audio encoders; (3) We present results across multiple datasets and diverse out-of-domain scenarios, including evaluation on L2 speakers; and (4) Our system’s code is openly accessible for academic use, setting it apart from many similar KWS projects. This paper is structured as follows. Section III introduces our proposed method