Implementing Keyword Spotting on the MCUX947 Microcontroller with Integrated NPU
1 st Petar Jakuš, 2 nd Hrvoje Džapo · 2025
Read Implementing Keyword Spotting on the MCUX947 Microcontroller with Integrated NPU by 1 st Petar Jakuš, 2 nd Hrvoje Džapo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08911.txt.
Introduction
Petar Jakus and Hrvoje Dzapo implement, in a 2025 paper, a keyword spotting system on a microcontroller with a built in neural accelerator, aiming at voice control that never leaves the device. Always listening wake word detection usually leans on phones or the cloud. A tiny chip that hears the keyword itself enables cheap, private, low power interfaces.
Their pipeline extracts standard sound features from speech and classifies them with a small convolutional network. Training was quantization aware, so the model shrank to fit the chip with minimal accuracy loss. Running on the neural unit instead of the main processor cut inference time by a factor of fifty nine. The final model reached about 97 percent accuracy in just 30 kilobytes. Numbers like those move voice control into toys, appliances, and sensors that could never host a big model, without sending audio anywhere.
Catalog record
| CREATOR | 1 st Petar Jakuš, 2 nd Hrvoje Džapo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 12 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-08911
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Implementing Keyword Spotting on the MCUX947 Microcontroller with Integrated NPU 1 st Petar Jakuš, 2 nd Hrvoje Džapo Abstract This paper presents a keyword spotting (KWS) system implemented on the NXP MCXN947 microcontroller with an integrated Neural Processing Unit (NPU), enabling real-time voice interaction on resource-constrained devices. The system combines MFCC feature extraction with a CNN classifier, optimized using Quantization Aware Training to reduce model size with minimal accuracy drop. Experimental results demonstrate a 59 × \times speedup in inference time when leveraging the NPU compared to CPU-only execution, achieving 97.06% accuracy with a model size of 30.58 KB, demonstrating the feasibility of efficient, low-power voice interfaces on embedded platforms. I Introduction With the growing adoption of IoT devices, keyword spotting (KWS) has become essential for voice-controlled interaction in smart assistants and embedded systems. However, implementing KWS on resource-constrained devices remains challenging due to limited processing power, memory constraints, and strict energy requirements, demanding optimized solutions for microcontroller deployment. A typical keyword spotting system consists of signal acquisition, feature extraction and a neural network [ 7 ] . Signal acquisition in keyword spotting systems is typically performed using a microphone, most often a MEMS microphone, which converts pressure variations into electrical signals. The raw audio signal often contains noise and amplitude variations, requiring signal preprocessing to improve robustness and accuracy. Common noise reduction techniques include adaptive noise cancellation (ANC) [ 9 ] and beamforming [ 10 ] . Feature extraction transforms the input signal into a compact and informative representation suitable for classification. Widely used methods include Mel-frequency cepstral coefficients (MFCC) [ 1 , 11 ] and RASTA-PLP [ 18 , 8 ] . To reduce the computational complexity of the preprocessing stage, simplified approaches such as direct application of the Fast Fourier Transform (FFT) have been proposed [ 2 ] . For ultra-low-power applications, analog front-end approaches and featureless techniques have been explored [ 3 , 17 , 14 ] . Alternatively, feature extraction can be integrated into the neural network itself, as shown in [ 15 , 4 ] . The classification stage typically employs neural network architectures. Convolutional Neural Networks (CNNs) have shown strong performance in KWS tasks, especially when combined with model compression techniques such as pruning and quantization [ 12 , 5 , 19 ] . Hardware acceleration for neural network inference, such as using dedicated co-processors, has been shown to significantly reduce power consumption [ 14 ] . Recurrent architectures like Long Short-Term Memory (LSTM) networks [ 6 ] and Gated Recurrent Units (GRUs), including energy-efficient variants such as eGRU [ 2 ] , are also employed to capture temporal dependencies in speech signals. Advanced compression techniques such as network binarization have further reduced memory footprints for deployment on edge devices [ 3 ] . Hybrid architectures combining CNNs and RNNs have been proposed to improve classification performance while maintaining efficiency [ 19 ] . This work presents an optimized KWS system targeting the NXP MCXN947 microcontroller, leveraging its integrated NPU to accelerate a quantized CNN model with MFCC-based features. By employing quantization-aware training (QAT), the method minimizes accuracy loss while ensuring efficient quantization, significantly reducing model size and inference latency. Experimental results demonstrate the feasibility of deploying responsive, low-power voice interfaces on resource-constrained edge devices. II Methodology II-A Data The Google Speech Commands dataset [ 16 ] contains 105,829 one-second voice recordings of 35 words, recorded at 16 kHz sample rate on mobile devices. The dataset is divided into training (84,843), test (11,005), and validation (9,981) sets as originally proposed. II-A 1 Preprocessing Mel-Frequency Cepstral Coefficients (MFCC) were used for feature extraction through the following pipeline: • Framing and Windowing: Audio signals were segmented into 25 ms frames with 10 ms hop size (400 samples at 16 kHz), as illustrated in Figure 1 . A Hamming window was applied to each frame to reduce spectral leakage (Figure 2 ): w(n)=0.54-0.46\cdot\cos\left(\frac{2\pi n}{N}\right) Figure 1: MFCC original signal and frames. Figure 2: Frames after applying Hamming function. • Spectral Analysis: Fast Fourier Transform (FFT) converted each windowed frame to the frequency domain, followed by power spectrum calculation (Figure 3 ). For a given FFT output X[k] , the power spectrum P[k] is defined as: P[k]=\frac{1}{N}|X[k]|^{2} where N is the FFT length. Figure 3: Power spectrum. • Mel-scale Filtering: A bank of 40 filters spanning 40 Hz to 7.6 kHz was applied to align with human auditory perception and 4 ). The Mel scale provides higher frequency resolution at lower frequencies, matching human hearing characteristics. Figure 4: Frames Mel spectrums. • MFCC Computation: Discrete Cosine Transform (DCT) was applied to the log mel-spectrum to decorrelate coefficients, producing 20 MFCC features per frame following HTK convention (Figure 5 ). Figure 5: MFCC spectogram. This preprocessing pipeline transforms raw audio into perceptually-motivated features suitable for speech recognition tasks. II-B Model Architecture and Quantization The proposed architecture employs a compact CNN optimized for edge deployment, featuring two convolutional layers (each with 2D convolution, batch normalization, and max pooling) followed by two dense layers. The model is trained specifically for the "Marvin" keyword using Adam optimization (learning rate = 0.001) over 10 epochs, with class weights of 24.81 (Marvin) and 0.51 (non-Marvin) to address imbalance. The model architecture incorporates quantization-aware training (QAT) to enable efficient conversion from 32-bit floating-point to 8-bit fixed-point weight representation. The complete QAT approach applies quantization constraints to all layers during training, simulating 8-bit operations in forward passes while maintaining full precision for backward propagation. While this increases training time by approximately 20%, it ensures minimal accuracy loss when after quantization. Alternatively, a fine-tuning approach can be applied where a pre-trained full-precision model is adapted with QAT for fewer epochs, offering reduced training time with comparable accuracy. The regular model architecture, the QAT prepared version and final quantized implementation are detailed in Tables I , II , and III , respectively, including layer-specific quantization parameters. Table I: Regular Neural Network Architecture Layer (type) Output Shape Param # input (InputLayer) (None, 98, 20, 1) 0 conv1 (Conv2D) (None, 96, 18, 32) 320 bn1 (BatchNormalization) (None, 96, 18, 32) 128 pool1 (MaxPooling2D) (None, 48, 9, 32) 0 conv2 (Conv2D) (None, 46, 7, 64) 18496 bn2 (BatchNormalization) (None, 46, 7, 64) 256 pool2 (MaxPooling2D) (None, 23, 3, 64) 0 gap (GlobalAveragePooling2D) (None, 64) 0 dropout (Dropout) (None, 64) 0 fc1 (Dense) (None, 128) 8320 output (Dense) (None, 1) 129 Total params: 27649 (108.00 KB) Trainable params: 27457 (107.25 KB) Non-trainable params: 192 (768.00 B) Table II: QAT Prepared Neural Network Architecture Layer (type) Output Shape Param # input (InputLayer) (None, 98, 20, 1) 0 quantize_layer (QuantizeLayer) (None, 98, 20, 1) 3 quant_conv1 (QuantizeWrapperV2) (None, 96, 18, 32) 387 quant_bn1 (QuantizeWrapperV2) (None, 96, 18, 32) 129 quant_pool1 (QuantizeWrapperV2) (None, 48, 9, 32) 1 quant_conv2 (QuantizeWrapperV2) (None, 46, 7, 64) 18627 quant_bn2 (QuantizeWrapperV2) (None, 46, 7, 64) 257 pool2 (MaxPooling2D) (None, 23, 3, 64) 0 gap (GlobalAveragePooling2D) (None, 64) 0 quant_dropout (QuantizeWrapperV2) (None, 64) 1 quant_fc1 (QuantizeWrapperV2) (None, 128) 8325 quant_output (QuantizeWrapperV2) (None, 1) 134 Total params: 27864 (108.84 KB) Trainable params: 27457 (107.25 KB) Non-trainable params: 407 (1.59 KB) Table III: Quantized Model Architecture Layer (type) Output Shape Param # InputLayer [-1, 98, 20, 1] 0 Conv2D [-1, 96, 18,