Conserved Kinematic Representations enable Zero-Shot Decoding in Handwriting BCIs

Srinivas Ravishankar \affmark 1, Virginia de Sa \affmark 1, 2 · 2026

Read Conserved Kinematic Representations enable Zero-Shot Decoding in Handwriting BCIs by Srinivas Ravishankar \affmark 1, Virginia de Sa \affmark 1, 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19048.txt.

Introduction

Srinivas Ravishankar and Virginia de Sa ask if motor cortex handwriting rests on shared move bits. Felt handwriting links serve Latin well but need each sign in train, a wall for tongues with thousands of signs. Their frame lines brain fire to felt moves and trains a zero shot decoder. It scores sixty four percent top three hits on unseen letters, backing shared stroke codes and easing open word links with light retune.

Catalog record

CREATORSrinivas Ravishankar \affmark 1, Virginia de Sa \affmark 1, 2
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30997
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19048

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Conserved Kinematic Representations enable Zero-Shot Decoding in Handwriting BCIs Srinivas Ravishankar \affmark 1, Virginia de Sa \affmark 1, 2 Abstract While intracortical Brain-Computer Interfaces (iBCIs) that decode imagined handwriting have achieved high communication rates for Latin scripts, they rely on observing every character in the alphabet during training. This poses a challenge in scaling to logographic languages (e.g., Chinese, Japanese), where the character set exceeds thousands of classes. The limitation highlights a fundamental question in motor neuroscience: does the motor cortex represent handwriting through the composition of shared kinematic primitives, that can be exploited by decoders? We introduce a computational framework for aligning neural activity to imagined kinematics in large datasets, enabling the training of a zero-shot capable machine learning algorithm for decoding unseen characters. Our model achieves 64% hits@3 retrieval on unseen letters, suggesting that neural representations of kinematic strokes are robustly conserved across different character contexts. This study provides a framework for dissecting conserved neural dynamics in large-scale intracortical datasets and offers strong evidence for a compositional basis of complex motor control. It also establishes a new paradigm for open-vocabulary iBCI communication with minimal recalibration burden on the user, crucial to increasing adoption of neuroprosthetics in logographic languages. Department of Cognitive Science, UC San Diego Halicioglu Data Science Institute, UC San Diego \emails [email protected] 1 Introduction Individuals with severe paralysis or speech disorders face significant challenges in communication. Brain-computer interfaces (BCIs) translate brain signals directly into text or speech, bypassing traditional peripheral systems such as the hands or voice. Handwriting BCIs in particular decode neural activity related to imagined writing (mentally tracing penstrokes without overt movement). Such a system was demonstrated by 8 to achieve state-of-the-art accuracy and allow intuitive, high-throughput communication for those unable to use traditional modes of communication. Furthermore, these technologies are poised to offer new modes of interaction for healthy users as well. However, we anticipate key challenges in their adoption. Existing methods employ relatively simple machine learning methods to map neural activity to recognized characters, where neural training data must be collected for each character. Although they have been demonstrated in English with a limited character set, it is non-trivial to scale to large-character languages such as Chinese or Japanese, which requires \sim 2500 characters for regular use. This requirement blocks access to these technologies for large segments of the population. Figure 1: High performance has been demonstrated when decoding imagined handwriting in English, with a limited character set. Logographic languages may require more sophisticated methods. To address these limitations, we propose to enable zero-shot character decoding in handwriting BCIs to reduce data demands in existing systems, and extend usability to language with large character sets. While benchmarks such as FALCON ( 5 ) have been proposed to evaluate few-shot and zero-shot generalization to new sessions, a different dimension of zero-shot generalization has received very limited attention: decoding of new characters unseen during training. Some recent works have explored zero-shot handwriting decoding in Chinese ( 9 ; 7 ) , but are limited to slow single-letter writing. The participant was asked to imagine writing with their entire arm, and each character took 4 to 9 seconds to write. No existing system supports ballistic continuous handwriting-to-text comparable to the performance previously demonstrated by 8 while allowing zero-shot capability. Furthermore, prior work requires supervised single-letter data collection for initial training as well as recalibration. Recalibration is a continuous burden on BCI users, arising due to non-stationarity and electrode shift. Neural signals are inherently non-stationary; their statistical distribution changes across sessions due to biological noise or plasticity, impedance fluctuations, and physical electrode shifts relative to the brain tissue. These phenomena render the decision boundaries of the initial classifier increasingly obsolete with time, and necessitate the collection of new labeled data to restore accuracy. It is crucial to minimize recalibration requirements to have a functional BCI for daily use by participants. In this study, we demonstrate a method for handwriting decoding that is zero-shot capable, and can be trained and recalibrated without any supervised single-letter data collection. This is achieved through kinematics prediction, and building on unsupervised recalibration work introduced for handwriting decoding ( 4 ) . While an imagined logographic dataset is not yet publicly available, we demonstrate a proof of concept in this work. We evaluate this method by cutting out all snippets of each character in the training data of an existing English imagined handwriting dataset to simulate the zero-shot setting. 2 Methods 2.1 Dataset The neural dataset used in this study was obtained from intra-cortical micro-electrode recordings during imagined handwriting by a single participant in a prior study ( 8 ) . Data were collected using 2 Utah arrays implanted in the hand knob area of the precentral gyrus, with a total of 192 electrodes over 10 sessions and preprocessed in the same manner as the original work: Multi-Unit threshold crossing rates were computed, and binned into 20ms time steps. Data were smoothed with a causal Gaussian filter. Each block of data was mean-subtracted and divided by the standard deviation per electrode. We follow the same training and validation trial split as the original work to train continuous sentence classification models. For our zero-shot character recognition experiments, we use all data across training and validation trials, but ensure that all neural snippets associated with the chosen zero-shot character is excluded. Zero-shot character recognition performance is evaluated on the held-out character snippets from both training and validation trials. 2.2 Architecture To create a zero-shot capable system, we propose a two-stage architecture shown in Fig 2 . The first stage maps neural data to a velocity sequence of a hypothetical pen-tip. While we use a Recurrent Neural Network (RNN) for this stage, the architecture is agnostic to the specific choice of sequence model. The second stage uses a template matching approach to find the most likely candidate, given a library of kinematics sequences for different characters. Figure 2: Two-stage pipeline to enable zero-shot character recognition. The first stage (kinematics prediction) predicts a velocity sequence of a hypothetical pen-tip corresponding to the observed neural data. The second stage (template matching) ranks a character template dictionary using a DTW-based distance measure, enabling zero-shot recognition. There are significant challenges that need to be solved to train the kinematics prediction model. Given continuous sentence data, we lack the full ground truth kinematics of the entire sentence (consider the velocity profile for the gaps between characters). Moreover, even for characters where we have approximate ground truth of the participant’s imagined writing, we lack precise frame-wise timing. Due to this variability in timing between the predicted sequence \mathbf{\hat{y}} and the approximate ground truth \mathbf{y} , we employ a soft Dynamic Time Warping loss ( 3 ) to optimize our kinematics model. Unlike element-wise metrics such as Mean Squared Error, DTW aligns sequences by finding the optimal non-linear warping path that minimizes the cumulative alignment cost. However, the classical DTW formulation relies on a discrete \min operator, which is non-differentiable and unsuitable for gradient-based optimization. Soft-DTW addresses this by replacing the hard minimum with a generalized \text{softmin}_{\gamma} operator, defined as: \text{softmin}_{\gamma}(z_{1},\dots,z_{k})=-\gamma\log\sum_{i=1}^{k}\exp\left(-\frac{z_{i}}{\gamma}\right), (1) where \gamma>0 is a smoothing hyperparameter. As \gamma\to 0 , the objective converges to the exact DTW solution. The loss

Read the complete text