Encoding of speech modes and loudness in ventral precentral gyrus Authors: Aparna Srinivasan, Maitreyee Wairagkar, Carrina Iacobacci, Xianda Hou, Nicholas S Card, Brandon G Jacques, Anna L Pritchard, Payton H Bechefsky, Leigh R Hochberg, Nicholas AuYong, Chethan Pandarinath, David M Brandman, Sergey D Stavisky Published: 2026 ABSTRACT AbstractThe ability to vary the mode and loudness of speech is an important part of the expressive range of human vocal communication. However, the encoding of these behaviors in the ventral precentral gyrus (vPCG) has not been studied at the resolution of neuronal firing rates. We investigated this in two participants who had intracortical microelectrode arrays implanted in their vPCG as part of a speech neuroprosthesis clinical trial. Neuronal firing rates modulated strongly in vPCG as a function of attempted mimed, whispered, normal or loud speech. At the neural ensemble level, mode/loudness and phonemic content were encoded in distinct neural subspaces. Attempted mode/loudness could be decoded from vPCG with 94% and 89% accuracy for the two participants, and corresponding neural preparatory activity at 640 ms and 270 ms before speech onset enabled 80% decoding accuracy, respectively. We then developed a closed-loop loudness decoder that achieved 94% online accuracy in modulating a brain-to-text speech neuroprosthesis output based on attempted loudness. These findings demonstrate the feasibility of decoding mode and loudness from vPCG, paving the way for speech neuroprostheses capable of synthesizing more expressive speech.Subject terms: Brain-machine interface, Premotor cortex, Motor cortex, Neural decoding Introduction Human speech production is highly flexible, encompassing a range of modes such as mimed, whispered, and overt speech. In overt speech, airflow through vibrating vocal folds is modulated to produce voiced sounds1, whereas in whispered speech, vocal fold vibration is absent2. Mimed speech involves articulatory movements without accompanying sound. Speech mode (and loudness level within whispered and overt speech) is often modulated to signal an additional layer of communication intent or adapt to environmental demands3–5. For example, speaking loudly is often interpreted as confident and assertive3,5. Precise control over speech mode and loudness is essential for producing natural and expressive communication, and these abilities are often impaired in the case of speech motor disorders such as dysarthria6. Understanding how the brain encodes different speech modes and loudness is therefore not only of fundamental neuroscientific interest, but is also important for developing brain-computer interfaces (BCIs) aimed at restoring expressive communication to people living with paralysis7. The neurobiological study of speech modes is difficult because this uniquely human behavior lacks direct animal models, and because there are relatively few opportunities for direct neural recordings from people. Instead, prior work has used noninvasive neural measurements with human subjects. fMRI studies have shown that the speech motor cortex located in the precentral gyrus (PCG), particularly the mid-ventral regions associated with the larynx and articulators, modulates across different types of speech production8–11. For example, overt speech recruits stronger activation of both phonatory (laryngeal) and articulatory regions compared to whispered speech12. The laryngeal regions are also involved in voluntary breathing10,12–14. There have also been intracranial electrocorticography recordings exploring the neural correlates of overt speech15,16 and other paralinguistic speech features, such as pitch17,18. However, speech mode and loudness modulation have not yet been studied at the precise resolution of neuronal spiking activity. As speech neuroprostheses have been rapidly improving, characterizing neural ensemble encoding of speech mode and loudness at cellular resolution is becoming increasingly important so as to be able to achieve more precise control of speech output7,19–24. Recent BCI research has demonstrated that the phonemic content of attempted overt19–22 or mimed speech25–27 can be decoded from the speech motor cortex into text or voice output. However, the ability to modulate speech BCI output with sustained prosodic modulations, such as changing words’ loudness, has not been previously demonstrated. Here, we explore how attempted speech modes and loudness are encoded in ventral precentral gyrus (vPCG). We recorded neural activity using intracortical microelectrode arrays when speech neuroprosthesis clinical trial participants attempted to mime, whisper, or speak at normal or loud volumes. For writing simplicity, we will refer to these four behaviors as different “loudness levels”, reflecting their typical acoustic intensity, while recognizing that mimed, whispered, and overt speech are qualitatively different speech modes. Analyzing multi-unit and single unit spiking activity and neural ensemble dynamics across the implanted region, we found that firing rates in vPCG were strongly modulated by attempted loudness, and that words’ phonemic content and loudness levels were separable in a latent neural space. Preparatory activity reflecting intended loudness could be detected hundreds of milliseconds prior to speech onset. Finally, we developed a closed-loop loudness decoder that predicted the attempted loudness level (normal vs. loud) from neural activity and modified the output of a text-based speech neuroprosthesis in real-time by applying loudness-based formatting to the predicted text. These findings demonstrate the feasibility of decoding attempted loudness from vPCG and pave the way for improved speech neuroprostheses capable of producing more expressive communication. Results To investigate how attempted speech modes and loudness are encoded in the human vPCG, we collected intracortical recordings from two participants (“T15” and “T16”) enrolled in the BrainGate2 clinical trial while they attempted speech tasks that involved loudness modulation. Participant T15 was severely dysarthric due to ALS, and participant T16 was dysarthric due to a pontine stroke. T15 had four 64-microelectrode arrays implanted along the vPCG, spanning cortical areas 6v, 4, and 55b (Fig. 1b). T16 had two arrays in area 6d, one in 6v, and one in the border between area 55b and premotor eye field (PEF) (Supplementary Fig. 1a). Loudness and word encoding across vPCG. a Schematic of the word-loudness task. The participant attempted to speak a word at the instructed loudness level while neural activity was recorded from microelectrode arrays implanted in vPCG. b 3D reconstruction of participant T15’s brain, showing the locations of the Utah arrays (black squares) and relevant brain regions estimated from fMRI. c Firing rates (mean ± s.e.) from an example electrode in each array, computed by trial-averaging within loudness conditions. Activity was aligned to cue onset (left), speech onset (middle), and speech offset (right). All arrays exhibited loudness-related modulation, i.e., had some electrodes tuned to loudness levels. d Electrodes tuned to attempted speech loudness level, determined by significant differences in firing rates between loudness conditions (one-way ANOVA with post-hoc Tukey’s honestly significant difference test, p < 0.05). e Electrodes tuned to attempted words determined by significant differences in firing rates between words (one-way ANOVA with post-hoc Tukey’s honestly significant difference test, p < 0.05). Electrodes whose firing rates are shown in (c) are marked with orange circles. See Supplementary Fig. 1 for participant T16. We designed both open-loop and closed-loop tasks to probe how neural activity modulated with attempted speech behavior and to evaluate the feasibility of decoding loudness for applications in speech BCIs. Loudness encoding in ventral precentral gyrus To assess whether neural firing rates in vPCG modulated with respect to attempted loudness, we first analyzed the neural recordings (threshold crossings) from both participants performing a word-loudness task (Fig. 1a), in which they attempted to speak six different words at four different loudness levels: MIME, WHISPER, NORMAL, and LOUD. While some electrodes exhibited firing rate changes that were selectively elevated for a specific loudness level, with responses to other levels remaining similar, most electrodes were tuned to multiple loudness levels (Fig. 1c). Electrodes with firing rates that significantly differed across pairs of attempted loudness levels (p < 0.05, one-way ANOVA with post-hoc Tukey’s honestly significant difference test) were distributed throughout T15’s vPCG region sampled by the arrays (Fig. 1d). Similar firing rate modulations were observed in T16: within her 55b/PEF array, electrodes that exhibited significantly different firing rates for different loudness levels were predominantly located on the putative 55b cortical region (Supplementary Fig. 1c). Using the same statistical criterion, electrodes showing significant differences in firing rates across word pairs were also distributed throughout T15’s vPCG region (Fig. 1e). Overall, 86% of electrodes showed significant modulation to both loudness levels and words. However, there was markedly less word tuning on T16’s 55b/PEF array compared to the 6v array (Supplementary Fig. 1d): 85% of electrodes were significantly tuned to loudness whereas only 63% were tuned to words. Firing rate activity for all electrodes is shown in Supplementary Fig. 2 for T15 and Supplementary Fig. 3 for T16. Complementing these per-electrode threshold crossings analyses, we also examined task tuning at the level of individual neurons using spike-sorted single-unit activity (Supplementary Fig. 4). For each neuron, we measured the firing rate from [-0.5, 0.5] s around speech onset and assessed tuning for word and loudness (one-way ANOVA, Tukey’s honestly significant difference post- hoc test, p < 0.05). In T15’s recordings, 65% of neurons were tuned to both words and loudness, 22.8% to words only, and 8% to loudness only (Supplementary Fig. 4c). In T16’s recordings, 50.8% of neurons were tuned to both, with a higher proportion tuned exclusively to loudness compared to words (27% vs. 6.6%) (Supplementary Fig. 4d). Only a small fraction of neurons in both participants were not tuned to either behavior. Loudness representation in neural ensemble activity To examine how loudness was represented across the neural population activity, we first performed principal component analysis (PCA) on the ensemble neural activity (across each electrode’s spike band power) trial-averaged within all word-loudness conditions (Fig. 2a for T15, Supplementary Fig. 5a for T16). When projected onto the top three principal components, the trial-averaged neural activity was organized with a gradient according to attempted loudness (Fig. 2a top, Supplementary Fig. 5a top). When viewing the same data projections but labeled by the attempted words (Fig. 2a bottom, Supplementary Fig. 5a bottom), the neural ensemble is also organized according to word identity, but along a different axis. These results suggest that what is said (phonemic content), and how it’s said (loudness), are encoded along separable neural dimensions. Neural ensemble activity separably encodes loudness from words. a Principal Component Analysis (PCA) projections of trial-averaged spike band power from [−750, 750] ms around speech onset during the word-loudness task. Both subplots show the same data projections but with conditions colored according to loudness (top) or which word was spoken (bottom) to illustrate the independent encoding of loudness versus phonemic content. b demixed PCA (dPCA) applied to these same data. Each subplot shows the data projected onto the respective dPCA decoder axis. Each plot contains 24 curves (4 loudness levels × 6 words), with loudness represented by increasing saturation and linewidth from MIME up to LOUD, and words shown in different colors. c Explained variance of individual demixed PCs. The pie chart illustrates the proportion of total neural variance attributed to each task parameter. d Relationship between demixed PCs. The upper right triangle shows the dot product between all pairs of the first 15 demixed principal axes, and the lower left triangle shows the correlations between these components. Stars indicate pairs that are significantly and robustly non-orthogonal. See Supplementary Fig. 5 for T16. To further quantify this phenomenon, we applied demixed PCA (dPCA) to the same dataset. Figure 2b and Supplementary Fig. 5b show the projections onto the components that capture the most variance for each task parameter for T15 and T16, respectively. Most of the variance in the neural activity was captured by the condition-independent components, which together amounted to 56-60% of the neural variance (Fig. 2b, c and Supplementary Fig. 5b, c), reflecting the shared temporal modulations in the neural activity irrespective of task condition28. We also found word-specific and loudness-specific components where the neural projections separated out according to different words and loudness levels, respectively (Fig. 2b and Supplementary Fig. 5b). For participant T15, the word-related components captured the second highest variance in neural activity, whereas for T16, the second-highest share of variance was captured in the loudness-related components (Supplementary Fig. 2c). In both participants, little neural variance was attributed to the interaction of loudness and word. Even though dPCA does not enforce orthogonality between encoding axes corresponding to different task parameters, most loudness and word dimensions turned out to be close to orthogonal (Fig. 2d and Supplementary Fig. 5d, upper triangle; for example, dPCs 2 and 4, the largest T15 word and loudness dimensions, were close to orthogonal). The mean absolute pairwise cosine similarity between the top five word- and loudness-dPCs was 0.14 for T15 and 0.28 for T16. Control analyses found that the loudness-related neural differences could not be explained merely by differences in breath depth across conditions (Supplementary Fig. 6 and Supplementary Note 1). These findings suggest that word and loudness information in vPCG are represented in largely distinct, though partially overlapping, neural subspaces. This implies that a neuron could have strong tuning for different loudness conditions but not for different words, for words but not for loudness, for both (with different covariation patterns across neurons), or for neither. Offline decoding of loudness We next evaluated whether loudness level could be accurately decoded from neural activity; our goal was to assess whether the loudness encoding in vPCG could be pragmatically used to increase the expressivity of a speech neuroprosthesis. To analyze how decoding performance evolved over time, decoders (multinomial logistic regression models) were trained and evaluated to predict the four loudness levels. For T15, peak decoding accuracy of 93% was achieved at speech onset (Fig. 3a), while for T16, peak accuracy of 86% occurred 270 ms after speech onset (Supplementary Fig. 7a). For both participants, loudness could be decoded well before speech onset, with over 80% accuracy at 640 ms before onset for T15 and 270 ms before onset for T16, indicating preparatory encoding of speech loudness. Loudness decoding (and also word decoding) remained significantly above chance for up to ~2 s after speech offset in both participants (Supplementary Fig. 8). The present data do not allow us to resolve the specific cause of this phenomenon, which could reflect the neural correlates of lingering articulatory activity; cognitive-state signals such as recent effort; or potential differences in end-of-speech orofacial and/or diaphragm/body posture across behavioral conditions. Loudness could be accurately decoded from neural activity offline. a Loudness decoders were trained and evaluated on a 400 ms window of neural features with a 10 ms stride. Trial-averaged performance (mean ± s.e.) began to surpass chance at 500 ms after cue onset (green star) and decreased after speech offset (one-sided time-cluster permutation test, p = 2.9 × 10−2). Gray vertical lines mark when decoding accuracy exceeded 80% (640 ms before speech onset) and when maximum accuracy was achieved (at speech onset). b Classification accuracy (mean ± s.d., n = 6-fold cross-validation) for each array. Performance was significantly above chance for all arrays and is indicated by * (one-sided permutation test, p = 0.00 for each array). Gray circles overlaid on each bar plot show classification accuracy for individual test folds. c Confusion matrix of the decoder’s performance using all arrays. Typical confusions, though few, were between adjacent loudness levels. d Classification accuracy (mean ± s.d., n = 10 repeats) when randomly dropping electrodes. Performance was only slightly worse even with the removal of up to half the electrodes. See Supplementary Fig. 7 for T16. To quantify overall loudness-related information, we decoded ensemble activity across a longer window of neural data spanning 600 ms before to 600 ms after speech onset (Fig. 3b–d and Supplementary Fig. 7b–d). All arrays performed significantly above chance. Classification confusions, though rare, were typically between mime and whisper or between normal and loud conditions for the T15 data, and most frequently between the whisper and normal conditions in the T16 data. Although the exact error patterns differed between participants, a common element was that the neural correlates of conditions adjacent in attempted loudness were more likely to be confused (Fig. 3c and Supplementary Fig. 7c). For T15, offline decoders trained using both spike band power and threshold crossings as neural features achieved 94.9% performance accuracy, but similar performance of 93.8% accuracy was obtained by training the model on spike band power alone. Hence, in subsequent online decoding, we used only spike band power for efficiency. To assess the loudness decoding contribution of distinct brain locations, we trained decoders using data from each array separately (Fig. 3b and Supplementary Fig. 7b). Neural features from the v6v array yielded the highest accuracy in both participants (90% for T15, 87% for T16). Loudness encoding was robustly distributed across the neural ensemble. In a simulated neural drop-out analysis, we did not observe a substantial drop in decoding accuracy until decoding was attempted with fewer than ~30 electrodes (Fig. 3d and Supplementary Fig. 7d). We also evaluated the performance of decoders trained on the word-specific and loudness-specific neural dimensions as a complementary method of quantifying the (low) degree of overlap in these neural subspaces (Supplementary Fig. 9). Using only the loudness-dPC projections (5-dimensional for T15, 7-dimensional for T16), loudness levels could be decoded robustly (95% accuracy for T15, 87.8% for T16; 25% chance). Similarly, words could be decoded with high accuracy using only the word-dPC dimensions (8-dimensional for T15, 5-dimensional for T16; 98.5% accuracy for T15, 90.4% for T16; 16.6% chance). In contrast, accuracies were only slightly above chance when we switched the projections used for decoding: 25.4% (T15) and 36.4% (T16) when decoding words from the loudness subspace, and 36.9% (T15) and 49% (T16) when decoding loudness from the words subspace. Although the participants were only instructed to vary their speech mode and loudness, we expected that other elements of prosody would inevitably covary, including speech duration. To minimize the potential confounding effects that loudness-related differences in speech duration might have contributed to decoding, we also performed decoding using only neural data from speech onset to each participant’s across-condition shortest speech duration (300 ms for T15, 150 ms for T16; Supplementary Fig. 10). Decoding accuracy decreased by only 5% for T15 and 10% for T16, indicating that loudness could still be accurately decoded when minimizing potential contributions from speech duration–related neural correlates. Closed-loop loudness decoding in a speech BCI To demonstrate the feasibility of incorporating decoded loudness into a speech BCI, we designed a closed-loop sentence-loudness task in which participant T15 attempted to speak words at different loudness levels, which were then decoded in real-time into either lowercase or uppercase words. He was instructed to modulate his attempted loudness at the word level—NORMAL for lowercase words and LOUD for uppercase words in the cued sentence (Fig. 4a and Supplementary Video 1). We used a previously-described brain-to-text decoder19 and a loudness decoder developed for this study to predict phonemes (which were then assembled into words) and loudness levels, respectively. These predictions were then combined to apply uppercase formatting to a decoded word if the loudness decoder predicted LOUD for more than 50% of that word’s duration (Fig. 4b, c). The loudness-integrated speech BCI achieved a loudness classification accuracy of 94%, with a true positive rate of 89% and a false positive rate of 2% (Fig. 4d). We observed that despite the added cognitive and motor demands of modulating speech loudness in the sentence-loudness task, the 3% word error rate achieved was similar to previously reported brain-to-text accuracy for this participant19. However, it remains to be seen whether there are more subtle effects on word decoding if this were to be used when producing more complex sentences during day-to-day independent use. Closed-loop decoding of loudness in a speech BCI. a Schematic of the sentence-loudness task. The participant attempted to speak sentences while modulating loudness levels. Neural activity was fed into a brain-to-text decoder that predicted phoneme logits, which were then assembled into words. Simultaneously, a loudness decoder predicted the attempted loudness level. The phoneme and loudness predictions were combined to apply uppercase or lowercase formatting to the predicted words. b Reconstruction of the participant’s task window and a photograph of T15 performing the task (inset). The decoded words—with loudness modulation applied—appear at the bottom of the task window. c Loudness decoder output and brain-to-text alignment for an example trial. The loudness decoder outputs 0 when the participant attempts to speak at NORMAL loudness level (or remains silent) and 1 when speaking at LOUD loudness level. d Closed-loop loudness level confusion matrix of the loudness-modulated speech BCI. Discussion In this study, we found that vPCG firing rates encode attempted loudness across speech modes—mime, whisper, normal, and loud—in two participants implanted with intracortical microelectrode arrays. vPCG population activity had separable representations of phonemic content and loudness level in different neural subspaces. We also observed preparatory activity related to loudness prior to speech onset. Lastly, we demonstrated the feasibility of decoding loudness levels in real-time by altering the formatting of a text-based neuroprosthesis output. Prior work at coarser spatio-temporal resolution has established that the laryngeal motor cortex plays a central role in controlling voicing and laryngeal articulation during speech in healthy individuals8,11,15,29. Our findings align with prior work and show that spiking activity in vPCG encodes attempted loudness even in individuals with dysarthria. We observed separable encoding of whispered versus overt speech (NORMAL and LOUD conditions) similar to a study that observed fMRI activity in superior and middle regions of primary motor cortex to be significantly higher during voiced (overt) vs whispered speech12. Furthermore, we found that vPCG activity modulated during both overt speech and instructed breathing, consistent with fMRI studies that reported both tasks to engage the precentral gyrus10 and laryngeal motor cortex13, respectively. The presence of preparatory speech loudness- and mode-related activity prior to speech onset further supports vPCG’s role in volitional control of attempted loudness and prosody more broadly. Such preparatory encoding could also facilitate low-latency online decoding such as instantaneous prosodic modulation of synthesized voice20. For speech BCIs, our findings represent a step towards decoding not just what is said, but how the user is saying it. Recent efforts in intracortical speech BCIs have demonstrated the ability to decode intended speech as text from vPCG activity19,21. Speech BCIs have also started to convert neural activity into voice25,27,30–32 and capture some paralinguistic voice features including timing and intonation20. Our study shows that attempted loudness can also be decoded from spiking activity and be used to enrich brain-to-text BCI output. This proof-of-concept demonstrates the early translational potential of decoding loudness from vPCG to modulate the output of a speech BCI, enabling more expressive communication. Limitations and future directions This study investigated how attempted speech mode and loudness are encoded at the level of intracortical spiking activity in two individuals with dysarthria. While our participants could not produce fully intelligible speech, they were capable of varying their vocal effort across loudness levels. Unlike in most previous studies, which involved healthy individuals, here we focused on subjective loudness intent because the range of absolute loudness that each participant could actually produce was impacted by the extent of their paralysis. This intention-centered approach is relevant for our overarching goal of personalizing speech neuroprosthesis output for individual users. We used whispering as a lower loudness condition. However, we recognize that whispering differs fundamentally from softly voiced speech as it lacks vocal fold vibration. In future studies, it will be important to contrast whispered speech from softly voiced speech at similar perceived loudness levels to better isolate loudness-related neural signals from those related to speech mode. Different speech modes and loudness levels could produce distinct auditory feedback, which could influence motor cortical activity33. In our case, participants attempted to vocalize in all conditions except MIME and WHISPER, but their vocal output was substantially reduced due to dysarthria, particularly for T15. As a result, our study likely underestimates the potential contribution of feedback-mediated neural differences across conditions that would be present in healthy speakers. Furthermore, we observed robust neural differences between loudness conditions even before voice onset, suggesting that the reported effects are not strongly dependent on auditory feedback. This study examined loudness and speech mode modulation at the relatively coarse timescale of whole words, and our closed-loop proof-of-concept loudness decoder was limited to binary classification (normal vs. loud speech) for modulating the text formatting in a BCI. Future work should explore more rapidly varying continuous control of loudness and speech mode. Integrating such continuously decoded intent at a more granular level into brain-to-voice neuroprostheses would enable a more expressive neurally generated voice. Modulating voice loudness can complement other approaches, such as Wairagkar et al.20, which showed discrete levels of pitch and emphasis modulation in a brain-to-voice BCI. An important next step will be to determine whether multiple prosodic features, such as loudness, pitch, duration, stress, etc., which often covary, are represented in distinct neural dimensions and whether they can be independently controlled in speech BCI applications. The long-term goal of this line of research is to provide continuous and simultaneous control of multiple prosodic features for more natural neural speech synthesis. Methods Participants and ethics approvals This research was conducted as part of the BrainGate2 clinical trial (ClinicalTrials.gov: NCT00912041). Permission for the trial was granted by the U.S. Food and Drug Administration under an Investigational Device Exemption (Caution: Investigational device. Limited by U.S. federal law to investigational use) and approved by the Institutional Review Boards at Mass General Brigham (protocol #2009P000505), University of California, Davis (protocol #1843264), and Emory University (protocol #00003070). This study includes data from two participants, referred to by their trial identifiers, “T15” and “T16”. Both participants gave informed consent for this research to be conducted and published, including video recordings. Participants were not financially compensated for their participation in the clinical study. Sex determination was self-reported and did not impact study eligibility. At the time of data collection, T15 was a 45-year-old left-handed man with ALS. He had tetraplegia and severe dysarthria. He retained eye and neck movements and had limited orofacial muscle control. He had four 64-electrode Utah arrays (1.5 mm electrode length; Blackrock Neurotech, Salt Lake City, Utah) implanted in his left vPCG; one in area 55b, two in area 6v, and one in area 4 (Fig. 1b). See Card et al.19 for more details. At the time of data collection, T16 was a 54-year-old right-handed woman with a remote pontine stroke. She had tetraplegia and dysarthria. She had four 64-electrode Utah arrays implanted in her left PCG; two in area 6d (hand-knob, which are not analyzed in this study as they did not have speech-related modulation), one in ventral 6v, and one in the border between area 55b and PEF (Supplementary Fig. 1a). For both participants, the implant targets were identified through preoperative MRI scans and alignment of their brains to the Human Connectome Project34 cortical parcellation. Behavioral tasks Each participant performed three types of tasks: a word-loudness task, a sentence-loudness task, and an instructed breathing task. All tasks followed an instructed-delay paradigm. Each trial started with the cue displayed first and a “delay” period (indicated by a red square) of 2 s during which the participant read the cue and prepared for execution. This was followed by the “go” period (indicated by the red square turning green), when the participant attempted to perform the cue. The go-period was a fixed 3 s duration for the word-loudness task for T15. For T16, the first block had a 3 s go-period, but this was increased to 4 s thereafter based on the participant’s preference. For the other two tasks, the go-period duration was variable because the participants ended the trials using an eye tracker after attempting the cue. The cue was present for the entire duration of a trial. Each task was performed during a “session” on a scheduled day. Within each session, participants completed a series of 5- to 10-min “blocks”, during which they performed the task. Word-loudness task In the word-loudness task, the participants were instructed to attempt a specific word at a given loudness level or speech mode (Fig. 1a). The set of words—“be,” “do,” “my,” “know,” “have,” and “going”—was chosen to cover a wide range of phonemes. Each word was attempted in one of four ways: MIME, WHISPER, NORMAL, and LOUD. These speech modes and loudness levels were chosen such that they were subjectively discernible from one another for each participant. This was confirmed by having the participants tell us that they found these conditions intuitive. Both participants were able to perform the task, although their vocal output was reduced due to their dysarthria—especially for T15. For simplicity, we refer to the four conditions as different “loudness levels” and the actual loudness produced was relative to the participant’s own speech abilities. For the MIME condition, the participant was instructed to try to mouth the words silently without producing any sound, as if miming. In the WHISPER condition, they were asked to try to speak softly as if speaking to someone sitting next to them or whisper. For NORMAL, they were instructed to try to speak at a regular conversational volume, and for LOUD, they attempted to speak at a comfortably loud level, as if addressing someone across the room. In addition to these loudness conditions, there was also a “DO NOTHING” condition, in which the participant was instructed to remain silent and not attempt to speak. In total, there were 25 unique conditions (6 words × 4 loudness levels and a “DO NOTHING” condition). For both participants, we collected 25 repetitions per condition. Details about the dataset are provided in Supplementary Table 1. Sentence-loudness task In the sentence-loudness task, the participants were instructed to attempt speaking sentences while modulating the loudness level of individual words within each sentence (Fig. 4a). In a given sentence, each word appeared in uppercase or lowercase, indicating the loudness level to be attempted. The participants were instructed to maintain a desired NORMAL loudness level for lowercase words and LOUD loudness level for uppercase words. Sentences ranging from 3 to 6 words in length were chosen from the Switchboard corpus35. In each sentence cue, uppercase or lowercase formatting was applied to individual words pseudorandomly, with a probability of 0.5. We ensured that across all sentences there was approximately an equal distribution of lowercase and uppercase words. This task data was collected for different purposes across the two participants: (1) closed-loop implementation of the loudness decoder (T15 only) and (2) comparing attempted loudness to instructed breathing (T15 and T16). For the closed-loop loudness-modulated brain-to-text BCI implementation, T15 attempted 300 unique sentences across multiple blocks (25–50 sentences per block), completing a total of 8 blocks in the session. The number of sentences per block was chosen by T15. Sentences decoded by the brain-to-text BCI were displayed on a monitor in closed-loop (see Card et al.19 for details of the brain-to-text BCI). The first 6 blocks were used to train a loudness decoder (200 sentences; details in Section “Online loudness decoder training and closed-loop decoding”), while the final 2 blocks (50 total sentences) were used as evaluation blocks. During evaluation, uppercase or lowercase formatting was applied to the brain-to-text output based on the loudness decoder’s predictions and then displayed on the monitor (details in Section “Online loudness decoder training and closed-loop decoding”). For comparing attempted loudness with instructed breathing, we collected 30 unique sentences (repeated twice) in the same session as the instructed breathing task. In these sessions, the task was performed in open-loop, i.e., no brain-to-text output was displayed on the monitor. A breath belt (ADInstruments, Respiratory Belt Transducer, Model: MLT1132) was attached around the participant’s chest for the entire session to record the change in thoracic circumference due to respiration. The collected dataset is detailed in Supplementary Table 1. Instructed breathing task In the instructed breathing task, participants were prompted to breathe either regularly or deeply (prompt: “Inhale and exhale [NORMALLY, DEEPLY] five times”). For the NORMALLY condition, they were instructed to maintain their baseline breathing pattern. For the DEEPLY condition, they were asked to inhale to their maximum capacity and exhale fully before taking the next breath. These conditions were chosen to elicit breathing at different volumes, allowing for a direct comparison with the NORMAL and LOUD attempted loudness conditions in the sentence-loudness task. We note that both T15 and T16 have difficulties modulating how deep of a breath they can take; nevertheless, they found the task instructions intuitive and felt they could complete the task appropriately. We collected 6 trials per condition from each participant (see Supplementary Table 1 for dataset details). The breath belt was attached around the participant’s chest for the entire session. Data recording and processing Neural data The raw voltage signals from 256 electrodes were collected at 30 kHz and preprocessed using a neural data acquisition system (NeuroPort Neural Signal Processor, Blackrock Neurotech), which filtered (0.3 Hz–7.5 kHz) and digitized the data. The signals were then bandpass filtered (4th order zero-phase non-causal Butterworth filter) between 250 and 5000 Hz. To reduce noise artifacts, Linear Regression Referencing19,36 was applied separately within each array’s 64 electrodes for all arrays. Two neural features, threshold crossings and spike band power, were computed every 1 ms per electrode. Threshold crossings (putative spiking activity) were detected by setting a threshold at −4.5 times the root mean square voltage, and a spike was registered if the voltage in the 1 ms window exceeded this threshold. Spike band power was computed by squaring and then taking the mean of the signal within the same 1 ms window. The neural features (256-dimensional each) were then binned every 10 ms by summing the threshold crossings and averaging the spike band power in the window. Each bin was then log-transformed, normalized using rolling means and standard deviations from the past 10 s, and causally smoothed with a sigmoid kernel spanning 1.5 s of past activity. The resulting 512-dimensional vector of smoothed neural features served as the input to the loudness decoder for offline analyses. For closed-loop loudness decoding, only the smoothed spike band power was used, as offline analyses indicated that comparable performance could be achieved by using only spike band power. All real-time signal processing, feature extraction, and neural decoding were executed using the BRAND framework37, which modularized these components into separate software nodes that ran asynchronously on multiple Linux computers. Behavioral data Microphone and breath belt (when used) signals were recorded simultaneously at 30 kHz along with the neural data using the same NeuroPort data acquisition system. We determined speech onset times using different approaches depending on whether a brain-to-voice speech BCI was available for the participant. For T15, we relied on a brain-to-voice BCI20 that decoded his attempted speech into synthesized voice every 10 ms using the same neural features. The brain-to-voice decoder synthesized voice regardless of the attempted loudness level, producing a speech waveform even for the MIME condition. We then determined speech onsets and offsets by thresholding the amplitude of the synthesized voice waveform. For T16, in the absence of a brain-to-voice BCI, we manually annotated speech onsets and offsets. In trials where T16 vocalized (i.e., WHISPER, NORMAL, and LOUD conditions), we determined them by thresholding the microphone signal after denoising it with Python’s noisereduce v3.0.3 package38. For trials involving unvoiced speech (MIME), where no sound was registered in the microphone, we manually annotated speech onset by reviewing video recordings frame-by-frame and identifying the moment T16 opened her mouth to begin attempted articulation. For the breath-related analyses, we binned the breath belt signal every 10 ms by averaging the values within the window and applied Gaussian smoothing (σ = 40 ms). In the instructed breathing task, we analyzed the five consecutive breaths starting from the first inhalation in the go-period, discarding any extra breaths taken before trial completion. We identified inhalation (peaks) and exhalation (troughs) points in the binned breath signal using SciPy’s find_peaks function. Firing rate analyses To determine whether the spiking activity recorded at each electrode modulated with attempted loudness level (Fig. 1c, Supplementary Fig. 1b and Supplementary Figs. 2–3), we considered the un-smoothed (before normalization and smoothing) binned threshold crossings from each electrode aligned to [-0.1, 1] s around cue onset, [-1, 0.5] s around speech onset and [−0.25, 0.25] s around speech offset. For each electrode, we averaged the binned threshold crossings across all trials corresponding to a given loudness condition in the word-loudness task, then smoothed the trial-averaged activity using a Gaussian filter (σ = 40 ms). To determine if an electrode significantly encodes different loudness levels (Fig. 1d and Supplementary Fig. 1c), we considered the binned threshold crossings (firing rate) from [−0.5, 0.5] s around speech onset. We then compared the firing rates between different pairs of loudness conditions using the one-way ANOVA statistical test with post-hoc Tukey’s Honestly Significant Difference test. We followed a similar procedure to determine if an electrode significantly encoded different words by comparing the firing rates between different pairs of words (Fig. 1e and Supplementary Fig. 1d). Dimensionality reduction analyses To determine if the ensemble neural activity encoded attempted loudness, we performed PCA and demixed PCA39 on the word-loudness task data (Fig. 2 and Supplementary Fig. 5). Here, PCA was used as a standard unsupervised data visualization technique to show the across-electrodes patterns that captured the most variance. dPCA was used as a more detailed and hypothesis-driven targeted dimensionality reduction technique to isolate variance related to different task parameters39. We chose to perform these neural ensemble analyses on spike band power because it reflects the aggregate spiking activity of a number of neurons around each electrode and has been shown to capture more overall task-related neural activity in chronic human Utah array recordings than threshold crossings, likely due to the additional signal provided by detecting smaller neurons23,40. We note that the PCA and dPCA plots resulting from the following analyses, when performed on threshold crossings, looked very similar (not shown); and the threshold crossings and spike band power had correlations (mean ± s.d. across all electrodes) of 0.59 ± 0.17 for T15 and 0.56 ± 0.21 for T16. For PCA, we analyzed the smoothed spike band power from [−750, 750] ms around speech onset from all electrodes. We obtained the trial-averaged activity for each of the 24 word-loudness conditions (256 electrodes × 150 time bins per condition). We then applied PCA to reduce the electrode dimensionality of the condition-averaged spike band power, after flattening the data tensors into 2D matrices by concatenating the time dimension across conditions. The first 3 PCs were used for data visualization, with projections averaged across time bins (Fig. 2a and Supplementary Fig. 5a). For the dPCA analysis, which also reduced dimensionality across electrodes, we used the smoothed spike band power from [−750, 750] ms around speech onset from all electrodes and decomposed the neural activity into 20 components that captured the variance marginalized over different task parameters (Fig. 2b–d and Supplementary Fig. 5b–d). We performed PCA using the implementation from Python’s scikit-learn v1.5.1. and dPCA using the MATLAB package provided by Kobak et al.39. For plotting purposes, we chose the first 15 dPCA components. Empirically, all four (T15) or two (T16) arrays contributed substantially to the PCA and dPCA loadings (not shown). Offline loudness decoder training and analyses For offline loudness decoding (Fig. 3 and Supplementary Fig. 7), we trained multinomial logistic regression models on smoothed neural features from the word-loudness task to predict the four loudness levels. A separate loudness decoder was trained and applied to 400 ms windows of neural features with a 10 ms stride (Fig. 3a and Supplementary Fig. 7a). The decoding accuracy at every time point shown in the plots was obtained from a loudness decoder evaluated on the past 400 ms of neural features. The onset of above-chance decoding accuracy was identified using time-cluster permutation test. To report overall performance, we trained and evaluated loudness decoders using neural features from [−600, 600] ms around speech onset (Fig. 3b–d and Supplementary Fig. 7b–d). All decoders were trained using a 6-fold cross-validation setup, where 5 folds were used for training and the remaining fold for testing. Each fold consisted of trials belonging to a particular word, ensuring that the decoder learned to classify loudness independent of the attempted word. We report the mean and standard deviation performance across all folds. To estimate chance-level performance, we repeated the training procedure 100 times with shuffled class labels. For the electrode dropout analysis, electrodes were randomly sampled uniformly across all arrays for each electrode count. The decoder was trained on neural features from the selected electrodes using the same procedure. This process was repeated 10 times for each electrode count condition, and we report the mean ± standard deviation of the decoding accuracy across these 10 repetitions. For word decoding at every time point (Supplementary Fig. 8), we followed a similar process as that for loudness decoding, except the data was split into 4 folds, where each fold consisted of trials from a particular loudness level. Online loudness decoder training and closed-loop decoding For online closed-loop loudness decoding (Fig. 4), we trained a multinomial logistic regression-based loudness decoder to predict 3 classes: NORMAL, LOUD, and silent. NORMAL and LOUD are the two loudness levels attempted by the participant in the sentence-loudness task. Silent refers to periods when the participant did not attempt to speak and remained silent. Including the silent class helped the decoder predict NORMAL or LOUD loudness level only when the participant attempted to speak. When training the loudness decoder, at the end of each block, we used the phoneme logits output of the brain-to-text decoder and a logit-to-phoneme alignment algorithm to identify the time segments when the participant attempted to speak. The algorithm used a beam search approach to find the most probable alignment of frame-level phoneme probabilities (logits) to the target phoneme sequence (obtained from the cue)41. This allowed us to extract speech onsets and offsets for each word. We then used a 600 ms window of spike band power aligned to each uttered word for training the loudness decoder. For the silent category, we sampled 600 ms windows during periods when the participant was not speaking. We balanced the number of examples across all three categories by resampling as needed before training the final logistic regression model. In closed-loop mode, the loudness decoder predicted attempted loudness from neural data every 20 ms. We applied a confidence threshold of 0.5 for the LOUD class—i.e., a prediction of LOUD was accepted only if the predicted probability for that class exceeded 0.5. As the attempted words were decoded, we learnt their temporal alignment using the logit-to-phoneme alignment algorithm and applied on-screen text display formatting (uppercase for LOUD, lowercase for NORMAL) based on whether LOUD was predicted for ≥50% of the duration of the word (if not, it was displayed in lowercase as NORMAL) (Fig. 4c). We evaluated the performance of the closed-loop system by comparing the formatted predicted text to the cued sentence on a per-word basis to calculate a loudness classification accuracy (Fig. 4d). Breath analysis We quantified breath belt expansion as the difference between the maximum and minimum breath belt values during each breath cycle. To assess whether expansion significantly differed across conditions, we performed a two-sided Wilcoxon rank-sum test (Supplementary Fig. 6b). To investigate whether attempted loudness could be decoded from instructed breath-related neural activity, we trained logistic regression classifiers on neural features from either the instructed breathing task (NORMALLY, DEEPLY) or the sentence-loudness task (NORMAL, LOUD). We considered neural features from [−1.5, 1.5] seconds around exhalation or speech onset, respectively. The models were trained using five-fold stratified cross-validation, with four folds for training and one for testing. Each model was also evaluated on all data from the other task to assess cross-task generalization. We report the mean accuracy across folds. To estimate chance-level performance, we repeated the training procedure 100 times with shuffled class labels. Spike sorting and single-unit analyses To examine task tuning at the level of individual neurons (Supplementary Fig. 4), we performed spike sorting on the bandpass-filtered and denoised neural data using MountainSort542, implemented via the SpikeInterface framework43. Candidate spike events were detected when the signal exceeded –4 times the median absolute deviation of the noise. Detected events were automatically clustered into putative single units using the default MountainSort5 pipeline, followed by manual curation to remove noise clusters and merge duplicate units. To determine if a given neuron was tuned to behavioral conditions, for each neuron, we considered the firing rate from [−0.5, 0.5] s around speech onset. We then compared the firing rates between different pairs of loudness conditions or word conditions separately using the one-way ANOVA statistical test with post-hoc Tukey’s Honestly Significant Difference test. Each neuron was then labeled as tuned to word, loudness, both, or neither, depending on the significance value across the two tests (Supplementary Fig. 4c, d). Reporting summary Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article. Supplementary information Supplementary Information Description of Additional Supplementary Files Supplementary Video 1 Reporting Summary Transparent Peer Review file Source data Source Data 1 Source Data 2 Source Data 3 Source Data 4 Source Data 5 Source Data 6 Source Data 7 Source Data 8 Source Data 9 Source Data 10 Source Data 11 Source Data 12 Source Data 13 Source Data File Captions Acknowledgements We thank participants T15 and T16, their families and care partners for their contributions to this research. This work is supported by the National Science Foundation Research Traineeship NRT program Award #2152260 (A.S.); the A.P. Giannini Foundation Postdoctoral Research Fellowship (N.S.C.); the Office of the Assistant Secretary of Defense for Health Affairs through the Amyotrophic Lateral Sclerosis Research Program under award number AL220043; DP2 from the NIH Office of the Director and managed by NIDCD (1DP2DC021055); Searle Scholars Program; and a Career Award at the Scientific Interface from the Burroughs Wellcome Fund (S.D.S.). Author contributions A.S. led the experiments, analyzed the data, created the figures, and implemented the loudness decoder on the real-time data acquisition system. A.S. and C.I. interfaced with participant T15, scheduled research sessions, and collected the primary data for this study. M.W. developed T15’s brain-to-voice BCI. X.H. developed the logit-to-phoneme alignment algorithm. N.S.C. developed T15’s brain-to-text BCI. B.G.J., A.L.P., and P.H.B. interfaced with participant T16 and collected T16 data for this study. C.P. and N.A.Y. supervised research at Emory University. L.R.H. is the sponsor-investigator of the multisite clinical trial. D.M.B. was responsible for all clinical trial-related activities at U.C. Davis. A.S., S.D.S., and D.M.B. conceived the study and experimental design. S.D.S. and D.M.B. supervised all aspects of the project. A.S., S.D.S., and D.M.B. wrote the manuscript. All authors reviewed and helped edit the manuscript. Peer review Peer review information Nature Communications thanks Gregory Cogan, who co-reviewed with Zachary Spalding and Zachary Freudenburg, Amirhossein Khalilian-Gourtani, and Yuanning Li for their contribution to the peer review of this work. A peer review file is available. Data availability De-identified neural data reported in this study are publicly available on Dryad (10.5061/dryad.2547d7x5w)44. Source data are provided with this paper. Code availability Code to implement the analyses described in this study is publicly available on GitHub (https://github.com/Neuroprosthetics-Lab/srinivasan-speech-modes). Competing interests I.D.E. Caution Statement: CAUTION: Investigational Device. Limited by Federal Law to Investigational Use. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the Department of Veterans Affairs, or the United States Government. The Translational Research Center at Massachusetts General Hospital (MGH) has a clinical research support agreement (CRSA) with Axoft, Neuralink, Neurobionics, Precision Neuro, Synchron, and Reach Neuro, for which L.R.H. provides consultative input. L.R.H. is a co-investigator on an NIH SBIR grant with Paradromics and is a non-compensated member of the Board of Directors of a nonprofit assistive communication device technology foundation (Speak Your Mind Foundation). Mass General Brigham (MGB) is convening the Implantable Brain-Computer Interface Collaborative Community (iBCI-CC); charitable gift agreements to MGB, including those received to date from Paradromics, Synchron, Precision Neuro, Neuralink, and Blackrock Neurotech, support the iBCI-CC, for which L.R.H. provides effort. M.W., S.D.S., and D.M.B. have patent applications related to speech BCI owned by the Regents of the University of California (US patent application no. 63/461,507 and 63/450,317) including intellectual property which has been licensed to a neurotechnology startup. S.D.S. is an inventor on intellectual property owned by Stanford University (US patent no. 12008987) that has been licensed to Blackrock Neurotech and Neuralink Corp. He is an advisor to Sonera and a consultant to Neuralink. D.M.B. is an ad-hoc consultant for Globus Medical Inc. and was an ad-hoc consultant for Paradromics Inc. during the period of data collection for this manuscript, but not at the time of manuscript submission. The remaining authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. These authors contributed equally: David M. Brandman, Sergey D. Stavisky. Contributor Information Aparna Srinivasan, Email: apsrini@ucdavis.edu. David M. Brandman, Email: dmbrandman@health.ucdavis.edu Sergey D. Stavisky, Email: sstavisky@health.ucdavis.edu Supplementary information The online version contains supplementary material available at 10.1038/s41467-026-71284-4. References Associated Data Supplementary Materials Supplementary Information Description of Additional Supplementary Files Supplementary Video 1 Reporting Summary Transparent Peer Review file Source Data 1 Source Data 2 Source Data 3 Source Data 4 Source Data 5 Source Data 6 Source Data 7 Source Data 8 Source Data 9 Source Data 10 Source Data 11 Source Data 12 Source Data 13 Source Data File Captions Data Availability Statement De-identified neural data reported in this study are publicly available on Dryad (10.5061/dryad.2547d7x5w)44. Source data are provided with this paper. Code to implement the analyses described in this study is publicly available on GitHub (https://github.com/Neuroprosthetics-Lab/srinivasan-speech-modes).