An EEG Dataset for Multimodal Semantic Alignment and Neural Decoding during Reading and Listening Authors: Sitong Chen, Beiqianyi Li, Cuilin He, Dongyang Li, Mingyang Wu, Xinke Shen, Song Wang, Xuetao Wei, Xindi Wang, Haiyan Wu, Quanying Liu Published: 2025 ABSTRACT EEG-based neural decoding requires large-scale benchmark datasets. Paired brain-language data across speaking, listening, and reading modalities are essential for aligning neural activity with the semantic representation of large language models (LLMs). However, such datasets are rare, especially for non-English languages. Here, we present ChineseEEG-2, a high-density EEG dataset designed for benchmarking neural decoding models under real-world language tasks. Building on our previous ChineseEEG dataset, which focused on silent reading, ChineseEEG-2 adds two active modalities: Reading Aloud (RA) and Passive Listening (PL), using the same Chinese corpus. EEG and audio were simultaneously recorded from four participants during ~10.8 hours of reading aloud. These recordings were then played to eight other participants, collecting ~21.6 hours of EEG during listening. This setup enables precise temporal and semantic alignment across the RA and PL modalities. ChineseEEG-2 includes EEG signals, speech audio, aligned semantic embeddings from pre-trained language models, and task labels. Together with ChineseEEG, this dataset supports joint semantic alignment learning across speaking, listening, and reading. It enables benchmarking of neural decoding algorithms and promotes brain-LLM alignment under multimodal language tasks, especially in Chinese. ChineseEEG-2 provides a benchmark dataset for next-generation neural semantic decoding. Background & Summary Understanding how the human brain processes language is a longstanding challenge in both neuroscience and artificial intelligence, particularly in how linguistic information is represented across different modalities such as speech and text. Multimodal large language models (MLLM) such as CLAP41, Audiopalm2, and Unified-IO 23 have accelerated research in this area, demonstrated robust cross-modal semantic alignment is the foundation for language modeling, and showed an increased capacity to decode natural language from brain signals4. Many of these models employ contrastive pre-training to establish a shared semantic representational space, which in turn serves as a target for mapping neural signals in brain-to-language alignment. Recent approaches like BELT-25 and Thought2Text6 focus on tasks such as fine-grained brain-to-language alignment, sequential decoding of neural signals, and large-scale cross-lingual or cross-modal knowledge transfer, which rely on extensive token-level aligned multimodal datasets. As the language tasks targeted by MLLMs expand from coarse-grained classification to fine-grained generation and cross-modal inference, the demand for more synchronized multimodal data becomes increasingly urgent. The further development and validation of MLLMs depend on the availability of large-scale datasets that provide tightly aligned neural, acoustic, and linguistic data7. The current scarcity of multimodal and multilingual resources poses a fundamental challenge, as existing datasets are often constrained by a single-modality design or limited linguistic coverage8,9. Most datasets provide only single-modality recordings (e.g., silent reading in ZuCo10,11 or auditory listening in Broderick et al.12, Brennan et al.13), lacking the instance-level correspondence of linguistic content across modalities required for effective cross-modal and multi-task neural learning. More recently developed multimodal datasets, including CineBrain14, Natural Scenes Dataset (NSD)15, and the Narratives dataset16, offer expanded modality coverage, but lack the token-level alignment and multimodal semantic mapping required for speech modeling. Furthermore, nearly all these existing resources are limited to English, leaving a pronounced gap for Chinese and other non-English languages. The available Chinese EEG resources also suffer from fragmented linguistic materials and a confinement to single modalities. For example, the EEG dataset by Tan and Zhang17 focuses on picture-naming tasks, the Chisco18 dataset employs phrase-level imagined speech, and the TMNRED19 dataset uses short-sentence reading. Although these datasets are valuable for probing specific cognitive processes like syllable encoding or semantic priming, their non-naturalistic stimuli and limited temporal scope fundamentally limit their utility for modeling the brain’s continuous language processing. To address this limitation, we introduce ChineseEEG-220, a multimodal EEG dataset including both RA and PL tasks. Using the same linguistic corpus as ChineseEEG21, we recorded EEG and audio data from four participants for approximately 10.8 hours of reading aloud. Their recordings were then played to eight additional participants in a listening task, resulting in 21.6 hours of EEG data aligned to the same semantic content. ChineseEEG-220 offers the first large-scale token-aligned Chinese EEG benchmark supporting comparative studies across reading, speaking, and listening modalities. This dataset supports research in Chinese-specific neural dynamics, such as tonal processing in auditory cortex and character-level semantic activation during reading. Together with the original ChineseEEG21 dataset, ChineseEEG-220 extends token-level semantic alignment to include silent reading, speaking, and listening, all based on the same naturalistic corpus in non-English settings, offering researchers new opportunities for cross-modal neural decoding studies in Chinese language processing. In summary, by providing paired EEG-speech recordings, cross-modal temporal precision, and language-level alignment in a non-English context, ChineseEEG-220 overcomes language and modality barriers, making it an essential resource for cross-modal research and for non-English brain-language decoding. With its ability to align semantic information from brain activity and LLMs, ChineseEEG-220 supports the evolution of the next generation of BCI systems. It opens new possibilities for recent LLM-driven BCI paradigms, such as instance-wise cross-modality mapping, sequential decoding, and multimodal fusion, and helps close the gap between human brain representations and computational semantic models. With its rich multimodal structure and matched linguistic stimuli, ChineseEEG-220 contributes a scalable and culturally diverse resource to the larger goal of human-level language decoding from brain activity. Building on the original ChineseEEG21 dataset, ChineseEEG-220 broadens the possibilities for research. Together, our resources support three main research directions:Cross-modal neural decoding, enabling models to generalize across sensory input (e.g., reading) and output (e.g., speaking) modalities;Brain-to-MLLM alignment, grounding shared semantic representations learned by language-audio models in real neural data;Standardized benchmarking for Chinese language decoding, enabling fair comparisons across decoding models, fusion strategies, and cross-modality training methods in non-English settings. Cross-modal neural decoding, enabling models to generalize across sensory input (e.g., reading) and output (e.g., speaking) modalities; Cross-modal neural decoding, enabling models to generalize across sensory input (e.g., reading) and output (e.g., speaking) modalities; Brain-to-MLLM alignment, grounding shared semantic representations learned by language-audio models in real neural data; Brain-to-MLLM alignment, grounding shared semantic representations learned by language-audio models in real neural data; Standardized benchmarking for Chinese language decoding, enabling fair comparisons across decoding models, fusion strategies, and cross-modality training methods in non-English settings. Standardized benchmarking for Chinese language decoding, enabling fair comparisons across decoding models, fusion strategies, and cross-modality training methods in non-English settings. Methods Participants and Task Overview A total of 12 healthy participants were recruited for the study, ranging in age from 18 to 25 years (4 males, 8 females). All participants were right-handed, had normal or corrected normal vision, and had normal hearing. No participant had a self-reported history of neurological or psychiatric disorders. In the RA task, each participant completed 4 experimental runs. For the PL task, all participants completed 4 runs except participant 4, who experienced an interruption during the second run and therefore completed 5 runs in total. They received monetary compensation in the form of approximately 50 MOP coupons (Macanese Pataca, the official currency of the Macao Special Administrative Region of China) for each experimental run. All participants have given their written informed consent to participate in the research and for their data to be shared. This study complied with the Declaration of Helsinki and was approved by the Institutional Review Board of the University of Macau (Approval No. BSERE20 − APP011 − ICI). Before the formal experiment, all participants underwent a practice session that included prefaces outside the formal chapters to validate the experimental procedures and ensure that the time triggers were synchronized in the audio and EEG recordings. Among the 12 participants, four (2 males and 2 females) conducted the RA task, while the remaining eight conducted the PL task. In the formal experiment, in total, 10.8 hours of reading data (around 3 hours per participant) and approximately 21.6 hours of listening data (around 3 hours per participant) were collected, a total of 32.4 hours of data. Experimental Materials The experimental materials comprised the Chinese translation of Antoine de Saint-Exupéry’s The Little Prince and excerpts from Shen Shixi’s Garnett Dream. These texts were divided into uniform-length passages to standardize reading sessions and facilitate subsequent analysis. The choice of these works as semantic stimuli was primarily driven by the need to maintain compatibility with the existing ChineseEEG21 dataset, ensuring consistency and avoiding discrepancies that could arise from heterogeneous materials. Both texts are exemplary pieces of children’s literature, written in accessible language that predominantly utilizes high-frequency vocabulary, thereby minimizing cognitive load during comprehension. Their rich linguistic diversity and varied syntactic structures provide an ideal foundation for in-depth linguistic and semantic analyses, as well as cross-modal comparisons between reading and auditory processing. Importantly, The Little Prince is recognized as a classic in Western literature, while Garnett Dream holds significant cultural and educational value in Chinese children’s literature. In the experiment, the selected texts were divided into multiple sections. Specifically, The Little Prince consists of 27 chapters, encompassing the complete narrative, while Garnett Dream was divided into 9 sections for participants f1, f2, and m2 in the RA task and participants 01–04 in the PL task, and 8 longer sections for participant m1 in the RA task and participants 05–08 in the PL task. For participant m1 (male) in the RA task, the audio recording arrangement of Garnett Dream was modified to have one fewer section than participant f1 (female) but more overall content, totaling 8 sections. Therefore, participants 05-08 in the PL task provided more EEG data in fewer sections, corresponding to the segments provided by participant m1. To ensure equivalent session durations and control for potential confounding variables related to exposure time, the length of Garnett Dream sections was adjusted to match the total length of The Little Prince. In general, the text materials we used consisted of 46,591 Chinese characters, 21,829 characters from The Little Prince and 24,762 characters from Garnett Dream. Experimental Procedures Participants were seated in an ergonomic chair positioned around 67 cm from a 54 cm × 30.375 cm LCD monitor (1920 × 1080 resolution, 60 Hz refresh rate), replicating the ChineseEEG21 experimental conditions (Fig. 1a). The stimuli consisted of two Mandarin novels: The Little Prince and Garnett Dream, divided as in the Experimental Materials section (Fig. 2), segmented to balance duration (mean run time = 25 ± 3 min) (Table 1). The introduction passages served as practice trials to familiarize participants with the task requirements and minimize potential learning effects.Fig. 1Overview of the experiment and the modalities included in the dataset. (a) The experiment setup. Participants were instructed to sit quietly approximately 67cm from the screen and sequentially read the highlighted text. (b) The subject grouping. the audio from female and male subject 1 in Reading-aloud group were chosen as materials for the two groups of subjects in Passive-listening task. (c) The experimental paradigm. Participants’ 128-channel EEG signals were recorded while reading the highlighted sentence or listening to the audio. (d) Data alignment in ChineseEEG-220. The corresponding audio and EEG segments for the same sentence are converted into embeddings and semantically aligned using the triggers ROWS/ROWE, which indicate the start and end of the sentence.Fig. 2Overview of the experimental pipeline followed by the participants. During the reading phase, four participants independently read two novels while their EEG signals were recorded. In the listening phase, eight participants were divided into two groups (four participants per group), and each group listened to the novel audio recordings by Female Participant 1 and Male Participant 1 while their EEG activity was recorded.Table 1Overview of experiment sessions.TaskParticipantSessionSemantic MaterialChapterNumber of CharactersDurationRAf1, f2, m21The Little Prince1–27218291h32min42s2Garnett Dream1–9215121h29min23sm11The Little Prince1–27218291h32min42s2Garnett Dream1–924,7621h42min51sPL01-041The Little Prince1–7576224min43sGarnett Dream1–3646527min11s2The Little Prince8–14590425min16sGarnett Dream4–5631226min30s3The Little Prince15–24622426min50sGarnett Dream6–7452919min2s4The Little Prince25–27374615min52sGarnett Dream8–9396316min41s05-081The Little Prince1–7576224min43sGarnett Dream1-3984441min16s2The Little Prince8–14590425min16sGarnett Dream4488020min26s3The Little Prince15–24622426min50sGarnett Dream5–6501621min4s4The Little Prince25–27374615min52sGarnett Dream7–8479120min5sTriggerDescriptionBEGNStart of EEG data collection by the EGI deviceSTOPStop collecting EEG dataCHxxStart of each chapter, where xx is the chapter number (e.g., the first chapter is CH01)ROWSStart of a new line of textROWEEnd of a linePRESStart of the preface reading phasePREEEnd of the preface reading phase Overview of the experiment and the modalities included in the dataset. (a) The experiment setup. Participants were instructed to sit quietly approximately 67cm from the screen and sequentially read the highlighted text. (b) The subject grouping. the audio from female and male subject 1 in Reading-aloud group were chosen as materials for the two groups of subjects in Passive-listening task. (c) The experimental paradigm. Participants’ 128-channel EEG signals were recorded while reading the highlighted sentence or listening to the audio. (d) Data alignment in ChineseEEG-220. The corresponding audio and EEG segments for the same sentence are converted into embeddings and semantically aligned using the triggers ROWS/ROWE, which indicate the start and end of the sentence. Overview of the experimental pipeline followed by the participants. During the reading phase, four participants independently read two novels while their EEG signals were recorded. In the listening phase, eight participants were divided into two groups (four participants per group), and each group listened to the novel audio recordings by Female Participant 1 and Male Participant 1 while their EEG activity was recorded. Table 1Overview of experiment sessions.TaskParticipantSessionSemantic MaterialChapterNumber of CharactersDurationRAf1, f2, m21The Little Prince1–27218291h32min42s2Garnett Dream1–9215121h29min23sm11The Little Prince1–27218291h32min42s2Garnett Dream1–924,7621h42min51sPL01-041The Little Prince1–7576224min43sGarnett Dream1–3646527min11s2The Little Prince8–14590425min16sGarnett Dream4–5631226min30s3The Little Prince15–24622426min50sGarnett Dream6–7452919min2s4The Little Prince25–27374615min52sGarnett Dream8–9396316min41s05-081The Little Prince1–7576224min43sGarnett Dream1-3984441min16s2The Little Prince8–14590425min16sGarnett Dream4488020min26s3The Little Prince15–24622426min50sGarnett Dream5–6501621min4s4The Little Prince25–27374615min52sGarnett Dream7–8479120min5sTriggerDescriptionBEGNStart of EEG data collection by the EGI deviceSTOPStop collecting EEG dataCHxxStart of each chapter, where xx is the chapter number (e.g., the first chapter is CH01)ROWSStart of a new line of textROWEEnd of a linePRESStart of the preface reading phasePREEEnd of the preface reading phase Overview of experiment sessions. Reading-aloud task During the RA task, participants were presented with three scrolling lines of text and instructed to read aloud the centrally displayed red-highlighted line (Fig. 1b), which was implemented using PsychoPy v2023.2.322. Instructional text employed ’Open Sans’, while the novel text was displayed in ‘Times New Roman’. The font size was set to 0.05 height units relative to the window height, presented at a rate of 0.25 seconds per character, calibrated to approximate natural reading speed while ensuring clear articulation. The foreground color for the non-highlighted text was defined using PsychoPy’s built-in color name ‘white’ ((255, 255, 255) in standard 0-255 RGB), and the highlighted characters were presented in red by default ((255, 0, 0) in standard RGB). The main row text (highlighted characters) was rendered with full opacity (opacity = 1), while the contextual rows were displayed with reduced opacity (opacity = 0.5) to create a visual hierarchy and reduce distraction from non-focal text. The background color was consistently gray ((128, 128, 128) in standard RGB), which was achieved by the default display behavior when no explicit background color was set in the window configuration for this particular task. The display luminance was adjusted based on participant requirements, within a comfortable range for each individual while ensuring readability of the text stimuli. The materials were segmented with natural dialogue markers (e.g., period punctuation) and syntactic boundaries to preserve semantic coherence. Each trial was followed by a mandatory 20-second interval, with participants controlling progression via spacebar press, and extended one-hour breaks were provided between novels to prevent fatigue. The reading materials were divided into segments based on both content structure and temporal considerations. The Little Prince was organized into 27 content-based segments, while Garnett Dream was partitioned into 10-minute temporal segments, each preceded by a 5-second silent period to facilitate attentional preparation. This segmentation strategy allows for discrete analysis of cognitive processing while maintaining natural reading flow. The complete experimental protocol included one practice session followed by two formal reading sessions per participant, with each session divided into two experimental runs (Session 1: Run 1 - Ch.1-14, Run 2 - Ch.15-27 of The Little Prince; Session 1: Run 1 - Ch.1-5, Run 2 - Ch.6-9 of Garnett Dream for participant f1; Run 1 - Ch.1-4, Run 2 - Ch.5-8 of Garnett Dream for participant m1; where Ch. denotes manually segmented chapters). The session durations were capped at two hours to prevent fatigue-related performance degradation. High-fidelity audio recordings were collected throughout the reading sessions using standardized procedures. These recordings were post-processed using Auphonic and Audacity software for noise reduction and quality optimization. For subsequent experimental phases, gender-balanced stimulus sets were created by selecting recordings of one male and one female participant based on objective acoustic quality metrics, including clarity and articulation precision. The experimental design incorporated multiple safeguards to ensure data quality and participant comfort. The duration of the sessions was capped at 2 hours to prevent fatigue, and the self-paced progression system allowed participants to extend rest periods as needed. The controlled presentation parameters provide a reliable assessment of the reading processes while maintaining participant engagement throughout the study. Passive-listening task The PL task was conducted at the University of Macau under visual display conditions identical to those of the reading task. This task used processed audio recordings from the reading-aloud phase as experimental stimuli. Two representative voice samples (one male and one female) were selected based on objective acoustic parameters, including pronunciation clarity, speech fluency, and recording quality, to serve as standardized auditory stimuli. Eight participants were randomly assigned to the male-voice or female-voice condition (n = 4 per group). The stimuli were organized into four folders per voice condition, containing chronologically segmented audio files from both literary works (Run 1 - Ch.1-14, Run 2 - Ch.15-27 of The Little Prince; Session 1: Run 1 - Ch.1-5, Run 2 - Ch.6-9 of Garnett Dream for participants 01-04, Run 1 - Ch.1-4, Run 2 - Ch.5-8 of Garnett Dream for participants 05-08, where Ch. denotes manually segmented chapters) to ensure balanced exposure across experimental sessions. During stimulus presentation, the primary window is set to full-screen mode with a black background ((0, 0, 0) in standard 0-255 RGB) and a white cross in the center. Participants maintained visual fixation on the central white cross to minimize ocular artifacts in the EEG recordings (Fig. 1b). The display luminance was also adjusted according to participant requirements, within a comfortable range for each individual while ensuring clear visibility of the visual stimuli. Audio playback was handled via PsychToolBox (PTB) audio library for low-latency playback of WAV files. The playback device is system-default; no specific audio hardware was used. The audio volume was adjusted to a comfortable level for each participant, while ensuring clear audibility of the sound stimuli. The protocol incorporated structured breaks including mandatory 10-second inter-segment intervals and self-paced rest periods between experimental runs. Following each segment, participants completed attention monitoring questionnaires that contained both content-specific comprehension questions and subjective attentiveness ratings. Post-session verbal recall tasks were administered to assess memory retention and narrative comprehension. Task engagement was objectively validated through computational linguistic analysis comparing transcripts of participants’ oral recalls with original story embeddings (generated by whisper-large-v3 model23). The significantly higher semantic similarity between participant recalls and original texts compared to randomly generated narratives (Fig. 1c) confirmed appropriate cognitive engagement with the auditory stimuli throughout the experimental sessions. This quantitative validation complemented behavioral measures to ensure data quality and task compliance. EEG Data Collection and Analysis EEG data collection EEG signals were recorded using a 128-channel HydroCel Geodesic Sensor Net from Electrical Geodesics Inc., providing full scalp coverage. Dual acquisition protocols were implemented for both tasks. During the RA task, EEG data were sampled at 250 Hz with sentence-level triggers marking the onset and offset of each text line, synchronized to the presentation rate of 0.25 s/character. For PL sessions, the sampling rate was increased to 1000 Hz. To mitigate the risk of data loss resulting from issues such as trigger misfires or power outages, the EEG recordings during the PL task were divided into two to three segments. The triggers indicated chapter sequences relative to the start of segments, as opposed to the entirety of the novel, which was the case during the reading phase. Signal quality was assured by pre-experimental impedance checks, which maintained all electrodes below 50 kΩ through saline solution replenishment. The raw EEG data were exported to metafile format (.mff) files on the macOS system, preserving the task-specific data (task-reading/lis) and session identifiers (ses-littleprince/ses-garnettdream). EEG data pre-processing The EEG pre-processing pipeline implemented through MNE-Python v1.6.0 (Fig. 3b) followed a neurophysiologically-grounded sequence to ensure signal integrity for language-related analyses. The conversion to BrainVision format was performed using the MNE-BIDS v0.14 Python package to ensure complete compliance with the BIDS specification. Initial data trimming removed non-task intervals, including practice trials from both novels’ prefaces and 5-second silent buffers preceding each chapter. The retained epochs were downsampled to 250 Hz, preserving temporal resolution while reducing computational load. Powerline interference was mitigated via a 50 Hz notch filter, followed by 1–40 Hz bandpass filtering to isolate neurocognitive rhythms: delta (0.5-4 Hz) for sustained attention, theta (4–8 Hz) for semantic integration, alpha (8–12 Hz) for inhibitory control, and beta (12–30 Hz) for predictive processing, targeting the frequency bands most relevant to cognitive processing. Channels exhibiting excessive noise were reconstructed by spherical spline interpolation, preserving topographic continuity across the 128-channel montage. We employed ICA decomposition (FastICA algorithm, 40 components) for artifact rejection, with components manually identified by a human rater based on characteristic patterns. Artifact classification was performed entirely through manual inspection and without applying quantitative thresholds, relying on visual recognition of typical characteristic patterns in the component spatial maps, time courses, and power spectra. Specifically, artifacts were identified based on their distinct characteristics. Ocular components were recognized by a combination of periorbital activity in topographic maps and periodic sharp deflections consistent with blinks or saccades. Cardiac artifacts were identified by periodic peak patterns in their time course that matched the regular heartbeat. Finally, strong orofacial EMG artifacts were typically characterized by maximal activity over the lower frontal scalp (e.g., near electrodes Fp1/Fp2, AF7/AF8 of the 10–20 system) and extended toward the perioral and jaw regions. For details of the ICA components for each participant, please refer to the documentation in our code repository (https://github.com/ncclab-sustech/ChineseEEG-2/blob/main/data_preprocessing/README.md). Residual DC shifts and slow drift artifacts were removed through baseline correction and average re-referencing, normalizing signals across participants for group-level analyses. The fully processed data were stored in BIDS-derivatives format, ready for sensor-space and source-level investigations of semantic processing dynamics.Fig. 3Directory structure of the ChineseEEG-220 dataset and EEG pre-processing pipeline. (a) File structure of the dataset. The file structure encompasses experiment metadata, detailed electrode information (including location, type, and sampling rate), annotations for channels identified as defective during pre-processing, EEG data in BrainVision format, event files containing marker information, records of removed independent components during pre-processing, as well as the original text and embedding files. (b) EEG pre-processing pipeline. The pipeline includes data segmentation, band-pass filtering, interpolation of faulty channels, independent component analysis (ICA) for denoising, and re-referencing. Directory structure of the ChineseEEG-220 dataset and EEG pre-processing pipeline. (a) File structure of the dataset. The file structure encompasses experiment metadata, detailed electrode information (including location, type, and sampling rate), annotations for channels identified as defective during pre-processing, EEG data in BrainVision format, event files containing marker information, records of removed independent components during pre-processing, as well as the original text and embedding files. (b) EEG pre-processing pipeline. The pipeline includes data segmentation, band-pass filtering, interpolation of faulty channels, independent component analysis (ICA) for denoising, and re-referencing. Temporal Alignment of EEG, Text, and Audio Sequences We synchronized the EEG data and the corresponding text or audio stimuli using a temporal synchronization protocol. Trigger alignment was implemented through: (1) hardware-level trigger synchronization and (2) character-level refinement of temporal markers (Fig. 1d). During data acquisition in both RA and PL tasks, triggers were embedded in the EEG recordings time-locked to the onset and offset of each presented stimulus, marking the exact start and end of each text line. These triggers enabled initial alignment of EEG recordings with the stimulus. The character-level temporal mapping was achieved using the fixed presentation rate of 0.25 seconds per character. In the case of textual stimuli, temporal alignment was achieved through a direct comparison of individual character timestamps with the corresponding markers in the EEG data. For auditory stimuli, the temporal localization of each character was accomplished by determining its onset time and performing synchronized identification across both the EEG recordings and the audio signal. Data Records The complete dataset is publicly accessible at Science Data Bank20. The data modalities included in the structure of the ChineseEEG-220 dataset are shown in Fig. 3a, including raw data and derivatives. Raw data contains raw EEG, raw text, and audio materials, and derivatives contain pre-processed EEG data, audio, and text embeddings generated by model Wav2Vec224 and Bert-base Chinese25. The dataset adheres to the EEG-BIDS (Brain Imaging Data Structure) specification, which extends the standard BIDS framework for electroencephalography (EEG) data. As depicted in Fig. 3a, under the root directory are the folders Reading aloud and Passive Listening, respectively containing the EEG data and available participant information in the reading aloud task and in the listening task. The data of each participant are encapsulated in a compressed archive (.zip), containing standardized EEG file formats: .vhdr (header), .vmrk (marker), and .eeg (raw signal). The EEG data are organized into two session-specific subfolders, ses-littleprince and ses-garnettdream, corresponding to the two semantic materials. The EEG data for all participants are separated by chapter for efficient processing and analysis. To facilitate analysis, the EEG recordings are segmented with run numbers encoded as runXY, where X is the first number and denotes the run index, Y is the chapter sequence within that run. Note that chapter numbers are counted continuously across runs, therefore, the actual chapter index in the book is obtained by adding Y from run 2 to the total number of chapters from run 1 (e.g., run112 = chapter 12 in run 1; run21 = chapter 15 in run 2). The derivatives folder contains processed EEG data and filtering results, all structured the same as the raw data. The filtered folder contains band-pass filtered EEG data within the frequency range (1–40 Hz). These filtered data files enable focused analyses on different neural oscillatory activities associated with linguistic tasks. Each participant has an individual subfolder located under preprocessed, which contains EEG data that have undergone our pre-processing pipeline. Within the root directory, the materials&embeddings folder contains the text stimuli from the experiment and the generated embeddings. Original novels are preserved in .txt format, while embeddings of audio and text materials are preserved in .npy files. We publicly release audio embeddings. To protect participant privacy, the audio embeddings cannot identify individuals. A README file is located under the root folder, which covers details of this dataset. We note one data issue: for participant 05, a 10-minute trigger was missing from the EEG data for run 213 of the PL task, corresponding to chapter 27 of The Little Prince. Consequently, this segment could not be aligned with the semantic material at the word level. Despite this issue, the segment was processed identically to all other runs, as our pre-processing steps (e.g., ICA decomposition, baseline correction, and average re-referencing) rely solely on continuous EEG data and are not dependent on event markers. This incident was documented to ensure transparency and to acknowledge the data gap in subsequent analyzes. Technical Validation Inter-subject Correlation Analysis To assess whether the recorded EEG signals reflect neural responses to the linguistic stimuli shared across subjects, we performed inter-subject correlation (ISC) analysis at the electrode level across four canonical frequency bands: delta, theta, alpha, and beta. For each frequency band and each subject i, we computed the correlation of each pair of electrodes between subject i and all other subjects k, where k ∈ {1, …, n} and k ≠ i. n is the total number of subjects (Fig. 4a). To determine whether the observed correlations were significantly above chance, we constructed the baseline correlation using a circular time-shift shuffle procedure26. Specifically, for each electrode, the EEG time series was segmented at a random time point and temporally shuffled by concatenating the latter segment before the former, thereby disrupting temporal alignment while preserving spectral properties. This procedure was repeated 500 times for each subject pair and electrode, and the average correlation coefficients across all iterations, electrodes, and subject pairs were used as the baseline (Fig. 4a). Fig. 4Inter-subject correlation analysis of EEG data. (a) Schematic illustration of the validation procedure comparing electrode-wise correlation coefficients between pairs of subjects, with statistical testing against a shuffled baseline. (b) Topographic maps showing ISC coefficients across electrodes for the δ, θ, α, and β frequency bands during the RA and PL tasks. Warmer colors indicate higher ISC values. (c) Boxplots of correlation coefficients for each frequency band compared to the shuffled baseline, demonstrating significantly higher actual correlations (One-sample t-test, **p < 0.01, ***p < 0.001), validating the neural plausibility and authenticity of the EEG signals. Inter-subject correlation analysis of EEG data. (a) Schematic illustration of the validation procedure comparing electrode-wise correlation coefficients between pairs of subjects, with statistical testing against a shuffled baseline. (b) Topographic maps showing ISC coefficients across electrodes for the δ, θ, α, and β frequency bands during the RA and PL tasks. Warmer colors indicate higher ISC values. (c) Boxplots of correlation coefficients for each frequency band compared to the shuffled baseline, demonstrating significantly higher actual correlations (One-sample t-test, **p < 0.01, ***p < 0.001), validating the neural plausibility and authenticity of the EEG signals. Figure 4b displays the scalp topography of ISC coefficients. For the RA task, we observed clear activations in frontal and temporal areas, consistent with the functional demands of this semantic reading task27. For the PL task, while the topography is broadly distributed, the scalp topography showed a consistent pattern of delta band (1–4 Hz) neural activation that was most prominent over the temporal and occipito - parietal regions, which is consistent with both the engagement of auditory - linguistic and visuospatial processing systems during narrative comprehension and previous neurological studies of low frequency entrainment to speech28,29. Figure 4c shows the boxplot of ISC from each band. Paired-sample t-tests revealed that authentic ISC values were significantly higher than baseline correlation in all frequency bands (One-sample t-test, ps < 0.001), confirming that EEG signals contain neural information related to shared stimuli, supporting the validity and quality of the dataset. EEG Source Localization and Cross-modal Analysis The 128-channel EEG data supported source localization analyzes. All source-level analyses were performed with MNE-Python, following this pipeline: (1) Cortical surface reconstruction using the fsaverage MRI template30; (2) Source space construction (10,242 sources/hemisphere) via Connectome Mapper 331; (3) Implementation of a 3-layer Boundary Element Method (BEM) model (15,360 triangles; conductivities: brain = 0.3 S/m, skull = 0.006 S/m, scalp = 0.3 S/m) following the ChineseEEG21 protocol; and (4) Reconstruction of EEG sources using dynamic Statistical Parametric Mapping (dSPM), selected for its established reliability in EEG source imaging. The visualization of the source activities corresponding to each word in the example sentence is provided. The result of source localization demonstrates a more focused activation near the left middle temporal gyrus in subjects in the RA task, which is related to language comprehension27. For subjects in the PL task, it is observed that activation areas are more dispersed. Activation areas included not only the anterior temporal lobe and the temporal-parietal region, which are associated with language comprehension and primary processing32, but also the prefrontal region, which is associated with visual processing. The potential cause may be that the subjects in the PL task are less focused, as there is less engagement compared to the RA task. Furthermore, participant task engagement in PL task was guaranteed by measuring the similarity of the embeddings of the transcript of the collected audio where the subjects recalled the story in their own words and the embedding of the original story. The transcripts were generated using the whisper-large-v3 model23. For both semantic materials, the average similarity of the audio of the subjects is above the story randomly generated by the natural language model using words that appear frequently as in Fig. 1c. In general, the result shows that ChineseEEG-220 contains data where neural activations respond to the given semantic materials. To verify task-induced activation of language networks, we conducted source analysis in language-related regions guided by Lausanne2018.scale233, with language-specific ROIs identified per Bizley’s auditory-language mapping34. For details, please refer to our data analysis documentation (https://github.com/ncclab-sustech/ChineseEEG-2/blob/main/data_analysis/README.md). The analysis focused on beta-band oscillatory activity (14–30 Hz), extracted through the Hilbert transform of bandpass-filtered signals. Cross-modal correlation analysis was performed by computing the pairwise similarity of amplitude envelopes between the RA and PL task groups at each cortical parcel. For each group, individual amplitude envelopes were averaged within groups to obtain representative group-level signals, then pairwise correlations were calculated for each brain region across the two task groups. As shown in Fig. 5b, the left superior temporal gyrus and the left middle temporal gyrus, both related to language processing, exhibited stronger cross-task correlations than between the left superior temporal gyrus and the left pericalcarine cortex, a primary sensory area35 (Steiger’s Z-test, p < 0.05). The result was obtained after excluding two participants (03, 04) with excessive artifacts. The correlation between task-activated regions substantiates the dataset’s sensitivity to language-related neural dynamics.Fig. 5EEG Source-level analysis. (a) Source localization during the RA task. EEG sensor-level data from the RA task were localized using dynamic Statistical Parametric Mapping (dSPM)43, consistent with ChineseEEG21 dataset. The reconstructed source maps corresponding to the time window marked by the highlighted sentence were presented. (b) Cross-modal correlation analysis. The correlation coefficients between the neural representations in reading-aloud task and passive-listening task were illustrated. (c) Task-dependent activation of language regions. Average activation levels were computed for language-relevant regions (blue) and control regions (orange) across multiple subjects in both the RA and PL tasks. Activation in language regions was significantly higher during the reading-aloud task compared to control regions (Welch’s t-test, p < 0.001). EEG Source-level analysis. (a) Source localization during the RA task. EEG sensor-level data from the RA task were localized using dynamic Statistical Parametric Mapping (dSPM)43, consistent with ChineseEEG21 dataset. The reconstructed source maps corresponding to the time window marked by the highlighted sentence were presented. (b) Cross-modal correlation analysis. The correlation coefficients between the neural representations in reading-aloud task and passive-listening task were illustrated. (c) Task-dependent activation of language regions. Average activation levels were computed for language-relevant regions (blue) and control regions (orange) across multiple subjects in both the RA and PL tasks. Activation in language regions was significantly higher during the reading-aloud task compared to control regions (Welch’s t-test, p < 0.001). Usage Notes The code for the experiment and data analysis has been uploaded to GitHub (https://github.com/ncclab-sustech/ChineseEEG-2) to facilitate sharing and use. The code repository contains five main modules, including scripts for preparing the materials, reproducing the experiment, data processing, data embedding, and data analysis procedures. The script cut_chinese_novel.py in the novel_segmentation folder contains the code to prepare the stimulation materials from the source materials. Both files littleprince.xlsx and garnettdream.xlsx record the sections of the materials, with garnettdream.xlsx containing the end of each segment for participants f1 and m1 in the RA task. The script ui_experiment.py contains code for the RA task experiment and the recording.py enables trigger sending, which should be guaranteed to be initiated first in the experiment. The script experiment_listen.py contains code for the PL task experiment. The script preprocessing.py in data_preprocessing folder contains the main part of the code to apply pre-processing on EEG data and conversion to BIDS format. The script text_embed.py in the embeddings folder contains code to generate embeddings for semantic materials, while the script audio_embed.py is for generating audio embeddings. The script isc_analysis.py contains necessary code for ISC analysis, and the script source_analysis.ipynb is for calculating forward and inverse solutions to reconstruct the source space. The code for EEG data pre-processing is highly configurable, permitting flexible adjustments of various pre-processing parameters, such as data segmentation range, downsampling rate, filtering range, and choice of ICA algorithm, thereby ensuring convenience and efficiency. Researchers can modify and optimize this code according to their specific requirements. Before using our ChineseEEG-220 dataset, we encourage all users to check README.md and the updated information in the GitHub repository.