Sign language narrative reveals universal and modality-specific features of cortical timescale hierarchy Authors: Maria Zimmermann, Piotr Tomaszewski, Artur Marchewka, Marcin Szwed, Marina Bedny Published: 2026 ABSTRACT Human languages are unique in their capacity to tell stories. Prior neuroimaging studies show that spoken language narrative comprehension engages a neural time-scale hierarchy, from words and sentences to narratives. Sign languages share core features with spoken languages but also have modality-specific elements such as use of hands, face and spatial position to convey meaning and mark grammar. Here, we use naturalistic fMRI to examine which aspects of the language hierarchy are speech-specific versus modality-general. Native signers (n = 20) watched a 20-minute Polish Sign Language interpretation of The Fall of the House of Usher and its progressively scrambled versions during fMRI. We show that signed narratives synchronize hierarchical neural networks, revealing a modality-invariant dissociation between frontotemporal regions, which track sentence-level structure, and the Default Mode Network tracking plot. We identify two signatures not previously reported for speech: the superior parietal cortex synchronizes at long discourse timescales, while the ventral occipital cortex responds to word- and sentence-level processing. These results reveal both universal and potentially modality-specific aspects of language neurobiology. Introduction Human languages are unique among communication systems in their ability to convey complex narratives or stories1. Understanding stories engages a hierarchy of neural networks that encode progressively complex representations, from perceptual symbols, to words, sentences and the narrative plot2–4. Thus far, neuroimaging studies of linguistic narratives have focused exclusively on spoken languages5,6. However, hundreds of distinct sign languages worldwide employ the visuo-manual modality, using the hands and face to convey meaning and grammar7–9. Sign languages provide unique insight into which neural signatures of language are shared across modalities and which are modality specific. Like spoken languages, sign languages have large lexicons, use generative grammatical rules, morphological structure, word order, and convey rich, structured narratives10–12. At the same time, sign languages also exhibit consistent modality-specific features, including the use of hands, face, eye gaze, head tilt and torso position to convey meaning. All sign languages also make use of space to encode meaning and grammatical relations13–16. The linguistic use of space operates across all levels of representation: from phonology and morphology to syntax and discourse. For example, at the discourse level, spatial locations can be used as pronouns to bridge across sentences in a narrative17,18. A signer points to a spatial location marking it as a pronoun indexing one of the characters in the story. Pointing to the same location later in the story references that character19–21. Neurobiologically, sign and spoken languages also share core features. Prior work suggests that like spoken languages, signed languages depend on a left-lateralized frontotemporal neural network22–34. For both signed and spoken languages, left inferior frontal and lateral temporal cortices are active when processing words and the grammatical structure of sentences29,35–38. Damage to these regions leads to aphasia in signers39–41. Prior studies also suggest that sign languages may have distinctive neurobiological signatures25,32. One prominent example is enhanced recruitment of visuospatial parietal networks, including the superior parietal lobule (SPL), possibly due to the spatial nature of sign languages25,27,42. However, evidence of SPL recruitment comes from studies using isolated words or sentences, often involving tasks such as lexical decision or picture matching43. It remains unclear whether superior parietal contributions generalize to naturalistic language use or emerge primarily with artificial experimental tasks. In the current study we complement prior work by using a naturalistic narrative approach, providing a bird’s eye view of the neural networks that support sign language comprehension in a naturalistic context. This approach enables investigation of processing across linguistic levels, from perceptual form to lexical and syntactic structure, up to the narrative or plot level, where meaning is integrated over long timescales. We investigated the hierarchical flow of neural activity during naturalistic sign language narrative comprehension, from sensory cortices to frontotemporal language areas and then to higher-order semantic networks including Default Mode Network. Prior work with spoken language narratives finds neural separation between sentence-level and story-level processing, and we tested for such a separation in sign languages. Second, we ask whether sign languages recruit distinct modality-specific patterns of brain activity, especially in parietal regions involved in spatial processing, and at which level of the hierarchy such modality-specific effects emerge: words, sentences, or narratives. Since sign language discourse engages unique spatial referential strategies, this could lead to engagement of parietal spatial systems at the discourse level. To address these questions, we use three complementary approaches to dissociate levels along the language processing hierarchy. The first approach leverages inter-subject correlation (ISC). When multiple participants watch the same narrative, cortical regions that process the narrative show high correlation of activity across participants (ISC)44,45. To dissociate different levels of processing, ISC is compared across intact and scrambled versions of the narrative. Cortical regions that support higher-order integration of narrative plot are highly synchronized by the intact narrative and show a steep drop in synchrony when the narrative is disrupted by scrambling the sentences46,47. Cortical networks which are sensitive to within-sentence structure, show relatively preserved synchrony when the order of sentences is shuffled and a fall-off with scrambling of individual words within a sentence. At the lowest level of the hierarchy, early perceptual regions are insensitive to scrambling44,45. Cortical areas at the apex of the cognitive hierarchy are said to have a long ‘temporal receptive window’ (TRW) because they synchronize only for stimuli with coherent structure at longer timescales, whereas lower processing areas have a shorter TRW. To complement ISC analysis, inter-subject functional connectivity analysis (ISFC) measures correlations across cortical areas across participants48. Rather than assessing correspondence between spatially matched voxels across brains, ISFC relates the activity of a given region of interest in one participant to the rest of the cortex in other participants. This approach makes it possible to identify large-scale cortical systems that respond in a coordinated, network-like manner to shared narrative input. Finally, we use a data-driven event segmentation approach to measure the temporal dynamics of cortical activity during the narrative49. Using Hidden Markov Models (HMM), we divide the fMRI signal into temporally coherent segments based on patterns of brain activity during narrative processing. This method allows us to estimate the timescale over which different brain networks accumulate and integrate information during the narrative, i.e., how long each network maintains a stable representation before shifting to the next. Since data-driven segmentation uses only narrative data, it is complementary to comparisons across scrambling levels. Together these complementary approaches allow us to identify the neural hierarchy engaged by sign language narrative comprehension. We show that sign language narratives engage a cortical hierarchy of temporal integration spanning visual, language, and default mode networks. We further show that ventral occipital and superior parietal regions make distinct contributions to sign language processing, with parietal cortex specifically involved in discourse-level comprehension. Results Deaf native signers of Polish Sign Language (PJM) (n = 20) passively viewed a PJM interpretation of Edgar Allan Poe’s The Fall of the House of Usher, as well as progressively less meaningful, scrambled versions of the same narrative (Fig. 1.). In the sentence-scrambled condition, sentences were presented in a shuffled order, disrupting discourse while preserving grammatical structure. In the word-scrambled condition, short segments of one to three words (1–3 seconds long segments) were presented, disrupting sentence-level grammar but preserving lexical information. In the control pseudo-sign condition, participants watched a meaningless story, so even single-word meaning was disrupted. To minimize order effects, the scrambled conditions (words, sentences, pseudowords) were presented first, in a counterbalanced order across participants. The intact narrative was shown after scrambled conditions. This ensured that participants had not yet been exposed to the full story when viewing the control stimuli, and that their interpretation of the scrambled materials was not influenced by narrative context.Fig. 1Study design.Participants watched a story in Polish Sign Language as well as three scrambled versions of the same story. The scrambled conditions were presented first, in an order counterbalanced across participants. To create the scrambled stimuli, a 10-minute segment of the story was first divided into individual sentences and then further segmented into individual signs by a native signer. In the sentence condition, single sentences from the story were presented in a shuffled order at the same speed as in the original narrative. In the word condition, individual signs or short sequences of signs (1–4-second segments) were presented in a shuffled order. In the pseudo-sign condition, participants watched a meaningless story composed of pseudo-signs. The actor granted permission for the publication of identifiable images of his face. Study design. Participants watched a story in Polish Sign Language as well as three scrambled versions of the same story. The scrambled conditions were presented first, in an order counterbalanced across participants. To create the scrambled stimuli, a 10-minute segment of the story was first divided into individual sentences and then further segmented into individual signs by a native signer. In the sentence condition, single sentences from the story were presented in a shuffled order at the same speed as in the original narrative. In the word condition, individual signs or short sequences of signs (1–4-second segments) were presented in a shuffled order. In the pseudo-sign condition, participants watched a meaningless story composed of pseudo-signs. The actor granted permission for the publication of identifiable images of his face. Language- and non-language-preferring regions are intermingled in frontotemporal cortex and vary across individuals, motivating the use of subject-specific language networks50. Recent work shows that sentence- and narrative-level stimuli can be used to define such networks in sign languages32. Accordingly, we localized the language-responsive regions in each participant using a separate PJM language localizer, modeled after prior studies in English and American Sign Language31,50. Participants watched sentences in Polish Sign Language and a perceptually matched control condition consisting of backward sentences overlaid upon each other, thus containing faces and movement but not interpretable as language(see Methods). Inter-subject synchrony reveals hierarchical levels of narrative comprehension in sign language The PJM narrative evoked bilateral inter-subject synchrony throughout the typical narrative-comprehension hierarchy, ranging from visual perception regions (V1, lateral and ventral-occipitotemporal cortex, (vOTC)), the frontotemporal language network and the DMN (TPJ and precuneus)51,52. Additionally, synchrony in parietal cortex expanded superiorly into superior parietal lobule (SPL), not typically observed for spoken or written narratives (Fig. 2).Fig. 2Whole brain inter-subject correlation maps.shown separately for each condition. ISC was calculated using a leave-one-subject-out approach. Statistical significance was assessed using nonparametric permutation tests. Maps show Fisher z–transformed ISC values, thresholded at p < 0.01 (voxel-wise FDR corrected) with a minimum cluster size of 30 voxels. Whole brain inter-subject correlation maps. shown separately for each condition. ISC was calculated using a leave-one-subject-out approach. Statistical significance was assessed using nonparametric permutation tests. Maps show Fisher z–transformed ISC values, thresholded at p < 0.01 (voxel-wise FDR corrected) with a minimum cluster size of 30 voxels. Frontotemporal regions typically associated with language processing were synchronized by sign language across scrambling levels: intact narrative, sentences, and words, with little synchrony for the meaningless pseudo-sign condition. By contrast, the DMN network, including precuneus and TPJ, was uniquely synchronized by the intact narrative but not by shuffled sentences or words. The SPL was also uniquely synchronized by the intact narrative, with no significant synchrony for sentences, words, or pseudo-signs. A direct whole-cortex comparison between narrative- and sentence-level processing revealed clusters in the DMN (precuneus and TPJ), the superior parietal lobule (SPL) and supramarginal gyrus (SMG) as well as smaller clusters in the ATL and occipital cortices (see Supplementary Fig. S2). We next defined individual-subject functional language ROIs using language>control contrast in a separate localizer experiment (see Methods). This functionally defined frontotemporal language network showed a significantly different synchrony pattern from the DMN regions (condition x ROI type interaction: F(3, 893) = 5.14, p = 0.001) and from the SPL (condition x ROI type interaction: F(3, 1093) = 6.54, p < 0.001) (Fig.3B). By contrast, there was no statistically significant difference in synchrony pattern among frontotemporal language network ROIs (interaction of ROI and condition (F(3, 893) = 0.95, p = 0.535), or between VOTC and frontetemporal language network ROIs (condition x ROI type interaction F(3, 1093) = 1.01, p = 0.380).Fig. 3Comparison of levels of processing.A Narrative-, sentence- and word-level processing on the whole brain level. Green represents voxels that are significantly synchronized uniquely for the narrative level (p<.01 FDR corrected). Purple represents voxels which are significantly synchronized for narrative and sentence level (overlap). Orange represents voxels significantly synchronized for narrative, sentences, and words. B Normalized ISC values within ROIs in the left hemisphere. For the right hemisphere ROIs results see Supplementary Fig. S2. The ISC data are provided in source data file. Comparison of levels of processing. A Narrative-, sentence- and word-level processing on the whole brain level. Green represents voxels that are significantly synchronized uniquely for the narrative level (p<.01 FDR corrected). Purple represents voxels which are significantly synchronized for narrative and sentence level (overlap). Orange represents voxels significantly synchronized for narrative, sentences, and words. B Normalized ISC values within ROIs in the left hemisphere. For the right hemisphere ROIs results see Supplementary Fig. S2. The ISC data are provided in source data file. Next, we performed a temporal receptive window analysis (TRW). Regions are said to have a long ‘temporal receptive window’ (TRW) when they synchronize only for stimuli with coherent structure at longer timescales i.e., intact narrative. Previous studies of spoken narrative processing find that the slope of synchrony fall-off increases along the processing hierarchy, even though many cortical areas are more synchronized by narratives than unconnected words45,53. Regions at the apex of this hierarchy, involved in higher-order narrative comprehension, show the steepest slope, with strong synchrony for coherent narratives and sharp decrease in synchrony with progressive scrambling. Consistent with previous results with spoken narratives, regions of the frontotemporal language network, as well as VOTC, showed an intermediate slope, ranging from 0.31 to 0.59 (individual-subject functional ROIs language>control). By contrast, the DMN, including the PC and TPJ, showed a steeper slope of synchrony fall-off (TRW left = 0.77, TRW right = 0.73) (normalized to range from 0 to 1, see Methods). These results suggest that frontotemporal language and DMN networks dissociate for sign language narratives with the DMN at the apex of the processing hierarchy. The SPL showed the steepest fall-off slope of all the ROIs examined (TRW left = 0.86, TRW right = 0.79). Like the DMN, the SPL is sensitive to narrative structure (Fig. 3). Data-driven HMM analysis uncovers processing hierarchy of sign language narratives Here we apply a data-driven event segmentation approach (HMM) to examine the duration of neural events during the intact 20-min narrative. This analysis offers an independent way to compare processing timescales across networks, without comparing scrambling levels. The HMM estimates neural event durations in each network. While the absolute estimates of event duration are not highly interpretable, the relative duration of events across networks provide insight into their position in the processing hierarchy, with apex regions having longer processing events (Fig. 4A).Fig. 4Event segmentation using Hidden Markov Models (HMM).A The event segmentation models have been fitted to the BOLD response to watching the entire narrative. The illustration was prepared by the first author in Adobe Illustrator. B The parcels colored by corresponding preferred event length as revealed in the HMM analysis. Event segmentation using Hidden Markov Models (HMM). A The event segmentation models have been fitted to the BOLD response to watching the entire narrative. The illustration was prepared by the first author in Adobe Illustrator. B The parcels colored by corresponding preferred event length as revealed in the HMM analysis. HMM models significantly fit the data in all left and right hemisphere anatomically defined ROIs (language network, DMN, SPL, VOTC, V1), indicating reliable shared temporal structure across individuals during PJM narrative comprehension (p < 0.01, permutation tests). Consistent with their role in perceptual processing, early visual areas (e.g., V1) showed the shortest event segments (median number of events estimations: 50-60 events, ~15 s). Bilateral frontotemporal language ROIs showed intermediate length events (25-30 events, ~35 s) with longer events in the anterior temporal cortex (18- 20 events, ~50 s) (Fig. 4B). The bilateral DMN ROIs showed long events (17–19 events ~50 s), significantly longer than all ROIs in the language network (mean difference = 5.9, 95% CI = [3.48, 7.11], p < 0.001, permutation test) except the anterior temporal cortex, which was not significantly different from the DMN ROIs: precuneus and TPJ (mean difference = 1.52, 95% CI = [−0.43, 3.47], p = 0.401, permutation test). Consistent with the idea that Superior Parietal Lobule (SPL) is involved in higher-order narrative processing, it showed the longest temporal segments (17 events, ~60 seconds), possibly corresponding to narrative events in the story that span across multiple sentences. SPL showed significantly longer event length relative to all the frontotemporal language network ROIs (mean difference = 7.2, 95% CI = [3.81, 10.43], p < 0.001, permutation test) except the longest event length within the left Anterior Temporal Cortex (mean difference= 1.06, 95% CI = [−1.05, 3.16], p = 0.302, permutation test). Although the vOTC patterned with the frontotemporal language network in terms of sensitivity to scrambling, HMM analysis revealed relatively short events (36-44 events ~40 s), significantly shorter than those in frontotemporal regions (mean difference = 4.1, 95% CI = [0.47, 7.78], p = 0.027, permutation test). VOTC engagement in processing at a shorter temporal scale may reflect form-based visual processing relevant to sign-language comprehension (Fig. 4). The above event-segmentation method uses between-participant multivoxel similarity and therefore standard space anatomical ROIs. To capture individual differences in language region localization, we repeated our analyses using individually defined frontotemporal language ROIs using the Shared Response Model to align functionally corresponding voxels and observed largely similar results (Chen et al., 2015). In functionally defined frontotemporal language ROIs, event lengths remained intermediate between DMN, SPL on the one hand and visual regions on the other. Event length was consistent across parcels and hemispheres (all p < 0.05, permutation test). The time-scale difference between posterior language regions and anterior temporal regions seen with anatomical ROIs (mean difference = 13.66, 95% CI = [10.23, 17.09], p < 0.001, permutation test) was not observed when language-specific left posterior and anterior temporal ROIs were defined (mean difference = 4.3, 95% CI = [−0.48, 9.12], p = 0.085, permutation test). Therefore, those parts of the anterior temporal lobe that are part of the core language processing network are not different from the rest of that network. Anterior temporal cortex may contain distinct functional regions, some of which pattern with the default mode network, and others are part of the core sentence-processing language network. SPL, DMN and frontotemporal language regions show distinct inter-subject functional connectivity (ISFC) We used inter-subject functional connectivity (ISFC) to distinguish among networks involved in narrative processing. Regions with similar functions tend to show correlated activation over time, whereas regions that belong to different functional networks may show uncorrelated or anticorrelated activity, furthermore, inter-region correlations can be affected by stimulus scrambling, revealing sensitivity to stimulus structure52,54,55. ISFC analysis is complementary to measures of inter-subject synchrony (ISC), since operating over a similar temporal scale (i.e., ISC) does not necessitate functional synchrony across regions. The analysis revealed a clear dissociation among three networks engaged in processing sign language narratives: the frontotemporal network, the Default Mode network (DMN), and the SPL. The left MTG seed, part of the frontotemporal language network, showed significant connectivity with other components of the frontotemporal network bilaterally, including the bilateral middle temporal gyrus (MTG) and inferior frontal gyrus (IFG), as well as with the ventral occipitotemporal cortex (vOTC) (Fig. 5). The connectivity profile of the frontotemporal network remained consistent across narrative and sentence-level scrambled conditions (Supplementary Fig. S3).Fig. 5Inter-subject functional connectivity analysis.The analysis was performed using seeds in the frontotemporal language network, the precuneus, and the superior parietal lobule (SPL) in the left hemisphere. Inter-subject functional connectivity (ISFC) was computed by correlating each subject’s seed time series with the average whole-brain response of all other subjects (leave-one-out approach). Maps show Fisher z–transformed ISFC values, thresholded at p < 0.05 (voxel-wise FDR corrected) with a minimum cluster size of 30 voxels. Inter-subject functional connectivity analysis. The analysis was performed using seeds in the frontotemporal language network, the precuneus, and the superior parietal lobule (SPL) in the left hemisphere. Inter-subject functional connectivity (ISFC) was computed by correlating each subject’s seed time series with the average whole-brain response of all other subjects (leave-one-out approach). Maps show Fisher z–transformed ISFC values, thresholded at p < 0.05 (voxel-wise FDR corrected) with a minimum cluster size of 30 voxels. By contrast, the left precuneus seed (part of the DMN) showed connectivity with the rest of the DMN, including bilateral middle frontal cortex and the temporoparietal junction (TPJ). Finally, the SPL seed showed strong correlations with the rest of the parietal cortex, middle frontal cortex, frontal eye fields (FEF), as well as lateral and ventral occipital cortex: regions associated with the dorsal attention network (DAN). Unlike for the frontotemporal language network, the connectivity patterns of both the DMN and SPL were disrupted by sentence scrambling, suggesting that these two networks are specifically involved in narrative processing (Supplementary Fig. S3). Interestingly, in the narrative condition, the left SPL showed significant anticorrelation with canonical frontotemporal language regions, as well as with parts of the DMN (including TPJ). This pattern suggests that the SPL is functionally distinct from both the frontotemporal language network and the DMN. Inter-subject connectivity analyses performed on analogous right-hemisphere seeds show similar results (Supplementary Fig. S4). Right-lateralized and bilateral neural responses to sign-language narrative To assess lateralization of responses to the narrative and its scrambled versions, we computed a laterality index (LI) by comparing ISC values between left and right homologous voxels across the entire cortex, normalized by the absolute value of their summed ISCs. An LI of zero indicates a bilateral pattern, whereas negative values indicate right-hemispheric lateralization. The intact narrative and sentence conditions showed rightward lateralization (intact narrative: LI = −0.024, t(19) = − 4.50, p < 0.001, d = −1.01, 95% CI = [−0.035, −0.013]; sentences: LI = −0.020, t(19) = − 2.98, p = 0.008, d = −0.67, 95% CI = [−0.034, −0.006]). The word-segment and pseudoword conditions yielded LIs not significantly different from zero (words: LI = −0.018, t(19) = − 1.95, p = 0.067, d = −0.44, 95% CI = [−0.037, 0.001]; pseudowords: LI = −0.004, t(19) = − 0.51, p = 0.618, d = −0.11, 95% CI = [−0.021, 0.013]). There was no significant LI difference across scrambling levels (F(3,57) = 1.37, p = 0.261). A similar pattern was observed within language network parcels. The two hemispheres likewise showed similar temporal receptive windows and comparable event durations in HMM analysis (effect of hemisphere all p > 0.05). The only hemispheric difference emerged in the superior parietal lobule (SPL): the right SPL showed longer event segments than the left (mean difference = 7.3, 95% CI = [3.83, 11.61], p < 0.001, permutation test). This effect was observed only after functional alignment, which likely enhanced sensitivity to shared temporal structure across participants. Overall, we observed bilateral or slightly right-lateralized responses to the intact and scrambled versions of the narrative and comparable hierarchical processing across hemispheres. Discussion Segregation of narrative and sentence-level processing is independent of language modality A Polish Sign Language narrative of Edgar Allan Poe’s ‘The Fall of the House of Usher’ robustly synchronized neural activity across deaf native signers. Synchrony was observed not only in visual and the frontotemporal language networks, but also in the Default Mode Network (DMN), which has been associated with discourse-level meaning construction for spoken languages4,6,45 as well as nonlinguistic narratives such as silent movies44,56. Previous studies have shown that strong inter-subject synchronization of the DMN reflects a shared understanding of high-level story meaning5. The current sign language narrative led to robust DMN synchronization across deaf signers, suggesting a shared representations of narrative across individuals. Consistent with findings from spoken language, our results reveal a functional separation between sentence-level processing supported by the frontotemporal language network and plot-level meaning extraction supported by the DMN. Comparison of inter-subject synchrony across varying levels of narrative coherence revealed hierarchical processing by distinct neural networks. Visual areas were synchronized across all levels of scrambling, and for meaningless pseudo-signs. The frontotemporal language network occupied an intermediate level within the processing hierarchy, with synchrony for sentence and word conditions but less for pseudo-signs, and somewhat higher synchrony for narratives. At the highest level of the hierarchy, narrative-specific synchronization occurred within the Default Mode Network (DMN): TPJ and the precuneus uniquely synchronized to the intact narrative content, not to individual sentences or word segments. Data-driven event segmentation using HMM analysis likewise showed differences in preferred time scales across hierarchical networks. Sensory cortices integrated information on a short time scale (15–20 s segments). The frontotemporal language network was sensitive to information at an intermediate timescale (30–50 s), whereas the DMN integrated information across longer timescales (50–60 s), at the narrative level. Finally, inter-subject connectivity analyses revealed distinctive profiles across networks. Regions within the frontotemporal language network were correlated with each other and not correlated with DMN or SPL, indicating distinct temporal processing dynamics. These data suggest that for signed, as for spoken languages, there is a neural separation between plot-level meaning and sentence-level linguistic processing. Our results highlight the unique, modality-invariant role of the frontotemporal language network: across modalities, this network is not merely involved in extracting meaning from a communicative signal in general, but rather specifically, in sentence-level and word-level linguistic processes. Lateralization We observed right-lateralized and bilateral responses to a sign-language narrative. Left-lateralization is a characteristic features of spoken language processing and some studies of sign languages also report left lateralization27,33,34. At the same time, converging evidence across modalities indicates that narrative and discourse processing engages the right hemisphere57–60. Consistent with this, inter-subject synchrony during spoken narrative comprehension is typically bilateral, although lateralization is rarely assessed explicitly4,6,45. In sign languages, narrative devices such as facial affective gestures, role shifting, and classifier constructions have also been shown to recruit right-hemisphere30,31,61. In the current study, we find that under naturalistic narrative conditions, sign languages engage bilateral and even right-lateralized networks – the degree to which this is related to narrative processing, language modality or their interaction remains to be tested in future work. Parietal cortex (SPL) involved in sign language narrative comprehension Relative to previous studies with spoken and written narratives, synchrony for the PJM narrative extended beyond the frontotemporal language network and DMN into superior parietal cortex (SPL). In terms of processing level, the SPL exhibited a slow response timescale comparable to that of the DMN. However, SPL activity was not correlated with the DMN and was anti-correlated with classic frontotemporal language regions, indicating a distinct functional profile. Together, these results suggest that during naturalistic sign language narrative comprehension, the SPL is engaged largely at the discourse level, but with a functional profile different from canonical narrative network (DMN). The SPL is classically associated with spatial processing, including spatial working memory and visuospatial attention62,63. Prior research has implicated the SPL in sign language comprehension and production, particularly in the comprehension of classifier constructions and spatially topographic descriptions25–27,64. In the current study, inter-subject functional connectivity analysis showed that SPL activity is strongly synchronized with the spatial working memory network, also referred to as the dorsal attention network, including the frontal eye fields (FEF) as well as the ventral and lateral occipital cortices62. This network is involved in tracking spatial locations, maintaining visuospatial attention65,66 and it is involved in nonlinguistic narrative processing, such as watching audiovisual and visual movies5,53,56,67,68. The SPL may contribute to spatial processing during sign language narrative comprehension. Space plays a central role in sign languages. At the level of discourse, use of space is crucial for maintaining narrative coherence17,69. For example, signing space is used to introduce and track discourse referents. A referent (e.g., protagonist) can be associated with a specific spatial location via pointing or eye gaze, and then subsequently referred to in the narrative by pointing or gaze to that same location17,70,71. The narrative used in the present study featured a small number of protagonists who were repeatedly referenced, leading to extensive use of such spatial discourse devices (see Supplementary Fig. S5). Narratives also involve referential shifts, in which the narrator adopts a character’s perspective72,73 and spatial shifts can be used to signal changes in scene, topic, or time in the narrative74,75. In addition to supporting discourse organization, the signing space also conveys spatial information through classifier constructions15,18,76. For example, a swirling downward motion verb describes movement of a character down a long winding stairway. Understanding such constructions requires creating and updating a mental model of the spatial layout—tracking where referents are located relative to one another, how they move, and how their spatial relationships change over time64,69,70,72. The spatial working memory functions associated with the SPL may contribute to such model building77. The present findings are not inconsistent with the idea that SPL contributes to sentence-level processing, for example, when the signing space is used to describe real-world topographic space and/or in a referential context25–27. Rather, the present results suggest that during naturalistic sign language comprehension, the SPL plays an important role in discourse processes. Ventral occipital cortex involved in sign language comprehension In addition to the SPL, we observed synchrony in bilateral lateral ventral occipital cortex (vOTC) during sign language comprehension. The vOTC is associated with face perception and reading in both hearing and deaf individuals78–81. Lateral vOTC preferentially responded to meaningful sign-language, including narrative, sentences, and words and these responses were not driven solely by the presence of a face or facial movements. The temporal response window of the vOTC was shorter than that of the DMN and SPL. Event segmentation analysis revealed shorter events, intermediate in duration between frontotemporal language and early perceptual networks, possibly suggesting processing of linguistic form. Prior evidence implicates vOTC in facial grammar and fingerspelling perception (McCullough et al., 2005; see also82. Facial gestures (or non-manuals) play an important role in sign languages at all levels of the language hierarchy83–87. For example, in PJM, yes/no questions are distinguished from statements through eyebrow raising and an accompanying mouth shape can change a sign’s meaning88. Non-manual markers are used heavily in a narrative context. Eye gaze can index discourse referents69. Non-manuals are used in role shifting and constructed action, where the narrator takes on the role of the character in the narrative to report what they are saying, feeling or doing89–91. The naturalistic narrative used in the current study is rich in non-manual facial markers, even in the sentence and word scrambled conditions, which are created by scrambling the narrative. By contrast, prior work using single words and sentences is less rich in nonmanuals in some cases explicitly removes them from the stimuli29. Naturalistic stimuli have the potential to provide new insight into cortical systems involved in sign language processing. Our results highlight the role of SPL and vOTC in sign language narrative processing. Future studies combining naturalistic and experimental approaches are needed to probe the specific contribution these cortical areas to sign languages and to test whether there is specialization for linguistic functions. A Polish Sign Language (PJM) narrative robustly synchronized a hierarchy of cortical networks, spanning visual areas in V1, the ventral stream (vOTC), frontotemporal language networks, and the default mode network (DMN). We observed a clear functional separation between the frontotemporal language network, which supports processes at the sentence level and below, and the DMN, which supports higher-level discourse and ‘story’ processing. The vOTC showed a language-specific response and a temporal profile consistent with processing of rich visual and facial information intrinsic to sign languages. Unlike prior work with spoken narratives, the current sign language narrative synchronize activity in the superior parietal lobule (SPL), a core component of the parietal spatial processing system. The SPL showed sensitivity to discourse content and had a distinct functional signature. The SPL may support use of space to establish and maintain narrative coherence. Methods All procedures performed in this study were conducted in accordance with relevant ethical regulations and were approved by the Ethics Committee of Jagiellonian University, Kraków, Poland. Participants Twenty congenitally deaf native signers of Polish Sign Language (PJM) (mean age= 35.6, SD = 7.2, 10 women) took part in the study. Inclusion criteria for deaf participants included congenital deafness, severe level of deafness (above 90 dB threshold in both ears), and lack of a cochlear implant. All participants were Deaf native signers of PJM, meaning that they were born deaf into Deaf families and acquired PJM from birth through exposure to their parents’ signing. For detailed information on deafness etiology and language experience, see Table 1. Instructions were in Polish Sign Language (PJM) for deaf participants (by use of a sign language interpreter). Written informed consent was signed by all participants.Table 1Participant demographics and language experienceIDPJM onsethearing aidshearing loss both earsEducationDeaf caregiversPJM acquisitionEarly language (3–6)PJM usePolish usePJM proficiency1nativeno> 120 dBHigherYesParentsPJMAll the timeDailyFluent2nativeyes90-119 dBHigherYesParentsPJMAll the timeDailyFluent3nativesometimes90-119 dBVocationalYesParentsPJMAll the timeDailyFluent4nativeno90-119 dBHigherYesParentsPJMAll the timeWeeklyFluent5nativeno> 120 dBSecondaryYesParentsPJMAll the timeWeeklyFluent6nativeno> 120 dBSecondaryYesParentsPJMAll the timeDailyFluent7nativeno90-119 dBSecondaryYesParentsPJMAll the timeNeverFluent8nativesometimes90-119 dBSecondaryYesParentsPJMAll the timeDailyFluent9nativesometimes90-119 dBSecondaryYesParentsPJMAll the timeDailyFluent10nativesometimes90-119 dBHigherYesParentsPJMAll the timeDailyFluent11nativeno90-119 dBSecondaryYesParentsPJMAll the timeNeverFluent12nativeno90-119 dBSecondaryYesParentsPJMAll the timeSpecial occasionsFluent13nativesometimes90-119 dBHigherYesParentsPJMAll the timeDailyFluent14nativeno> 120 dBVocationalYesParentsPJMAll the timeSpecial occasionsFluent15nativeno> 120 dBSecondaryYesParentsPJMAll the timeDailyFluent16nativeno> 120 dBHigherYesParentsPJMAll the timeDailyFluent17nativesometimes> 120 dBSecondaryYesParentsPJMAll the timeSpecial occasionsFluent18nativesometimes90-119 dBSecondaryYesParentsPJMAll the timeDailyFluent19nativeno90-119 dBSecondaryYesParentsPJMAll the timeSpecial occasionsFluent20nativesometimes90-119 dBHigherYesParentsPJMAll the timeDailyFluent Table 1Participant demographics and language experienceIDPJM onsethearing aidshearing loss both earsEducationDeaf caregiversPJM acquisitionEarly language (3–6)PJM usePolish usePJM proficiency1nativeno> 120 dBHigherYesParentsPJMAll the timeDailyFluent2nativeyes90-119 dBHigherYesParentsPJMAll the timeDailyFluent3nativesometimes90-119 dBVocationalYesParentsPJMAll the timeDailyFluent4nativeno90-119 dBHigherYesParentsPJMAll the timeWeeklyFluent5nativeno> 120 dBSecondaryYesParentsPJMAll the timeWeeklyFluent6nativeno> 120 dBSecondaryYesParentsPJMAll the timeDailyFluent7nativeno90-119 dBSecondaryYesParentsPJMAll the timeNeverFluent8nativesometimes90-119 dBSecondaryYesParentsPJMAll the timeDailyFluent9nativesometimes90-119 dBSecondaryYesParentsPJMAll the timeDailyFluent10nativesometimes90-119 dBHigherYesParentsPJMAll the timeDailyFluent11nativeno90-119 dBSecondaryYesParentsPJMAll the timeNeverFluent12nativeno90-119 dBSecondaryYesParentsPJMAll the timeSpecial occasionsFluent13nativesometimes90-119 dBHigherYesParentsPJMAll the timeDailyFluent14nativeno> 120 dBVocationalYesParentsPJMAll the timeSpecial occasionsFluent15nativeno> 120 dBSecondaryYesParentsPJMAll the timeDailyFluent16nativeno> 120 dBHigherYesParentsPJMAll the timeDailyFluent17nativesometimes> 120 dBSecondaryYesParentsPJMAll the timeSpecial occasionsFluent18nativesometimes90-119 dBSecondaryYesParentsPJMAll the timeDailyFluent19nativeno90-119 dBSecondaryYesParentsPJMAll the timeSpecial occasionsFluent20nativesometimes90-119 dBHigherYesParentsPJMAll the timeDailyFluent Participant demographics and language experience Naturalistic stimuli Participants watched a story in Polish Sign Language during fMRI scan. The narrative was generated by a deaf native signer who interpreted Edgar Allan Poe’s ‘The Fall of the House of Usher’ (20 min). The story contains a rich structure at many levels of the linguistic hierarchy, from phonology to compositional syntax, semantics, and discourse. The first 10 min of the story were used to create the control conditions. This segment was first divided into individual sentences and then further segmented into approximately individual signs or short sign sequences by a native signer. In the sentence condition, single sentences from the story were presented in a shuffled order while preserving their original signing speed. In the word condition, short sign segments consisting of 1–3 signs (lasting approximately 1–3 s) were presented in a shuffled order. Due to coarticulation and simultaneity in sign language, most segments contained more than one sign, such as a sign accompanied by co-gesture or a coarticulated sign sequence. However, these segments did not combine to form full sentences. A scrambled sentence condition removes discourse structure, while a scrambled words condition also interferes with sentence-level grammar. Additionally, participants watched a control condition, a meaningless story composed of pseudo-signs (Fig.1). To avoid order confounds, the scrambled conditions (words, sentences, pseudowords) were presented first in a counterbalanced order. The full story was shown only afterward, preventing participants from reinterpreting the control stimuli based on the story. The 10 min of the intact narrative as well as 10 min control conditions were used in the inter-subject correlation analysis (ISC), and inter-subject functional connectivity analysis (ISFC). We used the entire 20 min narrative in the event segmentation analysis (Hidden Markov Models). Language Localizer To enable individual-subject analyses of language-selective regions, the language network was localized separately in each deaf participant using a Polish Sign Language (PJM) language localizer task31,50. Participants viewed blocks of PJM sentences and a perceptually matched control condition designed to preserve visual motion and facial information while eliminating linguistic content. The PJM sentences were translations of materials originally developed for a spoken-language localizer50 and were recorded by a deaf native signer. At least two approaches have been used in prior work to construct such control stimuli for sign-language localizers. One approach overlays backward-signed sentences to disrupt linguistic structure while preserving low-level visual properties31. A second approach relies on strong Gaussian blurring to remove linguistic intelligibility32. In the present study, we adopted the first approach. However, because backward, overlaid signing can occasionally reveal recognizable lexical forms, we validated the unintelligibility of the control condition by asking two native signers to watch the stimuli and extract as many words as possible. When isolated signs were still identifiable, we blurred the signer’s hands until no meaningful items were reliably recognized. Thirty coherent and thirty control sentences were presented in blocks, across two runs of approximately 7 min each. Data Acquisition MRI structural and functional data of the whole brain were collected on a 3 Tesla Siemens MAGNETOM Tim Trio scanner. The T1-weighted structural images were collected in 176 axial slices with 1 mm isotropic voxels using a magnetization-prepared rapid gradient echo (MP RAGE). Functional images were collected using a gradient echo planar imaging (EPI) sequence (36 sequential ascending axial slices, repetition time (TR) 1.4 s, echo time (TE) 30 ms, flip angle 70°, field of view (FOV), matrix 76 × 70, voxel size 2.5 × 2.5 × 2.5 mm, PE direction L/R, multiband MB = 4). Data analyses were performed using fmriprep (freesurfer), python (v. 3.7), the Brain Imaging Analysis Kit, http://brainiak.org. Preprocessing We performed minimal preprocessing using fmriprep92. Anatomical data preprocessing The T1-weighted (T1w) image was corrected for intensity non-uniformity (INU) with ‘N4BiasFieldCorrection‘ and used as T1w-reference throughout the workflow. The T1w-reference was then skull-stripped with a Nipype implementation of the ‘antsBrainExtraction.sh‘ workflow, using OASIS30ANTs as target template. Brain tissue segmentation was performed on the brain extracted T1w using ‘fast‘ [FSL 5.0.9]. Volume-based spatial normalization to one standard space (ICBM 152 Nonlinear Asymmetrical template version 2009c) was performed through nonlinear registration with ‘ants Registration‘ (ANTs 2.3.3). The BOLD reference was co-registered to the T1w reference using ‘flirt‘ [FSL 5.0.9]. Co-registration was configured with nine degrees of freedom to account for distortions remaining in the BOLD reference. The BOLD time-series were resampled onto their original, native space by applying the transforms to correct for head motion. The BOLD time-series were resampled into standard space, generating a preprocessed BOLD run in MNI152NLin2009cAsym space. Functional data preprocessing First, a reference volume and its skull-stripped version were generated using a custom methodology of fMRIPrep. Several confounding time series were calculated based on the preprocessed BOLD: framewise displacement (FD), DVARS, and three region-wise global signals. FD was computed using two formulations following Power (absolute sum of relative motions) and Jenkinson (relative root mean square displacement between affines). FD and DVARS are calculated for each functional run, both using their implementations in Nipype. The three global signals are extracted within the CSF, the WM, and the whole-brain masks. Additionally, a set of physiological regressors were extracted to allow for component-based noise correction. Principal components are estimated after high-pass filtering the preprocessed BOLD time-series (using a discrete cosine filter with 128 s cut-off) for the two *CompCor* variants: temporal (tCompCor) and anatomical (aCompCor). tCompCor components are then calculated from the top 2% variable voxels within the brain mask. For aCompCor, three probabilistic masks (CSF, WM and combined CSF + WM) are generated in anatomical space. Finally, these masks are resampled into BOLD space and binarized by thresholding at 0.99 (as in the original implementation). The confound time series derived from head motion estimates and global signals were expanded with the inclusion of temporal derivatives and quadratic terms for each. Frames that exceeded a threshold of 0.5 mm FD or 1.5 standardized DVARS were annotated as motion outliers. Denoising After preprocessing, we smoothed the data (fwhm=6 mm) and performed voxel-wise denoising using a GLM. To perform denoising, we fit a voxel-wise general linear model (GLM) for each participant. The effects of motion estimated during the realignment step using an expanded set of 24 motion parameters (six demeaned realignment parameters, their squares, their derivatives, and their squared derivatives), We also include dummy codes for spikes identified from global signal outliers and outliers identified from frame differencing (i.e., temporal derivative). No additional high-pass filtering was applied beyond that implemented within fMRIPrep; instead, low-frequency trends were modeled using linear and quadratic regressors. Data Analysis Whole brain inter-subject correlation analysis The aim of this analysis is to evaluate the degree of stimulus-driven synchronization (correlation) to that same voxel in other people’s cortices. In the whole brain level analysis for each voxel, the inter-subject correlation was calculated using the leave-one-out method. First, voxelwise synchrony was calculated as the average Pearson correlation coefficient (r) between the time course of one subject and the average time course of the rest of the experimental group (Hasson et al., 2004; Lerner et al., 2011). This procedure was repeated for all subjects in a group for each condition separately (narrative, sentences, words, pseudo-signs) Results were averaged across participants. The average ISC maps with values r were transformed to Fisher’s z values to allow comparisons of correlations between groups and conditions. Group ISC maps (Fig.2) were created using bootstrapping (implemented in BriainIAK, isc_bootstrap). All ISC whole-brain maps were thresholded at the level p <.05, FDR corrected voxel-wise, cluster size > 30 voxels. Differences in synchronization between stimuli were compared by subtracting the relevant z-maps (Supplementary Fig. S2). Because ISC analysis violates the assumptions required for parametric methods, we perform non-parametric hypothesis ISC testing using permutation tests. (implemented in BrianIAK isc_permutation). ROI definition The regions of interest (ROIs) were delineated within bilateral parcels: 5 distinct frontotemporal parcels in each hemisphere50. The Superior Parietal Lobule (SPL), the Ventral Occipital Cortex (vOTC) bilaterally, two parcels within the Default Mode Network (DMN): the precuneus and the temporoparietal junction (TPJ), primary visual cortex (V1), and lateral occipital cortex (LOC). The frontotemporal language network comprises anatomical locations in the frontotemporal cortices known to be responsive to spoken and written language: the inferior frontal gyrus (IFG), orbital part of the IFG (IFGorb), middle frontal gyrus (MFG), anterior temporal (AntTemp), and posterior temporal (PostTemp) regions50. The SPL, V1, LOC were anatomically defined using the Harvard-Oxford atlas for each hemisphere separately. The lateral vOTC was defined based on the localization of the fusiform face area93. DMN parcels (precuneus and TPJ) were delineated using atlas based whole-brain connectivity parcellation of 17 networks52. Voxels in the TPJ ROI that overlapped with the frontotemporal language parcels were excluded from TPJ ROI. Inter-subject correlation ROI analysis For the inter-subject correlation analysis in the frontotemporal “language” parcels, we first identified language-responsive voxels individually for each participant. Using the localizer contrast (sentences > control), we selected the top 10% of voxels showing the strongest response. To compare the level of brain synchronization across ROIs and conditions (narrative, sentences, words, pseudo-signs), we then computed ISCs within the ROIs defined above. For each participant and each ROI, we extracted a time course by averaging the BOLD signal across all voxels in that region. As in the whole-brain analysis, each participant’s ROI time course was correlated with the average ROI time course of all other participants in the group, and the resulting ISC values were averaged across participants. To normalize ISC values across ROIs, we followed the procedure described in Blank and Fedorenko (2020)94. Regional ISC for the intact narrative was treated as a “ceiling” value against which all other conditions were normalized. Specifically, ISC values were squared (converted correlations to percent variance explained), squared-ISC values for each scrambled condition were divided by the squared-ISC value for the intact narrative, and the square root of the resulting ratio was taken to return the values to correlation units. We performed four separate linear mixed-effects models with condition (narrative, sentences, words, pseudo-signs) and ROI type as within-participant fixed factors. For each model, we tested the ROI-type × condition interaction to assess whether ISC patterns differed across ROI types. Three models compared the frontotemporal network (FT) with each comparison network (DMN, SPL, vOTC), and a fourth model examined ISC patterns across conditions within the language-responsive ROIs. Degrees of freedom for all fixed effects were estimated using the Satterthwaite approximation, as implemented in the lmerTest package. To test for potential hemispheric differences, we fit an additional model including hemisphere (left, right) as a fixed factor and evaluated the ROI-type × condition × hemisphere interaction. The same analyses were conducted using bilateral ROIs (reported in the main text) as well as left- and right-hemisphere ROIs separately (reported in the Supplementary Fig.s). All results were corrected for multiple comparisons using FDR. Temporal receptive windows We used ISC measures for each condition to estimate the slope of the function of ISC measures and conditions. TRW index was estimated using formula: 3*Intact narrative ISC + sentences – words – 3*pseudo-sign45,56. The TRW index was first calculated for each voxel and for each functional ROI and normalized using the higher slope (trw=1) and the lowest slope (trw=0) on the whole brain level. To visualize slope, the ISC values were divided by the highest ISC in the region of interest. HMM event segmentation The event segmentation analysis was performed on the frontotemporal parcels, SPL, DMN ROIs, visual cortex ROIs, and vOTC. To ensure high alignment across individuals, each ROI was defined within anatomical parcels (see ROI definition) as the top 10% of voxels showing the highest inter-subject synchronization during narrative watching. For each ROI, we estimated the preferred event duration using event segmentation based on hidden Markov models, following procedures established in previous studies49. The model identifies a sequence of stable multivoxel activity patterns that best predicts held-out participants’ data. The resulting “event duration” reflects the timescale over which a region’s spatial pattern remains relatively stable. Notably, this duration does not necessarily match up with the duration of stimulus features for a variety of reasons. Because of fMRI’s hemodynamic smoothing and the constraint that event boundaries be consistent across participants, it is generally found that estimated event durations, even those in primary sensory regions, are longer than moment-to-moment changes in the stimulus. For this reason, the absolute duration of a detected event should not be interpreted as an intrinsic processing timescale of a specific brain region. Instead, the measure can be viewed as a relative index of how quickly or slowly different regions update their multi-voxels activity patterns. To determine the relative temporal scale of each ROIS, we determined the optimal number of events (k) at the group level using a train–test split approach. The minimum k was set to 10 and the maximum to 100. For each region, subjects were randomly divided into training and testing groups, and voxelwise time series were averaged within each group, yielding one voxel-by-time matrix for the training set and one for the test set. We then fit models to the averaged training data for every possible k and evaluated each model on the test data using log-likelihood as the goodness-of-fit metric. After iterating this procedure across all k values, we selected the k that produced the highest log-likelihood as the optimal number of events for that region in each split. This process was repeated 50 times with different train–test splits per region. The resulting vectors of 50 optimal k estimates were then compared across regions using a permutation test (1000 permutations). For each region, we computed the mean optimal k across iterations, which we report as the estimated event length for that region (see Fig. 4). Finally, we assessed statistical significance by comparing within-event and between-event correlations. To complement this analysis, we performed an additional event segmentation analysis using individually defined language-responsive ROIs. To enable alignment across individuals while preserving subject-specific ROI definitions, we used a leave-one-subject-out Shared Response Model (SRM). For each iteration, a shared functional space was learned from all participants except one held-out subject, who was subsequently aligned to this space using a regularized linear mapping. Event segmentation was then applied to the shared-space time courses. For each held-out participant, the optimal number of events was estimated by repeatedly training the HMM on bootstrapped subsets of the remaining subjects and evaluating model fit on the held-out individual using log-likelihood. The median k across bootstraps was taken as that participant’s preferred event length. This procedure yielded vectors of 50 estimated event lengths per region, which were compared across regions using permutation tests (1000 permutations). Inter-subject functional connectivity analysis In the inter-subject functional connectivity analysis, seeds were defined in three parcels: a frontotemporal parcel (left MTG), a parcel in the left SPL, and a parcel in the left DMN (precuneus). We performed analogical analysis using seeds in the right hemisphere (right MTG, right SPL and right precuneus, which is reported in Supplementary Fig. 4). In the parietal and frontotemporal parcels, we identified 20 top language-responsive voxels (sentences > control condition) at the group level. In the DMN (precuneus) the seed was defined as 20 voxels within the precuneus parcel with the highest synchronization during the second part of the narrative. Correlation (ISFC) is computed between each subject and average whole brain response in other subjects (leave-on-out method). The ISFC significance was calculated using permutation test, results were corrected for multiple comparisons using FDR. Lateralization analysis To examine hemispheric differences in neural synchrony, we calculated a voxelwise laterality index (LI) for each subject using the ISC values obtained from the whole-brain ISC analysis explained above. Homologous left and right hemisphere voxels were paired based on their spatial coordinates. For each voxel pair, LI was defined as the difference between left and right ISC values normalized by their combined magnitude, such that a value of zero indicates a bilateral response, negative values indicate right-lateralization, and positive values indicate left-lateralization. Voxelwise LI was computed for each participant, and mean LI values were obtained across the whole brain and separately within the frontotemporal network. Reporting summary Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article. Supplementary information Supplementary Figs. Reporting Summary Transparent Peer Review file Supplementary Figs. Reporting Summary Transparent Peer Review file Source data Source Data Source Data