Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments.

Driver J, Noesselt T. · 2008

Read Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. by Driver J, Noesselt T. (2008) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc2427054.txt.

Introduction

Driver and Noesselt surveyed how the senses influence each other in the brain in 2008. Classic neuroscience treated each sense, sight, hearing, touch, as a sealed module. Their review gathers human studies showing that the modules talk, early and often.

Brain scans and recordings now show meeting zones where signals from several senses converge. Some sit surprisingly close to the first processing areas for a single sense. Direct links may even join the primary cortices of different senses. What one sense reports can change both brain responses and conscious judgements about another sense, as when sound alters what people see. The authors weigh three wiring explanations: fast forward merging, relay stations in the thalamus, and feedback from meeting zones back to single-sense areas. They stress new methods can now separate these circuits.

Catalog record

CREATORDriver J, Noesselt T.
YEAR2008
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC2427054

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multisensory Interplay Reveals Crossmodal Influences on ‘Sensory-Specific’ Brain Regions, Neural Responses, and Judgments By Jon Driver, Toemme Noesselt Published in: Neuron DOI: 10.1016/j.neuron.2007.12.013 ABSTRACT Abstract Although much traditional sensory research has studied each sensory modality in isolation, there has been a recent explosion of interest in causal interplay between different senses. Various techniques have now identified numerous multisensory convergence zones in the brain. Some convergence may arise surprisingly close to low-level sensory-specific cortex, and some direct connections may exist even between primary sensory cortices. A variety of multisensory phenomena have now been reported in which sensory-specific brain responses and perceptual judgments concerning one sense can be affected by relations with other senses. We survey recent progress in this multisensory field, foregrounding human studies against the background of invasive animal work and highlighting possible underlying mechanisms. These include rapid feedforward integration, possible thalamic influences, and/or feedback from multisensory regions to sensory-specific brain areas. Multisensory interplay is more prevalent than classic modular approaches assumed, and new methods are now available to determine the underlying circuits. MAIN TEXT Introduction Sensory processing and perception have been studied intensively for decades, in both neuroscience and psychology. But most traditional research considered just a single sensory modality at a time (e.g., vision or audition or touch). By contrast, real-world situations often stimulate several of our senses concurrently. Moreover, subsets of the incoming stimulation across different modalities arise from common external objects or events, as when we both see and feel an object in our hand or both see and hear a person talking or a car moving. In psychology, it has long been known that perceptual judgments can reflect combined information from multiple senses (Welch and Warren, 1986; Spence and Driver, 2004). Moreover, neuroscience has identified various “multisensory” brain regions as convergence zones, where neurons receive afferent inputs from several senses and combine these according to various constraints. But in recent years the field of multisensory research has expanded and altered radically with the realization that multisensory influences are much more pervasive than classical views assumed and may even affect brain regions, neural responses, and judgments traditionally considered modality specific. Here we consider such cases, in which multisensory effects can arise for apparently sensory-specific processes or perceptions. We will often refer to multisensory “interplay” rather than the commonly used “integration,” so as to include cases where one modality might affect another without necessarily always implying a single unified percept. We focus primarily on human perceptual studies but refer to important animal work as relevant background. Behavioral and Perceptual Consequences of Multisensory Interplay in Humans Classic examples of multisensory perceptual “illusions” include spatial ventriloquism (mislocalization of sounds toward temporally correlated but displaced visual events), auditory driving (misperception of visual events as having the temporal frequency of apparently related auditory events), and the McGurk effect (perception of speech sounds influenced by seen lip movements). See Calvert et al. (2004), Macaluso and Driver (2005), Spence and Driver (2004), and Vroomen and de Gelder (2004) for more extensive reviews of such perceptual phenomena. In such cases, information is typically provided by two or more different senses about one particular external property. The textbook multisensory effects are traditionally considered to be misleading illusions. But an increasingly influential view is that they reflect combined use of information from the separate modalities to yield a joint estimate of an external property. Thus, location information from two or more modalities may be jointly considered when estimating the location of an apparently multisensory event. Such joint estimates may be “optimal” in a formal sense, weighting each modality's contribution by its reliability/variability for the property concerned (e.g., Alais and Burr, 2004; Ernst and Bulthoff, 2004; Helbig and Ernst, 2007). When more than one input is provided within each of several senses, as will often apply in the real world, a further issue arises concerning which particular inputs from one sense should be jointly weighted together with which particular selection of inputs from other senses. Spatial, temporal, and semantic/associative relations may be critical in constraining such selective combination of related or jointly parsed subsets from multiple inputs to multiple senses. For recent examples regarding spatial constraints, see Frassinetti et al. (2002), for temporal constraints, see Fendrich and Corballis (2001) and Recanzone (2003), for combined spatiotemporal constraints, see Zampini et al. (2003); see also Vatakis and Spence (2007) for further discussion. By contrast, other recent examples of multisensory influences on perception arguably demonstrate a different type of phenomenon. Rather than several modalities providing independent samples about the same external property, stimulation in one modality may now affect judgments of a property that logically applies only to another modality. Thus, presence/absence judgments concerning only one modality (e.g., for vision) can be enhanced when a sound co-occurs at the location of the visual event to be detected, affecting visual sensitivity rather than merely criterion (e.g., McDonald et al., 2000). Touch at a given location can even improve judgments of visual color nearby, although touch itself cannot convey color (see Spence et al., 2004). Here, events in a modulating modality may render a particular region of space (and/or time) salient for another modality, to facilitate modality-specific processing for that time or place in the latter modality (Driver and Spence, 2000; Frassinetti et al., 2002; Lovelace et al., 2003; McDonald et al., 2000; Vroomen and de Gelder, 2000). Such results provide an initial behavioral hint that events in one modality may sometimes affect sensory-specific processing for another modality, as now also indicated by some of the neural measures we consider later. Other recent examples of behavioral multisensory effects extend the classic phenomenon of auditory driving (Shipley, 1964) with the auditory-flash illusion. Shams et al. (2000) reported that a single flash can be misperceived as two flashes if paired with two beeps (see also Arden et al., 2003; Mishra et al., 2007; Shams et al., 2001; Watkins et al., 2006). Berger et al. (2003) showed that when multiple sounds produce the impression of more visual events than actually occurred, visual orientation discriminations can improve objectively (even though the sounds do not provide any orientation information), analogously to when more visual events actually did occur. Thus, multisensory interplay can affect sensory-specific judgments. Neural Studies of Multisensory Interactions: Traditional Focus on Multisensory Convergence Zones and Recent Findings on This Topic Converging evidence from single-cell studies, tracing work, and recent human neuroimaging indicate numerous multisensory convergence zones in the brain (e.g., Mesulam, 1998; Kaas and Collins, 2004; Wallace et al., 2004); that is, brain regions where neurons receive afferent inputs from multiple senses. This has now been observed for numerous cortical and subcortical regions (see Figures 1 and 2 for examples). [Figure: Response Properties of Multisensory Neurons(A) Response properties of a putatively illustrative multisensory neuron, in deep superior colliculus, which in this case shows the often-discussed nonlinearly superadditive pattern of firing. That is, the response for combined visual and auditory stimulation, with a particular spatiotemporal relation, greatly exceeds the sum of the responses to each modality alone (adapted from Stein et al., 2004, by permission of Oxford University Press).(B) Distribution of z scores for a population of sampled neurons within deep layers of the cat superior colliculus, where z scores relate to firing rates for combined

Read the complete text