walter_slides1 (1)

walter_slides1 (1) - Multisensory Interplay Jon Driver and...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Multisensory Interplay Jon Driver and Toemme Noesselt Sensory Research Sensory Research Behavioral Consequences of Multimodality Joint estimates of single property Spatial Ventriloquism Auditory Driving McGurk Effect Can modalities affect each other without creating a single unified percept? Touch at a location can help perception of color Sound-induced Illusory Flash Orientation discrimination improves with multiple beeps Convergence Zones Superior Colliculus inputs from somatosensory, auditory, and visual areas Super or Sub-additive responses found for combination Additivity in SC most likely with weak unisensory inputs Ceiling effect? Neural limitations? Late onset in development Depends on multisensory cortex Convergence Zones Testing for Convergence Anatomical studies: Direct connections between different sensory areas Single-cell studies: Response to stimulation from different modalities Neuroimaging: Large-scale responses based on BOLD signal Influences on 'Sensory-Specific' Areas Growing body of research shows that sensory-specific areas might be an artifact of the studies done with them Examine studies that use: fMRI EEG Invasive recording in animal models fMRI Caveat: fMRI has been shown to respond to attention and imagery For example, speech may be imagined when viewing lip movements fMRI analysis Inspired by Stein (SC), some look for sub-, super-additive responses Maybe linearity is normal, though, so some use max or mean criteria Difficult because of spatial resolution Convergence in V1? Amedi, Jacobsen, Hendler, Malach, and Zohary, 2002 ERP results example Tactile stimulation Visual cue ERP extracted ERP studies visual N1 enhanced when tactile stimulation occurred at same location as a visual event visual P1 modified by taskirrelevant sound P1 modified by attend-visual relative to attend-tactile conditions ERP studies ERPs show early multi-sensory effects (~30 ms) Poor localization Potential methodological confounds Invasive Studies Current-source densities (CSD) reflect local PSPs Region of auditory association cortex Location and timing of stimulation consistent with auditory feedforward, visual feedback Invasive Studies Posture may affect responses to auditory signals in A1 Tactile stimuli modulate initial response to auditory signals in A1 Multisensory Interplay Examples of converging zones of multi-sensory input Examples of interplay: one modality affects another What frameworks does this evidence suggest? Possible Frameworks A) All Multisensory B) Bimodal Brain Areas C) Critical Feedback Circuitry All Multisensory Unlikely to be completely undifferentiated Even primary sensory areas responsive to multiple modalities All Multisensory Thalamus might be source of multisensory interplay Tactile stimulation can affect first neural response in A1, hypothesized from thalamus Found in gerbils, hard to study in humans All Multisensory Direct coritco-cortical influences Anatomical evidence: single synapse from ACVC and AC-SSC, AC-OC However, not as many as to conventional Multisensory areas Role still unclear All Multisensory Still overwhelmingly "sensory-specific" Bimodal Brain Areas Less extreme version of account A Similar to current account, with more multisensory regions Parallel multi- and singlemodality processing could explain early EEG modulation Bimodal Brain Areas Different areas in auditory areas may be connected to distinct visual areas Bimodal interplay would be affected by transduction time, explaining BOLD response time differences Feedback Circuitry Effects in primary areas might be feedback from convergence zones Evidence from effective connectivity in fMRI, tactile stimulation increases visual response Evidence from EEG source-localization: STS - VC Feedback Circuitry Evidence from invasive recordings: late A1 stimulation from vision (speculation?) Feedback can be tested directly, but very little has been done Remarks Perhaps the rival frameworks are all valid for certain situations Perhaps primary cortex responses in the blind and deaf can help tease out what the multisensory roles are New techniques will allow testing causal interplay ...
View Full Document

Ask a homework question - tutors are online