CaltechTHESIS
  A Caltech Library Service

Crossmodal interaction in humans

Citation

Watanabe, Katsumi (2001) Crossmodal interaction in humans. Dissertation (Ph.D.), California Institute of Technology. http://resolver.caltech.edu/CaltechTHESIS:10122010-090303102

Abstract

Since different senses are attuned to the different aspects of the environment, crossmodal interactions can reduce perceptual ambiguity which may result from relying on a single sensory modality. While many studies have shown that visual information affects perception in the other modalities, little is known about how auditory and tactile information affect visual perception. The present thesis explores how the human perceptual system synthesizes visual event perception by using an ambiguous visual motion display: Two identical visual targets moving across each other can be perceived either to bounce off or to stream through each other. A brief sound at the moment the targets coincide biases perception toward bouncing. Extensive psychophysical investigations on this bounce-inducing effect show that: (1) A salient sensory transient biases visual perception toward bouncing, irrespective of the modality in which such a transient is presented. (2) The magnitude of the bounce-inducing effect depends on both the saliency of a sensory transient and the timing of a sensory transient relative to the visual coincidence. (3) The magnitude of the bounce-inducing effect increases as the sensory saliency of bounce-inducing transients increases. (4) The interaction window during which a sensory transient can bias visual perception is fixed in time. (5) Auditory and tactile transients have much larger temporal interaction windows than visual transients. (6) Auditory and tactile transients presented before the visual coincidence tend to have a stronger bounce-inducing effect than those presented after the visual coincidence; visual transients do not show this asymmetry. (7) The lack of attention to the visual moving targets in the streaming/bouncing motion display increases the likelihood of the bouncing percept. Based on these results, I conclude that the streaming percept arises when amodal attentional resource is available for moving stimuli around the moment of the visual coincidence and that the bouncing percept results from the lack of attentional resource. The role of salient sensory transients in the bounce-inducing effect is to distract attention from the coinciding moving stimuli. Thus, this thesis shows that dynamics of attentional resource allocation, not simply associative learning, can be the cause of amodal event perception in humans.

Item Type:Thesis (Dissertation (Ph.D.))
Subject Keywords:Computation and Neural Systems
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Computation and Neural Systems
Thesis Availability:Restricted to Caltech community only
Research Advisor(s):
  • Unknown, Unknown
Thesis Committee:
  • Allman, John Morgan
  • Andersen, Richard A.
  • Koch, Christof
  • Konishi, Masakazu
Defense Date:1 January 2001
Record Number:CaltechTHESIS:10122010-090303102
Persistent URL:http://resolver.caltech.edu/CaltechTHESIS:10122010-090303102
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:6131
Collection:CaltechTHESIS
Deposited By: John Wade
Deposited On:12 Oct 2010 16:44
Last Modified:26 Dec 2012 04:31

Thesis Files

[img] PDF - Final Version
Restricted to Caltech community only
See Usage Policy.

71Mb

Repository Staff Only: item control page