27.1. INTRODUCTION

Visually dominant behavior has been observed in many different species, including birds, cows, dogs, and humans (e.g., Partan and Marler 1999; Posner et al. 1976; Uetake and Kudo 1994; Wilcoxin et al. 1971). This has led researchers to suggest that visual stimuli may constitute “prepotent” stimuli for certain classes of behavioral responses (see Colavita 1974; Foree and LoLordo 1973; LoLordo 1979; Meltzer and Masaki 1973; Shapiro et al. 1980). One particularly impressive example of vision’s dominance over audition (and more recently, touch) has come from research on the Colavita visual dominance effect (Colavita 1974). In the basic experimental paradigm, participants have to make speeded responses to a random series of auditory (or tactile), visual, and audiovisual (or visuotactile) targets, all presented at a clearly suprathreshold level. Participants are instructed to make one response whenever an auditory (or tactile) target is presented, another response whenever a visual target is presented, and to make both responses whenever the auditory (or tactile) and visual targets are presented at the same time (i.e., on the bimodal target trials). Typically, the unimodal targets are presented more frequently than the bimodal targets (the ratio of 40% auditory—or tactile—targets, 40% visual targets, and 20% bimodal targets has often been used; e.g., Koppen and Spence 2007a, 2007b, 2007c). The striking result to have emerged from a number of studies on the Colavita effect is that although participants have no problem in responding rapidly and accurately to the unimodal targets, they often fail to respond to the auditory (or tactile) targets on the bimodal target trials (see Figure 27.1a and b). It is almost as if the simultaneous presentation of the visual target leads to the “extinction” of the participants’ perception of, and/or response to, the nonvisual target on a proportion of the bimodal trials (see Egeth and Sager 1977; Hartcher-O’Brien et al. 2008; Koppen et al. 2009; Koppen and Spence 2007c).

FIGURE 27.1

Results of experiments conducted by Elcock and Spence (2009) highlighting a significant Colavita visual dominance effect over both audition (a) and touch (b). Values reported in the graphs refer to the percentage of bimodal target trials in which participants (more...)

Although the majority of research on the Colavita effect has focused on the pattern of errors made by participants in the bimodal target trials, it is worth noting that visual dominance can also show up in reaction time (RT) data. For example, Egeth and Sager (1977) reported that although participants responded more rapidly to unimodal auditory targets than to unimodal visual targets, this pattern of results was reversed on the bimodal target trials—that is, participants responded more rapidly to the visual targets than to the auditory targets. Note that Egeth and Sager made sure that their participants always responded to both the auditory and visual targets on the bimodal trials by presenting each target until the participant had made the relevant behavioral response. ^* A similar pattern of results in the RT data has also been reported in a number of other studies (e.g., Colavita 1974, 1982; Colavita and Weisberg 1979; Cooper 1998; Koppen and Spence 2007a; Sinnett et al. 2007; Zahn et al. 1994).

In this article, we will focus mainly (although not exclusively) on the Colavita effect present in the error data (in line with the majority of published research on this phenomenon). We start by summarizing the basic findings to have emerged from studies of the Colavita visual dominance effect conducted over the past 35 years or so. By now, many different factors have been investigated in order to determine whether they influence the Colavita effect: Here, they are grouped into stimulus-related factors (such as stimulus intensity, stimulus modality, stimulus type, stimulus position, and bimodal stimulus probability) and task/participant-related factors (such as attention, arousal, task/ response demands, and practice). A range of potential explanations for the Colavita effect are evaluated, and all are shown to be lacking. A new account of the Colavita visual dominance effect is therefore proposed, one that is based on the “biased competition” model put forward by Desimone and Duncan (1995; see also Duncan 1996; Peers et al. 2005). Although this model was initially developed in order to provide an explanation for the intramodal competition taking place between multiple visual object representations in both normal participants and clinical patients (suffering from extinction), here we propose that it can be extended to provide a helpful framework in which to understand what may be going on the Colavita visual dominance effect. In particular, we argue that a form of cross-modal biased competition can help to explain why participants respond to the visual stimulus while sometimes failing to respond to the nonvisual stimulus on the bimodal target trials in the Colavita paradigm. More generally, it is our hope that explaining the Colavita visual dominance effect may provide an important step toward understanding the mechanisms underlying multisensory interactions. First, though, we review the various factors that have been hypothesized to influence the Colavita visual dominance effect.

27.2. BASIC FINDINGS ON COLAVITA VISUAL DOMINANCE EFFECT

27.2.9. Practice Effects

The largest Colavita visual dominance effects have been reported in studies in which only a small number of bimodal target trials were presented. In fact, by far the largest effects on record were reported by Frank B. Colavita himself in his early research (see Koppen and Spence 2007a, Table 1, for a review). In these studies, each participant was only ever presented with a maximum of five or six bimodal targets (see Colavita 1974, 1982; Colavita et al. 1976; Colavita and Weisberg 1979). Contrast this with the smaller Colavita effects that have been reported in more recent research, where as many as 120 bimodal targets were presented to each participant (e.g., Hartcher-O’Brien et al. 2008; Koppen et al. 2008; Koppen and Spence 2007a, 2007c). This observation leads on to the suggestion that the Colavita visual dominance effect may be more pronounced early on in the experimental session (see also Kristofferson 1965). ^* That said, significant Colavita visual dominance effects have nevertheless still been observed in numerous studies where participants’ performance has been averaged over many hundreds of trials. Here, it may also be worth considering whether any reduction in the Colavita effect resulting from increasing the probability of (and/or practice with responding to) bimodal stimuli may also be related to the phenomenon of response coupling (see Ulrich and Miller 2008). That is, the more often two independent target stimuli happen to be presented at exactly the same time, the more likely it is that the participant will start to couple (i.e., program) their responses to the two stimuli together.

In the only study (as far as we are aware) to have provided evidence relevant to the question of the consequence of practice on the Colavita visual dominance effect, the vigilance performance of a group of participants was assessed over a 3-h period (Osborn et al. 1963). The participants in this study had to monitor a light and sound source continuously for the occasional (once every 2½ min) brief (i.e., lasting only 41 ms) offset of either or both of the stimuli. The participants were instructed to press one button whenever the light was extinguished and another button whenever the sound was interrupted. The results showed that although participants failed to respond to more of the auditory than visual targets during the first 30-min session (thus showing a typical Colavita visual dominance effect), this pattern of results reversed in the final four 30-min sessions (i.e., participants made more auditory-only than visual-only responses on the bimodal target trials; see Osborn et al. 1963; Figure 27.2). It is, however, unclear whether these results necessarily reflect the effects of practice on the Colavita visual dominance effect, or whether instead they may simply highlight the effects of fatigue or boredom after the participants had spent several hours on the task (given that auditory events are more likely to be responded to than visual events should the participants temporarily look away or else close their eyes).

FIGURE 27.2

Graph highlighting the results of Koppen and Spence’s (2007b) study of Colavita effect in which auditory and visual targets on bimodal target trials could be presented at any one of 10 SOAs. Although a significant visual dominance effect was observed (more...)

27.3. INTERIM SUMMARY

To summarize, the latest research has confirmed the fact that the Colavita visual dominance effect is a robust empirical phenomenon. The basic Colavita effect—defined here in terms of participants failing to respond to the nonvisual stimulus more often than they fail to respond to the visual stimulus on the bimodal audiovisual or visuotactile target trials—has now been replicated in many different studies, and by a number of different research groups (although it is worth noting that the magnitude of the effect has fluctuated markedly from one study to the next). That said, the Colavita effect appears to be robust to a variety of different experimental manipulations (e.g., of stimulus intensity, stimulus type, stimulus position, response demands, attention, arousal, etc.). Interestingly, though, while many experimental manipulations have been shown to modulate the size of the Colavita visual dominance effect, and a few studies have even been able to eliminate it entirely, only two of the studies discussed thus far have provided suggestive evidence regarding a reversal of the Colavita effect in humans (i.e., evidence that is consistent with, although not necessarily providing strong support for, auditory dominance; see Osborn et al. 1963; Shapiro et al. 1984).

Having reviewed the majority of the published research on the Colavita visual dominance effect, and having ruled out accounts of the effect in terms of people having a predisposition to attend endogenously to the visual modality (see Posner et al. 1976), differences in stimulus intensity (Colavita 1974), and/or difficulties associated with participants having to make two responses at the same time on the bimodal target trials (Koppen and Spence 2007a), how should the effect be explained? Well, researchers have recently been investigating whether the Colavita effect can be accounted for, at least in part, by the prior entry of the visual stimulus to participants’ awareness (see Spence 2010; Spence et al. 2001; Titchener 1908). It is to this research that we now turn.

27.4. PRIOR ENTRY AND COLAVITA VISUAL DOMINANCE EFFECT

Koppen and Spence (2007b) investigated whether the Colavita effect might result from the prior entry of the visual stimulus into participants’ awareness on some proportion of the bimodal target trials. That is, even though the auditory and visual stimuli were presented simultaneously in the majority of published studies of the Colavita effect, research elsewhere has shown that a visual stimulus may be perceived first under such conditions (see Rutschmann and Link 1964). In order to evaluate the prior entry account of the Colavita visual dominance effect, Koppen and Spence assessed participants’ perception of the temporal order of pairs of auditory and visual stimuli that had been used in another part of the study to demonstrate the typical Colavita visual dominance effect. ^* Psychophysical analysis of participants’ TOJ performance showed that when the auditory and visual stimuli were presented simultaneously, participants actually judged the auditory stimulus to have been presented slightly, although not significantly, ahead of the visual stimulus (i.e., contrary to what would have been predicted according to the prior entry account; but see Exner 1875 and Hirsh and Sherrick 1961, for similar results; see also Jaśkowski 1996, 1999; Jaśkowski et al. 1990).

It is, however, important to note that there is a potential concern here regarding the interpretation of Koppen and Spence’s (2007b) findings. Remember that the Colavita visual dominance effect is eliminated when bimodal audiovisual targets are presented too frequently (e.g., see Section 27.2.5). Crucially, Koppen and Spence looked for any evidence of the prior entry of visual stimuli into awareness in their TOJ study under conditions in which a pair of auditory and visual stimuli were presented on each and every trial. The possibility therefore remains that visual stimuli may only be perceived before simultaneously presented auditory stimuli under those conditions in which the occurrence of bimodal stimuli is relatively rare (cf. Miller et al. 2009). Thus, in retrospect, Koppen and Spence’s results cannot be taken as providing unequivocal evidence against the possibility that visual stimuli have prior entry into participants’ awareness on the bimodal trials in the Colavita paradigm. Ideally, future research will need to look for any evidence of visual prior entry under conditions in which the bimodal targets (in the TOJ task) are actually presented as infrequently as when the Colavita effect is demonstrated behaviorally (i.e., when the bimodal targets requiring a detection/discrimination response are presented on only 20% or so of the trials).

Given these concerns over the design (and hence interpretation) of Koppen and Spence’s (2007b) TOJ study, it is interesting to note that Lucey and Spence (2009) were recently able to eliminate the Colavita visual dominance effect by delaying the onset of the visual stimulus by a fixed 50 ms with respect to the auditory stimuli on the bimodal target trials. Lucey and Spence used a between-participants experimental design in which one group of participants completed the Colavita task with synchronous auditory and visual targets on the bimodal trials (as in the majority of previous studies), whereas for the other group of participants, the onset of the visual target was always delayed by 50 ms with respect to that of the auditory target. The apparatus and materials were identical to those used by Elcock and Spence (2009; described earlier) although the participants in Lucey and Spence’s study performed the three-button version of the audiovisual Colavita task (i.e., in which participants had separate response keys for auditory, visual, and bimodal targets). The results revealed that although participants made significantly more vision-only than auditory-only responses in the synchronous bimodal condition (10.3% vs. 2.4%, respectively), no significant Colavita visual dominance effect was reported when the onset of the visual target was delayed (4.6% vs. 2.9%, respectively; n.s.). These results therefore demonstrate that the Colavita visual dominance effect can be eliminated by presenting the auditory stimulus slightly ahead of the visual stimulus. The critical question here, following on from Lucey and Spence’s results, is whether auditory dominance would have been elicited had the auditory stimulus led the visual stimulus by a greater interval.

Koppen and Spence (2007b) have provided an answer to this question. In their study of the Colavita effect, the auditory and visual stimuli on the bimodal target trials were presented at one of 10 stimulus onset asynchronies (SOAs; from auditory leading by 600 ms through to vision leading by 600 ms). Koppen and Spence found that the auditory lead needed in order to eliminate the Colavita visual dominance effect on the bimodal target trials was correlated with the SOA at which participants reliably started to perceive the auditory stimulus as having been presented before the visual stimulus (defined as the SOA at which participants make 75% audition first responses; see Koppen and Spence 2007b; Figure 27.3). This result therefore suggests that the prior entry of the visual stimulus to awareness plays some role in its dominance over audition in the Colavita effect. That said, however, Koppen and Spence also found that auditory targets had to be presented 600 ms before visual targets in order for participants to make significantly more auditory-only than visual only responses on the bimodal target trials (although a similar nonsignificant trend toward auditory dominance was also reported at an auditory lead of 300 ms; see Figure 27.2).

FIGURE 27.3

(a) Schematic illustration of the results of Sinnett et al.’s (2008; Experiment 2) speeded target detection study. The figure shows how the presentation of an accessory sound facilitates visual RTs (RT_V(A)), whereas the presentation of an accessory (more...)

It is rather unclear, however, what exactly caused the auditorily dominant behavior observed at the 600 ms SOA in Koppen and Spence’s (2007b) study. This (physical) asynchrony between the auditory and visual stimuli is far greater than any shift in the perceived timing of visual relative to auditory stimuli that might reasonably be expected due to the prior entry of the visual stimulus to awareness when the targets were actually presented simultaneously (see Spence 2010). In fact, this SOA is also longer than the mean RT of participants’ responses to the unimodal auditory (440 ms) targets. Given that the mean RT for auditory only responses on the bimodal target trials was only 470 ms (i.e., 30 ms longer, on average, than the correct responses on the bimodal trials; see Koppen and Spence 2007b, Figure 1 and Table 1), one can also rule out the possibility that this failure to report the visual stimulus occurred on trials in which the participants made auditory responses that were particularly slow. Therefore, given that the visual target on the bimodal trials (in the 600 ms SOA vision-lagging condition) was likely being extinguished by an already-responded-to auditory target, one might think that this form of auditory dominance reflects some sort of refractory period effect (i.e., resulting from the execution of the participants’ response to the first target; see Pashler 1994; Spence 2008), rather than the Colavita effect proper.

In summary, although Koppen and Spence’s (2007b) results certainly do provide an example of auditory dominance, the mechanism behind this effect is most probably different from the one causing the visual dominance effect that has been reported in the majority of studies (of the Colavita effect), where the auditory and visual stimuli were presented simultaneously (see also Miyake et al. 1986). Thus, although recent research has shown that delaying the presentation of the visual stimulus can be used to eliminate the Colavita visual dominance effect (see Koppen and Spence 2007b; Lucey and Spence 2009), and although the SOA at which participants reliably start to perceive the auditory target as having been presented first correlates with the SOA at which the Colavita visual dominance effect no longer occurs (Koppen and Spence 2007b), we do not, as yet, have any convincing evidence that auditory dominance can be observed in the Colavita paradigm by presenting the auditory stimulus slightly before the visual stimulus on the bimodal target trials (i.e., at SOAs where the visual target is presented before the participants have initiated/executed their response to the already-presented auditory target). That is, to date, no simple relationship has been demonstrated between the SOA on the audiovisual target trials in the Colavita paradigm and modality dominance. Hence, we need to look elsewhere for an explanation of vision’s advantage in the Colavita visual dominance effect. Recent progress in understanding what may be going on here has come from studies looking at the effect of accessory stimuli presented in one modality on participants’ speeded responding to targets presented in another modality (Sinnett et al. 2008), and from studies looking at the sensitivity and criterion of participants’ responses in the Colavita task (Koppen et al. 2009).

27.5. EXPLAINING THE COLAVITA VISUAL DOMINANCE EFFECT

27.5.2. Perceptual and Decisional Contributions to Colavita Visual Dominance Effect

Koppen et al. (2009) recently explicitly assessed the contributions of perceptual (i.e., threshold) and decisional (i.e., criterion-related) factors to the Colavita visual dominance effect in a study in which the intensities of the auditory and visual stimuli were initially adjusted until participants were only able to detect them on 75% of the trials. Next, a version of the Colavita task was conducted using these near-threshold stimuli (i.e., rather than the clearly suprathreshold stimuli that have been utilized in the majority of previous studies). A unimodal visual target was presented on 25% of the trials, a unimodal auditory target on 25% of trials, a bimodal audiovisual target on 25% of trials (and no target was presented on the remaining 25% of trials). The task of reporting which target modalities (if any) had been presented in each trial was unspeeded and the participants were instructed to refrain from responding on those trials in which no target was presented.

Analysis of Koppen et al.’s (2009) results using signal detection theory (see Green and Swets 1966) revealed that although the presentation of an auditory target had no effect on visual sensitivity, the presentation of a visual target resulted in a significant drop in participants’ auditory sensitivity (see Figure 27.4a; see also Golob et al. 2001; Gregg and Brogden 1952; Marks et al. 2003; Odgaard et al. 2003; Stein et al. 1996; Thompson et al. 1958). These results therefore show that the presentation of a visual stimulus can lead to a small, but significant, lowering of sensitivity to a simultaneously presented auditory stimulus, at least when the participants’ task involves trying to detect which target modalities (if any) have been presented. ^* Koppen et al.’s results suggest that only a relatively small component of the Colavita visual dominance effect may be attributable to the asymmetrical cross-modal effect on auditory sensitivity (i.e., on the auditory perceptual threshold) that results from the simultaneous presentation of a visual stimulus. That is, the magnitude of the sensitivity drop hardly seems large enough to account for the behavioral effects observed in the normal speeded version of the Colavita task.

FIGURE 27.4

Summary of mean sensitivity (d’) values (panel a) and criterion (c) (panel b) for unimodal auditory, unimodal visual, bimodal auditory, and bimodal visual targets in Koppen et al.’s (2009) signal detection study of the Colavita visual (more...)

The more important result to have emerged from Koppen et al.’s (2009) study in terms of the argument being developed here was the significant drop in participants’ criterion for responding on the bimodal (as compared to the unimodal) target trials. Importantly, this drop was significantly larger for visual than for auditory targets (see Figure 27.4b). The fact that the criterion dropped for both auditory and visual targets is inconsistent with the simple criterion shifting account of the asymmetrical cross-modal effects highlighted by Sinnett et al. (2008) that were put forward in Figure 27.3b. In fact, when the various results now available are taken together, the most plausible model of the Colavita visual dominance effect would appear to be one in which an asymmetrical lowering of the criterion for responding to auditory and visual targets (Koppen et al. 2009), is paired with an asymmetrical cross-modal effect on the rate of information accrual (Miller 1986; see Figure 27.3d).

However, although the account outlined in Figure 27.3d may help to explain why it is that a participant will typically respond to the visual stimulus first on the bimodal target trials (despite the fact that the auditory and visual stimuli are actually presented simultaneously), it does not explain why participants do not quickly recognize the error of their ways (after making a vision-only response, say), and then quickly initiate an additional auditory response.^† The participants certainly had sufficient time in which to make a response before the next trial started in many of the studies where the Colavita effect has been reported. For example, in Koppen and Spence’s (2007a, 2007b, 2007c) studies, the intertarget interval was in the region of 1500–1800 ms, whereas mean vision-only response latencies fell in the 500–700 ms range.

27.5.3. Stimulus, (Perception), and Response?

We believe that in order to answer the question of why participants fail to make any response to the auditory (or tactile) targets on some proportion of the bimodal target trials in the Colavita paradigm, one has to break with the intuitively appealing notion that there is a causal link between (conscious) perception and action. Instead, it needs to be realized that our responses do not always rely on our first becoming aware of the stimuli that have elicited those responses. In fact, according to Neumann (1990), the only causal link that exists is the one between a stimulus and its associated response. Neumann has argued that conscious perception should not always be conceptualized as a necessary stage in the chain of human information processing. Rather, he suggests that conscious perception can, on occasion, be bypassed altogether. Support for Neumann’s view that stimuli can elicit responses in the absence of awareness comes from research showing, for example, that participants can execute rapid and accurate discrimination responses to masked target stimuli that they are subjectively unaware of (e.g., Taylor and McCloskey 1996). The phenomenon of blindsight is also pertinent here (e.g., see Cowey and Stoerig 1991). Furthermore, researchers have shown that people sometimes lose their memory for the second of two stimuli as a result of their having executed a response to the first stimulus (Crowder 1968; Müsseler and Hommel 1997a, 1997b; see also Bridgeman 1990; Ricci and Chatterjee 2004; Rizzolatti and Berti 1990). On the basis of such results, then, our suggestion is that a participant’s awareness (of the target stimuli) in the speeded version of the Colavita paradigm may actually be modulated by the responses that they happen to make (select or initiate) on some proportion of the trials, rather than necessarily always being driven by their conscious perception of the stimuli themselves (see also Hefferline and Perera 1963).

To summarize, when participants try to respond rapidly in the Colavita visual dominance task, they may sometimes end up initiating their response before becoming aware of the stimulus (or stimuli) that have elicited that response. Their awareness of which stimuli have, in fact, been presented is then constrained by the response(s) that they actually happen to make. In other words, if (as a participant) I realize that I have made (or am about to make) a vision-only response, it would seem unsurprising that I only then become aware of the visual target, even if an auditory target had also been presented at the same time (although it perhaps reached the threshold for initiating a response more slowly than the visual stimulus; see above). Here, one might even consider the possibility that participants simply stop processing (or stop responding to) the target stimulus (or stimuli) after they have selected/triggered a response (to the visual target; i.e., perhaps target processing reflects a kind of self-terminating processing). Sinnett et al.’s (2008) research is crucial here in showing that, as a result of the asymmetrical cross-modal effects of auditory and visual stimuli on each other, the first response that a participant makes on a bimodal target trial is likely to be to a visual (rather than an auditory) stimulus.

If this hypothesis regarding people’s failure to respond to some proportion of the auditory (or tactile) stimuli on the bimodal trials in the Colavita paradigm were to be correct, one would expect the fastest visual responses to occur on those bimodal trials in which participants make a visual-only response. Koppen and Spence’s (2007a; Experiment 3) results show just such a result in their three-response study of the Colavita effect (i.e., where participants made one response to auditory targets, one to visual targets, and a third to the bimodal targets; note, however, that the participants did not have the opportunity to respond to the visual and auditory stimuli sequentially in this study). In Koppen and Spence’s study, the visual-only responses on the bimodal target trials were actually significantly faster, on average (mean RT = 563 ms), than the visual-only responses on unimodal visual trials (mean RT = 582 ms; see Figure 27.5). This result therefore demonstrates that even though participants failed to respond to the auditory target, its presence nevertheless still facilitated their behavioral performance. Finally, the vision-only responses (on the bimodal trials) were also found, on average, to be significantly faster than the participants’ correct bimodal responses on the bimodal target trails (mean = 641 ms).

FIGURE 27.5

Schematic timeline showing the mean latency of participants’ responses (both correct and incorrect responses) in Koppen et al.’s (2007a) three-button version of the Colavita effect. Significant differences between particular conditions (more...)

Interestingly, however, participants’ auditory-only responses on the bimodal target trials in Koppen and Spence’s (2007a) study were significantly slower, on average, than on the unimodal auditory target trials (mean RTs of 577 and 539 ms, respectively). This is the opposite pattern of results to that seen for the visual target detection data (i.e., a bimodal slowing of responding for auditory targets paired with a bimodal speeding of responding to the visual targets). This result provides additional evidence for the existence of an asymmetrical cross-modal effect on the rate of information accrual). Indeed, taken together, these results mirror those reported by Sinnett et al. (2008) in their speeded target detection task, but note here that the data come from a version of the Colavita task instead. Thus, it really does seem as though the more frequent occurrence of vision-only as compared to auditory-only responses on the bimodal audiovisual target trials in the Colavita visual dominance paradigm is tightly linked to the speed with which a participant initiates his/her response. When participants respond rapidly, they are much more likely to make an erroneous visual-only response than to make an erroneous auditory-only response. ^*

27.6. BIASED (OR INTEGRATED) COMPETITION AND COLAVITA VISUAL DOMINANCE EFFECT

How can the asymmetric cross-modal effects of simultaneously presented auditory and visual targets on each other (that were highlighted in the previous section) be explained? We believe that a fruitful approach may well come from considering them in the light of the biased (or integrated) competition hypothesis (see Desimone and Duncan 1995; Duncan 1996). According to Desimone and Duncan, brain systems (both sensory and motor) are fundamentally competitive in nature. What is more, within each system, a gain in the activation of one object/event representation always occurs at a cost to others. That is, the neural representation of different objects/events is normally mutually inhibitory. An important aspect of Desimone and Duncan’s biased competition model relates to the claim that the dominant neural representation suppresses the neural activity associated with the representation of the weaker stimulus (see Duncan 1996). In light of the discussion in the preceding section (see Section 27.5.2), one might think of biased competition as affecting the rate of information accrual, changing the criterion for responding, and/or changing perceptual sensitivity (but see Gorea and Sagi 2000, 2002). An extreme form of this probabilistic winner-takes-all principle might therefore help to explain why it is that the presentation of a visual stimulus can sometimes have such a profound effect on people’s awareness of the stimuli coded by a different brain area (i.e., modality; see also Hahnloser et al. 1999).

Modality-based biased competition can perhaps also provide a mechanistic explanation for the findings of a number of other studies of multisensory information processing. For example, over the years, many researchers have argued that people’s attention is preferentially directed toward the visual modality when pairs of auditory and visual stimuli are presented simultaneously (e.g., see Falkenstein et al. 1991; Golob et al. 2001; Hohnsbein and Falkenstein 1991; Hohnsbein et al. 1991; Oray et al. 2002). As Driver and Vuilleumier (2001, p. 75) describe the biased (or integrated) competition hypothesis: “ . . . multiple concurrent stimuli always compete to drive neurons and dominate the networks (and ultimately to dominate awareness and behavior).” They continue: “various phenomena of ‘attention’ are cast as emergent properties of whichever stimuli happen to win the competition.” In other words, particularly salient stimuli will have a competitive advantage and may thus tend to “attract attention” on purely bottom-up grounds. Visual stimuli might then, for whatever reason (see below), constitute a particularly salient class of stimuli. Such stimulus-driven competition between the neural activation elicited by the auditory (or tactile) and visual targets on bimodal target trials might also help to explain why the attentional manipulations that have been utilized previously have proved so ineffective in terms of reversing the Colavita visual dominance effect (see Koppen and Spence 2007d; Sinnett et al. 2007). That is, although the biasing of a participant’s attention toward one sensory modality (in particular, the nonvisual modality) before stimulus onset may be sufficient to override the competitive advantage resulting from any stimulus-driven biased competition (see McDonald et al. 2005; Spence 2010; Vibell et al. 2007), it cannot reverse it.

27.7. CONCLUSIONS AND QUESTIONS FOR FUTURE RESEARCH

Research conducted over the past 35 years or so has shown the Colavita visual dominance effect to be a robust empirical phenomenon. However, traditional explanations of the effect simply cannot account for the range of experimental data that is currently available. In this article, we argue that the Colavita visual dominance effect may be accounted for in terms of Desimone and Duncan’s (1995; see also Duncan 1996) model of biased (or integrated) competition. According to the explanation outlined here, the Colavita visual dominance effect can be understood in terms of the cross-modal competition between the neural representations of simultaneously presented visual and auditory (or tactile) stimuli. Cognitive neuroscience studies would certainly help to further our understanding of the mechanisms underlying the Colavita effect. It would be particularly interesting, for example, to compare the pattern of brain activation on those trials in which participants fail to respond correctly to the nonvisual stimulus to the activation seen on those trials in which they respond appropriately (cf. Fink et al. 2000; Golob et al. 2001; Sarri et al. 2006; Schubert et al. 2006). Event-related potential studies could also help to determine just how early (or late, see Falkenstein et al. 1991; Quinlan 2000; Zahn et al. 1994) the processing of ignored and reported auditory (or tactile) stimuli differs (see Hohnsbein et al. 1991).

REFERENCES

1. Battelli L, Pascual-Leone A, Cavanagh P. The ‘when’ pathway of the right parietal lobe. Trends in Cognitive Sciences. 2007;11:204–210. [PMC free article: PMC3613278] [PubMed: 17379569]
2. Baylis G.C, Driver J, Rafal R. D. Visual extinction and stimulus repetition. Journal of Cognitive Neuroscience. 1993;5:453–466. [PubMed: 23964918]
3. Baylis G.C, Simon S. L, Baylis L. L, Rorden C. Visual extinction with double simultaneous stimulation: What is simultaneous? Neuropsychologia. 2002;40:1027–1034. [PubMed: 11900754]
4. Bender M.B. Disorders in perception. Springfield, IL: Charles Thomas; 1952.
5. Besle J, Bertrand O, Giard M. H. Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex. Hearing Research. 2009;258(1-2):143–151. [PubMed: 19573583]
6. Bolognini N, Senna I, Maravita A, Pasqual-Leone A, Merabeth L. B. Auditory enhancement of visual phosphene perception: The effect of temporal and spatial factors and of stimulus intensity. Neuroscience Letters. 2010;477:109–114. [PMC free article: PMC3538364] [PubMed: 20430065]
7. Bonneh Y.S, Belmonte M. K, Pei F, Iversen P. E, Kenet T, Akshoomoff N, Adini Y, Simon H. J, Moore C. I, Houde J. F, Merzenich M. M. Cross-modal extinction in a boy with severely autistic behavior and high verbal intelligence. Cognitive Neuropsychology. 2008;25:635–652. [PubMed: 18651259]
8. Bridgeman B. The physiological basis of the act of perceiving. In: Relationships between perception and action: Current approaches. Berlin: Springer; 1990. pp. 21–42.
9. Brozzoli C, Demattè M. L, Pavani F, Frassinetti F, Farnè A. Neglect and extinction: Within and between sensory modalities. Restorative Neurology and Neuroscience. 2006;24:217–232. [PubMed: 17119300]
10. Brugge J.F, Volkov I. O, Garell P. C, Reale R. A, Howard M. A. Functional connections between auditory cortex on Heschl’s gyrus and on the lateral superior temporal gyrus in humans. Journal of Neurophysiology. (third) 2003;90:3750–3763. [PubMed: 12968011]
11. Calvert G.A, Spence C, Stein B. E, editors. The handbook of multisensory processes. Cambridge, MA: MIT Press; 2004.
12. Cappe C, Barone P. Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. European Journal of Neuroscience. 2005;22:2886–2902. [PubMed: 16324124]
13. Cappe C, Morel A, Barone P, Rouiller E. M. The thalamocortical projection systems in primates: An anatomical support for multisensory and sensorimotor interplay. Cerebral Cortex. 2009;19:2025–2037. [PMC free article: PMC2722423] [PubMed: 19150924]
14. Clavagnier S, Falchier A, Kennedy H. Long-distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive, Affective, and Behavioural Neuroscience. 2004;4:117–126. [PubMed: 15460918]
15. Colavita F.B. Human sensory dominance. Perception and Psychophysics. 1974;16:409–412.
16. Colavita F.B. Visual dominance and attention in space. Bulletin of the Psychonomic Society. 1982;19:261–262.
17. Colavita F.B, Tomko R, Weisberg D. Visual prepotency and eye orientation. Bulletin of the Psychonomic Society. 1976;8:25–26.
18. Colavita F.B, Weisberg D. A further investigation of visual dominance. Perception and Psychophysics. 1979;25:345–347. [PubMed: 461094]
19. Cooper R. Visual dominance and the control of action. In: Gernsbacher M. A, Derry S. J, editors. Proceedings of the 20th Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum Associates; 1998. pp. 250–255.
20. Costantini M, Bueti D, Pazzaglia M, Aglioti S. M. Temporal dynamics of visuo-tactile extinction within and between hemispaces. Neuropsychology. 2007;21:242–250. [PubMed: 17402824]
21. Cowey A, Stoerig P. The neurobiology of blindsight. Trends in the Neurosciences. 1991;14:140–145. [PubMed: 1710851]
22. Critchley M. Tactile thought, with special reference to the blind. Brain. 1953;76:19–35. [PubMed: 13041920]
23. Crowder R.G. Repetition effects in immediate memory when there are no repeated elements in the stimuli. Journal of Experimental Psychology. 1968;78:605–609.
24. De Reuck T, Spence C. 2009. Attention and visual dominance. Unpublished manuscript.
25. Desimone R, Duncan J. Neural mechanisms of selective visual attention. Annual Review of Neuroscience. 1995;18:193–222. [PubMed: 7605061]
26. Driver J, Vuilleumier P. Perceptual awareness and its loss in unilateral neglect and extinction. Cognition. 2001;79:39–88. [PubMed: 11164023]
27. Duncan J. Cooperating brain systems in selective perception and action. In: Inui T, McClelland J. L, editors. Attention and performance XVI: Information integration in perception and communication. Cambridge, MA: MIT Press; 1996. pp. 549–578.
28. Egeth H.E, Sager L. C. On the locus of visual dominance. Perception and Psychophysics. 1977;22:77–86.
29. Elcock S, Spence C. 2009. Caffeine and the Colavita visual dominance effect. Unpublished manuscript.
30. Ernst M. A Bayesian view on multimodal cue integration. In: Knoblich G, Thornton I, Grosejan M, Shiffrar M, editors. Perception of the human body from the inside out. New York: Oxford Univ. Press; 2005. pp. 105–131.
31. Exner S. Experimentelle Untersuchung der einfachsten psychischen Processe (Experimental study of the most simple psychological processes). Archiv für die gesammte Physiologie des menschens und der Thiere (Pflüger’s Archive). 1875;11:403–432.
32. Falchier A, Clavagnier S, Barone P, Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience. 2003;22:5749–5759. [PMC free article: PMC6758216] [PubMed: 12097528]
33. Falchier A, Schroeder C. E, Hackett T. A, Lakatos P, Nascimento-Silva S, Ulbert I, Karmos G, Smiley J. F. Projection from visual areas V2 and prostriata to caudal auditory cortex in the monkey. Cerebral Cortex. 2010;20:1529–1538. [PMC free article: PMC2882821] [PubMed: 19875677]
34. Falkenstein M, Hohnsbein J, Hoormann J, Blanke L. Effects of crossmodal divided attention on late ERP components: II. Error processing in choice reaction tasks. Electroencephalography and Clinical Neurophysiology. 1991;78:447–455. [PubMed: 1712280]
35. Farnè A, Brozzoli C, Làdavas E, Ro T. Investigating multisensory spatial cognition through the phenomenon of extinction. In: Haggard P, Rossetti Y, Kawato M, editors. Attention and performance XXII: Sensorimotor foundations of higher cognition. Oxford: Oxford Univ. Press; 2007. pp. 183–206.
36. Felleman D.J, Van Essen D. C. Distributed hierarchical processing in primate cerebral cortex. Cerebral Cortex. 1991;1:1–47. [PubMed: 1822724]
37. Fink G.R, Driver J, Rorden C, Baldeweg T, Dolan R. J. Neural consequences of competing stimuli in both visual hemifields: A physiological basis for visual extinction. Annals of Neurology. 2000;47:440–446. [PubMed: 10762154]
38. Foree D.D, LoLordo V. M. J. Attention in the pigeon: Differential effects of food-getting versus shock-avoidance procedures. Journal of Comparative and Physiological Psychology. 1973;85:551–558. [PubMed: 4762141]
39. Foxe J.J, Morocz I. A, Murray M. M, Higgins B. A, Javitt D. C, Schroeder C. E. Multisensory auditory–somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cognitive Brain Research. 2000;10:77–83. [PubMed: 10978694]
40. Foxe J.J, Strugstad E. C, Sehatpour P, Molholm S, Pasieka W, Schroeder C. E, McCourt M. E. Parvocellular and magnocellular contributions to the initial generators of the visual evoked potential: High-density electrical mapping of the “C1” component. Brain Topography. 2008;21:11–21. [PubMed: 18784997]
41. Foxe J.J, Wylie G. R, Martinez A, Schroeder C. E, Javitt D. C, Guilfoyle D, Ritter W, Murray M. M. Auditory–somatosensory multisensory processing in auditory association cortex: An fMRI study. Journal of Neurophysiology. 2002;88:540–543. [PubMed: 12091578]
42. Fu K.-M. G, Johnston T. A, Shah A. S, Arnold L, Smiley J, Hackett T. A, Garraghty P. E, Schroeder C. E. Auditory cortical neurons respond to somatosensory stimulation. Journal of Neuroscience. 2003;23:7510–7515. [PMC free article: PMC6740753] [PubMed: 12930789]
43. Gallace A, Tan H. Z, Spence C. Multisensory numerosity judgments for visual and tactile stimuli. Perception and Psychophysics. 2007;69:487–501. [PubMed: 17727102]
44. Ghazanfar A.A, Maier J. X, Hoffman K. L, Logothetis N. K. Multisensory integration of dynamic faces and voices in Rhesus monkey auditory cortex. Journal of Neuroscience. 2005;25:5004–5012. [PMC free article: PMC6724848] [PubMed: 15901781]
45. Godey B, Schwartz D, de Graaf J. B, Chauvel P, Liegeois-Chauvel C. Neuromagnetic source localization of auditory evoked fields and intracerebral evoked potentials: A comparison of data in the same patients. Clinical Neurophysiology. 2001;112:1850–1859. [PubMed: 11595143]
46. Golob E.J, Miranda G. G, Johnson J. K, Starr A. Sensory cortical interactions in aging, mild cognitive impairment, and Alzheimer’s disease. Neurobiology of Aging. 2001;22:755–763. [PubMed: 11705635]
47. Gondan M, Fischer V. Serial, parallel, and coactive processing of double stimuli presented with onset asynchrony. Perception. 2009;38 Suppl:16.
48. Gorea A, Sagi D. Failure to handle more than one internal representation in visual detection tasks. Proceedings of the National Academy of Sciences of the United States of America. 2000;97:12380–12384. [PMC free article: PMC17350] [PubMed: 11050253]
49. Gorea A, Sagi D. Natural extinction: A criterion shift phenomenon. Visual Cognition. 2002;9:913–936.
50. Green D.M, Swets J. A. Signal detection theory and psychophysics. New York: Wiley; 1966.
51. Gregg L.W, Brogden W. J. The effect of simultaneous visual stimulation on absolute auditory sensitivity. Journal of Experimental Psychology. 1952;43:179–186. [PubMed: 14927820]
52. Hackett T.A, De La Mothe L, Ulbert I, Karmos G, Smiley J, Schroeder C. E. Multisensory convergence in auditory cortex: II. Thalamocortical connections of the caudal superior temporal plane. Journal of Comparative Neurology. 2007;502:924–952. [PubMed: 17444488]
53. Hahnloser R, Douglas R. J, Mahowald M, Hepp K. Feedback interactions between neuronal pointers and maps for attentional processing. Nature Neuroscience. 1999;2:746–752. [PubMed: 10412065]
54. Harris J.A, Miniussi C, Harris I. M, Diamond M. E. Transient storage of a tactile memory trace in primary somatosensory cortex. Journal of Neuroscience. 2002;22:8720–8725. [PMC free article: PMC6757763] [PubMed: 12351747]
55. Hartcher-O’Brien J, Gallace A, Krings B, Koppen C, Spence C. When vision ‘extinguishes’ touch in neurologically-normal people: Extending the Colavita visual dominance effect. Experimental Brain Research. 2008;186:643–658. [PubMed: 18301885]
56. Hartcher-O’Brien J, Levitan C, Spence C. Out-of-touch: Does vision dominate over touch when it occurs off the body? Brain Research. 2010;1362:48–55. [PubMed: 20850418]
57. Hecht D, Reiner M. Sensory dominance in combinations of audio, visual and haptic stimuli. Experimental Brain Research. 2009;193:307–314. [PubMed: 18985327]
58. Hefferline R.F, Perera T. B. Proprioceptive discrimination of a covert operant without its observation by the subject. Science. 1963;139:834–835. [PubMed: 13963703]
59. Hirsh I.J, Sherrick C. E Jr. Perceived order in different sense modalities. Journal of Experimental Psychology. 1961;62:423–432. [PubMed: 13907740]
60. Hohnsbein J, Falkenstein M. Visual dominance: Asymmetries in the involuntary processing of visual and auditory distractors. In: Blum B, editor. Channels in the visual nervous system: Neurophysiology, psychophysics and models. London: Freund Publishing House; 1991. pp. 301–313.
61. Hohnsbein J, Falkenstein M, Hoormann J. Visual dominance is reflected in reaction times and event-related potentials (ERPs). In: Blum B, editor. Channels in the visual nervous system: Neurophysiology, psychophysics and models. London: Freund Publishing House; 1991. pp. 315–333.
62. Howard M.A, Volkov I. O, Mirsky R, Garell P. C, Noh M. D, Granner M, Damasio H, Steinschneider M, Reale R. A, Hind J. E, Brugge J. F. Auditory cortex on the human posterior superior temporal gyrus. Journal of Comparative Neurology. 2000;416:79–92. [PubMed: 10578103]
63. Humphreys G.W, Romani C, Olson A, Riddoch M. J, Duncan J. Nonspatial extinction following lesions of the parietal lobe in man. Nature. 1995;372:357–359. [PubMed: 7969493]
64. Inui K, Wang X, Tamura Y, Kaneoke Y, Kakigi R. Serial processing in the human somatosensory system. Cerebral Cortex. 2004;14:851–857. [PubMed: 15054058]
65. Jaśkowski P. Simple reaction time and perception of temporal order: Dissociations and hypotheses. Perceptual and Motor Skills. 1996;82:707–730. [PubMed: 8774006]
66. Jaśkowski P. Reaction time and temporal-order judgment as measures of perceptual latency: The problem of dissociations. In: Aschersleben G, Bachmann T, Műsseler J, editors. Cognitive contributions to the perception of spatial and temporal events. North-Holland: Elsevier Science; 1999. pp. 265–282.
67. Jaśkowski P, Jaroszyk F, Hojan-Jesierska D. Temporal-order judgments and reaction time for stimuli of different modalities. Psychological Research. 1990;52:35–38. [PubMed: 2377723]
68. Johnson T.L, Shapiro K. L. Attention to auditory and peripheral visual stimuli: Effects of arousal and predictability. Acta Psychologica. 1989;72:233–245. [PubMed: 2618791]
69. Kayser C, Petkov C. I, Logothetis N. K. Visual modulation of neurons in auditory cortex. Cerebral Cortex. 2008;18:1560–1574. [PubMed: 18180245]
70. Koppen C, Alsius A, Spence C. Semantic congruency and the Colavita visual dominance effect. Experimental Brain Research. 2008;184:533–546. [PubMed: 17885751]
71. Koppen C, Levitan C, Spence C. A signal detection study of the Colavita effect. Experimental Brain Research. 2009;196:353–360. [PubMed: 19488743]
72. Koppen C, Spence C. Seeing the light: Exploring the Colavita visual dominance effect. Experimental Brain Research. 2007a;180:737–754. [PubMed: 17333012]
73. Koppen C, Spence C. Audiovisual asynchrony modulates the Colavita visual dominance effect. Brain Research. 2007b;1186:224–232. [PubMed: 18005944]
74. Koppen C, Spence C. Spatial coincidence modulates the Colavita visual dominance effect. Neuroscience Letters. 2007c;417:107–111. [PubMed: 17408855]
75. Koppen C, Spence C. Assessing the role of stimulus probability on the Colavita visual dominance effect. Neuroscience Letters. 2007d;418:266–271. [PubMed: 17398003]
76. Kristofferson A.B. Attention in time discrimination and reaction time. Washington D.C: Office of Technical Services, U.S. Department of Commerce; 1965. NASA Contractors Report 194.
77. Lamme V. A. F. Blindsight: The role of feedforward and feedback corticocortical connections. Acta Psychologica. 2001;107:209–228. [PubMed: 11388136]
78. Lamme V.A. F, Supèr H, Landman R, Roelfsema P. R, Spekreijse H. The role of primary visual cortex (V1) in visual awareness. Vision Research. 2000;40:1507–1521. [PubMed: 10788655]
79. Liegeois-Chauvel C, Musolino A, Badier J. M, Marquis P, Chauvel P. Evoked potentials recorded from the auditory cortex in man: Evaluation and topography of the middle latency components. Electroencephalography and Clinical Neuroscience. 1994;92:204–214. [PubMed: 7514990]
80. LoLordo V. M. Selective associations. In: Dickinson A, Boakes R. A, editors. Mechanisms of learning and motivation: A memorial to Jerzy Konorsky. Hillsdale, NJ: Erlbaum; 1979. pp. 367–399.
81. LoLordo V. M, Furrow D. R. Control by the auditory or the visual element of a compound discriminative stimulus: Effects of feedback. Journal of the Experimental Analysis of Behavior. 1976;25:251–256. [PMC free article: PMC1333457] [PubMed: 932617]
82. Lu Z.-L, Williamson S. J, Kaufman L. Behavioral lifetime of human auditory sensory memory predicted by physiological measures. Science. 1992;258:1669–1670. [PubMed: 1455246]
83. Lucey T, Spence C. 2009. Visual dominance. Unpublished manuscript.
84. Macaluso E, Driver J. Multisensory spatial interactions: A window onto functional integration in the human brain. Trends in Neurosciences. 2005;28:264–271. [PubMed: 15866201]
85. Macknik S.L. The role of feedback in visual attention and awareness. Perception. 2009;38 Suppl:162.
86. Macknik S, Martinez-Conde S. The role of feedback in visual masking and visual processing. Advances in Cognitive Psychology. 2007;3:125–152. [PMC free article: PMC2864985] [PubMed: 20517504]
87. Macknik S, Martinez-Conde S. The role of feedback in visual attention and awareness. In: Gazzaniga M. S. A, editor. The new cognitive neurosciences. Cambridge, MA: MIT Press; pp. 1163–1177.
88. Manly T, Robertson I. H, Galloway M, Hawkins K. The absent mind: Further investigations of sustained attention to response. Neuropsychologia. 1999;37:661–670. [PubMed: 10390027]
89. Marks L.E, Ben-Artzi E, Lakatos S. Cross-modal interactions in auditory and visual discrimination. International Journal of Psychophysiology. 2003;50:125–145. [PubMed: 14511841]
90. Martuzzi R, Murray M. M, Michel C. M, Thiran J. P, Maeder P. P, Clarke S, Meuli R. A. Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cerebral Cortex. 2007;17:1672–1679. [PubMed: 16968869]
91. McDonald J. J, Teder-Sälejärvi W. A, Di Russo F, Hillyard S. A. Neural basis of auditory-induced shifts in visual time-order perception. Nature Neuroscience. 2005;8:1197–1202. [PubMed: 16056224]
92. Meltzer D, Masaki M. A. Measures of stimulus control and stimulus dominance. Bulletin of the Psychonomic Society. 1973;1:28–30.
93. Miller J.O. Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology. 1982;14:247–279. [PubMed: 7083803]
94. Miller J.O. Time course of coactivation in bimodal divided attention. Perception and Psychophysics. 1986;40:331–343. [PubMed: 3786102]
95. Miller J.O. Channel interaction and the redundant targets effect in bimodal divided attention. Journal of Experimental Psychology: Human Perception and Performance. 1991;17:160–169. [PubMed: 1826309]
96. Miller J, Ulrich R, Rolke B. On the optimality of serial and parallel processing in the psychological refractory period paradigm: Effects of the distribution of stimulus onset asynchronies. Cognitive Psychology. 2009;58:273–310. [PubMed: 19281972]
97. Miyake S, Taniguchi S, Tsuji K. Effects of light stimulus upon simple reaction time and EP latency to the click presented with different SOA. Japanese Psychological Research. 1986;28:1–10.
98. Murray M.M, Molholm S, Michel C. M, Heslenfeld D. J, Ritter W, Javitt D. C, Schroeder C. E, Foxe J. J. Grabbing your ear: Auditory–somatosensory multisensory interactions in early sensory cortices are not constrained by stimulus alignment. Cerebral Cortex. 2005;15:963–974. [PubMed: 15537674]
99. Müsseler J, Hommel B. Blindness to response-compatible stimuli. Journal of Experimental Psychology: Human Perception and Performance. 1997a;23:861–872. [PubMed: 9180047]
100. Müsseler J, Hommel B. Detecting and identifying response-compatible stimuli. Psychonomic Bulletin and Review. 1997b;4:125–129.
101. Neumann O. Direct parameter specification and the concept of perception. Psychological Research. 1990;52:207–215. [PubMed: 2281129]
102. Nickerson R. Intersensory facilitation of reaction time: Energy summation or preparation enhancement? Psychological Review. 1973;80:489–509. [PubMed: 4757060]
103. Noesselt T, Rieger J. W, Schoenfeld M. A, Kanowski M, Hinrichs H, Heinze H.-J, Driver J. Audiovisual temporal correspondence modulates human multisensory temporal sulcus plus primary sensory cortices. Journal of Neuroscience. 2007;27:11431–11441. [PMC free article: PMC2957075] [PubMed: 17942738]
104. Occelli V, Hartcher O’Brien C, Spence C, Zampini M. Assessing the audiotactile Colavita effect in near and rear space. Experimental Brain Research. 2010;203:517–532. [PubMed: 20431874]
105. O’Connor N, Hermelin B. Sensory dominance in autistic children and subnormal controls. Perceptual and Motor Skills. 1963;16:920.
106. Odgaard E.C, Arieh Y, Marks L. E. Cross-modal enhancement of perceived brightness: Sensory interaction versus response bias. Perception and Psychophysics. 2003;65:123–132. [PubMed: 12699315]
107. Odgaard E.C, Arieh Y, Marks L. E. Brighter noise: Sensory enhancement of perceived loudness by concurrent visual stimulation. Cognitive, Affective, and Behavioral Neuroscience. 2004;4:127–132. [PubMed: 15460919]
108. Oray S, Lu Z. L, Dawson M. E. Modification of sudden onset auditory ERP by involuntary attention to visual stimuli. International Journal of Psychophysiology. 2002;43:213–224. [PubMed: 11850087]
109. Osborn W.C, Sheldon R. W, Baker R. A. Vigilance performance under conditions of redundant and nonredundant signal presentation. Journal of Applied Psychology. 1963;47:130–134.
110. Partan S, Marler P. Communication goes multimodal. Science. 1999;283:1272–1273. [PubMed: 10084931]
111. Pascual-Leone A, Walsh V. Fast backprojections from the motion to the primary visual area necessary for visual awareness. Science. 2001;292:510–512. [PubMed: 11313497]
112. Pashler H. Dual-task interference in simple tasks: Data and theory. Psychological Bulletin. 1994;116:220–244. [PubMed: 7972591]
113. Peers P.V, Ludwig C. J. H, Rorden C, Cusack R, Bonfiglioli C, Bundesen C, Driver J, Antoun N, Duncan J. Attentional functions of parietal and frontal cortex. Cerebral Cortex. 2005;15:1469–1484. [PubMed: 15689522]
114. Posner M.I, Nissen M. J, Klein R. M. Visual dominance: An information-processing account of its origins and significance. Psychological Review. 1976;83:157–171. [PubMed: 769017]
115. Quinlan P. The “late” locus of visual dominance. Abstracts of the Psychonomic Society. 2000;5:64.
116. Randich A, Klein R. M, LoLordo V. M. Visual dominance in the pigeon. Journal of the Experimental Analysis of Behavior. 1978;30:129–137. [PMC free article: PMC1332708] [PubMed: 16812092]
117. Rapp B, Hendel S. K. Principles of cross-modal competition: Evidence from deficits of attention. Psychonomic Bulletin and Review. 2003;10:210–219. [PubMed: 12747510]
118. Ricci R, Chatterjee A. Sensory and response contributions to visual awareness in extinction. Experimental Brain Research. 2004;157:85–93. [PubMed: 14968280]
119. Rizzolatti G, Berti A. Neglect as a neural representation deficit. Revue Neurologique (Paris). 1990;146:626–634. [PubMed: 2124720]
120. Rizzolatti G, Fadiga L, Fogassi L, Gallese V. The space around us. Science. 1997;277:190–191. [PubMed: 9235632]
121. Rockland K.S, Ojima H. Multisensory convergence in calcarine visual areas in macaque monkey. International Journal of Psychophysiology. 2003;50:19–26. [PubMed: 14511833]
122. Rodway P. The modality shift effect and the effectiveness of warning signals in different modalities. Acta Psychologica. 2005;120:199–226. [PubMed: 15993828]
123. Romei V, Murray M. M, Cappe C, Thut G. Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Current Biology. 2009;19:1799–1805. [PubMed: 19836243]
124. Romei V, Murray M. M, Merabet L. B, Thut G. Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: Implications for multisensory interactions. Journal of Neuroscience. 2007;27:11465–11472. [PMC free article: PMC6673223] [PubMed: 17959789]
125. Rorden C, Mattingley J. B, Karnath H.-O, Driver J. Visual extinction and prior entry: Impaired perception of temporal order with intact motion perception after unilateral parietal damage. Neuropsychologia. 1997;35:421–433. [PubMed: 9106271]
126. Rutschmann J, Link R. Perception of temporal order of stimuli differing in sense mode and simple reaction time. Perceptual and Motor Skills. 1964;18:345–352. [PubMed: 14166017]
127. Sarri M, Blankenburg F, Driver J. Neural correlates of crossmodal visual–tactile extinction and of tactile awareness revealed by fMRI in a right-hemisphere stroke patient. Neuropsychologia. 2006;44:2398–2410. [PubMed: 16765998]
128. Schroeder C.E, Foxe J. J. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Research: Cognitive Brain Research. 2002;14:187–198. [PubMed: 12063142]
129. Schroeder C.E, Foxe J. J. Multisensory convergence in early cortical processing. In: Calvert G. A, Spence C, Stein B. E, editors. The handbook of multisensory processes. Cambridge, MA: MIT Press; 2004. pp. 295–309.
130. Schroeder C.E, Lindsley R. W, Specht C, Marcovici A, Smiley J. F, Javitt D. C. Somatosensory input to auditory association cortex in the macaque monkey. Journal of Neurophysiology. 2001;85:1322–1327. [PubMed: 11248001]
131. Schroeder C.E, Mehta A. D, Givre S. J. A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. Cerebral Cortex. 1998;8:575–592. [PubMed: 9823479]
132. Schroeder C.E, Molholm S, Lakatos P, Ritter W, Foxe J. J. Human simian correspondence in the early cortical processing of multisensory cues. Cognitive Processing. 2004;5:140–151.
133. Schubert R, Blankenberg F, Lemm S, Villringer A, Curio G. Now you feel it, now you don’t: ERP correlates of somatosensory awareness. Psychophysiology. 2006;43:31–40. [PubMed: 16629683]
134. Sereno M.I, Dale A. M, Reppas J. B, Kwong K. K, Belliveau J. W, Brady T. J, Rosen B. R, Tootell R. B. H. Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science. 1995;268:889–893. [PubMed: 7754376]
135. Shapiro K.L, Egerman B, Klein R. M. Effects of arousal on human visual dominance. Perception and Psychophysics. 1984;35:547–552. [PubMed: 6483557]
136. Shapiro K.L, Jacobs W. J, LoLordo V. M. Stimulus–reinforcer interactions in Pavlovian conditioning of pigeons: Implications for selective associations. Animal Learning and Behavior. 1980;8:586–594.
137. Shapiro K.L, Johnson T. L. Effects of arousal on attention to central and peripheral visual stimuli. Acta Psychologica. 1987;66:157–172. [PubMed: 3687484]
138. Sinnett S, Soto-Faraco S, Spence C. The co-occurrence of multisensory competition and facilitation. Acta Psychologica. 2008;128:153–161. [PubMed: 18207117]
139. Sinnett S, Spence C, Soto-Faraco S. Visual dominance and attention: The Colavita effect revisited. Perception and Psychophysics. 2007;69:673–686. [PubMed: 17929691]
140. Smiley J, Hackett T. A, Ulbert I, Karmos G, Lakatos P, Javitt D. C, Schroeder C. E. Multisensory convergence in auditory cortex: I. Cortical connections of the caudal superior temporal plane in Macaque monkey. Journal of Comparative Neurology. 2007;502:894–923. [PubMed: 17447261]
141. Smith A. Effects of caffeine on human behavior. Food Chemistry and Toxicology. 2002;40:1243–1255. [PubMed: 12204388]
142. Smith A.P, Kendrick A. M, Maben A. L. Effects of breakfast and caffeine on performance and mood in the late morning and after lunch. Neuropsychobiology. 1992;26:198–204. [PubMed: 1299795]
143. Smith W.F. The relative quickness of visual and auditory perception. Journal of Experimental Psychology. 1933;16:239–257.
144. Soto-Faraco S, Alsius A. Conscious access to the unisensory components of a cross-modal illusion. Neuroreport. 2007;18:347–350. [PubMed: 17435600]
145. Soto-Faraco S, Alsius A. Deconstructing the McGurk–MacDonald illusion. Journal of Experimental Psychology: Human Perception and Performance. 2009;35:580–587. [PubMed: 19331510]
146. Spence C. Cognitive neuroscience: Searching for the bottleneck in the brain. Current Biology. 2008;18:R965–R968. [PubMed: 18957255]
147. Spence C. Prior entry: Attention and temporal perception. In: Nobre A. C, Coull J. T, editors. Attention and time. Oxford: Oxford Univ. Press; 2010. pp. 89–104.
148. Spence C, Baddeley R, Zampini M, James R, Shore D. I. Crossmodal temporal order judgments: When two locations are better than one. Perception and Psychophysics. 2003;65:318–328. [PubMed: 12713247]
149. Spence C, Nicholls M. E. R, Driver J. The cost of expecting events in the wrong sensory modality. Perception and Psychophysics. 2001a;63:330–336. [PubMed: 11281107]
150. Spence C, Shore D. I, Klein R. M. Multisensory prior entry. Journal of Experimental Psychology: General. 2001b;130:799–832. [PubMed: 11757881]
151. Spence C, Soto-Faraco S. Auditory perception: Interactions with vision. In: Plack C, editor. Auditory perception. Oxford: Oxford Univ. Press; 2009. pp. 271–296.
152. Sperdin H.F, Cappe C, Foxe J. J, Murray M. M. Early, low-level auditory–somatosensory multisensory interactions impact reaction time speed. Frontiers in Integrative Neuroscience. 2009;3(2):1–10. [PMC free article: PMC2659167] [PubMed: 19404410]
153. Stein B.E, London N, Wilkinson L. K, Price D. P. Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis. Journal of Cognitive Neuroscience. 1996;8:497–506. [PubMed: 23961981]
154. Stone J.V, Hunkin N. M, Porrill J, Wood R, Keeler V, Beanland M, Port M, Porter N. R. When is now? Perception of simultaneity. Proceedings of the Royal Society (B). 2001;268:31–38. [PMC free article: PMC1087597] [PubMed: 12123295]
155. Tajadura-Jiménez A, Kitagawa N, Väljamäe A, Zampini M, Murray M. M, Spence C. Auditory–somatosensory multisensory interactions are spatially modulated by stimulated body surface and acoustic spectra. Neuropsychologia. 2009;47:195–203. [PubMed: 18760293]
156. Taylor J.L, McCloskey D. I. Selection of motor responses on the basis of unperceived stimuli. Experimental Brain Research. 1996;110:62–66. [PubMed: 8817257]
157. Thompson R.F, Voss J. F, Brogden W. J. Effect of brightness of simultaneous visual stimulation on absolute auditory sensitivity. Journal of Experimental Psychology. 1958;55:45–50. [PubMed: 13502524]
158. Titchener E.B. Lectures on the elementary psychology of feeling and attention. New York: Macmillan; 1908.
159. Turatto M, Benso F, Galfano G, Gamberini L, Umilta C. Non-spatial attentional shifts between audition and vision. Journal of Experimental Psychology: Human Perception and Performance. 2002;28:628–639. [PubMed: 12075893]
160. Uetake K, Kudo Y. Visual dominance over hearing in feed acquisition procedure of cattle. Applied Animal Behaviour Science. 1994;42:1–9.
161. Ulrich R, Miller J. Response grouping in the psychological refractory period (PRP) paradigm: Models and contamination effects. Cognitive Psychology. 2008;57:75–121. [PubMed: 18262510]
162. Uusitalo M.A, Williamson S. J, Seppä M. T. Dynamical organisation of the human visual system revealed by lifetimes of activation traces. Neuroscience Letters. 1996;213:149–152. [PubMed: 8873136]
163. Van Damme S, Crombez G, Spence C. Is the visual dominance effect modulated by the threat value of visual and auditory stimuli? Experimental Brain Research. 2009a;193:197–204. [PubMed: 18953530]
164. Van Damme S, Gallace A, Spence C, Moseley G. L. Does the sight of physical threat induce a tactile processing bias? Modality-specific attentional facilitation induced by viewing threatening pictures. Brain Research. 2009b;1253:100–106. [PubMed: 19094970]
165. Vibell J, Klinge C, Zampini M, Spence C, Nobre A. C. Temporal order is coded temporally in the brain: Early ERP latency shifts underlying prior entry in a crossmodal temporal order judgment task. Journal of Cognitive Neuroscience. 2007;19:109–120. [PubMed: 17214568]
166. Wang Y, Celebrini S, Trotter Y, Barone P. Visuo-auditory interactions in the primary visual cortex of the behaving monkey: Electrophysiological evidence. BMC Neuroscience. 2008;9:79. [PMC free article: PMC2527609] [PubMed: 18699988]
167. Wilcoxin H.C, Dragoin W. B, Kral P. A. Illness-induced aversions in rat and quail: Relative salience of visual and gustatory cues. Science. 1971;171:826–828. [PubMed: 5541167]
168. Zahn T.P, Pickar D, Haier R. J. Effects of clozapine, fluphenazine, and placebo on reaction time measures of attention and sensory dominance in schizophrenia. Schizophrenia Research. 1994;13:133–144. [PubMed: 7986770]
169. Zwyghuizen-Doorenbos A, Roehrs T. A, Lipschutz L, Timms V, Roth T. Effects of caffeine on alertness. Psychopharmacology. 1990;100:36–39. [PubMed: 2296626]
170. Zylberberg A, Dehaene S, Mindlin G. B, Sigman M. Neurophysiological bases of exponential sensory decay and top-down memory retrieval: A model. Frontiers in Computational Neuroscience. 2009;3(4):1–16. [PMC free article: PMC2659975] [PubMed: 19325713]

Footnotes

*

That is, the visual target was only turned off once the participants made a visual response, and the auditory target was only turned off when the participants made an auditory response. This contrasts with Colavita’s (1974) studies, in which a participant’s first response turned off all the stimuli, and with other more recent studies in which the targets were only presented briefly (i.e., for 50 ms; e.g., Koppen and Spence 2007a, 2007b, 2007c, 2007d).

*

Note that researchers have also manipulated the relative probability of unimodal auditory and visual targets (see Egeth and Sager 1977; Quinlan 2000; Sinnett et al. 2007). However, since such probability manipulations have typically been introduced in the context of trying to shift the focus of a participant’s attention between the auditory and visual modalities, they will be discussed later (see Section 27.2.7).

*

Caffeine is a stimulant that accelerates physiological activity, and results in the release of adrenaline and the increased production of the neurotransmitter dopamine. Caffeine also interferes with the operation of another neurotransmitter: adenosine (Smith 2002; Zwyghuizen-Doorenbos et al. 1990).

*

Note that if practice were found to reduce the magnitude of the Colavita visual dominance effect, then this might provide an explanation for why increasing the probability of occurrence of bimodal target trials up to 90% in Koppen and Spence’s (2007d) study has been shown to eliminate the Colavita effect (see Section 27.2.5). Alternatively, however, increasing the prevalence (or number) of bimodal targets might also lead to the increased coupling of a participants’ responses on the bimodal trials (see main text for further details; Ulrich and Miller 2008).

*

Note the importance of using the same stimuli within the same pool of participants, given the large individual differences in the perception of audiovisual simultaneity that have been reported previously (Smith 1933; Spence 2010; Stone et al. 2001).

*

Note here that a very different result (i.e., the enhancement of perceived auditory intensity by a simultaneously-presented visual stimulus) has been reported in other studies in which the participants simply had to detect the presence of an auditory target (see Odgaard et al. 2004). This discrepancy highlights the fact that the precise nature of a participant’s task constitutes a critical determinant of the way in which the stimuli presented in different modalities interact to influence human information processing (cf. Gondan and Fisher 2009; Sinnett et al. 2008; Wang et al. 2008, on this point).

†

Note here that we are talking about the traditional two-response version of the Colavita task. Remember that in the three-response version, the participant’s first response terminates the trial, and hence there is no opportunity to make a second response.

*

One final point to note here concerns the fact that when participants made an erroneous response on the bimodal target trials, the erroneous auditory-only responses were somewhat slower than the erroneous vision-only responses, although this difference failed to reach statistical significance.

*

Note here also the fact that visual influences on primary and secondary auditory cortex are greatest when the visual stimulus leads the auditory stimulus by 20–80 ms (see Kayser et al. 2008), the same magnitude of visual leads that have also been shown to give rise to the largest Colavita effect (see Figure 2; Koppen and Spence 2007b).

*

It would be interesting here to determine whether the feedforward projections between primary auditory and tactile cortices are any more symmetrical than those between auditory and visual cortices (see Cappe and Barone 2005; Cappe et al. 2009; Hackett et al. 2007; Schroeder et al. 2001; Smiley et al. 2007, on this topic), since this could provide a neural explanation for why no Colavita effect has, as yet, been reported between the auditory and tactile modalities (Hecht and Reiner 2009; Occelli et al. 2010). That said, it should also be borne in mind that the nature of auditory-somatosensory interactions have recently been shown to differ quite dramatically as a function of the body surface stimulated (e.g., different audio–tactile interactions have been observed for stimuli presented close to the hands in frontal space vs. close to the back of the neck in rear space; see Fu et al. 2003; Tajadura-Jiminez et al. 2009; cf. Critchley 1953, p. 19). The same may, of course, also turn out to be true for the auditory–tactile Colavita effect.

*

One slight complication here though relates to the fact that people typically start to couple multiple responses to different stimuli into response couplets under the appropriate experimental conditions (see Ulrich and Miller 2008). Thus, one could argue about whether participants’ responses on the bimodal target trials actually counts as a third single (rather than dual) task, but one that, in the two-response version of the Colavita task involves a bi-finger, rather than a unifingered response. When considered in this light, the interference of performance seen in the Colavita task does not seem quite so surprising.