Immediate post-saccadic
information mediates space constancy
Heiner Deubel1, Bruce Bridgeman2
and Werner X. Schneider1
1Department of
Experimental Psychology, Ludwig-Maximilians-University
Leopoldstrab e 13,
80802 Munich
and
2Max-Planck-Institute
for Psychological Research
Leopoldstrab e 24, 80802 Munich
Germany
2Permanent
address:
Program
in Experimental Psychology
University
of California, Santa Cruz
Santa
Cruz, Ca. 95064 USA
Running
head: Post-saccadic information mediates space constancy
Keywords:
Displacement threshold; Eye movement; Saccade; Extraretinal eye position
information; Transsaccadic memory; Saccadic suppression; Space perception;
Spatial vision
ABSTRACT
We recently demonstrated that the
perceived stability of a visual target that is displaced during a saccade
critically depends on whether the target is present immediately when the
saccade ends; blanking a target during and just after a saccade makes its intrasaccadic
displacement more visible (Deubel, Schneider & Bridgeman, 1996). Here, we
investigate the interaction of visual context and blanking. Subjects saw a
saccade target and an equal-sized distractor. During a saccade one or the other
was displaced left or right. At the same time, one of the objects could be
blanked briefly. Subjects reported whether the target or the distractor had
jumped. The object that was blanked was more often seen as jumping (Experiment
1), regardless of which object really jumped, implying that continuously
visible objects are preferentially perceived as stable. When both objects were
blanked, longer blanking led to better accuracy at identifying which had jumped
during a saccade. When one object was jumped and the other, stationary object
was blanked (Experiment 2), the blanked object was mistakenly seen as jumping
until the jump covered 50% or more of the saccade amplitude. In Experiment 3 a
large continuously present texture underwent an undetected jump during a
saccade, biasing judgements of simultaneous jumps of a blanked target. The
results demonstrate that space constancy in normal situations is dominated by
the assumption that a continuously present pattern is stable -- this pattern
becomes the spatial reference for the postsaccadic recalibration of perceptual
space.
INTRODUCTION
Space constancy, limited here to
maintaining apparent stability of the visual world despite saccadic eye
movements, is normally perfect -- the world does not appear to jump in the
slightest when the eye moves. The earliest attempts to account for space
constancy were cancellation theories, in which the sensory effects of an eye
movement are compensated by a simultaneous, equal and opposite extraretinal
signal about the position of the eyes in the orbit. The retinal and
extraretinal signals cancel each other somewhere in the brain, resulting in a
space-constant representation of visual space. In these theories an oculomotor
efference copy, proprioception, or some combination of both subtracts from the
disturbing effects of a displaced retinal image following a saccade (Von Holst
& Mittelstaedt, 1950).
Cancellation theories cannot support
space constancy unaided, however, because the extraretinal signals are not
exact copies of the actual eye movement. Their gain (ratio of extraretinal
signal to actual eye movement) is usually less than one (Grüsser, Krizic &
Weiss, 1987), so they are too small to afford complete compensation. And the
gain also depends on other parameters such as oculomotor dynamics (reviewed by
Bridgeman, 1995). Even a small error would result in a disturbance of
constancy.
One compelling solution to this
problem is that the visual system has the built-in assumption that the world as
a whole does not change during an eye movement. The large background region of
the optic array (as opposed to smaller foreground objects) usually does not
move in the real world. A visual system that took advantage of this fact would
simply ignore all displacements of the optic array as a whole, assigning them
instead to movements of the eye. In this case the solution to the problem of
why the world does not seem to move during an eye movement would be simple:
"Why should it move? The movement of the eye and its retina is registered
instead; The retina is proprioceptive." (Gibson, 1966, p. 256).
Unfortunately this simple and powerful idea is easily disproved: a tap on the
side of the eye results in apparent movement of the entire optic array,
background and all; further, a large afterimage, viewed in darkness, also seems
to jump with each saccade (Helmholtz, 1866). So a more subtle solution must be
sought.
An important component of space
constancy is saccadic suppression of image displacement (SSID). SSID is an
increase in the threshold for detecting a target displacement that takes place
during or near a saccadic eye movement. Subjects fail to detect a substantial
displacement of a continuously visible image, if the displacement takes place
around the time of a saccade (Bridgeman, Hendry & Stark, 1975; Mack, 1970).
Obviously, visual information about object motion that normally would enable
detection of the jump at low threshold is severely degraded during a saccade. Without
this direct evidence for a target jump, detection of intrasaccadic image
displacement requires the comparison of pre- and postsaccadic target locations;
SSID implies either that such transsaccadic information about the location of
objects is not available to the visual system, or that the required precise
comparison is normally not performed. It has been suggested that SSID provides
an important function in the maintenance of visual stability: it may bridge the
errors that occur when imprecise extraretinal signals and retinal input are
combined for spatial localizations (Bridgeman, Van der Heijden, &
Velichkovsky, 1994).
SSID seems to imply that precise
spatial information about the location of objects is not available across the
saccade. Yet perceptual-motor coordination remains intact even for actions
toward stimuli whose displacements are not perceived (Bridgeman, Lewis, Heit
& Nagle, 1979; Prablanc & Martin, 1992). So, although spatial
information about the location of objects cannot mediate perception of changes,
it remains available at a motor level.
We recently demonstrated that a
simple manipulation of the stimulus allows the perceptual system to regain
access to precise spatial information: SSID largely disappears if a target is
absent for a short temporal period when the eye stops at the end of a saccade
(Deubel & Schneider, 1994; Deubel, Schneider & Bridgeman, 1994; 1996). Blanking
a target during a saccade, and restoring it 50 - 300 ms later, restores the
detectability of even quite small target displacements. This blanking effect
occurs even for targets in darkness, implying that displacement detection under
this condition relies on extraretinal signals rather than retinal information
from the structured environment. The beneficial effect of blanking disappears
if the target reappears before the saccade ends (Deubel et al. 1996, exp. 3) or
if onset of the blank interval is delayed beyond the time range of SSID. In
contrast to the interpretation of SSID that transsacadic information about
spatial positions is poor, this effect requires both the maintenance of
high-quality information about presaccadic target position across the saccade,
and a precise extraretinal signal.
These findings imply that
information about pre-saccadic target position and precise extraretinal signals
is available for stimulus localizations after the saccade, but they ordinarily
are not used in perception. We have suggested that this is because the visual
system assumes by default the stability of an object that is continuously
available both before and after the saccade. A very large discrepancy between eye
movement magnitude and image position is normally required to break this
assumption. This assumption is also broken, however, when the object is not
present immediately after the saccade, only under this condition are precise
transsaccadic information and extraretinal signals used to achieve displacement
detection. Because of its strong effect in unveiling information available
transsaccadically, blanking presents a tool for studying visual stability and
the nature of spatial information transferred across the saccade.
The blanking effect shows the
importance of a stimulus that is present immediately after a saccade. According
to our theoretical interpretation, the visual system seeks the saccade goal
immediately after a saccade (Deubel, Wolf, & Hauske, 1984; Irwin, McConkie,
Carlson-Radvansky, & Currie, 1994). If this target is found within a
certain spatial and temporal window, the visual system assumes it to have
remained stable during the saccade, and the target becomes a 'reference object'
to determine the positions of other objects and textures (Deubel et al., 1996;
McConkie & Currie, 1996). This reference object idea will be further
investigated here, and the theory will be developed more fully in the general
discussion.
The aim of this study is to use the
blanking effect to analyze the role of the immediate postsaccadic information
in the maintenance of visual stability. We first investigate the effect of
blanking on perceived stability in a more complex visual context consisting of
two stimuli, a saccade goal and a distractor. For this situation, the reference
object theory makes two predictions. First, in a two-object stimulus field, the
visual system should exhibit a bias to perceive the saccade target as stable,
and to attribute relative displacements to the distractor. Second, if one of
the two objects is blanked beyond the critical temporal period, but the other
is not, the continuously available stimulus should take the role of the
reference object which is then seen as stable even if it jumps during the
saccade -- the jump should be attributed to the blanked object.
Experiment 1 examines the predicted
interactions between the saccadic target and a distractor of similar size;
blanking interval and jump are varied factorially. Experiment 2 establishes the
minimum displacement size required to perceive the continuous stimulus as
displaced when the other is blanked, while experiment 3 examines the
interaction between a small saccade target and a large background texture.
GENERAL METHODS
Subjects
Five
paid subjects participated in Experiment 1 and 3, and six subjects in
Experiment 2. Their ages ranged from 21 to 49 years. With one exception they
were naive with respect to the object of the study, but all were experienced
with the equipment from other eye-movement related tasks. Each performed at
least three separate sessions in each paradigm. For each experiment, the
results are based on 100-200 trials per condition from each subject. We used
within-subjects designs in which all of the conditions of an experiment are
replicated in each subject.
Apparatus
The experiments took place in a
laboratory room providing an ambient illumination of approximately 0.1 cd/m2.
All stimuli were presented on a 21" video monitor (CONRAC 7550 C21) in
combination with a TIGA graphics board (KONTRAST 8000). The monitor's spatial
resolution was 1024x768 pixels at a frame rate of 100 Hz. Screen background
luminance was set to 3 cd/m2; the luminance of the stimuli was 25
cd/m2. In order to assure that results were not affected by phosphor
persistence, we measured the temporal decay of the phosphor luminescence with a
linear PIN diode (Wolf and Deubel, 1997). Due to the steady background
luminance, contrast of the stimuli decayed to undetectable levels within 10 ms,
excluding an effect of phosphor persistence on the data. In a previous
experiment (Deubel et al., 1996, experiment 2a) we demonstrated that similar
psychophysical effects are obtained in complete darkness, with fixation points
and targets defined by a laser that could be turned on and off within a few
microsec. Comparison of this experiment with our other experiments showed that
phosphor persistence or visual frame effects did not have a measurable
influence on the blanking effect.
The subject viewed the screen
binocularly from a distance of 80 cm. Head movements were restricted by a
biteboard and a forehead rest. Eye movements were measured with a SRI
Generation 5.5 Purkinje-image eyetracker (Crane & Steele, 1985) sampled at
400 Hz. Its frequency response is better than 250 Hz with a noise level
equivalent to about 20 arc sec rms. The eyetracker can follow saccades of 15
degrees or more without losing the eye.
Experiments were controlled by a 486
PC, which also performed automatic off-line analysis of the eye movement data
in which saccadic latencies and saccade start and end positions were
determined. The computer detected saccade onset by digital differentiation of
the sampled eye position signal. Saccade-related sensory events were triggered
when instantaneous eye velocity exceeded 30 deg/sec. Early triggering is
critical because of an unavoidable delay in Purkinje-image eyetracker records
due to lens slippage within the eye (Deubel & Bridgeman, 1995) and a
display delay of up to 10 ms because of screen raster sampling. Early
triggering insured that stimulus modifications occurred before the eye reached
maximum velocity.
Calibration
and data analysis
Each session started with a
calibration procedure: the subject sequentially fixated 9 positions arranged on
a circular array of 8 degrees radius. The eyetracker behaved linearly within
this range. Static accuracy of the eyetracker was better than 0.1 degrees. Dynamically,
however, the eyetracker registers a delayed saccade onset and artifactual
overshoots at the end of each saccade due to the movement of the eye lens
relative to the optical axis of the eye (Deubel & Bridgeman, 1995). To
determine direction of gaze, an off-line program searched the eye position
record for the end of the overshoot and then calculated mean eye position over
a 40 ms time window. The eye movement analysis program calculated latencies and
start and landing positions of all saccades occurring in each trial.
Behavioral
paradigm
In each trial a target jumped left
or right 6 degrees or 8 degrees. The subject's task was to maintain fixation on
the target, and to track it with a saccade as it jumped across the visual
field. The two amplitudes and two directions were randomized and equally
probable to minimize anticipation and adaptation by the subjects. Saccades
beginning earlier than 100 ms or later than 400 ms after the target step were
discarded. When the computer detected the saccade elicited by the first target
step, a second, smaller jump and/or a blanking of either target or distractor
was triggered (Figure 1). All stimulus jumps occurred between single frames of
the display. At the end of each trial, in a two-alternative forced choice
procedure, the subject's task in experiment 1 and 2 was to report which of two
patterns, the target or a distractor, had moved during the saccade. In
experiment 3, the subject had to decide whether the target moved in the same
direction as the saccade or in the opposite direction. The final position of
the target served as the starting position for the next trial.
<Figure
1 about here>
EXPERIMENT
1: INTERACTION OF TARGET AND DISTRACTOR
In the first experiment we extend
our previous analysis of the blanking effect to a situation where the saccade
target and a distractor are present. One or the other stimulus could be blanked
during and for a short period after the primary saccade. A blank period should
prevent this object from being found and used as a reference object following
the saccade. This object should not be perceived as stable. We investigate the
strength of this blanking effect, and also whether the object that was not the
saccade target can function as a reference object. In this two-object
environment, the visual system cannot take advantage of the normally correct
generalization that the largest part of the visual field is the spatially
stable part (Gibson, 1966).
METHOD
Subjects began each trial by
fixating a small x-shaped target, 0.2 degrees in width (Figure 1). After a random
delay of 800 - 1200 ms the fixation was extinguished and the saccade target was
presented, 6 or 8 degrees in the visual periphery. Above this target appeared
an outline circle of the same angular size, referred to as the distractor. The
distractor's position was to be ignored for purposes of saccade targeting.
Before the saccade the target and
the distractor were vertically aligned, but during the saccade one of them was
displaced left or right so that vertical alignment was broken. Displacement
magnitude was fixed at 0.5 degrees. During and after the saccade either or both
of the patterns could be blanked for a short temporal interval. Two
displacement directions, times displacement of either pattern, times target
blank/continuous presentation times distractor blank/continuous presentation
yielded 16 experimental conditions in factorial combination. The subject's task
was to determine which of the two objects had jumped during the saccade, in a
two-alternative forced-choice procedure. Subjects were explicitly instructed to
report displacements, and to ignore blanking.
The blanking interval was set to 100
ms. In addition, 8 of the conditions were repeated with a 50 ms blanking
interval (see Figure 2). All 24 of the resulting conditions were run simultaneously
in random order for each subject. All five subjects were experienced
psychophysical observers; one of them (BB) was familiar with the aims of the
experiment.
The matrix of results was analyzed
with Statistica's MANCOVA routine. A single summary statistic was calculated
for each subject in each condition and results analyzed between subjects. Separate
analyses were done for target displacement and for distractor displacement
conditions, since the number of cells was different in the two conditions. Data
for all cells where one or both of the stimuli were blanked were entered as
differences from the proportion correct in the corresponding no-blanking cell. Because
the data are proportions, a square root arcsin transformation was applied to
each cell. Specific contrasts were interpreted with t-tests.
RESULTS
Four
of the five subjects showed no consistent bias toward correctly detecting
displacements in the direction of the saccade versus displacements in the
opposite direction. The fifth subject (BB) was consistently more often correct
for jumps in the direction opposite the saccade, but because this did not
affect the overall pattern of results, the two directions of motion were
collapsed in all subjects for further analysis and graphical display. Mean latency
of the primary saccades to the appearance of the stimuli was 171.8 +- 44
ms(SD). Mean saccade amplitudes to the targets at 6 degrees and 8 degrees were
5.78 +- 0.61 degrees and 7.56 +- 1.0 degrees, respectively.
<Figure
2 here>
In the top row of Figure 2 the
target is displaced in each trial. The leftmost points in each graph (filled
squares) represent subject bias when both stimuli were presented continuously,
without blanking. There is a range of biases centered around an overall mean
close to 50%, representing the tendencies of individual subjects to perceive
motion of either the target or the distractor. The two subjects below 50% show
a bias of perceiving the distractor as displaced, while the other three
subjects rather tend to attribute the relative displacement to the target. Thus,
when both patterns are present continuously, there are large intersubject
differences in preference for seeing either target or distractor as jumping,
but a between-subjects t-test for matched pairs shows that the tendency to see
the target move under the target-moved condition is not significantly different
from the tendency to see the distractor move under the distractor-moved
condition (t4 = 0.36, p = 0.73).
The open circles in the top row of
graphs in Figure 2 result from including a blanking period for the target,
while the distractor remains continuously present. The filled circles show the
result of the converse manipulation, applying the blanking period for the
distractor. All subjects tended to see the blanked stimulus as displaced,
regardless of which was actually displaced. ANOVA showed that blanking had a
significant effect for the conditions where the target was displaced (F(2,6) =
32.13, p < 0.001). Because measures were made at two blanking intervals, 50
and 100 ms, the effect of interval could also be tested. The effect of interval
was not significant (F(1,3) = 0.045, p = 0.845), showing that the blanking
effect was not significantly stronger at 100 ms than at 50 ms. Thus the effects
of blanking the target are already strong 50 ms after saccade detection. At the
longest blank interval (right side of each graph in Figure 2) the individual
differences were considerably reduced, probably due to ceiling and floor
effects. At this interval the difference between the target blanked and the
distractor blanked conditions was statistically significant (t-test for matched
pairs, t4 = 19.98, p < 0.001).
An analogous result was found when
the distractor was displaced, as shown in the bottom row of graphs in Figure 2.
Again the main effect of blanking was statistically significant (F(2,6) =
137.4, p < 0.0001). The target was seen as jumping if it was blanked,
resulting in a low percent correct; if the distractor was blanked, however, it
was almost always perceived correctly if it indeed jumped. Again the difference
is statistically significant for a 100 ms blank (t3 = 16.72, p =
0.0005).
When both target and distractor were
given identical blanking periods, the blanking induced a slight but significant
tendency to identify the jumped stimulus more accurately, whether it was the
target (inverted triangles in Figure 2, top) or the distractor (bottom). For
statistical analysis the difference between the percent correct in the 100ms
blanking condition and in the corresponding no-blank condition was calculated
for both the target-moved and the distractor-moved paradigms, and the resulting
data averaged in each subject. A between-subjects t-test for matched pairs
showed a significant enhancement in accuracy (t4 = 5.75, p = 0.0045)
demonstrating that blanking increases the subjects' accuracy even when both
patterns are given the same blanking intervals. The intermediate blanking
interval was not run for the conditions shown at the bottom in Figure 2.
DISCUSSION
The results from the 24 conditions
probed here can be described compactly in a few generalizations. First, without
any blanking the saccade target has no systematic advantage to obtain the role
as a stable reference over the distractor; some subjects then rather perceive
the target as displaced, while others tend to see the distractor moving. Second,
whatever stimulus has a blank is more likely to be seen as displaced, whether
that stimulus was actually displaced or not -- space constancy is extended
preferentially to objects that remain in the visual field throughout a trial,
regardless of which pattern actually jumped. Third, even a brief blank interval
has a strong effect. The effect of blanking the target appears gradually, not
abruptly, though it is clearly present even at 50 ms. Finally, a blank interval
improves performance when the entire configuration is blanked, again with a
tendency for a longer blank to yield a greater advantage.
The data from Experiment 1 clarify
the role of post-saccadic target information and context in space constancy. First,
they confirm our previous results that the presence or absence of a stimulus
immediately after a saccade determines whether extraretinal eye position
information and information stored in transsaccadic memory are discarded or
used for spatial localization (Deubel et al., 1996). This is reflected in the
finding that a blank interval improves performance when it is extended to both
of the two stimuli. Our interpretation is that blanking both stimuli simply
allows the visual system to process both stimuli in the same way, allowing the
blanking effect to overcome the tendency to perceive an object as stable
regardless of its actual intrasaccadic displacement. Only when no visual
information is available after saccade end is extraretinal information used in
the perception of displacement. In our experiment the motion transient of the
continuous but displaced object was masked by saccadic suppression, so that the
transient did not affect judgements of which object had been displaced.
Second, and more importantly, the
data demonstrate that immediate post-saccadic information determines whether
objects are perceived as stable or as moving across the saccade -- presence
within the first 50 ms after a saccade is crucial for a stimulus to become the
spatial reference. Accordingly, the object that is present in the visual field
in the critical time when the eye lands is always perceived as stable --
displacements are consistently attributed to the blanked target. Thus, in this
case, extraretinal information is of no importance; perception of stability is
dominated by presence and spatial position of the continuous (reference)
object. This is demonstrated by the strong illusions of transsaccadic
instability through manipulation of the immediate post-saccadic stimulus.
Finally, the results speak against
our previous prediction that the saccade target rather than the distractor
should be perceived as stable; at least for the two-stimulus configuration with
closely spaced target and distractor used in this experiment both stimuli are
equally likely to be seen as stable.
EXPERIMENT 2: SPATIAL
LIMITS FOR DISPLACEMENT DISCRIMINATION
The first experiment demonstrated
that a manipulation of immediate postsaccadic visual information can lead to
the illusory perception of displacements of other, temporarily blanked objects.
The induced effects are amazingly large and consistent; with a 0.5 degrees
displacement and a 100 ms blank, biases for perceiving the blanked object as
moving approached 100% in all subjects. Experiment 2 expands the range of
displacements under these conditions to find a displacement large enough to
overcome the tendency to perceive a continuous (but shifted) target as stable. With
this experiment we titrate the stabilizing effect of the temporal continuity of
the reference object against the calibrating effect of extraretinal information
that indicates stationarity of the blanked (undisplaced) object. The effects of
extraretinal signals are in equilibrium with the assumption of stability of
continuous targets when the displaced (but continuous) stimulus is seen as
moving 50% of the time.
METHOD
During the primary saccade one of
the two visual objects jumped while the other was blanked. Gap duration was
fixed at 100ms. For the conditions where the continuously presented object was
displaced, displacement magnitude was varied in a range from -3.5 degrees to +
3.5 degrees in steps of 0.5 - 1 degrees. From the results of the previous
experiment we expected close to perfect performance for the cases where the
blanked object is also displaced; therefore, in these conditions only
displacement magnitudes of +-0.5 degrees were applied. Two subjects from
Experiment I and four new subjects were run. Other methods are as described
above.
RESULTS
Mean latency (+-SD) of the primary
saccades to the appearance of the stimuli was 176.5 +- 32.2 ms. Mean saccade
amplitude to the targets at 6 degrees and 8 degrees was 5.8 +- 0.44 degrees and
7.78 +- 0.58 degrees, respectively.
Figure 3 shows the performance of
the six subjects in correctly attributing the displacement to the jumped
stimulus. It can be seen that for the cases where the continuous stimulus was
displaced (open circles and solid triangles) the jump had to be very large,
over 50% of the saccade size, to be reliably perceived. An ANOVA computed for
these cases showed a significant main effect of displacement size (F(7,35)=26.8,
p<0.001). Functions for target and distractor were not significantly
different (F(1,5)=0.05). Two of the subjects (AF and SP, fig. 3) did not
perceive the continuously present patterns to jump even at the largest (inward)
displacement of -3.5 degrees. For all subjects the minimum of the function was
reached when the pattern actually jumped slightly in the direction opposite the
saccade, by 0.5 to 1 degrees.
In contrast, all subjects correctly
perceived displacement of a stimulus, either distractor or target, if that
stimulus was blanked. Discrimination was nearly perfect even at the small
displacement magnitudes of +-0.5 degrees (short lines near the top center of
each graph in fig. 3). This replicates the result of Deubel et al. (1996) in
the presence of a second pattern.
Due to the relatively large stimulus
displacements applied in the trials where the continuous stimulus was
displaced, the subjects produced secondary, corrective saccades in 74% of these
cases. The question arises whether these corrective saccades are directed - as
instructed - to the target. Figure 4 shows the mean amplitudes of corrective
saccades as a function of stimulus displacement, for the cases where the
continuously present stimulus - either target or distractor - was displaced. Negative
amplitude values indicate displacements opposite to the primary saccade
detection. It can be seen that size and direction of the corrective saccades
correlates well with displacement size when the target was displaced (open
symbols). When the distractor was displaced, however, corrective saccade size
was unaffected (solid symbols). The data clearly show that the secondary
saccades are always correctly directed to the instructed target --
nevertheless, in these cases, the subjects systematically misattribute the
displacement to the blanked stimulus (Fig. 3). This indicates that the oculomotor system and the perceptual system
access different types of information. <Figures
3 and 4 about here>
DISCUSSION
The
large displacement magnitude at which intrasaccadic jumps can overcome the
blanking effect indicates the very poor sensitivity of subjects to
displacements of continuously present stimuli in the presence of a blanked
object. That is, only at this amplitude subjects correctly perceive the object
as jumping despite the stabilizing effect of its continuous presence. The
system seems biased to accept the position of a saccade target to be constant
if it is continuously present, even for stimulus displacements on the order of
half the size of the saccade.
The
finding that the secondary saccades are corrective in the sense that they are
all directed to the target demonstrates that visual stability and the
perception of intrasaccadic displacements are independent of oculomotor
behavior; obviously, the subjects cannot make use of information about
correction saccade amplitude for determining whether target or distractor were
displaced.
EXPERIMENT
3: SHIFT OF VISUAL BACKGROUND TEXTURE
Visual
target positions are normally evaluated relative to a visual context of
background objects, textures and surfaces. Under normal perceptual conditions,
the background might take the role of the reference. In this experiment we
examine the influence of a larger and more complex visual field on the
localization of a small blanked target. The question arises whether the visual
system can use this larger visual background to recalibrate postsaccadic target
position. Intrasaccadic displacements of the background should then be
misattributed to target displacements.
METHOD
In
addition to a target, we presented a background pattern of 14-16 elliptical
shapes (Figure 5a). The ellipses appeared in a circular area around the target
that extended about 6 degrees in diameter. This background pattern remained the
same throughout a trial, but could be displaced as a whole during the primary
saccade (Figure 5b). The background pattern was presented continuously, and in
randomly ordered trials it was displaced upon saccade detection either 0.75
degrees to the right, 0.75 degrees to the left, or not at all. The target was
always blanked for 200 ms, beginning at the same time as the background shift,
and was shifted in a range from -1.0 degrees to +1.0 degrees in 0.5 degrees
increments. Subjects indicated whether the target had moved either in the same
direction as the saccade (forward) or in the opposite direction (backward).
<Figure
5 about here>
RESULTS
Mean
latency (+-SD) of the primary saccades to the appearance of the stimuli was 175
+- 34.7 ms. Mean saccade amplitude to the targets at 6 degrees and 8 degrees
was 5.77 +- 0.66 degrees and 7.63 +- 0.91 degrees, respectively.
Figure
6 shows the discrimination results for five subjects. The graphs display
percent "forward" judgements (in the same direction as the primary
saccade) as a function of displacement size and background displacement. Though
subjects reported to have never perceived the intrasaccadic displacement of the
background, the displacement had a consistent effect on perceived target jumps,
shifting the psychometric functions in the direction of the background
displacements.
The
magnitude of the interaction of target with background can be estimated by
examining the deviation of the curves for leftward and for rightward
displacement of the background where they cross the 50% "neutral"
position (horizontal dashed line in each graph in Figure 5). For a further
statistical analysis we fitted each psychometric function separately with a
cumulative gaussian and calculated the 50% point, i.e. the actual target displacement
where the subjects perceived a perfectly stable target. In other word, these
are the target displacements that are necessary to compensate for the effect of
the background shift on displacement detection. The results are displayed in
Figure 7, showing this target displacement as a function of the background
jump. The average total effect in the judgments is 0.73 degrees, a figure that
estimates the effect of a 1.5 degrees difference in background positions. Thus
48.7% of the background shift is reflected in target position judgments. It is
important that the background displacement did not eliminate the perceptual
advantage of blanking; it merely biased the judgements of displacement, which
still took place with low thresholds and with the steep psychophysical
functions that have characterized the blanking effect in our other experiments.
The
effect is analogous to induced motion, with apparent target displacement being
biased in the direction opposite the background displacement. The result
implies that target location is evaluated with reference to the continuously
visible background when the target is blanked after the saccade.
<Figure 6 and 7 about here>
DISCUSSION
Displacement
of the background had a strong effect on target localization, even though the
background displacement itself was not perceived due to saccadic suppression.
Displacement discrimination was biased, in the sense that perceived forward
target displacements were sometimes seen in the presence of backward background
displacements. The subjects described this as an apparent motion of the target,
which is surprising given the fact that the presaccadic pattern was in the
retinal periphery and the postsaccadic pattern was centered near the fovea. The
effect can be interpreted as a form of transsaccadic induced motion; the
subjects perceive an apparent target motion with respect to a stable background
despite the fact that both the target and background undergo a displacement of
about 6 degrees on the retina due to the saccade. The relative positions of
target and background in space, rather than on the retina, determine what is
perceived across the saccade.
The
magnitude of the interaction of target with background can be estimated by
examining the deviation of the curves for leftward and for rightward
displacement of the background showing that 48.7% of the background shift is
reflected in target position judgments. This is close to the estimate of
Bridgeman and Graziano (1989) that half of an intrasaccadic background
deviation transfers to perceived visual target position, if the background has
texture but no meaningful structure. The consistency of these two estimates,
despite differences in method and background texture, suggests that background
texture parameters are relatively unimportant, as long as the background has a
large area.
Honda
(1993) also found that background is important in stabilizing perception
transsaccadically: the presence of a background results in smaller and briefer
mislocalization errors. And as noted above, perceptual space constancy is
generally more robust in a complex visual field (Matin et al., 1982). These
results in combination with ours speak for an explanation of the mechanism of
the blanking effect at a central level that integrates retinal location,
extraretinal signals and visual context information.
GENERAL
DISCUSSION
This
study addresses two theoretical issues: the transsaccadic integration of visual
information, and the general problem of oculomotor space constancy (the
appearance of a stable visual world despite movements of the eyes). These
problems are often treated separately, but our findings suggest that both are
necessary components of the more general problem of perceptual continuity. In
the present context this issue reduces to the questions of how spatial
orientation can be maintained across saccadic eye movements, and how a stable
and consistent visual world can be perceived across the discontinuities of the
retinal image due to saccades. Theories of what information is transferred
across saccades can be placed on two dimensions, beginning with a theory that
only semantic or symbolic information is transferred (O'Regan &
Levy-Schoen, 1983; O'Regan, 1992). On one dimension, the semantic theory can be
contrasted with object-oriented theories and data. These theories support the
availability of quantitative spatial information transfer across the saccade,
but only about a small number of visual objects (Irwin, 1992; Hayhoe, Lachter
& Feldman, 1991). These objects must be subjects of visual attention to be
transferred to the next fixation.
The
second dimension contrasts the semantic theory with theories that handle visual
space as a continuum. Saccadic displacements are corrected with a series of
vectors, allowing quantitative compensations for the coordinate shifts produced
during saccades (Von Holst & Mittelstaedt, 1950; Sperry, 1950). For this
dimension the correction is not of objects but of space itself, or of the
entire image as a unit, regardless of its content. These theories are the
traditional accounts of space constancy across saccades.
The
evidence presented here forces a move away from the interpretation of
perceptual space constancy as a quantitative compensation and toward a more
object-oriented conception. The existence of SSID, measured quantitatively
since the 1970s, already implied that a quantitative compensation by
extraretinal signals could not be responsible for space constancy, for stimulus
displacements of several degrees could go unnoticed during large saccades, but
our perceptual calibration is far better than this. The theoretical
implications of this discrepancy did not result in a new theory of space
constancy at the time, however.
The
present data can best be interpreted in terms of a reference object theory that
has emerged recently. Several versions of this theory have been described
(Irwin et al., 1994; Deubel et al., 1996; McConkie & Currie, 1996). In our
previous paper (Deubel et al., 1996, p. 995), we were led to such a theory by
our data on the blanking effect. In that paper we found that blanking a target
during and just after a saccade greatly reduced the threshold for detecting its
intrasaccadic displacement, even though such a blanking actually interferes
with displacement detection during fixation. We explained these phenomena by
suggesting a three-stage process of recalibrating visual space after a saccade.
First, a particular object is selected as a target for a future saccade. This
object receives preferential perceptual processing equivalent to an obligatory
shift of attention to the target stimulus (Deubel & Schneider, 1996).
Second, higher-level visual features (geometric properties, etc.) of this
reference object are stored in memory so that it can be identified after the
saccade. Third, the visual system seeks the target after fixation is
re-established, comparing the stored features with the new image. If a match is
found, the matching object is identified as the reference object, other parts
of the new visual scene are localized relative to it, and no further
computation or comparison takes place (Bridgeman et al., 1994). Extraretinal
signals do not enter into this process. The data presented here suggest that
this mechanism relies on the visual information available immediately after the
saccade -- the initial 50 ms are crucial for space constancy.
Several
lines of research provide evidence consistent with this theory, and
inconsistent with other theories of space constancy. The reference-object
theory requires only relatively little information to be stored from previous
fixations; confirming this prediction, only qualitative information about most
of the visual field is available (Irwin, 1992). For example, a visual scene can
be moved or changed in size (McConkie & Currie, 1996), or objects in a
scene can be moved or replaced by other objects (Pollatsek, Rayner &
Henderson, 1990), and the changes are not detected if they occur during
saccades. The extensive SSID literature confirms this property of intersaccadic
integration. Only changes in the saccade goal and possibly a few other attended
objects are transferred accurately across saccades (Irwin, 1992). The mechanism
concentrates on the region near the saccade target, with only secondary
influence from other locations (Irwin et al., 1994).
The
present experiments provide a more detailed characterization of the timing of
the transsaccadic integration process. The results suggest that the presence or
absence of an object at the moment when the eye lands is an essential
determining factor for that object to become a spatial reference. This implies
that the reference object need not be the saccade target: another nearby object
can take that role, if the saccade target is blanked so that it is unavailable
for establishing a new calibration. The distractor's displacement is not
visible if it is continuously present; rather, the motion is attributed to the
blanked saccade target. This demonstrates that temporal continuity of an object
is more important even than selection as a saccade target in establishing a
reference object. In like manner, our third experiment shows that a stimulus
array that is not blanked will be perceived as stable even if it is displaced,
as long as the saccadic target is blanked. The "background" array
takes on the role of the reference object, again because of its temporal
continuity.
These
results necessitate a modification of the reference object theory that we
described earlier (Deubel et al., 1996, McConkie & Currie, 1996). The
visual system need not be committed to a single identified reference object
before the saccade begins, for a non-target object can become the reference
object, and the system does not know in advance which object will be
appropriate as the reference object. According to our data, at least two
objects near the saccade goal region might also serve as reference objects, and
the assignment of stationarity depends upon which one is found after the
saccade. This is consistent with indications that information about three to
four objects can be localized across a saccade (Irwin, 1992). Whether an object
is defined in advance as target or distractor seems to play little role in the postsaccadic
determination of the reference object.
Nevertheless,
there is some independent evidence that the saccade goal target has a special
role in postsaccadic visual calibration. Bischof and Kramer (1968), for
instance, found perceived locations to be corrected more quickly near the
saccadic goal than at other retinal positions. In a saccadic suppression
experiment, Heywood and Churcher (1981) showed that subjects often misattribute
an intrasaccadic displacement of the saccade goal to a displacement of another
visual object such as the previous fixation, tending to preserve space
constancy preferentially for the saccade goal. Finally, Ross, Morrone, and Burr
(1997) demonstrated that stimuli flashed shortly before a saccade are
mislocalized such that they are perceived close to the saccade target. Whether
this "spatial attraction" by the saccade target is reminiscent of the
effect of our "reference object" mechanism that tries to anchor
presaccadically attended objects on the target found after the saccade must be
clarified by further research.
Another
refinement of the theory is made possible by our result with two blanked
objects. If both stimuli are blanked, performance is better than if neither is
blanked. This is quite unexpected at first glance, since spatial information
must be held in transsaccadic memory for a longer period, and delay should lead
to decay of performance. However, it is consistent with findings from our
previous work on the blanking effect. In that work, detectability of intrasaccadic
target displacements was even better than the detectability of similar
displacements during fixation. With target blanking, displacement detection
seems to be aided by information that is not available if the reference object
is found. In the experiments where two objects are blanked, no object receives
an advantage over the other, but localization of both targets is aided by the
availability of extraretinal information that is discarded if a reference
object is found. There is no reason for the visual system to still assume
visual stability for either object, because neither is found immediately after
the saccade.
Taken
together, our results can be combined with earlier evidence to suggest that
space constancy depends on comparison of common elements in the pre- and
post-saccadic images. This comparison takes place in a "constancy
window" that is about 50 ms in duration and has a size that can reach more
than 50% of the size of the saccade, depending on stimulus conditions. Neither
the spatial nor the temporal limits can be exceeded if constancy is to be
maintained.
Neurons
in lateral intraparietal cortex (LIP) described by Duhamel, Colby and Goldberg
(1992) may be performing some of the computations required by our theory.
Receptive fields in this area shifted to compensate for a saccade about 80 ms
before the start of the movement. Thus the LIP seems to store presaccadic,
visual information across the saccades and possesses quantitative spatial
information about the saccade. Similar properties have been recently reported
from neurons in the superior colliculus (Walker, Fitzgibbon & Goldberg,
1995).
Our
finding that corrective saccades are generally accurate, even when the
perceptions of displacements of the targets to which they are directed are grossly
in error, highlights a difference between cognitive and sensorimotor visual
functions. Several authors have differentiated a cognitive or perceptual
system, governing visual experience and pattern recognition, and a sensorimotor
system, controlling visually guided behavior (e.g., Bridgeman et al., 1979;
Paillard, 1987; Milner & Goodale, 1995). These authors show that motor
information can be accurate even under conditions where perception is in error.
Our present result shows a conclusion also reached by these authors:
information can flow from the cognitive to the sensorimotor systems, under some
conditions, making the sensorimotor function inaccurate; but information cannot
flow the other way, using accurate motor planning information to inform the cognitive
system about locations of objects in space.
Acknowledgements
The study was supported by the Deutsche
Forschungsgemeinschaft, Sonderforschungsbereich 462 ("Sensomotorik"),
and by the 1996 TransCoop Programme of the German-American Academic Council
Foundation.
REFERENCES
Bischof, N. & Kramer, B. (1968).
Untersuchungen und Überlegungen zur Richtungswahrnehmung bei willkürlichen
sakkadischen Augenbewegungen. Psychologische Forschung, 32, 195-218.
Bridgeman, B. (1995). Extraretinal
signals in visual orientation. In Prinz, W. & Bridgeman, B. (Eds.) Handbook
of Perception and Action Vol. I: Perception. London: Academic Press, 191-223.
Bridgeman, B. & Graziano, J.
(1989). Effect of context and efference copy on visual straight ahead. Vision
Research 29, 1729-1736.
Bridgeman, B., Hendry, D. & Stark,
L. (1975). Failure to detect displacement of the visual world during saccadic
eye movements. Vision Research 15, 719-722.
Bridgeman, B., Lewis, S., Heit, G.
& Nagle, M. (1979). Relationship between cognitive and motor-oriented
systems of visual position perception. Journal of Experimental Psychology:
Human Perception and Performance 6, 692-700.
Bridgeman, B., Van der Heijden, A. H.
C., & Velichkovsky, B. M. (1994). A theory of visual stability across
saccadic eye movements. Behavioral and Brain Sciences 17, 247-292.
Crane, H. D. & Steele, C. M.
(1985). Generation-V dual-Purkinje-Image eyetracker. Applied Optics 24,
527-537.
Deubel, H., Wolf, W., & Hauske, G.
(1984). The evaluation of the oculomotor error signal. In Gale, A.G. &
Johnson, F.W. (eds) Theoretical and applied aspects of eye movement research.
Amsterdam: Elsevier, pp. 55-62.
Deubel, H. & Schneider, W.X.
(1994). Can man bridge a gap? Behavioral and Brain Sciences 17, 259-260.
Deubel, H. & Schneider, W.X.
(1996). Saccade target selection and object recognition: Evidence for a common
attentional mechanism. Vision Research 36, 1827-1837.
Deubel, H., Schneider, W. X. &
Bridgeman, B. (1994). Stimulus blanking diminishes saccadic suppression of
image displacement. Investigative Ophthalmology and Visual Science 35 (Suppl.),
2033.
Deubel, H. & Bridgeman, B. (1995).
Fourth Purkinje image signals reveal lens deviations and retinal image
distortions during saccadic eye movements. Vision Research, 35, 529-538.
Deubel, H., Schneider, W. X. &
Bridgeman, B. (1996). Post-saccadic target blanking prevents saccadic
suppression of image displacement. Vision Research 36, 985-996.
Duhamel, J., Colby, C., & Goldberg,
M. E. (1992). The updating of the representation of visual space in parietal
cortex by intended eye movements. Science, 27, 227-240.
Gibson, J. J. (1966). The Senses
Considered as Perceptual Systems. Boston: Houghton Mifflin.
Grüsser, H.-J., Krizic, A. & Weiss,
L.-R. (1987). Afterimage movement during saccades in the dark. Vision Research,
27, 215-226.
Hayhoe, M., Lachter, J. & Feldman,
J. A. (1991). Integration of form across saccadic eye movements. Perception 20,
393-402.
Helmholtz, H. von (1866). Handbuch der
physiologischen Optik, Bd. 3. Leipzig: Voss.
Heywood S. & Churcher, J. (1981).
Saccades to step-ramp stimuli. Vision Research 21, 479-490.
Holst, E. von & Mittelstaedt, H.
(1950). Das Reafferenzprinzip: Wechselwirkungen zwischen Zentralnervensystem
und Peripherie. Naturwissenschaften, 37, 464-476.
Honda, H. (1993). Saccade-contingent
displacement of the apparent position of visual stimuli flashed on a dimly
illuminated structured background. Vision Research 33, 709-716.
Irwin, D. E. (1992). Memory for
position and identity across eye movements. Journal of Experimental Psychology:
Learning, Memory and Cognition 18, 307-317.
Irwin, D. E. & Andrews, R. V.
(1995). Integration and accumulation of information across saccadic eye
movements. In: Inui, T. & McClelland, J.L. (Eds.) Attention and Performance
XVI (p. 125-155), Cambridge:MIT Press.
Irwin, D. E., McConkie, G.,
Carlson-Radvansky, L. & Currie, C. (1994). A localist evaluation solution
for visual stability across saccades. Behavioral & Brain Sciences, 17,
265-266.
Mack, A. (1970). An investigation of
the relationship between eye and retinal image movement in the perception of
movement. Perception & Psychophysics 8, 291-298.
Matin, L., Picoult, E., Stevens, J. K.,
Edwards, Jr., M. W., Young, D. & MacArthur, R. (1982). Oculoparalytic
illusion: Visual-field dependent spatial mislocalizations by humans partially
paralyzed with curare. Science 216, 198-201.
McConkie, G. & Currie, C. (1996).
Visual stability across saccades while viewing complex pictures. Journal of
Experimental Psychology: Human Perception and Performance 22, 563-581.
Milner, A. D., & Goodale, M. A.
(1995). The visual Brain in action. Oxford: Oxford University Press.
O'Regan, K. (1992). Solving the
"real" mysteries of visual perception: The world as an outside
memory. Canadian Journal of Psychology, 46, 461-488.
O'Regan, J. K. & Levy-Schoen, A.
(1983). Integrating visual information from successive fixations: Does
transsaccadic fusion exist? Vision Research 23, 765-768.
Paillard, J. (1987). Cognitive versus
sensorimotor encoding of spatial information. In P. Ellen & C. Thinus-Blanc
(Eds.), Cognitive Processes and Spatial Orientation in Animal and Man.
Dordrecht, Netherlands: Martinus Nijhoff Publishers.
Pollatsek, A., Rayner, K. &
Henderson, J. M. (1990). Role of spatial location in integration of pictorial
information across saccades. Journal of Experimental Psychology: Human
Perception and Performance, 16, 199-210.
Prablanc C. & Martin O. (1992).
Automatic control during hand reaching at undetected two-dimensional target
displacements. Journal of Neurophysiology 67, 455-69.
Ross, J., Morrone, M.C., & Burr,
D.C. (1997). Compression of visual space before saccades. Nature 386, 598-601.
Sperry, R. (1950). Neural Basis of the
spontaneous optokinetic response produced by visual inversion. Journal of
Comparative and Physiological Psychology 43, 482-489.
Walker, M.F., Fitzgibbon, E.J., &
Goldberg, M.E. (1995). Neurons in the monkey superior colliculus predict the
visual result of impending saccadic eye movements. Journal of Neurophysiology,
73, 1988-2003.
Wolf, W. & Deubel, H. (1997). P31
phosphor persistence at photopic mean luminance level. Spatial Vision 10,
323-333.
FIGURE
LEGENDS
Figure 1. Progress of a trial with
blanking of target (left column) or both target and distractor (right column).
Blanking duration is 100 ms. SRT: Saccadic Reaction Time from target
displacement to the start of the saccade.
Figure 2. Experiment 1. Each column
depicts the performance of one subject when the target, the distractor, both,
or neither are blanked, and when either the target or the distractor had been
displaced (upper and lower graphs, respectively). The solid square at the left
of each graph shows accuracy of the subject in correctly identifying a jump of
the target (top graphs) or of the distractor (bottom graphs) when both were
present continuously. Other points on each graph show accuracy when the target
is blanked (open circles), when the distractor is blanked (filled circles), and
when both are blanked (open triangles) for the intervals indicated on the
horizontal axis. Each pair of graphs shows responses of one subject, ordered
from left to right in decreasing order of percent correct when the distractor
is moved.
Figure 3. Experiment 2. Percent correct
discrimination of one of two patterns as displaced when the other pattern is
blanked. Negative displacements indicate that stimulus displacement and saccade
occurred in opposite directions.
Figure 4. Experiment 2. Amplitudes of
corrective saccades as a function of size and direction of stimulus
displacement. Negative displacements indicate stimulus jumps opposite to the
primary saccade (backward displacements).
Figure 5. Experiment 3. Top: stimulus
configuration. Only the target was blanked, and the background was present
continuously. Bottom: position (vertical axis) versus time (horizontal axis)
during a trial. The background jump and the start of target blanking coincide
with the detection of the saccade.
Figure 6. Experiment 3. Proportion of
judgements that the target jumped in the same direction as the main saccade,
when the background was held steady, displaced in the direction of the saccade,
or displaced in the opposite direction. Each graph shows data from one subject.
Figure 7. Experiment 3. Perceived shift
of the blanked target induced by the background shift.