Change Detection Performance in Naturalistic Scenes: The Influence of Visual Working Memory for Identity and Spatial Locations

The present studies examined the roles of identity and spatial working memory in change detection. Observers completed a spatial or identity working m emory task concurrently with a change detection task. In the change detection task, participants we re presented naturalistic scenes that contained eit her a color or location change to one object. Concurren tly, participants remembered either the colors or locations of four squares. There was specific disru ption of performance when the working memory task and the change detection task loaded the same ubsystem of working memory. There was also evidence that spatial information is processed more readily than identity information. This suggests that although there are separate systems for identi ty and spatial working memory, these subsystems are not necessarily created equal in that processing in the spatial processing may have priority over identity processing. However, this priority can be overridden during change detection if spatial memory is already occupied.


INTRODUCTION
The amount of information that can be maintained in Visual Working Memory (VWM) is limited and research suggests that there are likely separate capacity limited mechanisms for spatial location versus identity information. The limits on VWM capacity are at least one of the reasons why people have difficulty detecting changes in visual stimuli, a phenomenon referred to as change blindness (Rensink et al., 1997;Simons and Levin, 1997). Successful change detection requires not only focused attention and encoding (Hollingworth and Henderson, 2002;Levin and Simons, 1997), but also retention in VWM and comparison of the pre and post presentations of the stimulus (Angelone et al., 2003;Mitroff et al., 2004;Simons and Rensink, 2005). What is of concern here is whether limits imposed by VWM in change detection are specific to separate capacity or resource limits within VWM for spatial location and identity information, or if the limits are more general. The current studies examined the degree to which change detection requires separate capacity limited mechanisms for object identity versus object spatial locations and the degree to which storage in one mechanism may receive priority over storage in another mechanism.
The discussion of multiple memory systems is not a new one. For decades researchers have speculated that our complex memory can be subdivided into many different systems (Schacter et al., 2000). Working memory is conceptualized as a limited capacity system used for not only storing auditory and visual information, but also manipulating information in reasoning, learning and comprehension (Baddeley, 1987;Baddeley and Logie, 1999). More specifically, within VWM there are different systems (identity and spatial location) that may or may not Science Publications CRP share resources. Some studies have demonstrated that the spatial location and identity of an object are automatically linked in VWM, suggesting a more general capacity limit across identity and location VWM (Jiang et al., 2000;Olson and Marshuetz, 2005). Olson and Marshuetz (2005) reported that memory for the identity of an object was dependent on memory for the relative location of that object, suggesting that the identity of an item is automatically linked to its relative location. However, other behavioral studies have demonstrated dissociations for identity and spatial location systems in VWM. Research has shown that engaging in a spatial task, only disrupts performance on other spatial tasks, not identity tasks (Hecker and Mapperson, 1997;Logie, 1995). For example, Tresch et al. (1993) asked participants to remember the location of a stimulus and found that there was interference if, in the retention interval, participants completed a motion discrimination task, but not when they completed a color discrimination task. Alternatively, when asked to remember the color of a stimulus, interference was caused by the color discrimination task but not the motion discrimination task. Finally, still other research suggests VWM is a flexible system in which to-be-remembered information is resistant to changes in size and viewpoint. When irrelevant spatial information was altered between sample and test displays, observers were still able to accurately detect color changes to stimuli (Woodman et al., 2012). Therefore, there is mixed evidence in regards to the degree to which identity and spatial location information share VWM resources. If identify and spatial location systems have separate resource capacities in VWM, then loading each may affect change detection performance for changes in identity versus spatial location differently.
In the visual search literature there is evidence that spatial location and identity VWM systems have independent resource capacities (Oh and Kim, 2004;Woodman and Luck, 2004;Woodman et al., 2001). Oh and Kim (2004) study participants completed a visual search task in conjunction with either a spatial location or an identity VWM task. Visual search performance was impaired when participants performed the concurrent spatial VWM task and not when they performed the concurrent identity VWM task. This suggests that similar processing mechanisms are utilized with spatial VWM and visual search for a target and supports the conclusion that identity working memory tasks require resources not needed in a visual search task. However, these studies did not demonstrate a task in which the spatial VWM task did not interfere, leaving open the possibility that the spatial VWM task interferes with general VWM resources rather than resources specific to spatial VWM.
Accurately detecting a change requires encoding of the pre-change and post-change stimuli and then a comparison of these two representations to determine if a change has occurred. While maintenance of spatial location and identity information may require separate storage and resource capacities, the encoding and comparison processes may require a more general capacity process. Change detection performance is better for some types of stimuli compared to others, suggesting that the some types of information may be preferentially encoded over others (Woodman and Vogel, 2008). For example, spatial location changes are detected more quickly and accurately than identity changes or color changes (Simons, 1996). This suggests that location information may be encoded into memory more quickly and more completely than identity information. Research has shown that configuration information can be extracted and compared to memory in the first few hundred milliseconds of scene presentation (Chun and Jiang, 1998). Encoding color information may be dependent on configuration information and therefore, must occur later in the encoding process (Jiang et al., 2000).
In the current studies we examined whether change detection is associated with a general load on visual working memory or if the load is specific to the type of change. The task required participants to detect a change between alternating views of an original and modified scene. The modified scene contained either a spatial location change (one object moved from one location to another) or an identity change (the color of one of the objects changed). We then examined whether a concurrent working memory task that required maintenance of either location or identity information during the change detection task would lead to a specific impairment (i.e., task interference only when the type of working memory task and the type of change were congruent) or if it would lead to a general impairment (i.e., task interference regardless of the type of change). If maintenance can rely on separate resource capacities, then the interference should be specific to the type of maintenance task being completed. However, if maintenance relies on a general resource, then interference should not be specific to the type of maintenance task.
In the current experiments, participants complete either a location working memory task (Experiment 1) or an identity working memory task (Experiment 2) simultaneously with an identity or location change detection task. If change detection relies on separate resources for identity and location information, then Science Publications CRP there will be specific interference of identity working memory with identity change detection and location working memory with location change detection. If change detection relies on a general working memory resource, both working memory tasks should interfere with location and identity change detection.

Experiment
Using a dual task paradigm we investigated change detection performance while participants completed a spatial working memory task that required them to remember a spatial array of 4 black squares. Images in the change detection task were naturalistic scenes that contained location or identity changes. If change detection uses a general working memory capacity, then performance on the working memory task and/or the identity and location change detection should be impaired when the two tasks are completed concurrently. However, if identity based and spatial based working memory systems do not share resources, then performance on the working memory task and/or the location change detection task should be impaired when the two tasks are completed concurrently.

Participants
A total of 38 undergraduates from Rowan University completed this experiment as partial fulfillment of Introductory Psychology course requirements. The average age of the participants was 19.8 years (SD = 1.91) and 16 identified themselves as female. All participants reported having normal or corrected-to-normal vision.

Stimuli
Stimuli were presented using Direct RT v2004.3.0.24 (Empirisoft Corporation) on 15 inch monitors set at a resolution of 800×600 (96 dpi, 75 hertz refresh rate) controlled by Dell Intel Pentium 4 computers at an unrestricted distance of approximately 50 cm. Each scene (array on grey background or picture) was 24×18 cm presented at the center of a 30×23 cm screen and subtended a 25° horizontal visual angle.
The memory stimuli consisted of an array of four black squares (each subtended 0.8° visual angle) that could occur in eight possible locations around a center fixation cross. The location for each square was randomly chosen without replacement from the eight possible locations. The change detection stimuli were life-like scenes created using the computer software program Sims2 (EA Games). There were four scenes for each of eight scene types, (adult bedroom, child bedroom, living room, deck, bathroom, kitchen, den and dining room). Each of these 32 scenes had two corresponding post-change scenes; one contained a color change (identity) and another contained a location change for a total of 64 change scenes. For example, in one of the living room scenes, for the color change the couch changes from red to tan and for the location change a plant moves from the back of the room closer to the middle of the room (the pre-change scene was the same for both the color and location changes). Participants viewed each pre-change scene twice, once in the change detection single task and once in the dual task. However, each pre-change scene was paired with each post-change scene only once. Each scene contained approximately 12-18 objects with limited object occlusion ( Fig. 1).

Procedure
Participants completed three randomly ordered blocks of trials, corresponding to the three experimental conditions: memory single task, change detection single task and dual task (memory and change detection). Each block contained 4 practice and 28 experimental trials. The practice trials were the same trials presented in the same order for each participant and the remaining 28 trials were presented in random order.
In the memory single task, participants began each trial by fixating a white cross on a grey background. Once fixated, they pressed the spacebar to begin. They saw the first array of black squares for 500 m sec followed by a black screen for 4000 m sec and then the location probe (a black outline of a square) for 2000 m sec or until a response was made. Participants reported whether there was a black square present in the probed location in the initial display (participants responded by pressing 1 (Yes) or 2 (No) on the keyboard using their left hand). On half of the trials, the probe appeared in a location where nothing was present in the memory array. In addition, participants received response feedback in the form of a red "X" at the bottom of the screen for incorrect answers to the memory probe. They were not allowed to proceed to the next trial until they provided the correct answer to the memory probe (memory single task and dual task condition only).
In the change detection single task, participants began each trial by fixating a cross and then pressing the spacebar to begin. After a 500 m sec black screen the change detection scenes were shown until the participant responded. Example stimuli for Experiments 1 and 2. In the memory single task condition participants only see and respond to the appropriate memory array (location for Experiment1 and color for Experiment 2) and in the change detection single task they only see and respond to the changing scene. In the dual task condition participants see and respond to the appropriate memory array and the changing scene. In this scene the chair on the right hand side of the picture changed color from brown to blue.
In the location change version of this scene the table with the flowers moves out in front of the couch. Images presented here in grayscale were in color during the experiments Scenes were shown using the flicker paradigm (Rensink et al., 1997); the first scene was presented for 400 m sec, followed by a blank screen (ISI, 130 m sec) and then the second modified scene was presented for 400 m sec. The flicker continued for approximately 60 s or until a response was made. Participants responded by hitting the space bar with their right hand when they saw the change and reaction time was recorded (a 2000 m sec black screen occurred before the start of the next trial). Of the 64 total scenes, 32 were randomly chosen to be shown in this block of trials (only 28 were experimental trials). Half of the experimental trials were identity changes, half were location changes and participants did not see the same pre-change scene twice in the same block. For example, if for a particular scene the location change was shown in this block of trials, the identity change was shown in the dual task block. Finally, participants did not know what kind of changes to expect.
In the dual task condition, participants perform both tasks simultaneously. After fixating the cross and pressing the space bar they saw the array of four black squares (500 m sec), then a blank screen (500 m sec), followed by the flickering scenes, another blank screen (500 m sec) and then the location probe (2000 m sec). Participants pressed the space bar with their right hand when they saw the change and the 1 or 2 key with their left hand to indicate yes or no for the location probe ( Fig.  1 for a sample trial). Feedback was then provided for the memory task as described above and the next trial began after a 2000 m sec inter trial interval.
In all conditions experimenters monitored participants as they performed an articulatory suppression task by repeating a string of 4 letters or numbers ("ABCD", "EFGH", "1234", or "5678") for the entirety of each trial. At the beginning of each trial, participants received a randomly selected new set of letters or numbers to repeat for that trial. They repeated the string of letters or numbers aloud at a rate of approximately four items per second. This procedure was added to reduce participants' ability to verbalize any visually presented material (Baddeley, 1987).

Location Memory Performance
A total of three participants were excluded from the analyses because their memory performance in the dual task condition was below 55% correct, suggesting that they were not focusing memory resources on the identity working memory task (analyses conducted on the remaining sample, N = 35). In addition, memory and change detection trials in the dual task condition were removed if performance on the change detection task was three or more standard deviations beyond the overall mean reaction time for change detection (average percentage of trials removed per participant = 6.7%). All statistical analyses for all experiments were conducted using and alpha level of 0.05.

Change Detection Performance
A total of four additional change detection trials were removed (2 were dual task trials and the corresponding memory trials were also removed) because reaction time was less than necessary for seeing one complete alteration of the flicker (<530 m sec).

Fig. 2.
Mean percent correct with standard error on the spatial working memory task for single and dual tasks conditions in Experiment 1. Memory performance for the dual task condition is divided by change detection for color and location

CRP
A 2 (change type: identity or location) X 2(task type: single task or dual task) repeated measures ANOVA revealed no significant main effects (F's<1). However, the interaction between change type and task type was significant (F (1, 34)

DISCUSSION
Memory for the spatial configuration of four squares was disrupted when performed concurrently with either a location or identity change detection task. When the prechange scene is presented, it is unknown whether a location change or an identity change will occur. Therefore, location information is likely encoded regardless of the type of change that occurs and encoding location information interferes with storage of location information for the memory task. With respect to change detection performance, in the dual task condition, location change detection performance declined relative to the single task condition. This suggests that location change detection requires spatial working memory resources used by the location working memory task. Therefore, these results indicate that the spatial working memory task uses location specific resources needed to complete location change detection but not identity change detection.
In addition, identity change detection improved when a location memory task was completed concurrently. In the single change detection task condition, location changes were detected faster than identity changes supporting previous research showing a preference for location information (Simons, 1996). However, this preference was not demonstrated in the dual task condition, identity changes were detected faster than location changes. This suggests that in the single task condition, location information is encoded and compared before identity information. However, in the dual task condition, a spatial working memory load lead to identity information being encoded and compared earlier. This also supports the idea of separable memory systems, showing that identity and location information is encoded and compared separately and not necessarily concurrently. In addition, the comparison process may be limited in that resources cannot be directed to more than one item at a time.

Experiment
Experiment 1 demonstrated that a specific working memory capacity for location information is used in detecting changes in naturalistic scenes. In addition, identity and spatial location information during these tasks were encoded and processed separately. In Experiment 2 we examined the effects of working memory for identity on change detection. In addition, we further examined whether identity and spatial working memory systems share resources. In this experiment participants performed a change detection task for location and color (identity) changes concurrently with an identity working memory task (to assess working memory for identity). If change detection uses separate identity based and spatial based working memory systems, then performance on the working memory task and/or the identity change detection task should be impaired when the two tasks are completed concurrently. However, if change detection relies on a general working memory capacity, then performance on the working memory task and/or the identity and location change detection tasks should be impaired when the two tasks are completed concurrently.

Participants
A total of 43 undergraduates from Rowan University completed this experiment as partial fulfillment of Introductory Psychology course requirements. The average age of the participants was 20.8 years (SD = 2.73) and 23 identified themselves as female. All participants reported having normal or corrected-to-normal vision.

Stimuli
The stimuli were similar to Experiment 1 except the memory task required participants to remember the colors of the four squares instead of their locations. The memory stimuli consisted of an array of four colored squares (each subtended 0.8° visual angle) each placed equal distance from a center fixation cross: one above, one to the right, one below and one to the left (the array subtended 2.6×2.6° visual angle; Fig. 1). The color for each square was randomly chosen without replacement from seven easily discriminable colors (red, blue, violet, green, yellow, black and white).

Procedure
In the memory single task condition, participants begin each trial by fixating a white cross (subtended 0.8×0.8 o visual angle) on a grey background. Once fixated, they pressed the spacebar to begin. The first array of colored squares was presented for 500 m sec followed by a blank, black screen for 4000 m sec and then the color probe for 2000 m sec (response terminated). Participants responded whether the color probe was present in the initial display (participants responded by pressing 1 (Yes) or 2 (No) on the keyboard using their left hand). On half of the trials, the color probe was a color that was present in the first array and on the other half it was a color not present in the first array.
In the change detection single task condition, the procedure was the same as in Experiment 1. In the dual task condition, participants performed both of these tasks simultaneously. After fixating the cross and pressing the space bar they saw the array of 4 colored squares (500 m sec), then a blank screen (500 m sec), followed by the flickering scene, another blank screen (500 m sec) and then the color probe (2000 m sec or a response). Participants pressed the space bar with their right hand when they saw the change and the 1 or 2 key with their left hand to indicate yes or no for the color probe (Fig. 1). As in Experiment 1, experimenters monitored participants as they performed an articulatory suppression task in all conditions by repeating aloud a string of four letters or numbers ("ABCD", "EFGH", "1234", or "5678") for the entirety of each trial. Also, participants received response feedback in the form of a red "X" at the bottom of the screen for incorrect answers to the memory probe. They were not allowed to proceed to the next trial until they provided the correct answer to the memory probe (memory single task and dual task condition only).

Color Memory Performance
A total of 16 participants were excluded from the analyses because their memory performance in the dual task condition was below 55% correct, suggesting that they were not focusing memory resources on the identity working memory task. Additionally, two participants were removed because in the dual task condition, change detection reaction time on several trials was less than necessary to see one alteration of the change (530 m sec; analyses conducted on the remaining sample, N = 25). In addition, trials in the dual task condition were removed if trial performance was three or more standard deviations beyond the mean of all participants' reaction time for change detection (average percentage of trials removed per participant = 5.8%) (An analysis including the 16 excluded participants is presented here and yielded similar memory and change detection results (N = 41). A one-way repeated measures ANOVA revealed a significant effect for memory (F (2, 80) = 57.48, p<0.0001). In the dual task condition, performance differed based on whether participants were detecting color changes (M = 46.8%, SD = 0.17) or location changes (M = 68.6%, SD = 0.18; t(40) = 5.61. p<0.0001). Memory performance concurrent with location and color change detection was lower than single task performance (M = 80.7%, SD = 0.11; t (40) = 4.22, p<0.0001; t (40) = 12.40, p<0.0001, respectively). Although memory performance during location change detection was worse than single task performance, memory performance during color change detection was still much worse. A 2×2 repeated measures ANOVA revealed a significant main effect for change type such that location changes were detected faster than color changes (F(1,40) = 19.72, p<0.0001). The main effect for task type and the interaction between change type and task type were not significant (F's<2). A one-way repeated measures ANOVA revealed a significant effect for memory performance in the different conditions (single task, dual task location change detection, dual task identity change detection; F (2, 48) = 47.39, p<0.0001, η 2 = 0.664). In the dual task condition, performance differed based on whether participants were detecting identity changes (M = 53.5%, SD = 0.13. SE = 0.027) or location changes (M = 77.9%, SD = 0.11, SE = 0.022; t (24) = 6.51. p<0.0001). Memory performance concurrent with location change detection was not different from single task performance (M = 83.0%, SD = 0.11, SE = 0.022; t (24) = 1.65, p = 0.11), but memory performance concurrent with identity change detection was lower than single task performance (t (24) = 10.45, p<0.0001; Fig. 4).

Change Detection Performance
A 2 (change type: Identity or location) X 2(task type: single task or dual task) repeated measures ANOVA revealed a significant main effect for change type such that location changes were detected faster than identity changes (F(1,24) = 10.91, p = 0.003, η 2 = 0.313). The main effect for task type and the interaction between change type and task type were not significant (F's<1; Fig. 5).
Change detection overall was similar in single and dual tasks. Furthermore, consistent with previous research, location changes were detected more quickly than identity changes (Simons, 1996). Although change detection performance was not affected by the identity working memory task, color working memory performance was affected by identity change detection but not location change detection. This provides evidence in support of identity and spatial change detection using separate working memory systems. Furthermore, given that performance on the working memory task was not impaired on identity change detection trials, it appears that the identity working memory task does not use the same general working memory capacity as change detection.

General Discussion
The current experiments investigated the effects of working memory load on naturalistic scene change detection. Using a dual task paradigm similar to prior studies, Experiment 1 demonstrated that performing a spatial working memory task interfered with location change detection. In addition, performance on the spatial working memory task was impaired by both location change detection and object change detection suggesting, regardless of the type of change that occurs, spatial location information is encoded. Dual task change detection performance suggests spatial working memory uses resources similar to those used in detecting location changes. However, color/identity information may be processed earlier in a serial process when spatial working memory is occupied. Experiment 2, examined whether identity working memory and change detection utilize the same general capacity system, or whether specific memory systems are involved. Identity working memory performance was impaired when participants concurrently completed identity change detection, but not location change detection. These data support previous studies arguing for separable identity and spatial working memory systems. Together these studies indicate that change detection requires specific working memory systems. Furthermore, they suggest that location information is generally processed faster and earlier than identity information unless spatial working memory is occupied by the task, allowing identity working memory to process identity information earlier.
The role of working memory in change detection may occur at any one or several stages of the change detection process. Working memory may be involved in encoding and retention stage and/or the comparison stage of change detection. The specific role of working memory in change detection likely occurs during the encoding and retention stages. Information is encoded into memory and then held in the working memory system consistent with the type of information encoded. Therefore, holding the color of four squares in identity VWM could occupy this system such that it is difficult to also encode and retain the color of objects in the change detection scenes. Something in the store will need to be eliminated, either the colors of the squares or the colors of objects in the scenes. It appears here that participants opted to eliminate information about the colors of the squares in order to complete the color change detection task as evidenced by the decrease in identity working memory performance and not identity change detection in the dual task condition (Experiment 2). Also, comparison and decision processes associated with location information may be completed initially in a serial process if the change signals for location are stronger (Yang, 2011). However, the comparison and decision process for detecting identity changes may happen earlier when location working memory is full (Experiment 1) because less location information from the change detection task is stored resulting in a weaker change signal. As such, identity change detection improved in a dual task situation where a location working memory task interfered with the encoding of location information during the change detection task.
Overall, these results suggest that there are separable object and spatial working memory systems within visual working memory. Unlike visual search, change detection in naturalistic scenes utilizes both working memory systems. While visual search performance declined only when spatial working memory was full (Oh and Kim, 2004;Woodman and Luck, 2004), both identity and spatial working memory differentially affected change detection. However, location information appears to be processed preferentially to identity information in visual change detection. Therefore, the results presented here, together with the results found in visual search literature, suggest that there are separate spatial and identity working memory systems involved in common visual tasks and that processing of spatial information is dominant over processing of identity information. Specifically, location information may be processed automatically, while identity information is processed only if the task requires processing of this information.
This study along with prior research suggests that location information, compared to other object information, is processed in a different manner by the visual system. Research suggests that initially upon viewing a scene and attending an object, a representation (or object file) is created (Kahneman et al., 1992). As new information is received by continued viewing of a scene or introduction to a new scene, a spatial index is created to maintain the visual information. Scene information is indexed such that spatial information is processed and indexed first and this is followed by feature information. Not only is this true for adults, but research on infants suggests that even as early as 3 months of age, infants create representations of spatial information. By 6 months of age, they can form more complex spatial representations and use this information to maintain spatial relations of several objects (Casasola et al., 2003;Quinn, 1994). The current experiments are consistent with this view in that spatial information is processed more Science Publications CRP readily than identity information. However, when spatial capacity is occupied it is possible that other information, like the property of color, may receive priority, depending on task requirements.
Spatial and identity working memory may also differ in sensitivity to distraction. Research suggests that identity working memory is more sensitive to a verbal distraction task compared to spatial working memory (Postle et al., 2005). In the current experiment participants completed a verbal suppression task while performing object (Experiment 1) and spatial (Experiment 2) working memory tasks. Identity working memory performance steeply declined when participants concurrently completed the object change detection task, but this was not the case for spatial working memory performance in Experiment 1. In addition, single task performance of the identity working memory task was slightly lower than single task performance for spatial working memory. Although it was not the question of interest, increased sensitivity of identity working memory to verbal distraction may have been demonstrated in the current experiments. In addition, this may help to explain the difference in the number of participants excluded in each experiment (3 in Experiment 1 and 16 in Experiment 2) due to poor memory task performance in the dual task condition. However, follow-up analyses revealed a similar pattern of change detection results upon the inclusion of the 16 participants in Experiment 2, which suggests that their ability to detect changes remained stable. While it is possible that the memory performance during detection of identity changes in our experiments reflect a differential sensitivity to verbal load, it is also possible that object memory tasks are more difficult regardless of verbal load because object information is not processed as readily as location information.
Overall, visual working memory is important for change detection in naturalistic scenes. In particular, change detection utilizes specific working memory systems depending on the type of change. This provides evidence for the separate resources for identity and spatial working memory systems. In addition, spatial information about a scene is processed in a different manner by the visual system compared to other object information and may even receive priority processing.