Cerebral Cortex Advance Access originally published online on March 26, 2007
Cerebral Cortex 2007 17(12):2914-2932; doi:10.1093/cercor/bhm017
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Representing Spatial Relationships in Posterior Parietal Cortex: Single Neurons Code Object-Referenced Position
1 Department of Neuroscience, University of Minnesota Medical School, Minneapolis, MN 55455, USA, 2 Brain Sciences Center, Veterans Affairs Medical Center, Minneapolis, MN 55455, USA, 3 Center for Cognitive Sciences, University of Minnesota, Minneapolis, MN 55455, USA
Address correspondence to Matthew V. Chafee, Brain Sciences Center (11B), Veterans Affairs Medical Center, 1 Veterans Drive, Minneapolis, MN 55417, USA. Email: chafe001{at}umn.edu.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
The brain computes spatial relationships as necessary to achieve behavioral goals. Loss of this spatial cognitive ability after damage to posterior parietal cortex may contribute to constructional apraxia, a syndrome in which a patient's ability to reproduce spatial relationships between the parts of an object is disrupted. To explore neural correlates of object-relative spatial representation, we recorded neural activity in parietal area 7a of monkeys performing an object construction task. We found that neurons were activated as a function of the spatial relationship between a task-critical coordinate and a reference object. Individual neurons exhibited an object-relative spatial preference, such that different neural populations were activated when the spatial coordinate was located to the left or right of the reference object. In each case, the representation was robust to translation of the reference object, and neurons maintained their object-relative preference when the position of the object varied relative to the angle of gaze and viewer-centered frames of reference. This provides evidence that the activity of a subpopulation of parietal neurons active in the construction task represented relative position as referenced to an object and not absolute position with respect to the viewer.
Key Words: attention • constructional apraxia • hemispatial neglect • monkey • object-centered
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
Humans assemble and utilize complex objects such as tools or buildings, a fact often considered to provide a physical demonstration of human intelligence. Most objects that humans build and use are assemblies of discrete components. Constructed objects are therefore the products of a set of spatial cognitive operations on the part of an architect, engineer, or builder that has computed and specified how the parts of an object ought to be fit together. Some of the most basic spatial computations involved can be isolated in a simple behavioral context defined by the requirement to assemble a copy of a model object. To do this successfully, the spatial relationships between the components of the model must be analyzed and reproduced. The process is captured by the actions of a child assembling building blocks to conform to a configuration provided by printed instructions. Spatial information embedded in the instructions is periodically sampled and analyzed to direct each action during assembly. This cycle of spatial analysis followed by physical assembly is nicely reflected by the pattern of eye and arm movements as humans assemble copies of model objects. Subjects direct gaze first to the model object and then to the corresponding location within the copy object immediately before placing new parts at the proper relative position in the copy (Ballard et al. 1992
; Hayhoe et al. 1998). This suggests that the brain strategically samples spatial information embedded in the model object before directing each movement during construction of the copy and further that neural correlates of the spatial analysis involved and the spatial motor control required may be dissociable in time during this process.
Spatial cognitive processes that enable object construction appear to rely particularly on the functional integrity of posterior parietal cortex. Damage to this cortical area often produces constructional apraxia, a syndrome in which patients are unable to accurately reproduce the spatial structure of model objects (Kleist 1934; Black and Strub 1976; Villa et al. 1986; Ruessmann et al. 1988). Given the above, a simple prediction is that neurons in posterior parietal cortex represent spatial relationships among object components. Loss of these neurons after parietal damage could then provide part of the explanation why parietal patients construct objects that are spatially disorganized. To examine the possibility that parietal neurons support relational representations of space that localize object components with respect to each other, we developed an object construction task that monkeys could perform and recorded neural activity in posterior parietal area 7a.
We previously reported that, during this task, the firing rate of area 7a neurons varied systematically with a task-critical spatial datum, namely, where component parts were missing from incomplete copies relative to model objects stored in working memory (Chafee et al. 2005). This neural signal suggested that parietal neurons were involved in a comparative operation by which differences between copies and models were assessed in order to direct subsequent construction. Support for this was provided by the observation that the neural signal predicted where monkeys were going to place the next component in the copy object on correct and error trials. Importantly, this neural signal did not correlate with sensorimotor aspects of the task because neural activity did not correspond to the position of a visual stimulus or the direction of the required motor response. We interpreted this neural activity as a physiological correlate of a spatial cognitive operation involved in directing object construction.
In the present experiment, we test the prediction that during object construction, single parietal neurons support an object-referenced representation of space. The object construction task required determining whether object components were located on the left or right side of the object to which they belonged. We sought to determine whether parietal neurons coded this spatial relationship consistently when the position of the reference object changed relative to the angle of gaze and the position of the viewer. The objective was to determine if neural activity was selective for an intraobject spatial relationship and if this selectivity could be dissociated from retinocentric or viewer-centered representations of space that have been previously described in parietal cortex. We present supporting evidence below that parietal neurons encode not only spatial locations or directions but also spatial relationships, when these are important to a given behavioral objective.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
Subjects
Neural activity was recorded bilaterally from area 7a in the posterior parietal cortex of 2 male rhesus macaques (4 and 6 kg). Two recording chambers (7 mm internal diameter) were implanted over area 7a bilaterally in each animal in an aseptic surgery under isoflurane (1–2%) gas anesthesia. Four titanium posts were fixed to the skull with screws, and a halo was attached to provide a mechanical anchor to stabilize head position. Postsurgical analgesia was administered for several days (Buprenex, 0.05 mg/kg bid, intramuscularly). Further information regarding recording technique and reconstruction of electrode penetrations in area 7a can be found in our prior report (Chafee et al. 2005). Care and treatment of the animals conformed to the Principles of Laboratory Animal Care of the National Institutes of Health (NIH) (NIH publication no. 86–23, revised in 1995). The Internal Animal Care and Use Committees of the University of Minnesota and the Minneapolis Veterans Affairs Medical Center approved all experimental protocols.
Visual Stimuli
The object stimuli used in the construction task consisted of varying arrangements of identical square elements and were adapted from stimuli developed by Driver and Halligan (1991). Squares were blue and subtended 1.4° of visual angle; adjacent squares were separated by a gap of 0.25°. Object squares were laid out in a 5 x 5 square grid, 8° on a side. All objects included a frame consisting of squares in the base row and central column of the grid forming an inverted "T" configuration. The frame provided a base and central axis to all objects. Distinct model configurations consisted of this frame plus 1 or 2 additional squares. Monkeys were trained to perform the task using a set of 36 distinct model configurations differing in the position of the squares present in addition to those making up the object frame. (Additional squares were located either on the left or on the right side of the object, at one of 4 different vertical levels within the 5 x 5 grid. For a depiction of the full set of object configurations employed in training, see Fig. 1 of Chafee et al. 2005.) Neural recording was conducted with a subset of these configurations. This subset consisted of objects in which additional object squares were present in either the topmost or the middle row of the object grid, on either the left or the right side of the object midline. The construction task was administered in 2 experimental series (Fig. 1), differing in whether the position of the model object (Series A) or the copy object (Series B) varied relative to the fixation target across trials (as described further below). In Series A (Fig. 1A–G), model objects consisted of the object frame plus 1 or 2 additional squares present on either the left or the right side of the object, in either the top or the middle row in the grid. In Series B (Fig. 1H–M), model objects consisted of the object frame plus one additional square, present on either the left or the right side of the object, in either the top or the middle row of the grid. We restricted the total number of different objects used during recording in order to restrict the number of trials required to complete each trial set. This was advantageous as the time that cortical neurons can be stably isolated is limited. The activity of each set of isolated neurons was recorded as monkeys performed either 128 trials (Series A) or 160 trials (Series B) of the object construction task.
|
Behavioral Tasks
Both monkeys performed Series A; the second monkey performed Series B of the construction task, as well as a probe task, in blocks. Each trial began with the appearance of a red dot serving as a gaze fixation target presented at the center of the visual display. Monkeys were required to maintain gaze within 1.3° to 1.7° of this central target throughout the trial up to the delivery of reward. Moving gaze fixation beyond this window terminated the trial. After a preliminary period of fixation (500 ms), a model object was presented for 750 ms (Fig. 1A,H; Model). In Series A, the model object was presented offset 4.2° horizontally either to the left or to the right of the fixation target (just more than half the object width) so that the object fell entirely within the left or right visual hemifields (Fig. 1A). The direction of the shift, left or right, was randomized across trials. In Series B, the model object was always presented centered on the fixation target at the center of the display (Fig. 1H). After the model disappeared, a 750-ms delay period followed in which only the gaze fixation target was visible (Fig. 1B,I; Delay). After the delay period, a copy object was presented (Fig. 1C,J; Copy). In Series A, the copy object was always centered on the fixation target (Fig. 1C). In Series B, the copy object was offset 4.2° horizontally to the left or right of the fixation target, at random across trials (Fig. 1J). In both series, the copy object was identical to the model object preceding it on each trial except that one square had been removed (squares within the central column or base row comprising the object frame were never removed). In considering the model object, we refer to the square removed to produce the copy as the "critical square." In considering the copy object, we refer to the location where a square was absent relative to the preceding model as the "missing critical square." After the copy object had been visible for 750 ms, a pair of choice squares appeared flanking the copy object (Fig. 1D,K; Choice array). Choice squares were arranged either in a horizontal array, with one square on either side of the copy object (Fig. 1D), or in a vertical array, with both squares on the same side of the copy object at different vertical positions (Fig. 1K). The configuration of the choice array, whether horizontal or vertical, varied randomly across trials (in both Series A and B). The monkey replaced the missing critical square by selecting one of the 2 choice squares for addition to the copy object. The selection was controlled by varying the timing of the motor response and not its direction. After the choice squares had been visible for 300–600 ms, first one choice square (Fig. 1E,L; first choice) and then the other choice square (Fig. 1F; second choice) increased in brightness for a period of 700–1000 ms in random sequence (represented by the open choice square in Fig. 1E,F,L). Only one of the 2 choice squares was brightly illuminated at any one time. The monkeys depressed a single response key with their left foot to indicate their selection. The choice square that was bright at the time that the response key was pressed translated smoothly inward in a horizontal direction to occupy the grid position in the same row and immediately adjacent to the copy object. (In the case that the monkey depressed the response key during the first choice period, the second choice was not shown.) Addition of the selected choice square to the copy object produced a new configuration. If the correct choice square was selected, it replaced the missing critical square and reproduced the configuration of the preceding model (Fig. 1G; Completion). If the incorrect choice square was selected, the configuration of the constructed object differed from the preceding model (Fig. 1M; Completion; note the constructed object configuration in Fig. 1M is different from the model object in Fig. 1H). After the selected choice was added to the copy object, the newly constructed configuration remained visible for a period of 300 ms (Fig. 1G,M). If the choice was correct, the monkey was then rewarded with a drop (0.2 mL) of juice. The sequential choice response mechanism was used to report the monkeys' decision in order to dissociate locations in objects from targets for movement. The spatial direction of the movement vector the monkey had to execute to depress the response key was invariant across trials. Neural signals associated with spatial variables in the task therefore were not likely to reflect spatial aspects of this required motor response.
In a probe task (Fig. 11), we flashed a visual probe stimulus on a minority (25%) of Series B construction trials. The monkey could not anticipate which trials were probe trials, and the reaction time to detect the probe stimulus measured behavioral correlates of spatial representation during object construction. The probe stimulus consisted of a red square the size of the other squares in the copy object. It was flashed for 50 ms during the copy period, 150–750 ms after the appearance of the copy object. The visual probe was located either at the same position in the copy object as the missing critical square or at the mirror location on the opposite side of the copy object. On probe trials, the monkey was required to depress the response key between 50 and 600 ms after probe onset for reward. On the remaining 75% of trials randomly interleaved with the attention probe trials, the monkey performed the construction task as described above.
Data Collection
Neural activity was recorded using a 16-channel multielectrode recording matrix (Thomas Recording, GMbH, Giessen, Germany). The electrode signals were amplified (at a gain of 20 000), filtered (bandpass between 0.5 and 5 kHz), and action potentials were discriminated in each channel using a time-amplitude window discriminator (DDIS-1, Bak Electronics, Mount Airy, MD) or waveform discriminator (Multi Spike Detector, Alpha Omega Engineering, Nazareth, Israel). Two people performed and monitored the unit isolation during recording. Typically, the action potentials of between 20 and 30 neurons were isolated simultaneously. If isolation of a neuron's activity was not maintained throughout the set of administered trials, the data from that neuron were discarded. Spike timing was sampled with 40 µs resolution (DAP 5200a Data Acquisition Processor, Microstar Laboratories, Bellevue, WA). Computer files containing the timing of action potentials relative to stimulus and behavioral events were saved for all neurons isolated. Neurons in which the mean discharge rate averaged across the model, delay, and copy periods of the task exceeded 0.5 Hz were included in subsequent analyses. In both animals, 
15% of isolated neurons failed to meet this spontaneous activity criterion; the remaining 85% of all cells isolated comprise the present database. Electrode penetrations were confined to area 7a in the inferior parietal gyrus; recording locations were determined by magnetic resonance imaging visualization of electrodes and confirmed by postmortem localization (Chafee et al. 2005).
Data Analysis
The relation between neural firing rate and the spatial position of object squares was assessed by a 2-way analysis of covariance (ANCOVA). The ANCOVA was implemented as a least-squares regression estimating the parameters of a general linear model (GLM) in which the categorical factors were represented by dummy variables. The 2 factors in the ANCOVA defined the relative position of the critical square in 2 alternative spatial frameworks. The first factor was object-referenced relative position. This factor had 2 levels that specified whether the critical square was located on the left or right side of the reference object relative to its midline. The second factor was viewer-referenced position. This factor had 2 levels specifying whether the critical square was located to the left or right of the gaze fixation target. Object-referenced and viewer-referenced positions of the critical square, coded as categorical spatial variables (left or right in each framework), were statistically independent in the design. In the analysis of the Series A data, the dependent variable was firing rate during the model period, and the 2 spatial factors represented the object and viewer-referenced position of the critical square in the model object. In the analysis of the Series B data, the dependent variable was firing rate during the copy period, and the 2 spatial factors represented the object and viewer-referenced position of the critical square missing from the copy object.
In Series A, we focused our analysis on neural activity during the model period because we varied the position of the model object in Series A. In Series B, we focused our analysis on neural activity during the copy period because we varied the position of the copy object in Series B. By examining activity during the period in which we varied the position of the corresponding reference object, we could determine whether neural activity varying as a function of the position of squares in (or missing from) the reference object coded position relative to the object or the viewer.
Two continuous covariates were included in the model, the baseline firing rate (during the 500 ms interval of central gaze fixation preceding model onset) and experimental time (elapsed time since the start of neural recording). These covariates corrected for trial-to-trial fluctuations in baseline activity and linear trends in firing rate across the recording session. We estimated the regression coefficients for the parameters of the linear model using the REGRESS function in the Matlab Statistical Toolbox (The Mathworks Inc., Natick, MA), and the significance of object and viewer-referenced factors was assessed at P < 0.05.
Critical squares located on the left side of the reference object occupied one pair of retinocentric positions (Fig. 2A; open squares labeled L). Critical squares located on the right side of the reference object occupied another, slightly offset pair of retinocentric positions (Fig. 2A; filled squares labeled R). Figure 2A illustrates the retinocentric receptive fields (dotted outlines varying in size from approximately 4° to 16°) of 3 hypothetical neurons overlapping 1 (neuron B), 2 (neuron A), or 3 (neuron C) critical square locations. The firing rates of neurons A–C (Fig. 2A) would be expected to vary as a function of square position in approximately the manner illustrated by the bar graphs of corresponding color below (the 4 bars indicate relative levels of activity evoked by critical squares at the 4 retinocentric positions above). In each case, mean firing rate is generally greater when critical squares are located in the left (neurons A, B) or right (neuron C) visual hemifields. Such a pattern of spatial preference would be expected to yield significance for the viewer-referenced factor in the ANCOVA.
|
A different type of spatial selectivity is illustrated in Figure 2B. In this case, elevated activity is associated with critical squares located within one of 2 spatially separated retinocentric regions corresponding to the preferred side of the reference object when the object appears at 2 different locations in the display. Hypothetical neurons preferring object-left (Fig. 2B; neuron A) and object-right (Fig. 2B; neuron B) are shown. Neurons with this form of spatial selectivity during the construction task would be expected to be significantly affected by the object-referenced factor in the ANCOVA.
Neurons significantly affected by both viewer- and object-referenced position or their interaction could exhibit several types of spatial selectivity. One such type is illustrated by neuron B in Figure 2A. In neurons such as this, selective for a single retinocentric location, firing rate could be significantly higher when trials were segregated according to both object-referenced and viewer-referenced position. (Neuron B exhibits elevated activity both when critical squares are located on the right side of the reference object and in the left visual hemifield). Thus, it was particularly important to confirm that single parietal neurons existed in which activity related significantly to object-referenced position only, and not to viewer-referenced position, or to the interaction between these factors.
To examine population activity, we plotted normalized population spatial tuning functions based on the position of the critical square expressed in viewer-referenced coordinates (Figs 5 and 6). We plotted separate tuning functions for neurons with left and right spatial preference defined either with respect to the reference object or with respect to the viewer. To construct the population spatial tuning functions, we first computed the mean firing rate of each neuron during either the model or copy period when the critical square was located in each of either 8 (Series A) or 4 (Series B) different horizontal locations on either side of the fixation target in the display. (Tuning functions were defined with respect to the horizontal position of critical squares.) We then normalized each neuron's spatial tuning function to its peak (by dividing each rate by the maximum observed) and then averaged these single-neuron, normalized spatial tuning functions across all of the neurons in the population. We further constructed population spike density functions (SDFs) to visualize the time course of activity in neurons exhibiting the strongest object-referenced and viewer-referenced signals (neurons were included in each population if their activity related significantly to the corresponding spatial factor in the ANCOVA at P < 0.001). In each population, we constructed separate SDFs using 4 groups of trials. In the object-referenced neuronal population (e.g., Fig. 8A,B), trials were divided according to the combination of 2 binary factors describing the position of the critical square in object-referenced (preferred or nonpreferred) and viewer-referenced (ipsilateral or contralateral) spatial reference frames. In the viewer-referenced neural population (e.g., Fig. 8C,D), trials were similarly divided according to the combination of 2 binary factors describing the position of the critical square in viewer-referenced (preferred or nonpreferred) and object-referenced (left or right) spatial reference frames. Each trial, represented by a vector of spike times, was converted to a continuous SDF by convolution with a Gaussian kernel (
= 20 ms) using the KSDENSITY function of the Matlab Statistical Toolbox (The Mathworks Inc.). Then for each neuron, the single-trial SDFs in each of the 4 groups of trials above were averaged to produce 4 mean SDFs for that neuron. These single-neuron SDFs were then averaged across all neurons in the population to produce the population activity time courses illustrated.
In the visual probe experiment (Fig. 11), reaction time data were modeled as a linear function of 2 independent variables using multiple linear regression (univariate GLM in the SPSS statistical package; SPSS, Inc., Chicago, IL). The independent variables were the position of the visual probe stimulus relative to the missing critical square, treated as a categorical variable (either the same or mirror opposite location in the copy object), and the retinal eccentricity of the visual probe stimulus, expressed in degrees visual angle.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
Behavior
In Series A, Monkey 1 performed 83% of object construction trials correctly. In the choice sequence, it responded correctly during the first choice period on 79% of trials and during the second choice period on 87% of trials. The average reaction time of Monkey 1 in the first and second choice periods was 386 and 335 ms, respectively (relative to the change in the illumination of each choice that signaled its availability to the monkey). Monkey 2 performed 90% of Series A object construction trials correctly. In the choice sequence, it responded correctly during the first choice period on 88% of trials and during the second choice period on 91% of trials. The average reaction time of Monkey 2 in the first and second choice periods was 324 and 303 ms, respectively. In Series B, Monkey 2 performed 93% of trials correctly. In the choice sequence, it responded to the first and second choices correctly on 92% and 93% of trials. The mean reaction time to the presentation of the first and second choice was 324 and 308 ms.
Neural Database
We recorded the activity of 1517 neurons in 2 monkeys performing the object construction task. (This neural database is partially overlapping with the database described in Chafee et al. 2005. Some neurons tested on various control conditions in that report are excluded from the present study, whereas the Series B data described here were not included in the prior report.) Neural recordings were confined to area 7a in the inferior parietal gyrus. (The regions of area 7a sampled during neural recording in the 2 monkeys are illustrated in Chafee et al. 2005.) In Series A, we recorded the activity of 748 and 265 neurons in Monkeys 1 and 2. In Series B, we recorded the activity of 504 neurons in Monkey 2. The neuronal sample was divided between the left and right cerebral hemispheres of both monkeys. (Series A: 374 neurons in both the left and right hemispheres of Monkey 1; 178 and 87 neurons in the left and right hemispheres of Monkey 2. Series B: 238 and 266 neurons in left and right hemispheres of Monkey 2).
Two Predictions Regarding the Neural Representation of Object-Referenced Spatial Position
Neural activity coding the position of the critical square relative to the reference object during the construction task should satisfy at least 2 conditions. First, neural activity ought to exhibit translation invariance. The activity of neurons preferring a given spatial relationship should be equivalently elevated by any configuration of critical square and reference object that satisfies the preferred relationship, regardless of where the square and the object appear in space relative to the viewer. Second, neural activity coding the spatial relationship between the critical square and the reference object should vary when the critical square remains fixed in space relative to the viewer, but the reference object moves so as to alter its spatial relationship to the critical square. To test the first prediction, we kept the spatial relationship between the critical square and the reference object fixed, but varied the position of the pair relative to the gaze fixation target and therefore the viewer. To test the second prediction, we kept the position of the critical square fixed relative to the viewer, while changing the position of the reference object. Confirming these 2 predictions tests the hypothesis that neural activity during object construction codes the relative position of the critical square with respect to the reference object and not the absolute position of the critical square with respect to the viewer.
Effects of Varying Viewer-Referenced Position while Holding Object-Referenced Position Constant
Figure 3 illustrates the activity of 2 neurons in area 7a demonstrating an object-referenced spatial preference. One neuron exhibited a spatial preference for the right side of the reference object (Fig. 3A–D). The other neuron exhibited a spatial preference for the left side of the reference object (Fig. 3E–H). The data are from Monkey 2 in Series B. The signal coding object-referenced position was clearest in single neurons during the copy period (see below for quantification), although it was present in both task periods.
|
In the first neuron, activity during the copy period (Fig. 3A–D) was elevated when the missing critical square was located on the right side of the copy object (Fig. 3B,D). Less activity was evident when the missing critical square was located on the left side of the copy object (Fig. 3A,C; the position of the missing critical square is specified by a comparison between model and copy objects illustrated above each panel). The activity of this neuron exhibited translation invariance in so far as firing rate was comparably elevated when the missing critical square was located on the preferred side of the copy object, irrespective of whether the square and object were located to the left (Fig. 3B) or right (Fig. 3D) of the gaze fixation target. Neural activity was not a simple function of the pattern of visual input in that firing rate varied to reflect the changing position of the missing critical square when visual input during the copy period did not vary (as in Fig. 3A,B). In the ANCOVA we employed (Materials and Methods), the discharge rate of this neuron during the copy period related exclusively to the object-referenced position of the missing critical square (Fobject = 56.40, P < 0.001). Activity did not relate to the viewer-referenced position of the missing critical square (Fviewer = 2.67, P = 0.104) or to the interaction between the 2 spatial factors (Finter = 2.13, P = 0.146). In the second neuron (Fig. 3E–H), activity was elevated to a comparable level when the missing square was on the left side of the copy object, regardless of whether the copy object was presented to the left (Fig. 3E) or right (Fig. 3G) of the gaze fixation target. Similarly, the discharge rate of the second neuron during the copy period related exclusively to the object-referenced position of the missing critical square (Fobject = 410.58, P < 0.001). Activity in this case again did not relate either to the viewer-referenced position of the missing critical square (Fviewer = 0.15, P = 0.703) or to the interaction between the 2 factors (Finter = 0.09, P = 0.767). In both neurons, activity varied as a function of a virtual spatial coordinate (the missing critical square) derived from a sequence of stimuli by application of a cognitive rule and appeared to represent that coordinate in an object-referenced framework.
We applied the above 2-way ANCOVA to firing rates during the model period (Series A) and the copy period (Series B) of the trial to examine the influence of critical square position on neural activity. The 2 factors in the ANCOVA were the object-referenced position of the critical square (left or right with respect to the object midline) and the viewer-referenced position of the critical square (left or right with respect to the fixation target). The Venn diagram in Figure 4 illustrates the percentage of neurons (of those exhibiting any spatial effect in the analysis) in which activity related significantly (P < 0.05) either to the object-referenced factor, the viewer-referenced factor, or their interaction (numbers of neurons in parentheses). In one group of neurons, activity varied as a function of the object-referenced factor exclusively. This group accounted for 28% and 49% of neurons with spatially selective activity during the model and copy periods, respectively. The activity of other neurons varied as a function of the viewer-referenced position of the critical square. Neurons coding square position in object-referenced and viewer-referenced coordinates coexisted in posterior parietal cortex.
|
To examine the spatial tuning of neurons, we constructed normalized population spatial tuning functions (Fig. 5). We defined the position of the critical square as its horizontal distance from the fixation target in degrees of visual angle. Neurons were separated into groups on the basis of whether their activity related to the object-referenced or viewer-referenced factor in the ANCOVA (P < 0.05) and by the spatial preference of the neuron (left or right) in each reference frame. Thus, for example, the object-referenced population included all neurons for which activity related significantly (P < 0.05) to the object-referenced factor (165 neurons in the model period and 223 neurons in the copy period, see Fig. 4). The spatial tuning of these neurons was bimodal—2 peaks in activity were aligned to the 2 retinocentric positions in our design that corresponded to a single, preferred side of the reference object when the object was located to the left or right of the fixation target (Fig. 5A–C; trials with both ambiguous and determinate models as defined below were included; error bars indicate ±1 standard error of the mean). Object-right preferring neurons (Fig. 5A–C; blue lines) were maximally activated when the critical square fell within 2 discrete ranges of retinocentric space (corresponding to the blue-shaded regions) located at different distances from the fixation target. Neurons with object-left preference exhibited a similarly asymmetrical and bimodal spatial tuning function mirror-reflected about the vertical meridian (Fig. 5A–C; red lines). Object-referenced spatial tuning was evident during both the model period (Fig. 5A,B) and the copy period (Fig. 5C) and in both Monkey 1 (Fig. 5A) and Monkey 2 (Fig. 5B,C). We confirmed that the bimodal pattern of spatial tuning at the population level (Fig. 5) was also characteristic of the single neurons in the population, the large majority of which also exhibited bimodal spatial tuning with 2 peaks in activity aligned either with the left (Fig. 6A) or with the right (Fig. 6B) sides of the reference object. Neural activity during the copy period is shown (Fig. 6). A similar pattern of bimodality in spatial tuning was evident during the model period, though the data were noisier, in part because the signal was weaker and in part because more critical square locations were tested in Series 1.
|
|
In viewer-referenced neurons, by contrast, neural population tuning functions were monotonic. Activity was greater when the critical square was located either to the left (Fig. 5D–F; red spatial tuning functions) or to the right (Fig. 5D–F; blue spatial tuning functions) of the gaze fixation target. This was the case in Monkey 1 (Fig. 5D) and Monkey 2 (Fig. 5E,F) and during both the model (Fig. 5D,E) and copy periods (Fig. 5F).
Effects of Varying Object-Referenced Position while Holding Viewer-Referenced Position Constant
The second prediction above is that neural activity coding the position of critical squares relative to objects should vary when critical squares remain fixed in viewer-referenced space, but the reference object changes position to alter the spatial relationship between square and object. To test this prediction, we compared the spatial preference of neurons during the model and copy periods of the same trial in Series A. Model and copy objects were offset by one half of the object width so that fixed regions of retinocentric space corresponded first to one and then the other side of the reference object within the same trial. For example, when the model object appeared offset to the left of fixation (Fig. 7A), the region of retinocentric space immediately to the left of the fixation target (shaded red) corresponded to the right side of the reference object. When the copy object appeared centered on the fixation target later within the same trial (Fig. 7B), the same retinocentric region (shaded red) now corresponded to the left side of the reference object.
|
Neurons activated during the model period were often activated during the copy period as well although typically to a lesser extent (e.g., compare the increase in population activity in model and copy periods in Fig. 8A). This allowed us to compare the spatial preference of neurons across the 2 task periods within the same trial. Figure 7 illustrates population spatial tuning functions computed using neural activity recorded during the model period (blue lines) and copy period (red lines), for neurons with object-left (Fig. 7C) and object-right (Fig. 7D) preference (neurons were included in this analysis if their activity in the model period related significantly to object-referenced position at P < 0.05). Peaks and troughs in the spatial tuning functions indicate retinocentric positions that were preferred and nonpreferred by the neural population during each task period. Population activity evoked by critical squares appearing in fixed retinocentric regions depended on the relative position of the reference object. For example, in neurons with object-left preference (Fig. 7C), critical squares immediately to the right of the fixation target (in region 2) were preferred during the model period but nonpreferred during the copy period. This can be seen by the fact that this retinocentric region contains a relative maximum in the blue tuning function constructed from model period activity and a relative minimum in the red tuning function constructed from copy period activity (Fig. 7C). The opposite pattern is seen for critical squares to the left of fixation (in region 1). Thus, the activity of this neural population represented each fixed retinocentric region as corresponding to opposite sides of the reference object (preferred and nonpreferred) depending on the relative position of the reference object. Neurons with object-right preference exhibited a similarly flexible representation of fixed retinocentric regions as corresponding to opposite sides of the reference object depending on relative location, although the spatial tuning functions were reflected about the fixation target (Fig. 7D). These data are evidence that firing rate did not bear a fixed relationship to retinocentric position, but varied as a function of the relative position of the critical square with respect to the reference object.
Comparing Object-Referenced Signals in the Model and Copy Periods
To compare the strength of neural activity related to the object-referenced position of the critical square in the model and copy periods, we first identified the population of neurons in which activity related to object-referenced position in the ANCOVA at P < 0.05 (165 neurons in the model period, 223 neurons in the copy period). We then compared the distributions of P values associated with the object-referenced factor in these populations of neurons. The average P value associated with the object-referenced factor was 0.017 in the population of neurons active in the model period and 0.009 in the population of neurons active in the copy period, a significant difference (Wilcoxon rank-sum test, P < 10–8) indicative of a more consistent object-referenced signal in the copy period. We also computed an index quantifying the difference in neuronal firing rate attributable to object-referenced position. To compute this index, we determined the mean firing rate of each neuron during the model or copy periods when the critical square was located on the left or right side of the reference object (collapsing across trials in which the reference object appeared to the left or right of the fixation target). We then computed the absolute value of the difference between these rates divided by their sum; |Rleft – Rright|/(Rleft + Rright); where R is the mean firing rate during the model or copy periods and "left" and "right" refer to the object-referenced position of the critical square on each trial. The index is bounded between 0 and 1. The average object-referenced activity index was 0.19 in the population of neurons active in the model period and 0.23 in the population of neurons active in the copy period, a significant difference (Wilcoxon rank-sum test, P = 0.0056) again indicative of a stronger signal in the copy period.
Considering whether Object-Referenced Tuning Arises by a Random Process
We utilized the ANCOVA to identify neurons in which activity varied with object-referenced position and then constructed population tuning (Fig. 5) and activity (Fig. 8) functions using this statistically defined group of cells. One can question whether object-referenced spatial tuning emerged in the analysis as a result of selecting neurons from a population of neurons with random spatial tuning for retinocentric position. In such a population, by chance some neurons would exhibit greater activity when critical squares were on the left side of objects, whereas others would exhibit greater activity when critical squares were on the right side of objects. Dividing the population into right- and left-preferring groups on this basis and averaging their activity would be expected to produce object-referenced spatial tuning, even in the case that the neurons were not, as a population, selective for object-referenced position.
|
To assess the likelihood of this possibility, we tested 2 further predictions of the alternative hypothesis that spatial tuning arises by chance association between mean firing rate and retinocentric position. If spatial tuning is random and neurons are selective for random subsets of retinocentric positions, then alternative patterns of spatial preference for various retinocentric positions should be equally prevalent. We performed an additional ANCOVA to determine the relative frequencies of 2 specific alternative types of spatial selectivity defined in retinocentric coordinates. In the first type, neural activity was selective for the specific pair of retinocentric positions corresponding either to the left or to the right sides of the reference object. Neurons of this type exhibited bimodal, object-aligned spatial tuning and were significantly influenced by the object-referenced factor in the ANCOVA. In the second type of spatial tuning we assessed, neural activity was elevated when the critical square was located in the pair of retinocentric positions that were either closest to or farthest from the fixation target. Neurons with this type of spatial selectivity exhibited either upright or inverted U-shaped spatial tuning functions. We detected U-shaped spatial tuning by including a second factor in the ANCOVA that coded the retinocentric eccentricity of the critical square as a binary factor (near to or far from the fixation target). We found that bimodal tuning was more common than U-shaped tuning (although both forms were detected). Considering neurons significantly affected by one and not the other of the 2 spatial factors, or the interaction, in the ANCOVA above (at P < 0.05), the ratio of bimodal to U-shaped spatial tuning was 134 to 60 neurons in the model period and 182 to 28 neurons in the copy period. The probabilities of obtaining samples skewed in favor of object-referenced tuning by drawing from a population in which the 2 forms were equally prevalent by chance were P < 10–7 and P < 10–27 during the model and copy periods, respectively (binomial distribution assuming the 2 forms of spatial tuning were equally probable). The results were similar when neurons significantly influenced by both factors or by the interaction were included in the counts of neurons with each type of tuning, in which case the probabilities of obtaining the corresponding samples by chance were P < 10–5 and P < 10–14 during the model and copy periods.
To further confirm that object-referenced spatial tuning was not an artifact of selecting the subset of randomly tuned neurons that happened to conform to an object-referenced hypothesis, we conducted a bootstrap analysis. We computed the above object-referenced activity index using the model period activity of every neuron in the sample (we did not prescreen the data by limiting this analysis to the subset of significantly object-referenced neurons as determined by the ANCOVA). During the model period, object-referenced indices in the population ranged from 0 to 0.75, with a mean value of 0.092. (The mean value of the index was reduced relative to the values above because all neurons were included in the present case, those with and without object-referenced activity, as well as neurons that were not active in the task.) We then shuffled the data by randomly reassigning the 8 mean firing rates observed in each neuron to the 8 possible retinocentric positions of the critical square and recomputed the object-referenced index for each neuron as well as the mean value of the index in the population. If object-referenced tuning occurred by chance, shuffling firing rate with respect to retinocentric position on a cell-by-cell basis should have no net effect on the average strength of object-referenced tuning seen in the population. We shuffled mean firing rate and retinocentric position in each neuron 10 000 times and recomputed the mean population object-referenced index after each iteration. The number of times that the shuffling produced a mean population object-referenced index equal to or greater than the value we observed provided a measure of the likelihood of obtaining the observed value by chance. Only 2 of the 10 000 iterations produced a greater mean object-referenced index in the population than the observed value. This indicates that the probability of obtaining by chance a population of randomly tuned neurons with a degree of object-referenced tuning equal to or greater than what was observed was 2 in 10 000 or P 
0.0002. We conclude object-referenced tuning occurs more frequently and with greater strength than predicted by chance.
Population Activity Time Course Is Consistent with Object-Referenced Spatial Representation
We constructed population SDFs (Materials and Methods) to examine the time course of the average activity of statistically defined populations of object-referenced and viewer-referenced neurons. In the object-referenced population, population activity functions nearly overlap during the model period (Fig. 8A) and copy period (Fig. 8B) when critical squares were located on or missing from the preferred side of the reference object (red and green SDF), regardless of whether the critical square and reference object were located in the visual hemifield that was contralateral (red SDF) or ipsilateral (green SDF) to the recorded neuron. Activity of the viewer-referenced populations (Fig. 8C,D) depended in contrast on whether the critical square (and reference object) was located in the preferred (red and orange SDFs) or the nonpreferred (blue and green SDFs) visual hemifield.
It was possible that neurons included in the object-referenced population had a viewer-referenced preference, but that this preference was not systematic on a neuron-to-neuron basis with respect to whether critical squares and objects were located in the ipsilateral or contralateral visual hemifield. To examine this question, we replotted activity time courses for the object-referenced neural population after determining the viewer-referenced preference (significant or not) of each neuron. In this case, population activity exhibited a joint influence of object-referenced and viewer-referenced position such that activity was stronger when critical squares (and objects) fell on the preferred side of the object (Supplementary Fig. 1A; red and green lines vs. blue and orange lines) and also within the preferred visual hemifield (Supplementary Fig. 1A; red vs. green line). When the population was limited to object-referenced neurons (removing neurons for which the probability associated with the viewer-referenced factor in the ANCOVA was P 
0.1), the difference in population activity as a function of viewer-referenced position was substantially reduced (Supplementary Fig. 1B; red and green lines more nearly overlap). This observation confirms the existence of a relatively pure object-referenced signal in some neurons. However, object-referenced and viewer-referenced factors influenced the activity of some neurons jointly to varying degrees, and the separation between the neural populations coding these factors was not absolute.
In the population significantly influenced by object-referenced position during the copy period (Fig. 8B), the population signal coding the position of the critical square in the copy object emerged prominently during the preceding delay period, before the appearance of the copy object. In a prior report, we demonstrated that the emergence of this population signal ahead of the appearance of the copy object reflects the degree to which the position of the missing critical square can be predicted on the basis of the model object alone (Chafee et al. 2005). Thus, the early emergence of the population signal coding the object-referenced position of the missing critical square in the present experiment (Fig. 8B) might reflect foreknowledge of this relative position afforded by the model object. To test the influence of prediction on neural activity, we contrasted population activity on trials in which the model object did (Fig. 9A; Determinate model trials) and did not (Fig. 9B; Ambiguous model trials) specify the location of the critical square with certainty (model and copy configurations on determinate and ambiguous model trials are shown in Fig. 9C,D, respectively). Determinate models had a single square that could be removed to produce the copy object (Fig. 9C; in models with 2 squares present in the same row and on the same side of the object, only the outermost square could be removed). Ambiguous models had 2 squares at different positions that could be removed to produce the copy, so that monkeys could not know the location of the missing square in advance until the copy object was presented (Fig. 9D). (We restricted this analysis to the 30 neurons in which activity related significantly, P < 0.001, to the object-centered factor during the model period in Fig. 8A.) When the model object determined whether the critical square was going to be on the left or right side of the model and copy objects (regardless of the eventual position of those objects in the display), neural activity coding the object-referenced relative position of the critical square emerged shortly after model onset (Fig. 9A; note the divergence of red and green preferred activity functions considered together from orange and blue nonpreferred activity functions during the model period at the arrow). When the model by itself did not determine whether the critical square was going to be on the left or right side of the copy object (and additional information provided by the configuration of the copy object was required), the population signal specifying the object-referenced position of the critical square did not emerge until after the appearance of the copy object (Fig. 9B; note the delayed divergence of red and green preferred activity functions from orange and blue nonpreferred activity functions at the arrow, after the onset of the copy object). Population activity coded the object-referenced position of the critical square specifically at the time in the trial when enough information had been provided to fix its relative position with respect to the reference object.
|
Object-Referenced Spatial Preference Persists when Object Shape Does Not Vary
On ambiguous trials, distinct sets of copy object shapes localized the missing critical square to the left and right side of the copy object (compare copy objects on object-left and object-right trials; Fig. 9D). We sought to confirm that this difference in copy object shape was not responsible for the emergence of the population signal reflecting the position of the missing critical square during the copy period on ambiguous model trials (Fig. 9B). Toward that end, we identified 2 sets of trials (Supplementary Fig. 2A; Set 1 and Set 2) selected to equate the shapes of copy objects on trials in which the missing critical square was located on the left and right side of the copy object. (Note that the same copy object shapes were presented when the missing critical square was on the left and the right side of the copy object in each set. Object-left and object-right trials were further equated for the number of repetitions of each copy object shape.) In these trial sets, half of the model objects were determinate, half ambiguous with respect to the side of the critical square. At the time of the onset of the copy object, population activities on preferred trials (red and green activity functions) and nonpreferred trials (orange and blue activity functions) overlapped (Supplementary Fig. 2B). Approximately 200 ms after the appearance of the copy object, population activity again diverged as a function of whether the missing critical square was located on the preferred or nonpreferred side of the copy object (Supplementary Fig. 2B; arrow). This divergence of population activity during the copy period was not due to a difference in the shape of the copy objects because the same copy objects were presented on preferred and nonpreferred trials. Object shape may influence neural activity in parietal cortex during the object construction task. However, it does not appear that object shape alone can account for object-referenced spatial selectivity.
Hemispheric Bias in Object-Referenced Spatial Representation
We found a contralateral bias in the neural representation of object-referenced space. Most object-referenced neurons (significantly related to the object-referenced factor in the ANCOVA at P < 0.05) were most active when the critical square was located on the contralateral side of reference objects (Fig. 10). Of object-referenced neurons active during the model period, 108 neurons exhibited a preference for the contralateral side of the reference object and 57 neurons exhibited a preference for the ipsilateral side of the reference object (P < 0.001, Fisher's exact test on counts in a 2 x 2 tabulation of neurons by cerebral hemisphere and object-referenced preference). This bias for the contralateral side of the reference object was evident when the reference object appeared alternatively in the ipsilateral and contralateral visual hemifields relative to each recorded neuron. Similarly, of object-referenced neurons active during the copy period, 137 neurons exhibited a preference for the contralateral side of the reference object and 86 neurons exhibited a preference for the ipsilateral side of the reference object (P = 0.002, Fisher's exact test). Considering viewer-referenced neurons active during the model period, a contralateral bias was also found with 126 neurons preferring critical squares (and reference objects) presented in contralateral viewer-referenced space and 71 neurons preferring critical squares presented in ipsilateral viewer-referenced space (Fisher's exact test, 2-sided P < 0.001). No such bias was seen for neurons coding the viewer-referenced position of the missing critical square during the copy period (46 neurons with contralateral preference, 53 neurons with ipsilateral preference; Fisher's exact test, 2-sided P = 0.295).
|
Behavioral Correlates of Object-Referenced Spatial Representation
We measured reaction time on visual probe trials to detect a probe stimulus flashed briefly next to the copy object across trials in which the copy object appeared to the left or right of the fixation target (Fig. 11). On visual probe trials, the monkey viewed a model object followed by the copy object and presumably computed the position of the missing critical square as on normal construction trials (probe trials were unpredictable, occurring on a random 25% of construction trials, and no indication was given that a probe trial was in progress). On a probe trial, 150–750 ms after the copy object appeared, a red square was flashed for 50 ms on one or the other side of the copy object. When the probe stimulus appeared, the monkey was rewarded for immediately pressing the response key, after which the probe trial ended.
|
