Behavioral and Neuropshysiological Correlates of Regret in Rat Decision-Making on a Neuroeconomic Task. Steiner, Redish. Nature Neuroscience 2014

1. Deals with regret as described in the RL literature as rats make decisions (they make a choice they can’t go back on, but may have gotten something better with a different choice)
2. “In humans, the orbitofrontal cortex is active during expressions of regret, and humans with damage to the orbitofrontal cortex do not express regret.  In rats and nonhuman primates, both the orbitofrontal cortex and the ventral striatum have been implicated in reward computations.”
3. In situations of high regret <presumably, the reward of the other option is presented, but not accessible> “… rats looked backwards toward the lost option, cells within orbitofrontal cortex and ventral striatum represented the missed action, rats were more likely to wait for the long delay, and rats rushed through eating the food after that delay.
4. “Dissapointment is the realization that a realized outcome is due to one’s own mistaken action “
5. <Didn’t get to finish – posting for spring cleaning>

Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Rao, Ballard. NatNeuro 1999Q

1. “We describe a model of visual processing in which feedback connections from a higher- to a lowerorder visual cortical area carry predictions of lower-level neural activities, whereas the feedforward connections carry the residual errors between the predictions and the actual lower-level activities. When exposed to natural images, a hierarchical network of model neurons implementing such a model developed simple-cell-like receptive fields. A subset of neurons responsible for carrying the residual errors showed endstopping and other extra-classical receptive-field effects. These results suggest that rather than being exclusively feedforward phenomena, nonclassical surround effects in the visual cortex may also result from cortico-cortical feedback as a consequence of the visual system using an efficient hierarchical strategy for encoding natural images.”
2. “Why should a neuron that responds to a stimulus stop responding when the same stimulus extends beyond the classical RF?”
   1. Why is it that a neuron would be end-sensitive? (respond more if a line terminates in its receptive field than continuing through it)
3. There are some existing explanations but they end up being very complex when they have to explain everything we know
4. “Here we show simulations suggesting that extra-classical RF effects may result directly from predictive coding of natural images. The approach postulates that neural networks learn the statistical regularities of the natural world, signaling deviations from such regularities to higher processing centers. This reduces redundancy by removing the predictable, and hence redundant, components of the input signal. “
5. “Because neighboring pixel intensities in natural images tend to be correlated, values near the image center can often be predicted from surrounding values. Thus, the raw image-intensity value at each pixel can be replaced by the difference between a center pixel value and its spatial prediction from a linear weighted sum of the surrounding values. This decorrelates (or whitens) the inputs17,19 and reduces output redundancy, providing a functional explanation for center–surround receptive fields in the retina and LGN. “
6. This also means one color gives information about others nearby “Thus, the color-opponent (red – green) and blue – (red + green) channels in the retina might reflect predictive coding in the chromatic domain similar to that of the spatial and temporal domains”
7. “Using a hierarchical model of predictive coding, we show that visual cortical neurons with extra-classical RF properties can be interpreted as residual error detectors”
8. Each level in the hierarchy attempts to predict what is lower through feedback connections – this is used to correct the estimate
9. Lower levels operate on smaller temporal and spatial scales, so receptive field expands as go up, until the top covers everything
10. Idea is that natural images have properties that can be organized hierarchically, and this should build a model of it
11. So the model is that higher areas report expected neural activity to lower areas, and lower areas report back up the residual
12. Trained a 3-level system based on this on natural images seems like standard NN weight-activation scheme
13. Updates are done to maximize posterior probability
14. After training, the first level becomes sensitive to oriented edges or line segments (like Gabors), level 2 is a mixture of these
15. A short bar elicits a strong residual while a long bar a small residual, because the latter are more common in natural images
16. “The removal of feedback from level 2 to level 1 in the model caused previously endstopped neurons to continue to respond to bars of increasing lengths (Fig. 5a), supporting the hypothesis that predictive feedback is important in mediating endstopping in the level-1 model neurons. “
17. “Our simulation results suggest that certain extra-classical RF effects could be an emergent property of the cortex using an efficient hierarchical and predictive strategy for encoding natural images.”
18. “when the stimulus properties in a neuron’s receptive field match the stimulus properties in the surrounding region, little response is evoked from the error-detecting neurons because the ‘surround’ can predict the ‘center.’”
19. “In anesthetized monkeys, inactivation of higher-order visual cortical areas disinhibits responses to surround stimuli in lower-area neurons”
20. ” For example, some neurons in MT are suppressed when the direction of stimulus motion in the surrounding region matches that in the center of the classical RF6. This suggests a hierarchical predictive coding strategy for motion analogous to the one suggested here for image features.”
21. “Certain neurons in the anterior inferotemporal (IT) cortex of alert behaving monkeys fire vigorously whenever a presented test stimulus does not match the item held in memory, though showing little or no response in the case of a match”

Different Spatial Scales of Shape Similarity Representation in Lateral and Ventral LOC. Drucker, Aguirre. Cerebral Cortex 2009

Uses the same stimulus as earlier study of Fourier basis and shape

1. ” The results, indicating a coarse spatial coding of shape features in lateral LOC and a more focused coding of the entire shape space within ventral LOC, may be related to hierarchical
   models of object processing.”
2. Small regions of cortex respond to different classes of images, and furthermore “… small regions of cortex contain populations capable of representing the entire space of images in a category”
3. “A counterpoint to this apparent specialization has been the demonstration that information regarding object category is also contained in the distributed pattern of voxel responses across
   and between these specialized regions (…).”
4. The results show there are also coarse-scale representations.  “This type of representation might
   correspond to the ‘‘chorus of fragments’’ model of Edelman and Intrator (1997), where individual properties of objects are represented by separate neural populations.”
5. “Our focus here is upon the representation of variations in stimulus identity within a simplified object category… In practice, the structure of a parameterized space of shapes can be recovered from human behavioral responses (e.g., reaction times or similarity judgments) …”
   1. This similarity may also be reflected in neural patterns, which is what they check out here
6. “Does a similar system of neural representation exist within human visual cortex? The human lateral occipital complex (LOC) shows similar functional properties to those previously ascribed to IT structures in the macaque. This region responds more strongly when a viewer is presented with images of parseable objects, as opposed to images that have no 2- or 3- dimensional interpretation, and appears largely indifferent to the method of object perception, for example, objects may be defined by luminance, texture, motion, or stereo difference”
   1. For IT and macaque, see this
   2. Ventral LOC is also called posterior fusiform sulcus (pFS)
7. “Two recent studies have demonstrated a relationship between the perceptual similarity and the
   distributed pattern of neural activity in LOC (Op de Beeck et al. 2008; Haushofer et al. 2008),”
8. During fMRI scanning, subjects viewed 16 different shapes defined by radial frequency components (RFCs; a series of sine waves of various frequencies describing perturbations from
   a circle; Zahn and Roskies 1972; Fig. 1)
   1. Idea of RFCs actually being used in shape recognition was eventually “experimentally rejected” but it makes convenient stimuli, also totally abstracted with no categorical boundaries
9. Shapes were modified by altering amplitude and phase of a particular frequency component
10. Here neural adaptation (depends on habituation) to shape is studied on neural level
11. “We asked in this study if the degree of recovery from neural habituation at different cortical sites was proportional to the transition in similarity between 2 stimuli.”
12. “In this study, we investigated if the distributed pattern of response can inform as to the identity
    of stimulus variation within an object category;”
13. “Continuous Neural Adaptation in Ventral LOC Is Proportional to Shape Similarity”
    1. No adaptation effects found in lateral LOC
    2. Magnitude in change in shape matched linearly with change in ventral LOC
14. “An alternative explanation for the proportional recovery from adaptation in ventral LOC is that the extreme stimuli (those from the corners of the stimulus space) may evoke a larger neural response generally (e.g., Kayaert et al. 2005). As the larger distance stimulus transitions tend to include these
    extreme stimuli to a greater extent, perhaps the apparent recovery from adaptation is actually a larger response to these extreme stimuli independent of an adaptation effect.”
    1. This is not the case, however, as the results show that “the proportional recovery from adaptation seen in ventral LOC indicates the presence of a population code for stimulus shape and cannot be attributed to a generally greater neural response to extreme stimuli.”
15. “Distributed Pattern Responses Distinguish between Shapes”
16. Use SVMs to analyze data at coarse spatial level, which worked well
17. “The accuracy of the SVM analysis and the identified patch within lateral LOC indicates that the distributed voxel pattern of activity in that area carries information about shape.However, the pattern difference between shapes need not reflect the similarity of the stimuli or indeed have any particular structure. The SVM requires only that patterns be different in order to distinguish them—no assumptions about similarity structure are made or used.”
18. “Within lateral LOC, the strongly discriminant responses seen in the SVM analysis were found to also reflect stimulus similarity consistently across subjects (t4 = 10.0, P = 0.001). In contrast,
    the distributed pattern of response in ventral LOC had a weaker correlation with the perceptual similarity of the stimuli (t4 = 1.2, P = 0.3) (Fig. 4A). The difference between these subregions of
    area LOC was significant (t4 = 11.4, P = 0.0003).”
    1. Mixed evidence for this being attributable strictly to retinotopic similarity
19. “The RFC-Amplitude and RFC-Phase Axes Are Differentially Represented at Coarse and Fine Neural Scales”
20. “Although the distributed neural similarity matrix measured from lateral LOC was strongly correlated with the stimulus similarity matrix, there appeared to be aspects of the structure
    of the neural response not evident in the stimulus matrix”
21. Earlier studies on these shapes showed results that had phase and amplitude being recognized as orthogonal and equally important but that wasn’t completely replicated here.  Results here say the dimensions are “equally perceptually salient”, but that they are not perceived equivalently
22. “…both aspects of the stimulus space [amplitude, phase] are represented by the within-voxel population code within ventral LOC… A rather different result was observed for the distributed
    pattern of response within lateral LOC. There, the distributed pattern across subjects reflected the shapes primarily in terms of RFC-amplitude but not RFC-phase”
23. “For example, clusters of neurons might represent the tightness of the ‘‘knobs’’ of the shapes (defined by RFC-amplitude) independent of the direction that those knobs point within the overall shape (defined by RFC-phase). RFC-amplitude and RFC-phase may be taken as similar to ‘‘feature’’ and ‘‘envelope’’ parameters of Op  de Beeck et al. (2008), respectively; we thus contribute a similar finding in that features are represented in the distributed pattern in lateral LOC much more reliably than the overall shape envelope.”
24. “Based upon the differential sensitivity to shape identity for the adaptation and distributed pattern methods, we argue that although both the lateral and ventral components of area LOC contain neural population codes for shape, the spatial scale of these representations differ. Specifically, the
    absence of a distributed pattern effect within ventral LOC is evidence for a homogeneous representation of the shape space, such that the average response of any one voxel does not differentiate between the shapes, whereas the presence of a distributed code and the absence of an
    adaptation effect in lateral LOC suggests that there is a heterogenous distribution of shape representation,”
25. “Within ventral LOC, no meaningful tuning for the shape space can be identified: The amplitude of the response is no different for different shapes. This indicates that ventral LOC voxels are broadly tuned for shape identity. In contrast, lateral LOC voxels show relatively narrow tuning: there is a progressive decline in the response of a voxel for shapes more distant from the shape for which the voxel is best tuned (which was frequently a stimulus from the edges of the stimulus space). Moreover, lateral LOC voxels appear more narrowly tuned for the RFC-amplitude, as compared with the RFC-phase dimension of the shape space, consistent with our previous observation”
26. “The narrow tuning observed in lateral LOC may also explain the absence of a linear adaptation response in this region to transitions in shape space. If a given voxel is narrowly tuned to a particular region of the shape space, then it may only show recovery from adaptation for stimulus transitions within its tuned area.”
27. <Discussion>
28. ” By using a continuous carryover design, our study was capable of examining neural similarity both on a coarse, across-voxel scale by distributed pattern analysis, as well as on a fine, within-voxel scale using continuous neural adaptation. We can thus compare the information provided at distributed and focal levels.”
29. “Unlike ventral LOC, the lateral portion of LOC did not show adaptation responses that were linearly related to shape similarity. We found that the narrow tuning of lateral LOC voxels could explain this finding, indicating that each particular voxel has a population of neurons that are tuned to one specific region of the shape space. Consequently, most of the transitions between stimuli would not induce neural adaptation within the voxel as they would be transitions between stimuli not within the voxel’s receptive field.”

Categorical Clustering of the Neural Representation of Color. Brouwer, Heeger. JNeuro 2013.

1. fMRI study where subjects viewed 12 colors did either a color-naming or distractor task
2. “A forward model was used to extract lower dimensional neural color spaces from the high-dimensional fMRI responses.”
3. Vision areas of V4 and V01 showed clustering for color naming task but not for distractor
4. “Response amplitudes and signal-to-noise ratios were higher in most visual cortical areas for color naming compared to diverted attention. But only in V4v and VO1 did the cortical representation
   of color change to a categorical color space”
5. We can perceive thousands of colors but have only a handful of descriptive categories for colors, so we can see two different colors but would still potentially call it the same thing
6. Inferotemporal cortex (IT) is believed to deal with categorization of color
7. “…performing a categorization task alters the responses of individual color-selective neurons in macaque IT (…).”
8. Similar colors cause overlapping patterns of neural activity, “… neural representations of color can be characterized by low-dimensional ‘neural color spaces’…”
9. “Activity in visual cortex depends on task demands (…).”
   1. Use fMRI to study this
10. “Forward model” is used to reduce fMRI signals to a lower dimensional space of bases
11. ” Normal color vision was verified by use of the Ishihara plates (Ishihara, 1917) and a computerized version of the Farnsworth–Munsell 100 hue scoring test (Farnsworth, 1957).”
    1. <Need to learn about this>
12. “The 12 stimulus colors were defined in DKL (Derrington, Krauskopf and Lennie) color space… We chose the DKL space because it represents a logical starting point to investigate the neural representation of color in visual cortex. Although there is evidence for additional higher-order color mechanisms in visual cortex (Krauskopf et al., 1986), the color tuning of neurons in V1 can be approximated by linear weighted sums of the two chromatic axes of DKL color space
    (Lennie et al., 1990).”
13. Color categorization task was done outside the scanner, involved putting 64 colors into one of 5 categories
14. When in the fMRI, there were two types of episodes.  In one, subjects had to press one of 5 buttons to categorize the color (R,G,B,Y, or purple).  Distractor task was a 2-back test (is color the same as the color 2 steps ago)
15. <details on fMRI processing>
16. Used the forward model from this paper.
17. ” We characterized the color selectivity of each neuron as a weighted sum of six hypothetical
    channels, each with an idealized color tuning curve (or basis function) such that the transformation from stimulus color to channel outputs was one to one and invertible. Each basis function was a half-wave-rectified and squared sinusoid in DKL color space.”
18. Assume voxel response is proportional to the number of responding neurons in that voxel
19. Channel responses C (an n x c matrix where n was number of colors, and c # channels(6)).  Then did PCA on this to “extract neural color spaces from the high-dimensional space of voxel responses(…)”
20. “According to the model, each color produces a unique pattern of responses in the channels, represented by a point in the six-dimensional channel space.  By fitting voxel responses to the forward model, we projected the voxel responses into this six dimensional subspace.”
    1. PCA <A competing method they previously used to do this analysis> did not work as well – had similar results but more variability because it tries to fit noise where the forward model throws it out
21. To visualize the forward model, they ran PCA to project the 6D space to 2D (these 2 dimensions accounted for almost all the variance)
22. “Reanalysis of the current data using PCA to reduce dimensionality directly from the number of voxels to two also yielded two-dimensional neural color spaces that were similar to those published previously. Specifically, the neural color spaces from areas V4v and VO1 were close to circular, whereas the neural color spaces of the remaining areas (including V1) were not circular, replicating our previously published results and supporting the previously published conclusions (Brouwer and Heeger, 2009).”
23. Used many different clustering methods to see if colors labeled in the same color category had a more similar response than those in other categories
24. On to results
25. Subjects were pretty consistent where they put color class boundaries.  Blue and green were the most stable
26. Subjects weren’t told category labels–basically that they were doing clustering–but still categories were intuitively identifiable and pretty stable
27. Color clustering was strongest in V01 and V4v, during the color-naming task.  Responses from neighboring area V3 were more smoothly circular and therefore not as good at clustering.
28. 
29. “The categorical clustering indices were significantly larger for color naming than diverted attention in all but one (V2) visual area (p 0.001, nonparametric randomization test), but the
    difference between color naming and diverted attention was significantly greater in VO1 relative to the other visual areas (p 0.01, nonparametric randomization test). One possibility is that all visual areas exhibited clustering of within-category colors, but that the categorical clustering indices were low in visual areas with fewer color-selective neurons, i.e., due to a lack of statistical power”
30. “… no visual area exhibited categorical clustering significantly greater than baseline for the diverted attention task.”
31. Manual clustering done by subjects matched that done from the neural data, aside from the fact that neurall turqoise/cyan matched with blues, whereas people matched it with greens
32. “Hierarchical clustering in areas V4v and VO1 resembled the perceptual hierarchy of color categories”
    1. In V01 when doing color naming.
    2. The dendogram resulting from the distractor task looks pretty much like garbage
33. <Shame on the editor.  Use SNR without defining the abbreviation – I assume its signal to noise ratio?>
34. “Decoding accuracies from the current data set were similar; forward-model decoding
    and maximum-likelihood decoding and were nearly indistinguishable.”
35. <Between this and the similarity of the result of PCA, what does their forward model buy you?  Is it good because it matches results and is *less* general?>
36. “… we propose that some visual areas (e.g., V4v and VO1) implement an additional color-specific change in gain, such that the gain of each neuron changes as a function of its selectivity relative to the centers of the color categories (Fig. 8C). Specifically, neurons tuned to a color near the center of a color category are subjected to larger gain increases than neurons tuned to intermediate colors”
    1. <It is only shown that doing this in simulation helps clustering, which is in the neural data, but they don’t show that the neural data specifically supports this over other approaches>
37. “Task-dependent modulations of activity are readily observed throughout visual cortex, associated with spatial attention, feature-based attention, perceptual decision making, and task structure (Kastner and Ungerleider, 2000; Treue, 2001; Corbetta and Shulman, 2002; Reynolds and Chelazzi, 2004; Jack et al., 2006; Maunsell and Treue, 2006; Reynolds and Heeger, 2009). These task-dependent modulations have been characterized as shifting baseline responses, amplifying gain and increasing SNR of stimulus-evoked responses, and/or narrowing tuning widths. The focus in the current study, however, was to characterize task-dependent changes in distributed neural representations, i.e., the joint encoding of a stimulus by activity in populations of neurons.”
38. <Need to read all references in section “Categorical specificity of areas V4v and VO1”>
39. Lots of results that show V4 and nearby areas respond to chromatic stimuli.  They have a previous paper (their one from 2009) that V4v and V01 better match perceptual experience of color than other regions, but there aren’t many results dealing with “… the neural representation of color categories, the representation of the unique hues, or the effect of task demands on these representations”
40. Previous EEG studies show that the differences in EEG when looking at one color and then another “…  appear to be lateralized, providing support for the influence of language on color  categorization, the principle of linguistic relativity, or Whorfianism (Hill and Mannheim, 1992; Liu et al., 2009; Mo et al., 2011). Indeed, language-specific terminology influences preattentive color perception. The existence in Greek of two additional color terms, distinguishing light and dark blue, leads to faster perceptual discrimination of these colors and an increased visual mismatch negativity of the visually evoked potential in native speakers of Greek, compared to native speakers of English (Thierry et al., 2009).”
    1. Here however, no evidence of lateralized categorical clustering from fMRI
41. Neural research on Macaques and color, but there are differences in brain structure and sensitivities in photoreceptors between them and us so we need to keep that in mind when examining the results from animal experiments on color
42. “We proposed a model that explains the clustering of the neural color spaces from V4v and VO1, as well as the changes in response amplitudes (gain) and SNR observed in all visual areas. In this model, the categorical clustering observed in V4v and VO1 is attributed to a color-specific gain change, such that the gain of each neuron changes as a function of its selectivity relative to the centers of the color categories.”

Automated monitoring and analysis of social behavior in Drosophila. Dankert, Wang, Hoopfer, Anderson, Perona. Nature Methods 2009.

1. Machine vision approach to studying aggresion and courtship in flies.
   1. Computes location, orientation, and wing posture
2. “Ethograms may be constructed automatically from these measurements, saving considerable time and effort.”
3. “Both aggression and courtship consist of rich ensembles of stereotyped behaviors, which often unfold in a characteristic sequence.”
4. Approach automates a task that was previously done by hand, so saves labor and is more objective, and may pick up on smaller or shorter-duration details
5. Being able to study behavior in minute detail will allow better analysis of impact of genetics and environment on behavior
6. Each video frame corresponds to 25 measurements/features like pose, velocity, wing pose, (size as well because it determines male/female, other special markings like a dot are used for male-male interaction) etc…
7. Able to identify a number of different high-level actions/behaviors like fighting, courting (accuracy was 90+%)
8. They then looked if they could pick up on behavioral changes by suppressing some neurons that had previously been determined to reduce aggressive behavior, and they were able to confirm the same result, looked at mutants as well
9. System allows for visualization of behavior in a number of ways, such as by making histograms or graphs depicting transitions between behaviors with weights

Toward a Universal Law of Generalization for Psychological Science. Shepard. Science 1987.

1. “A psychological space is established for any set of stimuli by determining metric distances between the stimuli such that the probability that a response learned to any stimulus will generalize to any other is an invariant monotonic function of the distance between them.  To a good approximation, this probability of generalization (i) decays exponentially with this distance, and (ii) does so in accordance with one of two metrics, depending on the relation between the dimensions along which the stimuli vary.  These empirical principles are mathematically derivable from universal principles of natural kinds and probabilistic geometry that may, through evolutionary internalization, tend to govern the behaviors of all sentient organisms.”
2. Psychology is about generalization, because nothing happens exactly the same way twice
   1. This idea though, is often left as a secondary topic in psychology.
   2. This generalization occurs according to some sort of metric
3. Aristotle’s principle of association by resemblance goes back 2000 years, but this was only studied more formally at the beginning of the 1900s with Pavlov (the original whistle or bell caused a response, but he also tested other bells and whistles of differing levels of similarity)
4. Since Pavlov, a common basis of experimentation was around “‘gradients of stimulus generalization’ relating the strength, probability, or speed of a learned response to some measure difference between each test stimulus and the original training stimulus.”
   1. Measuring this accurately began in ’56, when Guttman and Kalish examined Skinner’s work
   2. Author then expanded upon this by testing people in a passive noisy n to n association task, gradients were found when distributions for items in terms of their mapping were similar
5. These gradients were originally defined in terms of hand-designed features (such as the wavelength of light emitted by each button in a set of buttons), but in some cases generalization was nomonotonic, (such as tones separated by an octave) or varied across individuals, species, and stimuli in differing ways
6. Lashley, along with others like Robert R Bush and Frederick Mosteller felt like there was not going to be any invariant law of generalization
7. The idea was, that instead of measuring things based on objective properties (such as the wavelength of the light) to do so according to how that physical parameter space maps to that individuals psychological space.
8. More specifically, consider if there is “… an invariant monotonic function whose inverse will uniquely transform those data into numbers interpretable as distances is some appropriate metric space?… Thus, in a K-dimensional space, the distances between points within each subset of K+2 points must satisfy definite conditions…”
9. The function must be unique based on the properties of the constraints set up: “Provided that the number, n, of points in a space is not too small relative to the number of dimensions of the space, teh rank order of teh n(n-1)/2 distances among those n points permits a close approximation to the distances themselves, up to multiplication by an arbitrary scale factor.”
10. This unknown function can be determined by “nonmetric” multidimensional scaling.  “The plot of the generalization measures gij against the distances dij between points in the resulting configuration is interpreted as the gradient of generalization.  It is a psychological rather than psychophysical function because it can be determined in the absence of any physical measurements on the stimuli.”
11. Basically the P matrix consists of how confusable pairs of stimuli are, and MDS is commonly done on a normalized version of that matrix
    1. Applying this to data from all sorts of experiments, even on different animals, yields basically the same exponential decay function.  This is not something that must fall out of MDS, but is in the data itself that MDS picks up on
12. MDS will not impose monotonicity, so when MDS yielded something nonmonotonic, going up to higher dimensional representations has done the trick.
    1. Interesting discussion about what exactly it yields in terms of colors (for example, colors should be 2D so a circle can be formed connecting red and violet instead of putting them at opposite ends of a line), tones
13. When you can define a reasonable metric (such as lightness, saturation in color) those are usually the closest thing to the MDS results.  Sometimes different metrics are needed though, such as Euclidian or Manhattan
14. “Are these regularities of the decay of generalization in psychological space and of the implied metric of that space reflections of no more than arbitrary design features… Or do they have a deeper, more pervasive source?  I now outline a theory of generalization based on the idea that these regularities may be evolutionary accommodations to universal properties of the world.”
15. Different organisms have different things they have to attend to in order to survive, and how they need to be able to distinguish between a particular stimulus varies.  This is from both evolutionary and individual perspectives.
16. Assume psychological space is in some dimension K.   Color might be 3D in terms of lightness, hue, saturation
17. The exponential law is derived from a set of assumptions about how an organism considers this feature space.
    1. All locations are equally probably
    2. Probability that the region (of the test stimulus) has a size s is based on density function p(s).  The way p(s) looks exactly doesn’t actually make much of a difference in practice, for reasonable distributions
    3. Region is convex and “centrally symmetric” <whats that>.  As is the case for the probability distribution, for the most part things are quite robust to the particular shape of the region
18. The theory of generalization described “applies only to the highly idealized experiment in which generalization is tested immediately after a single learning trial with a novel stimulus.”  Empirical evidence from other test settings of either very long training times on very similar stimuli, or delayed test stimuli will lead to deviations from what is discussed here, which may happen in a few ways:
    1. Instead of exponential, an inflected Gaussian function
    2. “deviation away from rhombic and toward elliptical curves of equal generalization” <?>
19. Brief discussion of how to extend the theory to deal with these cases (such as how to deal with sharply bounded “consequential regions”
20. “We generalize from one situation to another not because we cannot tell the difference between the two situations but because we judge that they are likely to belong to a set of situations having the same consequence.”
21. “probability of generalization approximates an exponential decay function of distance in psychological space”
22. “to the degree that the spreads of consequential stimuli along orthogonal dimensions of that space tend to be correlated, psychological distances in that space approximate the Euclidian or non-Euclidian metrics associated, respectively with the L2- and L1- norms for that space.”

Decoding and Reconstructing Color from Responses in Human Visual Cortex. Brouwer, Heeger. JNeuro 2009

1. Tried to decode color from FMRI with “conventional pattern classification, a forward model of idealized color tuning, an d ” PCA
   1. The conventional classifier was able to match training data to colors, but the forward model was able to extrapolate to new colors
2. Color was decoded accurately from:
   1. V1, V2, V3, V4, and V01
   2. But not L01, L02, V3A/B or MT+
3. In V4 and V01, 1st 2 principcal components “revealed progression through perceptual color space” (closeness defined a special way)
   1. This similarity didn’t manifest itself anywhere else, even though classification may have been accurate, and classification was actually most accurate in V1, where this similarity effect didn’t manifest itself.
   2. “This dissociation implies a transformation from the color representation in V1 to reflect color space in V4 and V01.”
4. There is color sensitivity throughout visual cortex
5. Classification of visual information through fMRI has been done previously on object categories, hand gestures, and visual features
6. <mostly skipping notes on materials and methods>
7. Stimulus was a slowly drifting series of concentric rings<, actually a little unclear about this, the description of the colors, and motion are not clear to me>
8. Classification was done through an 8-way (8 colors originally presented) classifier, not some means of regression
9. “The first two principal components of the simulated cone-opponency responses revealed results similar to those observed in V1.”
10. “The forward model assumed that each voxel contained a large number of color-selective neurons, each tuned to a different hue.” <There are more details>
11. The cone-opponency model, however was worse at recreating a space that pushed all the colors apart, their forward model was successful at that, however
12. Forward model not only allowed for decoding, but also reconstructing stimulus colors from test data
13. <Skipping to discussion, running out of time>
14. Mean voxel responses themselves did not reliably distinguish color
15. Here saturation didnt vary, only hue varied
16. “Obviously, the lack of progression in the early visual areas (in particular V1) should not be taken as an indication that these areas are colorblind…  An alternative model of color selectivity, based on cone-opponent tuning rather than hue tuning, reproduced many features of the non-circular and self-intersecting color space derived from teh V1 PCA scores”
17. “…spatially distributed representations of color in V4 supported ‘interpolation’ to decode a stimulus color based on the responses to perceptually similar colors.”
18. “Nonetheless, our results support the hypothesis that V4 and VO1 play a special role in color vision and the perception of
    unique hues…”

Quantifying the Internal Structure of Categories Using a Neural Typicality Measure. Davis, Poldrack. Cerebral Cortex 2014

1. Deals with the internal structure/representation of category information
2. <Seems like assumption is there is something of an exemplar representation>
3. “Internal structure refers to how the natural variability between-category members is coded so that we are able to determine which members are more typical or better examples of their category. Psychological categorization models offer tools for predicting internal structure and suggest that perceptions of typicality arise from similarities between the representations of category members in a psychological space.”
4. Based on these models, develop a “neural typicality measure” that checks if a category member has a pattern of activation similar to other members of its group, as well as what is central to a neural space.
5. Use an artificial categorization task, find a connection between stimulus and response
   1. “find that neural typicality in occipital and temporal regions is significantly correlated with subjects’ perceptions of typicality.”
6. “The prefrontal cortex (PFC) is thought to represent behaviorally relevant aspects of categories such as
   rules associated with category membership (…). Motor and premotor regions may represent habitual responses associated with specific categories (…). The medial temporal lobe (MTL) and subregions of the striatum are thought to bind together aspects of category representations from these other systems.”
7. Different areas and different neurons and patterns of activation in an area can “reliably discriminate
   between many real world object categories”
8. Consider examples of category data as having some sort of “internal structure” or feature representation specific to that class.
   1. These features can say things like how typical a concrete example is, and is related to how quickly and accurately classification occurs
9. “Depending on the specific model, a category representation may be a set of points associated with a given category (exemplar models; …), a summary statistic ( prototype models; …), or a set of statistics (clustering models; …) computed over points associated with a category.”
10. Items closer to other examples in the class, or to the prototype are considered to be most typical or likely
11. But they don’t propose that an accurate model is exactly the same thing a computer does, as there are examples of where nonintuitive things happen.
    1. Ex/ culture can influence how things are categorized, as can a current task or other context
12. “Here, our goal is to develop a method for measuring the internal structure of neural category representations and test how it relates to physical and psychological measures of internal structure.”
13. The neural typicality measure is related to nonparametric kernel density estimators, but “A key difference between our measure and related psychological and statistical models is that instead of using psychological or
    physical exemplar representations, our measure of neural typicality is computed over neural activation patterns…”
14. Use a well studied research paradigm of categorizing simple bird illustrations into 4 categories based on neck angle and leg length.  Previous results show people reconstruct classes based on average item for each category
15. “Our primary hypothesis is that psychological and neural measures of internal structure will be linked, without regard to where in the brain this might occur.”
    1. Also expect that some categorization will happen in visual cortex, and higher level temporal and medial-temporal regions, which “…. are theorized to bind together features from early visual regions into flexible conjunctive category representations (…).”
    2. There are other parts relevant to categorization, but not particularly this form of visual categorization, and other parts may be sensitive to things like entropy
16. “To foreshadow the results, we find that neural typicality significantly correlates with subjects’ perceptions of typicality in early visual regions as well as regions of the temporal and medial temporal cortex. These results suggest that neural and psychological representational spaces are linked and validate the neural typicality measure as a useful tool for uncovering the aspects of category representations coded by specific brain regions.”
17. “For analysis of behavioral responses, response time, and typicality ratings, a distance-to-the-bound variable was constructed that gave each stimulus’ overall distance from the boundaries that separate the categories in the stimulus space. Distance-to-the-bound is a useful measure of idealization: items that are distant from the bound are more idealized than items close to the bound (…).”
18. “For the psychological typicality measure, a value for each of the Test Phase stimuli was generated by interpolating, on an individual subjects basis, a predicted typicality rating from the subjects’ observed typicality ratings…”
19. Also did a physical typicality measure, which is pretty simple to understand (just neck angle, leg length measurements)
20. Then a neural typicality <too much details to list here>
    1. “Our neural typicality measure is based on similarities between multivariate patterns of activation elicited for
       stimuli in the task. Stimuli that elicit activation patterns that are like other members of their category are more neurally typical than those that elicit dissimilar patterns of activation.”
21. Subjects’ behavioral responses were predicted by SVM
22. Typicality ratings were highly correlated with distance-to-the-bound
    1. Reveals that most typical items, and not the average item are the one that is used for category representation.  There are a few other results that show this is the case through other methodology
23. Neural typicality is linked to psychological typicality
24. Found activity in visual cortex and MTL that have been found to be linked to categorization
25. “These results suggest that, in the present task, the internal structure of neural category representations in temporal and occipital regions are linked to subjects’ psychological category representations such that objects that are idealized or physical caricatures of their category elicit patterns of activation that are most (mathematically) similar to other members of their category.”
26. “… in the present task, physical similarity is not a significant contributor to the internal structure of neural category representations, at least not at a level that is amenable to detection using fMRI.”
27. Also did MDS for classification on the neural data, <results don’t seem amazing, but only ok>
28. SVM for classification “The SVMs are given no information about the underlying stimulus space, and unlike
    the MDS analysis, do not make any assumptions about how the dimensions that separate the categories will be organized. Thus, the SVMs can be sensitive to regions that code rulebased or behavioral differences between categories, regions that encode information about their perceptual differences, or regions that code some combination of behavioral and perceptual information.”
29. “Although there is strong overlap in the visual and MTL regions that discriminate between categories and represent
    internal structure, the motor/premotor, insula, and frontal regions were only identified in the between-category analysis. These results are consistent with the hypothesis that PFC and motor/premotor regions are more sensitive to behavioral aspects of categories (…). However, because behavioral responses are strongly associated with the perceptual characteristics of each category, the SVM results are also consistent with the hypothesis that these regions contain some perceptual information about the categories.”
30. “The present research adds to the growing consensus that categorization depends on interactions between a number of
    different brain regions… An important point that this observation highlights is that there may not be any brain region that can be thought of representing all aspects of categories, and thus it might be most accurate to think of brain regions in terms of the aspects of category representations that they code.”
31. “…in the present context, the deactivation of regions of the striatum with increasing typicality likely indicates an uncertainty signal, as opposed to category representation…”
32. “Because our neural typicality measure is not based on mean activation-level differences between stimuli, it may be
    more directly interpretable and less susceptible to adjacency effects in studies of longer term internal category structure.”
    1. <Hm, should read their methodology more carefully on another read-through>
33. They don’t have results that indicate suppression of adjacent stimulus
34. Says their methodology should be tested in real-world, and more artificial settings
35. Evidence of “dimensional selective attention” where not all features are attended to for classificaiton
    1. “Attentional mechanisms in the PFC that instantiate rule-based strategies (…) may contribute to selective attention effects by influencing neural representations in a top-down manner.”
    2. Although: “In the present context, dimensional selective attention is insufficient for explaining the idealization effect because dimensional selective attention affects an entire dimension uniformally… additional mechanisms are required.”
36. “Attention has been found to create a spotlight around salient regions of visual space such that the processing of stimuli
    close to this location in space is enhanced (not just differences along a specific dimension of visual space; …). It is conceptually straightforward to predict that the same or similar spotlight mechanisms may affect the topography of stored neural stimulus representations, such that regions of a category space that contain highly idealized category members are enhanced and contribute more to categorization and typicality judgments than exemplars in ambiguous regions of category space.”
37. Another model is one that specifically tries to “… to reduce prediction error and confusion between categories (…). In these models, category members are simultaneously pulled toward representations/members of their own categories and repelled by members of opposing categories.”
    1. But this doesn’t seem to be a possible explanation here because “… the neural effects as actual neuronal changes in regions of early visual cortex happen on a much longer scale than our task.”
38. This study only tried to find correlation between “psychological” and “neurological” responses, but more in-depth exploration of their relationship is a good idea and left for future work
39. “Our task involves learning to distinguish multiple categories, akin to A/B tasks, and so our finding that early visual cortex is involved with representing category structure may be at odds with theories emphasizing the role of task demands (as opposed to featural qualities) in determining which perceptual regions will be recruited to represent categories.”
    1. Although these distinctions may be an artifact of the type of analysis used

Behavioral and neurophysiological correlates of regret in rat decision-making on a neuroeconomic task. Steiner, Redish. Nature Neuroscience.

1. Deals with regret as described in the RL literature as rats make decisions (they make a choice they can’t go back on, but may have gotten something better with a different choice)
2. “In humans, the orbitofrontal cortex is active during expressions of regret, and humans with damage to the orbitofrontal cortex do not express regret.  In rats and nonhuman primates, both the orbitofrontal cortex and the ventral striatum have been implicated in reward computations.”
3. In situations of high regret <presumably, the reward of the other option is presented, but not accessible> “… rats looked backwards toward the lost option, cells within orbitofrontal cortex and ventral striatum represented the missed action, rats were more likely to wait for the long delay, and rats rushed through eating the food after that delay.
4. “Disappointment is the realization that a realized outcome is worse than expected; regret is the realization that the worse than expected outcome is due to one’s own mistaken action… that the option taken resulted in a worse outcome than an alternative option or action would have.”
5. “Orbitofrontal cortical neurons represent the chosen value of an expected future reward… [and] has been hypothesized to be critical for learning and decision making, particularly in the evaluation of expected outcomes.”
6. Ventral striatum is also implicated in evaluation of outcomes
7. In rats, neural recordings show VS and OFC deal with reward, reward/value predictions.  “In the rat, lesion studies suggest that orbitofrontal cortex is necessary for recognition of reward-related changes that require inference, such as flavor and kind, while vStr is necessary for recognition of any changes that affect value.  In rats deliberating at choice points, vStr reward representations are transiently active before and during the reorientation process, but reward representations in OFC are only active after the reorientation process is complete.”
8. The experiments are somewhat along the lines of the secretary problem – rats run around a loop and can stop at one of a number of places to get food.  When entering an area, a stochastic wait time (until food was released) was introduced.  A tone after entering indicated how long the wait would be.  Rats could either decide to wait for the food or proceed on.  Delays were IID and uniformly distributed
9. Experiment run on 4 different rats – they all basically took a threshold approach where if the wait was less than a certain value they would wait, and otherwise they would move on.
   1. If they decided to skip the reward and move on, the delay on this was independent of delay.  Data indicates they made a decision upfront and did not simply wait for a period of time to move on (either they left after a short period of time, or waited around completely until food was delivered)
   2. Threshold between waiting and skipping tended to be related to each of the 1 of 4 flavors of possible food, one for each quadrant
   3. Upon delivery of reward, rats usually waited 20-25s to leave to get next rewards
10. In variation of the task where one zone provided 3x amount of food, rats chose to wait longer for that larger amount
11. 81, 86% of OFC, VS neurons responded to reward, respectively.  “Responses in bot OFC and vStr often differentiated among the four reward sites (…).”
    1. Used Bayesian classification to determine food quadrant from activity pattern (they included an extra zone from the 4 food quadrants to correspond to locations in the track between quadrants where food could not be obtained
    2. Results of classification between OFC and VS are overall qualitatively quite similar
12. Both OFC and VS signals distinguish between zones both at time of entering zones, as well as at time reward was produced
13. Responses in both areas when food delay was below threshold, but not so in cases when delay was above threshold
    1. “This suggests that these structures were indicating expected value, and predicting future actions.”
    2. This was tested specifically when presenting delays right around threshold (so the difference in possible delays was small, but rats would either choose to stay or go).  The same results were found, that activity was related to the decision and not the environment
14. Now moving on to regret specifically
15. For regret to be induced, the agent needs to know what outcome occurred, as well as what the expected outcome of all actions are – these conditions exist in this domain
16. “Because the rats were time-limited on the Restaurant Row task, encountering a high-cost delay after not waiting through a low-cost delay means that skipping the low-cost delay was a particularly expensive missed opportunity.”
    1. These conditions did occur in the experiment, in some cases the rats would skip a below-threshold reward and then be faced with a high delay
17. “Theoretically, the key to regret is a representation of the action not taken.”
    1. <Their interpretation of this is that is “… that there should be representations of the previous choice during the regret-inducing situations, particularly in contrast to control conditions that are merely disappointing.”  Usually in the RL literature it is done w.r.t. the expectation, but I guess this is reasonable here because the rat knows exactly what it passed up on – the noise is presented to the rat in the form of an audio cue.>
18. To tease apart disappointment from regret they took sequences where the waits were the same, but the rat behavior differed.
    1. In this case, the rat acted optimally (by taking the short wait) but may be disappointed by the long wait following, as it may want to eat another time
    2. The second control is when cue 1 + 2 are above threshold (as opposed to just #2) and where the rat skipped both options.  Again, in this case, the options presented weren’t optimal, but the rat behaved optimally given the circumstances
    3. Experimental set-up had regret and control instances evenly distributed
19. “Behaviorally, rats paused and looked backwards toward the previous option upon encountering a potentially regret-inducing sequence, but they did not do so in either control condition (…).”  In both the control instances, (where the rat acted correctly), the rat did not look back.
20. “During potential regret instances, individual reward-responsive neurons in OFC and vStr showed activity patterns more consistent with the previous reward than the current one (…). Neural activity peaked immediately after the start of the look back toward the previously skipped, low-cost reward.”
    1. That is for individual neurons.  For the population as a whole, representation of the previous reward was weak, whereas representation related to the previous zone was stronger
    2. The representation of the previous zone (where the regret-inducing decision was made), did not occur in non-regret situations
21. Regret didn’t only manifest through immediate behavioral <looking back> and neural responses but also in terms of future decision making.
    1. This is the case – rats tended to take the subsequent long delay (that they normally would reject) after rejecting the previous short delay (that they normally accept, and should do so optimally)
    2. They also ate *much* more quickly in the regret case where they accepted the bad offer than normal – the otherwise average case and both other controls are basically the same and are markedly different
22. In regret cases, increased representation of the previous zone was correlated to accepting the bad offer, this wasn’t the case in controls, which both had a high-cost second choice
23. There was a clear representation of the previous zone, but not other zones
24. Earlier work implicates OFC and VS in calculating expectation of reward.
    1. “Our data indicate that violation of an expectation initiates a retrospective calculation of expectation, this retrospective calculation of expectation influences future behavior: rats are more willing to wait for reward after a regret instance.”
25. “While some evidence suggests that OFC represents economic value, the representation of regret is more consistent with the hypothesis that OFC encodes the outcome parameters of the current, expected, or imagined state.  The data presented here are also consistent with the essential role of OFC in proper credit assignment.  Previous studies have identified potential representations of the counterfactual could-have-been-chosen option in rats, monkeys, and humans.”
26. “The connectivity between OFC and vStr remains highly controversial, with some evidence pointing to connectivity and other analyses suggesting a lack of connectivity.”

Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement Learning. Gustafson, Daw. PLOS Computational Biology. 2011.

1. How does representation make learning possible in spatial navigation.
   1. In particular, considering (activity of pyramidal) hippocampal place and grid cells can contribute to estimates of a value function
2. As opposed to localization, here transfer between earlier and present experience is considered
3. “Accordingly, although neural populations collectively offer a precise representation of position, our simulations of navigational tasks verify the suggestion that RL gains efficiency from the more diffuse tuning of individual neurons, which allows learning about rewards to generalize over longer distances given fewer training experiences.”
4. As opposed to Euclidian distances that may not respect the manifold of the domain (and cause problems in the case of boundaries), a geodesic distance is what is useful
5. The brain has at least 2 representations of location:
   1. “Hippocampal place cells fire when a rat passes through a confined roughly concentric, region of space…”
   2. “… grid cells of the dorsomedial enthrohinal cortex (dMEC) discharge at vertices of regular triangular latices […].”
6. Most studies based on place and grid cells consider what type of stimulus/environment makes them active.  Here, the consideration is how does the brain use that activity to organize behavior
7. “Importantly, this exercise views the brain’s spatial codes less as a representation for location per se, and instead as basis sets for approximating other functions across space.”
   1. That is, activity of place cells alone, would be capable of producing value functions and policies, but is there something about the combination of place and grid cells that makes path planning even easier (or in particular, producing value functions that respect topography, as discontinuities in dynamics lead to discontinuities in value)
8. Activity of place and grid cells are dependent somewhat on characteristics of domain
9. In simulated experiments, there were grid worlds with different numbers of boundaries.  Start positions were randomized, but goal position always remained the same
   1. To simulate place cells, Gaussian basis functions were used
   2. To simulate grid cells, sine waves were used
10. Naturally, the tabular approach was the worst (no generalization).  In the simplest domain the place cells are clearly better than grid cells, which are clearly better than tabular.  In the hardest domain, the performance of grid+place cells are equivalent and still better than tabular
11. But, looking at the value functions, it is clear that goodness from the goal is “bleeding” across boundaries in a way that is not appropriate
12. Because of the overgeneralization, in another set of more complex tasks, the tabular representation does better
13. To fix this, and so that basis functions would respect topology, points were assigned new x-y coordinates, basically by running connectivity through ISOMAP
    1. After doing this, there wasn’t spillage across boundaries.
14. <Although they use shortest geodesic distance, there is no reason why that would be the only method that would produce these results.  Basically, you just need something that respects the fact that you cant cross walls (for example a random walk also respects this>
15. <I’m not sure why, but they keep comparing tabular vs grid vs place cells.  The brain has the latter two together, so why not show the results of their combined activity?  Maybe there is also some prevention of spillage across boundaries in the naive case when used together, or something else interesting….>
16. There are results from place cell activity that respect domain topology w.r.t. doors/boundaries
    1. This is true also when the domain is nonstationary/changes during activity.  This can even cause new place cell activity
17. “One of the hallmarks of model-based planning (and the behavioral phenomena that Tolman [67] used to argue for it, albeit not subsequently reliably demonstrated in the spatial domain), is the ability to plan novel routes without relearning, e.g. to make appropriate choices immediately when favored routes are blocked or new shortcuts are opened. Interestingly, rather than by explicit replanning, some such behaviors could instead be produced more implicitly by updating the basis functions to reflect the new maze, while maintaining the weights connecting them to value. This is easy to demonstrate in the successor representation [16], a model closely related to ours.”