Perceptual Similarity of Shapes Generated from Fourier Descriptors. Cortese, Dyre. Journal of Experimental Psychology. 1996.

1. A metric representation of shape is preserved by a Fourier analysis of the cumulative angular
   bend of a shape’s contour. Three experiments examined the relationship between variation in
   Fourier descriptors and judgments of perceptual shape similarity. Multidimensional scaling of
   similarity judgments resulted in highly ordered solutions for matrices of shapes generated by
   a Fourier synthesis of a few frequencies. Multiple regression analyses indicated that particular
   Fourier components best accounted for the recovered dimensions. In addition, variations in
   the amplitude and the phase of a given frequency, as well as the amplitudes of 2 different
   frequencies, produced independent effects on perceptual similarity. These results suggest that
   a Fourier representation is consistent with the perceptual similarity of shapes, at least for the
   relatively low-dimensional Fourier shapes considered.”
2. Although many things are useful for object recognition (color, texture, etc) earlier work shows outline (contour) shape being the most important
3. Mention approach for shape representation as having an alphabet of shape-piece prototypes that are then assembled – can be represented hierarchically or spatially in some other manner.
   1. Pinker was a proponent of this
4. But there hasn’t been any real traction on this from a practical sense, as “The difficulty lies in representing he infinite variety of shapes with a small set of primitives. Typically, the parts are distinguished only by qualitative differences in shape. ”
   1. Idea of geons, codons, but they dont deal with metric variations which seems important
   2. Marr also had idea of some form of decomposition
5. An alternative is to use a system that doesn’t involve parsing an object into parts
   1. Fourier descriptors is one system from computer vision
6. ” In this method, given an arbitrary starting point on a closed contour, the function relating cumulative arc length to local contour orientation is expanded in a Fourier series ”
   1. Has some nice properties, including that global shape characteristics can be determined just by the first few low-frequency terms, also its basically invariant to starting point
7. Fourier descriptors were used in early computer vision and have been considered in biological vision as well
8. One study found “…hat approximately half of the visually responsive neurons in the inferior temporal cortex were selectively tuned to the frequency of FD stimuli ”
   1. “… all frequencies were about equally represented, except for a reduced incidence of the frequency 64 cycles per perimeter. ”  Fits werent quite linear but were still good
9. “In the present experiments, we tested this prediction [that FDs are related to categorization] by obtaining ratings of perceived shape similarity and subjecting them to multidimensional scaling”
10. “…if, on the one hand, perceived shape similarity is related to variation in the amplitude and phase parameters of the contour, then vectors representing these Fourier components should account for the dimensions of the recovered similarity space. If, on the other hand, qualitative
    stimulus attributes are used to represent shape (e.g., smoothness, number of parts, or orientation), then vectors representing these qualities should account for the majority of variability in similarity judgments. For this reason, we also obtained ratings on a number of unidimensional scales representing qualitative aspects of the stimuli.”
11. “In Experiment 1, we varied the amplitude and the phase of a single FD frequency. A Fourier representation of shape would predict that the perceptual similarity space should reflect variation of these two parameters. Also, because of the independence of amplitude and phase in a Fourier representation, we made an additional prediction: The amplitude and the phase of a given FD frequency should show independent effects on perceived similarity.”
12. 
13. Participants were shown 45 pairs, and were told to rate them for similarity on a numeric scale, and then after that they rated each shape on 7 independent numeric scales (width, straightness, smoothness, # of parts, complexity, symmetry, orientation) – these criteria were intended to be alternatives for doing classification
14. MDS using euclidian distance on the similarity ratings – there was a sharp elbow with 2 dimensions
15. 
16. This reproduces almost exactly the earlier figure (just rotated and flipped), “….which suggests that perceived dissimilarity is monotonically related to distance in a 2-D Euclidean space with, in this case, amplitude and phase as the two dimensions. Indeed, the relationship between distances in this space and perceived dissimilarities may be linear: A linear multidimensional-scaling analysis produced a 2-D solution with virtually the same pattern as that for the monotonic analysis”
17. ” that the phase and the amplitude of Frequency 6 accounted for more variability in the judgments of similarity than did any of the unidimensional scales, with the exception of smoothness.”

18. Experiment 2

19. ” Fourier theory also predicts another pattern of effects on similarity judgments: the independence of amplitude values at different frequencies. The purpose of Experiment 2 was to test this prediction…”
20. Based on MDS “it appears that the perceived dissimilarities of these shapes are monotonically related to distance in a Fourier space, with amplitude of frequency 2 and amplitude of frequency 4 as the two dimensions “
21. “Fitted vectors for the amplitudes of the two frequency components were found to be orthogonal
    (angular difference = 88.8°, suggesting that there were independent perceptual effects of variations in amplitude on two different frequencies. This observation, along with the observed independence of amplitude and phase in Experiment 1, is consistent with a representation of shape based on FDs.”
22. Variation in amplitude of freq 2 was highly correlated with judgements of “width” and freq 4 was with “smoothness”

23. Experiment 3

24. “Experiment 3 tested the effects of variation in the phases of two different frequencies on
    judgments of similarity. As in Experiments 1 and 2, this was an investigation of the perceptual effects of variation in two parameters of the Fourier expansion. However, unlike the previous experiments, the parameters manipulated in this experiment did not exhibit independent effects on the shape of the contour, because the relative phases, and not the absolute phases, determined the shape”
25. Here stimuli were constructed from freqs of 4,6,8 cycles/perimiter, with amplitudes held constant.
    1. Phases of freqs 6, 8 were varied indep (need an extra freq of 4 around for the comparison to work)
26. Here stress plot from MDS didn’t have a clear elbow, but plotting with 2 dimensions made items in a ‘U’ shape (a linear manifold), implying a one-dimensional solution – they are dependent
27. “This relationship, taken together with the results of Experiments 1 and 2, which found a significant relationship between number of parts and amplitude of frequencies 4 and 6, suggests that object parsing may be related to the amplitude and the relative phase of frequencies in this range (4 to 8 cycles per perimeter). “
28. “Of particular importance was the evidence found for independent perceptual effects for variations of amplitude and phase on a single frequency and for variations of amplitudes on two different frequencies. Both of these results predicted by a Fourier theory.”

Categorical Clustering of the Neural Representation of Color. Brouwer, Heeger. JNeuro 2013.

1. fMRI study where subjects viewed 12 colors did either a color-naming or distractor task
2. “A forward model was used to extract lower dimensional neural color spaces from the high-dimensional fMRI responses.”
3. Vision areas of V4 and V01 showed clustering for color naming task but not for distractor
4. “Response amplitudes and signal-to-noise ratios were higher in most visual cortical areas for color naming compared to diverted attention. But only in V4v and VO1 did the cortical representation
   of color change to a categorical color space”
5. We can perceive thousands of colors but have only a handful of descriptive categories for colors, so we can see two different colors but would still potentially call it the same thing
6. Inferotemporal cortex (IT) is believed to deal with categorization of color
7. “…performing a categorization task alters the responses of individual color-selective neurons in macaque IT (…).”
8. Similar colors cause overlapping patterns of neural activity, “… neural representations of color can be characterized by low-dimensional ‘neural color spaces’…”
9. “Activity in visual cortex depends on task demands (…).”
   1. Use fMRI to study this
10. “Forward model” is used to reduce fMRI signals to a lower dimensional space of bases
11. ” Normal color vision was verified by use of the Ishihara plates (Ishihara, 1917) and a computerized version of the Farnsworth–Munsell 100 hue scoring test (Farnsworth, 1957).”
    1. <Need to learn about this>
12. “The 12 stimulus colors were defined in DKL (Derrington, Krauskopf and Lennie) color space… We chose the DKL space because it represents a logical starting point to investigate the neural representation of color in visual cortex. Although there is evidence for additional higher-order color mechanisms in visual cortex (Krauskopf et al., 1986), the color tuning of neurons in V1 can be approximated by linear weighted sums of the two chromatic axes of DKL color space
    (Lennie et al., 1990).”
13. Color categorization task was done outside the scanner, involved putting 64 colors into one of 5 categories
14. When in the fMRI, there were two types of episodes.  In one, subjects had to press one of 5 buttons to categorize the color (R,G,B,Y, or purple).  Distractor task was a 2-back test (is color the same as the color 2 steps ago)
15. <details on fMRI processing>
16. Used the forward model from this paper.
17. ” We characterized the color selectivity of each neuron as a weighted sum of six hypothetical
    channels, each with an idealized color tuning curve (or basis function) such that the transformation from stimulus color to channel outputs was one to one and invertible. Each basis function was a half-wave-rectified and squared sinusoid in DKL color space.”
18. Assume voxel response is proportional to the number of responding neurons in that voxel
19. Channel responses C (an n x c matrix where n was number of colors, and c # channels(6)).  Then did PCA on this to “extract neural color spaces from the high-dimensional space of voxel responses(…)”
20. “According to the model, each color produces a unique pattern of responses in the channels, represented by a point in the six-dimensional channel space.  By fitting voxel responses to the forward model, we projected the voxel responses into this six dimensional subspace.”
    1. PCA <A competing method they previously used to do this analysis> did not work as well – had similar results but more variability because it tries to fit noise where the forward model throws it out
21. To visualize the forward model, they ran PCA to project the 6D space to 2D (these 2 dimensions accounted for almost all the variance)
22. “Reanalysis of the current data using PCA to reduce dimensionality directly from the number of voxels to two also yielded two-dimensional neural color spaces that were similar to those published previously. Specifically, the neural color spaces from areas V4v and VO1 were close to circular, whereas the neural color spaces of the remaining areas (including V1) were not circular, replicating our previously published results and supporting the previously published conclusions (Brouwer and Heeger, 2009).”
23. Used many different clustering methods to see if colors labeled in the same color category had a more similar response than those in other categories
24. On to results
25. Subjects were pretty consistent where they put color class boundaries.  Blue and green were the most stable
26. Subjects weren’t told category labels–basically that they were doing clustering–but still categories were intuitively identifiable and pretty stable
27. Color clustering was strongest in V01 and V4v, during the color-naming task.  Responses from neighboring area V3 were more smoothly circular and therefore not as good at clustering.
28. 
29. “The categorical clustering indices were significantly larger for color naming than diverted attention in all but one (V2) visual area (p 0.001, nonparametric randomization test), but the
    difference between color naming and diverted attention was significantly greater in VO1 relative to the other visual areas (p 0.01, nonparametric randomization test). One possibility is that all visual areas exhibited clustering of within-category colors, but that the categorical clustering indices were low in visual areas with fewer color-selective neurons, i.e., due to a lack of statistical power”
30. “… no visual area exhibited categorical clustering significantly greater than baseline for the diverted attention task.”
31. Manual clustering done by subjects matched that done from the neural data, aside from the fact that neurall turqoise/cyan matched with blues, whereas people matched it with greens
32. “Hierarchical clustering in areas V4v and VO1 resembled the perceptual hierarchy of color categories”
    1. In V01 when doing color naming.
    2. The dendogram resulting from the distractor task looks pretty much like garbage
33. <Shame on the editor.  Use SNR without defining the abbreviation – I assume its signal to noise ratio?>
34. “Decoding accuracies from the current data set were similar; forward-model decoding
    and maximum-likelihood decoding and were nearly indistinguishable.”
35. <Between this and the similarity of the result of PCA, what does their forward model buy you?  Is it good because it matches results and is *less* general?>
36. “… we propose that some visual areas (e.g., V4v and VO1) implement an additional color-specific change in gain, such that the gain of each neuron changes as a function of its selectivity relative to the centers of the color categories (Fig. 8C). Specifically, neurons tuned to a color near the center of a color category are subjected to larger gain increases than neurons tuned to intermediate colors”
    1. <It is only shown that doing this in simulation helps clustering, which is in the neural data, but they don’t show that the neural data specifically supports this over other approaches>
37. “Task-dependent modulations of activity are readily observed throughout visual cortex, associated with spatial attention, feature-based attention, perceptual decision making, and task structure (Kastner and Ungerleider, 2000; Treue, 2001; Corbetta and Shulman, 2002; Reynolds and Chelazzi, 2004; Jack et al., 2006; Maunsell and Treue, 2006; Reynolds and Heeger, 2009). These task-dependent modulations have been characterized as shifting baseline responses, amplifying gain and increasing SNR of stimulus-evoked responses, and/or narrowing tuning widths. The focus in the current study, however, was to characterize task-dependent changes in distributed neural representations, i.e., the joint encoding of a stimulus by activity in populations of neurons.”
38. <Need to read all references in section “Categorical specificity of areas V4v and VO1”>
39. Lots of results that show V4 and nearby areas respond to chromatic stimuli.  They have a previous paper (their one from 2009) that V4v and V01 better match perceptual experience of color than other regions, but there aren’t many results dealing with “… the neural representation of color categories, the representation of the unique hues, or the effect of task demands on these representations”
40. Previous EEG studies show that the differences in EEG when looking at one color and then another “…  appear to be lateralized, providing support for the influence of language on color  categorization, the principle of linguistic relativity, or Whorfianism (Hill and Mannheim, 1992; Liu et al., 2009; Mo et al., 2011). Indeed, language-specific terminology influences preattentive color perception. The existence in Greek of two additional color terms, distinguishing light and dark blue, leads to faster perceptual discrimination of these colors and an increased visual mismatch negativity of the visually evoked potential in native speakers of Greek, compared to native speakers of English (Thierry et al., 2009).”
    1. Here however, no evidence of lateralized categorical clustering from fMRI
41. Neural research on Macaques and color, but there are differences in brain structure and sensitivities in photoreceptors between them and us so we need to keep that in mind when examining the results from animal experiments on color
42. “We proposed a model that explains the clustering of the neural color spaces from V4v and VO1, as well as the changes in response amplitudes (gain) and SNR observed in all visual areas. In this model, the categorical clustering observed in V4v and VO1 is attributed to a color-specific gain change, such that the gain of each neuron changes as a function of its selectivity relative to the centers of the color categories.”