Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Rao, Ballard. NatNeuro 1999Q

  1. “We describe a model of visual processing in which feedback connections from a higher- to a lowerorder visual cortical area carry predictions of lower-level neural activities, whereas the feedforward connections carry the residual errors between the predictions and the actual lower-level activities. When exposed to natural images, a hierarchical network of model neurons implementing such a model developed simple-cell-like receptive fields. A subset of neurons responsible for carrying the residual errors showed endstopping and other extra-classical receptive-field effects. These results suggest that rather than being exclusively feedforward phenomena, nonclassical surround effects in the visual cortex may also result from cortico-cortical feedback as a consequence of the visual system using an efficient hierarchical strategy for encoding natural images.”
  2. “Why should a neuron that responds to a stimulus stop responding when the same stimulus extends beyond the classical RF?”
    1. Why is it that a neuron would be end-sensitive? (respond more if a line terminates in its receptive field than continuing through it)
  3. There are some existing explanations but they end up being very complex when they have to explain everything we know
  4. “Here we show simulations suggesting that extra-classical RF effects may result directly from predictive coding of natural images. The approach postulates that neural networks learn the statistical regularities of the natural world, signaling deviations from such regularities to higher processing centers. This reduces redundancy by removing the predictable, and hence redundant, components of the input signal. “
  5. “Because neighboring pixel intensities in natural images tend to be correlated, values near the image center can often be predicted from surrounding values. Thus, the raw image-intensity value at each pixel can be replaced by the difference between a center pixel value and its spatial prediction from a linear weighted sum of the surrounding values. This decorrelates (or whitens) the inputs17,19 and reduces output redundancy, providing a functional explanation for center–surround receptive fields in the retina and LGN. “
  6. This also means one color gives information about others nearby “Thus, the color-opponent (red – green) and blue – (red + green) channels in the retina might reflect predictive coding in the chromatic domain similar to that of the spatial and temporal domains”
  7. “Using a hierarchical model of predictive coding, we show that visual cortical neurons with extra-classical RF properties can be interpreted as residual error detectors”
  8. Each level in the hierarchy attempts to predict what is lower through feedback connections – this is used to correct the estimate
  9. Lower levels operate on smaller temporal and spatial scales, so receptive field expands as go up, until the top covers everything
  10. Idea is that natural images have properties that can be organized hierarchically, and this should build a model of it
  11. So the model is that higher areas report expected neural activity to lower areas, and lower areas report back up the residual
  12. Trained a 3-level system based on this on natural images seems like standard NN weight-activation scheme
  13. Updates are done to maximize posterior probability
  14. After training, the first level becomes sensitive to oriented edges or line segments (like Gabors), level 2 is a mixture of these
  15. A short bar elicits a strong residual while a long bar a small residual, because the latter are more common in natural images
  16. “The removal of feedback from level 2 to level 1 in the model caused previously endstopped neurons to continue to respond to bars of increasing lengths (Fig. 5a), supporting the hypothesis that predictive feedback is important in mediating endstopping in the level-1 model neurons. “
  17. “Our simulation results suggest that certain extra-classical RF effects could be an emergent property of the cortex using an efficient hierarchical and predictive strategy for encoding natural images.”
  18. “when the stimulus properties in a neuron’s receptive field match the stimulus properties in the surrounding region, little response is evoked from the error-detecting neurons because the ‘surround’ can predict the ‘center.’”
  19. “In anesthetized monkeys, inactivation of higher-order visual cortical areas disinhibits responses to surround stimuli in lower-area neurons”
  20. ” For example, some neurons in MT are suppressed when the direction of stimulus motion in the surrounding region matches that in the center of the classical RF6. This suggests a hierarchical predictive coding strategy for motion analogous to the one suggested here for image features.”
  21. “Certain neurons in the anterior inferotemporal (IT) cortex of alert behaving monkeys fire vigorously whenever a presented test stimulus does not match the item held in memory, though showing little or no response in the case of a match”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: