Feb 23, 2025	PSY 340 Brain and Behavior Class 18: Visual Coding

Introduction:  What does it mean to see something? What is vision about?

Too frequently people have a simple and utterly incorrect understanding of what human vision is all about or how it works. The diagram above illustrates one of the major WRONG models of how we see. It makes an analogy between how the human eye/brain works and how television works. In this model the eye and nervous system is believed to make a duplicate copy of what it sees and then sends that copy to the brain. This is a modern version of the same mistaken model of René Descartes and other early modern philosophers.

The visual system does not make a duplicate copy of what it sees. Such a model presumes that there is a kind of little person in the brain who views the copy. But, if that were true, wouldn't that little person need to have another little person view what he or she was seeing and so on? Ultimately, this model assumes an impossible set of processes and contradicts what we know the brain actually does.

When this lecture and the next one are finished, you will understand that

• a brain constructs or builds what someone sees by combining many different but separate elements of the scene in front of that person's eyes. A fundamental factor that affects this construction is the brain's predictive nature: its expectation of seeing what it has seen before.
    
  
• the elements themselves are coded by the visual system itself (beginning at the retina of the eye all the way to processing by the cortex) and the brain uses the coded information when it combines the elements; and,

• what we experience visually is NOT a reproduction of the world as it is, but the brain's best guess at what we are seeing.

Vernon Mountcastle, the late American neuroscientist and perceptual psychologist at Johns Hopkins University, put it this way: "Each of us believes himself to live directly within the world that surrounds him, to sense its objects and events precisely, and to live in real and current time. I assert that these are perceptual illusions. Sensation is an abstraction, not a replication, of the real world" (1974).
 

Components of Vision: What Are You Seeing When You See Something? 

      

• Now, let's look at a single image above on the left. I took it one summer from my window during a very heavy thunderstorm.
• What it might be able to tell us about the components of vision. What can you recognize or not recognize in each image: #1, 2, & 3?

• Consider the differences among the 5 images above on the right. What is the color of the grass under the trees in the upper center of the photographs? Why do you have little trouble seeing the green grass despite the overall coloration? We will come back to this phenomenon at the end of this module.

Our visual system reveals its constructive nature when we encounter visual illusions

The 2023 Best Illusion of the Year Contest Winners:    

• 1st Place: Platform  9 3/4s (Matt Pritchard) (YouTube version, 45")
• 2nd Place: Tower of Cubes (John Salmon) (YouTube version, 1'08)
  
  
  
• 2021 • 2nd Place: The Changing Room Illusion (YouTube version, 1'03") • Very clear example of the predictive brain approach to perception

A further illustration of the importance of "top down" or predictive expectancy in visual perception comes from the way the visual system (which we will outline later in this class and in the next class) actually compresses the richness and detail of the visual information at every step as shown in the figure below.

A. The Eye and Its Connections to the Brain

What happens when light enters the eyes?

• It passes through the pupil, an opening in the center of the iris, that changes size depending upon the amount of light in the environment
   
  
• It passes through the lens, a clear ovoid-shaped body, which can also change shape (by the movement of the ciliary muscles) and which inverts the visual scene in front of it.
   
  
• It reaches the back inside wall of the eye which is composed of the retina, a three-layered assembly of cells which contains, on the deepest layer, the visual photoreceptors that absorb the light.

The Retina (see diagram above and below)

• is comprised of three layers. Even though light has to pass through the topmost layers to reach the photoreceptors at the bottom, those layers are essentially transparent and do not interfere with vision.
   
  
• The bottom-most layer contains the photoreceptor cells (rods & cones). These cells send their signals up to
• The second layer which is comprised of horizontal and bipolar cells. These cells send their signals up to
• The third layer which is comprised of ganglion and amacrine cells. The axons of the ganglion cells exit toward the middle of the retina where they form the optic nerve. 
  

• The human blind spot comes at the point in the retina where the millions of ganglion axons exit the eye and form the optic nerve. Any visual image striking that spot cannot be seen and all people who see have such a spot. However, we are not aware of it because the brain fills in the empty region in the visual field.

The Fovea & the Periphery of the Retina (see the topmost diagram above)

• When you look at objects in the very center of your visual field, their image is projected to a small area of the retina called the fovea.
   
  
• This area is physiologically designed to provide very detailed images of what you look at. How?
• (a) The photoreceptors (almost all are cones) are densely packed
• (b) There are almost no blood vessels or ganglion axons near the fovea to impede light
  
• (c) Receptor cells connect one-to-one with bipolar cells and each bipolar cell to a single ganglion  cell.
• (d) The ganglion cells in the fovea are called midget ganglion cells because they are very small and respond to single cones.
   
  
• The photoreceptors in the periphery of the retina connect up to a smaller number of bipolar and ganglion cells. Because of the process of summation of multiple photoreceptor cells converging on a single bipolar cell, however, the photoreceptors in the periphery respond to much lower or fainter light levels.
  

B. Visual Receptors: Rods and Cones

Rods: respond to low levels of light (not to bright light) and most abundant in the periphery of the retina.

Cones: respond to bright light, essential for color vision, and most abundant in the fovea. There is about 1 cone for every 20 rods (ca. 6 million cones and 120 million rods).

Both rods & cones contain photopigments: these are substances that release energy when they are struck by light (but are stable in the dark). Each photopigment is bound to proteins called opsins. Different kinds of opsins make the photopigments sensitive to different wavelengths of light.

When light strikes the photopigments, they change their form very briefly and release energy. This energy causes second messengers within the photoreceptor to send out a receptor potential. This whole process is generally called transduction.

C. Color Vision

Human beings see light with wavelengths between 350 nm (violet) and about 700 nm (red). 

Other animals see below 350 nm in the ultraviolet range while some reptiles can perceive in the infrared range. These abilities allow animals to deal with aspects of the environment human do not naturally experience.

The Three Present Day Models of Color Vision

1. Trichromatic (Young-Helmholtz) Theory of Vision

Thomas Young (d. 1829) and Herman von Helmholtz (d. 1894) proposed the trichromatic theory of color, i.e., human color vision is the result of the relative rates of response to differing wavelengths of light by three different types of color receptor cells (the cones).
 

• The differing types of cones are maximally receptive to
• Short-wavelengths ["blue"] (ca. 420 nm) [Less abundant than "green" or "red" cones]
  
• Medium wavelengths ["green"] (ca. 540 nm)
• Long wavelengths ["red"] (ca. 570 nm)
   
  
• The types of cones are distributed randomly within the retina and different people have different proportions of each. The brain corrects for these differing proportions.
   
  
• The cones are found mostly in the broad middle of the visual field (= what you can see at any one point in time) rather than on the far periphery of the field.

2. The Opponent-Process Theory of Vision

The trichromatic theory has difficulty in explaining

(1) the phenomenon of negative afterimages (the tendency to replace one color with another, i.e., reds->greens; greens->reds; blues->yellows; yellows->blues; black->white; white->black) and
 

(2) color-vision deficiencies

• red-green color (most common; 8% in M & < 1% in women; sex-linked gene defect)
• blue-yellow (< 1% in both M & W; not sex-linked; Chromosome 7 defect)
• achromatopsia: complete lack of any color vision (black/white/grays; 1 in 33,000 in USA)

Ewald Hering (d. 1918) proposed the opponent-process theory of vision, i.e., we perceive color as a set of three pairs of opposites (red vs. green; yellow vs. blue; white vs. black).

The diagram on the right suggests one way in which the visual system can accommodate both the trichromatic and opponent-process theories of vision. At the level of

the cones, the trichromatic theory is operative. However, at the level of the bipolar, horizontal, and ganglion cells and/or in the geniculate nucleus of the thalamus (see next class), the three colors are wired to produce the opponent-process configuration.

3. The Retinex Theory of Vision

How do we maintain color consistency?

 

Dr. Edwin Land (d. 1991; the inventor of the Polaroid camera) proposed the retinex theory of vision, i.e., the cortex compares information from multiple parts of the retina in order to determine both the brightness and the color of each area in the visual field. According to this theory, the cortex will adjust the color of an object in order to render it closer to its true color based upon information other than the object itself.

We can say, therefore, that vision is not only a bottom-up process driven by the stimuli, but also a top-down process in which higher levels of the brain are involved to correct, alter, and figure-out what it is seeing. The "top-down process" is a function of the brain's predictive functioning: seeing what it expects to see.

---

References

Anderson, C. H., Van Essen, D. C., & Olshausen, B. A. (2005). Directed visual attention and the dynamic control of information flow. In L. Itti, G. Rees, & J. Tsotos (Eds.), Neurobiology of attention (pp. 11-17). Burlington, MA: Elsevier.

Mountcastle, V. (1975). The view from within: Pathways to the study of perception. Johns Hopkins Medical Journal, 136, 109-131.

Raichle, M. E. (2010). Two views of brain function. Trends in Cognitive Science, 14(4), 180-190. https://doi.org/10.1016/j.tics.2010.01.008

Raichle, M. E. (2015, December 8). The restless brain (Kavli Prize Laureate Lecture). Washington, DC: Carnegie Institution for Science. Downloaded from https://www.youtube.com/watch?v=_p8LJ95IChs

The first version of this page was posted on February 22, 2007