Predictive Coding in the Visual Cortex

Have you ever wondered which piece of the predictive processing story sits on the firmest empirical ground? The visual cortex is a fair answer. Nowhere in the brain has the case for predictive coding been made as carefully or replicated as thoroughly.

The standard textbook story of vision was hierarchical and feedforward. Light hits the retina. Signals move from primary visual cortex outward, becoming more abstract at each stage. Consciousness compiles the result. That story was elegant, well-supported for decades, and, it turns out, incomplete.

The first serious challenge came from anatomy. When researchers actually counted the connections in the visual system, they found something surprising: the top-down projections outnumbered the bottom-up projections by a substantial margin (Bastos et al., 2012) [Externally Validated Evidence]. Whatever the brain was doing with the visual signal, it was clearly not merely receiving it. Something roughly equal in magnitude was flowing the other way.

Rao and Ballard's 1999 computational model proposed what that something might be. In their framing, each stage of the visual hierarchy generates a prediction about what the stage below should be seeing, and the stage below sends back only the prediction error — the part that differs from expectation. Higher-order areas then update their predictions based on that error signal. What reaches conscious perception is the negotiated result.

Subsequent imaging work has produced impressive support for this framing. Studies using fMRI show that early visual cortex activity is measurably reduced when a stimulus matches expectation and elevated when it violates expectation — exactly what the model predicts. Single-unit recordings in primates show similar patterns at the neuron level. The evidence has held up across labs and methodologies.

What this means for a non-neuroscientist is worth being concrete about. When you look at a face, the visual system is not passively assembling a face from lines and shadows. It is running an active hypothesis — this is a face, probably my sister's, probably feeling roughly the emotion the last few seconds of context suggested — and updating the hypothesis wherever incoming data forces a change. The face you consciously see is that hypothesis, rendered with just enough correction to stay accurate.

This is why optical illusions work. They exploit the visual system's willingness to complete a scene from prior expectation, filling in what the model expects to be there even when the sensory data disagrees. It is also why you can misread someone's expression under emotional load — the prediction the nervous system is running about that person, in that moment, shapes what your visual system delivers to consciousness.

The visual-cortex evidence is the reason the broader predictive processing frame is taken seriously. Whether or not every claim built on top of it holds up, the core observation — that perception is a controlled hallucination stabilised by sensory correction — is difficult to argue with once the anatomy and imaging are on the table.

For the broader framework, see The Brain Is a Prediction Machine. For where prediction error fits, Prediction Error — The Currency of Learning is the companion.