Do We See Reality As It Is?

In Visual Intelligence: How We Create What We See Donald Hoffman presents the compelling scientific evidence for vision’s constructive powers, unveiling a grammar of vision – a set of rules that govern our perception of line, color, form, depth, and motion.

Cognitive scientist Donald Hoffman is trying to answer a big question: Do we experience the world as it really is … or as we need it to be? In this ever so slightly mind-blowing talk, he ponders how our minds construct reality for us.

What is the relationship between your brain and your conscious experiences, such as your experience of the taste of chocolate or the feeling of velvet?

This mystery is not new. In 1868, Thomas Huxley wrote, “How it is that anything so remarkable as a state of consciousness comes about as the result of irritating nervous tissue is just as unaccountable as the appearance of the genie when Aladdin rubbed his lamp.” Now, Huxley knew that brain activity and conscious experiences are correlated, but he didn’t know why. To the science of his day, it was a mystery.

Do we see reality as it is?

I open my eyes and I have an experience that I describe as a red tomato a meter away. As a result, I come to believe that in reality, there’s a red tomato a meter away. I then close my eyes, and my experience changes to a gray field, but is it still the case that in reality, there’s a red tomato a meter away? I think so, but could I be wrong? Could I be misinterpreting the nature of my perceptions?

We have misinterpreted our perceptions before. We used to think the Earth is flat, because it looks that way. Pythagorus discovered that we were wrong. Then we thought that the Earth is the unmoving center of the Universe, again because it looks that way. Copernicus and Galileo discovered, again, that we were wrong.

Neurons that create, in real time, all the shapes, objects, colors and motions

Neuroscientists tell us that about a third of the brain’s cortex is engaged in vision. When you simply open your eyes and look about this room, billions of neurons and trillions of synapses are engaged.

Now, this is a bit surprising, because to the extent that we think about vision at all, we think of it as like a camera. It just takes a picture of objective reality as it is. Now, there is a part of vision that’s like a camera: the eye has a lens that focuses an image on the back of the eye where there are 130 million photoreceptors, so the eye is like a 130-megapixel camera. But that doesn’t explain the billions of neurons and trillions of synapses that are engaged in vision. What are these neurons up to?

Well, neuroscientists tell us that they are creating, in real time, all the shapes, objects, colors, and motions that we see. It feels like we’re just taking a snapshot of this room the way it is, but in fact, we’re constructing everything that we see. We don’t construct the whole world at once. We construct what we need in the moment.

Why would neuroscientists say that we don’t just construct, we reconstruct?

Well, the standard argument given is usually an evolutionary one. Those of our ancestors who saw more accurately had a competitive advantage compared to those who saw less accurately, and therefore they were more likely to pass on their genes. We are the offspring of those who saw more accurately, and so we can be confident that, in the normal case, our perceptions are accurate. You see this in the standard textbooks. One textbook says, for example, “Evolutionarily speaking, vision is useful precisely because it is so accurate.” So the idea is that accurate perceptions are fitter perceptions. They give you a survival advantage.

Does natural selection really favor seeing reality as it is?

Fortunately, we don’t have to wave our hands and guess; evolution is a mathematically precise theory. We can use the equations of evolution to check this out. We can have various organisms in artificial worlds compete and see which survive and which thrive, which sensory systems are more fit.

We’re inclined to think that perception is like a window on reality as it is. The theory of evolution is telling us that this is an incorrect interpretation of our perceptions. Instead, reality is more like a 3D desktop that’s designed to hide the complexity of the real world and guide adaptive behavior. Space as you perceive it is your desktop. Physical objects are just the icons in that desktop.

What does this mean for the mystery of consciousness?

Well, it opens up new possibilities. For instance, perhaps reality is some vast machine that causes our conscious experiences. I doubt this, but it’s worth exploring. Perhaps reality is some vast, interacting network of conscious agents,simple and complex, that cause each other’s conscious experiences. Actually, this isn’t as crazy an idea as it seems, and I’m currently exploring it.

But here’s the point: Once we let go of our massively intuitive but massively false assumption about the nature of reality, it opens up new ways to think about life’s greatest mystery. I bet that reality will end up turning out to be more fascinating and unexpected than we’ve ever imagined.

The theory of evolution presents us with the ultimate dare: Dare to recognize that perception is not about seeing truth, it’s about having kids. And by the way, even this TED is just in your head.

Complement Visual Intelligence: How We Create What We See with Understanding philosphy through jokes.