immense electrochemical patterns into a useful understanding of the world? It does so by comparing the signals it receives from the different sensory inputs, detecting patterns that allow it to make its best guesses about what’s “out there”. Its operation is so powerful that its work seems effortless. But let’s take a closer look.
Let’s begin with our most dominant sense: vision. The act of seeing feels so natural that it’s hard to appreciate the immense machinery that makes it happen. About a third of the human brain is dedicated to the mission of vision, to turning raw photons of light into our mother’s face, or our loving pet, or the couch we’re about to nap on. To unmask what’s happening under the hood, let’s turn to the case of a man who lost his vision, and then was given the chance to get it back.
I was blind but now I see
Mike May lost his sight at the age of three and a half. A chemical explosion scarred his corneas, leaving his eyes with no access to photons. As a blind man, he became successful in business, and also became a championship paralympic skier, navigating the slopes by sound markers.
Then, after over forty years of blindness, Mike learned about a pioneering stem cell treatment that could repair the physical damage to his eyes. He decided to undertake the surgery; after all, the blindness was only the result of his unclear corneas, and the solution was straightforward.
But something unexpected happened. Television cameras were on hand to document the moment the bandages came off. Mike describes the experience when the physician peeled back the gauze: “There’s this whoosh of light and bombarding of images on to my eye. All of a sudden you turn on this flood of visual information. It’s overwhelming.”
SENSORY TRANSDUCTION
Biology has discovered many ways to convert information from the world into electrochemical signals. Just a few of the translation machines that you own: hair cells in the inner ear, several types of touch receptors in the skin, taste buds in the tongue, molecular receptors in the olfactory bulb, and photoreceptors at the back of the eye.
Signals from the environment are translated into electrochemical signals carried by brain cells. It is the first step by which the brain taps into information from the world outside the body. The eyes convert (or transduce) photons into electrical signals. The mechanisms of the inner ear convert vibrations in the density of the air into electrical signals. Receptors on the skin (and also inside the body) convert pressure, stretch, temperature, and noxious chemicals into electrical signals. The nose converts drifting odor molecules, and the tongue converts taste molecules to electrical signals. In a city with visitors from all over the world, foreign money must be translated into a common currency before meaningful transactions can take place. And so it is with the brain. It’s fundamentally cosmopolitan, welcoming travelers from many different origins.
One of neuroscience’s unsolved puzzles is known as the “binding problem”: how is the brain able to produce a single, unified picture of the world, given that vision is processed in one region, hearing in another, touch in another, and so on? While the problem is still unsolved, the common currency among neurons – as well as their massive interconnectivity – promises to be at the heart of the solution.
Mike’s new corneas were receiving and focussing light just as they were supposed to. But his brain could not make sense of the information it was receiving. With the news cameras rolling, Mike looked at his children and smiled at them. But inside he was petrified, because he couldn’t tell what they looked like, or which was which. “I had no face recognition whatsoever,” he recalls.
In surgical terms, the transplant had been a total success. But from Mike’s point of view, what he was experiencing couldn’t be called vision. As he summarized it: “my brain was going ‘oh my gosh’”.
With the help of his doctors and family, he walked out of the exam room and down the hallway, casting his gaze toward the carpet, the pictures on the wall, the doorways. None of it made sense to him. When he was placed in the car to go home, Mike set his eyes on the cars, buildings, and people whizzing by, trying unsuccessfully to understand what he was seeing. On the freeway, he recoiled when it looked like they were going to smash into a large rectangle in front of them. It turned out to be a highway sign, which they passed under. He had no sense of what objects were, nor of their depth. In fact, post-surgery, Mike found skiing more difficult than he had as a blind man. Because of his depth perception difficulties, he had a hard time telling the difference between people, trees, shadows, and holes. They all appeared to him simply like dark things against the white snow.
The lesson that surfaces from Mike’s experience is that the visual system is not like a camera. It’s not as though seeing is simply about removing the lens cap. For vision, you need more than functioning eyes.
In Mike’s case, forty years of blindness meant that the territory of his visual system (what we would normally call the visual cortex) had been largely taken over by his remaining senses, such as hearing and touch. That impacted his brain’s ability to weave together all the signals it needed to have sight. As we will see, vision emerges from the coordination of billions of neurons working together in a particular, complex symphony.
Today, fifteen years after his surgery, Mike still has a difficult time reading words on paper and the expressions on people’s faces. When he needs to make better sense of his imperfect visual perception, he uses his other senses to crosscheck the information: he touches, he lifts, he listens. This comparison across the senses is something we all did at a much younger age, when our brains were first making sense of the world.
Seeing requires more than the eyes
When babies reach out to touch what’s in front of them, it’s not only to learn about texture and shape. These actions are also necessary for learning how to see. While it sounds strange to imagine that the movement of our bodies is required for vision, this concept was elegantly demonstrated with two kittens in 1963.
Richard Held and Alan Hein, two researchers at MIT, placed two kittens into a cylinder ringed in vertical stripes. Both kittens got visual input from moving around inside the cylinder. But there was a critical difference in their experiences: the first kitten was walking of its own accord, while the second kitten was riding in a gondola attached to a central axis. Because of this setup, both kittens saw exactly the same thing: the stripes moved at the same time and at the same speed for both. If vision were just about the photons hitting the eyes, their visual systems should develop identically. But here was the surprising result: only the kitten that was using its body to do the moving developed normal vision. The kitten riding in the gondola never learned to see properly; its visual system never reached normal development.
Inside a cylinder with vertical stripes, one kitten walked while the other was carried. Both received exactly the same visual input, but only the one who walked itself – the one able to match its own movements to changes in visual input – learned to see properly.
Vision isn’t about photons that can be readily interpreted by the visual cortex. Instead it’s a whole body experience. The signals coming into the brain can only be made sense of by training, which requires cross-referencing the signals with information from our actions and sensory consequences. It’s the only way our brains can come to interpret what the visual data actually means.
If from birth you were unable to interact with the world in any way, unable to work out through feedback what the sensory information meant, in theory you would never be able to see. When babies hit the bars of their cribs and chew their toes and play with their blocks, they’re not simply exploring – they’re training up their visual systems. Entombed in darkness, their brains are learning how the actions sent out into the world (turn the head, push this, let go of that) change the sensory input that returns. As a result of extensive experimentation, vision becomes trained up.