Technology Review - May/June 2017

to recognize dogs, for instance, the lower layers recognize simple things like outlines or color; higher layers recognize more complex stu; like fur or eyes; and the topmost layer identifies it all as a dog. The same approach can be applied, roughly speaking, to other inputs that lead a machine to teach itself: the sounds that make up words in speech, the letters and words that create sentences in text, or the steering-wheel movements required for driving.

Ingenious strategies have been used to try to capture and thus explain in more detail what’s happening in such systems. In 2015, researchers at Google modified a deep-learning-based image recognition algorithm so that instead of spotting objects in photos, it would generate or modify them. By e;ectively running the algorithm in reverse, they could discover the features the program uses to recognize, say, a bird or building. The resulting images, produced by a project known as Deep Dream, showed grotesque, alien-like animals emerging from clouds and plants, and hallucinatory pagodas blooming across forests and mountain ranges. The images proved that deep learning need not be entirely inscrutable; they revealed that the algorithms home in on familiar visual features like a bird’s beak or feathers. But the images also hinted at how di;erent deep learning is from human perception, in that it might make something out of an artifact that we would know to ignore. Google researchers noted that when its algorithm generated images of a dumbbell, it also generated a human arm holding it. The machine had concluded that an arm was part of the thing.

Further progress has been made using ideas borrowed from neu-roscience and cognitive science. A team led by Je; Clune, an assistant professor at the University of Wyoming, has employed the AI equivalent of optical illusions to test deep neural networks. In 2015, Clune’s group showed how certain images could fool such a network into perceiving things that aren’t there, because the images exploit the low-level patterns the system searches for. One of Clune’s collaborators, Jason Yosinski, also built a tool that acts like a probe stuck into a brain. His tool targets any neuron in the middle of the network and searches for the image that activates it the most. The images that turn up are abstract (imagine an impressionistic take on a flamingo or a school bus), highlighting the mysterious nature of the machine’s perceptual abilities.

We need more than a glimpse of AI’s thinking, however, and there is
no easy solution. It is the interplay of calculations inside a deep neural
network that is crucial to higher-level pattern recognition and complex
decision-making, but those calculations are a quagmire of mathemati-
cal functions and variables. “If you had a very small neural network, you
might be able to understand it,” Jaakkola says. “But once it becomes very
large, and it has thousands of units per layer and maybe hundreds of
layers, then it becomes quite un-understandable.”
In the o;ce next to Jaakkola is Regina Barzilay, an MIT professor
who is determined to apply machine learning to medicine. She was diag-
nosed with breast cancer a couple of years ago, at age 43. The diagnosis
s
things that aren’t there, because the images exploit the low-level patterns
the system searches for. One of Clune’s collaborators, Jason Yosinski, also
built a tool that acts like a probe stuck into a brain. His tool targets any
neuron in the middle of the network and searches for the image that
activates it the most. The images that turn up are abstract (imagine an
impressionistic take on a flamingo or a school bus), highlighting the