to recognize dogs, for instance, the lower layers recognize simple things
like outlines or color; higher layers recognize more complex stu; like
fur or eyes; and the topmost layer identifies it all as a dog. The same
approach can be applied, roughly speaking, to other inputs that lead a
machine to teach itself: the sounds that make up words in speech, the
letters and words that create sentences in text, or the steering-wheel
movements required for driving.
Ingenious strategies have been used to try to capture and thus
explain in more detail what’s happening in such systems. In 2015,
researchers at Google modified a deep-learning-based image recognition
algorithm so that instead of spotting objects in photos, it would generate or modify them. By e;ectively running the algorithm in reverse,
they could discover the features the program uses to recognize, say, a
bird or building. The resulting images, produced by a project known
as Deep Dream, showed grotesque, alien-like animals emerging from
clouds and plants, and hallucinatory pagodas blooming across forests
and mountain ranges. The images proved that deep learning need not
be entirely inscrutable; they revealed that the algorithms home in on
familiar visual features like a bird’s beak or feathers. But the images
also hinted at how di;erent deep learning is from human perception,
in that it might make something out of an artifact that we would know
to ignore. Google researchers noted that when its algorithm generated
images of a dumbbell, it also generated a human arm holding it. The
machine had concluded that an arm was part of the thing.
Further progress has been made using ideas borrowed from neu-roscience and cognitive science. A team led by Je; Clune, an assistant
professor at the University of Wyoming, has employed the AI equivalent
of optical illusions to test deep neural networks. In 2015, Clune’s group
showed how certain images could fool such a network into perceiving
things that aren’t there, because the images exploit the low-level patterns
the system searches for. One of Clune’s collaborators, Jason Yosinski, also
built a tool that acts like a probe stuck into a brain. His tool targets any
neuron in the middle of the network and searches for the image that
activates it the most. The images that turn up are abstract (imagine an
impressionistic take on a flamingo or a school bus), highlighting the
mysterious nature of the machine’s perceptual abilities.
We need more than a glimpse of AI’s thinking, however, and there is
no easy solution. It is the interplay of calculations inside a deep neural
network that is crucial to higher-level pattern recognition and complex
decision-making, but those calculations are a quagmire of mathemati-
cal functions and variables. “If you had a very small neural network, you
might be able to understand it,” Jaakkola says. “But once it becomes very
large, and it has thousands of units per layer and maybe hundreds of
layers, then it becomes quite un-understandable.”
In the o;ce next to Jaakkola is Regina Barzilay, an MIT professor
who is determined to apply machine learning to medicine. She was diag-
nosed with breast cancer a couple of years ago, at age 43. The diagnosis
s
things that aren’t there, because the images exploit the low-level patterns
the system searches for. One of Clune’s collaborators, Jason Yosinski, also
built a tool that acts like a probe stuck into a brain. His tool targets any
neuron in the middle of the network and searches for the image that
activates it the most. The images that turn up are abstract (imagine an
impressionistic take on a flamingo or a school bus), highlighting the
HOW WELL
CAN WE
GET ALONG
WITH
MACHINES
THAT
ARE
UNPREDICT;
ABLE AND
INSCRUTA;
BLE?