The walk from the Vector Institute to Hinton’s office at Google,
where he spends most of his time (he is now an emeritus professor at the University of Toronto), is a kind of living advertisement for the city, at least in the summertime. You can
understand why Hinton, who is originally from the U.K.,
moved here in the 1980s after working at Carnegie Mellon
University in Pittsburgh.
When you step outside, even downtown near the financial
district, you feel as though you’ve actually gone into nature. It’s
the smell, I think: wet loam in the air. Toronto was built on top
of forested ravines, and it’s said to be “a city within a park”; as
it’s been urbanized, the local government has set strict restrictions to maintain the tree canopy. As you’re flying in, the outer
parts of the city look almost cartoonishly lush.
Toronto is the fourth-largest city in North America (after
Mexico City, New York, and L.A.), and its most diverse: more
than half the population was born outside Canada. You can
see that walking around. The crowd in the tech corridor looks
less San Francisco—young white guys in hoodies—and more
international. There’s free health care and good public schools,
the people are friendly, and the political order is relatively left-leaning and stable; and this stuff draws people like Hinton,
who says he left the U.S. because of the Iran-Contra affair. It’s
one of the first things we talk about when I go to meet him,
just before lunch.
“Most people at CMU thought it was perfectly reasonable
for the U.S. to invade Nicaragua,” he says. “They somehow
thought they owned it.” He tells me that he had a big breakthrough recently on a project: “getting a very good junior engineer who’s working with me,” a woman named Sara Sabour.
Sabour is Iranian, and she was refused a visa to work in the
United States. Google’s Toronto office scooped her up.
Hinton, who is 69 years old, has the kind, lean, English-
looking face of the Big Friendly Giant, with a thin mouth, big
ears, and a proud nose. He was born in Wimbledon, England,
and sounds, when he talks, like the narrator of a children’s
book about science: curious, engaging, eager to explain things.
He’s funny, and a bit of a showman. He stands the whole time
we talk, because, as it turns out, sitting is too painful. “I sat
down in June of 2005 and it was a mistake,” he tells me, letting
the bizarre line land before explaining that a disc in his back
gives him trouble. It means he can’t fly, and earlier that day
he’d had to bring a contraption that looked like a surfboard to
the dentist’s office so he could lie on it while having a cracked
tooth root examined.
In the 1980s Hinton was, as he is now, an expert on neural networks, a much-simplified model of the network of neurons and synapses in our brains. However, at that time it had
been firmly decided that neural networks were a dead end in
AI research. Although the earliest neural net, the Perceptron,
which began to be developed in the 1950s, had been hailed as
a first step toward human-level machine intelligence, a 1969
book by MIT’s Marvin Minsky and Seymour Papert, called
Perceptrons, proved mathematically that such networks could perform only the most basic functions. These networks had just
two layers of neurons, an input layer and an output layer. Nets
with more layers between the input and output neurons could
in theory solve a great variety of problems, but nobody knew
how to train them, and so in practice they were useless. Except
for a few holdouts like Hinton, Perceptrons caused most people
to give up on neural nets entirely.
Hinton’s breakthrough, in 1986, was to show that backpropagation could train a deep neural net, meaning one with
more than two or three layers. But it took another 26 years
before increasing computational power made good on the
discovery. A 2012 paper by Hinton and two of his Toronto
students showed that deep neural nets, trained using backpropagation, beat state-of-the-art systems in image recognition. “Deep learning” took off. To the outside world, AI seemed
to wake up overnight. For Hinton, it was a payoff long overdue.
Reality distortion field
A neural net is usually drawn like a club sandwich, with layers stacked one atop the other. The layers contain artificial
neurons, which are dumb little computational units that get
excited—the way a real neuron gets excited—and pass that
excitement on to the other neurons they’re connected to. A
neuron’s excitement is represented by a number, like 0.13 or
32. 39, that says just how excited it is. And there’s another crucial number, on each of the connections between two neurons,
that determines how much excitement should get passed from
one to the other. That number is meant to model the strength
Maybe we’re not
actually at the
beginning of a