Schalk’s results, however, were surprisingly robust. As
one might expect, when Leuthardt’s subjects vocalized a
word, the data indicated activity in the areas of the motor
cortex associated with the muscles that produce speech. The
auditory cortex, and an area in its vicinity long believed to
be associated with speech processing, were also active at the
exact same moments. Remarkably, there were similar yet
slightly di;erent activation patterns even when the subjects
only imagined the words silently.
Schalk, Leuthardt, and others involved in the project
believe they have found the little voice that we hear in our
mind when we imagine speaking. The system has never been
perfect: after years of e;ort and refinements to his algorithms,
Schalk’s program guesses correctly 45 percent of the time.
But rather than attempt to push that number higher (they
expect performance to improve with better sensors), Schalk
and Leuthardt have focused on decoding increasingly complex
components of speech.
In recent years, Schalk has continued to extend the
findings on real and imagined speech (he can tell whether a
subject is imagining speaking Martin Luther King Jr.’s “I Have
a Dream” speech or Lincoln’s Gettysburg Address). Leuthardt,
meanwhile, has attempted to push on into the next realm:
identifying the way the brain encodes intellectual concepts
across di;erent regions.
The data on that e;ort is not published yet, “but the honest
truth is we’re still trying to make sense of it,” Leuthardt says.
His lab, he acknowledges, may be approaching the limits of
what’s possible using current technologies.
“THE MOMENT WE GOT EARLY EVIDENCE that we could decode
intentions,” Leuthardt says, “I knew it was on.”
Soon after obtaining those results, Leuthardt took seven
days o; to write, visualize the future, and think about both
short- and long-term goals. At the top of the list of things to
do, he decided, was preparing humanity for what’s coming, a
job that is still very much in progress.
With su;cient funding, Leuthardt insists, reclining in a
chair in his o;ce after performing surgery, he could already
create a prosthetic implant for a general market that would
allow someone to use a computer and control a cursor in
three-dimensional space. Users could also do things like
turn lights on and o;, or turn heat up and down, using
their thoughts alone. They might even be able to experience
artificially induced tactile sensations and access some
rudimentary means of turning imagined speech into text.
“With current technology, I could make an implant—but how
many people are going to want that now?” he says. “I think
it’s very important to take practical, short interval steps to
get people moved along the pathway toward this road of the
To that end, Leuthardt founded NeuroLutions, a company
aimed at demonstrating that there is a market, even today,
Though the technology is decidedly modest compared
with Leuthardt’s grand designs for the future, he believes this
is an area where he can meaningfully transform people’s lives
right now. There are about 700,000 new stroke patients in
the U.S. each year, and the most common motor impairment
is a paralyzed hand. Finding a way to help more of them
regain function—and demonstrating that he can do it faster
and more e;ectively—would not only demonstrate the power
of brain-computer interfaces but meet a huge medical need.
Using noninvasive electrodes that sit on the outside of the
scalp makes the invention much less o;-putting for patients,
but it also imposes severe limitations. The voltage signals
coming from brain cells may be mu;ed as they travel through
the scalp to reach the sensors, and they may be di;used as
they pass through bone. Either makes them harder to detect
and their origins harder to interpret.
Leuthardt can achieve far more transformative feats using
his implanted electrodes that sit directly on the cortex of the
brain. But he has learned through painful experience that
elective brain surgery is a tough sell—not just with patients,
but with investors as well. When he and Schalk founded
NeuroLutions, in 2008, they hoped to restore movement to
the paralyzed by bringing just such an interface to market.
But the investment community wasn’t interested. For one
thing, neuroscientist-led startups have been testing brain-