three adages that have come to be known
as Clarke’s Three Laws:
When a distinguished but elderly scientist states that something is possible,
he is almost certainly right. When he
states that something is impossible, he
is very probably wrong.
The only way of discovering the limits
of the possible is to venture a little way
past them into the impossible.
Any su;ciently advanced technology is
indistinguishable from magic.
Personally, I should probably be wary
of the second sentence in his first law, as
I am much more conservative than some
others about how quickly AI will be ascendant. But for now I want to expound on
Clarke’s Third Law.
Imagine we had a time machine and
we could transport Isaac Newton from
the late 17th century to today, setting him
down in a place that would be familiar to
him: Trinity College Chapel at the University of Cambridge.
Now show Newton an Apple. Pull out
an iPhone from your pocket, and turn it
on so that the screen is glowing and full
of icons, and hand it to him. Newton, who
revealed how white light is made from
components of di;erent-colored light by
pulling apart sunlight with a prism and
then putting it back together, would no
doubt be surprised at such a small object
producing such vivid colors in the darkness of the chapel. Now play a movie of
an English country scene, and then some
church music that he would have heard.
And then show him a Web page with the
500-plus pages of his personally annotated copy of his masterpiece Principia,
teaching him how to use the pinch gesture
to zoom in on details.
Could Newton begin to explain how
this small device did all that? Although
he invented calculus and explained both
optics and gravity, he was never able to sort
out chemistry from alchemy. So I think he
would be flummoxed, and unable to come
up with even the barest coherent outline
of what this device was. It would be no dif-
ferent to him from an embodiment of the
occult—something that was of great inter-
est to him. It would be indistinguishable
from magic. And remember, Newton was
a really smart dude.
If something is magic, it is hard to know
its limitations. Suppose we further show
Newton how the device can illuminate the
dark, how it can take photos and movies
and record sound, how it can be used as
a magnifying glass and as a mirror. Then
we show him how it can be used to carry
out arithmetical computations at incredible speed and to many decimal places. We
show it counting the steps he has taken as
he carries it, and show him that he can use
it to talk to people anywhere in the world,
immediately, from right there in the chapel.
What else might Newton conjecture
that the device could do? Prisms work
forever. Would he conjecture that the
iPhone would work forever just as it is,
neglecting to understand that it needs to
be recharged? Recall that we nabbed him
from a time 100 years before the birth of
Michael Faraday, so he lacked a scientific
understanding of electricity. If the iPhone
can be a source of light without fire, could
it perhaps also transmute lead into gold?
This is a problem we all have with
imagined future technology. If it is far
enough away from the technology we have
and understand today, then we do not
know its limitations. And if it becomes
indistinguishable from magic, anything
one says about it is no longer falsifiable.
This is a problem I regularly encounter
when trying to debate with people about
whether we should fear artificial general
intelligence, or AGI—the idea that we will
build autonomous agents that operate
much like beings in the world. I am told
that I do not understand how powerful
AGI will be. That is not an argument. We
have no idea whether it can even exist.
I would like it to exist—this has always
been my own motivation for working in
robotics and AI. But modern-day AGI
research is not doing well at all on either
being general or supporting an independent entity with an ongoing existence. It
mostly seems stuck on the same issues in
reasoning and common sense that AI has
had problems with for at least 50 years.
All the evidence that I see says we have no
real idea yet how to build one. Its properties are completely unknown, so rhetorically it quickly becomes magical, powerful
Nothing in the universe is without
Watch out for arguments about
future technology that is magical. Such
an argument can never be refuted. It is
a faith-based argument, not a scientific
3. Performance versus competence
We all use cues about how people perform
some particular task to estimate how well
they might perform some di;erent task.
In a foreign city we ask a stranger on the
street for directions, and she replies with
confidence and with directions that seem
to make sense, so we figure we can also
ask her about the local system for paying
when you want to take a bus.
Now suppose a person tells us that
a particular photo shows people playing
Frisbee in the park. We naturally assume
that this person can answer questions like
What is the shape of a Frisbee? Roughly