Reinforcement learning is a way of
making a computer learn through experience to make a series of decisions that
yield positive outcomes—even without
any prior knowledge of how its actions
will affect its immediate environment. A
software-based tutor, for example, would
alter its activities in response to how students perform on tests after using it.
If we hope to create artificial teaching
agents using reinforcement learning, we’ll
need algorithms that are “data smart.” We
might gather data from online educational systems and use it to help the agent
estimate the effectiveness of different
teaching approaches. When a student logs
in, should the system provide him with a
problem to solve? Or would starting with
an explanatory video be better? The data
can help it decide.
But in some cases there’s not enough
data, or not the right kind of data, which
makes it challenging to develop systems
that make good decisions. It would be nice
if we could create a system that didn’t
need so much data in the first place. And
that’s exactly what my group is working
on—we’re developing reinforcement-learning algorithms and statistical techniques to allow computers to develop
good suggestions while using less data.
We still have a lot of work to do, but we’re
tightening the gap between theory and
In the end, we shouldn’t leave it all to
the computers. So-called “human-in-the-
loop” reinforcement learning can acceler-
ate the process, allowing algorithms to
“reason” about their own limited perfor-
mance and reach out to humans for help
when they need, for example, to expand
the set of possible decisions. My group
and our collaborators at the University of
Washington are now testing algorithms
for a tutoring system that can tell if its
current curriculum isn’t enabling all stu-
dents to learn well, and then asks people
to add new hints to the system. Such
human-computer collaborations could
help students to learn using approaches
we can’t yet imagine. This vision of rein-
forcement learning has artificially intel-
ligent agents redefining what outstanding
human performance looks like—and
enabling all of us to achieve it.
Emma Brunskill is an assistant professor of
computer science at Stanford University.
Cash is passé. But digital money makes you
easier to track.
One great challenge facing society is
where to draw the line between an individual’s right to privacy and the government’s right to tax, regulate, and enforce
the law. Few areas illustrate this problem
as well as the way we spend our money.
Our transactions are increasingly
digital (and thus easily tracked), and in
places like China many companies are
adopting biometrics (like fingerprints or
eye scans) to verify who we are (see “ 10
Breakthrough Technologies: Paying with
Your Face,” page 72). In India, the government has taken biometric data from 1. 1
billion people. But these developments
alone don’t give us a good answer to the
question of what we should do with good
old-fashioned paper currency.
The demand for cash has dwindled in
the legal, tax-compliant economy, but the
underground economy uses it as much as
ever. Incredibly, given that 95 percent of
Americans report that they’ve never held a
$100 bill (the rest say they hold one occa-
sionally), there are 34 $100 bills floating
around for every man, woman, and child
in the country. Similar figures hold for
big bills in other advanced economies.
What are they being used for? The evi-
dence seems clear: a huge amount of the
world’s cash supply is used to facilitate tax
evasion, crime, and corruption.
Given that, going to a completely
cashless society might appear to be a
great idea. But it’s not so simple. Ordinary people rely on cash to protect their
privacy, and cash still comes in handy
during prolonged power outages. One
way to deal with the problem might be to
phase out large-denomination notes such
as the U.S. $100 bill, the 500-euro note,
and the 1,000 Swiss franc note—anything
worth $50 or more. (Although I wouldn’t
suggest following the example of India,
which recently phased out 85 percent of
its currency supply almost overnight. This
move had disastrous effects that could
have been avoided if the change had been
made more gradually, over a period of
We shouldn’t get rid of cash entirely.
Even with the rapid evolution of new technologies such as Bitcoin, paper currency
provides ordinary citizens with a critical
safety valve. The government’s objective
in regulating new or old transaction technologies should be to discourage wholesale tax evasion and crime while leaving
ordinary people a margin of privacy and
convenience in their ordinary lives. Putting the economy on a cash diet is a good
idea. Literally going cashless is not.
Kenneth Rogoff is a professor of economics
at Harvard University and the author of
The Curse of Cash.
There are 34 $100 bills floating around for every man,
woman, and child in the country. A huge amount of that cash
is being used to facilitate corruption and tax evasion.