Imagine walking into a casino to play the one-armed bandits. You’ve heard that one of them pays out more than the others, so your goal is to find out which. But how much of your resources should you pour into exploring the machines before you decide to exploit one of them?
This problem is known as the exploration-exploitation dilemma, and it crops up in all kinds of circumstances, from oil exploration to gambling to business planning. The difficulty, of course, is that the machines are probabilistic, so it is always possible to be tricked into playing a poorly paying machine by a good run that occurs by chance.
Advertisement
There is no shortage of algorithms that can help. These are computer programs of varying degrees of complexity that simulate what is going on and use probability theory to calculate the best next move.
This story is only available to subscribers.
Don’t settle for half the story.
Get paywall-free access to technology news for the here and now.
But now there is another way. Today, Makoto Naruse at the National Institute of Information and Communications Technology in Tokyo, Japan, and pals point out that the laws of quantum mechanics are probabilistic, and so provide a natural environment for decision theory.
And that’s allowed them to build an extraordinary decision-making device. Their new toy is a photon gun and a single photon detector that uses the laws of physics to make decisions, rather than complex algorithms. The device raises the possibility that this kind of photonic intelligence could help make complex decisions.
The theory is straightforward. One strategy to find the best paying one-armed bandit is to start feeding the machines and count how much they pay out each time. Then, as you learn, feed more into the machines that pay better until you find the best.
The single photon decision maker does all this automatically. It consists of a gun that fires photons through a polarizing filter. This filter can be adjusted to polarize the photons horizontally or vertically.
When it is at 45 degrees, however, each photon has an equal chance of being horizontally or vertically polarized. So this set up produces a steady stream of photons polarized either horizontally or vertically with a 50 percent chance of each.
Rotating the filter changes these odds. Moving the filter toward the vertical increases the chances of vertical polarization while reducing the chances of horizontal polarization. So in this setup, more photons will be vertically polarized.
Next, Naruse and co use the polarization of the photons to choose which of two slot machines to play. For example, a vertical photon means a play on the first bandit while a horizontal photon means a play on the second bandit.
Advertisement
The decision-making process comes by creating a feedback loop that changes the orientation of the polarizing filter after each play. So if the first bandit pays out, the filter moves to the vertical making it more likely to be played in future. But if a horizontal photon triggers a payout, the filter moves to the horizontal.
In this way, the photon machine “learns” which machine pays out more often.
What’s interesting about this setup is that it requires no standard computing, no simulation of the system, and no number crunching of probabilities. Instead, the laws of physics make the decision via a simple feedback mechanism.
This is not just theoretical work. Naruse and co have built their own photon decision-maker using a photon gun made from a nitrogen vacancy in nanodiamaond. The photons from this device determine which of two slot machines to play with the results fed back to change the orientation of a polarization filter.
The results are impressive. The single photon decision-maker quickly finds the best paying bandit and even changes its “mind” when Naruse and co change the payout odds of the machines. “We experimentally demonstrate accurate and adaptive decision making,” they say.
That’s interesting work that has important implications for intelligent machines. One of the advantages of this new machine is its simplicity—it requires no standard computing machinery at all. It can also be tiny, operating on a scale of nanometers while using very little power.
But it is conceptually where the new device might have the biggest impact. There is a clear feeling that intelligent machines must rely on electronic computation to make decisions. Naruse and co show that this is not the case. Instead, a new kind of photonic intelligence uses the laws of physics and clever design seem to decide for itself.
