In an effort to make eye tracking cheap, compact, and accurate enough to be included in smartphones, a group of researchers is crowdsourcing the collection of gaze information and using it to teach mobile software how to figure out where you’re looking in real time.
Researchers at MIT, the University of Georgia, and Germany’s Max Planck Institute for Informatics are working on the project, and they say that so far they’ve been able to train software to identify where a person is looking with an accuracy of about a centimeter on a mobile phone and 1.7 centimeters on a tablet.
This isn’t all that accurate, especially if you consider the overall size of your smartphone screen. It’s still not exact enough to use for consumer applications, says Aditya Khosla, a graduate student at MIT and coauthor of a paper on the work that was presented at a computer vision conference this week.
But he believes the system’s accuracy will improve with more data. If it does, it could make eye tracking a lot more widespread and useful; typically, it’s been expensive and has required hardware that has made it tricky to add the capability to gadgets like phones and tablets. And the technology could be helpful as a way to let you play games or navigate your smartphone without having to tap or swipe.
The researchers started out by building an iPhone app called GazeCapture that gathered data about how people look at their phones in different environments outside the confines of a lab. Users’ gaze was recorded with the phone’s front camera as they were shown pulsating dots on a smartphone screen. To make sure they were paying attention, they were then shown a dot with an “L” or “R” inside it, and they had to tap the left or ride side of the screen in response.
GazeCapture information was then used to train software called iTracker, which can also run on an iPhone. The handset’s camera captures your face, and the software considers factors like the position and direction of your head and eyes to figure out where your gaze is focused on the screen.
About 1,500 people have used the GazeCapture app so far, Khosla says, and he thinks that if the researchers can get data from 10,000 people they’ll be able to reduce iTracker’s error rate to half a centimeter, which should be good enough for a range of eye-tracking applications.
Andrew Duchowski, a professor at Clemson University who studies eye tracking, thinks iTracker could be “hugely” useful if the researchers can get it working well on mobile devices, though he cautions that it will also need to work quickly and not consume too much battery life.
He doesn’t think it’s possible to get pixel-level accuracy from it, but he says “it could still be pretty good.”
How AI is reinventing what computers are
Three key ways artificial intelligence is changing what it means to compute.
These weird virtual creatures evolve their bodies to solve problems
They show how intelligence and body plans are closely linked—and could unlock AI for robots.
Our brains exist in a state of “controlled hallucination”
Three new books lay bare the weirdness of how our brains process the world around us.
We reviewed three at-home covid tests. The results were mixed.
Over-the-counter coronavirus tests are finally available in the US. Some are more accurate and easier to use than others.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.