Point, Click, and Fire in Virtual Reality—With Just Your Eyes
Kickstarter-funded startup Fove wants to be the first virtual-reality device controlled entirely by vision.
The technology behind Fove could greatly reduce the computational power needed for virtual reality, opening it up to many more types of applications.
It’s testament to the speed at which the current crop of virtual-reality headsets are evolving that newcomer Fove is able to make its competitors seem, if not outdated, then at least behind the curve.
Fove, founded by former Sony Computer Entertainment producer Yuka Kojima and image-processing wunderkind Lochlainn Wilson, makes a device that is the first of its kind to use eye tracking as its primary user interface, following the user’s pupils with infrared light. It’s a simple but science-fiction-esque invention. You become Iron Man, able to navigate digital menu screens merely by allowing your gaze to linger on a particular button, or Cyclops, capable of firing lasers from your eyes to shoot down alien spaceships.
This latter game was explored in an inspiring, if brief, demo at the E3 video-game conference, held in Los Angeles last week: aliens swarm around your head, and you score points for blowing them up with speed and precision. But retina-controlled shooting is perhaps the most primitive use for a technology that has a grander ambition to revolutionize the way virtual-reality programs run.
Wilson, Fove’s CTO, who learned his hardware skills as president of the “hackerspace” HSBNE in Brisbane, Australia, says that foviated rendering is the future of virtual reality. Using this technique, a computer allocates the majority of its rendering resources to the exact point at which the user is looking at any given moment. The rest of the scene, which is in the user’s peripheral vision, is left at a lower resolution (much the way the human eye works in reality).
Foviated rendering is a clever way to circumvent one of VR’s pressing issues: the fact that full immersion in a simulated world requires two high-resolution displays running at an extremely high frame rate. Most makers of VR headsets recommend a refresh rate of 120 frames per second so that the lag you experience when you turn your head is kept to a minimum. This places a huge computational demand on the hardware that’s running the VR headset, a cost that can be significantly offset by Fove’s innovative technique.
A second demo at E3 demonstrated how close Fove’s inventors are to solving the issue. A scene in what looked like the corridor of a Star Trek spaceship sharpened depending on where you focused your attention, with objects in the near distance blurring when you looked toward the rear of the scene and vice versa. Fove has a little way to go before the shift is imperceptible.
The eye-tracking technology will, Wilson claims, solve another looming issue for makers of VR goggles. “From an immersion point of view, if you deal with VR characters that are completely unaware of your gaze, it creates an unnatural, even unnerving experience,” he says. “We can bridge that uncanny barrier.” Indeed, the company’s cofounders hope that in the near future, digital characters will be able to react if you look them directly in the eye; currently, they can only “know” that you are looking in their general direction. A mock-up video shows a man smiling back at a user who looks him in the eye.
The technology could also have applications outside of video games. One of the first tests the team performed explored how users with Asperger’s syndrome and autism interact with others. “We were able to quantify their eye contact and eye avoidance rates,” explains Wilson.
The headset reached its Kickstarter funding target of $250,000 weeks before the end of the campaign, which is due to end in early July. The company plans to sell the finished product for about $500, although its true value may yet be as an influencer rather than a consumer product. Kojima and Wilson own patents on the technology and have been in recent talks with Sony about how it might benefit the Japanese company’s forthcoming Morpheus VR headset.
Regardless of whether a partnership comes to fruition, Fove clearly demonstrates how VR’s leading innovations are currently spread across multiple rival products, yet to settle into a single, ideal device.
Become an MIT Technology Review Insider for in-depth analysis and unparalleled perspective.Subscribe today