Microsoft has joined the wave of companies betting that 3-D is the next big thing for computing. At a recent talk at MIT, chief research and strategy officer Craig Mundie said he sees the technology as an innovation that “will get people out of treating a computer as a tool” and into treating the device as a natural extension of how they interact with the world around them. Microsoft plans to introduce consumers to the change through its gaming products, but Mundie outlined a vision that would eventually have people shopping and searching in 3-D as well.
The combination of better chips, better displays, and better sensors, Mundie said, is finally making it possible to move computing from today’s graphical user interfaces to the “natural user interface,” by allowing people to interact with 3-D content through the gestures they normally use. Today’s interfaces require users to learn about menu bars and double-clicks, but Mundie believes natural user interfaces, which work through gesture and voice, will be faster and easier to learn, and will prove more flexible in the long run.
Mundie also argued that natural user interfaces would reduce the mental effort required for people to operate software. Even people who are good at using controllers, keyboards, and mouses might find that a natural interface frees up attention and concentration so that they can focus better on the task at hand, he said. He believes that natural interfaces will make it easier to introduce software to people unfamiliar with computers, as well as make software generally easier to use, and therefore more attractive to consumers.
He also noted that today many programs come with what is essentially “an application-specific prosthetic”–for example, some driving games come with a steering-wheel device. Natural user interfaces may require some peripherals, such as depth-sensing cameras that can detect users’ movements, but Mundie sees these as ultimately having broader purpose than most of today’s devices.
The first step in this strategy, Mundie said, is Microsoft’s release next month of the Kinect sensor for the Xbox 360 gaming console; Kinect incorporates a depth-sensing camera and voice recognition and will cost about $150. It will allow users to play games by gesturing, without the need for a controller or additional equipment. This opens the way to 3-D interaction with games that Mundie hopes will lead to broader use of 3-D displays.
Mundie demonstrated how Kinect would allow a user to interact with 3-D game content through hand gestures, virtually picking up clues to examine them or show them to friends. “We’re trying to create a genre of games where you don’t have to think about how what you would do naturally would map to the controls,” Mundie said.
He also showed a concept video for a real-time 3-D multiplayer game called “The Spy from the 2080s” that included a TV show and a game that players could interact with using multiple devices. For example, they might watch an episode in 3-D on TV, then log in through a gaming console to work with friends to solve clues from the show. Mobile devices might provide additional updates. In the video, the outcome of gameplay even influenced the course of the TV show.