In a keynote speech this morning at the Society for Information Display’s annual Display Week conference in Seattle, Steven Bathiche, the research director of Microsoft’s Applied Sciences Group, demonstrated an immersive computing system that expand on the company’s Surface technology. Surface is a tabletop display that uses a set of four cameras to detect the location of objects placed on its surface, and special software to identify objects. Even with fantastic software, this technology can only do so much.

During his talk, Bathiche played a video that shows what’s possible when this concept is combined with better hardware–some nifty (but sketchily explained) optics and a transparent display. Transparent displays can do more than provide heads-up information while allowing you to see in front of you (for example showing traffic information on a windshield). A transparent display can look back at you. Bathiche’s group has combined a flat lens called a wedge lens with a transparent light-emitting diode display. This system can act as a touch screen; it can also detect gestures made from several feet away.

In video of a demo system where the display is mounted on top of the flat lens, a man walks up to the display and then walks back several feet, while the display shows his image. That image is captured using the lens rather than an external camera. Using this form factor, each hand can be assigned a different function–the left hand draws while the right moves the “paper” on screen. Even when the hands cross, the system keeps track of which hand is which and what it does.

Another system Bathiche showed on video uses a camera and image-processing sofrware to determine two viewers’ positions, and the positions of their eyes, and then processes that information to sequentially and directionally display different images to each viewer. Bathiche said this enables side-by-side, glasses-free 3-D viewing.