Microsoft Mulls a Stylus for Any Screen

Microsoft is considering whether to release a stylus that could, after a software upgrade, interact with almost any existing display or device. Researchers at the company’s Silicon Valley site designed the stylus to good internal reviews and are waiting to hear if the company will continue its development with an eye on testing its potential as a product.

While styluses are available that work with any touch-screen device, such as an iPad or iPhone, they are relatively inaccurate. True stylus support requires an extra layer of sensors built into a device’s display, which adds costs. If the new Microsoft stylus concept were to become available, it would allow precise stylus use on any display, even on those that aren’t already touch-sensitive.

The debut of the iPhone and other multi-touch phones and tablets in recent years caused a shift away from stylus input, which became associated with outdated mobile devices, such as the Palm Pilot. More recently, though, several phones have launched with stylus input (see “Review: Galaxy Note”), stylus accessories for the iPad have become popular, and Microsoft has unveiled a future tablet designed around stylus input. The company has also just launched a new version of Windows and of its Office productivity suite with design changes suited to stylus as well as touch input.

Andreas Nowatzyk and colleague Anoop Gupta hit upon the idea of using the grid of pixels that make up a digital display as a navigational system for their backwards-compatible stylus. In their design, a small camera inside the stylus looks down at the display and counts off pixels as they pass by to track its movement. That is fed back to the device via a wireless link, much as a wireless mouse reports its motion to a computer. The way the stylus tracks its motion is similar to the way “smart pens” such as the LiveScribe, a device for aiding note-taking, use a camera to track dots on special paper (see “Taking Apart the LiveScribe Pulse”).

The new Microsoft design has the camera looking out of the side of the stylus body at an angle, so it views the display aslant. That lets it infer the angle it is being held at based on how different pixels are in and out of focus.

However, for the stylus to work, it also needs to know precisely where on the screen it is at any time. The Microsoft researchers’ solution was to have the related software “massage” the color of the blue pixels in a display so that their pattern of brightness encodes their position; the stylus then knows where it is. “Blue is chosen because the human eye doesn’t have many blue cones in the fovea,” the area of the retina used for our central vision, says Nowatzyk.

In his design, the stylus needs to note the average brightness of around five groups of four pixels to learn exactly where it is. It can constantly report that back to the computer, which can update its display and react appropriately. Matched images of a Windows desktop with and without that tweaking of blue pixels shows that it isn’t noticeable.  The software needed to alter a device’s blue pixels to include the location signal could be bundled with its driver, says Nowatzyk.

The design Nowatzyk and colleagues sketched out should be workable on stand-alone and mobile displays, he says, even very high resolution ones on tablets and phones.

However, researchers would need a new type of image sensor to actually test prototypes. A good quality wireless mouse now uses a compact image sensor with a resolution of 30 by 30 pixels. To work, the new stylus design would require one with a resolution of 512 by 512 pixels to see the details as small as a tenth of a millimeter and to capture images at a relatively high rate to track motion smoothly. “The next step [in development] requires making that chip,” says Nowatzyk. “It’s currently being looked at in other groups inside Microsoft.” He says that no decision has yet been made on whether to develop the design into a product or not.

Nowatzyk and colleagues have built a working prototype of a different stylus design, albeit one that would be usable only with devices with special hardware. They added four radio antennae to the frame of a regular LCD display that allow it to be used with a stylus with coils of wire in. The coils distort the radio beacons from the antennae, giving multiple sources of feedback that can be used to track the location and motion of the stylus in three dimensions.