We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Intelligent Machines

Microsoft Mulls a Stylus for Any Screen

Adding a camera to a stylus could let it interact with any device—even one without a touch screen.

Microsoft is considering whether to release a stylus that could, after a software upgrade, interact with almost any existing display or device. Researchers at the company’s Silicon Valley site designed the stylus to good internal reviews and are waiting to hear if the company will continue its development with an eye on testing its potential as a product.

While styluses are available that work with any touch-screen device, such as an iPad or iPhone, they are relatively inaccurate. True stylus support requires an extra layer of sensors built into a device’s display, which adds costs. If the new Microsoft stylus concept were to become available, it would allow precise stylus use on any display, even on those that aren’t already touch-sensitive.

The debut of the iPhone and other multi-touch phones and tablets in recent years caused a shift away from stylus input, which became associated with outdated mobile devices, such as the Palm Pilot. More recently, though, several phones have launched with stylus input (see “Review: Galaxy Note”), stylus accessories for the iPad have become popular, and Microsoft has unveiled a future tablet designed around stylus input. The company has also just launched a new version of Windows and of its Office productivity suite with design changes suited to stylus as well as touch input.

Andreas Nowatzyk and colleague Anoop Gupta hit upon the idea of using the grid of pixels that make up a digital display as a navigational system for their backwards-compatible stylus. In their design, a small camera inside the stylus looks down at the display and counts off pixels as they pass by to track its movement. That is fed back to the device via a wireless link, much as a wireless mouse reports its motion to a computer. The way the stylus tracks its motion is similar to the way “smart pens” such as the LiveScribe, a device for aiding note-taking, use a camera to track dots on special paper (see “Taking Apart the LiveScribe Pulse”).

The new Microsoft design has the camera looking out of the side of the stylus body at an angle, so it views the display aslant. That lets it infer the angle it is being held at based on how different pixels are in and out of focus.

However, for the stylus to work, it also needs to know precisely where on the screen it is at any time. The Microsoft researchers’ solution was to have the related software “massage” the color of the blue pixels in a display so that their pattern of brightness encodes their position; the stylus then knows where it is. “Blue is chosen because the human eye doesn’t have many blue cones in the fovea,” the area of the retina used for our central vision, says Nowatzyk.

In his design, the stylus needs to note the average brightness of around five groups of four pixels to learn exactly where it is. It can constantly report that back to the computer, which can update its display and react appropriately. Matched images of a Windows desktop with and without that tweaking of blue pixels shows that it isn’t noticeable.  The software needed to alter a device’s blue pixels to include the location signal could be bundled with its driver, says Nowatzyk.

The design Nowatzyk and colleagues sketched out should be workable on stand-alone and mobile displays, he says, even very high resolution ones on tablets and phones.

However, researchers would need a new type of image sensor to actually test prototypes. A good quality wireless mouse now uses a compact image sensor with a resolution of 30 by 30 pixels. To work, the new stylus design would require one with a resolution of 512 by 512 pixels to see the details as small as a tenth of a millimeter and to capture images at a relatively high rate to track motion smoothly. “The next step [in development] requires making that chip,” says Nowatzyk. “It’s currently being looked at in other groups inside Microsoft.” He says that no decision has yet been made on whether to develop the design into a product or not.

Nowatzyk and colleagues have built a working prototype of a different stylus design, albeit one that would be usable only with devices with special hardware. They added four radio antennae to the frame of a regular LCD display that allow it to be used with a stylus with coils of wire in. The coils distort the radio beacons from the antennae, giving multiple sources of feedback that can be used to track the location and motion of the stylus in three dimensions.

Hear more from Microsoft at EmTech Digital.

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    Print + Digital Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

    Technology Review PDF magazine archive, including articles, images, and covers dating back to 1899

    10% Discount to MIT Technology Review events and MIT Press

    Ad-free website experience

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.