Intelligent Machines

Microsoft Mulls a Stylus for Any Screen

Adding a camera to a stylus could let it interact with any device—even one without a touch screen.

Microsoft is considering whether to release a stylus that could, after a software upgrade, interact with almost any existing display or device. Researchers at the company’s Silicon Valley site designed the stylus to good internal reviews and are waiting to hear if the company will continue its development with an eye on testing its potential as a product.

While styluses are available that work with any touch-screen device, such as an iPad or iPhone, they are relatively inaccurate. True stylus support requires an extra layer of sensors built into a device’s display, which adds costs. If the new Microsoft stylus concept were to become available, it would allow precise stylus use on any display, even on those that aren’t already touch-sensitive.

The debut of the iPhone and other multi-touch phones and tablets in recent years caused a shift away from stylus input, which became associated with outdated mobile devices, such as the Palm Pilot. More recently, though, several phones have launched with stylus input (see “Review: Galaxy Note”), stylus accessories for the iPad have become popular, and Microsoft has unveiled a future tablet designed around stylus input. The company has also just launched a new version of Windows and of its Office productivity suite with design changes suited to stylus as well as touch input.

Andreas Nowatzyk and colleague Anoop Gupta hit upon the idea of using the grid of pixels that make up a digital display as a navigational system for their backwards-compatible stylus. In their design, a small camera inside the stylus looks down at the display and counts off pixels as they pass by to track its movement. That is fed back to the device via a wireless link, much as a wireless mouse reports its motion to a computer. The way the stylus tracks its motion is similar to the way “smart pens” such as the LiveScribe, a device for aiding note-taking, use a camera to track dots on special paper (see “Taking Apart the LiveScribe Pulse”).

The new Microsoft design has the camera looking out of the side of the stylus body at an angle, so it views the display aslant. That lets it infer the angle it is being held at based on how different pixels are in and out of focus.

However, for the stylus to work, it also needs to know precisely where on the screen it is at any time. The Microsoft researchers’ solution was to have the related software “massage” the color of the blue pixels in a display so that their pattern of brightness encodes their position; the stylus then knows where it is. “Blue is chosen because the human eye doesn’t have many blue cones in the fovea,” the area of the retina used for our central vision, says Nowatzyk.

In his design, the stylus needs to note the average brightness of around five groups of four pixels to learn exactly where it is. It can constantly report that back to the computer, which can update its display and react appropriately. Matched images of a Windows desktop with and without that tweaking of blue pixels shows that it isn’t noticeable.  The software needed to alter a device’s blue pixels to include the location signal could be bundled with its driver, says Nowatzyk.

The design Nowatzyk and colleagues sketched out should be workable on stand-alone and mobile displays, he says, even very high resolution ones on tablets and phones.

However, researchers would need a new type of image sensor to actually test prototypes. A good quality wireless mouse now uses a compact image sensor with a resolution of 30 by 30 pixels. To work, the new stylus design would require one with a resolution of 512 by 512 pixels to see the details as small as a tenth of a millimeter and to capture images at a relatively high rate to track motion smoothly. “The next step [in development] requires making that chip,” says Nowatzyk. “It’s currently being looked at in other groups inside Microsoft.” He says that no decision has yet been made on whether to develop the design into a product or not.

Nowatzyk and colleagues have built a working prototype of a different stylus design, albeit one that would be usable only with devices with special hardware. They added four radio antennae to the frame of a regular LCD display that allow it to be used with a stylus with coils of wire in. The coils distort the radio beacons from the antennae, giving multiple sources of feedback that can be used to track the location and motion of the stylus in three dimensions.

Get stories like this before anyone else with First Look.

Subscribe today
Already a Premium subscriber? Log in.

Uh oh–you've read all of your free articles for this month.

Insider Premium

$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe and become an Insider.

  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look. Exclusive early access to stories.

    Insider Conversations. Join in and ask questions as our editors talk to innovators from around the world.

  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning magazine and daily delivery of The Download, our newsletter of what’s important in technology and innovation.

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.