Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

A View from Erica Naone

New Computer Interface Goes Beyond Just Touch

Manual Deskterity combines a touch with the trusty pen.

  • April 12, 2010
Touch screen interfaces may be trendy in gadget design, but that doesn’t mean they do everything elegantly. The finger is simply too blunt for many tasks. A new interface, called Manual Deskterity, attempts to combine the strengths of touch interaction with the precision of a pen.

“Everything, including touch, is best for something and worse for something else,” says Ken Hinckley, a research scientist at Microsoft who is involved with the project, which will be presented this week at the ACM Conference on Human Factors in Computing Systems (CHI) in Atlanta.

The prototype for Manual Deskterity is a drafting application built for the Microsoft Surface, a tabletop touchscreen. Users can perform typical touch actions, such as zooming in and out and manipulating images, but they can also use a pen to draw or annotate those images.

The interface’s most interesting features come out when the two types of interaction are combined. For example, a user can copy an object by holding it with one hand and then dragging the pen across the image, “peeling” off a new image that can be placed elsewhere on the screen. By combining pen and hand, users get access to features such as an exacto knife, a rubber stamp, and brush painting.

Hinckley says the researchers videotaped users working on visual projects with sketchbooks, scissors, glue, and other typical physical art supplies. They noticed that people tended to hold an image with one hand while making notes about it or doing other work related to it with the other. The researchers decided to incorporate this in their interface–touching an object onscreen with a free hand indicates that the actions performed with the pen relate to that object.

Hinckley acknowledges that the interface includes a lot of tricks that users need to learn. But he thinks this is true of most interfaces. “This idea that people just walk up with an expectation of how a [natural user interface] should work is a myth,” he says.

Hinckley believes that natural user interfaces can ease the learning process by engaging muscle memory, rather than forcing users to memorizes sequences of commands or the layout of menus. If the work is successful, Hinckley says it will show how different sorts of input can be used in combination.

Hinckley also thinks it’s a mistake to focus on devices that work with touch input alone. He says, “The question is not, ‘How do I design for touch?’ or ‘How do I design for pen?’ We should be asking, ‘What is the correct division of labor in the interface for pen and touch interactions such that they complement one another?’”

The researchers plan to follow up by adapting their interface to work on mobile devices.

Want to go ad free? No ad blockers needed.

Become an Insider
Already an Insider? Log in.
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to MIT Technology Review.
  • Print + All Access Digital {! insider.prices.print_digital !}* Best Value

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivered daily

  • All Access Digital {! insider.prices.digital !}*

    {! insider.display.menuOptionsLabel !}

    The digital magazine, plus unlimited site access, our online archive, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    Digital magazine (6 bi-monthly issues)

    Access to entire PDF magazine archive dating back to 1899

    The Download: newsletter delivered daily

  • Print Subscription {! insider.prices.print_only !}*

    {! insider.display.menuOptionsLabel !}

    Six print issues per year plus The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Print magazine (6 bi-monthly issues)

    The Download: newsletter delivered daily

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.