Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

John Pavlus

A View from John Pavlus

Think Gestural Interactions Suck? Design Your Own

Creating your own gestural patterns could make these interactions easier to remember, researchers say.

  • April 26, 2013

Gestural interactions may be the new hotness in interface design, but there's something about these hand-wavey children of Minority Report that's hard to get exactly right. It can feel like learning a pidgin sign language: instead of pressing, clicking, swiping or tapping a physical surface, you're “speaking” to the computer by making (often arbitrary-seeming) symbols with your hands. Computer scientists at the University of St. Andrews (who've been making a lot of interesting moves in UI design lately) think that letting users design their own gestures will make it much easier to remember them.

The idea certainly makes intuitive sense. Every day I watch my two-year-old protest “I do it!” when I try to show her some new physical skill. She wants to do it in her own way, so it sticks. And when it comes to learning unfamiliar gestural UIs—with no obvious physical affordances to exploit, and few instances of useful previous experience to call upon for reference—we adults aren’t much better off than toddlers. So why not design these systems to let each user “speak” to them in their own way, and streamline the learning process?

The researchers discovered something interesting about the pain-in-the-ass-ness of gestural UIs: the difficulty wasn’t in remembering exactly how to do the gestures (“do I swipe with two fingers or three?”) but in so-called “association errors” (“what does swiping with two fingers do, again?”) . Letting users define the gestures themselves reduced these association errors. If decide, “OK, swiping upwards with two fingers means ‘undo’”, I’m more likely to remember the interaction and correctly perform it.

Still, not everyone is a professional interaction designer. Supposedly “simple” interactions like “pinch to zoom” on iOS only emerged from doing intense research on many gestural variations. DIY gestures, created on the fly by amateurs, could feel simple and easy the first few times and then turn annoying or even physically harmful with repeated use. If you give yourself some gestural-UI version of carpal tunnel, who are you going to sue when you’re the person who created the harmful pattern in the first place?

OK, I kid. And you could always just change any gestural pattern that started to bug you. Asking users to assume the cognitive load not just of learning these new interactions, but also designing them, seems like it could backfire—but it also might have unintended benefits as well. Perhaps by “crowdsourcing” gestural-interaction patterns in this way, we’d be able to converge upon useful conventions for gestural UIs in general—just as Twitter users invented “RT” and “@”-messaging conventions in a bottom-up manner.

In any case, gestural interfaces have a long way to go before they feel as legible and evident as touchscreens, keyboards and mice. That old cliché—“I hear and I forget; I see and I remember; I do and I understand”—could help point the way forward.

Cut off? Read unlimited articles today.

Become an Insider
Already an Insider? Log in.

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

    Bimonthly digital/PDF edition

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special interest publications

    Discount to MIT Technology Review events

    Special discounts to select partner offerings

    Ad-free web experience

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.