We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

John Pavlus

A View from John Pavlus

Samsung’s “Eye Scroll” Hints at Post-Interactive Interfaces

What if the future of human-computer interaction had a lot less interaction in it?

  • March 4, 2013

The best essay on human-computer interaction I’ve read this year was a fake news piece in The Onion. Its title: “Internet Users Demand Less Interactivity.” What if people just “want to visit websites and look at them”? What if “using” a piece of software is simply not what we want to do with it, most of the time? 

I couldn’t help but think of that Onion article when I came across the news that Samsung’s latest phablet will use eye-tracking software to scroll its display for you. It’s a fine idea: after all, how much of our interaction with our smartphones consists of merely dragging the next tiny “page” of content up into view? Not that it’s a terribly taxing thing to do. But it’s not a very high-value physical interaction to repeat hundreds of times a day, either. Why not automate it?

Bret Victor, an ex-Apple interface designer, wrote a serious treatise on the idea of post-interactive software interfaces way back in 2006 – before glass-slab smartphones even existed. Its central argument is, essentially, exactly the same as that Onion headline: interaction is not most what software is actually for. Nine times out of ten, we engage with a piece of software primarily because we want to “read” it like a text, not manipulate it like an object. The latter is something we’re forced to do in order to achieve the former. But why should it be this way? The beauty of software, according to Victor, is that it’s really just graphic (or typographic) design rendered in “magic ink”: it can rearrange itself into exactly the right pattern for exactly the right context, from moment to moment. 

Of course, in order for software to fully exploit its “magic ink” potential, it has to be able to accurately sense our intent. Cheap, ubiquitous sensors and machine-learning algorithms (like the eye-tracking technology in Samsung’s new phone) make this possible. To be fair, it’s not exactly removing the interaction from software. Instead, it’s submerging it: treating it as noise and complication best abstracted away from the user’s direct attention. 

That’s the best-case scenario, of course. In practice, “interaction-less” software interfaces are likely to introduce their own annoying cognitive loads, simply because they won’t be smart enough to accurately anticipate our intent 100% of the time. To take the example of Samsung’s phone: how is the eye-tracking software going to know, perfectly, when certain eye movements near the bottom of the screen mean “advance the page now, please” while others may just be semi-random saccades, or some other, subtler sort of attentional behavior (perhaps you might be re-reading a certain word or phrase to savor or study it, and you don’t want to advance the page)?

What Samsung’s eye-tracking feature sounds like isn’t really “post-interactive” software behavior at all – instead, it’s simply replacing one kind of manipulation with another. Instead of dragging your finger (or pressing a button) to advance the page, you direct your gaze to a specific place in a specific way. The software isn’t really acting like “magic ink” that can anticipate your intent; it’s just making you issue the same old UI-manipulation commands with your eyes instead of your hand (or mouse). 

But your eyes are not hands. You use them to sense, not act. Software will have to get a heck of a lot more magical before it can really act like Victor’s magic ink. Until then, jabbing, pushing and poking at our software – er, interacting with it – will probably still be a necessary evil. 

Keep up with the latest in intelligent machines at EmTech Digital.

The Countdown has begun.
March 25-26, 2019
San Francisco, CA

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Print + All Access Digital.
  • Print + All Access Digital {! insider.prices.print_digital !}*

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivery each weekday to your inbox

    The MIT Technology Review App

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.