Skip to Content

Are Physical Interfaces Superior to Virtual Ones?

Humans have enormous capacity for spatial memory. Why don’t our user interfaces take advantage of that?

Something’s been bothering me ever since I started reading books, especially non-fiction, on my Kindle:

I can’t remember where anything is. Physical books are full of spatial reference points; an especially beloved book is a physical topography in which we develop a vague sense of which chapters contain relevant information; even where, on a page, a particularly striking sentence or diagram lies.

Ebooks have none of these referents. They’re searchable (or at least, some are) which mitigates this issue somewhat. But I’m unlikely to remember that a fact was at “41% through a book” for one simple reason: my hands never got a chance to find out what 41% through a particular ebook feels like.

This isn’t to say that physical books are perfect – perhaps if we read off of giant scrolls laid out across a gymnasium floor, I’d have an even better memory of where I saw a fact: “upper left quadrant, approximately the fourth row…” or something like that. And perhaps some day a virtual interface for reading will give me those kinds of spatial referents.

But in the meantime, millions of years of evolution are going to waste. It’s no secret that mnemonists – the mental athletes of the world of competitive memorization – use tricks like placing facts and sequential information on the walls of mansions they imagine walking through. And why? Because our brains are exquisitely well-tuned to remember where things are. Exactly what you’d expect from a species with a migratory, hunter-gatherer past; a species that re-applied those abilities to the navigation of cities long after it settled into an agricultural pattern.

Here’s another interface that lacks spatial referents: the web. Compare the average river-of-news to these images, of a control board for a soviet nuclear power plant, as captured by the photographer and blogger Ilya Varlamov:

How could anyone make sense of such a layout? But on the other hand, what nuclear power plant’s operation doesn’t have a steep learning curve? At least this way, every day, you come in, everything is where you left it.

Compare that to the few parts of the power plant’s interface that are fully computerized – and therefore terrifying in their changeability.

If you’re used to hunting through piles of browser tabs, alphabetized files or the like, perhaps you can appreciate that the only mechanism for arriving anywhere in this virtual environment is search, i.e. teleportation. This engages our verbal but not our spatial memory – and in some respects, the latter is the stronger of the two.

There is a solution, of course – immersive virtual interfaces with a spatial characteristic. In other words, re-creating these soviet-style interfaces inside a virtual world that you navigate as you would the real, only you’re wearing goggles that are perfectly registered to your immediate environment. It’s the old saw of virtual (or augmented) reality versus the virtual desktop, and we know how well VR has worked out so far.

Still, all the programs we use whose elements don’t change are perhaps the last bastion of sanity we have. I wonder if that’s the reason that we cling to metaphors as tired as the desktop even as the computers we use become ever more capable of much more sophisticated user experiences – at least we know papers go in folders, which live on the desktop, which is after all a fiction in the branching tree file structure of our computer; itself an abstraction laid on top of the quasi-random scattering of bits on our hard drives…

Update - Credit where it’s due, here’s the post that inspired this one, by Mark Changizi at Psychology Today:

The Problem With the Web and E-Books Is That There’s No Space for Them

Follow Mims on Twitter or contact him via email.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

How scientists traced a mysterious covid case back to six toilets

When wastewater surveillance turns into a hunt for a single infected individual, the ethics get tricky.

The problem with plug-in hybrids? Their drivers.

Plug-in hybrids are often sold as a transition to EVs, but new data from Europe shows we’re still underestimating the emissions they produce.

It’s time to retire the term “user”

The proliferation of AI means we need a new word.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.