A View from Christopher Mims
Are Physical Interfaces Superior to Virtual Ones?
Humans have enormous capacity for spatial memory. Why don’t our user interfaces take advantage of that?
Something’s been bothering me ever since I started reading books, especially non-fiction, on my Kindle:
I can’t remember where anything is. Physical books are full of spatial reference points; an especially beloved book is a physical topography in which we develop a vague sense of which chapters contain relevant information; even where, on a page, a particularly striking sentence or diagram lies.
Ebooks have none of these referents. They’re searchable (or at least, some are) which mitigates this issue somewhat. But I’m unlikely to remember that a fact was at “41% through a book” for one simple reason: my hands never got a chance to find out what 41% through a particular ebook feels like.
This isn’t to say that physical books are perfect – perhaps if we read off of giant scrolls laid out across a gymnasium floor, I’d have an even better memory of where I saw a fact: “upper left quadrant, approximately the fourth row…” or something like that. And perhaps some day a virtual interface for reading will give me those kinds of spatial referents.
But in the meantime, millions of years of evolution are going to waste. It’s no secret that mnemonists – the mental athletes of the world of competitive memorization – use tricks like placing facts and sequential information on the walls of mansions they imagine walking through. And why? Because our brains are exquisitely well-tuned to remember where things are. Exactly what you’d expect from a species with a migratory, hunter-gatherer past; a species that re-applied those abilities to the navigation of cities long after it settled into an agricultural pattern.
Here’s another interface that lacks spatial referents: the web. Compare the average river-of-news to these images, of a control board for a soviet nuclear power plant, as captured by the photographer and blogger Ilya Varlamov:
How could anyone make sense of such a layout? But on the other hand, what nuclear power plant’s operation doesn’t have a steep learning curve? At least this way, every day, you come in, everything is where you left it.
Compare that to the few parts of the power plant’s interface that are fully computerized – and therefore terrifying in their changeability.
If you’re used to hunting through piles of browser tabs, alphabetized files or the like, perhaps you can appreciate that the only mechanism for arriving anywhere in this virtual environment is search, i.e. teleportation. This engages our verbal but not our spatial memory – and in some respects, the latter is the stronger of the two.
There is a solution, of course – immersive virtual interfaces with a spatial characteristic. In other words, re-creating these soviet-style interfaces inside a virtual world that you navigate as you would the real, only you’re wearing goggles that are perfectly registered to your immediate environment. It’s the old saw of virtual (or augmented) reality versus the virtual desktop, and we know how well VR has worked out so far.
Still, all the programs we use whose elements don’t change are perhaps the last bastion of sanity we have. I wonder if that’s the reason that we cling to metaphors as tired as the desktop even as the computers we use become ever more capable of much more sophisticated user experiences – at least we know papers go in folders, which live on the desktop, which is after all a fiction in the branching tree file structure of our computer; itself an abstraction laid on top of the quasi-random scattering of bits on our hard drives…
Update - Credit where it’s due, here’s the post that inspired this one, by Mark Changizi at Psychology Today:
AI is here. Will you lead or follow?
Join us at EmTech Digital 2019.