Last week Apple released the Magic Mouse, a new computer mouse with a “multitouch” interface that responds to movement of fingertips across its surface in addition to conventional click-and-drag actions. Archrival Microsoft isn’t ready to launch a competing product just yet, but the company does have plans for its own multitouch mice. Earlier this month, researchers presented five prototypes at the User Interface Software and Technology in Victoria, British Columbia, and their work won the symposium’s best paper award.
With a multitouch mouse, a user can, for example, browse through a virtual stack of digital photos by flicking a finger across the mouse’s surface, rotate an image by stroking the mouse, or zoom in on a picture by drawing an arrowhead with a fingertip.
“If the [traditional] mouse pointer is your virtual fingertip, we’re giving you a virtual hand,” says Dan Rosenfeld, a researcher with Microsoft’s Applied Sciences Group in Redmond, WA. There are multitouch surfaces for tabletops, computer monitors, and cellphone screens, he says, but aside from Apple’s new device, “there’s really nothing addressing the kind of tasks that lots of people do all day long, sitting in front of a desk at a computer.”
The first mouse outlined in the Microsoft research paper consists of a piece of clear acrylic lit with infrared light along its edge, where it attaches to a palm rest. Fingertips on the acrylic scatter the light, and an infrared camera captures the light patterns to track the movement of the fingers. The technique, known as frustrated total internal reflection (FTIR), has been used for other multitouch systems before, but this is the first design that also integrates the classic features of a mouse such as an optical sensor underneath and clickable buttons.
Another prototype, the dome-shaped Orb Mouse, also uses an infrared camera and light, but it reflects the light out of its center to make its entire hemisphere touch-sensitive. The dome also acts as a giant click button.
SideMouse, in contrast, positions the palm of the user’s hand on top and projects infrared light out of its side to track the user’s fingers as they move along the table next to the mouse.
The Cap Mouse abandons the infrared scheme altogether, instead tracking finger movements with a grid of capacitive sensors on its surface. Unlike the mice that rely on infrared technology, Cap Mouse isn’t affected by ambient lighting, consumes less power, and offers a less detailed account of finger movements.
Finally, the Articulated Mouse, also known as Arty, features two mini-mouse finger rests connected to the base. Each of the three parts contains an optical sensor for tracking movement, so that Arty can be manipulated by moving the base as well as each of the mini-mice separately.
In each of the five prototypes, although the user still moves the cursor across the screen by moving the entire mouse across the desktop, multitouch functions are accessed by moving individual fingers. Software created by Microsoft lets the user control their computer using these multitouch functions.
Shahram Izadi, a TR35 award winner who worked on Microsoft’s multitouch mice, says there’s still much work to be done on all of the prototypes. In particular, he says, the researchers need to determine the most natural way for users to switch between multitouch capabilities and standard mousing action. Activating the multitouch features with an extra mouse click makes the device slightly more difficult to use, but having those features available all the time means they might be accidentally triggered. “Users wanted to click the device to trigger the multitouch,” Izadi says. “But it’s not the most ergonomic form to click and then gesture.”
Alan Hedge, a professor of ergonomics at Cornell University who has worked on multitouch devices in the past, says multitouch can be useful, but he isn’t sure it’s right for computer mice. “The real benefit of multitouch is when you can take the whole top of a table or desk and use that to drag things around,” he says. “Confining it to the size of a mouse might actually slow you down,” particularly because clicking is so efficient.
Rosenfeld says that while efficiency is important, it’s not the only goal. A good tool, he says, should be both efficient and delightful. Ideally, a device lets you focus on your task and gets the job done, “but it’s also just a really lovely object to use.”
This new data poisoning tool lets artists fight back against generative AI
The tool, called Nightshade, messes up training data in ways that could cause serious damage to image-generating AI models.
Rogue superintelligence and merging with machines: Inside the mind of OpenAI’s chief scientist
An exclusive conversation with Ilya Sutskever on his fears for the future of AI and why they’ve made him change the focus of his life’s work.
The Biggest Questions: What is death?
New neuroscience is challenging our understanding of the dying process—bringing opportunities for the living.
How to fix the internet
If we want online discourse to improve, we need to move beyond the big platforms.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.