Precision Pointing with Fat Fingers
A new technique for touch screens could make them easier to use.
Retrieving the stylus for a personal digital assistant takes time. But for detailed work, a stylus is usually better than a finger. Microsoft researchers believe that they’ve found a better way to activate tiny targets, such as a name on a contact list or a street on a map.
Microsoft’s solution, called Shift, allows users to employ their fingertips to select pixels in a new way. First, the user presses a finger on the screen over the area of interest. Holding down her finger activates the Shift software. A detailed view of the area of interest appears nearby on the screen, in a pop-up window on top of the original image. With slight movements of her finger, the user can guide a pair of crosshairs over her desired target within the pop-up window and then make her selection by lifting her finger off the screen.
“You want to give people the sensation of skill and control,” says Patrick Baudisch, lead researcher on the Shift team. “We wanted it to be transparent and help people get their job done.”
Shift only kicks in when users touch the screen long enough–generally for about a third of a second–for it to know that they need help finding small targets with their finger. As a result, users can seamlessly move from quick finger work on detailed screens like maps and calendars to more careful interaction with their stylus when it’s required.
“It’s important for a device to help people only when they need it,” Baudisch says. “Other researchers have looked at [personal digital assistant] designs that are touch only. But as a result, you can’t sketch, write, or annotate on them anymore, and you are limited to around 15 targets on the screen.” On the other hand, Baudisch notes, it shouldn’t be necessary to always use the stylus. “The trick is to have something that presents itself as a very simple device 90 percent of the time, yet when you need it to, it can become very powerful on demand.”
Researchers at the University of Maryland developed their own approach to the problem in the late 1980s. Their approach, called Offset Cursor, gave users a set of crosshairs just north of their finger to aim with every time they touched the screen. But Offset Cursor never took off. The Microsoft team points to three fundamental flaws with this early system: users are forced to guess where they should place their finger to aim the crosshairs; they are unable to access locations near the screen’s edges; and they always have to select targets with the cursor offset no matter the size of the target.
An experiment conducted by the Microsoft team last summer compared Shift with Offset Cursor and with unassisted touch-screen use. During the study, 12 participants had to point their finger and press on a series of targets as they popped up on the screen. In these trials, the researchers found that Shift outperformed the other methods because it could adapt to different target sizes.
“The biggest benefit of Shift is the simplicity of pressing on the target itself,” says Daniel Vogel, a PhD candidate at the University of Toronto, who assisted Baudisch on the project. “We found a way to help people without them having to think about doing something different than they would normally do with their finger.”
In the experiment, the smallest targets Baudisch and Vogel presented were six-by-six pixels. For even smaller tasks, such as pinpoint drawing, the Shift team created a special application that lets users isolate individual pixels by magnifying the area copied in the pop-up window and increasing the sensitivity of the screen to the users’ movements.
Ultimately, Baudisch believes that Shift could make it easier for users to switch tasks on their mobile devices, without eliminating the stylus altogether. He also hopes to use the same approach to streamline the interactions between phones, notebooks, and wall displays. “In the future, we’ll be able to use very small screens to give us overviews of complex information and then quickly delve deeper into it on larger devices when necessary.”
Baudisch declined to say exactly when Shift will be brought to market. But he is confident it won’t be long.
The AI revolution is here. Will you lead or follow?
Join us at EmTech Digital 2019.