Skip to Content

How to Operate Your Smart Watch with the Same Hand That Wears It

Researchers used a smart watch’s existing sensors to add gesture controls like pinching and waving.

Smart watches aren’t exactly models of efficiency—they require one hand to operate while the other wrist wears it. A team of Carnegie Mellon University researchers figured there had to be a better way to pinch, swipe, and click using just the hand that wears it.

They tapped into the existing smart-watch sensors like gyroscopes and accelerometers and, using machine learning, taught an off-the-shelf Samsung Galaxy Gear smart watch to recognize five different gestures performed by the hand wearing the watch. It included relatively fine gestures like pinch and tap, plus larger hand movements such as rub, squeeze, and wave.

"We wanted to do gestures that are not awkward for people to do while they’re on their smart watch,” says Julian Andres Ramos Rojas, a PhD student in Carnegie Mellon’s Human-Computer Interaction Institute.

Sensors in the smart watch can recognize simple gestures from the hand that wears it.

The research team turned to 10 volunteers to test the accuracy of its system. Overall, it was about 87 percent accurate. Rojas estimates that a commercial version would have to be more than 95 percent accurate.

Smart watches in their modern form are still relatively new, and their makers are experimenting with different input methods. They often involve bulky buttons or lots of tapping or scrolling with the user’s finger.

Gesture recognition would dramatically improve the functionality of smart watches, which have struggled to take off. Tying up both hands to interact with a smart watch often doesn’t make much sense. Someone with dirty hands might not want to touch his screen just to change the music.

“(There) are times when speed, subtlety, hands-free interaction, or pure enjoyment call for gesture control,” according to Stephen Lake, CEO of gesture control armband maker Thalmic Labs. “When we have the opportunity to further immerse ourselves into an experience or provide a shortcut to the alternative, gesture control can prevail.”

Smart-watch makers are in the early stages of adding gesture control. Late last year, Google made it possible to add a few gesture controls to Android smart watches. Users can flick, shake, or lift their wrist to interact. The Carnegie Mellon team’s approach differs in that users can actually make relatively fine finger movements and still have their watch pick them up.

The researchers believe their work could be especially useful in health care, where it could be used by patients with motor neuron diseases. It would also be possible to use the technology to control more than a smart watch; phones, laptops, and virtual- or augmented-reality headsets can also benefit from gesture control.

Rojas says researchers are already looking beyond smart watches for interacting with our devices. Another Carnegie Mellon team is working on turning skin into a touch pad (see “Use Your Arm as a Smart-Watch Touch Pad”). But we aren’t likely to all immediately jump to the most sci-fi solution.

"I don’t think we’re going to settle into a single mode of interaction. Instead, it’s going to be this really nice combination of techniques,” Rojas says.


Keep Reading

Most Popular

Geoffrey Hinton tells us why he’s now scared of the tech he helped build

“I have suddenly switched my views on whether these things are going to be more intelligent than us.”

Meet the people who use Notion to plan their whole lives

The workplace tool’s appeal extends far beyond organizing work projects. Many users find it’s just as useful for managing their free time.

Learning to code isn’t enough

Historically, learn-to-code efforts have provided opportunities for the few, but new efforts are aiming to be inclusive.

Deep learning pioneer Geoffrey Hinton has quit Google

Hinton will be speaking at EmTech Digital on Wednesday.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.