Skip to Content

How to Operate Your Smart Watch with the Same Hand That Wears It

Researchers used a smart watch’s existing sensors to add gesture controls like pinching and waving.

Smart watches aren’t exactly models of efficiency—they require one hand to operate while the other wrist wears it. A team of Carnegie Mellon University researchers figured there had to be a better way to pinch, swipe, and click using just the hand that wears it.

They tapped into the existing smart-watch sensors like gyroscopes and accelerometers and, using machine learning, taught an off-the-shelf Samsung Galaxy Gear smart watch to recognize five different gestures performed by the hand wearing the watch. It included relatively fine gestures like pinch and tap, plus larger hand movements such as rub, squeeze, and wave.

"We wanted to do gestures that are not awkward for people to do while they’re on their smart watch,” says Julian Andres Ramos Rojas, a PhD student in Carnegie Mellon’s Human-Computer Interaction Institute.

Sensors in the smart watch can recognize simple gestures from the hand that wears it.

The research team turned to 10 volunteers to test the accuracy of its system. Overall, it was about 87 percent accurate. Rojas estimates that a commercial version would have to be more than 95 percent accurate.

Smart watches in their modern form are still relatively new, and their makers are experimenting with different input methods. They often involve bulky buttons or lots of tapping or scrolling with the user’s finger.

Gesture recognition would dramatically improve the functionality of smart watches, which have struggled to take off. Tying up both hands to interact with a smart watch often doesn’t make much sense. Someone with dirty hands might not want to touch his screen just to change the music.

“(There) are times when speed, subtlety, hands-free interaction, or pure enjoyment call for gesture control,” according to Stephen Lake, CEO of gesture control armband maker Thalmic Labs. “When we have the opportunity to further immerse ourselves into an experience or provide a shortcut to the alternative, gesture control can prevail.”

Smart-watch makers are in the early stages of adding gesture control. Late last year, Google made it possible to add a few gesture controls to Android smart watches. Users can flick, shake, or lift their wrist to interact. The Carnegie Mellon team’s approach differs in that users can actually make relatively fine finger movements and still have their watch pick them up.

The researchers believe their work could be especially useful in health care, where it could be used by patients with motor neuron diseases. It would also be possible to use the technology to control more than a smart watch; phones, laptops, and virtual- or augmented-reality headsets can also benefit from gesture control.

Rojas says researchers are already looking beyond smart watches for interacting with our devices. Another Carnegie Mellon team is working on turning skin into a touch pad (see “Use Your Arm as a Smart-Watch Touch Pad”). But we aren’t likely to all immediately jump to the most sci-fi solution.

"I don’t think we’re going to settle into a single mode of interaction. Instead, it’s going to be this really nice combination of techniques,” Rojas says.


Keep Reading

Most Popular

10 Breakthrough Technologies 2024

Every year, we look for promising technologies poised to have a real impact on the world. Here are the advances that we think matter most right now.

Scientists are finding signals of long covid in blood. They could lead to new treatments.

Faults in a certain part of the immune system might be at the root of some long covid cases, new research suggests.

AI for everything: 10 Breakthrough Technologies 2024

Generative AI tools like ChatGPT reached mass adoption in record time, and reset the course of an entire industry.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.