Tactile instruction is crucial to learning certain tasks, but unlike video and audio, touch is difficult to record and transfer. Now researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and their colleagues have developed a smart glove that can capture, reproduce, and relay touch-based instructions, along with a machine-learning agent that adapts to how different users react to the feedback. 

The system could help teach people physical skills, improve the performance of remote-controlled robots, and help train surgeons, firefighters, and pilots in virtual reality.

ALEX SHIPPS/MIT CSAIL

To make the glove, an embroidery machine stitches tactile sensors and haptic actuators—devices that provide touch-based feedback—into fabric that is cut out to fit a user’s hand measurements. It’s ready to wear in 10 minutes, and the machine-learning model needs just 15 seconds of user data to personalize the feedback.

In experiments, the system helped users learn to play mobile games, and a demonstration showed how it could even teach people to play the piano. The researchers also found that their glove could transfer force sensations to robotic arms, helping them with more delicate grasping tasks.

“This work is the first step to building personalized AI agents that continuously capture data about the user and the environment,” says Wojciech Matusik, SM ’01, PhD ’03, an MIT professor of electrical engineering and computer science and senior author of a paper on the work. “These agents then assist them in performing complex tasks, learning new skills, and promoting better behaviors.”