Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Connectivity

Electrodes for Your Face Bring Your Emotions to Augmented and Virtual Reality

Tracking facial expressions can make VR and AR more realistic, but the technology is still bulky.

MindMaze’s Mask device includes a foam face pad embedded with electrodes that can capture facial muscle movements.

Tej Tadi is really excited about the foam face pad that sits inside a virtual-reality headset on a desk in his office.

It’s the kind of mundane piece of cushioning normally found on many kinds of head-mounted displays, but the one his startup, MindMaze, has is different. It’s embedded with electrodes so it can pick up the electrical signals associated with your facial muscle movements, and it’s connected to a computer where software analyzes these signals to reproduce your facial expressions on an on-screen avatar.

Called Mask, Tadi sees it as a way to bring natural-looking grimaces, smiles, and eyebrow raises to virtual characters without adding much bulk to headsets. Making it easier for users to express emotions—and interact with each other—in virtual reality could encourage more people to try it out, he thinks, and make it more effective.

Would the addition of expression-tracking make you more interested in trying virtual reality?

Tell us in the comments.

“The only way to do it is to bring emotions back into the game,” he says. “That makes us human, right? The non-verbal cues.”

For now, virtual and augmented reality headsets are still a tough sell to most consumers for many reasons: they’re annoying to wear, don’t yet have many clear practical applications, and can feel isolating, to name just a few.

Switzerland-based MindMaze, which has raised $100 million, already offers virtual-reality hardware that combines features like games, motion tracking, and brain-signal monitoring to help rehabilitate stroke victims; it’s currently used in several dozen hospitals in Europe.

Mask builds on the company’s existing work, Tadi says, and can currently recognize 10 different expressions, including winking, smiling, smirking, grimacing, and eyebrow-raising. And with a microphone attached to it, it can mimic the wearer’s mouth while they’re speaking, too.

Tadi expects such a gadget will be available to consumers later this year, either as a product from MindMaze itself or from a headset maker.

As I watched a member of the MindMaze team try out a prototype of the device, it seemed to work quite well: with an OSVR virtual-reality headset outfitted with MindMaze’s technology, he made a variety of expressions that a cartoony male character also made on a desktop display. The speaking mimicry didn’t seem much better than basic “talking” motions I’ve seen on virtual characters’ mouths, however.

There was no need to calibrate the headset for me to try it, but it had issues recognizing some of my facial expressions. Tadi and his team explained that this may have been due to stray hairs getting in the way of one of the electrodes.  

And it’s still unwieldy and messy: the electrode-laden foam insert was connected to some wires and electronics as well as to a computer, and a gel-swabbed electrode was connected to each of my earlobes to serve as a reference.

Tadi says the ear electrodes could be replaced by dry ones that are embedded in headphones connected to the headset, and that the electronics could be whittled down. But the fact remains that the feature would be adding even more baggage to headsets that are already criticized for their size—it remains to be seen whether the addition of emotional expression can outweigh that issue.

Hear more about augmented reality from the experts at the EmTech Digital Conference, March 26-27, 2018 in San Francisco.

Learn more and register

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Connectivity

What it means to be constantly connected with each other and vast sources of information.

Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

    Bimonthly digital/PDF edition

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special interest publications

    Discount to MIT Technology Review events

    Special discounts to select partner offerings

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning print magazine, unlimited online access plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

  • Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.