Skip to Content

Conjuring Music, Sans Instruments

At the TEDGlobal Conference, Imogen Heap debuts futuristic gloves that let her compose, arrange, and perform music on the fly.
July 31, 2011

I’ve never conducted an orchestra before, but I imagine it must bring with it an enormous sense of power. With the mere waving of your arms, a symphony pours forth. The tempo, the rhythm, the dynamics of the music are all conjured with the motions of your body – and you’re not even holding an instrument.

Imogen Heap, the Grammy Award-winning musician, must be feeling something similar, after debuting a pair of musical gloves at the recent TEDGlobal conference in Edinburgh. With a team of sound engineers at Fifth Dimension Technologies and x-IO, Heap contributed to the development of technology-laden gloves that allow her to compose, arrange, and perform music.

It all sounds something like that elusive (and over-referenced) “Minority Report interface,” only applied to music. Sadly, the TED video doesn’t appear to be live yet, but Mashable’s description gives a taste of how it works: “Those movements include, for example, the ability to record a loop by opening her hand, filtering sound by bringing her hands together and panning by pointing in the desired direction. Volume can also be manipulated with some fitting gestures — a ‘shh’ movement initiates quiet mode and a horn sign prompts ‘rock out mode.’” Heap has said that she’d like to start incorporating the gloves into her conferences, and someday might like to build an entire hour-long concert “walking on stage with nothing but the gloves.”

Heap got the idea for the gloves after seeing a similar pair developed by an MIT researcher named Elly Jessop (a member of the “Opera of the Future” research group there). As Tom Mitchell, a University of West England in Bristol researcher involved in the project, tells me in an e-mail, “Imogen subsequently started to dream about a system that would enable her to perform an entire show live without having to disengage from the audience to perform unknown interactions with electronic equipment.” (Mitchell’s collaboration with Heap is slightly longstanding.)

That notion–that the gestures that control the music should themselves become part of the performance–was at the heart of the project. Continues Mitchell: “One of the main challenges and successes of the work is the emphasis on performance which preceded any developmental work. Imogen has been there every step of the way to discuss the design of the gestures and their mapping onto audio processes.”

Mitchell cautions that gloves like Heap’s probably won’t become widespread anytime too soon. “[T]he main factor here is the availability, convenience and cost of the required technology,” he says. “It’s not convenient to have to set up external tracking equipment like cameras whenever you want to do something.” The gloves themselves are a bundle of a dozen different technologies – there’s various sensors, a gyroscope, wireless microphones, an accelerometer, a magnetometer. It would take considerable R&D to commercialize a package like that.

There has been more research and development in the realm of gestural music than you might imagine, though – we’ve moved well beyond that gestural sci-fi music staple, the theremin. Michel Waisvisz, who debuted his musical interface, dubbed “the Hands,” back in the ’80s, was a pioneer here. Here’s a video of him performing with them at the New Interfaces for Musical Expression conference, which for 11 years has helped spur research of the sort that led to Imogen Heap’s gloves.

Someday in the future, then, will a handmade cello seem as obsolete a curio as the paperbound book?

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

The problem with plug-in hybrids? Their drivers.

Plug-in hybrids are often sold as a transition to EVs, but new data from Europe shows we’re still underestimating the emissions they produce.

Google DeepMind’s new generative model makes Super Mario–like games from scratch

Genie learns how to control games by watching hours and hours of video. It could help train next-gen robots too.

How scientists traced a mysterious covid case back to six toilets

When wastewater surveillance turns into a hunt for a single infected individual, the ethics get tricky.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.