Skip to Content

Collaborative Augmented Reality Makes Beautiful Music

With ARmonica, two players bounce virtual balls off musical note bars using Wii remotes.

Researchers demonstrate a collaborative augmented reality game.
Credit: Courtesy of Columbia University.

Augmented reality (AR) places interactive, virtual objects and effects over the real world, and it has huge potential for gaming. Imagine wearing an AR headset to play chess with animated virtual pieces on a real chess board, or to fight computerized zombies running into your own home.

For this to happen, computer scientists need to make sure players’ views and motions are in sync, which is doubly hard with two players, because both players need to see the virtual objects in the same space. A new AR environment designed by computer scientists at Columbia University is a step toward this kind of two-player AR gaming. A game, dubbed “ARmonica,” lets two people create a floating, musical world to collaboratively make music.

Each player wears a pair of Vuzix AR glasses, through which they see a real-time video feed of their environment (slightly different for each eye). The players use Wii remotes superimposed with floating controls to place virtual bars in the space around them. Then, they can launch virtual balls at the bars. When the balls bounce off specific bars, the players hear a unique note. The headsets and remotes are tagged to help the program make sure both users see the virtual objects in the same locations.

The group, led by professor Steven Feiner, presented the work at the 2010 Symposium on User Interface Software and Technology in New York last month.

Keep Reading

Most Popular

individual aging affects covid outcomes concept
individual aging affects covid outcomes concept

Anti-aging drugs are being tested as a way to treat covid

Drugs that rejuvenate our immune systems and make us biologically younger could help protect us from the disease’s worst effects.

close up of baby with a bottle
close up of baby with a bottle

The baby formula shortage has birthed a shady online marketplace

Desperate parents just want to feed their babies. They’re having to contend with misinformation, price gouging, and scams along the way.

"Olive Garden" NFTs concept
"Olive Garden" NFTs concept

I tried to buy an Olive Garden NFT. All I got was heartburn.

Our newest issue spells out what you need to know about the dizzying world of digital money.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.