Edit

Computing

Google Aims to Make VR Hardware Irrelevant Before It Even Gets Going

Smartphones have sidelined digital cameras and other special-purpose devices. Now Google thinks mobile phones will shove virtual-reality headsets like the Oculus Rift into the shadows, too.

The bright black sand at Diamond Beach, Iceland, is dotted with smooth gray pebbles and glassy chunks of ice. The crash of foamy waves fills my ears as I turn my head to look up and down the beach and see the dark strip of sand fade into white fog as if it goes on forever. I don’t feel the cold, because I’m not really there. Moments earlier, I had been standing in a cable car climbing over Onomichi, Japan.

I’m in a windowless room at Google’s headquarters, holding a cardboard box against my face in a way that positions a smartphone two inches from my eyes. Google Cardboard, as both the phone holder and the accompanying app are called, transforms the computer in your pocket into a virtual-reality headset. It’s something of a gimmick. But it’s also a serious attempt to create a new mass communication and entertainment medium. “Virtual reality will have an important role to play in entertainment, communications, work, and learning,” says Clay Bavor, who leads Google’s virtual-reality project. “Cardboard will be the way that we make these immersive experiences available for everyone.”

Mark Zuckerberg, CEO of Facebook, also thinks virtual reality will change the world. He has predicted that it is “the next major computing and communication platform,” a very serious claim in Silicon Valley business-speak. But Zuckerberg and Google are trying to make it come true in almost opposite ways. In 2013, Facebook spent $2 billion to acquire Oculus VR, a company developing a sophisticated virtual-reality headset called the Rift that is held to your face with a thick elasticated strap and attaches via a cable to a powerful PC. The Rift goes on sale early next year. The price is unknown, but a version that until recently was available to developers cost $350, and an Oculus-approved PC to power its forthcoming headset costs at least $950. Facebook has also collaborated with Samsung on a $99 headset called the Gear VR, which can use some Samsung smartphones as a screen, and boasted that it will widen virtual reality’s audience this way. But Samsung is unlikely to make the Gear VR compatible with its competitors’ devices, and Facebook’s main project is the high-end Rift experience.

In contrast, Cardboard works with iPhones and almost any phone running Google’s Android software. Google has released the design of the Cardboard phone holder for free, and other companies sell versions for $10, or more if you want one in plastic, metal, or wood. Google estimates that it and other companies have sold or given away more than a million Cardboard kits. This weekend the New York Times will send out over a million more to its subscribers, to promote a virtual-reality documentary on children displaced by war. Google has also begun sending Cardboard kits to schools with a special version of the app that lets a teacher take a class on a 3-D tour of a coral reef or Machu Picchu.

Students in a classroom at the Bronx Latin School in New York City tried Cardboard as part of Google’s Expeditions Pioneer Program for schools.

The Rift offers a far better experience than Cardboard and has spurred tech and media companies to invest hundreds of millions in virtual-reality startups, content, and copycat headsets from companies such as Sony and phone manufacturer HTC. But the technological tide that made the Rift possible also works against it—in Cardboard’s favor. Even as the smartphone industry slashed the costs of the displays and sensors needed to build a good virtual-reality headset, smartphones have made people less inclined to spend money on PCs or on single-purpose gadgets such as cameras or GPS devices—and perhaps virtual-reality technology. “There’s a set of enthusiastic users, me being one of them, that’ll be willing to charge and plug in and assemble these things, and we’re going to have a great experience,” says Google’s Bavor. And indeed, many people on his team are working on applications for high-end VR technologies, he says. But for virtual reality to go anywhere right now—and to answer the question of what on earth it’s good for—it has to spread beyond that small niche, says Bavor.

Google has several initiatives designed to help its lightweight take on virtual reality take off. It is working with phone makers to ensure that future smartphones support Cardboard better. It has invented a new kind of camera to make it easier to produce virtual-reality movies, and it is developing YouTube into a platform to deliver them. If it works, Google’s gimmick might neutralize virtual-reality headsets before they even get properly established.

No head strap required

Cardboard originated in a sequence of events that seem divinely chosen to help Google executives boast of their company’s openness to new ideas. In the spring of 2014, David Coz, a software engineer in Google’s Paris office, traveled to company headquarters in Mountain View, California, carrying the first prototypes of Cardboard, which he’d made with a colleague, Damien Henry, in the time Google allows for side projects. Coz demonstrated it to a few people, who liked it and showed it to others. Within hours somebody handed Cardboard to Google cofounder and CEO Larry Page after he stepped off stage at the company’s weekly all-hands meeting. He liked it too. The next day, Bavor’s boss, Sundar Pichai (who is now Google’s CEO), called him in to try it. Bavor lifted the cardboard box to his face and was transported to Paris, thanks to imagery from Google’s Street View service. When Bavor’s mind returned to Mountain View, Pichai told him to turn the prototype into a real product, to be launched two months later at Google’s annual conference for developers.

That June, thousands of developers, Google employees, and journalists shuffling out of the almost three-hour keynote were each handed a brown cardboard packet the size of a slim paperback. It opened to reveal a set of cardboard goggles; a few Velcro patches, two plastic lenses, a magnet, and a rubber band were also included. Simple instructions explained how to fold the device together, download an app, and put your phone inside to jump into virtual space. Cardboard’s design was updated this summer, but it remains essentially the same. The app splits a phone’s display in two, and the goggles’ lenses project a slightly different viewpoint to each of your eyes, so your brain perceives depth. To interact with what you see on the screen—to select from a virtual menu, say, or pause a movie—you press a button on top of the goggles. It swings a piece of metalized fabric against the phone’s touch screen, which registers the tap of a fingertip.

Cardboard is easy to fold and put to use with many kinds of smartphones.

Putting your expensive glass-and-metal pocket computer into a dowdy brown box you just folded together and raising it to your face feels a little silly. But watch someone try Cardboard for the first time, and you will usually see a grin quickly appear beneath the plain brown goggles. “There is magic in the difference between your expectations and what it delivers,” says Bavor. “You already have this device in your pocket that, when you add some cardboard, turns into a good VR viewer.”

He is both right and wrong. It is somewhat surprising that your phone can function as virtual-reality headset. But right now Cardboard is only okay, not good. For a few minutes it’s great, but the discomfort and even nausea that VR can cause—when your body’s sense of how it is moving conflicts with what it is seeing—strike more readily than they do when you use a headset like the Rift. Current smartphone hardware can’t update the virtual world in close enough synchrony with the movements of your head as you look around, which is what causes motion sickness. The Rift works well in part because it has cut that lag to less than 20 milliseconds, and even then, viewing some kinds of content can still leave some users nauseated. Cardboard’s latency is significantly greater.

Google designed Cardboard without a head strap in recognition of that deficiency—and to make it more social. The idea is that your arms will get tired before you overexpose yourself or your friends get antsy for their turn. But one reason the Google VR team is now large enough to occupy its own building on the Google campus is that Bavor thinks smartphones offer a good enough experience for some purposes and will soon get much better. Google is talking with manufacturers about how their devices could help fix Cardboard’s limitations—for example, by upgrading their motion sensors to be more accurate and to update their readings more frequently, like those in Samsung’s Gear VR devices. Because its Android operating system powers 1.4 billion smartphones around the world, Google has significant influence on the companies that make them and how they perform.

Qualcomm, which leads the market in smartphone processors, has made better support for virtual reality a selling point of its new flagship chip, the Snapdragon 820, which is due to appear in many Android devices in 2016. Manufacturers see support for virtual reality as a priority for high-end handsets, says Jay Wright, who until this week was a vice president of product management at Qualcomm who worked on virtual and augmented reality. He joined the 3-D design software company PTC, which bought a Qualcomm group working on augmented reality software. “It’s been proven there is a viable experience,” he says. “Now it’s about improving the performance.”

Cardboard could also benefit from a Google research effort known as Project Tango, which is developing 3-D sensors and software to help phones and tablets accurately track their position in space. Qualcomm and competing chipmakers Infineon and Intel are all on board. Google is unlikely to get much help from its competitor Apple, although its devices currently work with Cardboard as well as high-end Android devices do. Nonetheless, with the progress being made, add-on devices like Samsung’s Gear VR aren’t necessary, according to Bavor. “We prefer to tune the software and components of the smartphone to work well as opposed to adding complexity and things you need to charge,” he says.

Strategy games

When you ask virtual-reality developers to predict what we all will do with the technology and why, you often hear predictions about how we will physically engage with imaginary worlds and the objects and people in them. The Oculus Rift will ship with a games controller from Microsoft’s Xbox, and Facebook will also sell complicated controllers that can be used to grab and manipulate virtual objects. Google is instead trying to make virtual reality into a new kind of casual, mostly passive, entertainment like watching TV or surfing online videos—one perhaps more likely to win a mass audience and thus become a lucrative platform for ads or premium content.

The Cardboard app offers a handful of such experiences, including a version of the Google Earth 3-D map program where you fly over cities or mountain ranges, and a cute animated short by a former Pixar director. Other companies can create mobile apps to display their own content through Cardboard and will take you climbing in Yosemite, freak you out with a short horror movie, or put you on stage next to Paul McCartney’s piano. And this summer, YouTube was upgraded to host what are described as “360-degree videos,” which don’t present your eyes with 3-D imagery but give a sense of being somewhere else by enveloping you with a panoramic video. They can surround you with frenetic dancers in a music video, or take you to snowy Svalbard to watch a solar eclipse. Soon, much more impressive 3-D videos like the one that transported me to Iceland will appear on YouTube. For this purpose, Google invented a camera called Jump; it’s built from a ring of 16 small cameras positioned so that every point is captured from three different perspectives, allowing software to merge them into a 3-D view.

Jump’s basic design has been released for free, and GoPro will begin selling a version of the camera later this year. Google will offer an “assembler” service that converts raw Jump footage into 3-D experiences ready for viewing or editing at no charge. Bavor predicts that the cameras will become standard for film crews large and small; indeed, major media companies such as ABC and the Discovery Channel are already producing content for this nascent virtual-reality platform. Mercedes and Volvo have made Cardboard apps for promotional purposes. “Anyone with a smartphone and a Cardboard will be able to see the most beautiful places on earth and participate in important events and moments in history,” Bavor says.

The students in New York viewed Aztec ruins through Cardboard.

Google is taking on a chicken-and-egg problem common to any company trying to establish itself as the middleman in a new marketplace. Consumers won’t buy into virtual reality unless there’s something to experience; content companies won’t provide it without a ready audience. “The only way to get around it is to subsidize one side of the market,” says Michael Cusumano, a management professor at MIT who has studied how software and consumer electronics companies conjure up new businesses. “Give away 3-D viewers, or pay software developers to create applications or other content.” Facebook is trying to fill the content gap by hosting 360-degree video as YouTube does, partnering with content companies such as Netflix, and building an in-house production effort called Oculus Story Studio. Google is effectively subsidizing both sides of the market. By making handsets more or less for free out of devices we already own, it’s offering its content partners a potentially much larger audience.

Philip Rosedale, founder of the virtual world Second Life and now the CEO of High Fidelity, a company that has raised $16 million to develop ways for people to socialize in virtual-reality environments, predicts that both Google’s and Facebook’s projects will initially disappoint people who expect it to become popular. “Next year there’s going to be a big push to make VR a success, and it’s not going to succeed,” he says. Rosedale thinks it will take a few years for Oculus to reach beyond a niche gaming audience, or for smartphones to be capable of providing a widely acceptable experience. Referring to Bavor’s group, he says: “I have a lot of respect for that team. But I don’t think that doing VR in software on top of a mobile device can be compelling today.”

Bavor acknowledges that Google and other companies investing in virtual reality have much to learn, but he says the only way to do it is to get the technology to a significant audience. And the device for doing that is already—perhaps all too often—in our hands and in front of our eyes. “The technology’s here,” he says. “We’ve seen the proof that you can create wonderful VR experiences with today’s smartphone hardware; the sensors and the hardware and the content are only going to get better.”

Uh oh–you've read all five of your free articles for this month.

Insider Online Only

$19.95/yr US PRICE

Computing

From the latest smartphones to advances in quantum computing, the hardware behind today's digital age is rapidly changing.

You've read of free articles this month.