Skip to Content

AR Is Making Its Way into the OR

Doctors may soon be able to augment their view of your body, but it will be some time before it’s commonplace.

“Scalpel. Forceps. Suction. Oh, and nurse, pass me the HoloLens.”

That’s not the kind of conversation you’ll currently overhear in an operating room, but it’s certainly the direction in which we’re headed. Late last year, engineers at Duke University announced that they were developing systems to test how Microsoft’s augmented reality device might provide doctors with extra information during brain surgery. And now, in the past week, two firms have offered up further takes on how the HoloLens could be used during procedures.

First up is a system being developed by the software company Scopis that's designed to help during spinal surgery. It can put scans into the field of view of a surgeon, or provide a simulated view of how pedicle screws—the metal rods used to repair vertebrae—are being positioned during the procedure.

More ambitious is a project by Cambridge Consultants that seeks to provide surgeons with the ability to load results from MRI and CT scans into software that renders them into a 3-D visualization of a patient’s organs. The idea: surgeons would be able to perform keyhole procedures with an extra layer of information in front of them, allowing them to take better-informed actions.

The concept is certainly a good one. Such approaches could allow the technology to alert surgeons if they, say, stray too close to a particularly delicate part of anatomy during a procedure. Or else they could help guide less experienced doctors during complex procedures.

But that, of course, raises some interesting questions. Namely, how comfortable would most people feel about surgeries carried out based on data provided via a HoloLens? As Shafi Ahmed, a surgeon at Royal London Hospital, explained to New Scientist: “We can’t trust these systems at the moment.” So the answer is: probably not very, for now at least.

There are clearly some hurdles to overcome. One of the largest will be that of image registration: loading MRI scans into a HoloLens is all well and good, but the body itself moves around naturally, and can look different during surgical procedures—the device will need to adjust the AR image accordingly, which is harder than it sounds. And at any rate, most doctors like to be sure of the technology they’re dealing with, so adoption of the most advanced surgical AR systems will be slow at first.

It is, however, easy enough to imagine the systems being used for less complex augmentation—such as providing relevant numbers or stills of medical scans in the periphery of a surgeon’s vision. So don't be too alarmed if your surgeon asks for the HoloLens as you're about to go under the knife.

(Read more: Engadget, New Scientist, “Microsoft’s New Idea: A Hologram Headset to Rewrite Reality”)

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.