Skip to Content

Heart Surgeons as Video Gamers

Stereoscopic glasses inspired by video games help surgeons operate on beating hearts.
June 10, 2008

Cardiac surgeons have borrowed a 3-D stereoscopic imaging technology from the video-game industry to help them guide their tools during intricate beating-heart surgeries. In tests of the new imaging device, a surgeon was able to more accurately navigate into pigs’ hearts and then to more quickly repair the hearts’ torn walls. Eventually, the stereoscopic system may make beating-heart surgery more efficient and less dangerous, perhaps expanding its use into relatively complicated heart repairs.

3-D view: Researchers at Children’s Hospital Boston are bringing video-game technology into the operating room. Cardiac surgeon Nikolay Vasilyev gives his stereo goggles a test run while performing surgical simulations. The goggles, coupled with a monitor displaying real-time ultrasound data, provide a stereoscopic 3-D view. Vasilyev tested the technology in beating-heart surgery on pigs’ hearts and found that it allowed him to patch a hole more efficiently than with traditional 3-D imaging.

For certain complex heart procedures such as valve repairs, a surgeon must stop the heart and cut it wide open. These surgeries require a heart-lung bypass machine and come with substantial risks to the patient. In recent years, some surgeons are opting to use beating-heart surgery, a less invasive approach that can often obviate the need for heart-lung bypass. But working on complex, delicate internal structures that move with every heartbeat presents major challenges–particularly because without opening the heart and diverting the blood, it’s hard for surgeons to see what they’re doing.

For several years, a team at Children’s Hospital Boston has been developing a real-time 3-D imaging system for use in beating-heart surgery. An ultrasound device gathers a stream of 3-D data and feeds it to a two-dimensional display, indicating depth with shades of gray. By keeping an eye on the display, a surgeon can see inside the heart during the operation.

But while early results of the technology, which has recently found its way into the operating room, have been promising, surgeons found that the two-dimensional display interfered with their depth perception. “If you’re navigating a three-dimensionally complicated structure like the heart, where you’re making right turns and left turns and going up and down and all over, using a two-dimensional image makes it very difficult,” says Pedro del Nido, chief of cardiac surgery at Children’s Hospital and leader of the research team.

For a solution, the team looked to an industry that’s well versed in creating an ever-improving 3-D experience: video gaming. Recent advances in image-processing technology have allowed game designers to simulate 3-D environments–complete with depth perception–using stereoscopic vision displays. “To address structural heart disease, you need a three-dimensional map with depth perception,” says Marc Gillinov, chief experience officer at Cleveland Clinic’s Heart and Vascular Institute. “And that’s what the stereoscopic glasses give you.”

A stereoscopic vision display works by generating a separate, slightly tilted 3-D image for each eye. The monitor rapidly flicks back and forth between the two versions about 70 times per second. Meanwhile, the viewer wears specialized glasses that alternately block the left and right eyes at the same rate. The flickering is fast enough that the eyes, which can only process some 25 to 30 frames per second, don’t notice it.

Split screen: An ultrasound device sends a constant stream of three-dimensional data from the patient’s heart to a computer, which splits it into a pair of slightly tilted images–one for each eye. Specialized glasses flicker back and forth between the left and right eyes while the two images flicker back and forth on a computer screen. The result is akin to a holographic video of the heart’s inner workings.

“Your brain knows how to integrate that information and give you depth perception automatically,” says del Nido. “You don’t have to have a computer that tells you you’re two and a half centimeters from the target–your brain intuitively figures that out, and it does it on the fly.” The result is a realistic, hologram-like 3-D experience.

Using this technology, the Children’s Hospital team built a custom-made display that could process data from an ultrasound probe. Then cardiac surgeon Nikolay Vasilyev donned a pair of stereoscopic glasses and tried out the display while operating on the beating hearts of pigs.

First, Vasilyev created a tear in the wall dividing the left and right atria, simulating a condition called an atrial septal defect. Then he repaired the hole with a tiny patch, fastening it in place with several tiny anchors. Because the entire surgery was performed with specialized tools that could be inserted through a very small opening in the heart’s surface, it was noninvasive enough to let the heart keep beating the whole time. In total, Vasilyev operated on six pigs: three with the stereoscopic display, and three with the regular 3-D display.

In all six surgeries, Vasilyev successfully affixed the patch, with no difference between the two groups in terms of how accurately he placed the anchors–perhaps because he has extensive experience with this particular procedure. But the new device did allow him to more precisely navigate his instruments to their destination–reducing the risk of damage to surrounding structures–and to get the job done 44 percent faster. The results of the study were announced yesterday in the Journal of Thoracic and Cardiovascular Surgery.

The stereoscopic system may help extend the uses of beating-heart surgery, expanding its use in children with congenital heart defects and in complicated heart repairs like extensive holes and faulty valves. Indeed, del Nido sees real-time stereoscopic imaging–combined with cutting-edge surgical instruments–as a platform that might ultimately revolutionize heart surgery the way that laproscopic procedures revolutionized abdominal surgery. “What we’re developing here is really a new platform for doing heart repairs in a very different way,” says del Nido. “We’ve taken the initial step: we’ve done the proof of concept. Now we want to start seeing how far we can go with it.”

“Is it going to be ready for prime time tomorrow? No, it’s not,” says Gillinov. “But could it open up a whole bunch of new ways to fix things in the heart? Yes.”

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.