Heart Surgeons as Video Gamers
Cardiac surgeons have borrowed a 3-D stereoscopic imaging technology from the video-game industry to help them guide their tools during intricate beating-heart surgeries. In tests of the new imaging device, a surgeon was able to more accurately navigate into pigs’ hearts and then to more quickly repair the hearts’ torn walls. Eventually, the stereoscopic system may make beating-heart surgery more efficient and less dangerous, perhaps expanding its use into relatively complicated heart repairs.
For certain complex heart procedures such as valve repairs, a surgeon must stop the heart and cut it wide open. These surgeries require a heart-lung bypass machine and come with substantial risks to the patient. In recent years, some surgeons are opting to use beating-heart surgery, a less invasive approach that can often obviate the need for heart-lung bypass. But working on complex, delicate internal structures that move with every heartbeat presents major challenges–particularly because without opening the heart and diverting the blood, it’s hard for surgeons to see what they’re doing.
For several years, a team at Children’s Hospital Boston has been developing a real-time 3-D imaging system for use in beating-heart surgery. An ultrasound device gathers a stream of 3-D data and feeds it to a two-dimensional display, indicating depth with shades of gray. By keeping an eye on the display, a surgeon can see inside the heart during the operation.
But while early results of the technology, which has recently found its way into the operating room, have been promising, surgeons found that the two-dimensional display interfered with their depth perception. “If you’re navigating a three-dimensionally complicated structure like the heart, where you’re making right turns and left turns and going up and down and all over, using a two-dimensional image makes it very difficult,” says Pedro del Nido, chief of cardiac surgery at Children’s Hospital and leader of the research team.
For a solution, the team looked to an industry that’s well versed in creating an ever-improving 3-D experience: video gaming. Recent advances in image-processing technology have allowed game designers to simulate 3-D environments–complete with depth perception–using stereoscopic vision displays. “To address structural heart disease, you need a three-dimensional map with depth perception,” says Marc Gillinov, chief experience officer at Cleveland Clinic’s Heart and Vascular Institute. “And that’s what the stereoscopic glasses give you.”
A stereoscopic vision display works by generating a separate, slightly tilted 3-D image for each eye. The monitor rapidly flicks back and forth between the two versions about 70 times per second. Meanwhile, the viewer wears specialized glasses that alternately block the left and right eyes at the same rate. The flickering is fast enough that the eyes, which can only process some 25 to 30 frames per second, don’t notice it.
“Your brain knows how to integrate that information and give you depth perception automatically,” says del Nido. “You don’t have to have a computer that tells you you’re two and a half centimeters from the target–your brain intuitively figures that out, and it does it on the fly.” The result is a realistic, hologram-like 3-D experience.
Using this technology, the Children’s Hospital team built a custom-made display that could process data from an ultrasound probe. Then cardiac surgeon Nikolay Vasilyev donned a pair of stereoscopic glasses and tried out the display while operating on the beating hearts of pigs.
First, Vasilyev created a tear in the wall dividing the left and right atria, simulating a condition called an atrial septal defect. Then he repaired the hole with a tiny patch, fastening it in place with several tiny anchors. Because the entire surgery was performed with specialized tools that could be inserted through a very small opening in the heart’s surface, it was noninvasive enough to let the heart keep beating the whole time. In total, Vasilyev operated on six pigs: three with the stereoscopic display, and three with the regular 3-D display.
In all six surgeries, Vasilyev successfully affixed the patch, with no difference between the two groups in terms of how accurately he placed the anchors–perhaps because he has extensive experience with this particular procedure. But the new device did allow him to more precisely navigate his instruments to their destination–reducing the risk of damage to surrounding structures–and to get the job done 44 percent faster. The results of the study were announced yesterday in the Journal of Thoracic and Cardiovascular Surgery.
The stereoscopic system may help extend the uses of beating-heart surgery, expanding its use in children with congenital heart defects and in complicated heart repairs like extensive holes and faulty valves. Indeed, del Nido sees real-time stereoscopic imaging–combined with cutting-edge surgical instruments–as a platform that might ultimately revolutionize heart surgery the way that laproscopic procedures revolutionized abdominal surgery. “What we’re developing here is really a new platform for doing heart repairs in a very different way,” says del Nido. “We’ve taken the initial step: we’ve done the proof of concept. Now we want to start seeing how far we can go with it.”
“Is it going to be ready for prime time tomorrow? No, it’s not,” says Gillinov. “But could it open up a whole bunch of new ways to fix things in the heart? Yes.”
Geoffrey Hinton tells us why he’s now scared of the tech he helped build
“I have suddenly switched my views on whether these things are going to be more intelligent than us.”
ChatGPT is going to change education, not destroy it
The narrative around cheating students doesn’t tell the whole story. Meet the teachers who think generative AI could actually make learning better.
Meet the people who use Notion to plan their whole lives
The workplace tool’s appeal extends far beyond organizing work projects. Many users find it’s just as useful for managing their free time.
Learning to code isn’t enough
Historically, learn-to-code efforts have provided opportunities for the few, but new efforts are aiming to be inclusive.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.