Whatever your thoughts on the way the Matrix trilogy wound down, most everyone can agree on one thing: the special effects, well, rocked. Especially those in the first film. The innovative “Bullet Time” sequences were the first time audiences saw a character slow down and the camera spin around him.
Fans of those effects would do well to keep their eyes on new software being developed by a team led by Rick Szeliski , a senior researcher at Microsoft Research. Szeliski presented the concept last week at the annual SIGGRAPH conference on leading-edge computer graphics in Los Angeles. The new algorithms will allow film studios, videogame design houses, and potentially consumers to create Matrix-like effects for a fraction of the cost and CPU power.
Currently, Matrix-like special effects are created by shooting a scene with hundreds of cameras meticulously assembled around a scene. In the editing room, the shots from various cameras are spliced together in order, and software interpolates, or fills in, between the hundreds of angles to make the action appear continuous. Szeliskis Microsoft Research team has engineered a new, much more powerful interpolation algorithm that can create the same seamless, fluid sequencesusing only eight cameras.
Interpolationthe act of artificially filling the space between two shotshas been around for some time, but the Microsoft Research development marks the third major advance in the field of video manipulation.
About 15 years ago, the first breakthrough occurred when tweening technology was first introduced. Tweening offered the same kind of fluidity of movement but only for two-dimensional images. The technology is currently in use in animated movies and video games. A second, more recent innovation was the advent of MPEG compression for motion picture images. While not interpolation, per se, the compression algorithms made it possible to even consider using computers to accomplish effects like Bullet Time, since the uncompressed files are too enormous for even the most powerful processors.
Graphics experts hail the new interpolation algorithm as just as important for computerized visual effects. “The work is inspiring,” says Jon Peddie, president of Jon Peddie and Associates, a research firm specializing in graphics technology. “[Microsofts] interpolation has the potential for being as big as MPEG and tweening.”
Our breakthough is that we have a much better matching algorithm between cameras, says Szeliski. The algorithm used by the visual effects team that created The Matrix wasnt powerful enough to interpolate a large distance between cameras, necessitating hundreds of camera placements. Using complex mathematical formulas created by the research team, Microsofts algorithm can mimic movement between much greater distances with fewer cameras. In short, it can create the same picture by connecting fewer dots.
Commercial applications for the technology are anywhere from two to five years away. The first logical areas where the technology could be used are in videogames and DVDs. Videogames already make use of tweening technology, and could incorporate 3D interpolation into new games, whereby a players character could seamlessly circle other characters to view threats from any angle. Szeliski predicts the interpolation algorithm will make its way first into the increasingly popular extra features found on most DVDs. With the right partner and the right applications to justify the cost, Szeliski says, fans could pick the camera angles or whirl around a battle scene for a 360-degree view.
Peddie believes that with a development of this magnitude, its hard to predict exactly what people with do with it. When ever a technology is made this much easier, it very often generates unexpected uses, he says. He believes one things for certain: It will stimulate peoples imagination.
This startup wants to copy you into an embryo for organ harvesting
With plans to create realistic synthetic embryos, grown in jars, Renewal Bio is on a journey to the horizon of science and ethics.
VR is as good as psychedelics at helping people reach transcendence
On key metrics, a VR experience elicited a response indistinguishable from subjects who took medium doses of LSD or magic mushrooms.
This nanoparticle could be the key to a universal covid vaccine
Ending the covid pandemic might well require a vaccine that protects against any new strains. Researchers may have found a strategy that will work.
This artist is dominating AI-generated art. And he’s not happy about it.
Greg Rutkowski is a more popular prompt than Picasso.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.