Researchers at Intel have developed an algorithm that, by leveraging the power of multiple microprocessors, can boost the resolution of a video as it plays in real time. The technology, called super resolution, can run on machines with as few as two cores and as many as hundreds, potentially letting people enhance video captured with a cheap webcam, improve old home movies, or turn a DVD-quality video into a high-resolution flick.
Intel’s super-resolution research is part of the company’s push to find the best applications to run on its multicore machines, says Jerry Bautista, codirector of Intel’s tera-scale computing research program. While multicore computers–machines with more than one processing core–are currently available to consumers in dual- and quad-core varieties, Intel has a research-grade microprocessor with 80 cores. (See “The Promise of Personal Supercomputing.”) And as researchers get closer to their goal of achieving tera-scale computing on desktop computers–in which trillions of calculations per second are enabled by massively multicore systems–the company is ramping up its software research; improving video quality using multicore machines is one of the top priorities on Intel’s to-do list, says Bautista.
To be sure, the chip maker isn’t the first to explore the idea of adding resolution to video. Super-resolution theory dates back to the 1980s, says Peyman Milanfar, a professor of electrical engineering at the University of California, Santa Cruz. But in the early days, the algorithms just didn’t work well, and the computing power wasn’t there to process the videos quickly. In 2003, Milanfar and his group developed computationally efficient algorithms that were able to improve the resolution for most video, although not in real time. Indeed, Milanfar’s approach has been the basis of other research by academics and companies.
Super-resolution algorithms upgrade video in two main steps, explains Oscar Nestares, senior research scientist at Intel. First, the algorithm examines pixels in the video frames to see how fast each pixel is moving and in which direction. For instance, if a car is moving down a street, the pixels that compose it will all be moving in a predictable way.
Watch a demonstration of Intel's super-resolution algorithm.
The data collected in the first step is then used to estimate the movement of new pixels that are added to increase the video resolution. The result is a cleaner video that appears to be captured at the same time as the original. “We’re trying to get information that’s not there between frames,” says Intel’s Bautista. “The only way we can do this is if we collect lots of data and make better educated guesses at what those intermediate pixels should be.”
Bautista explains that one thing that differentiates Intel’s super-resolution algorithm from others is its ability, in real time, to generate what’s known as a robust result. This means that the algorithm is able to toss out any erroneous pixels that could be a result of electrical noise in the sensor or dust on the lens, for instance. These erroneous pixels tend to lead to inaccurate guesses and a video that isn’t true to reality.
In a recent demonstration at Intel’s Santa Clara campus, Nestares ran the algorithm on a dual-core Intel machine to add resolution to a webcam (click here to see a video of a demonstration). The algorithm was able to add four times the resolution to captured video, with no noticeable delay. Nestares says that the algorithm is written to scale to the number of cores that it runs on: the more cores available, the more the processing gets divided up. “Different regions [of the video] are assigned to different cores,” says Nestares. Using this method, it would take about 150 cores to convert DVD video into HD DVD in real time, he says.
There’s an economic driver to be able to add resolution to video, says William Freeman, a professor of electrical engineering and computer science at MIT. More people are buying HD TV sets, and there’s still a lot of content that hasn’t caught up with the resolution that those sets can provide. “There’s a real need … People are going to want to look at high-definition content on their new HD TV sets,” Freeman says.
At this point, however, Intel doesn’t have any plans to make the algorithm commercially available. But Bautista says that the team has talked with product groups at Intel about the possibility.