Technology Review - Published By MIT
Log in to My.TechnologyReview.com | Register
Advertisement
[1] 2 Next »

Tuesday, June 26, 2007

Converting DVD into HD DVD

Software for multicore computers could add resolution to video in real time.

By Kate Greene

smaller text tool iconmedium text tool iconlarger text tool icon
Resolve to be sharper: Intel’s new algorithm, which leverages multicore architecture, can improve the resolution of video in real time.
Credit: Technology Review
Multimedia
•  Slideshow

Researchers at Intel have developed an algorithm that, by leveraging the power of multiple microprocessors, can boost the resolution of a video as it plays in real time. The technology, called super resolution, can run on machines with as few as two cores and as many as hundreds, potentially letting people enhance video captured with a cheap webcam, improve old home movies, or turn a DVD-quality video into a high-resolution flick.

Intel's super-resolution research is part of the company's push to find the best applications to run on its multicore machines, says Jerry Bautista, codirector of Intel's tera-scale computing research program. While multicore computers--machines with more than one processing core--are currently available to consumers in dual- and quad-core varieties, Intel has a research-grade microprocessor with 80 cores. (See "The Promise of Personal Supercomputing.") And as researchers get closer to their goal of achieving tera-scale computing on desktop computers--in which trillions of calculations per second are enabled by massively multicore systems--the company is ramping up its software research; improving video quality using multicore machines is one of the top priorities on Intel's to-do list, says Bautista.

To be sure, the chip maker isn't the first to explore the idea of adding resolution to video. Super-resolution theory dates back to the 1980s, says Peyman Milanfar, a professor of electrical engineering at the University of California, Santa Cruz. But in the early days, the algorithms just didn't work well, and the computing power wasn't there to process the videos quickly. In 2003, Milanfar and his group developed computationally efficient algorithms that were able to improve the resolution for most video, although not in real time. Indeed, Milanfar's approach has been the basis of other research by academics and companies.

Super-resolution algorithms upgrade video in two main steps, explains Oscar Nestares, senior research scientist at Intel. First, the algorithm examines pixels in the video frames to see how fast each pixel is moving and in which direction. For instance, if a car is moving down a street, the pixels that compose it will all be moving in a predictable way.

The data collected in the first step is then used to estimate the movement of new pixels that are added to increase the video resolution. The result is a cleaner video that appears to be captured at the same time as the original. "We're trying to get information that's not there between frames," says Intel's Bautista. "The only way we can do this is if we collect lots of data and make better educated guesses at what those intermediate pixels should be."

Bautista explains that one thing that differentiates Intel's super-resolution algorithm from others is its ability, in real time, to generate what's known as a robust result. This means that the algorithm is able to toss out any erroneous pixels that could be a result of electrical noise in the sensor or dust on the lens, for instance. These erroneous pixels tend to lead to inaccurate guesses and a video that isn't true to reality.

[1] 2 Next »

Comments

  • Nice article, poor presentation
    inboulder on 06/26/2007 at 3:28 AM
    Posts:
    10
    Avg Rating:
    5/5
    The video on this site of the upscaling in question is so small as to be useless. This is especially ironic considering the content of this article. Did Tech Rev contemplate this issue before publishing?
    Rate this comment: 12345
    • Re: Nice article, poor presentation
      Mr.Doerr on 06/26/2007 at 11:24 AM
      Posts:
      1
      Avg Rating:
      5/5
      I'm sorry that you did not see the difference in the video, which by the way was clearly visible. The idea was to show you an example, not to market the product.
      Rate this comment: 12345
      • Re: Nice article, poor presentation
        3danim8 on 06/26/2007 at 11:53 AM
        Posts:
        1
        Not only was the video in the left and right panels indistinguishable, the text that described the video panels was also in error. It stated: "The original video, on the left, was captured with a standard USB 2.0 webcam with an input of 160 by 120 pixels. The output on the left is 320 by 240 pixels."

        Should the description read: "output on the right..."
        Rate this comment: 12345
    • Re: Nice article, poor presentation
      brunascle on 06/26/2007 at 12:15 PM
      Technology Review TR Staff
      Web Developer
      Posts:
      70
      Avg Rating:
      4/5
      i agree. at this scale it's hard to make a judgement. i can see a difference between the two, but i wouldnt call the one on the right _better_, just different. the one on the left is blurry, the one on the right is blocky. and they're both equally legible, at least IMO.

      and judging the result by the resolution you're able to achieve is totally meaningless. i can put a video into whatever resolution i want simply by filling in random pixels, or upscaling. the only way to judge it is to either compare it to an original taken at the same resolution, or by human perception.
      Rate this comment: 12345
    • Re: Nice article, poor presentation
      GaryB on 06/26/2007 at 5:59 PM
      Posts:
      42
      Avg Rating:
      3/5
      While I agree that say an 85 year old would see no difference in these videos, those of us who are younger and retain some visual acuity can quite clearly read the text better on the right than on the left. 

      I'd like to see this work on non-text, say a sports clip with near/play far/crowd differences.  Or better yet, a youtube clip which have just awful resolution and terrible compression artifacts.  But, as is, it's pretty cool work.
      Rate this comment: 12345
    • Re: Nice article, poor presentation
      mem11363 on 07/08/2007 at 2:16 AM
      Posts:
      1
      I believe the original video steadily cycles from clear to blurry to clear.  This is consistent with what I have experienced with webcams when the amount of motion in the input stream exceeds their processing capability.  The "enhanced" video remains clear.  So this demonstration tells me that the algorithm is successful at determining the clarity of each frame it processes and uses the better ones - ignores or puts a lower weight on the worse ones.  But that seems like a different goal than taking a good quality video at resolution X (like a high quality dvd) and converting it to a high quality video at say double or quadruple the initial resolution. 
      Rate this comment: 12345
  • Other super-resolution solutions
    pookien on 08/22/2007 at 1:18 PM
    Posts:
    1
    Nothing new. Just google a bit and you'll see that this super resolution technology already exists as an Adobe After Affects and Premiere Pro plug-in and also as a stand-alone Windows application. It  actively uses multiple cores too and works much faster (320x240 -> 640x480 with 20 fps on a quad core).
    http://www.thedeemon.com/super_resolution/
    http://www.thedeemon.com/VideoEnhancer/
    Rate this comment: 12345
  • Just saw this article
    jkent on 04/22/2008 at 6:18 PM
    Posts:
    1
    Found something from CNBC about super-resolution in the film industry. Looks like all the indie filmmakers are doing backflips now.

    http://www.cnbc.com/id/23954891/

    The company's called Topaz Labs. www.topazlabs.com . They make some bold claims.
    Rate this comment: 12345
Advertisement

Current Issue

Technology Review September/October 2008
How Obama Really Did It
Social technology helped bring him to the brink of the presidency.
•  Subscribe
Save 41%
•  Table of Contents
•  MIT News

Magazine Services

Career Resources

MIT Technology Insider

Stories and breaking news from inside MIT about the latest research, innovations, and startups--in a convenient monthly e-newsletter. Subscribe today

Follow us on Twitter

Twitter

Get Technology Review updates via the web, cellphone, or Instant Messager – Follow techreview on Twitter!

Advertisement

More Technology News from Forbes

Advertisement
Advertisement
Advertisement
TECHNOLOGY RESOURCES
Advertisement
MIT Massachusetts Institute of Technology