Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Intelligent Machines

A Camera That Sees like the Human Eye

IBM’s brainlike computer architecture paves the way for a new kind of artificial vision.

Better artificial vision will make it easier to sort through vast sets of visual data.

The retina is an enormously powerful tool. It sorts through massive amounts of data while operating on only a fraction of the power that a conventional digital camera and computer would require to do the same task.

Dynamic vision: The camera’s strength is in capturing movement, like the milk drops seen here.

Now, engineers at a company called iniLabs in Switzerland are applying lessons from biology in an effort to build a more efficient digital camera inspired by the human retina.

Like the individual neurons in our eyes, the new camera—named the Dynamic Vision Sensor (DVS)—responds only to changes in a given scene. This approach eliminates large swaths of redundant data and could be useful for many fields, including surveillance, robotics, and microscopy.

“Your eye and my eye are digital cameras too. [They’re] just a different kind of digital camera,” says Tobi Delbruck, the chief scientific officer at iniLabs. “We had machine vision that was as good as possible with existing architecture and hardware. But compared to biology, machine vision is pathetically poor.”

An ordinary camera will take in everything it sees, storing the information to be processed later. This uses up a lot of power and a lot of space. Neurons in the eye, however, fire only when they sense a change—such as when a particular part of a scene gets brighter or dimmer. The DVS mimics that selectivity, transmitting information only in response to a shift in the scene. That takes less power and leaves less information to be processed.

Artificial retina: The Dynamic Vision Sensor (DVS) responds only to changes in the scene, eliminating large swaths of irrelevant data.

This feature could be especially useful for recording scenes that do not change often. For example, when sleep researchers videotape their subjects, they are later forced to comb through hours of uneventful footage. With a sensor like the DVS, important, active portions of the data are automatically highlighted.

The pixels in the DVS also mimic the way an individual eye neuron will calibrate itself to a particular location: that cell and those responsible for another area will respond to incoming data in slightly different ways, so one neuron might be very sensitive to input while another takes more stimulation to fire. Similarly, each pixel of the DVS adjusts its own exposure. This allows the DVS to handle uneven lighting conditions, though it also requires enormous pixels that are 10 times the size of those in a modern cell-phone camera.

The DVS is built to work with IBM’s new TrueNorth computer architecture (see “IBM Researchers Show Blueprints for Brainlike Computing”). TrueNorth is a programming approach that mimics biology—information is stored, processed, and shared in a network of “neuromorphic” computer chips, inspired by the neural networks in the brain.

“What we’re talking about—the cameras sending information when something changes—is actually a very central theme to how the brain works, or at least how neuroscientists think it works,” says Nabil Imam, a computer scientist at Cornell University, who is part of the Cornell team that helped IBM develop its neuromorphic chips. “We’re capturing brain features at a high level.”

By combining their camera with TrueNorth architecture, Delbruck and his team believe, they will achieve a device that’s better at handling dynamic, real-time problems.

The DVS is available for purchase for about $2,700 and has been used in several research projects, including one that recorded traffic and another that involved tracking particles in a fluid. The team plans to continue improving the device. The next goals are to add color sensitivity and to enlarge the camera’s retina from its current resolution of 240x180.

Keep up with the latest in intelligent machines at EmTech Digital.

The Countdown has begun.
March 25-26, 2019
San Francisco, CA

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to MIT Technology Review.
  • Print + All Access Digital {! insider.prices.print_digital !}* Best Value

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivery each weekday to your inbox

    The MIT Technology Review App

  • All Access Digital {! insider.prices.digital !}*

    {! insider.display.menuOptionsLabel !}

    The digital magazine, plus unlimited site access, our online archive, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    Digital magazine (6 bi-monthly issues)

    Access to entire PDF magazine archive dating back to 1899

    The Download: newsletter delivery each weekday to your inbox

  • Print Subscription {! insider.prices.print_only !}*

    {! insider.display.menuOptionsLabel !}

    Six print issues per year plus The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Print magazine (6 bi-monthly issues)

    The Download: newsletter delivery each weekday to your inbox

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.