We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Intelligent Machines

A Sidewalk Disappearing Act

Automatically removing people from street-level imagery could help prevent privacy complaints.

In 2008, responding to privacy concerns, Google started to blur the faces of people caught by car-mounted cameras and shown in its Google Street View mapping service. Researchers in California believe they have now come up with a better solution–software that automatically removes any trace that a person was in a scene.

Now you see me: Software that removes pedestrians from Google Street View could help to alleviate privacy concerns.

The approach protects people’s privacy while also providing a cleaner street-level view, says Arturo Flores, a computer-science graduate student in the Artificial Intelligence Group at the University of California, San Diego. “Even with face blurring, it is still possible to identify a person,” Flores says. Clothing, body shape, and height, combined with a location, can be enough to recognize someone, he says.

Google’s Street View vans use nine roof-mounted cameras to take regular shots of the scene around them. These are then stitched together to produce a near-seamless panoramic view. But automatically removing people from thousands of varied images, each showing different scenes, is a challenge.

Flores’s software first has to detect any pedestrians in a scene. This is done using a standard object-recognition algorithm called implicit shape model (ISM), which was developed at the Swiss Federal Institute of Technology. “The idea is to find a rough contour of pedestrians,” says Bastian Leibe, a codeveloper of ISM who is now at RWTH Aachen University. Because there is so much variability in human appearance, the algorithm takes a probabilistic approach–looking for similarities between the shapes in images and hundreds of images of pedestrians that it has been trained to recognize.

Once a pedestrian has been identified and cut from an image, the hole left behind has to be filled in. Flores’s software does this by using photographs captured before and after the image in question by Google’s Street View vans. These images show a view of the background from slightly different angles–the algorithm can reorient the background and stitch it into the space left behind by the missing pedestrian.

Flores and his advisor, Serge Belongie, recently presented the work at the IEEE International Workshop on Mobile Vision in Chicago. Flores says some real-world images are simply too unusual for the software to process properly. Odd artifacts have been left behind in some shots, such as dogs on a leash without their owner, or pairs of shoes apparently abandoned on the sidewalk.

The system also struggles to generate a background when a pedestrian is walking in the same direction as the Google van, says Flores. “It just isn’t possible to get an unobstructed view of the background,” he says.

Andrew Blake, deputy managing director of Microsoft Research Cambridge, in the U.K., and an expert in computer vision, says most of the techniques used in Flores’s software are well-known. He adds that recent breakthroughs have made it easier to stitch together images to create a smooth-looking composite. “State-of-the-art panorama-stitching software allows the photographer to stitch together the best possible panorama from any disorderly pile of pictures of a scene,” he says.

Leibe notes that this software, or something similar, could ultimately be used to create street-level imaging for turn-by-turn navigation systems, to replace the computer-generated animations now used by such systems.

Hear more from Google at EmTech Digital.

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    Print + Digital Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

    Technology Review PDF magazine archive, including articles, images, and covers dating back to 1899

    10% Discount to MIT Technology Review events and MIT Press

    Ad-free website experience

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.