Skip to Content

As It Searches for Suspects, the FBI May Be Looking at You

Questions about accuracy and transparency plague the bureau’s five-year-old face matching system.
June 23, 2016
Illustration by Daniel Zender

The FBI has access to nearly 412 million photos in its facial recognition system—perhaps including the one on your driver’s license. But according to a new government watchdog report, the bureau doesn’t know how error-prone the system is, or whether it enhances or hinders investigations.

Since 2011, the bureau has quietly been using this system to compare new images, such as those taken from surveillance cameras, against a large set of photos to look for a match. That set of existing images is not limited to the FBI’s own database, which includes some 30 million photos. The bureau also has access to face recognition systems used by law enforcement agencies in 16 different states, and it can tap into databases from the Department of State and the Department of Defense. And it is in negotiations with 18 other states to be able to search their databases, too.

The size of the total pool of photos the bureau can access, which was not clear until the new report from the Government Accountability Office, is shocking even to those who have been paying close attention to the FBI’s growing use of biometric data, says Jennifer Lynch, senior staff attorney at the Electronic Frontier Foundation. And the degree to which the FBI has access to photos in state-owned face image databases, which contain mostly driver’s license images, has Lynch and other privacy advocates concerned.

Deploying face recognition is the “logical next step” in the FBI’s use of biometrics, says Anil Jain, a professor of computer science and engineering and head of the biometrics research group at Michigan State University. The bureau already had an automatic fingerprint matching system, and adding face images to that data will lead to more reliable identification, he says. Surveillance cameras are everywhere, and facial recognition technology has improved to a point where “it makes sense to collect this additional data,” says Jain, who works closely with the Michigan state police.

Jain points out that the FBI’s fingerprint database also contains images taken for noncriminal purposes, like employee background checks. But photos of faces are different because they can be captured covertly, argues Alvaro Bedoya, executive director of the Center on Privacy and Technology at Georgetown Law. “I know what I touch, and I certainly know if I give fingerprints for a background check,” he says. “I don’t think there’s anyone who keeps track of every surveillance or smartphone camera.”

Adding to the privacy concerns is another finding in the GAO report: that the FBI has not properly determined how often its system makes errors and has not “taken steps to determine whether face recognition systems used by external partners, such as states and federal agencies, are sufficiently accurate” to support investigations. By taking those steps, the bureau “could better ensure the data received from external partners is sufficiently accurate and do not unnecessarily include photos of innocent people as investigative leads,” the report concludes.

According to the GAO, the FBI said it does not have the authority to audit the external systems. There is no evidence that the FBI’s face recognition system has ever mistakenly implicated someone in a crime.

State-of-the-art face recognition systems are generally very accurate, says Jain. But poor image quality—in either the images being tested or those in the database—can significantly hinder accuracy. If a lot of time has gone by since an image in the database was taken, that could also make errors more likely. And the larger the number of images in the database, the greater the chance of such errors—either incorrect matches or failure to match photos of people already in the database. This is why it’s so important that human biometrics experts make the final determination as to whether a match is correct, he says.

Keep Reading

Most Popular

Rendering of Waterfront Toronto project
Rendering of Waterfront Toronto project

Toronto wants to kill the smart city forever

The city wants to get right what Sidewalk Labs got so wrong.

windows desktop with anime image from Wallpaper Engine
windows desktop with anime image from Wallpaper Engine

Chinese gamers are using a Steam wallpaper app to get porn past the censors

Wallpaper Engine has become a haven for ingenious Chinese users who use it to smuggle adult content as desktop wallpaper. But how long can it last?

Yann LeCun
Yann LeCun

Yann LeCun has a bold new vision for the future of AI

One of the godfathers of deep learning pulls together old ideas to sketch out a fresh path for AI, but raises as many questions as he answers.

Linux hack concept
Linux hack concept

The US military wants to understand the most important software on Earth

Open-source code runs on every computer on the planet—and keeps America’s critical infrastructure going. DARPA is worried about how well it can be trusted

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.