Scientists studying a colony of rare penguins on a remote South African island are using sophisticated object-recognition software to identify and track individual animals–an approach that they believe could transform conservation fieldwork.

The software involved–originally developed for recognizing individual human faces–has developed rapidly in recent years. But so far, the so-called Penguin Recognition Project, run by Bristol University, in England, is the first large-scale attempt to use this technology to catalogue and monitor an entire population of animals in the field.

Robben Island is home to roughly 20,000 African penguins, a threatened species that has declined by 90 percent in the past century. Bristol scientists set up several cameras along paths well-traveled by the penguins. Software picks up the penguins’ fingerprint-like patterns of black and white feathers, and uses these patterns to identify individual animals. By tracking individual penguins through time, scientists can learn how long they live, how frequently they’re reproducing, and what times of year they are most vulnerable.

“There are quite a lot of people working on computer vision to try to identify objects in images,” says Peter Barham, a physicist and one of the project’s chief researchers. “No one has applied that technology to looking for animals.” He adds that the approach can potentially be used to track any animal that has individually distinct visual patterns. Many animals, from humpback whales to giraffes, have individually distinct patterns of coloration.

Conventional animal population-monitoring techniques are typically very costly and difficult, and inflict stress on animals. Most population biology studies are done by capturing animals and tagging them, or by following and photographing individuals so that they can be catalogued by hand in visual databases.

The penguin project does the same thing without these downsides and gets better data too. “We can work out monthly survival rates for penguins. Not annual–monthly,” Barham says. “You get a frightening amount of data.” Right now, the Bristol scientists are installing a system that will permanently monitor the entire island, following the success of prototypes tested over the past four years.

This week, the project’s scientists are presenting their research at the British Royal Society’s annual summer science exhibition in London.

The penguin-recognition software uses a learning algorithm that gets better the more data it encounters. Using a large collection of penguin photographs, Bristol University computer scientist Tilo Burghardt “taught” the software to identify a penguin-shaped object by its chest outline and stripe, a black band with a characteristic shape. Individual penguins are recognized by the unique patterns of spots on their chests, each of which is described in the system by its distance from all other spots. The detector is robust enough to correctly identify individual penguins even when one or more of the spots are covered, says Burghardt.

Object-recognition software adds green boxes when any penguin is recognized and yellow boxes when a specific penguin is recognized.
Credit: Tilo Burghardt

“You can encode the animal pattern even more effectively and efficiently than human faces,” he says. “You don’t have to use a lot of description to make the system work.”

The system uses fairly cheap components: ordinary security cameras connected to laptops, which communicate via a wireless LAN. With a power source and a connection to stream the data to a central server, it operates in the field with minimal human interference. In a month of observation, Barham says, the system will capture data on almost the entire colony.

The challenge in generalizing this approach to other species, of course, is in simply collecting the images efficiently. For wide-ranging species that don’t travel along well-used paths, passive cameras won’t capture enough images to track an entire population.

But even for mobile animals that can’t be photographed passively, object-recognition software can take the place of the painstaking work of hand-matching images, a job that takes great expertise and eats up limited conservation research budgets. Sophie Grange, a zebra biologist at Wits University, in South Africa, is optimistic about the technology’s potential, and she is currently working with Burghardt and his colleagues to develop a similar system for her fieldwork. “These studies are essential to improving our scientific knowledge on animal demography, which is central if you want to manage and preserve animal populations,” she says.

Burghardt thinks that the field of conservation biology is ripe for technological innovation. “It took a long time to realize you can use similar technology to solve seemingly very different problems,” he says. “We’ve basically opened up a new field of collaboration between science and engineering.”