Enhancements to image-processing technologies for colorizing black-and-white images are helping curators divine the colors used by the French artist Henri Matisse on his landmark work Bathers by a River–while the painting was still a work in progress

The tricks deployed by curators could be more widely relevant to other colorizing applications where it’s not obvious what the colors should be in a black-and-white image of a piece of art, or in cases where subtle differences are important and should be highlighted, such as in medical images.

Researchers at Northwestern University used information about Matisse’s prior works, as well as color information from test samples of the work itself, to help colorize a 1913 black-and-white photo of the work in progress. Matisse began work on Bathers in 1909 and unveiled the painting in 1917.

In this way, they learned what the work looked like midway through its completion. “Matisse tamped down earlier layers of pinks, greens, and blues into a somber palette of mottled grays punctuated with some pinks and greens,” says Sotirios A. Tsaftaris, a professor of electrical engineering and computer science at Northwestern. That insight helps support research that Matisse began the work as an upbeat pastoral piece but changed it to reflect the graver national mood brought on by World War I.

The process was more complex than the methods used routinely for colorizing old movies and family photographs. In those kinds of applications, backgrounds such as skies, clothing, and skin tones are “more homogeneous and thus easier to extrapolate,” says Tsaftaris. The color of an entire sky can be determined from a relatively small batch of pixel data, he said. It’s far harder in a black-and-white image of a piece of color art, because “the painter works from a very unique palette of colors that is particular to him, that he sees in his mind,” adds Aggelos Katsaggelos, a professor of electrical engineering and computer science at Northwestern, who collaborated with Tsaftaris.

The researchers made a high-resolution digital version of the 1913 photograph to work from. The photograph itself contained crucial clues to colors and their saturation levels. But to draw a more complete picture, the scientists and their collaborators needed more data.