They took multiple digital photos of Bathers in its current form, going quadrant by quadrant to obtain a resolution of 4,000-by-5,000 pixels. Finally, they included information from historical accounts of what the painting looked like in 1909 and again in 1913, drawing on research by curators at the Art Institute of Chicago.
Finally, they used some sample data from their collaborators at the Art Institute of Chicago: cross-sections of the hidden paint layers on Bathers, obtained by removing microscopic core samples of the painting for spectroscopic analysis.
They applied all of this information to help colorize the photograph, taken by photographer Eugene Druet in November 1913.
And when all of the data sources were combined, it allowed the researchers to transfer colors onto their digital photo of the old photo. The algorithm’s job at that point was to propagate the transferred colors across the entire digital photo, pixel by pixel, to rediscover some of the painting’s 1913 appearance.
“This research is an excellent example of collaborative research between computer science, art conservation, and art history,” says Roy S. Berns, a chemist and color scientist at the Rochester Institute of Technology. “The historians bring their connoisseurship of the artist and their oeuvre. The conservators contribute their knowledge of artist materials and the artist’s working method. The computer scientists facilitate the visualization in a physically realistic way. Because the physical data are sparse, collaboration is required to ensure the result is plausible.”
The effort took three years, and the scientists and conservators say they held back their findings until they reached a 95 percent confidence level about the colorized image.
The algorithm can be tweaked to work with other similar situations and other artists. While this algorithm was “customized to work on paintings and on the particular style of Matisse,” Tsaftaris said, “we can turn off some options, and it works on other paintings as well.”
Tsaftaris sees future applications of custom colorization, particularly in the medical field. The scientists are considering using their new methods to pseudo-colorize grayscale cardiac magnetic resonance images (MRI) to make it easier for doctors to read, analyze, and render a diagnosis. In this case, they might use cues gleaned from color images of diseased hearts, for example, to inform their work in how to properly colorize black-and-white MRI images to bring out the most relevant distinctions.