We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Business Impact

The Algorithm Expanding the Science of Color

There is no natural way to place colors in order, and that makes comparing color palettes difficult. One algorithm looks set to change that.

The color palettes extracted from fine art paintings.

The study of color is undergoing a revolution. Until now, color theory has been little more than a collection of ideas based loosely on science.

But the ability to analyze huge data sets of images using machine-vision algorithms is changing that. Suddenly it has become possible to study color and the way it is used in entirely different ways. For example, it is now easy to extract combinations of colors from one picture and paste them onto another.

But this process of color pasting is a blunt tool. It can replace one set of colors with another but it cannot compare these color palettes, decide how specific colors should be matched, or discover whether a palette is missing a color and replace it.

The problem is that there is no way of placing colors in order. And that makes it hard to compare palettes. So image specialists would dearly love to have a natural way to order colors extracted from images and so be able to compare them.  

Enter Huy Phan at the Technical University of Denmark in Copenhagen and a couple of pals, who have found a way to do just that. Their technique is straightforward to implement and immediately leads to a new generation of enriched image filters that are far more flexible than those currently available.

The basic problem is simple to state: given the color palette from two images, what order should the colors appear in to make a meaningful comparison? By meaningful, the team means that colors describing the same objects should be comparable.

Phan and co tackle this as a kind of sorting problem. They first assess an image by plotting the position of the colors it contains—its palette—in a three-dimensional color “space” (every color can be thought of as a three-vector describing how it is composed from a mixture of red, green, and blue, for example).

They then measure the distance between each pair of colors in the palette. Finally, they use an algorithm to work out how to map one palette onto another with minimal distortion. Effectively, this algorithm finds clusters within each palette’s color space.

The data set they use is important. “We consider color palettes extracted from fine art collections, which we believe to be an abundant source of stylistic and unique color themes,” say Phan and co.

That also limits the subject matter, which is crucial. Henri Matisse often used deep reds and greens in his paintings, for example, while Maximilien Luce used strong blues and yellows to shade objects. But because these artists use broadly similar scenes (for example, divided into sky and land), the colors are comparable.

It means that the team can assume that similar clusters of colors describe similar objects. The algorithm then maps these clusters onto each other. In that way, it can take the coloring of sky in one image and apply the same set of colors to the sky in another image.

And because the transformation is applied to the whole space, it becomes straightforward to map any color from palette onto a corresponding color in the other palette.

That leads to a natural way to order colors. Phan and co do it by first specifying the number of colors that appear in each cluster. Phan and co use five as their standard example (higher numbers require significantly more computational resources).

The algorithm then finds the five colors in the first cluster, then in the second, third and so on. Ordering these clusters then allows the palettes in each image to be easily compared.

The method immediately leads to some interesting applications. For example, they create an application called Photo-style Explorer which acts like an Instagram filter but in a continuous color space. “Instead of choosing among a few predefined themes, one can freely surf a continuous space of possible photo colorizations to pick a preferred theme,” they say.

Another option is to recolor photos with different palettes in different parts of the image. So it is possible to use one palette to recolor the sky and another to recolor trees.

It also allows a better analysis of the color signatures of artists and for these signatures to be transferred to other images. So it becomes possible to quickly recolor a photograph using the Renoir’s color palette or Vincent van Gogh’s.

That’s interesting work which expands the way color can be used in modern images.

Ref: http://arxiv.org/abs/1703.06003: Color Orchestra: Ordering Color Palettes for Interpolation and Prediction

Understand the technologies that will drive the global economy at EmTech MIT 2019

Register now
More from Business Impact

How technology advances are changing the economy and providing new opportunities in many industries.

Want more award-winning journalism? Subscribe to Print + All Access Digital.
  • Print + All Access Digital {! insider.prices.print_digital !}*

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivery each weekday to your inbox

    The MIT Technology Review App

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.