Picture This

Image-processing software is a hot commodity. But digital-photo files are getting so big that without some clever engineering, processing them would take a painfully long time. The tricks that engineers use to speed up their image-processing algorithms, however, make their code almost unreadable and, often, impossible to run on more than one type of machine.

MIT researchers aim to change that with a new programming language called Halide. Halide programs are easier to read, write, and revise than programs written in a conventional language. And because Halide helps automate code-optimization procedures that would ordinarily take hours to perform by hand, they’re also significantly faster.

The researchers used Halide to rewrite several common image-processing algorithms whose performance had already been optimized manually. The Halide versions were typically about one-third as long but offered significant performance gains—increasing speed by a factor of two, three, or even six.

Image processing generally requires a succession of discrete operations—default procedures like color correction, contrast enhancement, and error correction, and user-specified steps like red-eye correction or shadow softening.

Distributing chunks of an image to different cores in a multicore chip can improve processing efficiency, but the time it takes to aggregate data after each step can offset those gains. So software engineers try to keep individual cores busy for as long as possible before aggregating results. Tracking the dependencies between operations being performed on separate cores makes the code for image processors dreadfully complicated.

Halide doesn’t spare the programmer from thinking about how to distribute tasks among cores, but it detaches that problem from the description of the tasks themselves. A Halide program has two sections: one for algorithms and one for the processing “schedule.” The schedule can specify, among other things, the relationships between steps executed on different cores.

A programmer who wants to add a new processing step to the pipeline just inserts a description of it in the algorithm section. “When you have the idea that you might want to parallelize something a certain way or use stages a certain way, when writing that manually, it’s really hard to express that idea correctly,” says Jonathan Ragan-Kelley, a graduate student who led Halide’s design with CSAIL postdoc Andrew Adams. “If you have a new optimization idea that you want to apply, chances are you’re going to spend three days debugging it because you’ve broken it in the process. With this, you change one line that expresses that idea, and it synthesizes the correct thing.”