Many of the vulnerabilities that the researchers used to test the system had been rated as serious or critical by Microsoft, Brumley says. While in two cases, Microsoft had already issued warnings for the exploits that the researchers generated, in several other cases, they created exploits that were previously unknown.

"If you just look at it naively, you are distributing a patch for the betterment of the system, closing security holes," says Dawn Song, an assistant professor at the University of California, Berkeley, who was also involved in the research. "But the point of the work is that, even in such situations, you also need to carefully consider the security ramifications."

As a result, the researchers call for new methods for distributing patches that could make them more secure. Brumley suggests taking steps to hide the changes that a patch is making to the software, releasing encrypted patches that can't be decrypted and activated until a large portion of users have downloaded them, or exploring peer-to-peer distribution methods that could allow patches to go out in a single wave rather than in stages. "I'd like to see researchers get together with vendors to find out what their requirements are to make new solutions work," he says.

Gkantsidis agrees that changes should be made to patch distribution, but he says that further research is needed to ensure that those changes don't introduce new problems. For example, he says, while peer-to-peer distribution has the potential to help distribute a patch quickly, it could also make it easier for attackers to figure out which systems remained vulnerable. He suggests combining the new approaches, such as by both encrypting patches and using peer-to-peer distribution.

However, Bruce Schneier, chief security technology officer at BT Counterpane, says that, while it's interesting that the researchers have demonstrated this capability, he doesn't see that it changes anything. People know that you can reverse-engineer an exploit from a patch, he says, and this research simply shows how easy the process can be. "I think you just have to live with the fact that when you release the patch, the exploit is known," he says. "That's just the way the world works." People can try to make reverse engineering harder, he says, but they can't stop it altogether.

Song hopes that the automated techniques she's developed to generate attacks can also help defenders. By improving the tools for automatically analyzing software code, Song hopes that it will eventually become possible to make programs more secure.