Select your localized edition:

Close ×

More Ways to Connect

Discover one of our 28 local entrepreneurial communities »

Be the first to know as we launch in new countries and markets around the globe.

Interested in bringing MIT Technology Review to your local market?

MIT Technology ReviewMIT Technology Review - logo


Unsupported browser: Your browser does not meet modern web standards. See how it scores »

{ action.text }

Put to the Test

Collaborative approaches are also transforming the way software is tested. Traditionally, testing has been a two-step process. First, programmers write code based on requirements. Then a separate group tests the result. But when software includes millions of lines of code, the two-step process is analogous to designing an automobile on paper, building it, then checking to see if the design worked.

That approach just didn’t work for developers like Nicholas Stamos, chief technical officer of Waltham, MA-based Phase Forward. Phase Forward develops software for running pharmaceutical clinical trials, and its products must pass the stringent requirements of the U.S. Food and Drug Administration.

Stamos says Phase Forward meets those requirements, in part because of close and constant collaboration between the company’s programmers and quality assurance personnel. “You can’t throw it over the transom and hope,” he says. “Quality has to be built in from day one.” When programmers complete a component, the quality team checks their work. Then the teams swap places for another round of bug fixes. Finally, the integrated application is tested to find problems with the interaction between components. Instead of the old two-step approach, testing becomes an interactive, ongoing process.

Some companies are trying a similarly iterative approach not just to testing, but to designing code in the first place. Cognizant Technology Solutions, a consulting firm based in Teaneck, NJ, has found that existing methods of starting with formal specifications and then writing code to satisfy them might have worked 20 or 30 years ago. But in a world that runs on the Web and needs flexibility, formal can mean stiff.

“When we tried to use the traditional processes for these newer types of projects, either the customers got really frustrated because we forced them to freeze the process or the project teams would just not follow the process,” says Cognizant CEO Kumar Mahadeva. Instead, projects use a series of prototypes followed by an “industrialization” cycle, letting the firm handle sudden changes from customers in a way that has less impact on reliability than adjusting a program after it is completed. Future users provide a steady stream of feedback as the application takes shape.

But the ultimate limits on software quality may not lie in the skills of the programmers, or the strength of the development process. “This is the conundrum that we and many manufacturers face today: what is an acceptable level of quality?” says James Hymel, director of software engineering at Motorola. “If you have competitors turning out cute, cheap, but buggy [products] and the customers say that is acceptable-that is, they spend their money on them-you’d be challenged.”

With methods like agile development, partnering of development and testing, and the involvement of end-users, software developers are taking steps to improve their product. But at the end of the day, the greatest constraint to quality may be the same people demanding it the loudest: the consumers.

0 comments about this story. Start the discussion »

Tagged: Communications

Reprints and Permissions | Send feedback to the editor

From the Archives


Introducing MIT Technology Review Insider.

Already a Magazine subscriber?

You're automatically an Insider. It's easy to activate or upgrade your account.

Activate Your Account

Become an Insider

It's the new way to subscribe. Get even more of the tech news, research, and discoveries you crave.

Sign Up

Learn More

Find out why MIT Technology Review Insider is for you and explore your options.

Show Me