At a recent technology competition hosted by Microsoft, two students from Drexel University demonstrated a program they have developed that makes it possible to collect public health data on smart phones. But Bill Buxton, a software design specialist at Microsoft, didn’t just want to hear how the program worked. He wanted to know how the information would ultimately be used to help sick people.

“I don’t care about data,” Buxton said. “How does the architecture lead to what to do next?”­

This question gets at something that Buxton sees as an urgent challenge for product designers. It’s not enough to merely build something that is faster, smaller, or cheaper than what came before, he argues. Instead, designers need to think carefully about what kinds of innovations to pursue—and in his view, they should be things that improve quality of life. For Buxton, that might be anything that reduces complexity and is attuned to human needs, whether it’s a new multitouch interface that is well suited to the human body or a computing network that seamlessly recognizes and incorporates new devices.

Trained as a musician, Buxton stumbled into computer science and the subspecialty called “human-computer interaction,” or HCI, when he began designing digital musical instruments in 1975. Later, in the decades before the advent of the iPhone, he was designing programs and devices that people directly controlled with two hands. He wanted to go beyond mouse and keyboard to reflect how people naturally move and communicate. For instance, in the 1980s he and collaborators at the University of Toronto built a prototype multitouch tablet that sensed not only where the users’ fingers were but also how hard they were pressing on the screen.

Buxton, now 62, never completed a PhD, but his academic and professional credentials compensate: among other things, he has been an adjunct faculty member at the University of Toronto, a research scientist and consultant for Xerox’s PARC and its branch in England, and chief scientist of Alias/Wavefront and Silicon Graphics, both now part of the 3-D software company Autodesk. His work has helped define the field. Brad Myers, a computer science professor at Carnegie Mellon, studied under Buxton in the 1980s and took inspiration from the way he tried to improve image rendering in Alias’s software and to develop more intuitive controls for the software’s users. Myers has spent his own career on tools that help interactive designers express their ideas without having to learn computer programming.

Scott Klemmer, a professor and codirector of the HCI group at Stanford University, says Buxton’s research on “graspable interfaces,” which let people use Lego-shaped bricks to manipulate things on screen, informed his own graduate work. Today, Klemmer and his students are trying to build better software tools based on what he calls the psychological and social understanding of design. Their efforts draw on research Buxton published in 2006, which showed that people are more willing to give critical feedback if they are shown more than one prototype design at a time.

“I think he’s been probably one of the most influential people in the field of human-computer interaction, period,” Klemmer says. “I am amazed that no matter what I do, one of my go-to references is something that Bill did before.”

Buxton joined Microsoft Research in 2005, and his mandate, he says, was simply to “make a difference.” He spends about a third of his time working on products in Microsoft’s commercial pipeline, acting as a design consultant on projects that need his expertise with touch and gesture recognition. Among them are the most recent generation of Windows smart-phone software and Kinect, the video-game controller that understands voice commands and body language.

He has also tried to change how Microsoft operates, so that engineering and business concerns do not come before “humancentric” design considerations. In some cases, Microsoft computer scientists are now being joined on an equal footing by social scientists, industrial designers, and others in what he calls “renaissance teams.”

“Microsoft, along with any other company that wants to survive in this world, has to start saying the technology of the human—the technology of culture, if you want to use that bizarre terminology—is as important as the technology of silicon,” he says.

But even with Buxton on board, Microsoft has learned that the process of getting design right can be humbling.

One project Buxton has helped with is Microsoft Surface, which debuted a few years ago as an extravagantly expensive, coffee-table-size computer with a multitouch screen—a real Frankenstein’s monster compared with the iPhone that Apple would soon release. The machine was, by Buxton’s account, hacked together Rube Goldberg style with separate cameras and projectors, and marketed to a very small audience while the company worked on better technology and slicker software. Finally, after five years, the company released a new, thinner model whose screen doubles as a camera and a display, a technology Buxton says he has been dreaming of for more than 20 years. In fact in the early 1990s he built something called Active Desk, which was essentially a primitive Microsoft Surface on a drafting table.

Now he says the beating Microsoft took in the press over Surface worked in the company’s favor, because it made people discount the potential for the product and gave Microsoft the freedom to develop the idea in plain sight.

“In general, when you see companies that really are innovating, if you look back—this is what we don’t do well in our industry—you’ll say, ‘Oh my God, they were telegraphing from day one what they were going to do, and none of us saw it,’” Buxton says. “In the end, we’re not quite the boobs that some people thought we might have been.”