Teaching Robots New Tricks

Continued from page 1

At a recent demonstration at Stanford, an autonomous helicopter used this approach to perform several complicated tricks, including loops with pirouettes and a backward funnel maneuver known as the hurricane. The team was even able to demonstrate a particularly difficult stunt called the tic toc, in which the helicopter hovers with its tail down while its nose swings back and forth like an inverted pendulum. Such a trick had been impossible to perform using hard coding, and it represented an impressive achievement for the team. "We can now trust our helicopter controls a lot more [and achieve] higher-performance flight," says Abbeel, who now works as an assistant professor in the Department of Electrical Engineering and Computer Science at the University of California, Berkeley.

Eric Feron, a professor of aerospace engineering at the Georgia Institute of Technology, was not involved in the Stanford project but is impressed by the performance of the autonomous helicopters trained using the approach. He also appreciates the underlying methodology. "When I was involved in similar research back in early 2000," he says, "there was definitely what I would call human intervention in figuring out what the online control systems should be doing in order to repeat the maneuvers. We had to program the computers ourselves." Feron says the Stanford work represents a significant gain in efficiency, by cutting down the learning time to half an hour. "At the end of our research, we were able to maybe do a new maneuver in one day," he says.

Abbeel notes that while the autonomous helicopters have achieved a new level of reliability, there is room for improvement, and safety will be a key concern if such robots are ever flown over populated areas. The machines have to be able to fly at least as well as an expert human pilot, even while doing complicated maneuvers, he says, and simple back-and-forth flight won't be good enough for search-and-rescue missions. "I like to imagine a future in which someday, if there is an accident out on the ocean, a fleet of a dozen autonomous helicopters can be instantaneously deployed to search for survivors," he says. This could help offset the lack of human pilots qualified to perform such a task and increase the chance of locating survivors.

The learning system could be used on other kinds of robots as well, Ng says, such as those that do housework or work in factories. "It could also allow for the very precise control of cars, motorcycles, fixed-wing aircraft, and even sea-based vehicles," he says.

In the future, the team hopes to make their system more flexible. "When we as humans learn, there are many things that speed up the process besides demonstrations. An expert pilot might give advice in other forms," says Abbeel, such as verbal or written tips. Ideally, the team hopes to design a system that can incorporate such advice.