A Self-Driving Bus That Can Speak Sign Language
It’s been 15 years since a degenerative eye disease forced Erich Manser to stop driving. Today, he commutes to his job as an accessibility consultant via commuter trains and city buses, but he has trouble locating empty seats sometimes and must ask strangers for guidance.
A step toward solving Manser’s predicament could arrive as soon as next year. Manser’s employer, IBM, and an independent carmaker called Local Motors are developing a self-driving, electric shuttle bus that combines artificial intelligence, augmented reality, and smartphone apps to serve people with vision, hearing, physical, and cognitive disabilities. The buses, dubbed “Olli,” are designed to transport people around neighborhoods at speeds below 35 miles per hour and will be sold to cities, counties, airports, companies, and universities. If the buses enter production in summer 2018, as planned, they could be among the earliest self-driving vehicles on U.S. roads.
Since Olli is fully autonomous and does not have a human driver, it uses IBM’s AI-powered Watson technology to converse with passengers (via voice and text displayed on an iPad). Olli navigates using radar, lidar, and optical cameras from a company called Meridian Autonomous. Before deploying in a neighborhood, Meridian Autonomous constructs 3-D maps of the area that Local Motors says are accurate to the half-inch. A human fleet manager then determines the bus route. When Olli detects an emergency via its various sensors, it will stop, notify a (human) remote supervisor, and independently run through a checklist of possible problems. “If a passenger has a medical problem or [there’s a safety issue], Olli will call the authorities or drive itself to a hospital or police station,” says Gina O’Connell, a Local Motors general manager who is leading the project.
Local Motors and IBM started collaborating on Olli in early 2016 and produced a first iteration of the bus in June 2016. That vehicle is currently in trials in Germany and Switzerland. It is the next—second—generation of Olli that will include assistive technologies. That version, which the companies call “Accessible Olli,” will be manufactured starting in 2018, and will retain Watson as a tool for communicating with passengers and add additional Watson features.
Local Motors and IBM are still testing technologies, but have already identified some capabilities they are likely to add. Future Ollis, for example, might direct visually impaired passengers to empty seats using machine vision to identify open spots, and audio cues and a mobile app to direct the passenger. Olli could also guide passengers via a special type of haptic feedback that uses ultrasound to project sensations through the air. An array of haptic sensors could be designed into every seat, and when people walk down the aisle they would feel a vibration on their hand or arm to alert them that they were at an empty seat, explains Drew LaHart, the program director for IBM’s accessibility division.
For deaf people, the buses could employ machine vision and augmented reality to read and speak sign language via onboard screens or passengers’ smartphones. LaHart says that Olli could be trained to recognize sign language using machine learning and Watson’s image recognition capabilities. If the bus were equipped with AR technology, it might be able to respond via a hologram of a person signing.
Machine vision could also enable Olli to recognize passengers waiting at bus stops who have walkers and wheelchairs. The bus would then activate an automated ramp to help them board and then deploy equipment that would secure their assistive devices, locking a wheelchair into place, for example.
Another potential Olli technology combines machine vision and sensors to detect when passengers leave items under their seats and issues alerts so the possessions can be retrieved, a feature meant to benefit people with age-related dementia and other cognitive disabilities.
This would all be a significant improvement over the typical bus accommodations of today, which are limited to wheelchair ramps and lifts and audible and visual bus route updates. Local Motors, IBM, and the CTA Foundation, the charitable arm of the Consumer Technology Association, a trade group for the consumer electronics industry, and a partner in Accessible Olli, have spent the past three months soliciting ideas from disability rights organizations and retirement communities, among others. Manser, who works for IBM Accessibility, has organized a workshop with blindness organizations and public transit agencies and attended an MIT assistive technologies hackathon in March to explain the challenges he encounters on public transportation.
Local Motors plans to keep soliciting public input for several more months. In July, it will devise an engineering plan for the new version of Olli, select suppliers, and calculate the cost of fabricating the bus. It aims to sell the vehicle for about $250,000 and will also offer a leasing-subscription service that would cost $10,000 to $12,000 a month and include hardware upgrades. Because Olli is mostly manufactured on-demand, through 3-D printing, its design can be tweaked quickly in response to user feedback, says O’Connell.
The company expects public transportation operators will be its main customers and hopes that cities will buy the buses to fill in gaps in their regular transit systems and not just as paratransit vehicles for disabled people.
For those with disabilities, though, Olli could be a big improvement over the current options. Door-to-door paratransit service tends to be slow, has to be scheduled ahead of time, and is only available to people who qualify for it, says Henry Claypool, who is the policy director of the Community Living Policy Center at the University of California, San Francisco, and a wheelchair user. “It’s much more reliable to be able to get on and off a bus at the same place and have a predictable schedule, especially if the bus has this type of assistive technology,” he says.
Olli offers a way to address important limitations of public bus and train systems as well, says Susan Henderson, the executive director of the Disability Rights Education and Defense Fund. The Americans with Disabilities Act mandates only that “key” train and subway stations be accessible, which means that people with wheelchairs, walkers, and scooters often have to travel several stops out of their way to get home or to a destination, says Henderson. “If I still had 10 blocks to go after getting off at my local station, having an Olli rolling around my neighborhood would make a big difference,” she says.
How Rust went from a side project to the world’s most-loved programming language
For decades, coders wrote critical systems in C and C++. Now they turn to Rust.
The inside story of how ChatGPT was built from the people who made it
Exclusive conversations that take us behind the scenes of a cultural phenomenon.
Design thinking was supposed to fix the world. Where did it go wrong?
An approach that promised to democratize design may have done the opposite.
Sam Altman invested $180 million into a company trying to delay death
Can anti-aging breakthroughs add 10 healthy years to the human life span? The CEO of OpenAI is paying to find out.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.