Inside a Schnucks grocery store in St. Louis, Missouri, the toilet paper and baking ingredients are mostly cleared out. A rolling robot turns a corner and heads down an aisle stocked with salsa and taco shells. It comes up against a masked customer wearing shorts and sneakers; he’s pushing a shopping cart carrying bread.
The robot looks something like a tower speaker on top of an autonomous home vacuum cleaner—tall and thin, with orb-like screen eyes halfway up that shift left and right. A red sign on its long head makes the introductions. “Hi, I’m Tally! I check shelf inventory!” A moment of uncertainty ensues. Tally freezes, sensing the human, and the customer pauses, seeming unsure of what to do next. Should he maneuver around the robot? Or wait for it to move along on its own? After a few seconds, the customer chooses to divert, and heads down another aisle.
Tally carries on taking stock of Ritz crackers, tuna fish cans, and nutmeg. Customers—some wearing gloves, a few choosing to shop maskless—are unfazed by its presence.
What seemed a little strange to shoppers when Tally arrived a year ago is now, mid--pandemic, not even close to being the most unusual thing happening inside the store. The robot has become part of the backdrop, posing far less threat than other shoppers and arousing much less concern than more pressing topics such as personal safety, possible meat shortages, and when the next shipment of Clorox wipes might arrive.
Such machines are not just at grocery stores. Roboticists at Texas A&M University and the Center for Robot-Assisted Search and Rescue recently surveyed over 120 reports from around the world about how robots were being used during the covid-19 pandemic. They discovered them spraying disinfectants, walking dogs, and showing properties for real estate agents. But where they may be doing the most to save lives is in hospitals, helping with things like disinfection, patient intake, and delivery of supplies.
Life inside a covid-19 ward looks like this: tubes running through windows sucking out contaminated air, coronavirus patients lying inside “isopods” (plexiglass boxes placed over beds to prevent contamination), and nurses in goggles, caps, gloves, masks, and disposable gowns, cautiously administering medicine, providing care, and holding up iPads for family members not allowed in.
Here’s where Moxi steps in. So far, the health-care robot, which was already working at two hospitals in Texas before covid-19 hit, has been
delivering lab samples, intravenous pumps, medications, and protective gear during the pandemic. But it has not yet been put to work inside critical care, intensive care, or covid-19 units. The outbreak has compelled Moxi’s creators, Diligent Robots of Austin, Texas, to think about how it could help there too.
“If we can find ways for more dangerous activities to be automated, then we should.”
In May, Vivian Chu, one of the company’s founders, introduced me to her invention over a video call. Cloud-white, with a barrel-like torso, Moxi is a blend of cute and not too creepy. It has a camera on its moving head, which can turn, but not a neck-breaking 360 degrees, since that would feel weird to anyone watching. Its eyes are bursts of warm blue light—they can turn into softly glowing pink hearts at the right moment—and it rolls along on wheels, with a robotic arm that waves almost cheerfully to passersby. Moxi is very deliberately unimposing. As Chu, who is 5'4" (163 cm), talked to me from her company’s lab, she stood a few inches taller than the robot next to her, although she did explain that it can adjust its height, growing taller if a task requires.
For the most part, Moxi acts like a mechanical waiter. Inside its body, it can carry a tray of “lock tubes” that hold medications or supplies placed there by medical workers. Moxi’s headband turns red if it is locked, green if unlocked.
Moxi does not carry on conversations but makes adorable “meeps” while working, said Chu: “Very R2-D2. Different noises to convey if the robot is happy that it successfully delivered or upset because it opened something incorrectly.”
The designers put a lot of thought into creating a robot that is personable, like a teammate, Chu explained. Not too human-like, “but at the same time not like a toaster in the corner that you don’t care about.”
Chu and her cofounder, Andrea Thomaz, are experts in social robots, and their long-term vision has been to help frontline health-care workers. They’d already spent two and a half years with nurses—shadowing them, interviewing them, and watching them interact with patients. They saw how many nurses were being forced to run errands like fetching supplies and medicine instead of spending their time on face-to-face patient care.
Thomaz remembers one nursing assistant in Austin who set down her cup of coffee at the beginning of her shift and never touched it again, because she was so busy. “We would shadow them for entire shifts, and you realize 12 hours is a very long time to be on your feet,” she said.
When some medical staff realized that Thomaz and Chu were designing robots for hospitals, their first reaction was one of suspicion. “Wait, you want to develop a robot to do our job?” Thomaz recalls being asked.
“The robot can’t be a nurse. It’s not going to be a nurse,” says Chu. “But what it’s perfect for is going in and helping relieve the nurse that is so overburdened.”
When covid-19 overwhelmed hospitals in the states of Washington, New York, and New Jersey, “it really felt like a rallying call,” says Thomaz. “Nurses have always been a part of our mission. We just looked at each other like ‘Wow, they really need help more than ever.’”
Russell Taylor, head of the Laboratory for Computational Sensing and Robotics at Johns Hopkins University, says the need for robots will spread beyond nursing to intensive care units, surgeries, and home health care. When the pandemic hit, his lab began working on a small, inexpensive robot that could help in patients’ rooms.
“Oftentimes the nurse has to go in there just to hit a few buttons on a ventilator,” says Taylor. That requires wearing full protective gear, so some hospitals are running infusion pumps that they can operate from hallways outside patient rooms. Instead, says Taylor, a robot could go in.
Thomaz and Chu are now talking with hospitals about how robots could best help clinical staff, such as by performing riskier tasks in patient rooms or delivering lab samples. Robots could also take on cleaning and disinfecting. This would free up nurses for more important work like holding the hands of ill patients. “If we can find ways for more dangerous activities to be automated, then we should,” says Thomaz. “That’s what robots are for.”
But while robots may be useful to frontline workers in hospital wards and medical centers, they could more directly threaten the livelihood of others.
Brian Tieszen has loved robots ever since he was a kid. He’s a serious Star Wars fan, and now a single father with two kids of his own. His fascination with R2-D2, empires, and futuristic realities followed him into adulthood, and in 2000 he earned an associate’s degree in electronics. In 2014 he joined Amazon, an exciting opportunity he thought could be the beginning of a lifelong career. At first, he worked the night shift at a warehouse an hour away from home—it was a good job, but he barely saw his kids. Then, in 2016, he heard about a new, robot-filled facility opening in Eastvale, California, much closer to his home, and applied for a transfer straight away.
He was there for Eastvale’s official launch day. New employees posted smiling photos on social media, high-fiving as the warehouse opened for business. To celebrate, Tieszen and other employees autographed three orange robots.
Tieszen started out unpacking trucks full of items like televisions and barbecue grills, and worked his way up to training new hires. He worked hard. “I was really good at what I did,” he says, “and really fast.” As he quickly realized, the robots—rolling devices that navigate on their own virtual highway system carrying shelves of goods—were more like giant, trundling trays than futuristic droids. Inside the warehouse, they moved around with monotonous rigidity, carrying tubs of wrapping paper, ribbons, and shampoo. They were separated from human workers by metal fences, with yellow tape warning of the dangers of crossing the line, as if at a crime scene.
At 6'1" and heavyset, wearing size XXXL Star Wars T-shirts, Tieszen is a refrigerator of a man. But inside the Amazon warehouse, he was a speck. One day, six months into the Eastvale job, Tieszen was tasked with unloading books from a pallet as tall as he was. He spent eight hours bending over, putting away book after book. At one point he felt his back buckle, and by the end of the shift, he could no longer stand. Tieszen ended up with two herniated discs. He spent months on bed rest and has still not fully recovered. “Bezos,” he says, referring to Amazon’s founder. “We’re all like his little storm troopers.”
Tieszen found a lawyer, Brian Freeman, who has represented 72 clients from Amazon. “They are reaching down for boxes all day,” Freeman explains. “Bending in ways they are not used to, and all of a sudden, bam, their back is killing them and they can barely move.” Often it’s the wear and tear, a constant grind. Most humans, he adds, are not built to sustain that kind of physical demand. The Amazon employees, Freeman says, are like “human robots.”
The actual robots at Amazon, with names like Kiva, Pegasus, and Xanthus, already do carry many of the heavier loads. According to Amazon, they make the warehouses more efficient and the workers’ jobs safer and easier, and allow the company to pay higher wages. Future robots could free up human workers from tasks more likely to injure them.
But the pandemic may change this calculus. Before covid-19 hit, many companies—not just in logistics or medicine—were looking at using robots to cut costs while protecting humans from dangerous tasks. Today humans are the danger, potentially infecting others with the coronavirus. “Now the challenge is that a minimum-wage laborer might actually be a carrier,” says Henrik Christensen, director of the Contextual Robotics Institute at UC San Diego.
This makes human labor, increasingly, a liability. As online orders have ballooned, Amazon has hired 175,000 new workers. Labor activists and employees have demanded protective gear, warehouse disinfection, more time off, higher pay, and testing. Amazon won’t say how many of its employees have been infected with or died from covid-19, but it and other companies have a clear incentive to replace more workers with robots permanently. After all, robots don’t need face masks, health care, or social distancing, and they don’t go on strike for better conditions.
“Entry-level, unskilled-labor jobs are going away because of robots.”
This shift means that one day soon, maybe, robots could not just check inventory in grocery stores but clean floors and stock shelves too, leaving humans only for the more complex tasks. “You will see robots doing cleaning at hospitals at a level much higher than we’ve seen before,” says Christensen. “I would love to have my grocery store being disinfected once a day, so I know it’s not contaminated. I don’t think the cruise ship industry can reboot unless they find a way of doing cleaning in a very different manner than they did before.”
That means today’s “essential workers”—the people who deliver goods, work at store checkouts, drive buses and trains, and process meat at packing plants—could be replaced by machines even sooner than they would have been before the pandemic. Without job protection or access to retraining and education, they’re not only risking their lives to keep the economy afloat; they risk losing their livelihoods as it recovers.
Some of those people, Christensen predicts, will be able to get work helping the robots that replaced them: “There will be a number of new jobs where these robot wranglers will help robots do things still hard to do with software and artificial intelligence.”
Eighteen miles from Amazon’s Eastvale warehouse where Brian Tieszen used to work is the Industrial Technical Learning Center, or InTech. It’s a training center in Fontana, California, where students are preparing for the day when robots become mainstream workers. “Yeah, the robots are taking some of the jobs,” said instructional assistant Steve Ward, when I visited before the coronavirus pandemic hit. “But things change.”
Ward tells his students not to be in the jobs that robots steal. “You want to be the guy that fixes the robot,” he tells them. “That’s job security. And that’s good money.”
At the training center, students learn to operate a robotic system while stationed at one of the central machines. “Really, we are doing all that control in one little brain,” Ward said, standing in his short-sleeve shirt, jeans, and sneakers before a tangle of machinery with brightly colored buttons, knobs, switches, lights, and wires. He gestured to a blue control box the size of a briefcase.
In the mechatronics curriculum, students are trained to program a robot to know the difference between, say, an acrylic block and an aluminum block. They can tell it to detect watermelons or water bottles coming down a conveyor belt. “If this goes down in a big factory, you’re talking thousands of dollars an hour in loss of production,” Ward said. “There is somebody behind that robot making a good living.”
Not everyone is cut out for a university, Ward added, or wants to get saddled with student loan debt. But this emerging profession can pay well, and workers can often take classes for free thanks to grants or company contracts. “It’s not four years of college away, that’s for sure,” he said.
Ward moved to a machine that looked like a yellow metal arm a few times bigger and bulkier than his own. “In this case,” he said, “the robot just will pick the parts up and move them from station to station when it’s not feasible to do it some other way.”
Ward explained that he had seen an Amazon prototype robot during a recent visit to a manufacturer. It looked similar to the yellow robotic arm, except “theirs has vision.” Ward said he watched six testers toss addressed envelopes at it. “As these people are throwing things, this creepy robot is picking things up, and turning them over, and looking at them, and putting them away. It read each bar code, and each address, and put everything in the right spot.” Even for a robot guy like himself, Ward said, “it’s a little weird to watch.”
But will there be enough new robot-keeper jobs to make up for all the losses? What happens as robots become increasingly sophisticated and less reliant on human guidance?
A report from Oxford Economics last year estimated that 20 million global manufacturing jobs could be lost to automation by 2030, 8.5% of the worldwide total. It’s clear already that “entry-level, unskilled-labor jobs are going away because of robots,” said Jon Fox, who coordinates workforce training through a local community college at InTech. “Those are the sorts of jobs most people don’t want to stay in for their entire life.” The people who can retrain as robot wranglers might end up making better money in the long run.
But not everyone will. Aging workers who don’t want to go back to school, people who can’t take the time to retrain for a new field, or those who just don’t have the physical or mental wherewithal to become robot fixers could end up being left behind.
The pandemic may forever change the way we work and shop. We don’t know exactly what the outcome will be: there is no algorithm that can tell us exactly how people will end up faring alongside robots like Moxi or Tally. But tomorrow won’t remain cloudy forever.
For the founders of Diligent Robotics, the problem isn’t having enough operators—it’s time. The most frustrating part of the pandemic has been knowing that Moxi could step in to help more than it’s already doing. Its design is ready. But the robots are still built on demand, and it takes time for the technology to get oriented to a new location: maps and sensors help it integrate into the workflow, but that requires programmers to spend time on site. Launching a robot workforce in the middle of a pandemic is not ideal, Thomaz says—not with hospitals in survival mode.
So they are looking to a future where medical-assistant robots are on the rise. They recently raised $10 million for their projects and plan to roll out more hospital robots in the next year and a half. “We could have them up and running a few months from now, maybe at the tail end of this pandemic,” Thomaz says, “but really we are thinking about being ready for the next one.”
A horrifying new AI app swaps women into porn videos with a click
Deepfake researchers have long feared the day this would arrive.
The therapists using AI to make therapy better
Researchers are learning more about how therapy works by examining the language therapists use with clients. It could lead to more people getting better, and staying better.
DeepMind says its new language model can beat others 25 times its size
RETRO uses an external memory to look up passages of text on the fly, avoiding some of the costs of training a vast neural network
AI fake-face generators can be rewound to reveal the real faces they trained on
Researchers are calling into doubt the popular idea that deep-learning models are “black boxes” that reveal nothing about what goes on inside
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.