Delivery by drone may be legal within two years. Just don’t expect many pizzas or packages to wing their way through your neighborhood by then. Despite huge interest in unmanned aerial vehicles (UAVs), and considerable hype around the idea of using them to deliver goods, experts say significant challenges still need to be solved for drone delivery to get off the ground.

Google and Amazon are leading the development of delivery drones, while UPS, FedEx, and a host of startups are also researching the technology. Earlier this month, the U.S. Senate Transportation Committee drafted a bill that paves the way for regulation of delivery drones within two years. But the technology may need much more time.

While some critics have raised concerns that drones carrying valuable cargo could be shot down or stolen, those issues are trivial next to questions about reliability, autonomy, and coördination with other aircraft. “You have to assume they’ll fall out of the sky,” says Nicholas Roy, a professor at MIT who worked with Google on its drone delivery effort, called Project Wing. “So how do you make sure these vehicles are reliable enough—both the hardware and the software?”

Roy says drone delivery could work in niche situations where safety isn’t a concern, such as on an oil rig, but that wider commercialization will require several problems to be solved. Chief among these is how to navigate through an unfamiliar and unpredictable urban environment. The challenge is comparable to the one facing self-driving cars but even more complicated, given that drones would need to travel over and between buildings and then negotiate an unfamiliar landing spot.

“There are some major technical challenges there that people are working on right now, in terms of manufacturing processes, aerodynamics and stability, materials and structures, and especially in terms of the autonomy,” Roy says.

Google unveiled project Wing in a promotional video in 2014. Around the same time, Amazon showed off its own drone, a conventional multi-rotor aircraft, developed through a project called Prime Air.

The Google clip shows the aircraft delivering dog food to a farmer in the Australian outback. Though an impressive technical feat, the demonstration highlights the fact that drones cannot yet be flown over populated areas.

Indeed, as it stands, drones need to be kept on a tight leash in U.S. airspace. The Federal Aviation Administration prohibits commercial drone flights, although hundreds of companies, including Google and Amazon, have received exemptions in order to develop and test drone technology. But those waivers still require that a drone is not flown without a pilot controlling it and in visual contact, and does not go above 400 feet or within five miles of an airport.

Enthusiasm and experimentation may now be getting ahead of practical concerns. A report issued by the FAA on Friday shows that over the past several months, U.S. pilots have spotted an alarming number of drones buzzing through the sky. A total of 582 were spotted from the cockpits of regular aircraft.

The FAA provided a statement about ongoing testing of delivery drones. “Amazon and Google have regulatory exemptions allowing them to do research and development on their concepts,” the statement said. “Those concepts are still in their preliminary stages […] and a detailed technical review will take time.”

Some entrepreneurs aim to develop technology that could make drone delivery more realistic. A startup called Skydio, for example, is working on autonomous navigation system for drones, which is challenging given the weight constraints.

The idea of drone delivery “ties together many of the hardest problems in the industry: navigation, flight range, ultrahigh reliability, and airspace management,” says Adam Bry, founder and CEO of Skydio. “Eventually all of these things will be solved, but the economics compared to autonomous ground delivery may not turn out to be favorable for many situations.”

Missy Cummings, a professor of mechanical engineering and director of the Humans and Autonomy Lab at Duke University, says drone delivery could perhaps be designed to overcome some of those remaining challenges. She suggests that drone operators could partner with local businesses that could offer safe, controlled landing spots. 

“Perhaps Starbucks could be your intermediary point,” Cummings says. “You’re not really going to deliver to everyone’s home. Do you want drones to land in a backyard with a dog?”