We Got a New Nine
Breakthrough and Bubble, Simultaneously
In 2015, Chris Urmson said that his goal was for his 11-year old son to never need a driver’s license. The field of self-driving cars was filled with energy and excitement. During that period, we saw many new capabilities as self-driving cars progressed from navigating highways to city streets, as Waymo drove its first 100k miles, as human interventions per hour began to drop. Many startups were founded - Cruise, May Mobility, Optimus Ride, Aroura, and more.
Self-driving cars have reached a new level of capability and robustness. For the first time, you could call a car in Mountain View, and it would come pick you up. It had two human safety operators, one with their hands on the wheel, and a second in the front seat, looking at a visualization of the car’s sense-compute-act system, narrating the vehicle’s future actions so the safety driver could decide whether to intervene. On my ride, they only had to help the car once, when it was edging out to make a left turn onto a busy road. It was too conservative about waiting for an opening in the traffic to make its turn, and the human driver had to take over to bring it out into the traffic. This level of robustness was unprecedented. The first self-driving car was developed in 1977 by Japan’s Tsukuba Mechanical Engineering Laboratory and used two cameras to visually detect lane markings, and the field had advanced by leaps and bounds, until it was now possible to navigate Castro Street in Mountain view alongside bikes, pedestrians, and other cars.
And yet, it wasn’t enough. Rod Brooks said, “Technologies for robots need 10+ years of steady improvement beyond lab demos of the target tasks to mature to low cost and to have their limitations characterized well enough that they can deliver 99.9% of the time. Every 10 more years gets another 9 in reliability.” This level of repeatability and reliability is essential for many applications, including self-driving cars. What happened in 2015 is that we got a new nine. We went from maybe 50% reliability to 90%. It was an amazing breakthrough! At the same time, it was not robust enough to field autonomous cars on the road at scale, not then, not on anything like the timeline Chris Urmson aimed for, and not now. It was only in 2022 when Waymo began offering rides with no safety driver to employees in San Francisco. Right now consumers can call Waymo cars in 11 US cities, but not yet in Boston, where I used to live, or in Providence, where I live now, or in Rochester, where I grew up. The breakthrough was real, and the promise is being kept, but not on the original timeline.
All this history changes how I look at the modern influx of resources into data-driven AI for humanoid robots. Right now, in learning for robotic manipulation, it feels like it felt in 2015 in self-driving cars. There is a real breakthrough: data-driven methods like diffusion policy and leveraging the bitter lesson to apply more data and compute than ever before to hard robotics problems have led to amazing progress. We got a new nine: robotic manipulation is more successful and more robust than ever before. We are taking concrete steps towards a robot with one hardware/software loadout that can accomplish many different tasks. Simultaneously, we are also in a bubble. We need more research into how to efficiently teach a robot a manipulation policy that it can execute quickly enough and robustly enough to return on its investment in two years. We need even more research into safety and verifiability before we trust them in unstructured environments with vulnerable people, like our homes.
In robotics, there is a spectrum: a video means, “I made it work once, and I got it on camera.” A demo is, “If someone sufficiently important walks into the lab, I can make it work.” A user study means, “I can make it work for 20-50 people who walk into the lab.” That’s as far as I’ve gotten in my work, but there are many steps after that, on the road to productization, which looks like: “Someone else turned on the robot in a new place it’s never been, and it worked robustly enough to get return on investment in two years or less.” Many of the companies starting now will fail, but the survivors will change the world.