Waymo today announced the results of a new study where they claim their Waymo driver system outperformed a model of an idealized top-grade human driver at taking evasive action to avoid crashes caused by somebody else. If that’s true, the next question is, what’s holding Waymo back from full scale commercial deployment?
Earlier, Waymo published a study where they recreated all fatal accidents which took place in the recent past in Chandler, Arizona, where they have operations. In the simulator, they had their system relive those accidents, playing the role of the at-fault driver (where they easily prevented all of them) and also playing the role of the driver who was hit or forced into a crash (where they avoided many of them with good defensive driving.)
To go further, they have both improved their driver system to include evasive action — swerving and other maneuvers to avoid the crash. They also created a simulation of a really good human driver, one who is always paying full attention to the road and has good reaction times (as studied in analysis of real world accidents.)
In the study, they ended up looking at only 16 crashes. That’s a pretty small set, and so may call into question where the results meet the bar for statistical significance. In real life, all the human drivers failed to avoid these crashes. In their simulation, the ideal human avoided 10 of the 16 crashes and the Waymo driver avoided 12. They Waymo driver reduced the crash severity in the other 4, and the ideal human did so in 5 of the 6 it missed — in the final case doing nothing to mitigate the accident. Waymo claims that as a result their system reduced serious injury risk by 93% compared to 84% for the ideal human.
Aside from the small sample, this is a good and somewhat expected result. Accidents pulled from the real world are likely to involve people who didn’t pay perfect attention on both sides. Humans avoid accidents both by anticipating them in advance (ie. you notice somebody is being erratic and you stay away from them) or by real evasive action (somebody is going to hit you and you swerve away.) Robots should naturally be better than humans at the latter — they have perfect knowledge of physics and a 360 degree awareness of the world around them. They don’t have to check their blind spot to swerve.
That 360 degree view can also aid the robots in spotting trouble before it happens, but right now humans are better at intuition about what other humans are likely to do — but the machines are improving.
While Waymo excluded accidents where a vehicle had to dodge being rear-ended, it’s not out of the question that a robot could do things there. Unlike a human, a robot is always looking behind it, not just in front. They will see somebody coming up from behind too fast. They will know, without needing to check a blind spot, if they can move and get out of the way, if it’s possible. It may even be possible in an illegal manner — for example when stopped at a light, one might zoom into the intersection if it’s clear — emergency vehicle drivers do this safely every day, after all, and it’s not just the siren that makes that happen. The fast acceleration of electric cars an also help here. Of course, sometimes it won’t be possible — they won’t deliberately trigger an un unsafe situation in the intersection they go to, or drive onto a sidewalk that has people on it.
So what’s stopping them?
They Waymo result, limited as it is, is good to see. We should want the robots on our roads to be superhuman at avoiding crashes. We want them to never cause a crash and we want them to save us if others will cause a crash. This is the safety case that’s been made as the core goal since the dawn of robocars — make vehicles that improve safety on our roads over having human drivers.
Waymo has data to suggest they have met that goal, at least in the relatively easy driving situation of suburban Arizona. That they have surpassed it. While we would like to see data like that for San Francisco and other more complex worlds, one can hope that is coming soon.
So now it’s time to ponder why Waymo, who since day one has been widely viewed as the leader in robocars, hasn’t moved from pilot projects to real commercial deployment. This question is discussed in my 2-part series of articles and videos on what the blocking factors are. The difference is that for Waymo, the two biggest factors — making it safe and proving you’ve done that — seem to be attained.
Waymo has built their app and rider support infrastructure for their pilot projects. But the projects are all incomplete in some way. Chandler is not a real market for robotaxis — it’s a suburb where everybody owns cars. Downtown Phoenix is a bit better but still not great. A robotaxi has to go to a really usable service area to be a suitable replacement for car ownership. San Francisco is such an area (especially if you include trips to the airport and a bridge to a Silicon Valley robotaxi service which Waymo also has worked on) but Waymo’s operations in SF are still limited and don’t include the downtown.
The time to deploy a commercially viable service is hopefully soon. Perhaps better road citizenship is needed. Perhaps some of the other factors need more work. It is known that Waymos will avoid certain areas and left turns that they are not 100% confident on, sometimes lengthening the trip. They avoid the highway in Chandler. Perhaps this is what stands in the way.
Last week, GM’s Cruise unit (which also has severe limits on their San Francisco operations) announced they would start service in Phoenix and Austin by the end of 2022. It is unknown if these service areas will be commercially viable, or still just pilots. Cruise is saying they can expand quickly to new cities. MobilEye — which has yet to get as far as Cruise or Waymo — also says they have the ability to move to a new city in a very short time. But we’re waiting on that commercially viable service. That is not just a cheaper Uber
UBER
- Cover a large enough service area that, combined with taxi-style service and carshare, makes people feel they can give up ownership of at least one of their cars because of the service. (In many cities, people give up a car for transit so this is not impossible.)
- Has the apps and glue all together to meet all the transport needs of those customers, along with possibly the finances — ie. an app where they can get rides, transit, car rental/share and micromobility in a seamless experience.
- Have a population which might do that which is large enough to support the service, combined with the market that comes from being a cheaper Uber-style service in that area.
The safe robotaxi is just one component of this, though it is the most important. We look forward to seeing it.
