Another key aspect of Friday’s release: Waymo offers insight on its simulation practices.
One of the conundrums in real-world testing on public roads is that companies seek information on challenging scenarios, but human safety backups must commandeer control of the test vehicle should safety concerns arise — incidents known as “disengagements” or “interventions” in the industry.
Preventing a real collision is, of course, the priority. But a disengagement can limit the insights self-driving tech companies can learn from an unusual or hazardous situation. Waymo’s safety framework provides detail on how Waymo has simulated the way those scenarios would have played out.
As a result of these “counterfactual” analyses, Waymo says it could measure performance metrics, and examine 29 instances in which a “contact event” would have occurred had a human driver not intervened, in addition to the 18 instances in which contact actually did happen.
Waymo says none of the 47 cases would have been expected to lead to serious injury, and that nearly all incidents, including the eight most-serious actual and serious ones, involved rules violations or other errors committed by human road users.
“The counterfactual examination is super cool,” McGehee said. “This is something generally used in crash reconstruction, but you’re working with hindsight bias. This is very unique. There’s a lot of proprietary testing going on like this. But for them to publicize it, it really moves the needle on crash avoidance.”
Many of the “contact events” were minor, such as three that involved bicyclists and pedestrians walking into the right side of stopped Waymo minivans. Others were more serious. There were eight instances — three actual and five simulated — in which airbags deployed.
Many of the incidents are further described in the report, entitled “Waymo Public Safety Performance Data.” It warrants further scrutiny and analysis. Now that Waymo has issued the information, industry experts, safety advocates and other outsiders can accomplish exactly that.
And at a time when others are conducting driverless testing, such as Yandex in Ann Arbor, Mich., or plotting driverless deployments, such as Cruise in San Francisco, it bears watching whether competitors will follow suit in releasing similar documents.
