July 16, 2019 Mountain View / CA / USA – Waymo self driving car performing tests on a street near Google’s offices, Silicon Valley
Getty
Recent leaks of Waymo rider satisfaction scores reported by “The Information” (paywall) suggest that in the Phoenix area, Waymo has gotten 70% 5-star and 30% lesser feedback from riders over about 6,100 trips. Another 4,000 trips in the San Francisco Bay Area, taken by employees, have a lower score, with 47% having some problem. Problems included bad routes, occasional driving issues, being dropped off at less convenient spots and even a few worries about safety but no actual incidents.
The analysis in the original piece is somewhat negative, feeling this feedback rate is problematic. I think in many ways it’s a triumph. This is, after all, one of the most complex and difficult consumer product challenges ever attempted. There are a variety of successful operating businesses who don’t get 70% 5-star customers.
Some of the problems will be easily tolerated in a service that’s half the price of Uber (and eventually less.) Safety problems or fear problems are obviously not in that class, but issues with where one is dropped off, routing delays and minor traffic hesitations are frequent in most other transportation services and easily tolerated.
This is not necessary, however, for two core reasons. First, limitations can be explained in advance. If the vehicle is not able to take the quickest route, it can tell a customer that, and offer a human-driven ride as an alternative — possibly at a subsidized price until that problem is solved. Drop off points can also be explained. It is only necessary, at first, to keep the number of unexpected problems minimal.
Waymo is not operating its own human-driven taxi service as Lyft and Uber do. Competing with those companies is not easy at all, but there are a few companies which have the best shot at doing so, and at the top of the list are Alphabet and Apple. They can also contract out rides to other providers as needed, and easily eat the cost of a temporary need for that. (After all, they are currently paying for safety drivers in the pilot phase.) Waymo does have a remote operations desk it uses to handle vehicle confusion problems.
Secondly, customer experience problems are an entirely different class of problems than safety issues. For all teams, safety issues remain an extremely high priority. Driving safely only 70% of the time would be a non-starter. Driving safely 99.99% of the time is a non-starter. You need to get a lot of nines, but there is a strange paradox about this. By one measure, getting from 99.99% to 99.9999% is 100 times harder — but this definition you are only 1% of the way there.
The saving factor is that teams don’t improve at a linear pace. As they get more experienced, they get better at getting better. If it takes 2 years to get from 99.9% to 99.99%, it might well take only another two years to get to 99.999% and another 2 to get to 99.9999%. So while it requires going 100 times as far without a problem, it doesn’t take 100 times as long to get there.
With non-safety customer issues, you don’t have this issue. If you get to 99% satisfied, you are pretty much done. You want to get better, and you will — also at an increasing rate, but you really don’t have that far to go.
Some of these problems are still a lot of work to solve. For example, the complaint about dropping customers off in inconvenient spots stems from the fact that it’s difficult to map out and certify every address for the right place to do drop-offs. It’s a lot of work to make sure that every street, every unprotected left turn, is well handled. But you get better and better at adding more of them. This is something Google is good at. It took a lot of work to get the Streetview cars to drive their first miles, and then a whole city, but surprisingly while it was an immense effort, going to driving an entire country didn’t take 1,000 times as long as driving a single city even if it meant 1,000 times as many roads.
One of the greatest reasons I have seen behind pessimism expressed around robocars has been the mistaken belief that you must solve everything to make a useful product. That’s not true at all, especially in a taxi service. There are lots of limited subset markets that are quite reasonable, especially if you can just send a human driver on the routes you can’t do yet. This lets you provide service from anywhere to anywhere in a wide enough service area to be useful. You can either charge more for the human drivers, or subsidize them all the way down to the cost of the robots. Even car companies intend to deploy with limited service areas (such as “only highways”) to begin, and this is also the plan of all the trucking companies. The concept of “Level 5” is an aspirational science fiction goal, not a real commercial one. (And since Level 3 is dangerous and Level 2 not very interesting, this is one reason the whole idea of levels is misleading and less than useless.)
Rather than disappointed by Waymo’s feedback scores, I think they are heartening. They show a project on the accelerating (but long) path to real commercial deployment. Because of Waymo’s substantial lead, they can also afford to wait until their customer satisfaction is at an exemplary level.
Disclosure: The author worked for Google’s car project 2010-2013, and owns a modest amount of Alphabet stock.
