Cruise Reports Lots Of Human Oversight Of Robotaxis, Is That Bad?

A recent report from the New York Times revealed leaked data that General Motors’ Cruise robotaxis were seeing a remote “intervention” every 2.5 to 5 miles, and that Cruise had 1.5 staff for every car on the road. Kyle Vogt, Cruise’s CEO, responded by saying that they do indeed have many instances of human overslight, though he expressed it as saying humans were guiding the vehicles about 2-4% of the time, and that further, this was a perfectly acceptable number that they did not see a big need to improve at this time; that other needs were more important.

While Cruise’s vehicles are off the roads right now due to an order from the California DMV and the company’s own decision in other states, it’s worth unpacking just what these numbers mean and whether they are bad or good, both for safety, and for the financial viability of the Cruise service. In short summary, Cruise’s numbers can’t be called good, but neither are they ruinous, nor a sign of economic impossibility. Indeed, their approach reminds me of the path we took at Starship, the world’s top robotic delivery company, to get to high autonomy.

Cruise has been in a tailspin of bad news of late. In addition to the leaks from the Times, other internal information disclosed to Forbes and the Intercept alleges that GM has paused the custom Origin robotaxi vehicle, and that there have been safety concerns over the ability of Cruise’s software to distinguish children from adults, or to spot pits and holes in the road. The pause of the Origin would suggest serious peril for Cruise as its corporate parent rethinks their strategy with it. Most charitably, it could mean GM has just decided it’s not wise to have a line churn out vehicles when it’s uncertain how long it will be until Cruise resumes operations.

Cruise, Waymo and almost all other companies manage their fleets with a remote operations center. Rather than try to solve every single problem with software, some problems are punted to humans who can look out through the cameras and sensors and advise the car how to solve then. Cruise and most other teams don’t have the remote staff drive the car with a video game wheel. Rather, they will give it advice, like whether to turn around, or what lane to pick, or a set of waypoints to navigate through on the map. The car does the driving. The companies have been wary of trusting the data networks to be good enough for remote driving, though there are companies that do that, and some debate on whether Cruise and Waymo were wise to avoid that approach entirely.

In San Francisco, 5 miles can mean about 25 minutes of driving, so these events are not so frequent as you would first imagine, but it still contrary to the image of an autonomous vehicle. Because the remote operators do not control vehicles in real time, these are (almost) never safety-related interventions. Rather they relate to the car getting confused about the best course, and they might involve the car pausing and blocking traffic, or being a bad road citizen. That’s not good, but it’s not a catastrophe either. The main safety implication, which has shown up, involves interactions with emergency vehicles, where it’s not road safety that’s at issue, but things like response to a fire.

Cruise has indicated that 80% of requests for assistance by the systems are resolve before the human can connect in and of the remaining group, most are done in <30 seconds, but around 10% take a minute or more, which is every 40 miles or 13 trips. (There is some confusion about the breakdown of the events that do end up needing a human, and I’ve reached out to Cruise for clarification.)

A typical request might come from encountering construction, or an object that can’t be identified, or a question of whether it is safe to proceed. Unusual crowds of people, or people touching the vehicle will trigger a request, along with tricky situations like complex U-turns were also situations cited by Cruise.

So while the first reaction to Cruise confirming that they have lots of remote assist interventions was to say, “wow, that’s way too many,” the more accurate approach is to realize that Cruise, Waymo and anybody operating in this style has zero truly urgent safety interventions. If there’s an urgent problem which will cause the vehicle to crash without human help, the remote operators aren’t set up to be able to stop it. If they happen to be watching in advance and it looks like a crash might come in several seconds, they could possibly do something to stop it, but they are not usually watching.

This is part of why it’s so hard to put vehicles out there with nobody in them. They must get very good at handling all urgent situations. They can’t discover a pedestrian suddenly in front of them and ask to have a human look at the situation and decide whether to veer left or right or slam the brakes in the next second. The human can’t provide that answer in time, the system has to do it now. The main number that looks poor here is the one minute intervention every 40 miles (which also equates to about 4 hours of travel on San Francisco streets.) While one minute of confusion out of 4 hours isn’t horrible, it has been causing frustration for drivers in the city. Humans don’t usually pause for a minute to figure things out, and Cruise has not made clear what sort of problems trigger this. This is an example of the tradeoff between caution and good road citizenship which demands more tolerance by other road users in the early days of a robocar project in exchange for getting exemplary drivers down the road.

While the numbers aren’t where drivers might like them to be, it’s not something that has to be at zero to have a working robotaxi. Humans do pause from time to time—if you’ve every felt the urge to honk you’ve seen it, but minute long delays are pushing it.

Contrast With Safety Drivers

There is some confusion because the word “intervention” has been used to describe these events, and this word was also used for the actions of human safety drivers who used to sit behind the wheel of Cruise and Waymo cars, and still sit there with most other companies. Tesla’s “FSD Beta” system is also run in a supervised way by the driver of the car, who intervenes by grabbing the wheel or using the pedals. Safety driver interventions are physical, and usually in response to urgent safety events, such as the car departing a lane. Those interventions need to be rare as the quality of a vehicle improves, to the point when they are happening extremely rarely, on the order of once every 100,000 miles or much more.

Remote assist interventions aren’t interventions of the same sort. They are requested by the car, not by the human, and in Cruise’s case, many need no response. They are more similar to the frequent times a Tesla driver will press the accelerator when the car is being hesitant, to let it know the human says it can go. It would be better if the word intervention were use only for actions initiated by a human supervisor, in particular those in the interests of safety.

Starship

I worked on the deployment of Starship delivery robots. I helped craft a strategy that relied on remote advisors. Starship had the advantage that its robots are small, light and slow, so they can stop in a foot and won’t hurt you even if they were to hit you. We began by having humans follow the robots on foot to make sure they were operating well, and later remote operators did this. At the very start, the robots got confused a lot, they did only a small portion of the day autonomously, on their own. This was fine, we could still deliver packages and learn about all the other facets of the business.

In time, the autonomy got much better, and the robots spent more and more of their time autonomous. Today, they often spend a whole day without needing any remote assist. That’s as good as they need to be. If you can get robots to need assist only 2% of the time, you can have one remote staffer for every 50 active robots, on average. That’s quite affordable, particularly when you consider today’s delivery has a human driving 100% of the time, and also wasting time moving without cargo from place to place, or commuting. Uber drivers spend a huge amount of time without a passenger in the car—either waiting, or repositioning, or doing pick-up, or commuting to the busy zone. The robots don’t need any human when waiting and they don’t generally commute. The “Autonomy most of the time” strategy has allowed Starship to make over 5 million paid deliveries, which is more trips than any robotaxi company.

If Cruise needs one staffer for every 25 robots, as they seem to, they may not have reached as high a percentage as Starship, but they don’t really need to. If the salary, facilities and other costs for the worker are $50/hour, that’s just $2/hour per car, or around 16 cents/mile. They are charging around $3/mile, so that’s not too hard to afford, and it will go down as they improve. This explains why Vogt feels he has better things to spend his money on that immediate improvement of that.

The more urgent question is whether those events are causing any traffic disruptions or risks over and above those caused by the same things humans do. If so, that would be a reason to work more urgently on this problem. Truth is, though, we otherwise want more human supervision of these machines in their early days, as they are probably helping the robots along, rather than slowing them. Robots in autonomous mode sometimes will be slower and more cautious than we might like if we are waiting behind them. If a human can glance at the screens and give the robot confidence, we might like that. Of course, human operators, being humans, will make mistakes, which can factor things the other way.

We might instead want to ask why this system didn’t work better in the incident that has Cruise shut down. Their vehicle made an autonomous decision to pull over after things went horribly bad. In review, it would probably have been better if the vehicle had waited for a human’s assessment of the situation, though even that won’t be perfect. Indeed, in many of the incidents Cruise has faced in the last few months, some of them might have been solved with more frequent calls for oversight.

In addition, Cruise is sometimes so wary of using just remote advice to get out of situations that they send a rescue driver to recover a car, which can leave it blocking traffic for 10-15 minutes, and that has made nobody happy.

Having 1.5 support staff per car is obviously not a number one would sustain at scale, but it’s not at all surprising to be oversupplied on service staff during the early stages of growing a fleet. Cruise’s fleet was much larger just a couple of months ago when the DMV made them cut it in half. It’s pretty certain that Cruise does not intend to have 15,000 support staff when they have 10,000 cars on the road. They will have the remote operators (probably about 300 of them on shift half the day, fewer at night) and perhaps a bit larger crews managing charging, maintenance, depots, customer support and cleaning. They may still be working out how big those crews will be.

I would assess that Cruise probably needs to improve these ratios a bit. They have to hit a higher bar than Starship. It’s not because of money, though. One reason showed up at the Outside Lands concert in August, where a cell network overload meant the remote advisors were slowed down in solving problems, and Cruise made traffic work. (While this was incorrectly reported, even by Cruise itself, the cell overload didn’t affect Cruise vehicles across town in North Beach where another jam the same night was caused by a pedestrian blocking the vehicles.) It’s better when the robot can solve the problems on its own, but not a calamity.