“Disciplined Innovation:” Kodiak Robotics Releases Safety Self-Assessment

Automated truck developer Kodiak Robotics submitted a Voluntary Safety Self-Assessment (VSSA) report to USDOT’s National Highway Traffic Safety Administration (NHTSA) today. The nearly fifty-page document comprehensively covers the many factors, technical and non-technical, of developing Automated Driving Systems (ADS) as well as interacting with road users and public officials.

VSSA’s are the brainchild of Dr. Mark Rosekind, NHTSA Administrator during the last part of the Obama administration. While development of nascent ADS was accelerating, he and the rest of the USDOT team were motivated to take a definitive step in an environment in which requiring information would run headlong into current government regulations and limitations.  In 2015, Dr. Rosekind spearheaded a unprecedented voluntary agreement across a wide range of automakers to make Automatic Emergency Braking standard equipment by 2022. Then in 2016, NHTSA published their most in-depth and definitive ADS document to that time: Federal Automated Vehicles Policy: Accelerating the Next Revolution in Roadway Safety. This AV Policy document was a treatise on NHTSA’s view of responsible approaches to ADS development, providing guidance to developers as well as public agencies. The VSSA concept was introduced, requesting entities developing ADS to describe their approach and level of compliance with the guidance across these areas:

·     Data Recording and Sharing

·     Privacy

·     System Safety

·     Vehicle Cybersecurity

·     Human Machine Interface

·     Crashworthiness

·     Consumer Education and Training

·     Registration and Certification

·     Post-Crash Behavior

·     Federal, State and Local Laws

·     Ethical Considerations

·     Operational Design Domain

·     Object and Event Detection and Response

·     Fall Back (Minimal Risk Condition)

·     Validation Methods

While VSSA’s were hatched during the Obama administration, today’s USDOT continues to be highly engaged with industry with a light touch on requirements. “Automated Driving Systems, A Vision for Safety,” dubbed AV 2.0, was issued in 2017. The document updated and discussed the same issues in a pared down format, add new perspectives. Nevertheless, the VSSA aspects remained. There is no Federal approval or comment process; once submitted, VSSA’s are posted in a cozy corner on NHTSA’s website for all to see.  

I’ve always supported the VSSA process, as compared to a more draconian approach by government agencies. There was then and still are too many ways to “skin the ADS cat” to require anything more than voluntary submissions. Of course, a request to submit a voluntary letter that includes saying whether you are complying with government guidance tends to have a wee bit of selection bias. If you’re not complying, are you likely to submit a letter? Nevertheless, the general community finds these documents valuable, depending on where a particular VSSA lands on a scale ranging from marketing fluff, to being usefully educational, to providing engineering depth. In some cases, you can find chest thumping “we alone are changing the world” talk, but in other cases one can get a feel for a company’s stated values, culture, and overall proficiency.

VSSA Number Twenty One

Twenty VSSA’s have previously been submitted. Of the AV truck players, Ike Robotics, TuSimple, and Starsky Robotics (now defunct) have contributed. Truck ADS developers Aurora and Waymo submitted VSSAs as well, but these were more about their passenger vehicle ADS development.  

If you follow the NHTSA VSSA guidelines to the letter, it’s hard to distinguish yourself. The Ike Robotics VSSA, published last year, argued that testing on public roads is not as essential as others say, plus provided significant detail appealing to AV nerds. TuSimple provided a more basic VSSA document, perfectly adequate, checking all the boxes. Others have expressed their safety approach but set their own topics and format, such as Peloton Technology’s Safety Report.  

USDOT officials frequently remind the ADS developer community that new VSSA submissions are welcome. So, no doubt Kodiak made their day today.  Kodiak’s Self-Assessment offers a highly coherent document addressing the truck ADS space comprehensively. The basics are not unique. If a copy was being perused in the break room of other Silicon Valley ADS firms you would – for the most part – hear the engineers saying, “Yeah, us too. Of course.” Just like they would have said in reading (most) previous VSSA’s.

But this is far from a ho-hum document. Kodiak’s report is titled “Safety, first and always.” Their stated audience is members of the public and regulatory officials. “As developers of self-driving commercial vehicles, we understand that most Americans will never get to ride in a Kodiak truck. Because we all share the same roads, we feel a responsibility to communicate why people should trust the Kodiak Driver. We see the Kodiak Safety Report as the beginning of a conversation with our fellow drivers about what we’re building,” said Don Burnette, Kodiak co-founder and CEO.

Kodiak clearly seeks to distinguish themselves within the industry. The company stresses the importance of collaboration, stating their “duty to share safety-relevant advances with the industry” and aspiration to “contribute meaningfully to the important conversation happening globally around how to define safety for self-driving vehicles.”

They detail a straightforward go-to-market approach: driving only on limited access highways.  Their trucks will exit to “truckports” adjacent to the highway, transferring loads to and from human driven trucks handling last mile operations. While other truck ADS companies seek to sell automated trucks to existing truck fleets, Kodiak will be a freight carrier competing with those fleets.

Ducks and Pigeons

There are several key points and assertions in the Kodiak Safety Report worth noting. They join many other voices, mine included, in making the case that automated trucking will be the first automated vehicle application to scale. They also contend that self-driving trucks can reduce traffic by driving at periods when traffic is light. Sure, regular trucks can do this, but it is harder to hire and retain truck drivers when they have to drive in the wee hours of the night, for instance.

Given years of hype about lidar in the self-driving space, Kodiak emphasizes that no one sensor is primary over another. Lidar, radar, and camera data are complementary, and when fused, the perception is better overall. I highly recommend reading their “ducks and pigeons” discussion, with the memorable line “Kodiak’s perception system does not assume that an object is a duck, just because a single sensor says it is one.”

When it comes to satellite navigation and other external data, their message is clear: “Kodiak believes that our trucks’ safety envelope must be entirely contained within the vehicle, i.e. that all safety-critical processing must happen on the truck.” External data is seen as an additional safety input. To my knowledge, this is a core tenet across all of the solo driverless truck ADS developers.

In the “Planning for Uncertainty” discussion they admit that the Kodiak Driver may encounter a situation it cannot handle, giving the example shown in the image. The system has the option to “safely pull to the side of the road and wait for the situation to become interpretable.” I’ve heard this from other companies and the strategy makes sense for the initial wave of deployment. But reaching full scale will require meeting the expectations of shippers insisting on reliable delivery times.  Kodiak also emphasizes that the system has sufficient intelligence to come to a safe stop when it reaches the limits of its Operational Design Domain, an issue frequently brought up by regulators.

Kodiak’s approach to “disengagements,” i.e. the safety driver taking over from the ADS as a safety precaution, is particularly interesting. The company is adamant that their safety drivers are free to disengage when they’re not sure the truck will handle a situation correctly. Kodiak  pushes back strongly against the metric of “miles per disengagement” as put in place by California DMV, because results can be misleading depending on the type of roads and traffic involved. They see disengagements as opportunities to learn more about their system and the operating environment. They also recognize that on-road testing carries some degree of risk and seek to ensure “that every piece of data we collect is worth the risk.” Alternatively, their metric is “learnings per mile,” noting that going too long without disengagements “means we aren’t learning and we need to stress the system some more.”  Kodiak calls this approach “disciplined innovation,” saying “we see our lower mileage count not as a risk, but as a sign of our commitment to safety.”  

Going further, Kodiak commits to proving “mathematically and in plain English, that our vehicles are comprehensively safe even without a safety driver behind the wheel.” I discussed this briefly with Paz Eshel, Kodiak’s Co-Founder and COO, as I was preparing this article. The document does not address this topic very deeply, but I wouldn’t expect it to do so. The industry is on the cusp of reaching this milestone. All of the ADS developers are internally defining what it takes to “go driverless” but this is an evolving process. When self-driving companies can openly elaborate how they certify a system for driverless operation, this will be another milestone vitally important to policy-makers. I don’t see this coming anytime soon.

Useful Pieces of a Daunting Puzzle

The people I know at Kodiak are not prone to hype. They have their feet on the ground. In fact, in their 26 months of existence, Kodiak has been relatively quiet, not prioritizing “progress by press releases” the way some others have. They tell me this  Safety Report provides substantially more information than they’ve shared before, which they see as an important step in maturing as a company and moving towards commercialization.

I’ll be surprised to hear any particular pushback on Kodiak’s Safety Report precisely because it is well in line with general industry thinking. The document is a fresh reaffirmation of major tenets, making the challenges of ADS development understandable. Kodiak makes a convincing case that they know the issues and challenges and want to “get it right.” Documents like this are useful pieces of the puzzle to gain broad acceptance of automated driving.

The bottom line will be the actual safety performance of these systems when they hit the road.