Transportation

Throwing Wrenches At Autonomous Vehicles


If you need to quickly figure out whether or not an AV is safe, you might want to throw a wrench at it.  If it doesn’t crash, it’s worthy of further review.  If it does, case closed.

Of course, you don’t necessarily throw a literal wrench at the AV, you just a proverbial wrench in it’s plans – add some forced entropy to an otherwise uneventful test drive.  Demand a last minute turn (if it’s reasonable that users might do so), push a shopping cart into it’s lane (if it might ever encounter them), pour black paint over some lane lines.

Few industries receive as much speculative scrutiny as autonomous vehicles (AVs).  The infamous trolley problem is case in point – a google of “trolley problem” + “autonomous” yields 4000 search results (probably 40x the total number of vehicles that have ever driven unmanned on public roads).  When it comes to regulating autonomous vehicles there are intensive conversations about which sensors should be required, which technologies banned, whether or not photorealistic simulation is required, et cetera et cetera.

Out of all of these conversations the consistent outcome tends to be something akin to a complex driving test.  A closed driving course with pre-set obstacles designed to pop out at predetermined times.  Since these tests would be much more thorough than what we’d give a student driver, the thinking goes, passing these tests will help prove an AV is safer.

The problem is that AVs are really really good at cramming for tests.

Almost every AV uses a fair amount of machine learning (ML), which works by looking at a set of relevant data and then determining whether or not current conditions look like that data.  These lane lines look like those it was trained on, so it recognizes them, etc.  From a testing perspective a problem emerges – what if you train on a model just on different attempts at doing that test?

You could imagine an essay writing AI, trained on 100,000 essays on Romeo & Juliet (as well as the original text and grade scores).  Such an AI would be incredibly good at writing an essay on Romeo & Juliet, but lackluster at a question on Hamlet and terrible on a Ulysses question.  For the AV – you could have a system that got top marks on the track but was utterly unable to drive anywhere else in the world.

The solution is actually something that humans do naturally and without advanced technical degrees: adding forced entropy, or throwing the odd wrench in the pre-scheduled test.  What regulators really care about is that these systems can be deployed fairly safely, and a relatively small amount of entropy can break all but the most robust AI systems.  

Asking for a last minute left turn during a ridealong, for example, is the easiest way to get a sense of whether the AV can move in the free world or is actually following an invisible track.  Pushing a shopping cart into a path (when you haven’t previously told the AV developer) is a great way to tell whether the AV system can deal with weird obstacles.

What’s useful about this approach is that it tests the AV system being deployed.  Depending on use case and proposed stage of deployment, that system might include a teleoperator and/or a safety driver; and a successful test is one where the AV doesn’t cause an unavoidable accident.

High res simulation, 3rd party review of design docs, reading VSSAs – all of those steps are still useful in evaluating the safety of an AV, but are only worth the long process of evaluating after an AV passes the wrench test.  

And when it comes to unmanned deployment, my guess is that most AV companies worth over $50m wouldn’t be able to dodge a wrench.



READ NEWS SOURCE

This website uses cookies. By continuing to use this site, you accept our use of cookies.