California DMV Publishes Self-Driving Car Disengagement Report, Waymo is Still the Leader

author: Eric Walz   

A self-driving Waymo minivan

Despite all of the attention surrounding self-driving cars, they are not yet allowed to roam freely on roads in the U.S. In California for example, companies looking to test autonomous vehicles on public roads must first apply (and be granted) an autonomous testing permit issued by the state's Division of Motor Vehicles (DMV).

In turn, permit holders are required to notify the state every time a self-driving vehicle "disengages", meaning a human operator was required take over to navigate a senario the vehicle's autonomous software could not handle.

On Wednesday, the California DMV released its annual disengagement report. The newly released report, which is also available to the public, provides some surprising insights into the progress made by each company testing autonomous vehicles every month.

The report also shows that out of 48 companies with the autonomous testing permit, Waymo leads the pack with the fewest amount of disengagements, while Apple is in last place with the most.

According to the report, over a one year period from Dec 2017 to Nov 2018, many of Waymo's vehicles traveled close to 30,000 miles in full autonomous mode with less than five disengagements, some with only two disengagements in 30,000 miles of driving.

One Waymo vehicle traveled 37,914 miles with only two disengagements the entire time. In total, Waymo racked up 1.26 million real-world miles in 2018 with it autonomous fleet of Chrysler Pacifica minivans.

The report shows the reasons for each disengagement and it varies widely from company to company. Most of Waymo's disengagements were reported by backup safety drivers as "unwanted maneuvers that were undesirable under the circumstances", indicated that there still work to be done to improve the software.

Waymo also reported multiple times that disengagements were caused by a "recklessly behaving road user" led to Waymo's safety driver to manually take control of the vehicle.

Apple on the other hand, cannot seem to travel one mile without having a human driver take over for some reason. One of Apple's self-driving vehicles reported 3,407 disengagements but only traveled 2,423 miles in autonomous mode.

While certains situations are more challenging than others, such as navigating a self-driving car through busy urban areas, the data clearly shows that Apple and other companies are far behind Waymo.

Perception Problems

Waymo has said that its main focus is on safety. However, some of the disengagements were due to perception problems, meaning the self-driving vehicle failed to recognize an object correctly. Waymo is not required to report what the object was, but it may have been a pedestrian, street sign, or bicyclist.

After a self-driving Uber vehicle fatally struck a pedestrian in Tempe, Arizona last year, many tech companies have scaled back their autonomous testing programs to take a closer look at the safety of their technology. That incident was blamed on Uber's software, for failing to identify and react in time to avoid striking the pedestrian.

In March 2018, following the crash in Tempe, Uber ATG immediately grounded its test vehicle fleets in every operating location to conduct a top-to-bottom internal review of safety approaches and the company's self-driving systems.

Like Apple, Uber is also far behind Waymo in miles driven without disengagement. Uber's report to the DMV spans from  Mar 8, 2017 through Nov 30, 2018. Uber reported 26,899 miles driven in autonomous mode for the period, but reported 70,165 disengagements—about once every 0.3 miles.

Its not too surprising that Google's Waymo is the leader in the self-driving race. The tech giant has been working on self-driving cars for the past decade, well before anyone else. In addition, Waymo's nearly unlimited financial resources allow it to custom build hardware and test more vehicles in more locations and many of the Silicon Valley startups working alongside them on the technology.

Right now, Waymo is averaging about 11,000 miles between disengagements across its entire fleet in California—a feat that was unreachable just a decade ago. That's like driving from New York to Los Angeles then back again twice, without ever touching the steering wheel or brakes.

Waymo is launching a commercial robo-taxi service called Waymo One with its autonomous fleet beginning in Arizona. The pilot is already underway with backup drivers behind the wheel to monitor the vehicles is case something goes wrong. However, Waymo is confident enough in its software that soon the human backups will be removed.

Other companies testing driverless vehicles in California include Toyota, Tesla, Lyft, BMW and General Motors.

The full DMV report is available here.

Eric Walz

Originally hailing from New Jersey, Eric is a automotive & technology reporter covering the high-tech industry here in Silicon Valley. He has over 15 years of automotive experience and a bachelors degree in computer science. These skills, combined with technical writing and news reporting, allows him to fully understand and identify new and innovative technologies in the auto industry and beyond. He has worked at Uber on self-driving cars and as a technical writer, helping people to understand and work with technology.