Leaked Uber Self-Driving Metrics Show Alarming, Inconsistent Performance Rates

author: Michael Cheng   

Uber is having a rough year – and it's about to get worse for the ridesharing giant. The main issue hindering the company's advancements in the driverless sector is a monumental legal battle with Waymo over trade secrets. The Alphabet-backed business previously requested the federal court to block Uber's self-driving project.

Concerns over Uber's capabilities to develop autonomous technology was never really questioned in the court filings, until now. A set of leaked documents, courtesy of Recode, revealed just how well its fleet of self-driving cars is actually doing on public roads. Generally speaking, the results do not look good for the business. One of the most daunting metric involving "miles per intervention measure" showed that in January Uber's driverless vessels required human intervention every 0.9 miles. In February, that figure fell to 0.7 miles.

"To take out the safety drivers, you would want far better performance than these numbers suggest, and you'd want that to be consistently better performance," explained Walker Smith, a member of the US Department of Transportation's Advisory Committee on Automation in Transportation.

Critical Interventions

To be fair, it is important to consider that Uber hasn't spent a lot of time developing this technology. By comparison, Google (before Waymo) has been clocking in driverless miles for over half a decade, and its system still isn't perfect.

Uber's metrics are divided into three classifications: the number of miles recorded before a human is required to take over ("miles per intervention"), miles recorded before a driver is needed to avoid critical obstructions that may lead to extensive harm or damage and miles recorded before the ride is interrupted (measuring smoothness of experience).

The first metric was addressed earlier in the article, so let's take a closer look at the second metric, related to avoiding critical damage. At the beginning of February, an Uber self-driving car could operate for 125 miles before encountering critical intervention. A week later, that figure fell to 50 miles, then to 160 miles and down to 115 miles toward the end of the month.

When paired against the US Department of Transportation's 2016 report on average annual miles driven for males and females between 20-34 – roughly 15,100 miles, this type of intervention would be encountered around 94 times per year, assuming the vehicle sees such need every 160 miles.

Bad Experiences

Moving on to the final metric, which measures smoothness of ride, Uber's driverless fleet succumbed to rough driving maneuvers every 4.5 miles (average) in January. The following month, that figure decreased to two miles. During an interview with BuzzFeed News, Uber mentioned that such disengagements includes the number of times the platform pushes back control to the driver at the end of the trip or session.

"As a whole, Uber's self-driving system is putting on many more miles than it did in January. Last week, the company's 43 active cars drove 20,354 miles autonomously, according to the documents. This is only the second time since late December 2016 that its cars have driven more than 20,000 miles in a week," said Johana Bhuiyan from Recode.

Michael Cheng

Michael Cheng is a legal editor and technical writer with publications for Blackberry ISHN Magazine Houzz and Payment Week. He specializes in technology business and digesting hard data. Outside of work Michael likes to train for marathons spend time with his daughter and explore new places.