Researchers Trick Tesla Vehicles into Driving 50 MPH Over the Speed Limit with a Traffic Sign Altered with Black Tape

author: FutureCar Staff    

Tesla's Chief Executive Elon Musk often boasts about the safety of the company's advanced driver assist systems (ADAS) including the semi-autonomous feature Autopilot. Musk has said publicly stated that Tesla's Autopilot performs "better than a human driver." For example, in 2016, Musk said that "half a million" people would be saved if Autopilot were more widely available in the auto industry.

However, security firm McAfee discovered some serious flaws in Telsa's forward facing cameras, which are used for the automaker's Autopilot system as well as for its advanced cruise control, particularly in how machine learning software detects common speed limit signs. 

McAfee is an American global computer security software company headquartered Silicon Valley California and claims to be the world's largest dedicated technology security company. 

The experiment by McAfee is designed to identify potential security problems and flaws that the auto industry has not kept up with.

McAfee calls its experiment "Model Hacking" which is the concept of exploiting weaknesses in machine learning algorithms to achieve adverse results. Tesla's cruise control system uses machine learning for image recognition, primarily to identify road signs.

McAfee researchers Shivangee Trivedi and Steve Povolny were able to run repeated tests on a 2016 Tesla Model S and a 2016 Model X SUV that use the MobilEye camera with the embedded EyeQ3 MobilEye chip, which is used in Tesla's hardware pack 1, which has since been superseded.

Mobileye is a subsidiary of the Intel Corporation that develops vision-based self-driving car and advanced driver-assistance systems.

In McAfee's lab tests, the researchers placed adhesive stickers on a 35 MPH speed limit sign that the Tesla vehicles rely on for its Traffic Aware Cruise Control (TACC) feature.

The test was varied using different angles, lighting and reflectivity to more closely replicate real-world conditions. The team positioned the stickers in various locations to try and trick the machine learning algorithms to read a number other than a "35."

"By making a tiny sticker-based modification to our speed limit sign, we were able to cause a targeted misclassification of the MobilEye camera on a Tesla and use it to cause the vehicle to autonomously speed up to 85 mph when reading a 35-mph sign," the researchers said.

The next step for the researchers was to reduce the number of stickers to determine at which point they failed to cause a misclassification. During this part of the test, the team realized that the head-up-display (HUD) continued to misclassify the speed limit sign when the stickers were present.

Finally, the team used a single piece of black electrical tape, approximately 2 inches long, and extending the middle of the 3 on the traffic sign and were able to trick the car's software, making it think the speed limit was 85 mph, 50 mph over the speed limit, with just a piece of black tape.

Each time, the vehicle misread the speed limit sign. The results were the same, with "a high rate of reproducibility" McAfee wrote in its blog post.

"Our results were relatively consistent in getting the MobilEye camera to think it was looking at a different speed limit sign than it was." McAfee wrote.

While the test was done in a controlled environment, it showed the limitations of image recognition software used in Tesla vehicles, or any other vehicles that rely on camera-based machine learning image recognition software for automated driving or advanced, speed-sensitive cruise control, or even vandals tampering with highway signs.

The sign on the left was altered with a small piece of black tape, tricking the Tesla into identifying the speed limit as 85 mph.

McAfee stressed that the experiment was not to discredit Tesla or its partner Mobileye that developed the camera used for the image recognition software. Rather, the researchers were trying to highlight the vulnerability of such systems so automakers can make improvements.

Prior to making their findings public, McAfee reached out to both Tesla and Mobileye 90 days ago, giving both companies ample time to address the issues or deliver a response to the findings.

However, the same Mobileye camera is used by other automakers for their advanced driver assists systems (ADAS), including General Motors, Audi, BMW, Volvo and Nissan, so the problem may not be isolated to older Tesla models.

McAfee said that MobilEye's EyeQ3 camera is used by some other vehicles to determine the speed limit, which it then displays on the vehicle's heads-up display (HUD). The company said that some vehicles may feed that speed limit to certain features of the car related to automated driving.

Tesla acknowledged McAfee's findings and says the issues would not be fixed in that older generation of hardware. While Mobileye doesn't consider tricking the camera to be an attack, despite the vital role of the camera in supporting Tesla's cruise control. MobilEye said that its EyeQ3 camera is not designed for autonomous driving, but for ADAS. 

However, there are still many older Tesla models on the road with the vulnerable hardware. McAfee said that Teslas with the first version of hardware cannot be upgraded to newer hardware. 

Tesla writes in the owner's manual of these older Tesla cars that, "Traffic-Aware Cruise Control is designed for your driving comfort and convenience and is not a collision warning or avoidance system. It is your responsibility to stay alert, drive safely, and be in control of the vehicle at all times."

Tesla no longer uses the Mobileye EyeQ3 cameras and developed its own proprietary cameras and said that the new version is not susceptible to this same attack.

FutureCar Staff