Self-Driving Tech Was Supposed To Be Safer, Experts Say It May Bring More Risks

It might seem counterintuitive, but a new report argues that the better consumer autonomous driving systems get, the more dangerous they actually can be. It comes as practically every automaker has some form of autonomous driving system either available or in development, with companies like Tesla pushing regulators to allow its fully autonomous Cybercab to be available for public purchase.

Experts have dubbed the phenomenon “automation complacency,” says the New York Times. It happens when users quite literally become too complacent, assuming that the automated systems that are in use can handle any situation. Level 2 systems, such as Tesla’s dubiously named Full Self-Driving system, still require a human to be attentive and ready to take control. And, importantly, in the event of a collision, that distinction means that the legal liability rests solely on the driver.

Read: The City That Never Sleeps Now Has Taxi Drivers That Don’t Need To Sleep

It’s something that many drivers have already found out the hard way. Kyle Shanahan, head coach of the San Francisco 49ers had used Tesla’s FSD for years before he crashed into head-on traffic. Speaking at a press conference in August, he told reporters how he had wrongly assumed that the system was engaged and bent over to pick up his phone. He took full responsibility for the crash, which left him with broken bones and a concussion after his car veered into the oncoming lane.

He also said that the crash had changed the way he thinks about the use of autonomous driving. “When you’re going at 20 miles an hour, and you’re on autopilot, I honestly have zero worries,” he said, according to Business Insider. “That was a good lesson that you can’t ever have zero worries. I think it definitely could be an example for a lot of people.”

How Do You Pitch Autonomous Tech?

Current self-driving systems available in consumer cars still need to be supervised by a driver and differ greatly from driverless taxis such as the ones deployed by Waymo. However, people are already relying on autonomous tech for many of their activities. The NYT quotes 72-year-old Carolyn McBride, a Tesla Model Y owner. She claims that she struggles to see at night and that her Tesla prevented her from hitting a pedestrian.

Tesla itself leans into the idea of using FSD as an aid. On X, it reposted a testimonial from a man with cerebral palsy who claimed that his Tesla with FSD allowed him to go wherever he wanted, whenever he wanted. The company described its system as “a guardian angel for you, and others on the road.”

It’s clear to see why this kind of advertising could be seen as dangerous. Despite the immense strides that Tesla’s autonomous systems have undoubtedly made, they’re not designed to compensate for any impairment and still require competent and able drivers to be behind the wheel, alert and ready to take over.

See Also: I Tried Out GM’s Smarter Super Cruise And It Feels Promising

Guy Mangiamele is the director of AMCI Testing, a firm dedicated to evaluating new car tech. While he praised Tesla’s advancements, he cautions against its limitations. “Maybe it will take 500 miles or 1,000 miles, but at some point it’s going to get something wrong,” he said. “If you’re not 100 percent aware and ready to take control, you’re going to be a danger to yourself and others around you.”

Rivian is getting ready to launch its own version of self-driving tech. Similar to Tesla’s FSD, it will allow drivers to take their hands off the wheel and will be capable of navigating any well-marked street from point to point. Also similar to Tesla’s system, it remains at Level 2, meaning that drivers are still responsible for the vehicle, and will need to remain alert.

But rather than pitching it as a system to help make up for driver shortcomings, James Philnin, Rivian’s Senior Vice President For Autonomy, angles it as a way for drivers to avoid fatigue and stay awake. “The biggest danger is actually drivers getting tired,” he says. “These systems can dramatically improve that.”

The Arms Race For Real Autonomy

Tesla may dominate the headlines, but a number of mainstream manufacturers are edging in on the autonomous race. Ford has announced plans to launch a Blue Cruise Level 3 driver-assistance system in 2028, which will allow drivers to take their hands off the wheel and their eyes off the road. Meanwhile, GM is doing similar with the next generation of Super Cruise, and Rivian will be following suite too, with a system that allows its cars to naviaget around any clearly marked road.

What is clear is that there’s a race to offer fully autonomous cars to the public. To do so, there needs to be significant regulatory changes. But even with those potentially being enacted, the timelines remain murkey. James Philbin, senior vice president for autonomy and artificial intelligence at Rivian estimates it will take five years or less. Bryan Reimer, a research scientist in the M.I.T. Center for Transportation and Logistics, offers a more pessimistic view. “It’s decades,” he said, “not years.”