The new version of Tesla Motors' Autopilot includes several significant safety improvements, including tighter limits on how long drivers can keep their hands off the wheel. But the semi-autonomous driving system still raises real concerns.

That's what we found after driving our Tesla Model S test car in Autopilot mode after the car received a software update called Tesla 8.0. (Tesla updates its cars wirelessly, much like Apple does with iPhones.) The update included several changes to Autopilot. It now has additional warnings for drivers to keep their hands on the wheel, and forces a restart of the car to use the feature if the driver ignores repeated warnings.

After evaluating an earlier version of the semi-autonomous system, Consumer Reports in July called on Tesla to make a number of changes to Autopilot.

  • Disable the Autosteer feature of Autopilot until the company requires that drivers keep their hands on the wheel. Autosteer allows the car to steer itself with minimal input from the driver.

  • Quit calling the system Autopilot because the term is misleading and potentially dangerous.

  • Issue clear guidance on how to use the system and its limitations.

  • Test all safety-critical systems before public deployment; no more beta releases.

We made these recommendations after our evaluations of the previous Autopilot version found that despite Tesla’s instructions for drivers to keep their hands on the wheel at all times, the system sometimes took more than three minutes after our tester’s hands were removed from the wheel before the vehicle gave any warning. We were also worried that the ability to drive for extended periods of time without actively steering the car, as well as the name Autopilot, could give drivers a false sense that Model S sedans and Model X SUVs are more capable of driving themselves than they actually are.

Meantime, Tesla's Autopilot came under scrutiny by regulators after a fatal accident in May. The National Highway Traffic Safety Administration and National Transportation Safety Board are investigating the accident. Consumer Reports is monitoring the investigations, and will report the findings from both as soon as they are made public.

And just last week the California Department of Motor Vehicles released draft regulations that seek to define the circumstances under which carmakers can use terms including “self-driving,” “automated,” or “auto-pilot.” Specifically, the proposed rules would bar the use of these terms unless a vehicle is truly autonomous.

More carmakers are offering semi-autonomous driving systems. For instance, Mercedes-Benz has a system called Drive Pilot, BMW’s is Active Driving Assistant Plus, and Volvo’s is Pilot Assist. They have the potential to improve safety, but we can’t make that determination until there is a sufficient body of evidence in the field comparing accident records of cars with the systems, to those without. In the meantime, Consumer Reports believes that as long as these systems require driver engagement, then they also should require drivers to keep their hands on the wheel.

Tesla’s new Autopilot is better but still needs improvement
Model S instrument panel in Autopilot mode.

Tesla's recent changes to Autopilot have addressed some, but not all, of our concerns.

“We heard some encouraging statements from Tesla a few weeks ago when the company first announced its 8.0 upgrade in terms of improving the safety of Autopilot,” says Jake Fisher, auto test director at Consumer Reports. “But the only way to know if the company has really addressed our concerns is for us to evaluate the upgrade. Now that we have the new software on our Model S, it’s clear that they’ve made improvements to the system. We appreciate these changes and encourage Tesla to keep working to make the system safer."

During our evaluation, we found that Autopilot checks up on you more frequently than before. While the timing depends on the character of the road—whether it's curvy or has many lanes, for instance—you'll generally see the first indicator to put your hands back on the wheel within a minute of taking them off. One similar system, Mercedes Drive Pilot, which we’ve also evaluated, warns drivers within about the same time frame. Other systems do so after about 15 seconds.

Autopilot has also made the visual notification to the driver more apparent. When the car alerts the driver to put his hands back on the wheel, it first displays the message “hold steering wheel” on the instrument panel and then begins flashing the edges of the panel, a new feature.

If you still don't put your hands on the wheel, you'll hear an audible warning. After the car beeps loudly three times, Autopilot shuts down. The only way to restart the system is to pull over, turn the car off and turn it back on. The Mercedes system, by comparison, does not force drivers to restart the car to re-engage the system if they fail to hold the steering wheel.

Autopilot also is more restrictive now about the roads where it allows you to activate the system. If the car senses that the road is too twisty or the road markings disappear, Autopilot won't engage. Our experience with the system also found more roads where the driver can only set Autopilot to 5 miles per hour over the speed limit.

However, despite Tesla’s improvements to Autopilot we have some lingering issues. Someone can drive hands-free for about a minute and even longer on highways. The system still is called Autopilot. It also remains a beta release, a term used in the technology world when essentially unfinished software is rolled out to the public. (Tesla CEO Elon Musk said last month that the system is not a true beta release and that the company calls it that to reduce people’s comfort level with turning the system on.)

"While Tesla and other automakers may be continuously improving their semi-autonomous systems, we believe they should take stronger steps to ensure that these systems are designed, deployed and marketed safely,” Consumer Reports’ Fisher says.