Autonomous vehicles would have to be driven hundreds of millions of miles and, under some scenarios, hundreds of billions of miles, to create enough data to clearly demonstrate their safety, according to a new report by the Rand Corporation. Under even the most aggressive test-driving assumptions, it would take existing fleets hundreds of years to log sufficient miles to adequately assess the safety of the vehicles when compared to human-driven vehicles, according to the analysis.

The report—“Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?”—suggests that, in order to advance autonomous vehicles into daily use, alternative testing methods must be developed to supplement on-the-road testing. Such methods might include accelerated testing, virtual testing and simulators, mathematical modeling, scenario testing and pilot studies.

“The most autonomous miles any developer has logged are about 1.3 million, and that took several years. This is important data, but it does not come close to the level of driving that is needed to calculate safety rates,” says Susan Paddock, co-author of the study and senior statistician at Rand. “Even if autonomous vehicle fleets are driven 10 million miles, one still would not be able to draw statistical conclusions about safety and reliability.”

The report's authors note that, while the total number of crashes, injuries and fatalities from human drivers is high, the rate of these failures is low in comparison with the number of miles that people drive. Americans drive nearly 3 trillion miles every year, according to the Bureau of Transportation Statistics. In 2013, there were 2.3 million injuries reported, which is a failure rate of 77 injuries per 100 million miles driven. The related 32,719 fatalities correspond to a failure rate of 1.09 fatalities per 100 million miles driven.

By comparison, Google’s autonomous vehicle fleet, which currently has 55 vehicles, after 1.3 million miles in autonomous mode had been involved in 11 crashes from 2009 to 2015. The report's authors note that a study carried out earlier this year by the Virginia Tech Transportation Institute, comparing Google’s fleet performance with human-driven performance, found that while Google’s fleet might result in fewer crashes resulting in only property damage, conclusions could not be drawn about the relative performance in terms of two critical metrics: injuries and fatalities.

"Given the rate of human and autonomous vehicle failures, there were simply not enough autonomously driven miles to make statistically significant comparisons," Rand notes.

Miles and years required to be logged to demonstrate autonomous vehicle reliability. Image credit: Rand Corporation.Miles and years required to be logged to demonstrate autonomous vehicle reliability. Image credit: Rand Corporation. Rand uses statistical theory to determine how many miles, and how long, a fleet of 100 autonomous vehicles (larger than any known existing fleet) would have to be driven—assuming they operated 24 hours a day, 365 days a year, at an average speed of 25 miles per hour—to demonstrate certain levels of confidence about autonomous vehicle reliability. Among their findings: it would take 400 years and 8.8 billion miles to demonstrate with 95% confidence their failure rate to within 20% of the actual (human) rate of 1.09 fatalities per 100 million miles.

To contact the author of this article, email GlobalSpeceditors@globalspec.com