To show, statistically, that an AI driver is 25% better than a human (incident rate is 75% of human rate), one power analysis (with 80% power) estimated that about 28 million driverless miles are needed when the incident is defined as “police reported accident.”
Rarer incidents like injuries and fatalities will require more miles. More common incident measures (less severe) will require fewer miles.
As an aside on 80% power:
This means if you were to do this measurement of 28 million miles and the AI driver incident rate is actually 75% of the human rate, 80 out of 100 times of doing this measurement (collecting 28 million AI driver miles 100 different times), you would observe a statistical difference between the AI rate and the human rate.
Waymo study (https://www.tandfonline.com/doi/full/10.1080/15389588.2024.2380522)
In one month in Austin, Tesla did 7,000 miles of supervised miles (not human driverless).
So, today, as far as we know, Tesla is not even getting the data needed to estimate true AI (no human supervision) safety.
It’s difficult to read the above and think Tesla is close to releasing an unsupervised driving product at scale for public driving scenarios.
(If they can’t build the product in any reasonable time, maybe Tesla should try to buy the product from someone else?
Are there any AI driving products for sale or license?)
This is one reason why one person cannot conclude from their 10s of thousands of miles of FSD driving (and likely much, much fewer miles) that an AI driving product is “good enough.”
Albaby has been saying the above over and over across many posts and threads.
But people keep citing single examples or experiences as evidence of product capability. Such examples show “possibility” not “performance generalizable to a large scale product.”