NTSHA data say that one accident occurs every 600,000 miles.
I asked Perplexity:
Prompt: on average, one event occurs in each 600,000 episodes.
What are the odds that three events would occur in 7000 episodes?
Response: The probability that three events would occur in 7,000 episodes—when the average rate is one event per 600,000 episodes—is approximately 0.000026%, or about 2.6 in 10 million episodes.
Calculation Details
The average event rate (λλ) for 7,000 episodes is 7000/600000≈0.011677000/600000≈0.01167.
Using the Poisson distribution, the probability for exactly three events is very low due to the rare event rate.
Interpretation
Such an occurrence would be extremely unlikely under the stated conditions, essentially expected less than once in many millions of similar trials.
The wild thing about this is that the passengers could see the debris hundreds of feet away and leaned in (as if getting 18 inchers closer in a car moving 60 MPH helps one see better) to get a better look - but the car of course didn’t slow down at all. You can see the driver hesitate to take the wheel - and probably should have as the resulting broken sway bar and damaged suspension will likely run over $1000.
Props to the influencers for being honest and not editing it out.
I didn’t check the math or specific assumptions, but this is a hypothesis test:
“Under the null hypothesis that Tesla robotaxi has the same accident rate as the average human driver, what is the probability that I would observe 3 accidents?”
(depending in your exact null hypothesis “Tesla’s rate is the same or lower than a human”, often you would calculate the p-value, the probability of observing an event as or more extreme (in the larger direction) than the 3 events you observed, so the total probability of 3, 4, 5, …, etc accidents in 7k miles)
That’s why people should think carefully about what is typically a rare event (ballpark 1 accident per million miles, but very dependent on exact driving scenario) and then what it means when
we don’t observe any events over few miles (10s of thousands) versus
we do observe accidents over few miles
The former doesn’t tell us too much about Tesla’s rate, other than “it’s not super large.”
The latter tells us with much more certainty “it’s not small.”
Again, I think the above reasoning has been communicated in a few threads.
So that’s why when people make small sample statements “It drives great for me.”
Or statements like:
They should think about the reasoning above when we repeatedly observe events like the rail crossing
and many others that have been documented, like this one:
All of this is silly. All of it refers to testing a new mode of operation. Hardly surprising there are a few accidents … about which we know very little.
It is a little surprising, though. Given that this isn’t being presented as a “test” of a possible robotaxi, but instead is the launch of their service to paying customers, one would not expect them to have had multiple accidents on so little driving. Especially with employees acting as safety monitors still in the cars.
It’s also surprising given how confident Tesla has been about how close FSD is to being able to….well, fully self-drive. The sample size here is ridiculously tiny, so you’re absolutely right that the “real” accident rate might be very different indeed from what we observe in this first bit of data. But multiple accidents in so little driving - with souped-up FSD and an employee in the car as a safety monitor - makes it very unlikely that FSD is anywhere close to human levels of safety. There’s every chance that it’s much better than an accident every 2.5K miles, of course. But there’s very little chance that this version of FSD is anywhere close to human driving.
Plus, you wouldn’t normally expect to “test” this kind of service on paying customers. “Fake it till you make it” is part of Silicon Valley start-up DNA, but you typically don’t do that when people’s physical safety is at stake. Part of what got Theranos in so much trouble was failing to recognize that the rules are different for your free photo-sharing app than for medical services where people can actually get hurt if they don’t work as advertised.
I actually think it would be very surprising that any company would actually launch if they had any expectation that they would rack up several accidents in so minute a level of operation. But given Musk’s intensely high risk tolerance, I suppose it makes sense that he’d be willing to move ahead with product launch under these circumstances.
Anyone know when the next tranche of data gets reported? All of this will out, eventually….
I was surprised. Tesla is quick to brag they have 50 bazillion miles or whatever of FSD data that informs their driving model. A person could reasonably conclude this is not an entirely new mode of operation.
If they are actually serious about this robotaxi release, one would think they have a reasonably good idea how well it functions. Keep in mind the service is overseen by a safety operator as well. In this case we have humans + technology that perform worse than just humans.
That would also be quite surprising. If this is due to an identifiable and reparable flaw, why go ahead with the launch? It’s not like they didn’t test the new version before launch, presumably by driving far more than 7K miles with it before actually launching to the public. Though if they didn’t do that, it would also be very surprising.
History is replete with examples of new versions of software that passed a bunch of tests and then came up with flaws, some embarrassing, when released more generally.
Sure. Fake it until you make it. Software often gets released in a buggy condition, with patches to come after. Because most software isn’t used in a context where an “embarrassing” flaw can end up getting people hurt or killed. Motor vehicle transportation is different. There’s vastly more effort to make sure that there aren’t any embarrassing flaws by the time a car is released more generally. It’s not perfect either, and recalls happen, but still.
Plus is there any indication that the first phase here involved robotaxi being “released more generally” than the test phase? They had only a dozen or so cars, and a still limited geographic area, and employees serving as safety monitors in the launch phase. It’s not like they went from a small invite-only beta to a global launch and exposed themselves to vastly more (and different) use cases - these seem to have been the same cars, doing the same thing, in the same location, during the test phase as the launch phase. Or if not, there’s no earthly reason why Tesla couldn’t have tested exactly the same conditions that would exist at launch during the test.
Again - it’s just strange that they would run into multiple accidents in so limited a launch. But I imagine that we’ll see whether that persists, as Tesla continues to release their accident data.