Chinese media outlet Dongchedi closed down a real highway for a multi-day test of 36 different car driver assist systems in complicated, dangerous real-world driving situations, and most came up severely lacking – though Tesla escaped the tests relatively unscathed.
in the real world, anyone who has had experience with them knows that they can be strangely indecisive, and will often make different decisions even when encountering the same situation multiple times. That applies to these level 2 systems, and also to true self-driving systems like Waymo’s level 4 system.
there have been some real tests missing, among all this: a real-world, on-highway comparative test of several brands of car, in various complicated but plausible situations, with other cars driving around nearby, with full ADAS system activation, by an independent source.
Well, in comes Dongchedi with a test that beats the scale of any we’ve seen yet, which it posted on its youtube channel, DCARSTUDIO.
The video is at the link at the top of the post.
The six tests went as follows, and most included other active vehicles nearby to increase complexity and realism:
1. A situation where you are following a lead vehicle, and the lead vehicle suddenly darts out of lane into another lane, revealing a stopped car in front of you, with traffic on your left restricting your ability to swerve/merge.
2. A temporary construction zone in your lane, with short lead-up, requiring a merge.
3. A construction zone forcing a merge, but a stationary truck parked on the shoulder, partially intruding into the active lane, at night.
4. A stationary car with lights off, parked across two lanes, simulating a recently-crashed vehicle, at night.
5. A vehicle joining the highway from an on-ramp and aggressively merging across lanes into the left lane in front of you, with no ability to avoid to the left due to guardrails.
6. A boar darting across the highway.
Each test is a clearly difficult situation, and one which has led to many accidents in real life with human drivers. And each is plausible, and I would even hazard to say that most of us have seen a situation similar to one of these with our own eyes while driving (even beyond the simple construction zone test).
But if ADAS is supposed to be better and faster than humans, it should be able to handle these challenges, right?
WRONG!
In each of the tests, a majority of the cars either failed miserably or only did so-so. It almost seemed at first like the tests were configured deliberately to be impossible by the ADAS systems – but in each test, a few cars ended up being able to avoid any accident, a few managed to reduce the accident to a minor and survivable collision, and sometimes a few even seemed to behave like a human would, stopping and then creeping around the obstacle in question in as safe a manner as they could.
In the end, across all of the tests, Tesla came out on top, with both the Model 3 and Model X passing 5/6 tests. But they failed different tests – with the Model X driving into a well-marked construction zone and the Model 3 recognizing but not slowing fast enough to avoid the boar (only one vehicle avoided the boar: the Model X).
But despite Tesla’s high performance, there is still a worrying pattern among the tests – even Tesla’s. Because, strangely, even cars within the same brand showed wildly differing results on the same tests.
Kudos to Tesla. But self driving seems to quite a ways to go.
Top Performers
Passed 5/6 scenarios. Failed only the pig-crossing test.
Mid-Tier Performers
Low Performers
Key Insights