They probably collect data from driving in more countries than just the USA? For example, Norway has a quite large penetration of Teslas for a number of years now. Maybe they collect a lot of snow driving data from there?
I don’t think a single commercial (sold to public) car on the road today will ever be truly fully self-driving. I think none of them have the processing power, and maybe also lack sufficient sensors, necessary to do so. I drive a Tesla for nearly 2 years, and the way it reacts, and more importantly, the time it takes to react, to certain things will likely preclude true full self-driving. Maybe a switch from HW3 to HW4 could do it, but not if additional sensors are needed. And I am still somewhat skeptical if HW4 will be sufficient anyway.
After a surprisingly strong first quarter set against the backdrop of a sluggish economy, GM (GM) is ready to supercharge its EV production to take on Tesla (TSLA).
“Demand remained firm,” GM CFO Paul Jacobson told Yahoo Finance Live. Jacobson said GM saw sales strength in both lower-priced vehicle trims such as Chevy and higher-end plays like Cadillac.
Jacobson told us GM will aim to double its EV production in the second half of this year as it continues to improve its battery capacity and introduce new models. The company is targeting the production of 50,000 EVs by the end of the first half of this year, he said.
50,000 plus the 20,000 they sold in the first quarter 2023 is still just a fraction of Tesla’s. But it is building. And their new EV pickup enters the fray in 2024.
Thanks for the link. Unfortunately it is written for insiders and in my old age I’m an outsider.
Spot on! Reminds me of a sales meeting at NCR. One sales rep reported that a prospect had objected to buying a cash register saying, “I have 20 years experience!” Sr. Hansen, our sales manager, suggested that maybe the prospect had one year’s experience repeated 20 times.
Tesla is quite aware of this problem and the solution they are using is to generate training scenarios using the kind of software game developers use. While this is a good idea in some ways it also has drawbacks. These scenarios can have the biases of their authors. You can see this kind of bias in schools, a Marxist professor, a conservative professor, a transgender professor likely teach quite differently.
I definitely agree that quality and diversity of the data is important. One of the fundamental concepts in machine learning is the bias-variance trade-off, where models will tend to over-fit if the training data is too small and/or not diverse enough compared to the prediction or application it will be used for. That’s the reason why these transformer neural nets require so much data–so that they can begin to “learn” the true edge-case correlations without being over-fit to spurious correlations.
While data quantity doesn’t necessarily mean more quantity/diversity, I think on the whole as the dataset expands so will the variation embedded within that dataset–one of the best known ways to reduce variance in a model is larger datasets (unless there’s a huge sampling bias). Tesla’s dataset likely is biased towards California and the southwest US, but as they scale production and FSD enabled cars, they will collect more and more data from other parts of the US (and worldwide) as well. Tesla also uses synthetic data and simulations, but I have to imagine that some edge cases can only be experienced through real-world data where the probability and frequency increases with larger datasets (which can then be augmented for further simulations).
When zooming out and looking at competitors in this space, Waymo for examples does have a driver-less taxi currently available, but it’s geo-fenced in an area like Phoenix, avoids highways, and takes routes that have less difficult turns (here’s an interesting comparison video: https://www.youtube.com/watch?v=2Pj92FZePpg).
Tesla’s main potential advantage in this space is their scale and the fact they have millions of FSD enabled cars to collect this massive, variable dataset to (hopefully) build a generalizable autonomous system that can respond to almost any environment.
I totally agree that lots of edge cases must be generated by collecting lots of real world driving. There are several reasons for this I’m sure. Certainly one of them is that in order to simulate an edge case one has to be creative enough to think about it and real life can probably think of lots more things than a group of people sitting around computers can.
OTOH, I know of at least a few real world edge cases that it would be very difficult to record in billions of miles of data collection. For example a friend of mine was in the left lane of a freeway, a very small shoulder on the left, multiple cars on the right and an 18-wheeler in front of him when the trailer detached. How often is an edge case like that going to get recorded by a Tesla? Certainly not enough times to significantly affect the training of the neural net.
(Note: my friend survived and was able to attend a wedding the next day. His wedding)
Glad to hear that your friend was OK in a scary situation.
And, it sounds like a low probability event, but low probability events have more and more frequency with massive datasets. Something with a 1 in 1 million mile chance of happening will have 50 occurrences in a 50 million mile dataset, but will have 5000 occurrences in a 5 billion mile dataset.
I believe Elon’s quote was “on the order of 6 billion miles”, which maybe it takes 10 or 20 billion (or more), but ultimately the only way to experience low-probability events is to have more and more opportunity to catch them.
Also, maybe that exact edge case won’t be experienced many times, but generalizing to large debris or vehicle flying off path will increase the frequency of that “population” of events. These neural nets learn through correlations of similar events, so it might not have experienced a trailer detaching exactly, but has seen a car losing traction in the rain on the highway and swerving into on-coming traffic. The model sees a large mass object with unusual velocity/trajectory and can “group” those events together within certain nodes/area of the net, similar to how LLM’s will create nodes that recognize certain long-range language patterns and context.
I think his IQ is fine. Musk is pathologically addicted to being the center of attention, and he’s found he can get that attention by making, wild, unsupportable predictions. For some reason, we keep listening to him. To clear, not everything he says is bogus. But he’s cried wolf enough I’m am extremely skeptical of anything he says.
His IQ is fine but I think his EQ has degraded substantially - probably due to a confluence of factors.
Not many years ago, he was being lauded for his high EQ:
I think he may have never had much in the first place (some of which might be due to his self proclaimed Asperger’s) and his recent elevation to being the world’s richest person, coupled with his addiction to attention via twitter, have simply exposed the absence of such.
I think my point was that the only (or best?) way to experience unknown low probability events is to collect data for billions of miles, thus collecting enough of these events to train with. But the best way to collect enough known low probability events is to create them in a simulator. This may involve combining multiple low probability events, such as a tire blow out in a construction zone and a car ahead blows through a red light. Easy to think of, easy to simulate but hard to record in real life.
As a bonus, the simulator can repeat this at all times of day and night, in all weather conditions with varying amounts of car and pedestrian traffic, etc.
True. Then there’s mud on the roads in Alabama, hail in Texas, severe wind in Oklahoma, and lumber coming loose from a trailer and spilling on the road right in front of you, and so on.,
Edge cases. A billion miles might not get you a single one, but a human can spot them even without having seen one before.
The bicyclist killed by a self-driving car in Arizona was pushing the bike across a four lane highway at dusk, not in a crosswalk where the software (it is claimed) would have reacted. But the company says “she appeared out of nowhere” (or words to that effect) even though she had already walked across two open lanes to the collision point.
An edge case, to be sure, but there are probably a billion of those.
I don’t know how you get to the edge cases without real world driving, but I don’t know that doing so is safe, either.
Part of the problem is human psychology. We would prefer to have humans drive 10 billion miles and accidentally kill 10 people than to have computers drive those 10 billion miles and accidentally kill 1 person*. Either way those 10 billion miles will be driven.
It’s probably a matter of how much we pay attention to things - when a human has an accident and kills someone, it makes the local paper, maybe it makes the local news for 15 seconds, but that’s it. But when a computer kills someone, it makes ALL the papers, and the news has special full segments about it, and it continues for months, everywhere, across the world, and even years later people talk about it and post links to it. And it gets a full wikipedia entry that remains as part of documented history forever.
* And it’s even worse because if we humans allow the computers to drive, eventually they will only kill 0.1 people every 10B miles, and if we allow them to start talking to each other in real-time, perhaps even only 0.001 people every 10B miles. Etc.
I think that’s why we’re likely to see regulators force these various driving systems to rack up some serious real-world driving miles under constraints to see how they actually perform in the real world. That’s going to also be valuable data - not just the training data set of watching oodles and oodles of miles driven by humans, but the far smaller but still critical data set of the miles that are actually driven by the AI, so that we can try to figure out how it’s doing and what it’s doing.
While that is valid, I think it still misses the mark a bit.
To quote your link:
No, I think the fundamental problem to adoption is that the machines are going to make mistakes that we never would, even though there will be many more times when they save our lives in scenarios where any other driver would just get themselves buried. Literally buried. In other words, the mistakes the machine is going to make will not be the types of mistakes we would make. And the mistakes are going to be remembered far more often than the successes.
Machines are very likely safer in the aggregate but not necessarily so for the individual. We are all above average, of course.
As individuals, we don’t usually make decisions based on the aggregate. Take 9/11 for example. After 9/11, people decided to drive instead of fly - even though, in the aggregate, there are far more accidents and fatalities from driving than flying. Regardless, everyone alive today that decided to drive instead of fly after 9/11 can say that they avoided the risk of dying on a plane and of course did not die from driving.
As individuals, we have some control, some ability to reduce our individual risk (even if only in our heads) but when sitting in a plane, we have no control over the randomness of an air disaster.
People will apply that same decision-making process to self-driving. We don’t care that in the aggregate, it is safer. As an individual, I may drive safer than the average (or think that I do) and I also have the means to eliminate those weird edge cases - those different mistakes.
It really puts wide adoption of SD in a tough position - and that is without someone of influence coming out and trying to stop it from wide adoption (our modern day cancel culture) for whatever reason.
Edge cases are something we have to do in the verification of new microprocessor designs, my career for about 30 years now. These designs are verified through a testing strategy known as directed random testing. Testcases are random, but not completely random. We can influence the range of likely values for any parameter we want in order to steer testing to some part of the design or another. And then you measure the coverage of those tests – where did the design actually go? Are we not stimulating certain features or aspects of the design?
Sometimes the constraints on randomness are minor and tests are very random. Sometimes the constraints are tight and the tests is more directed than others.
The good part of this is it works, the bad part of it is you need to think up all these corner cases. Or rather, you think up all the coverage points, the parts of the design that you need to measure if it was hit or not.
The bicycle case baffles me. It was an obstacle in the path, why did it matter if it was in the crosswalk or not? And it should not matter that it was a bicycle, person, another car, a deer - it is an obstacle that could be hit.
Isn’t this pretty much what Tesla has been doing? They have the car drive itself, but the constraint that they require a human driver in the driver’s seat, while paying attention to the road, with their hands on/near the wheel.
IMO Musk has more than a bit of P T Barnum in his personality. He knows any utterances will be reported upon. And any publicity (bad or good) bring Tesla to public notice. Trump used that factor in his 2016 run. The Main Stream Media thought they were killing him with their criticism. But it just brought him national recognition.
Today I read this assinine article:
It might be time for Tesla TSLA –4.18% (ticker: TSLA) to advertise.
LOL Musk/Tesla is always in the news which is free publicity. Why paid for it?
I don’t know about you but when I hear or read EV…Tesla pops into my mind.
Mostly that’s because, as we discussed in the other thread, it’s up to the driver to choose whether or not to activate FSD (and Autopilot, I believe). You’ve got a lot of self-selection there. Drivers are going to be less likely to activate FSD and Autopilot in challenging circumstances - which are where a lot of the “edge cases” are going to be hiding, and where the highest rates of disengagement are going to take place. Their observed rates of disengagement are going to be artificially improved, because the systems are running disproportionately in situations where they are especially likely to do well.
Also, you don’t have complete data about what the vehicle will/would do when disengaging isn’t an option. With Cruise or Waymo, their vehicles are (mostly) on their own - when they encounter an edge case, the AI in the car is going to make the first response to that edge case, not a human driver. That doesn’t happen (I believe?) with FSD or Autopilot - when the FSD disengages, it’s now up to the human to start driving again.
