Tesla's AI Day 2022

Stocks A to Z
41

Not sure I understand why something like that would be especially useful for robots or GAI mind development, where you don’t have (or necessarily need) billions of hours of video that needs to be processed.

GAI tries to emulate human intelligence. As I mentioned above, we have two brains, the rational, boolean brain and the pattern matching brain that we share with most animals. For pattern matching to work we store all the patterns we have ever encountered in our whole life. They have a recall gradient, recent ones are easy to remember, early ones are much more difficult to recall. Apparently hypnotism or drugs can bring them back.

Nature is not efficient, it relies on abundance. Just one s-perm gets to fertilize one egg, all the other s-perms and eggs are redundant! Many species lay thousands of eggs but few survive to adulthood. Predators seldom destroy whole herds, it would be collective suicide. The large size of herds is what guarantees their survival. Pattern matching is no different, there is no way of knowing if a new pattern will or not be useful in the future, the safe thing is to store it. It’s the huge brain that makes this possible. You are right, no way of knowing “why something like that would be especially useful for robots.” I can imagine one use case, an Optimus Robot is tasked to drive a vehicle that has no auto pilot!

GAI really needs a new way of looking at the world! It’s the hugest paradigm shift of our lifetimes. The secret of GAI is overkill in storage, machine learning, and pattern matching. That’s why Dojo is designed for overkill.

The Captain

Sorry, you can’t post the word ‘s-perm’; it’s not allowed.

4 Likes
42

Sure, more computing power is probably better in almost any case - but if you don’t have billions of hours of video of people working in a factory or picking tomatoes or what have you, then is having that big of a supercomputer much of an advantage?

The Optimus robot is just a doll until it becomes intelligent. Initially it will be trained by humans, that’s the reason to start it on Tesla’s factories. The workers will be collecting the data, several thousands of them doing all sorts of chores. At the same time Tesla can make simulations to train the robots. As the robots start doing these jobs they will start collecting their own data. It won’t be long before Dojo needs to be expanded some more.

Like I said above, “GAI really needs a new way of looking at the world!”

The Captain

5 Likes
43

Will it?

I guess that’s the question. I can see why Dojo is necessary for the specific way that Tesla is trying to solve autonomous driving - using enormous amounts of video data. Because there’s so much data, you need a bigger-than-ever computer to efficiently teach your neural nets.

But when it comes to generalizable AI, you don’t have that kind of dataset. You’re not going to ever generate as much video data in a Tesla factory as is provided by the hundreds of thousands of vehicles driving hundreds of hours over several years. So do you need that big a computer for any real purpose? Is it useful for AI training if you don’t have the kind of massive dataset that Tesla’s using for FSD? Or is Dojo mostly a bespoke device that’s mostly useful for just this one type of problem?

44

Would it help if you knew that all the excitement over convolutional neural nets over the last few years was started in 2012 with Alexnet? FYI, Alexnet operates on the Imagenet dataset (a few million still images divided into 1000 classes ) and processed at 224 x 224 pixels.
Imagine the simple idea of processing HD video in 10 second clips and a million classes of actions to be trained on.
That is roughly 30,000,000 times as much processing power required.
Certainly some better data formats and some other algorithm tricks learned in the last 10 years will reduce that requirement by 10x or even 100x. But even when Moore’s Law was increasing chip throughput by ~2x every ~2 years that was only ~32x in 10 years.
So, there’s not really a shortage of potential demand to solve really hard problems when it becomes economically possible to do the computations.

Mike

2 Likes
45

Would it help if you knew that all the excitement over convolutional neural nets over the last few years was started in 2012 with Alexnet?

No because albaby’s doubt is not about the power of the computer but not understanding the Optimist robot’s learning scope. Of course it is small now but it will grow exponentially – with a large exponent-- once it starts operating.

The Captain

3 Likes
46

Interesting stats on this topic. 44 replies but only 8 users. Berkshire board attracts more users.

Where is everyone?

47

Eight people contributed to the Topic, but 43 likes gives a better picture of readership.

I’m guessing Berkshire had a broader appeal, with Tesla more of an acquired taste. Or distaste, for some.

48

OMG it’s great to see you here Denny! Been missing your posts. So Tesla, SpaceX and Twitter! Quite a combo huh? :grinning:

How is Porto? I know this is all off topic….

Lucky Dog

1 Like
49

It’s actually not a doubt about the scope of Optimus robot’s learning. I understand that the size of what it needs to learn to be a general-use robot is massive.

My doubt is about the size of the dataset available to Tesla.

Tesla has a dataset for FSD that is massive beyond description - untold hours of video from hundreds of thousands of cars driving in literally millions and millions of different places. That dataset is supposed to be the secret sauce for Tesla in cracking AV - no other car company has access to that much computer-analyzable data about real-world driving. It is easy to see why Tesla would need to build a massive computer in order to process that dataset.

But do they have an analogous dataset for training Optimus? You could put cameras all over the handful of Tesla factories, watching all the workers engage in their more-or-less repetitive tasks - and you’d never get within two or three orders of magnitude of the FSD data. Does GAI have - or need - a dataset that big in order to train on?

That’s my question. Is having a very big (but not the biggest) supercomputer analogous to having, say, one of the world’s biggest dry-docks for shipbuilding? Something that’s useful - indeed necessary - if you have enough materials to build one of the world’s biggest ships (like a cruise ship), but utterly unnecessary if you only have enough materials to build a moderate-sized yacht? It’s easy to see how Dojo is critical for solving the task that Tesla has set itself for FSD - creating Level 5 autonomous vehicles using mostly video data. But is it useful for solving GAI, which doesn’t have the same video dataset to crunch?

1 Like
50

But do they have an analogous dataset for training Optimus?

Of course not! Tesla didn’t have massive beyond description FSD data five years ago. In five years we’ll have “massive beyond description Optimus data.”

The Captain

2 Likes
51

Really? Because five years ago, Tesla had already started building the FSD data - turning on the video sharing in what was then almost a hundred thousand cars with nearly a million cameras. And it was obvious - and much talked about - that this number would expand exponentially as more cars were sold, and thus more cameras collecting more video would be in more parts of the world.

Is there anything similar happening now to collected data for Optimus? Any similar path forward for expanding that data collection? Even if Tesla were, in fact, stringing its factories with cameras to feed into an Optimus data file, how could that possibly expand as fast as turning the switch on a million already-out-there cameras and increasing that number exponentially year after year?

And honestly, how much video do you need in order to compile the training database for factory work? Driving conditions are insanely complicated, with gazillions of different possible parameters for any given moment; factory work is insanely repetitive, with relatively narrow parameters for what the worker will encounter playing their part on an assembly line.

1 Like
52

Thats because 10 or 12 years ago, Tesla/Musk asked themselves “where are WE/,Tesla going to be in 50 years and how do we get there? What do we need to have accomplished in 5 years, 10 years, 20 years, etc, in order to get to our goal?”

Tesla is following the "I skate to where the puck is going to be, not where it has been.

Wayne Gretzky"

Tesla will put a couple TeslaBots on the floor in its factories, collect data, and using that data put more bots to “work”.
And, using FIRST PRINCIPLES concepts, as more “learning” happens, using OTA, TeslaBots will IMPROVE EVERY DAY, perhaps even several times PER DAY?

Cause that’s Tesla’s philosophy.

Why are you limiting TeslaBots to Tesla factories? Sure, that’s where they’re likely to be deployed first.
But soon-ish (2024? 2026?) TeslaBots will be in other situations, INCLUDING highly public spaces?

Why are you limiting the dataset to smaller than that for FSD? Are there fewer unique or “edge cases” in “work environments” than in driving environments?
If the TeslaBot environment is expanded to private homes, nursing homes, health care, service industry applications, etc… are there fewer unique or edge cases?

Tesla/Musk has stated that they expect to eventually build more bots per year, than cars per year. Each TeslaBot will be IoT, always connected to the internet (and Dojo, the mothership), just as Tesla vehicles are now continually sharing data back to the mothership.
Will all the data those bots collect truly produce a “smaller data set than FSD”?

Dont forget, all the vehicles (cars, trucks, cybertrucks, etc), will continue to send data to Dojo.
Add to that, all the data from TeslaBots.
The compute needs are just going to increase?

Dojo is being built. The software to process the GAI data is being developed. Should Tesla NOT use it for the TeslaBot?

Dojo.
Tesla has floated the “rent processing time on Dojo” idea a few times. Maybe Dojo will be used for other tasks, external to Tesla’s needs?

Tesla is following Musk’s vision.
10 to 15 years ago, all the experts poopoo’d Musk’s vision.
But, Musk keeps on succeeding.

:alien:
ralph

4 Likes
53

I thought their philosophy was to accelerate the world’s transition to sustainable energy. :grinning: As with IGU, I"m not sure what robots have to do with that.

Anyway, I’m not sure that holds up, because you have a ‘chicken and the egg’ problem that you don’t have with cars. You don’t need the ‘brain’ in a car to be very advanced - or even a ‘brain’ at all - to get lots of cars on the road. Tesla could start collecting the data to make the product smart by selling a dumb version of the product - where the driver is doing all the work for now, while the cameras record what the driver is doing and the choices the driver made.

That doesn’t really work for robots, does it? It’s not like they can sell people a robot suit for them to wear, laden with cameras, to start collecting all that delicious data for them long before there’s enough AI to drive the robot on their own. Unlike cars, you can’t get the robots into public spaces until there’s enough brains behind the robots for them to be useful in public spaces.

So - to return to my question - why exactly would Dojo be all that attractive to people who might be working on robotics? From what you’ve descried, Tesla’s plan is to start having a fully functioning humanoid robot for use in public spaces (presumably different from getting an Asimo up on a convention stage) within the next 2-4 years. IOW long before there’s any dataset large enough to require Dojo. Which would be amazing - but also something that makes Dojo rather superfluous. Because to answer this question:

…the answer is yes, YES, a hundred times yes! Because Tesla was able to sell hundreds of thousands of cars without an AI. Because cars are useful without an AI. Which means you can use the non-AI cars to collect the data necessary to create the Car AI. But humanoid robots are not especially useful without an AI - which means you can’t really use them to collect the data necessary to create the AI. So Tesla will not be able to deploy hundreds of thousands of TeslaBots years in advance of having the AI that can completely operate them without human intervention.

Only if it’s useful for the TeslaBot. Right now, it’s clearly useful and needed for solving the specific way that Tesla is approaching automotive autonomy . As I’ve asked several times, I can’t see how having Dojo is useful for the TeslaBot, which doesn’t have - and isn’t likely to have - the massive video dataset that Tesla has (which is specific to driving). If you don’t have the dataset to feed into that massive a computer, then how is that attractive to robotics boffins?

1 Like
54

Not at all. That’s their mission. Their engineering philosophy is best captured by, I think, by several statements.

  1. Reason from first principles.
  2. The best part is no part.
  3. The pace of innovation is key.
  4. Manufacture at scale.

So everything Tesla makes is designed to be manufactured at scale. Other oft-stated design goals are safety, efficiency, and affordability. These are kind of obvious, especially affordability, as if you manufacture at scale it’s useless if people can’t afford what you’ve built.

3 Likes
55

The thing, perhaps, that you are missing (besides my prior calculation on how video datasets require massive more processing to train with) is that with a more powerful dojo (or dojo-like) AI training machine, you can train with bigger datasets or you can train more modest datasets more quickly. Reducing the turn around time for training from days and days to hour or minutes is a great accelerator for the humans.

All you say about Tesla (or anybody) lacking a big dataset to train a humanoid factory robot is true. But then if you don’t build a few bots and start building the dataset you’ll never get there. And you can also augment the dataset building with simulations and generate random or human assisted cases to help accelerate the training. Of course these simulations require heavy duty hardware as well.
You can train competing AI functionality with only real datasets vs those with simulations vs those with both and accelerate the human understanding of the progress. Of course, all this requires 3x or more access to HW capability.
And what potential robotics/AI company has the dozen or more factories that Tesla has access to that they can instrument with cameras for robotics vision? Factories include cars/trucks, motors, other car parts, batteries, grid electrical assemblies, solar panels, EV chargers, rockets, satellites, tunneling equipment, etc.) Tesla probably also has easy access to willing beta testers.

Mike

5 Likes