Tesla's AI Day 2022

Stocks A to Z
#32

But, I think, you only think that way because for your whole life this has been happening. If you go and study how ENIAC worked all the formulas were basically hard wired. There was no terminal where you typed in a program in ~english-like text and a compiler to convert it to the machine’s language.

No it can’t. But the tools and process used to create and train DALL-E are pretty much the same as the tools used to do most other AI tasks. There are multiple different tools sets that are similar, but better at different things, such as the “frameworks” like Tensorflow, PyTorch, MXNet just like there are different programming languages that one might use to solve other problems – like Fortran, C/C++, Basic, Pascal, Python (and a hundred others).

I think that was and is the point of Tesla’s AI Day. To recruit the best of the best to solve all these last few percent problems and build it. Not easy for sure, but probably not rocket science either. I doubt they will meet the stated time line. And I think the reality will be that the first products are much more restricted in what they are able to do. Maybe work on one stage of an assembly line in a car factory.

I recall working on big SW projects with a few dozen people that took overnight to compile and build, never mind the slow regression testing and manual testing after that. Progress was very slow due to the long turn around to test the most simple things. Lots of time was spent just trying to optimize the infrastructure. Many current ML models take many hours, days or even weeks to train. Iterating to fix things takes a LONG time. Improving this by 10x, for example, can get you more than a 10x productivity improvement just because the programmers can try more things.
Did you see the Tesla Dojo project?
What genius college grad wouldn’t want this?

Mike

2 Likes
#33

Yes, clearly. It reminded me very much of half the law firm pitch meetings I went to, when the managing partner would spend a lot of time talking about home awesome their pro bono environmental program was in an effort to lure recruits in - even though 98% of their firm was in securities and corporate restructuring.

I’m not in the field, so I have no way of assessing whether that “we’re not really a car company” pitch is something they can succeed at. Tesla is clearly presenting itself as an AI or Robotics company…but at the end of the day, I imagine whether it is or not depends on the consistent long-term strategy of the company’s leadership. If you’re a hotshot recruit, do you want to end up at Tesla - where the robotics team is a small part of a large company that has a hundred thousand employees who are almost entirely making and selling cars - or at a shop like OpenAI or Boston Dynamics, which are smaller enterprises but are entirely devoted to your field?

It’s the conglomerate problem. Not everything can be the main priority of the company. Tesla’s making batteries, solar cells/roofs, cars (obviously), AI, and now robots. I would think that a genius recruit in solar technology research probably wouldn’t put Tesla at the top of their list (though maybe that’s wrong?), given that it’s not really a priority of the company these days. Tesla’s core business today is making cars - it’s 90% of what they do. I would think their core future business priorities would be better batteries and teaching the cars to drive themselves, at least for the next decade or so. Is making real advances in something specific to robotics like, IDK, robot tactile feedback systems going to be a priority for a company like that, compared to other things?

I suspect that’s why Google Lab spins off their companies, so that someone who works for EveryDay Robots (like people who work for Boston Dynamics) know they’re working for a robotic shop, rather than the robotics side project of a Search Engine company.

I did - but I confess, I don’t have the technological background to understand it. From what I can gather, they’re in the process of trying to build an extremely powerful bespoke supercomputer. One that is capable of analyzing, if I’m using the technical term correctly, many many buttloads of video data for use in training an AI. So while some other computer might only be able to analyze several peta-buttloads of visual data, the Dojo will (when done) be able to analyze several exa-buttloads of visual data. As the man with the rather impenetrable accent pointed out, 3.7x more buttloads.

Not being a computer boffin, I don’t know if that has appeal to a genius college grad. From the AIDay presentation, I couldn’t tell whether Dojo is somehow better than other supercomputers around the world, or whether it’s able to do this because it’s designed specifically to solve this one type of problem and thus can do it faster than other supercomputers (which get used for disparate problems like climate models and economic forecasts and figuring out how the heck turbulence works or whatever). Certainly it’s going to almost entirely be used to solve this one specific type of problem. I don’t know whether that appeals to genius college grads generally, but it should appeal greatly to the ones who want to work on developing software for autonomous cars.

#34

They aren’t just building a supercomputer. They are building the chips that they use to make it. This will be either genius or a total fail as they may spin it off like Amazon did for AWS. Remember how dumb of an idea that was. Not.

The cars are so profitable that there seems to be leeway to go into new/related businesses. How interesting would Amazon be just being the best at selling books? Imagine Google just doing search.

Mike

3 Likes
#35

Well then, let’s try this. Imagine that you have to write a complex brief for a client. It requires at least half a dozen meetings for you to agree on exactly what’s needed, and then another half a dozen back and forth reviews of the document to get it into final form. You need to work through imperfect communication, imperfect understanding, and imperfect environmental issues. Not to mention applying your actual expertise of figuring out the relevant law and how it works in this case.

This is quite similar to what an engineer must go through to produce a software solution to a problem. Figuring out specifications, what the customer really wants, a good way to accomplish that, and then getting agreement that it’s what’s desired. Lots of back and forth. Lots of work in between.

What Dojo does is the engineering equivalent of making most of those client meetings go ten times as fast. So you finish your draft of the brief and instead of having to wait a day to get it back all marked up, you get it back in five minutes. Everything is still fresh in your mind and you can proceed with the next round of issues immediately.

I started writing software in the days when you punched up your program on cards, submitted your card deck to an operator, and got back a pile of paper to stare it. Eventually. At best you could do that a couple of times a day. And each time you did that it was expensive! Things have gotten better slowly over the years.

What Tesla showed machine learning engineers is that they have taken another leap in making turn-around time quicker. This means spending more of your time being productive, which means doing a better job faster.

Is that more digestible?

4 Likes
#36

For a moment forget everything you know about computing. Machine learning is nothing like what has come before. There are no human created algorithms that solve problems. You need to envision a completely different paradigm - pattern matching.

Let’s start with babies, how do they learn? Some of their ‘knowledge’ is built into their DNA which deals mostly with the physical, the mechanics of how their body works. The rest of their ‘knowledge’ is acquired through their senses. How did Pavlov train the dogs? Repetition. Do this, get reward. Do something else get no reward. Rinse, repeat. Rinse, repeat. Rinse, repeat. Data! The dog’s neural network stored these ‘patterns’ (don’t ask me how, I don’t have a clue). The next time Pavlov rang the bell the dog matched the sound or the event against its stored patterns and somehow (don’t ask me how, I don’t have a clue), the weighings of the stored patterns directed the dog to salivate. That is about all AI machine learning is.

Why rote learning? Training your neural nets with lots of repetitive data.

Why do pilots have to accumulate flying hours? Training their neural nets with lots of repetitive data.

Why Malcolm Gladwell’s Big Idea: 10,000-Hour Rule? Training their neural nets with lots of repetitive data.

That is about all AI machine learning is. What does Tesla’s FSD have to do with the Optimus robot? At Tesla they keep talking about ‘full stack FSD.’ What is a stack? Learning about a set of tasks like taking left turns, recognizing pedestrians, etc. On a greater scale city driving, highway driving, parking lots, etc. Full stack is just joining all these stacks into one big stack (don’t ask me how, I don’t have a clue).

The neural network algorithms don’t care what the subject matter is about, they just store weighted patterns and match incoming data against the stored patterns. Create a bunch of ‘stacks’ separately (in school, chemistry, geometry, algebra) and join all the stacks → General AI! That is about all AI machine learning is.

Why Dojo? For the same reason that humans have huge brains. You need vast amounts of memory, not on tape or disk but in chips, and vast amounts of parallel computing power to match incoming data against the vast amount of stored patterns. To reach their goals Tesla could not rely on off the shelf hardware so they built their own.

The Captain

Strange as it may seem the above ties in with my very first professional program. The algorithms I created were too big for the limited computers of the time (IBM 650 - 1960). I told my boss the program was too big and didn’t fit in the computer. His reply, “It fits.” Try as hard as I could my rational, boolean brain could not find a solution. Back to my boss, same reply, “It fits.”

One morning at around 4 AM I woke up with the solution, “It fits!” The punched cards I had to process had an unorthodox coding system which I tried to convert to standard code and that required a lengthy algorithm that exceeded the computer’s memory. The solution was to create a table with the valid unorthodox codes and the equivalent standard code. Just a small table and the software’s ‘table lookup’ did the rest. Pattern matching!

The AI related part of the story is that the subconscious brain gave me the solution, not the rational, boolean brain – pattern matching? And the solution was certainly pattern matching, match the input data against the valid data in the table, a go-no go solution. No need to know what the data actually was, just if this, do that!

4 Likes
#37

Yes, thank you. I think I have a better understanding of why Dojo might be exciting to someone trying to solve the problem of autonomous driving. Still not sure I understand why it would really be relevant to a young engineer interested in robotics, though, or why Tesla has decided to build a robot. It doesn’t seem like it would be relevant to anyone working on the physical body of a robot. And while it might be useful for developing a ‘robot brain,’ it doesn’t seem like it’s actually going to be used for that - or anything other than working out FSD - any time soon. I would think the FSD team will have top priority (and near-exclusive access) to the Dojo resource for quite a while going forward.

And is it going to be particularly useful or necessary for building a robot brain? I understand why having that type of a supercomputer would be useful and necessary for Tesla’s specific approach to AV. They’ve got available to them a gazillion video files of driving, and they’re trying to solve Level 5 with almost entirely visual data (ie. no Lidar and few other non-camera inputs) - so I can see why having a machine that can process a gazillion exa-widgets of video data is useful to them. Is that the bottleneck for other AI research? To use the legal brief analogy, a very fast reviewer is enormously helpful for meaningfully speeding up the review time of 400-page briefs - but far less useful for a 12-page motion. Is having access to massive amounts of video file review a real bottleneck for GAI research?

#38

Yes, I think I understand - thanks for your summary and IGU’s.

As noted in my prior response, I can see why Dojo is necessary for how Tesla is trying to solve the specific problem of Level 5 autonomy - they have literally billions of vehicle-hours of video to serve as ‘repetitive data’ that they need to expose their neural networks to. Not sure I understand why something like that would be especially useful for robots or GAI mind development, where you don’t have (or necessarily need) billions of hours of video that needs to be processed. Sure, more computing power is probably better in almost any case - but if you don’t have billions of hours of video of people working in a factory or picking tomatoes or what have you, then is having that big of a supercomputer much of an advantage?

#39

Totally OT, but it’s good to see you back on the boards, Captain.

2 Likes
#40

Totally OT, but it’s good to see you back on the boards, Captain.

SpeyCaster, thank you!

The Fool community has been a welcome companion for over 20 years, entertaining and educational, and often just plain silly. It would be sad to lose it. Fortunately I stumbled on a way to avoid the “New, Improved, Crappy, Useless, Website Navigation Calamity.” I get emails of the posts that interest me. I read them in my mail client and occasionally I ‘Visit Topic’ to reply.

Win-win: Keep the community, avoid the website clutter.

BTW, I edit my posts and replies in my own Editor and save them on my computer.

The Captain

3 Likes
I quit! Title must be at least 15 characters. You got it. Want more?
#41

Not sure I understand why something like that would be especially useful for robots or GAI mind development, where you don’t have (or necessarily need) billions of hours of video that needs to be processed.

GAI tries to emulate human intelligence. As I mentioned above, we have two brains, the rational, boolean brain and the pattern matching brain that we share with most animals. For pattern matching to work we store all the patterns we have ever encountered in our whole life. They have a recall gradient, recent ones are easy to remember, early ones are much more difficult to recall. Apparently hypnotism or drugs can bring them back.

Nature is not efficient, it relies on abundance. Just one s-perm gets to fertilize one egg, all the other s-perms and eggs are redundant! Many species lay thousands of eggs but few survive to adulthood. Predators seldom destroy whole herds, it would be collective suicide. The large size of herds is what guarantees their survival. Pattern matching is no different, there is no way of knowing if a new pattern will or not be useful in the future, the safe thing is to store it. It’s the huge brain that makes this possible. You are right, no way of knowing “why something like that would be especially useful for robots.” I can imagine one use case, an Optimus Robot is tasked to drive a vehicle that has no auto pilot!

GAI really needs a new way of looking at the world! It’s the hugest paradigm shift of our lifetimes. The secret of GAI is overkill in storage, machine learning, and pattern matching. That’s why Dojo is designed for overkill.

The Captain

Sorry, you can’t post the word ‘s-perm’; it’s not allowed.

4 Likes
#42

Sure, more computing power is probably better in almost any case - but if you don’t have billions of hours of video of people working in a factory or picking tomatoes or what have you, then is having that big of a supercomputer much of an advantage?

The Optimus robot is just a doll until it becomes intelligent. Initially it will be trained by humans, that’s the reason to start it on Tesla’s factories. The workers will be collecting the data, several thousands of them doing all sorts of chores. At the same time Tesla can make simulations to train the robots. As the robots start doing these jobs they will start collecting their own data. It won’t be long before Dojo needs to be expanded some more.

Like I said above, “GAI really needs a new way of looking at the world!”

The Captain

5 Likes
#43

Will it?

I guess that’s the question. I can see why Dojo is necessary for the specific way that Tesla is trying to solve autonomous driving - using enormous amounts of video data. Because there’s so much data, you need a bigger-than-ever computer to efficiently teach your neural nets.

But when it comes to generalizable AI, you don’t have that kind of dataset. You’re not going to ever generate as much video data in a Tesla factory as is provided by the hundreds of thousands of vehicles driving hundreds of hours over several years. So do you need that big a computer for any real purpose? Is it useful for AI training if you don’t have the kind of massive dataset that Tesla’s using for FSD? Or is Dojo mostly a bespoke device that’s mostly useful for just this one type of problem?

#44

Would it help if you knew that all the excitement over convolutional neural nets over the last few years was started in 2012 with Alexnet? FYI, Alexnet operates on the Imagenet dataset (a few million still images divided into 1000 classes ) and processed at 224 x 224 pixels.
Imagine the simple idea of processing HD video in 10 second clips and a million classes of actions to be trained on.
That is roughly 30,000,000 times as much processing power required.
Certainly some better data formats and some other algorithm tricks learned in the last 10 years will reduce that requirement by 10x or even 100x. But even when Moore’s Law was increasing chip throughput by ~2x every ~2 years that was only ~32x in 10 years.
So, there’s not really a shortage of potential demand to solve really hard problems when it becomes economically possible to do the computations.

Mike

2 Likes
#45

Would it help if you knew that all the excitement over convolutional neural nets over the last few years was started in 2012 with Alexnet?

No because albaby’s doubt is not about the power of the computer but not understanding the Optimist robot’s learning scope. Of course it is small now but it will grow exponentially – with a large exponent-- once it starts operating.

The Captain

3 Likes
#46

Interesting stats on this topic. 44 replies but only 8 users. Berkshire board attracts more users.

Where is everyone?

#47

Eight people contributed to the Topic, but 43 likes gives a better picture of readership.

I’m guessing Berkshire had a broader appeal, with Tesla more of an acquired taste. Or distaste, for some.

#48

OMG it’s great to see you here Denny! Been missing your posts. So Tesla, SpaceX and Twitter! Quite a combo huh? :grinning:

How is Porto? I know this is all off topic….

Lucky Dog

1 Like
#49

It’s actually not a doubt about the scope of Optimus robot’s learning. I understand that the size of what it needs to learn to be a general-use robot is massive.

My doubt is about the size of the dataset available to Tesla.

Tesla has a dataset for FSD that is massive beyond description - untold hours of video from hundreds of thousands of cars driving in literally millions and millions of different places. That dataset is supposed to be the secret sauce for Tesla in cracking AV - no other car company has access to that much computer-analyzable data about real-world driving. It is easy to see why Tesla would need to build a massive computer in order to process that dataset.

But do they have an analogous dataset for training Optimus? You could put cameras all over the handful of Tesla factories, watching all the workers engage in their more-or-less repetitive tasks - and you’d never get within two or three orders of magnitude of the FSD data. Does GAI have - or need - a dataset that big in order to train on?

That’s my question. Is having a very big (but not the biggest) supercomputer analogous to having, say, one of the world’s biggest dry-docks for shipbuilding? Something that’s useful - indeed necessary - if you have enough materials to build one of the world’s biggest ships (like a cruise ship), but utterly unnecessary if you only have enough materials to build a moderate-sized yacht? It’s easy to see how Dojo is critical for solving the task that Tesla has set itself for FSD - creating Level 5 autonomous vehicles using mostly video data. But is it useful for solving GAI, which doesn’t have the same video dataset to crunch?

1 Like
#50

But do they have an analogous dataset for training Optimus?

Of course not! Tesla didn’t have massive beyond description FSD data five years ago. In five years we’ll have “massive beyond description Optimus data.”

The Captain

2 Likes
#51

Really? Because five years ago, Tesla had already started building the FSD data - turning on the video sharing in what was then almost a hundred thousand cars with nearly a million cameras. And it was obvious - and much talked about - that this number would expand exponentially as more cars were sold, and thus more cameras collecting more video would be in more parts of the world.

Is there anything similar happening now to collected data for Optimus? Any similar path forward for expanding that data collection? Even if Tesla were, in fact, stringing its factories with cameras to feed into an Optimus data file, how could that possibly expand as fast as turning the switch on a million already-out-there cameras and increasing that number exponentially year after year?

And honestly, how much video do you need in order to compile the training database for factory work? Driving conditions are insanely complicated, with gazillions of different possible parameters for any given moment; factory work is insanely repetitive, with relatively narrow parameters for what the worker will encounter playing their part on an assembly line.

1 Like