After many failures AI finally landed on neural network technology which is the closest we have to how the brain works. The Universe is about size. The Universe is so huge we can’t see all the way to the edge, the speed of light is not fast enough. The human brain has billions of neurons and it has 100 trillion synapses, connection. Size matters.
Rocks have no brains, no intelligence. Ants have tiny brains yet they can travel great distances by just follow a scent. Many get lost but their numbers are so large that it doesn’t matter for the survival of the species. Size matters.
In essence neural networks store vast amount of patterns and match current input against the store. The inference output is the most statistically likely but not necessarily the correct one. The larger the network the higher the probability of getting the right answer. The faster the network, the higher the probability that the answer arrives in time to be useful. Size matters. Speed matters.
Neural network technology ran into a road block. After 25 to 30K GPUs they are so spread out, so far from each other that they cannot talk to each other with the current software based technology. From 30K GPUs to the brain’s billions of neurons is a long, long distance. Size matters. Speed matters.
Experts threw up their hands and forgot that anything you can do in hardware you can do in software and anything you can do in software you can do in hardware. Hardware is faster but static, software is slower but pliable. If you can solve a problem with software it will be more cost efficient than doing it in hardware. Most stuff is done with software.
Horizons
The earth’s curvature creates a horizon for the observer because light does not bend enough.
The universe is so vast that it creates a particle or cosmological horizon because light is not fast enough.
Black holes create an event horizon because they are so massive and dense that their gravity lets nothing escape (except Stephen Hawking proved they can).
Neural networks also have a horizon of around 25 to 30K GPUs because software is too slow. This horizon might be called the coherence horizon because beyond it the GPUs cannot talk to each other anymore.
Elon and his team, working from first principles, said, “If software is too slow let’s do it in hardware.”
Watch all about it!
The Captain
