Waymo shares some of its lessons from AI driving.
We watch how different companies execute AI driving and we learn.
Excerpts on black-box versus modular modeling with human intelligence:
fewer models are better, that does not mean we’re consolidating our processing into a single, black box
it’s hard to understand how decision making happens in full E2E systems
This is a separate, AI-based safety system that monitors every trajectory proposed by the Waymo Driver. It checks these plans against hard physics-based constraints and traffic laws by incorporating techniques like Reinforcement Learning and reasoning inspired by Generative AI.
Waymo’s 10 AI Lessons from Autonomous Driving
- Multimodal sensors are indispensable. Cameras alone are not enough for full autonomy; a combination of cameras, lidar, and radar creates a rich, redundant, and safe worldview.
- HD maps are a powerful “prior.” High-definition maps jump-start the validation process and act as an important layer of baseline knowledge for navigation.
- Fewer, larger models are better. The industry has shifted away from numerous specialized modules (e.g., separate modules for pedestrian detection or traffic light color) toward fewer, larger foundation models.
- You can’t build trust with a “black box.” AI safety relies on transparency and a system designed to be “demonstrably safe,” rather than relying on pure, uninterpretable end-to-end models.
- Closed-loop simulation reveals more edge cases. Testing rare, highly dynamic scenarios (like sudden highway cut-offs) is far more effective and safe inside closed-loop simulated environments than in the real world.
- Every great Driver needs a great Critic. Engineering a helpful, internal “critic” AI to evaluate choices and cross-check decisions improves overall navigation safety.
- Vision Language Models improve scene reasoning. Driving requires complex, chain-of-thought reasoning that is heavily enhanced by integrating Vision Language Models (VLMs).
- Our holistic approach shows AI is only as effective as the governance that evaluates it. Building the autonomous driver is only part of the challenge; robust, rigorous internal governance and evaluation frameworks are what make it scalable.
- A data flywheel enables continuous improvement. Continuous, real-world data collection and machine learning loops drive ongoing behavioral and safety enhancements.
- There is no substitute for fully autonomous experience. Simply iterating on a driver-assist system (L2) is a false summit; true L4 maturity requires a purpose-built system hardened by driving with no human in the car.