Recent interview with Waymo co-CEO that provides a window into their AI capabilities (long interview, summary further below as well).
(At 4 million miles per week, Waymo is on pace to more than double their cumulative miles this year to go from 170 to over 400 million. At 500,000 rides per week, they average 8 miles per ride. To continue their scaling across more metro areas, they need to not be at fault for a bad accident.)
Interview
interview_with_waymo_coceo_dolgov
Summary
The “Teacher-Student” AI Architecture
One of the most significant technical revelations is how Waymo has moved beyond simple modular systems to a sophisticated foundation-model-driven approach:
- Offboard Foundation Model: Waymo starts with a massive offboard model that understands the physical and social aspects of the world [07:31].
- The Three Teachers: This foundation is specialized into three high-capacity “teacher” models:
- The Waymo Driver: The core intelligence that handles driving.
- The Simulator: A generative model that creates realistic, closed-loop synthetic environments for training [08:15].
- The Critic: An opinionated model that identifies interesting events and judges whether a behavior was “good” or “bad” [09:18].
- Distillation: These large teacher models are then “distilled” into smaller, highly efficient “student” models that can run real-time inference on the vehicle’s onboard computer [09:50].
Generalizability & “Zero-Shot” Learning
Dolgov emphasizes that Waymo is no longer in the “research” phase but in a “global scaling” phase, enabled by AI that generalizes across domains [23:57].
- Inheriting World Knowledge: By hooking the Waymo Driver into VLMs, the system inherits general world knowledge. This allows for “zero-shot” or “few-shot” learning, where the car can understand new environments or signs without needing specific training data for every new city [25:30].
- The “Tick-Tock” Cycle: Waymo’s scaling strategy mimics a hardware/software “tick-tock” cycle. Generation 6 features a brand-new custom vehicle and sensor stack (the “tick”), while keeping the software relatively consistent (the “tock”) to prove generalizability across different vehicle platforms [36:04].
Novel Technical Insights
- “X-Ray” Vision through Multi-Modal Fusion: Dolgov shares a striking example where a Waymo vehicle “saw” a pedestrian through a bus. It wasn’t magic or literally seeing through metal; rather, the AI detected extremely noisy peripheral LiDAR reflections bouncing under the bus off the pedestrian’s feet. The models were sophisticated enough to interpret those few noisy bits as a human and predict their path [45:11].
- Cloud vs. Local Inference: While all driving-critical inference happens locally on the car, Waymo uses the cloud for “nice-to-have” tasks. For example, after a ride, an offboard model checks the car for left-behind items or messes to decide if it needs to go to a cleaning depot [06:13].
- Generative Simulation vs. Pixels: Dolgov explains the debate over “end-to-end” (pixels-to-actions) systems. He argues that while pure end-to-end is impressive (the “talking horse” effect), it is too inefficient for safety at scale. Waymo uses an augmented architecture that maintains intermediate representations (concepts like “roads” and “signs”) because they provide the “knobs” necessary to run efficient simulations and specify reward functions for the “Critic” [18:53].
Hardware: The 6th Generation & Cost Reduction
- Custom Vehicle Design: The upcoming 6th generation is a custom-designed vehicle (not a derivative of a consumer car like the Jaguar I-PACE) featuring sliding doors, a flat floor, and a living-room-like interior [32:43].
- Drastic Cost Reduction: The Gen 6 sensor stack is a “fraction of the cost” of previous versions, bringing the price of a fully autonomous hardware suite down to a level comparable to high-end consumer Driver Assist (ADAS) systems [35:40].
- Sensor Strengths: He clarifies why LiDAR and Radar are both essential: while LiDAR provides fine-grained mapping, imaging radar is critical for high-speed freeway driving in dense fog because its physics allow it to see through particulates that blind cameras and LiDAR [39:13].
Operational Efficiency
- The “Autonomous Dance”: Waymo depots are now highly automated. Cars automatically navigate to depots for charging or cleaning, use icons on their “sensor dome” to signal staff what they need (e.g., a cleaning emoji), and self-orchestrate their movements within the facility [52:14].
- Global Expansion: As of the recording, Waymo is operating in 11 U.S. cities (with Nashville being the most recent “ghost city” launch) and is preparing to launch in * London and Tokyo [24:42].