Waymo's AI Development
Evaluations are key to deploying reliable AI models, as seen in Waymo's self-driving car technology
Waymo, a self-driving car company, relies heavily on AI models to navigate streets and make split-second decisions. The company's approach to managing risks associated with AI deployment includes continuous evaluation, carefully curated data, human oversight, and clearly defined business outcomes. According to Manasi Joshi, Waymo's director of engineering for systems intelligence and machine learning, the company has driven over 220 million fully autonomous miles with significantly fewer serious crash injuries than human drivers.
Waymo's method of development, known as “eval-forced development” or “eval-centric development,” prioritizes evaluation as a core part of the engineering process. This approach ensures that AI models are thoroughly tested and validated before deployment.
The company's focus on evaluation and testing has yielded impressive results, with 17 times fewer serious crash injuries than human drivers over the same distance.
Why it matters
The success of Waymo's AI-powered self-driving cars demonstrates the importance of rigorous evaluation and testing in AI development. As AI is increasingly deployed in various industries, the need for reliable and safe AI models becomes more pressing.
The success of Waymo's AI-powered self-driving cars demonstrates the importance of rigorous evaluation and testing in AI development.
What you can learn from this
- Evaluation should be a core part of the AI development process, not just a final check before deployment
- Continuous testing and validation are crucial for ensuring the reliability and safety of AI models
- Clearly defined business outcomes and human oversight can help mitigate risks associated with AI deployment
- Prioritizing evaluation and testing can lead to significant improvements in AI model performance and safety
We teach this
Sources
- At Waymo, an AI project isn't ready until its evals are — not when the model performs well — VentureBeat
Our reporting is an original summary; full coverage is at the links above.
Don't just read about it — build it.
Square 1 teaches the skills behind the headlines, with every line of your work graded by AI. Find your starting point in 3 minutes.
Get your free skill report