This episode explores Google's transformation into an AI company, featuring insights from co-hosts Ben Gilbert and David Rosenthal. They examine how Google, despite inventing the foundational Transformer architecture, faces an innovator's dilemma as AI threatens their core search business.

The story traces Google's AI journey from early language models in 2001 through the current Gemini era, highlighting key figures like Jeff Dean, Noam Shazir, and the eight researchers who created the Transformer. The narrative reveals how Google's early AI dominance led to the founding of OpenAI and the current competitive landscape.

Key themes include Google's unique position with complete AI infrastructure, the strategic challenges of protecting search revenue while embracing AI, and major acquisitions like DeepMind. The discussion draws extensively from In the Plex by Steven Levy, Genius Makers by Cade Metz, and Supremacy by Parmi Olson to chronicle this technological transformation.

The Origins of Google's AI Ambitions

Larry Page's father was a computer science professor who did his PhD in machine learning at University of Michigan, influencing Larry's vision of Google as fundamentally an AI company from the beginning.

In 2000, Larry Page declared: 'Artificial intelligence would be the ultimate version of Google. If we had the ultimate search engine, it would understand everything on the web and give you the right thing.'

The foundational work began with Georges Herrick and Noam Shazir's lunch conversation about data compression and understanding, as documented in In the Plex, leading to Google's first language models.

The Cat Paper and Neural Network Breakthrough

Google Brain's 2012 'Cat Paper' trained a nine-layer neural network to recognize cats from unlabeled YouTube videos, proving large neural networks could learn meaningful patterns without supervision.

Jeff Dean described the breakthrough: 'After a little while, that model was actually able to build a representation where one neuron would get excited by images of cats. It had never been told what a cat was.'

This discovery enabled YouTube's recommendation system and content understanding, driving hundreds of billions in revenue over the next decade through improved video recommendations and content classification.

The DeepMind Acquisition and OpenAI's Formation

DeepMind was founded in 2010 by Demis Hassabis, Shane Legg, and Mustafa Suleiman with the tagline 'solve intelligence and use it to solve everything else,' as detailed in Supremacy.

Peter Thiel funded DeepMind's seed round after Demis pitched him at a Singularity Summit after-party, with Shane Legg noting they needed 'someone crazy enough to fund an AGI company.'

Google acquired DeepMind for $550 million in 2014, beating Facebook's higher offer because Larry Page understood and supported DeepMind's research mission better than Mark Zuckerberg.

Elon Musk's frustration over losing DeepMind to Google led him to co-organize a dinner with Sam Altman in 2015, recruiting AI researchers to found OpenAI as a nonprofit alternative.

The Transformer Revolution and Google's Missed Opportunity

Eight Google Brain researchers published 'Attention is All You Need' in 2017, introducing the Transformer architecture that would revolutionize AI, as chronicled in Genius Makers.

Noam Shazir rewrote the entire Transformer codebase from scratch, with teammates calling him 'a magician' and 'a wizard' for making the architecture actually work at scale.

The paper has been cited over 173,000 times, making it the seventh most cited paper of the 21st century, yet all eight authors eventually left Google for other companies.

Google built BERT and other Transformer-based models but treated it as incremental improvement rather than the platform shift that would enable ChatGPT and the modern AI era.

ChatGPT's Launch and Google's Code Red Response

ChatGPT launched November 30, 2022, reaching 100 million users in two months - the fastest product in history to hit that milestone, forcing Google to declare 'Code Red.'

Google's initial response with Bard was poorly executed, including a factual error in the launch video that caused Google's stock to drop 8% in a single day.

Sundar Pichai made two critical decisions: merging Google Brain and DeepMind under Demis Hassabis, and standardizing on one model called Gemini for all Google AI products.

The company brought back Noam Shazir through a $2.7 billion deal with Character AI, pairing him with Jeff Dean to co-lead Gemini development.

Google's AI Infrastructure Advantage

Google ordered 40,000 NVIDIA GPUs for $130 million in 2014, when NVIDIA's total revenue was only $4 billion, essentially funding NVIDIA's transformation into an AI company.

The company developed Tensor Processing Units (TPUs) in just 15 months, creating custom AI chips that provide 2-3x cost advantages over NVIDIA's 80% gross margin GPUs.

Google now processes over 500 trillion tokens quarterly across services, with inference volume growing 50x from April 2024 to June 2025, demonstrating unprecedented AI deployment scale.

Waymo's Autonomous Vehicle Success

Waymo originated from Sebastian Thrun's 2005 DARPA Grand Challenge victory, where Stanford's team used machine learning to combine laser and camera data for autonomous navigation.

The project completed Larry Page's '1000-mile challenge' within 18 months, autonomously driving difficult routes including Lombard Street and Highway 1 to Los Angeles.

Waymo now operates in five cities with hundreds of thousands of paid rides weekly, showing 91% fewer crashes with serious injuries compared to human drivers.

The potential market impact represents $420 billion in annual cost savings from accident reduction, making Waymo potentially as valuable as Google's search business.