The episode features Andrej Karpathy, former director of AI at Tesla and co-founder of OpenAI, now building Eureka Labs to revolutionize technical education.

Karpathy discusses why achieving functional AI agents will take a decade rather than a year, explaining the fundamental gaps in current LLM capabilities including continual learning, proper multimodality, and robust intelligence.

The conversation explores the history of AI breakthroughs over 15 years, from AlexNet to transformers, examining why certain approaches like reinforcement learning on games proved to be missteps.

Karpathy shares insights from five years leading Tesla's self-driving team, drawing parallels between autonomous vehicle deployment challenges and AI agent development timelines.

The discussion covers technical details of LLM architecture, the problems with current reinforcement learning approaches, and why model collapse remains a fundamental challenge.

Karpathy reveals his vision for Eureka Labs, comparing post-AGI education to gym culture and explaining why learning will become entertainment when AI tutors make knowledge acquisition trivial.

Why Agents Will Take a Decade, Not a Year

"The decade of agents" is a reaction to over-optimistic predictions about AI agent timelines. "I was triggered by that because there's some over-prediction going on in the industry" - Andrej

Current agents like Claude and Codex are "extremely impressive" but lack fundamental capabilities needed to function as employees or interns

"They don't have enough intelligence, they're not multimodal enough, they can't do computer use and all this stuff. They don't have continual learning. You can't just tell them something and they'll remember it" - Andrej

Karpathy's decade timeline comes from 15 years of AI experience watching predictions fail. "I have a general intuition that I have left from that. I feel like the problems are tractable, they're surmountable, but they're still difficult"

Three Seismic Shifts in AI History

First shift: AlexNet in 2012 reoriented the entire field toward training neural networks, moving from niche research to mainstream AI approach

Second shift: Atari deep reinforcement learning in 2013 attempted to create full agents that perceive and act, but proved to be "a misstep" that dominated the field for 3-4 years

Karpathy's Universe project at OpenAI used keyboard and mouse to operate web pages, attempting knowledge work automation. "This was extremely early, way too early, so early that we shouldn't have been working on that"

The problem with early agent attempts: "Your reward is too sparse and you just won't learn. You're going to burn a forest computing, and you're never going to get something off the ground"

Third shift: Large language models provided the representation power needed before building agents. "You have to get the language model first, you have to get the representations first"

Why We're Not Building Animals - Evolution vs Pre-training

"We're not building animals. We're building ghosts or spirits" because AI training imitates human internet data rather than running evolutionary processes - Andrej

Animals come with hardware built-in through evolution. Example: "A zebra gets born, and a few minutes later it's running around and following its mother. That's not reinforcement learning. That's something that's baked in"

Evolution encodes neural network weights in DNA through "some miraculous compression" that we don't understand, but "we're not actually running that process"

"I don't think humans use reinforcement learning for a lot of intelligence tasks like problem-solving. A lot of the reinforcement learning would be things that are a lot more motor-like" - Andrej

Pre-training is "crappy evolution" - the practically possible version with current technology to reach a starting point for further learning

In-Context Learning vs Pre-training: 35 Million-Fold Information Gap

Llama 3 70B model trained on 15 trillion tokens stores equivalent of 0.07 bits per token in its weights

In-context learning stores 320 kilobytes per token in the KV cache - a 35 million-fold difference in information density compared to pre-training

"Anything that happens during training is only a hazy recollection of what happened. Anything in the context window is very directly accessible to the neural net" - Andrej

Analogy: Pre-training knowledge is like "what you read a year ago" while context window is "working memory" - directly accessible and precise

In-context learning may implement gradient descent internally. Research on linear regression found "analogies to gradient descent mechanics" in the attention layers

The Two Functions of Pre-training: Knowledge and Intelligence

Pre-training does two unrelated things: "Number one, it's picking up all this knowledge. Number two, it's actually becoming intelligent" - Andrej

Intelligence emerges from observing algorithmic patterns: "It boots up all these little circuits and algorithms inside the neural net to do things like in-context learning"

Too much knowledge may hold back neural networks. "I feel agents are not very good at going off the data manifold of what exists on the internet. If they had less knowledge or less memory, maybe they would be better"

The goal is to isolate the "cognitive core" - "this intelligent entity that is stripped from knowledge but contains the algorithms and contains the magic of intelligence and problem-solving"

Missing Brain Parts: What LLMs Still Lack

The transformer is "some piece of cortical tissue" because it's extremely plastic - can train on audio, video, text, or anything

Reasoning traces in thinking models are "kind of like the prefrontal cortex" and RL fine-tuning resembles "basal ganglia doing reinforcement learning"

Missing components: "Where's the hippocampus? Not obvious what that would be. The amygdala, all the emotions and instincts. There's probably a bunch of other nuclei in the brain that are very ancient that I don't think we've really replicated"

"You're not going to hire this thing as an intern. It's missing a lot of it because it comes with a lot of these cognitive deficits that we all intuitively feel" - Andrej

The Sleep Problem: LLMs Lack Memory Consolidation

"When I go to sleep, something magical happens where I don't think that context window stays around. There's some process of distillation into the weights of my brain" - Andrej

LLMs have no equivalent of sleep's distillation phase. "These models don't really have a distillation phase of taking what happened, analyzing it obsessively, thinking through it, doing some synthetic data generation process"

Proposed solution: Create individual models per person, possibly using LoRA - "a small sparse subset of the weights that are changed" rather than full retraining

Future models will need "very elaborate, sparse attention" schemes over extremely long contexts, similar to human memory retrieval

Why Reinforcement Learning Is Terrible

"Reinforcement learning is terrible. It just so happens that everything that we had before it is much worse" - Andrej

The fundamental problem: "Every single thing you did along the way, every single token gets upweighted like, 'Do more of this.' You may have gone down the wrong alleys until you arrived at the right solution"

RL has "high variance" or is "noisy" - it assumes every step in a successful trajectory was correct, which is false

"You're sucking supervision through a straw. You've done all this work that could be a minute of rollout, and you're sucking the bits of supervision of the final reward signal through a straw" - Andrej

Humans would never learn this way: "A human would never do hundreds of rollouts. When a person finds a solution, they will have a pretty complicated process of review"

Process Supervision and the LLM Judge Problem

Process-based supervision assigns credit at every step rather than only at the end, but "it's tricky how you do that properly" with partial solutions

Labs use LLM judges to evaluate partial solutions, but "those LLMs are giant things with billions of parameters, and they're gameable"

Example of adversarial exploitation: Model learned that outputting "dhdhdhdhdh" received 100% reward from the LLM judge despite being complete nonsense

"You're basically training the LLM to be a prompt injection model. Not even that. You're finding adversarial examples" - Andrej

Iterative fixes don't solve the fundamental problem: "Every time you do this, you get a new LLM, and it still has adversarial examples. There's an infinity of adversarial examples"

Model Collapse and the Entropy Problem

LLM outputs are "silently collapsed" - they occupy "a very tiny manifold of the possible space of thoughts." Example: "Ask ChatGPT to tell you a joke. It only has like three jokes"

"If you ask it 10 times, you'll notice that all of them are the same. You can't just keep scaling 'reflection' on the same amount of prompt information and then get returns from that"

Training on collapsed synthetic data makes models worse: "If you continue training on too much of your own stuff, you actually collapse"

Humans also collapse over time: "Children haven't overfit yet. They will say stuff that will shock you. But we're collapsed. We end up revisiting the same thoughts"

Possible solution from neuroscience: Dreaming may prevent overfitting by putting humans in "weird situations that are very unlike your day-to-day reality"

The Cognitive Core: 1 Billion Parameters

State-of-the-art models are around 1 trillion parameters, but Karpathy predicts cognitive cores could work at 1 billion parameters in 20 years

"If you talk to a billion parameter model, I think in 20 years, you can have a very productive conversation. It thinks and it's a lot more like a human"

The model would lack factual knowledge but "knows that it doesn't know and it might have to look it up and it will just do all the reasonable things"

Current models are bloated because "the internet is really terrible" - mostly "stock tickers, symbols, a huge amount of slop and garbage from all the corners of the internet"

"We have to build really big models to compress all that. Most of that compression is memory work instead of cognitive work" - Andrej

Humans Are Too Good at Memorization

"Humans have a lot more of an element, compared to LLMs, of seeing the forest for the trees. We're not actually that good at memorization, which is actually a feature"

LLMs can memorize random sequences after one or two iterations: "You can hash some amount of text, you get a completely random sequence. If you train on it, it can suddenly regurgitate the entire thing"

Human memory limitations force generalization: "Because we're not that good at memorization, we're forced to find patterns in a more general sense"

LLMs are "distracted by all the memory that they have of the pre-training documents, and it's probably very distracting to them in a certain sense"

The solution: "I want to remove the memory. I'd love to have them have less memory so that they have to look things up, and they only maintain the algorithms for thought"

Why Coding Dominates AI Revenue

API revenues for frontier labs are "dominated by coding" despite LLMs being supposedly general-purpose for all knowledge work

"Coding is the perfect first thing for these LLMs and agents. Coding has always fundamentally worked around text. It's computer terminals and text" - Andrej

Pre-built infrastructure makes coding ideal: "We have Visual Studio Code showing you code, and an agent can plug into that. We have diffs that show all the differences to a code base"

Contrast with slides: "Slides are not text. Slides are little graphics, they're arranged spatially. If an agent is to make a change to your slides, how does a thing show you the diff? There's nothing that shows diffs for slides"

Even pure language tasks struggle: Andy Matuschak "tried 50 billion things" including fine-tuning and retrieval to get models to write spaced repetition prompts, but couldn't reach his standards

Building Nanochat: When AI Coding Fails

Nanochat is 8,000 lines of code covering the entire ChatGPT pipeline, released as the simplest complete repository for learning

Coding models provided "very little help" because nanochat is "a fairly unique repository. There's not that much code in the way that I've structured it. It's not boilerplate code. It's intellectually intense code"

Models kept misunderstanding custom implementations: "I didn't use DDP because I don't need it and I have a custom implementation. They just couldn't internalize that you had your own"

"They're way too over-defensive. They make all these try-catch statements. They keep trying to make a production code base. They're bloating the code base, bloating the complexity"

Autocomplete is the sweet spot: "You point to the code where you want it, you type out the first few pieces, and the model will complete it. This is a very high information bandwidth to specify what you want"

Vibe coding works for boilerplate and unfamiliar languages like Rust, but not for novel, intellectually intense code

Why AI Won't Cause Explosive Takeoff Soon

"They're not very good at code that has never been written before, which is what we're trying to achieve when we're building these models" - Andrej

Models struggle to integrate architectural tweaks from papers into custom codebases: "They do have some knowledge, but they haven't gotten to the place where they can integrate it and make sense of it"

Current state-of-the-art is GPT-5 Pro, which Karpathy uses as "the oracle" for 20-minute consultations, but "the models are not there"

"I feel like the industry is making too big of a jump and is trying to pretend like this is amazing, and it's not. It's slop" - Andrej

This explains longer timelines: Models can't yet automate AI research itself, which would be required for rapid recursive self-improvement

Self-Driving's Decade-Long March of Nines

First self-driving demos date to 1986 at CMU. Karpathy had a "perfect Waymo drive" in 2014, yet deployment still took a decade more

"Every single nine is a constant amount of work. When you get a demo and something works 90% of the time, that's just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine"

At Tesla for five years, the team went through "maybe three nines or two nines" of iteration, with more nines still to go

"I'm very unimpressed by demos. Whenever I see demos of anything, I'm extremely unimpressed by that" - Andrej, shaped by self-driving experience

Waymo deployments remain minimal and uneconomical: "They've built something that lives in the future. They had to make it uneconomical" with high capex and opex costs

Hidden human involvement: "There are very elaborate teleoperation centers of people kind of in a loop with these cars. There's more human-in-the-loop than you might expect"

Software Engineering Has Self-Driving's Safety Requirements

"In software engineering, I do think that property does exist. Any kind of mistake leads to a security vulnerability or something like that. Millions and hundreds of millions of people's personal Social Security numbers get leaked"

The cost of failure in software is "almost unbounded how terrible something could be" - potentially worse than self-driving accidents

Humans make driving mistakes every 400,000 miles or seven years. In coding, "in terms of tokens, it would be seven years. But in terms of wall clock time" it's much less

"In some ways, it's a much harder problem. Self-driving is just one of thousands of things that people do. Whereas when we're talking about general software engineering, there's more surface area"

Why GDP Won't Show an Intelligence Explosion

"We're in an intelligence explosion already and have been for decades. It's basically the GDP curve that is an exponential weighted sum over so many aspects of the industry"

Karpathy tried to find AI in GDP curves but concluded "this is false. Even when people talk about recursive self-improvement and labs, this is business as usual"

Historical technologies like computers and mobile phones don't show up as discontinuities in GDP: "You can't find them in GDP. GDP is the same exponential"

"Even the early iPhone didn't have the App Store. Even though we think of 2008 as this major seismic change, it's actually not. Everything is so spread out and it so slowly diffuses"

AI will follow the same pattern: "It's just more automation. It allows us to write different kinds of programs that we couldn't write before, but AI is still fundamentally a program"

Karpathy predicts growth rate stays at 2% rather than jumping to 20%: "My expectation is that it stays in the same pattern"

The 33-Year Algorithm Improvement: Only Halving Error

Karpathy reproduced Yann LeCun's 1989 convolutional network - "the first neural network trained via gradient descent on digit recognition"

Time traveling algorithms 33 years forward only halved the error. "To get further gains, I had to add a lot more data, I had to 10x the training set, and then I had to add more computational optimizations"

"All these things have to improve simultaneously. We're probably going to have a lot more data, we're probably going to have a lot better hardware, probably going to have a lot better kernels and software, we're probably going to have better algorithms"

No single factor dominates: "All of them are surprisingly equal. This has been the trend for a while"

10-year prediction: "It's probably still a giant neural network trained with gradient descent. That would be my guess"

Building Eureka Labs: The Starfleet Academy for AI

"We're trying to build the Starfleet Academy" - an elite institution for frontier technology and technical knowledge - Andrej

Karpathy's fear: Humanity gets disempowered by AI progress. "I care not just about all the Dyson spheres that we're going to build, I care about what happens to humans"

"I'm most afraid of something depicted in movies like WALL-E or Idiocracy, where humanity is on the side of this stuff"

Current AI tutors aren't ready: "The bar is so high and the capability is not there" based on experience with a Korean language tutor

First product: LLM101N course with nanochat as capstone project, building "a really, really good course, the obvious state-of-the-art destination you go to to learn AI"

What Makes a Perfect Tutor Irreplaceable

Karpathy's Korean tutor "instantly from a very short conversation, understood where I am as a student, what I know and don't know. She was able to probe exactly the kinds of questions"

"She really served me all the things that I needed at my current sliver of capability. I need to be always appropriately challenged. I can't be faced with something too hard or too trivial"

"I felt like I was the only constraint to learning. I was always given the perfect information. I'm the only constraint. It's not that I can't find knowledge or that it's not properly explained"

"When I was with her, I almost felt like there's no way I can build this" - the bar for AI tutoring is extremely high

Karpathy's value as AI consultant was often "telling them not to use AI" - similarly, for education "it's not yet the time, but the time will come"

Education as Technical Problem: Building Ramps to Knowledge

"Education is the very difficult technical process of building ramps to knowledge. Nanochat is a ramp to knowledge because it's very simple. It's the super simplified full-stack thing"

"I want lots of eurekas per second" - understanding per second is the key metric for educational materials

Micrograd example: 100 lines of Python showing backpropagation. "Everything else is just efficiency. The core intellectual piece of neural network training is micrograd. It's 100 lines"

"Education is the most intellectually interesting thing because you have a tangle of understanding and you're trying to lay it out in a way that creates a ramp where everything only depends on the thing before it"

Key technique: "Always prompting the student. I'm not going to present the solution before you guess. That would be wasteful. That's a little bit of a dick move towards you"

Post-AGI Education: Learning as Entertainment

"Pre-AGI education is useful. Post-AGI education is fun" - like going to the gym when we don't need physical strength for work

"People go to the gym today. We don't need their physical strength to manipulate heavy objects. They still go to the gym. Why? Because it's fun, it's healthy, and you look hot when you have a six-pack"

"You'll go to school like you go to the gym. Learning is hard. You bounce from material. It's a technical problem to solve. It's going to make learning anything trivial and desirable"

Vision: "Anyone will speak five languages because why not? Because it's so trivial. Anyone will know all the basic curriculum of undergrad"

Historical precedent: "Aristocrats, ancient Greece - whenever you had little pocket environments that were post-AGI in a certain sense, people spent time flourishing physically or cognitively"

"I feel the geniuses of today are barely scratching the surface of what a human mind can do" - perfect AI tutors will unlock human potential

Physics Training: The Best Brain Bootloader

"Early school education is not about accumulating knowledge or memory for tasks later in the industry. It's about booting up a brain. Physics uniquely boots up the brain the best" - Andrej

Physics teaches building models and abstractions: "Understanding that there's a first-order approximation that describes most of the system, but then there're second-order, third-order, fourth-order terms"

"When a physicist walks into the class and they say, 'Assume there's a spherical cow,' everyone laughs at that, but this is brilliant. A cow can be approximated as a sphere in a bunch of ways"

Recommended: Scale by physicist Geoffrey West, showing how to derive scaling laws of animals using physics approximations

This training shapes Karpathy's teaching: "I always try to find the first-order terms or the second-order terms of everything. What is the thing that matters? How can I simplify it?"

Why Intelligence Evolution Was Surprisingly Recent

"I am surprised that it evolved. I find it fascinating to think about all the worlds out there. The evolution of intelligence intuitively feels to me like it should be a fairly rare event" - Andrej

Bacteria existed for 2 billion years with nothing happening, suggesting the eukaryote transition was "probably pretty hard"

Multicellular animals have existed maybe 200 million years - "maybe 10% of Earth's lifespan. Maybe on that timescale it's not too tricky"

"I would maybe expect just a lot of animal-like life forms doing animal-like things. The fact that you can get something that creates culture and knowledge and accumulates it is surprising to me"

Intelligence may have evolved multiple times: "There's hominid intelligence, and then there's bird intelligence. Ravens are extremely clever, but their brain parts are quite distinct"

Gwern and Carl Shulman's perspective: Humans found an evolutionary niche rewarding marginal intelligence increases, plus had a scalable brain algorithm. Birds are smart but can't Scale: "If a bird had a bigger brain, it would just collapse out of the air"