What Andrej Karpathy Says About Education

2 episodes featuring Andrej Karpathy on education

Key Insights on Education

Current LLMs lack continual learning, multimodality, and sufficient intelligence - "they're cognitively lacking and it's just not working" - Andrej

In-context learning stores 320 kilobytes per token versus 0.07 bits per token in pre-training - a 35 million-fold difference in information density
"Reinforcement learning is terrible" - Andrej explains it's noisy, upweighting entire trajectories based on final outcomes without understanding which steps were actually correct
Eureka Labs aims to build "Starfleet Academy" - an elite institution for technical education that makes learning trivial through perfect AI tutoring
Andre Karpathy predicts this will be the 'decade of agents' not year, citing missing capabilities like continual learning and cognitive deficits that will take years to resolve
Reinforcement learning is fundamentally flawed because it's 'sucking supervision through a straw' - using sparse end rewards to judge entire complex trajectories
Future AI education will resemble having a perfect Korean tutor who instantly understands your knowledge gaps and serves appropriately challenging material