Key ideas from Andrej Karpathy

• "It's the decade of agents, not the year of agents" - Andrej, reacting to over-prediction in the industry about AI agent timelines
• Current LLMs lack continual learning, multimodality, and sufficient intelligence - "they're cognitively lacking and it's just not working" - Andrej
• Pre-training acts as "crappy evolution" - a practically possible way to build intelligence by imitating internet documents rather than running actual evolution
• In-context learning stores 320 kilobytes per token versus 0.07 bits per token in pre-training - a 35 million-fold difference in information density
• "Reinforcement learning is terrible" - Andrej explains it's noisy, upweighting entire trajectories based on final outcomes without understanding which steps were actually correct
• Self-driving progress is "a march of nines" where each additional nine of reliability requires constant work - demos are just the first nine
• Andrej predicts the cognitive core of intelligence could be as small as 1 billion parameters once knowledge is separated from reasoning capability
• Eureka Labs aims to build "Starfleet Academy" - an elite institution for technical education that makes learning trivial through perfect AI tutoring