Jared Kaplan’s insights offer a compelling window into the rapidly evolving world of artificial intelligence, revealing how a blend of physics, biology, and practical experimentation is shaping the future of AI. His journey from theoretical physics to AI research underscores a broader narrative: the quest to understand complex systems—whether the universe or intelligent machines—requires both big questions and simple curiosity. Kaplan’s story begins with his early fascination, sparked by a science fiction writer mother and dreams of faster-than-light travel, before shifting gears to AI when physics felt too slow. This personal backdrop sets the stage for a deep dive into how AI learns, improves, and scales.
At the heart of modern AI’s progress lies a surprisingly straightforward mechanism: predicting the next word. Kaplan explains that AI models undergo two main training phases. First, pre-training involves digesting vast amounts of human-written text to guess what word comes next, a process that might sound simple but is incredibly powerful. “What pre-training does is teach models what words are likely to follow other words,” he says. Then, reinforcement learning fine-tunes these models by rewarding helpful, honest, and harmless responses while discouraging mistakes. This two-step approach enables AI like ChatGPT to engage in surprisingly coherent and useful conversations.
What truly sets AI apart, Kaplan reveals, is the predictable way it improves as it scales. He shares how performance follows clear “scaling laws,” meaning that as compute power, dataset size, and model complexity increase, AI’s capabilities improve along reliable trajectories. These trends have held steady over “many, many, many orders of magnitude” since 2019, making future progress not just hopeful but almost certain. Even in reinforcement learning scenarios—like training AI to play board games—skill improvements align neatly with increased computational resources. This predictability is rare in technology and reminiscent of the precise laws found in physics and astronomy.
Kaplan breaks down AI’s growing capabilities along two key dimensions: flexibility and time horizon. Flexibility measures how many different tasks AI can handle, from specialized systems like AlphaGo, which excels only at Go, to large language models that process multiple input types and may soon incorporate senses like smell. Time horizon refers to how long AI can sustain a task, with some companies observing that this duration “is doubling roughly every seven months.” This means AI is steadily taking on more complex, longer-term challenges that once required extensive human effort.
Yet, despite these advances, Kaplan highlights three critical ingredients still needed for truly human-level AI: relevant organizational knowledge to understand context, memory to track progress over time, and nuanced oversight to manage fuzzy tasks like humor or creativity. Newer models, such as Claude 4, are beginning to incorporate these features, signaling a move toward more sophisticated and context-aware AI systems. Kaplan encourages bold experimentation, noting that rapid improvements mean what seems imperfect today could be revolutionary tomorrow.
Delving deeper, Kaplan draws fascinating parallels between AI research and the sciences of physics and biology. Neural networks, with their billions or even trillions of parameters, resemble massive physical systems studied through approximations in physics. Yet, understanding how these networks function internally—known as interpretability—is more akin to neuroscience. Unlike the human brain, where neurons are hidden, AI models offer complete measurability, providing researchers with unprecedented data to unravel their inner workings. This advantage fuels ongoing efforts to decode AI’s “black box.”
However, challenges remain. AI training is currently “really inefficient,” demanding enormous computational power. Kaplan predicts future breakthroughs in efficiency, such as using lower-precision computing formats like FP4 or ternary representations, but warns that increased demand might offset these gains, echoing the Jevons paradox where better technology leads to more consumption. Another intriguing insight is how AI’s ability to “self-correct” mistakes extends its effective time horizon, enabling it to tackle longer and more complex tasks.
Looking ahead, Kaplan envisions a layered approach to AI training, where models help train other models by providing detailed feedback rather than simple pass/fail signals. While AI-generated training tasks will grow, humans remain essential, especially for the most challenging problems. This human-AI collaboration extends beyond training: as Kaplan notes, skeptics rightly point out that AI still makes many mistakes, so “a major role people can play... is as managers to sort of sanity check the work.” Rather than replacing humans, AI is poised to become a powerful partner, handling vast knowledge while humans provide judgment and oversight.
In sum, Jared Kaplan’s perspective illuminates AI’s steady, predictable march forward, grounded in scaling laws and enriched by interdisciplinary insights. His blend of physics rigor, biological analogy, and practical wisdom paints a picture of a field brimming with potential—one where asking simple questions and embracing experimentation remain key. As AI grows more flexible, capable, and integrated into real-world workflows, Kaplan’s roadmap offers both hope and a clear-eyed view of the challenges ahead, inviting continued exploration into the fascinating frontier of artificial intelligence.