The Leadership Letter

Real correspondence from the people running real companies — and what it reveals about leadership.

Compute Is the Constraint, Not Cleverness

If the bottleneck is hardware, the winner is whoever buys the most of it first.

We usually decide that problems are hard because smart people have worked on them unsuccessfully for a long time. It's easy to think that this is true about AI. However, the past five years of progress have shown that the earliest and simplest ideas about AI - neural networks - were right all along, and we needed modern hardware to get them working. Historically, AI breakthroughs have consistently happened with models that take between 7-10 days to train. This means that hardware defines the surface of potential AI breakthroughs. It's not so much that AI progress is a hardware game, any more than physics is a particle accelerator game. But if our computers are too slow, no amount of cleverness will result in AGI, just like if a particle accelerator is too small, we have no shot at figuring out how the universe works. Fast enough computers are a necessary ingredient, and all past failures may have been caused by computers being too slow for AGI. We estimate that Brain has around 100k GPUs, FAIR has around 15-20k, and DeepMind allocates 50 per researcher no questions asked. There is good reason to believe that deep learning hardware will speed up 10x each year for the next four to five years. The world is used to the comparatively leisurely pace of Moore's Law, and is not prepared for the drastic changes in capability this hardware acceleration will bring. Within the next three years, robotics should be completely solved, AI should solve a long-standing unproven theorem, and programming competitions should be won consistently by AIs.

This edition is for members.

The daily letter is free. The archive — every prior edition, fully searchable — is for members. Sign in to start your free week.

Court Exhibit
Musk v. Altman (OpenAI)
4:24-cv-04722 (CAND), Doc. 379-77, filed 2026-01-06
July 12, 2017
Public domain
View the primary source →