From one piece Dylan Patel — The single biggest bottleneck to scaling AI compute 15 beliefs · youtube.com
-
Their words
As we move from, you know, hey, [clears throat] these companies are selling tokens where they provide the entire uh reasoning chain and all that to uh selling automated, you know, white collar work, right? Automated software engineer, send them the request, they give you the result back and there's a bunch of thinking on the back end that they don't show you. The ability to distill out of American models into Chinese models will be harder.
-
korrents.com
You do not have to believe in AGI to believe in the timelines on which the United States wins the semiconductor race.Their words
But I don't know like I don't know what fast timelines means, right? Like I I like don't think you have to believe in AGI to have the timelines where the US wins.
-
Their words
I think they'll have working tools. I don't think that they'll be able to manufacture a bunch yet, right? You know, there's they're sort of having it work and then there's production hell, right?
+ 12 more
-
Their words
So when you look at inference at let's say 100 tokens a second for deepseek and kimk 2.5 hopper versus blackwell the performance difference is on the order of 20x
-
Their words
Um in general the semiconductor supply chain has not right it's lived through the booms and bust and uh we can talk a bit more about it but basically no one you know some players as of very recently have like woken up but in general no one really sees demand for 200 gawatts a year of AI chips or you know trillions of dollars of spend a year in the semiconductor supply chain.
-
Their words
You can you can take the margin like Nvidia takes the margin. memory players are taking the margin, but ASML has never risen the price more than they've increased the capability of the tool. Um, and so in a sense, they've always provided net benefit to their customer.
-
Their words
Um, and then you stack on 70 this year, 80 next year, growing to 100 by 2030. You're at like 700 EV tools by the end of the decade. Um, 700 EV tools, three and a half tools per gigawatt. um assuming it's all allocated to AI which it's not but three and a half tools per gigawatt gets you to 200 gigawatts worth of AI chips for the data centers to deploy
-
Their words
oh 50 gigawatts of economic you know sort of capex in in the data center and what gets built on top of that in terms of tokens is even larger right it might be hundred billion dollars worth of AI value into the supply chain is held up by this $1.2 two billion dollars worth of tooling that simply just cannot expand its supply chain quickly.
-
Their words
So to scale compute further right there's some different bottlenecks this year next year uh but ultimately by 2829 the bottleneck falls to the lowest rung on the supply chain which is ASML right ASML makes the world's most complicated machine i.e. an EUV tool.
-
Their words
Yeah, I think the biggest bottleneck is compute and for that the longest lead time supply chains are not power or data centers. They're actually the semiconductor supply chain themselves, right? It switches back from being power and data center uh as a major bottleneck to chips.
-
korrents.com
Anthropic saw the coming shortage of chips before Google did, and bought Google's own TPUs out from under it.Their words
Because Enthropic saw it before Google. And then Google had Nano Banano and Gemini 3 which caused their user metrics to skyrocket and leadership at Google was like oh and then they started making the statement of we have to double compute every is it 6 months or I don't remember the exact number that they said.
-
korrents.com
TSMC would rather give wafers to a slow, stable CPU business than to its fastest-growing AI customer.Their words
TSMC is much more excited to give allocation to Graviton than they are to tranium because they view CPU business as more stable long-term growth right and as a company that is conservative and doesn't want to ride cycles of growth too hard you actually want to allocate to the uh the market that is more stable and lower growth rate first before you allocate all the incremental capacity to the fast growth rate market.
-
Their words
Um I think at least this year we're going to see margins for the model vendors go up a lot, right? Because they're so capacity constrained, they have to demand destroy demand, right? there is there's no way they can continue anthropic can continue at the current pace without destroying demand.
-
Their words
A companies that have locked up, you know, and and don't have commitment issues, you know, have these 5-year contracts for compute, they've kind of locked in a humongous margin advantage because they've locked in compute for 5 years at a price of what it transacted at 5 years ago or three years ago or two years ago, whatever it is.
-
Their words
if improvement stopped you know here the value of an H100 is now predicated on the value that GPD 5.4 four can get out of it instead of the value that GP4 can get out of it and the margins and all that stuff that these labs are doing and they're in a competitive environment so their margins can't go to infinity. Um so you sort of have this like dynamic that is quite interesting in that an H100 is worth more today than it was 3 years ago.