-
korrents.com
AI scaling laws require exponentially more compute and resources to produce only linear gains in model intelligence.Their words
all of our scaling laws show that you need exponential compute and resources to make linear improvements in intelligence.
Related posts
Elliot Hershberg Newsletter
The the words it uses that this site has seen least often elsewhere. Posts are matched on those words alone — nothing here is a summary of this one.
Top people
Showing Profile →
Hiding
Hiding
19 September
15 September
7 September
-
Their words
As LLMs continue to scale up, the value of any particular source continues to diminish.
-
korrents.com
Scaling up clean energy is more effective when it is paired with retiring fossil fuel infrastructure.Their words
Building clean energy is much more likely, and much more effective, if we're also shutting down dirty energy.
6 September
From one piece An Alien Mind 2 beliefs · openai.com
-
korrents.com
No lab has solved alignment and monitoring well enough to keep scaling at maximum speed much longer.Their words
Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.
-
korrents.com
How far AI systems are scaled must be constrained by how confident we are in their safety.Their words
Scaling AI systems has to be constrained by our confidence in safety.
5 September
-
korrents.com
Test-time scaling has gained a third axis of latent space reasoning iterations in looped transformers.Their words
Seems like test time scaling has gained a 3rd axis: latent space reasoning iterations in looped transformers.
4 September
-
Their words
I think three of us have total conviction about the scaling law. That that I think we do. I do think the exact architecture choices and data mixtures is where the the devils are in the details.
2 September
-
korrents.com
Applying scale-first, profit-later to every business is a cookbook that turns small failures into big ones.Their words
Any investor or founder who blindly follows the pathway of scaling first and profiting later for every business is using a cookbook approach to business building, and runs the risk of making small failures into big ones.
27 August
-
Their words
In verifiable domains, model capability scaling should remain unbounded. Models will simply keep improving by "absorbing more and more of the computational universe", which is infinite by construction.
26 August
-
Their words
So we should also have the humility. As amazing as the LLMs are now, it could be that they eventually plateau. We haven't seen any evidence of it yet, and I think this is also why we're seeing this absolute gobsmacking levels of investment, because so far the scaling laws are true, and the more billions are poured in, the more intelligence comes out.
21 August
11 August
-
korrents.com
Scaling up spending on expert human data has not been a major driver of progress in AI research.Their words
So my sense is that scaling up the amount of effort spent on getting expert human data has not been hugely important for AI R&D in general.
7 August
-
On changing their mind
In the era of base LLM scaling (2022-2024), I believed the LLM line of research would reach a capability plateau (as later seen with base LLMs). In late 2024, after the o3 test-time compute demo, I changed my views: the new models were showing genuine fluid intelligence, and with this new line of work, the LLM line of research could achieve unbounded capability scaling. "There will be no wall."
3 August
-
Their words
So, the lesson here is to bet on a system that's maximally learned and minimally constrained and leverage structure intentionally to boost performance and scaling laws both in training and in evaluation.
2 August
-
korrents.com
The engineer-designer-PM pod ratio was an HR artifact of scaling, not something anyone showed a product needed.Their words
somewhere along the line this kind of HR ratio of like a pod popped into being. You know, every time you hire six engineers, you add a designer or you add a PM, you add an EM
27 July
-
Their words
Like this is actually like a new form of test time compute. Like when we talk about the scaling laws and kind of we talk about the model getting more intelligent over time, historically, it's been a function of the size of the neural net, the amount of training data, and the number of flops that you put in to the training. And then recently, we also added test time compute. So this is essentially a fancy way a researcher way of saying how many tokens does it generate. And now dynamic workflows are essentially a new way to orchestrate test time compute.
7 July
2 July
26 June
-
Their words
the preparedness frameworks and responsible scaling policies, they don't really account for the amount of test time compute.
24 June
2 June
-
Their words
it's this nonscalable way to scale your organization and it's through like passing your vampire blood. That's what inside Zinga they called it, Pinkis' vampire blood. What he did that I started to do is you pick someone from the organization who's promising. I usually pick the people who didn't fit in the smart misfits and they become your tech assistant which is not your chief of staff, not your executive assistant.
16 May
-
Their words
As reasoning models and agent workflows keep more tokens around (for longer), KV-cache size, memory traffic, and attention cost quickly become the main constraints, and LLM developers are adding a growing number of architecture tricks to reduce those costs.
13 May
6 May
22 April
-
Their words
generally, like you just want the the cost and the computing capacity to be roughly proportional to the load that you have. And at the low end, that means actually being able to scale down to something that is extremely cheap to run.
15 April
-
Their words
the ability for general purpose computing to continue to scale has largely run its course and the only the the not the only way but the the way to do that is through domain specific acceleration
4 April
1 April
-
Their words
No new feature necessary. Just make sure that we can do that. And then that allow the business the company to just pour a whole bunch of hardware behind that and it will scale.
23 March
-
Their words
And so the next scaling law is the agentic scaling law. It's kind of like multiplying AI. Multiplying AI, we could spin off agents as fast as you want to spin off agents. And so, you know, I… You know, I have four scaling laws.
13 March
-
Their words
well the model the compute efficiency gains you get from research are so large you actually want most of your compute to go to research not to development because you know all these researchers are generating new ideas trying them out testing them and continuing to march along this and push the prao optimal curve of scaling laws further and further and further
11 March
-
Their words
But, I can tell you that there are at least two more cycles left in this, and that means they will be at least 16 times smarter than they are today, and that is going to cause all of knowledge work to be subsumed by this stuff.
1 February
24 January
19 January
7 January
-
Recommendsaffiliate linkrcmnd.app
Scaling People: Tactics for Management and Company BuildingTheir words
Claire has been a senior executive at Google and Stripe. Here she offers up a guide to leadership that is both practical and inspiring. Suitable for leaders of all levels.
1 January
-
Their words
Systems which do require remote operations assistance to get full reliability cut into that economic advantage and have a higher burden on their ROI calculations to make a business case for their adoption and therefore their time horizon to scaling across geographies.
30 December 2025
7 December 2025
25 November 2025
From one piece Ilya Sutskever – We're moving from the age of scaling to the age of research 2 beliefs, in the piece's order there
-
Their words
And it's just this, this is an example of how language affects thought. Scaling is what just one word, but it's such a powerful word because it informs people what to do.
-
korrents.com
The age of scaling sucked the air out of the room and left the field with more companies than ideas, by quite a bit.Their words
And so because scaling sucked out all the air in the room, everyone started to do the same thing. We got to the point where uh we are in a world where there are more companies than ideas by quite a bit.
22 November 2025
-
Recommendsrcmnd.app
The Scaling Era: An Oral History of AI, 2019-2025Their words
Dwarkesh Patel, and others, The Scaling Era: An Oral History of AI, 2019-2025
11 November 2025
23 October 2025
-
Their words
durable workflows are scaling errors as values to your infrastructure. errors as values lead to predictable, maintainable, and scalable software.
17 September 2025
11 September 2025
4 September 2025
-
Their words
But I'm being obviously overly facetious, but if ultimately part of your scaling is how big is the animal versus how big is its brain, that's most of a T-Rex brain. It's a fraction of the size of a chimp brain, and chimps don't weigh seven tons.
23 July 2025
-
korrents.com
Scaling what we already have, with no further breakthroughs, will be enough to reach AGI.Their words
And so we don't know, I would say it's kind of 50/50 whether new things are needed or whether the scaling the existing stuff is going to be enough.
5 June 2025
-
Their words
So I do think scaling laws are working, but it's tough to get, at any given time, the models we all use the most, this maybe a few months behind the maximum capability we can deliver because that won't be the fastest, easiest to use, et cetera.
-
Their words
While the fact that merely scaling up AI models often results in qualitative leaps in performance may seem empirically mysterious, in retrospect it should be seen as a rational necessity.
2 February 2025
-
korrents.com
AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard.Their words
GPT-2, R1, and nearly every other AI model released since 2018 have all been part of a consistent story: AI capabilities that rival and ultimately exceed human intelligence are easier and cheaper to build than almost anyone can intuitively grasp, and this gets easier and cheaper every month.
31 December 2024
-
Recommendsrcmnd.app
Is AI progress slowing down?Their words
To understand more about inference scaling I recommend Is AI progress slowing down?
20 December 2024
-
Their words
o3's improvement over the GPT series proves that architecture is everything. You couldn't throw more compute at GPT-4 and get these results. Simply scaling up the things we were doing from 2019 to 2023 -- take the same architecture, train a bigger version on more data -- is not enough.
OpenAI o3 Breakthrough High Score on ARC-AGI-Pubarcprize.org
11 October 2024
-
Their words
Neurosymbolic AI — combining such machinery with neural networks – is likely a necessary condition for going forward.
1 September 2024
-
Their words
In effect there are two different ways to run a company: founder mode and manager mode. Till now most people even in Silicon Valley have implicitly assumed that scaling a startup meant switching to manager mode.
2 June 2024
4 April 2024
-
Their words
The copper backplane in the data center rack is effectively the new advanced packaging in the system-level Moore’s Law race.
30 October 2023
29 August 2023
-
korrents.com
AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard.Their words
This rapid scaleup will probably drive another qualitative leap forward in capability like what we saw over the last 18 months.
5 July 2023
Nothing matches.
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.