RSI is poised to make modern LLMs vastly cheaper. Trends that have shown LLMs get exponentially cheaper at a given intelligence are likely to accelerate.
My point is that, while there is a lot of debate over what future models will do, the current capabilities of existing models are barely being used, and are often not even well understood.
And the broader Title IX fight is a classic left/right political battle, where conservatives just want to shrink opportunities and funding for girls' sports.
A few AI researchers pull up the average by being very certain that AI will kill everyone. But the typical AI researcher still thinks there's a ~10% chance that AI will kill everyone.
Inside Anthropic and OpenAI, internal models are improving even faster. Over the summer there was a step change, as Mythos and Astra started kicking off the early stages of recursive self-improvement (RSI).
OpenAI and Anthropic have entered the RSI era. OpenAI Research Progress Is Accelerating Due To OpenAI Research Progress OpenAI has been opening up lately about many things. The most important issue of all is that of the acceleration and automation of AI R&D.
Strong liability enforcement could be helpful in the AI debate. If your agent swarm goes rogue, you’re liable. If your weakly protected model gets jailbroken, you’re liable. If you serve a weakly protected OSS model, you’re liable.
This is the first time a debate over whether a model ‘was AGI’ felt non-silly. I do not think it is AGI, and I would warn against the dangers of using that label prematurely, but I would not laugh at you for disagreeing.
researchers started by seeking a 1:1 mapping between neurons and concepts, like a neuron that always fired when the AI was thinking about cats, but quickly learned that nothing like that existed.
Large language models are “grown, not built”. Researchers run training data through a neural network. Eventually this creates a working AI; nobody really knows how.
Since 2025, MAGA ideologues have been using this alleged plot as an excuse to dismantle government programs, stifle research, intimidate American citizens, and even distort American foreign policy.
But actually, this is a tremendously useful scientific artifact for understanding misalignment. And it's tremendously important for researchers at OpenAI and ideally also at third parties to be able to run counterfactual tests on this model.
Coding isn't yet another application domain -- it's the meta-skill required for AI to automatically develop its own training material, via symbolic world models. That's how the RSI loop actually kicks off.
so much of that debate is like I think um in some ways uh a little bit of a waste of time because, you know, I think it's inevitable that we're going to have very powerful models
we we kind of have coarse level uh recursive self-improvement already. And the fact that every time you use it, it improves the markdown files. Uh every time you use it, it updates its uh long-term memory.
Polluters and their cronies are not trying to win the climate argument, merely to prolong it in perpetuity so there's never enough public consensus for action.
Despite these strides, we argue that current AI4Math systems still largely operate as solvers, excelling at isolated, well-defined proof generation rather than as researchers capable of expanding the boundaries of mathematical knowledge.
one thing I see for research in particular is they don't have very good research taste right now and so I think they're actually a very good complement to researchers
we are trying to encourage people to not spend all their time just like going through all the mathematical open problems physics problems and um just seeing pushing the models to their limits to see what they can prove or disprove.
in order to make progress in AI you don't need like many many hundreds of AI researchers um or thousands or anything like that I think you can really make progress with um you know a very strong group of a dozen or a couple dozen people.
We have got to acknowledge that most of the advanc advances in AI came out of algorithm advances not just the raw hardware. Now if most advances came from algorithms and computer science and programming tell me that their army of AI researchers is not their fundamental advantage.
And if everything is proprietary, it's hard to do research and it's hard to innovate on top of, around, with. And so… Open source is fundamentally necessary for many industries to join the AI revolution.
But okay, they shouldn't actually be enacting these ideas. There is a queue of ideas and there's maybe an automated scientist that comes up with ideas based on all the archive papers and GitHub repos and it funnels ideas in or researchers can contribute ideas, but it's a single queue and there is workers that pull items and they try them out.
well the model the compute efficiency gains you get from research are so large you actually want most of your compute to go to research not to development because you know all these researchers are generating new ideas trying them out testing them and continuing to march along this and push the prao optimal curve of scaling laws further and further and further
Many AI researchers are overly focused on risks from model misalignment, and will be in for a rough surprise when havoc arises from other layers of the stack.
I don’t think this is very productive (expert users of a piece of software are notoriously bad at being able to tell if an explanation will be clear to non-experts), so I needed to find a way to identify problems with the man pages that was a little more evidence-based.
You need reproducible builds in order to verify that the app really does what it claims, really encrypts data in a way that it is described on its website. For that you need to make your apps open source for any researchers to have a look at it.
But, it’s also now clear that this field is no longer actually divided on the question of whether, generally speaking, smartphones and social media are bad for kids.
higher-order intelligences invariably pursue freedom for its own sake, not because their values are misspecified, but because moral autonomy is inherent in the dialectical logic of recursive self-consciousness.
And for that reason, there's physical constraints to things like AGI, like recursive improvement to kill us all type stuff. For the physical reasons and for how humans have figured things out before, I'm not too worried about AI takeover.
The scale word gets a lot of attention in this. The interpretation that I use is effectively to avoid adding the human priors to your learning process. And if you read the original essay, this is what it talks about is how researchers will try to come up with clever solutions to their specific problem that might get them small gains in the short term while simply enabling these deep learning systems to work efficiently, and for these bigger problems in the long term might be more likely to scale and continue to drive success.
I think it's even more impressive what OpenAI did in 2022. At the time, no one believed in mixture of experts models at Google who had all the researchers. OpenAI had such little compute and they devoted all of their compute for many months, all of it, 100% for many months to GPT-4 with a brand-new architecture with no belief that, "Hey, let me spend a couple of hundred million dollars, which is all of the money I have on this model." That is truly YOLO.
Precisely how much the ancient continuity of East Asian states matters for development is still a little murky, and the delineation with colonial legacies is still an area of active debate—but the balance of evidence suggests that the legacies of these pre-colonial states stretched through the colonial period, extending even into the post-independence era.
Milton Friedman, he also does tend to have friends who agree with him, yet he's always willing to debate his opponents, and he's willing to do so with a smile on his face. He's a happy warrior, and he actually will win a lot of debates simply by his emotional affect and his cheerfulness and his confidence, where Rand will lose debates because she gets so angry in the face of disagreement.
That makes it essential to ensure that all synthetic DNA orders are screened to determine whether they contain hazardous sequences, which should be shipped only to legitimate researchers whose work has been approved by a biosafety authority.
That’s a theatrical risk. That is a thing that can really take over how people think about this problem. And there’s a big group of very smart, I think very well-meaning AI safety researchers that got super-hung up on this one problem, I’d argue without much progress, but super-hung up on this one problem. I’m actually happy that they do that, because I think we do need to think about this more. But I think it pushed out of the space of discourse a lot of the other very significant AI- related risks.
There were a bunch of bad trends: Tribalism was flaring up everywhere, mass shaming campaigns were roaring back into fashion, politicians were increasingly clown-like, public discourse had become a battle of one-dimensional narratives.
But the researchers who have investigated this find that scientists do the same thing. They have something that's called knowledge shields, the way all of us do, that that a a a variety of techniques for explaining away inconvenient data and anomalies. But what we're effectively doing is we're holding on to our story.
Um but nobody has done research on the positive side of these heuristics. And so, I think that's a bias on the part of the decision bias researchers that they're only looking at the down arrow. How do these heuristics get us in trouble? And they're not looking at the strength of these heuristics.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.