A few AI researchers pull up the average by being very certain that AI will kill everyone. But the typical AI researcher still thinks there's a ~10% chance that AI will kill everyone.
researchers started by seeking a 1:1 mapping between neurons and concepts, like a neuron that always fired when the AI was thinking about cats, but quickly learned that nothing like that existed.
Large language models are “grown, not built”. Researchers run training data through a neural network. Eventually this creates a working AI; nobody really knows how.
Since 2025, MAGA ideologues have been using this alleged plot as an excuse to dismantle government programs, stifle research, intimidate American citizens, and even distort American foreign policy.
But actually, this is a tremendously useful scientific artifact for understanding misalignment. And it's tremendously important for researchers at OpenAI and ideally also at third parties to be able to run counterfactual tests on this model.
Despite these strides, we argue that current AI4Math systems still largely operate as solvers, excelling at isolated, well-defined proof generation rather than as researchers capable of expanding the boundaries of mathematical knowledge.
one thing I see for research in particular is they don't have very good research taste right now and so I think they're actually a very good complement to researchers
we are trying to encourage people to not spend all their time just like going through all the mathematical open problems physics problems and um just seeing pushing the models to their limits to see what they can prove or disprove.
in order to make progress in AI you don't need like many many hundreds of AI researchers um or thousands or anything like that I think you can really make progress with um you know a very strong group of a dozen or a couple dozen people.
We have got to acknowledge that most of the advanc advances in AI came out of algorithm advances not just the raw hardware. Now if most advances came from algorithms and computer science and programming tell me that their army of AI researchers is not their fundamental advantage.
And if everything is proprietary, it's hard to do research and it's hard to innovate on top of, around, with. And so… Open source is fundamentally necessary for many industries to join the AI revolution.
But okay, they shouldn't actually be enacting these ideas. There is a queue of ideas and there's maybe an automated scientist that comes up with ideas based on all the archive papers and GitHub repos and it funnels ideas in or researchers can contribute ideas, but it's a single queue and there is workers that pull items and they try them out.
well the model the compute efficiency gains you get from research are so large you actually want most of your compute to go to research not to development because you know all these researchers are generating new ideas trying them out testing them and continuing to march along this and push the prao optimal curve of scaling laws further and further and further
Many AI researchers are overly focused on risks from model misalignment, and will be in for a rough surprise when havoc arises from other layers of the stack.
You need reproducible builds in order to verify that the app really does what it claims, really encrypts data in a way that it is described on its website. For that you need to make your apps open source for any researchers to have a look at it.
But, it’s also now clear that this field is no longer actually divided on the question of whether, generally speaking, smartphones and social media are bad for kids.
The scale word gets a lot of attention in this. The interpretation that I use is effectively to avoid adding the human priors to your learning process. And if you read the original essay, this is what it talks about is how researchers will try to come up with clever solutions to their specific problem that might get them small gains in the short term while simply enabling these deep learning systems to work efficiently, and for these bigger problems in the long term might be more likely to scale and continue to drive success.
I think it's even more impressive what OpenAI did in 2022. At the time, no one believed in mixture of experts models at Google who had all the researchers. OpenAI had such little compute and they devoted all of their compute for many months, all of it, 100% for many months to GPT-4 with a brand-new architecture with no belief that, "Hey, let me spend a couple of hundred million dollars, which is all of the money I have on this model." That is truly YOLO.
That makes it essential to ensure that all synthetic DNA orders are screened to determine whether they contain hazardous sequences, which should be shipped only to legitimate researchers whose work has been approved by a biosafety authority.
That’s a theatrical risk. That is a thing that can really take over how people think about this problem. And there’s a big group of very smart, I think very well-meaning AI safety researchers that got super-hung up on this one problem, I’d argue without much progress, but super-hung up on this one problem. I’m actually happy that they do that, because I think we do need to think about this more. But I think it pushed out of the space of discourse a lot of the other very significant AI- related risks.
But the researchers who have investigated this find that scientists do the same thing. They have something that's called knowledge shields, the way all of us do, that that a a a variety of techniques for explaining away inconvenient data and anomalies. But what we're effectively doing is we're holding on to our story.
Um but nobody has done research on the positive side of these heuristics. And so, I think that's a bias on the part of the decision bias researchers that they're only looking at the down arrow. How do these heuristics get us in trouble? And they're not looking at the strength of these heuristics.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.