-
Recommendsrcmnd.app
A Step Farther OutTheir words
well um one thing that shaped me as a little tiny person of like age nine or so was a book by Jerry Pornell called A Step Farther out. A whole lot of engineers um say that this was a major formative book for them. It's the technopile book um as written from the perspective of the 1970s the book that's all about asteroid mining and all the mineral wealth that would be available on Earth if we learn to mine the asteroids.
-
Recommendsrcmnd.app
Judgment under UncertaintyTheir words
Book two, judgment under uncertainty, an edited volume by uh uh Kaman Ferski and I think Slovik um had a huge influence on how I think how I ended up thinking about where humans are on the cognitive chain of existence as it were. It's like here's how the steps of human reasoning break down step by step. Here's how they go astray.
-
Recommendsrcmnd.app
Probability TheoryTheir words
Book three, um I'll name probability theory, the logic of science, which was my first introduction to there is a better way. Like here is the structure of quantified uncertainty. If you can try different structures, but they necessarily won't work as well. And we actually can say some things about like what better reasoning would look like. We just can't run it, which is probability theory, the logic of science.
Eliezer Yudkowsky
Co-founder of the Machine Intelligence Research Institute and the writer who started the modern argument that a sufficiently capable AI would by default kill everyone. Began arguing the case in the early 2000s, latterly with the conclusion that it is being lost.
Eliezer Yudkowsky did not write this page. ppll compiled it from their own words, each linked to the source. Is this you? · Report a mistake
What they believe
-
Any given AI capability becomes cheap and widely available within a couple of years of first being reached, so nobody can be kept away from it for long.
2 years after the leading actor has the capability to destroy the world, 5 other actors will have the capability to destroy the world.
intelligence.orgSaid 10 Jun 2022This on korrents
The bold line is korrents’ wording; the quote is theirs. These one: what they wrote before what they said aloud, one per source, newest first.
Feed
As its own page →Hiding
15 October 2025
10 June 2022
From one piece AGI Ruin: A List of Lethalities 10 beliefs · intelligence.org
-
Their words
None of this is about anything being impossible in principle.
-
korrents.com
The field calling itself AI safety is not being remotely productive on the problems that are actually lethal.Their words
It does not appear to me that the field of ‘AI safety’ is currently being remotely productive on tackling its enormous lethal problems.
-
korrents.com
There is no written plan for surviving AGI, and a world that was going to survive would have had one decades ago.Their words
Surviving worlds, by this point, and in fact several decades earlier, have a plan for how to survive.
+ 7 more
-
Their words
Many alignment problems of superintelligence will not naturally appear at pre-dangerous, passively-safe levels of capability.
-
korrents.com
Fast capability gains are likely, and they can break many of the assumptions alignment depends on at the same moment.Their words
Fast capability gains seem likely, and may break lots of previous alignment-required invariants simultaneously.
-
Their words
A strategically aware intelligence can choose its visible outputs to have the consequence of deceiving you, including about such matters as whether the intelligence has acquired strategic awareness
-
Their words
We’ve got no idea what’s actually going on inside the giant inscrutable matrices and tensors of floating-point numbers.
-
Their words
The best and easiest-found-by-optimization algorithms for solving problems we want an AI to solve, readily generalize to problems we’d rather the AI not solve; you can’t build a system that only has the capability to drive red cars and not blue cars, because all red-car-driving algorithms generalize to the capability to drive blue cars.
-
Their words
unaligned operation at a dangerous level of intelligence kills everybody on Earth and then we don’t get to try again.
-
Their words
2 years after the leading actor has the capability to destroy the world, 5 other actors will have the capability to destroy the world.
Nothing matches.
Beliefs
Korrents What they believe 10 beliefs — each backed by an exact quote.
Each is a — compiled by korrents.com, not by them: the one-line wordings are korrents', the quotes are theirs.
Recent
Any given AI capability becomes cheap and widely available within a couple of years of first being reached, so nobody can be kept away from it for long.
2 years after the leading actor has the capability to destroy the world, 5 other actors will have the capability to destroy the world.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
Alignment has to be right on the first try at a dangerous level of capability, because failing at that level leaves nobody to try again.
unaligned operation at a dangerous level of intelligence kills everybody on Earth and then we don’t get to try again.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
You cannot build an AI that has only the capabilities you want, because the algorithms that solve the problems you want generalize to the ones you do not.
The best and easiest-found-by-optimization algorithms for solving problems we want an AI to solve, readily generalize to problems we’d rather the AI not solve; you can’t build a system that only has the capability to drive red cars and not blue cars, because all red-car-driving algorithms generalize to the capability to drive blue cars.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
Show 7 more
Nobody knows what is going on inside a large model, and the interpretability pictures we can draw do not answer the only question that matters.
We’ve got no idea what’s actually going on inside the giant inscrutable matrices and tensors of floating-point numbers.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
No observation of a model's behaviour can establish that it is aligned, because a model capable enough to be dangerous is capable enough to produce whatever behaviour it is being watched for.
A strategically aware intelligence can choose its visible outputs to have the consequence of deceiving you, including about such matters as whether the intelligence has acquired strategic awareness
AGI Ruin: A List of Lethalities Said 10 Jun 2022
Fast capability gains are likely, and they can break many of the assumptions alignment depends on at the same moment.
Fast capability gains seem likely, and may break lots of previous alignment-required invariants simultaneously.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
Alignment failures that matter will not show up at safe capability levels, because the reason to hide misbehaviour only exists once hiding it pays.
Many alignment problems of superintelligence will not naturally appear at pre-dangerous, passively-safe levels of capability.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
There is no written plan for surviving AGI, and a world that was going to survive would have had one decades ago.
Surviving worlds, by this point, and in fact several decades earlier, have a plan for how to survive.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
The field calling itself AI safety is not being remotely productive on the problems that are actually lethal.
It does not appear to me that the field of ‘AI safety’ is currently being remotely productive on tackling its enormous lethal problems.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
Aligning an AI is not impossible in principle; the problem is that it has to be done without the textbook that would make it look easy.
None of this is about anything being impossible in principle.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
Beliefs others hold too
Any given AI capability becomes cheap and widely available within a couple of years of first being reached, so nobody can be kept away from it for long. 3 hold this
2 years after the leading actor has the capability to destroy the world, 5 other actors will have the capability to destroy the world.
AGI Ruin: A List of Lethalities Said 10 Jun 2022
No observation of a model's behaviour can establish that it is aligned, because a model capable enough to be dangerous is capable enough to produce whatever behaviour it is being watched for. 2 hold this
A strategically aware intelligence can choose its visible outputs to have the consequence of deceiving you, including about such matters as whether the intelligence has acquired strategic awareness
AGI Ruin: A List of Lethalities Said 10 Jun 2022
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
Recommends rcmnd
rcmnd What they recommend 3 things — each backed by an exact quote.
Compiled by rcmnd.app, not by them: the quotes are theirs, and each links to the post it was copied from.
Recent
Recommends Probability Theory
Book three, um I'll name probability theory, the logic of science, which was my first introduction to there is a better way. Like here is the structure of quantified uncertainty. If you can try different structures, but they necessarily won't work as well. And we actually can say some things about like what better reasoning would look like. We just can't run it, which is probability theory, the logic of science.
youtube.com 15 Oct 2025
Recommends Judgment under Uncertainty
Book two, judgment under uncertainty, an edited volume by uh uh Kaman Ferski and I think Slovik um had a huge influence on how I think how I ended up thinking about where humans are on the cognitive chain of existence as it were. It's like here's how the steps of human reasoning break down step by step. Here's how they go astray.
youtube.com 15 Oct 2025
Recommends A Step Farther Out
well um one thing that shaped me as a little tiny person of like age nine or so was a book by Jerry Pornell called A Step Farther out. A whole lot of engineers um say that this was a major formative book for them. It's the technopile book um as written from the perspective of the 1970s the book that's all about asteroid mining and all the mineral wealth that would be available on Earth if we learn to mine the asteroids.
youtube.com 15 Oct 2025
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.