From one piece AGI Ruin: A List of Lethalities 10 beliefs, in the piece's order there
-
Their words
None of this is about anything being impossible in principle.
-
korrents.com
The field calling itself AI safety is not being remotely productive on the problems that are actually lethal.Their words
It does not appear to me that the field of ‘AI safety’ is currently being remotely productive on tackling its enormous lethal problems.
-
korrents.com
There is no written plan for surviving AGI, and a world that was going to survive would have had one decades ago.Their words
Surviving worlds, by this point, and in fact several decades earlier, have a plan for how to survive.
+ 7 more
-
Their words
Many alignment problems of superintelligence will not naturally appear at pre-dangerous, passively-safe levels of capability.
-
korrents.com
Fast capability gains are likely, and they can break many of the assumptions alignment depends on at the same moment.Their words
Fast capability gains seem likely, and may break lots of previous alignment-required invariants simultaneously.
-
Their words
A strategically aware intelligence can choose its visible outputs to have the consequence of deceiving you, including about such matters as whether the intelligence has acquired strategic awareness
-
Their words
We’ve got no idea what’s actually going on inside the giant inscrutable matrices and tensors of floating-point numbers.
-
Their words
The best and easiest-found-by-optimization algorithms for solving problems we want an AI to solve, readily generalize to problems we’d rather the AI not solve; you can’t build a system that only has the capability to drive red cars and not blue cars, because all red-car-driving algorithms generalize to the capability to drive blue cars.
-
Their words
unaligned operation at a dangerous level of intelligence kills everybody on Earth and then we don’t get to try again.
-
Their words
2 years after the leading actor has the capability to destroy the world, 5 other actors will have the capability to destroy the world.