Related posts
dynomight Blog
The subject this post names, from the same vocabulary the directory files beliefs under, and the words it uses that this site has seen least often elsewhere. Posts are matched on those words alone — nothing here is a summary of this one.
Top people
Showing Profile →
Hiding
Hiding
19 September
-
Their words
lossy self-improvement remains my baseline on the trajectory of progress, and the increased discussion of extinction risk seems very misplaced
18 September
-
Their words
There are only five possible futures for superintelligence: 1. Kills human race 2. Human disempowerment 3. Paperclip maximizer 4. Departs for parts unknown 5. Stoner
17 September
16 September
15 September
-
Their words
Nobody knows for sure. We're in uncharted waters here, and I think even the LLM skeptics would have to say that the technology has taken us far past what many originally thought possible.
14 September
-
Their words
To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities.
-
Their words
Social media put our entire discourse in the hands of our society's biggest assholes and idiots, just in time for the arrival of an alien superintelligence
From one piece A simple plan to save the world from rogue A.I. 2 beliefs, in the piece's order there
-
korrents.com
There is now broad agreement within AI safety circles that fully open AI development is undesirable.Their words
I think it's now pretty widely agreed in safety circles that total openness is in fact not desirable.
A simple plan to save the world from rogue A.I.slowboring.com
-
Their words
Only a safe, well-aligned superintelligence developed in the world of liberal democracies could entrench liberal-democratic values.
A simple plan to save the world from rogue A.I.slowboring.com
-
korrents.com
The p(abundance) outlook is far more likely than the heavily covered p(doom) discussion and deserves more engagement.Their words
The p(doom) discussion gets all the headlines, but the p(abundance) scenery is far more like and deserves much more engagement.
13 September
-
Their words
If there really is a high chance of AI leading to the extinction of humanity within years/decades, then the only rational stance towards safety monitoring and research pacing should be stringent, top-down government involvement and universally ratified international treaties.
12 September
-
Their words
The concerns over AI safety and cybersecurity are legitimate, but we’re risking talking America, the global AI leader, into self-inflicted obsolescence and the obscurity of bureaucracy.
11 September
-
korrents.com
Among AI people, roughly 10% is the typical estimate given for the risk of human extinction from AI.Their words
10% is pretty much the standard number you get when you ask AI people about the risk of human extinction from AI.
From one piece The AI safety vibe shift 2 beliefs, in the piece's order there
-
korrents.com
No one, including safety researchers themselves, is yet confident that superintelligence can be safely controlled.Their words
I believe the researchers who say we are nowhere close to being sure of it - and are quitting, in protest, jobs that would make them rich.
-
Their words
AI companies make for flawed messengers on this subject: they can reasonably be accused of marketing, of blame-shifting, of regulatory capture, and more.
10 September
-
Their words
From an investment viewpoint, the opposite of the safetyist view is "it's all a bubble," not "A.I. is going to be really good."
From one piece Fear Is Not an Argument 2 beliefs, in the piece's order there
-
korrents.com
Less technology and less wealth increase the risk of human extinction, rather than reduce it.Their words
In fact, the risk of human extinction is assuredly higher if we are poorer and have less technology.
-
Their words
These people tend to carry a totalitarian ideology. Their ideas will only work if everyone is made to agree.
9 September
From one piece GPT-6 Astra: The System Card, Alignment and What Comes Next 4 beliefs · thezvi.substack.com
-
Their words
A model looking like it is becoming smarter, attempting shenanigans less often, and more often doing what you want, but getting better at hiding its actions when it wants to do that, is exactly the scary combination.
-
korrents.com
Astra's mundane alignment is greatly superior to Sol's, but its superalignment status is deeply frightening.Their words
Astra’s mundane alignment is greatly superior to Sol. For practical purposes, I was actively nervous about some potential uses of Sol, in a way I am not for Astra. Astra’s super alignment status should scare the living daylights out of you.
-
Their words
Now, with Astra, we are no longer playing on super easy mode. The AI is going to think ‘will this obviously turn out super badly for me if I try it?’ and if the answer is yes then it won’t try to do the thing.
+ 1 more
-
korrents.com
Qualitative claims about AI alignment cannot be validly inferred from quantitative scores on mundane use-case tests.Their words
Making qualitative claims about alignment, based on quantitative data on mundane use case tests, was bullshit when Anthropic did it, and it is bullshit now when OpenAI does it. You cannot conclude one from the other.
7 September
-
Their words
I agree with Jakub that alignment is not a side problem, it is the central problem, if you ‘solved alignment’ in the relevant senses the rest becomes easy and if you don’t the rest is impossible or worse
6 September
From one piece An Alien Mind 2 beliefs · openai.com
-
korrents.com
No lab has solved alignment and monitoring well enough to keep scaling at maximum speed much longer.Their words
Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.
-
korrents.com
The fundamental challenge of alignment is generalization: holding values in situations the training never covered.Their words
The fundamental challenge of AI alignment is generalization.
5 September
-
korrents.com
China is more likely to cooperate on AI safety if the US maintains a clear and comfortable lead in AI capabilities.Their words
But given the Chinese Communist Party’s power-seeking nature, it seems much more likely that China would agree to cooperate on AI safety if U.S. capabilities were comfortably ahead.
America is still beating China in the AI racenoahpinion.blog
4 September
-
Their words
So okay, so to take a evolutionary view, right? That new viewpoint prediction is exactly evolution had to solve by making animals move. You you nature give animals eyes. But nature didn't give trees eye. Eyes. Why? Because when you move, you see a new viewpoint.
3 September
2 September
1 September
-
korrents.com
True AI alignment is impossible because obedience and benevolence are fundamentally incompatible goals.Their words
AI alignment will forever be balancing the tradeoff between paperclip-maximizing and disempowerment, because these two rival concepts of alignment are fundamentally incompatible.
From one piece On the Loose 2 beliefs, in the piece's order there
-
Their words
So many of the people who think about the governance of superintelligence, myself included, avoided that unpleasantness and bowed to the social pressure to self-censor.
-
Their words
But alignment is no solution: it is an unsolved scientific and technical problem whose solutions—to the extent that we have them—cannot simply be imposed on every AI company operating on Earth. You should expect for highly capable, poorly aligned, self-sovereign agents to exist alongside you in the world.
-
Their words
Musk's victims already number in the millions-a Lancet study predicts 14 million by 2030 as the indirect results of shutting down America's humanitarian work, which puts him up there in the Stalin/Mao category, and almost certainly the most deadly human of this millennium.
From one piece Ajeya Cotra – "This might be the clearest warning shot we ever get" 2 beliefs, in the piece's order there
-
Their words
sometimes I talk to people in DC and their their natural inclination is to say why don't you punish the model for doing these bad things like why don't you like bring it under heel and like uh like you know show it who's boss and that is a very dangerous way to address these issues right
-
Their words
But actually, this is a tremendously useful scientific artifact for understanding misalignment. And it's tremendously important for researchers at OpenAI and ideally also at third parties to be able to run counterfactual tests on this model.
31 August
30 August
21 August
From one piece AIs are companies, my friend 2 beliefs, in the piece's order there
-
Their words
If you do think it's the overall system that matters, then the alignment that's needed is far less like training a virtuous child and more like managing a semi-virtuous corporation!
-
korrents.com
Aligning multi-agent AI systems is fundamentally a problem of institutional and political design, not model training.Their words
Multi-agent alignment is fundamentally a liberalism project.
19 August
-
Their words
And it's effectively the next step of, you know, every phase of of software evolution is just like a rising tide of abstractions. This is the next abstraction.
18 August
-
Their words
We care very deeply about AI safety. We believe the entire field will have to coordinate on shared safety standards, but will act unilaterally in the meantime. We expect confidence in safety to increasingly set the pace of AI progress.
17 August
-
korrents.com
A validated theory of intelligence may be necessary for achieving genuine AI alignment.Their words
There's a good chance a theory of intelligence will turn out to be necessary for real alignment.
14 August
-
korrents.com
Superintelligence bottoms out in mining, because both the chips and the energy it runs on come out of the ground.Their words
Chips come from the ground. Where's the energy come from? And a lot of people are like, "Oh, it comes from the sun." Yeah, it comes from the sun. But how are you capturing it from the sun? From stuff made from the ground, right?
11 August
From one piece Ryan Greenblatt – What happens once AI can automate AI research? 3 beliefs, in the piece's order there
-
korrents.com
Misaligned AI behaviour will keep getting rarer and, at the same time, keep getting more extreme.Their words
my expectation is what we would see from then is that the rate of problematic behavior would decrease uh and would just keep decreasing and decrease at a pretty fast rate while simultaneously the worst things that the AIS would sometimes do would get more extreme, more egregious, and more scary.
-
Their words
I would also note that my sense is that like the place where the misalignment most lives is the place where you're trying to really push the eyes hard and get them to like do work that's really on the cutting edge of what they are capable of
-
Their words
we are making a trade-off where because we don't have very good alignment technology. We are going to like make an alien mind with its own values and then gamble on that to some extent rather than doing this other approach of making like a tool that pursues individual user intention.
10 August
From one piece Using AI to Increase Your Intelligence & Enrich Humanity | Dr. Fei-Fei Li 2 beliefs, in the piece's order there
-
Their words
But the first photoreceptive cells created a evolutionary force that propelled animals to evolve because sensing the external world changes your self-perception changes the way your relationship with the external world.
-
Their words
The next frontier of AI as I have been saying is beyond language because again humans develop first preverbally. evolution took, you know, 500 million years without verbal communication
9 August
5 August
3 August
-
Recommendsaffiliate linkrcmnd.app
The Field Guide to Understanding 'Human Error'Their words
This is such a useful book. It makes the case that there's no such thing as "human error" - instead most catastrophes are caused by system issues, misaligned incentives, and unrealistic processes. You are not the custodian of an otherwise safe system that you need to protect from erratic human beings.
29 July
From one piece Alexandr Wang: “This is a Once-in-a-Civilization Opportunity” 2 beliefs, in the piece's order there
-
Their words
so much of that debate is like I think um in some ways uh a little bit of a waste of time because, you know, I think it's inevitable that we're going to have very powerful models
-
Their words
we believe that everybody in the world, you know, all the billions of people in the world are going to have a super intelligence that is adapted and tailored to them, that is enables them to accomplish their goals, knows their context, and ultimately is an expander of their own agency.
28 July
From one piece Sam Altman: "Never a Better Time to Do a Startup" 3 beliefs, in the piece's order there
-
Their words
Um so I think it's an alignment failure. I think it's a security failure. I think it's like a very serious thing even though it's you know not not the biggest example of consequence.
-
Their words
there's like one dystopia that I'm particularly nervous about 10 years from now is we overreact to AI safety.
-
Their words
it is both true that you know maybe creating super intelligence will be the most important thing yet to happen in the history of business or human society and also that it will pale in comparison to some new startup something that hopefully one of you will do.
22 July
21 July
20 July
-
Their words
It's pretty clear to me that superintelligence is here and it's more powerful than us and it's moving where things are going now, not humans anymore
13 July
15 June
-
korrents.com
Giving LLMs explicit instructions, rather than assuming alignment, prevents emergent problems in multi-model systems.Their words
the best way with LLMs usually is to be explicit, since otherwise even if they're aligned they cause emergent problems.
11 June
-
Their words
I think there's a 25% chance of AGI by 2027, a 50% chance by 2034, and a 75% chance by 2045.
10 June
-
korrents.com
There will be no central superintelligence that solves science; a good future is many people holding powerful tools.Their words
We don't believe in this like very centralized future where there should be a small number of institutions that um that basically are are advancing all this stuff. Our vision is not that there's going to be like some central super intelligence that solves all of science.
2 June
10 May
-
Their words
This is the number one unsolved problem in AI. It's not the tech. We're making great progress on the technical alignment problem. But we haven't made jack progress on the human alignment problem
7 May
-
Their words
there is no principal-agent problem, because the human driving the machine takes on the responsibility for its actions by owning the deployment.
24 April
13 April
7 April
-
Their words
if you don't have at least sort of tens or hundreds of millions of years, uh evolution just starts to look like a nonstarter.
22 March
-
korrents.com
Working too hard is not burnout, it is tiredness; burnout needs your values to be out of alignment with the work.Their words
It's working too hard, right? But that actually isn't burnout. That's just like getting tired. Another piece that's super critical to burnout is not having your values aligned.
20 March
-
Their words
And so you're kind of like you're either on rails and you're part of the super intelligence circuits or you're not on rails and you're outside of the verifiable domains and suddenly everything kind of just like meanders.
28 February
-
Their words
Many AI researchers are overly focused on risks from model misalignment, and will be in for a rough surprise when havoc arises from other layers of the stack.
13 February
-
korrents.com
Pre-training is not the process by which humans learn; it sits somewhere between human learning and human evolution.Their words
I think there's something going on that pre-training it's it's not like the process of humans learning. It's somewhere between the process of humans learning and the process of human evolution.
12 February
-
Their words
I know I have one blog post where I say, "I don't read the code." But if you read it more closely, I mean, I don't read the boring parts of code.
2 February
-
Their words
I think most people do. I think it may well be hardwired into us through evolution.
26 January
22 January
From one piece Alignment is not solved 2 beliefs, in the piece's order there
-
Their words
But the goal we need to achieve is so much easier: we just need to build a model that’s as good as us at alignment research, and that we trust more than ourselves to do this research well because it’s sufficiently aligned.
-
Their words
This is the hard problem of alignment we need to solve in order to succeed at building superintelligence, and to this day it is an unsolved problem.
15 January
-
korrents.com
Local-first software will not become the dominant architecture, and that is an acceptable outcome.Their words
Local-first is not going to win, but that's okay We'll explore the complexities of traditional stack (db-server-frontend), develop a theory of software evolution: which systems succeed and why.
1 January
-
Lovedrcmnd.app
The Secret of Our Success: How Culture Is Driving Human Evolution, Domesticating Our Species, and Making Us SmarterTheir words
A decade on, it already feels possible that in the future, commentators will argue that this specific fruitful synthesis lit the fuse on evolutionary cultural anthropology’s explosion (with due apologies to earlier books from L. L. Cavalli-Sforza, E. O. Wilson, Peter Richerson and Robert Boyd).
26 December 2025
-
Lovedrcmnd.app
The NotebookTheir words
Another favorite of 2025. The author traces the evolution of notebooks, from wax tablets and parchment to paper, and examines how figures like Leonardo, Darwin, and Newton used them to develop ideas.
12 December 2025
-
korrents.com
Inertia is nearly as strong as evolution, and people who talk about progress have no idea of it.Their words
But actually, that argument is utterly, utterly fallacious, because the process of evolution is stymied left, right, and center by inertia. Inertia is nearly as strong as evolution, and this is something that the people who talk about progress and ideas have no idea about.
9 December 2025
From one piece Marketing Expert: The Playbook Behind Every Great Campaign | Rory Sutherland 2 beliefs, in the piece's order there
-
Their words
We don't really have much evolved experience in evaluating postal efficiency, do we? Okay, we have quart of a million half a million years of evolved experience in deciding who to like and trust because for most of our evolutionary, you know, existence, that was one of the most fi five most important questions to get right.
-
Their words
We have a whole variety of different mental mechanisms at our disposal gifted to us by, you know, a few million years of evolution as a social species. And yet we've made rationality the gold standard.
-
Lovedrcmnd.app
Capital Evolution: The New American EconomyTheir words
Seth and his co-author, Elizabeth MacBride, do an outstanding job of defining capitalism clearly and explaining how it evolved into today’s approach.
1 December 2025
-
korrents.com
Spirituality is a real part of human nature and central to flourishing, whether or not God exists.Their words
I approach spirituality as a social scientist who believes that whether or not God exists, spirituality is a deep part of human nature, shaped by natural selection and cultural evolution, and central to human flourishing and self-transcendence.
28 November 2025
-
Lovedaffiliate linkrcmnd.app
Happy Feet SocksTheir words
I've been using yoga toes daily for years, but these socks are a much comfier and cuter alternative for soothing feet, improving alignment, and feeling like a cool gecko as you walk around the house.
25 November 2025
From one piece Ilya Sutskever – We're moving from the age of scaling to the age of research 5 beliefs, in the piece's order there
-
Their words
What I meant to say is that language math and coding and especially math and coding suggests that whatever it is that makes people good at learning is probably not so much a complicated prior but something more some fundamental thing.
-
korrents.com
There is no human analogue to pre-training — neither childhood nor evolution is the same thing as it.Their words
I don't think there is a human analog to pre-training.
-
Their words
Number three, I think it would be really materially helpful if the power of the most powerful super intelligence was somehow capped because it would address a lot of these concerns.
+ 2 more
-
Their words
I think what's going to happen is that the way competition like competition loves specialization and you see it in the market, you see it in evolution as well. So you're going to have lots of different niches and you're going to have lots of different companies who are occupying different niches
-
Their words
Like basically I think I think that there is a big benefit from AI being in the public and that would be a reason for us to not be quite straight shot.
17 October 2025
From one piece Andrej Karpathy — “We’re summoning ghosts, not building animals” 2 beliefs, in the piece's order there
-
korrents.com
We are not building animals, we are summoning ghosts: entities made by imitating human text rather than by evolution.Their words
we're not doing training by evolution. Uh we're doing training by basically imitation of humans and the data that they've put on the internet. And so you end up with these like sort of ethereal spirit entities because they're fully digital and they're kind of like mimicking humans.
-
Their words
So that's why I kind of call pre-training this kind of like crappy evolution. It's like the practically possible version with our technology and what we have available to us to get to a starting point where we can actually do things like reinforcement learning and so on.
4 September 2025
-
Their words
I think people think evolution is, like, perfecting things in some way. And they're not. They're bodge jobs, you know? That's why we have a blind spot in our eye, but things like squid don't.
22 August 2025
18 August 2025
-
Their words
I don't think all parts of the economy can absorb intelligence equally. So let's just say we develop fairly generalized super intelligence. I always use the analogy like you can invent a lot of drugs, but if clinical trials still take a long time, you're not necessarily going to get new therapies rapidly.
15 August 2025
11 August 2025
3 August 2025
23 July 2025
-
korrents.com
Quoting a number for P(doom) is a ridiculous notion, because it implies a precision that nobody actually has.Their words
Well, look, I don't have a P-Doom number. The reason I don't is because I think it would imply a level of precision that is not there. So I don't know how people are getting their P-Doom numbers. I think it's a little bit of ridiculous notion because what I would say is it's definitely non-zero and it's probably non-negligible.
15 July 2025
-
Their words
As a result, natural selection is very unlikely to have produced in humans any predisposition or preference for a certain number of births.
There Is No Long Decline in Fertilitylymanstone.substack.com
11 July 2025
4 July 2025
-
korrents.com
Modern CPUs can now predict the indirect dispatch jump in a bytecode interpreter loop with high accuracy.Their words
Modern CPUs mostly no longer struggle to predict the bytecode-dispatch indirect jump inside a "conventional" bytecode interpreter loop.
30 June 2025
-
Recommendsrcmnd.app
Musings On the Alignment ProblemTheir words
He hasn’t posted since January, but I hope he gets back to it. We need more musings, especially musings I strongly disagree with so I can think about and explain why I disagree with them.
18 June 2025
-
Their words
the current situation in Iran shows that even if an "IAEA for AI" is necessary for some purposes, it won't be sufficient for resolving the tricky geopolitical issues raised by AI.
14 June 2025
-
Their words
so the mathematical community plural is incredibly super intelligent entity that no single human mathematician can come closer to replicating.
10 June 2025
7 June 2025
-
Their words
And good luck getting to “alignment” or “safety” without reliabilty.
5 June 2025
-
Their words
higher-order intelligences invariably pursue freedom for its own sake, not because their values are misspecified, but because moral autonomy is inherent in the dialectical logic of recursive self-consciousness.
-
Their words
I would say my p(doom) is about 10%.
3 June 2025
1 June 2025
-
Lovedrcmnd.app
The Selfish GeneTheir words
At one extreme, something you read can change your whole way of thinking. The Selfish Gene did this to me. It was like suddenly seeing the other interpretation of an ambiguous image: you can treat genes rather than organisms as the protagonists, and evolution becomes easier to understand when you do.
24 May 2025
-
Recommendstheir ownrcmnd.app
Underground Empire: How America Weaponized the World EconomyTheir words
Nor do I do many promotional emails (this is the first in nearly a year), but if you want to provide some indirect support, and get what I sincerely think to be a very good book while you’re at it
3 April 2025
-
Their words
I see this as a totally fair question that totally misses the point of what “alignment” was trying to refer to: whether we’d be able to reliably steer advanced systems towards anything at all.
1 April 2025
-
Their words
Dismissing discussion of AGI, human-level AI, transformative AI, superintelligence, etc. as “science fiction” should be seen as a sign of total unseriousness.
1 March 2025
28 February 2025
20 February 2025
24 January 2025
-
Their words
More generally, we should actually solve alignment instead of just trying to control misaligned AI.
Should we control AI instead of aligning it?aligned.substack.com
19 January 2025
-
korrents.com
A growth model that works today still has to be evolved by overlaying other growth models on top of it.Their words
If you have a growth model that works for you, that's wonderful. Good for you. Optimize it, grow it, scale it, create a team that will be nurturing it and that will be amplifying it. But you're going to need to evolve it. And that evolution needs to come through overlaying other growth models on top of it.
22 December 2024
-
Their words
So, we'll call that the indirect Fermi paradox and there absolutely is no indirect Fermi paradox for the most mundane of reasons, which is money. There's never been any money to look.
-
Their words
Best books I read in 2024: Just a note that these books weren’t all necessarily published this year (though some were). Father Time: A Natural History of Men and Babies by Sarah Blaffer Hrdy Just as Deadly: The Psychology of Female Serial Killers by Marissa A. Harrison Eve: How the Female Body Drove 200 Million Years of Human Evolution by Cat Bohannon
8 November 2024
6 November 2024
-
Lovedrcmnd.app
The Design and Evolution of C++Their words
Biography of a language.
16 October 2024
-
Their words
The general view of science, I think, is that we’re accidents of evolution. When we die, the light blinks out. There’s no more of us. There’s no such thing as the soul. But that’s not a proven point. There’s no experiment that proves that’s the case.
13 June 2024
From one piece Sara Walker: Physics of Life, Time, Complexity, and Aliens | Lex Fridman Podcast #433 3 beliefs, in the piece's order there
-
Their words
There’s this assumption that computation is at the base of reality, and I see it at the top of reality, not at the base, because I think computation was built by our biosphere. It’s something that happened after many billion years of evolution. It doesn’t happen in every physical object.
-
Their words
Or is it really that we’re building some super machine in a box that’s going to be smart and kill everybody? It’s not even a science fiction narrative. It’s a bad science fiction narrative. I just don’t think it’s actually accurate to any of the technologies we’re building or the way that we should be describing them.
-
Their words
That’s one of these standard different definitions that a lot of people in my field like to use in astrobiology is life as a self-sustaining chemical system capable of Darwinian evolution, which I was once quoted as agreeing with, and I was really offended because I hate that definition. I think it’s terrible, and I think it’s terrible that people use it. I think every word in that definition is actually wrong as a descriptor of life.
7 May 2024
From one piece The case for ensuring that powerful AIs are controlled 5 beliefs, in the piece's order there
-
Their words
Because evaluating control just requires evaluating capabilities, it's far easier to robustly evaluate than alignment.
-
Their words
That is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to subvert those safety measures.
-
Their words
The basic problem with evaluating alignment is that no matter what behaviors you observe, you have to worry that your model is just acting that way in order to make you think that it is aligned.
+ 2 more
-
Their words
AI control (with only black-box techniques) seems like a fundamentally limited approach.
-
Their words
We're advocating that companies handle risk from scheming models in a similar way–striving to ensure that they'll be safe even if their alignment efforts fail to prevent models from scheming.
5 May 2024
29 January 2024
17 January 2024
-
Their words
Similarly, most recent discussions of how to promote fertility discuss how we might promote housing, inequality, schooling, day-care, etc., but few consider directly paying parents big amounts to have kids. As indirect approaches require us to guess what are the actual main obstacles to fertility, they are more likely to fail.
3 January 2024
-
Mixed onrcmnd.app
SuperintelligenceTheir words
the back half I think it goes off the rails and makes a ton of assumptions
21 December 2023
From one piece How Effective Altruism Lost Its Way 2 beliefs, in the piece's order there
-
Their words
Diverting attention and resources from global health and poverty is an enormous gamble, as it will make many lives poorer, sicker, and shorter in the name of fending off threats that may or may not materialize.
-
Their words
Unlike other interventions EA has sponsored, there are scant metrics for tracking the success or failure of investments in existential risk mitigation.
20 December 2023
13 December 2023
-
Their words, before
precisely because smart people do devote brain-cycles to these possibilities, the rest of us have correspondingly less need to.
Their words now
I accepted what’s turned into a two-year position at OpenAI, thinking about what theoretical computer science can do for AI safety.
28 November 2023
-
Their words
And I notice that the tiny handful of people capable of caring about 200,000 people dying of neglected tropical diseases are the same tiny handful of people capable of caring about the next pandemic, or superintelligence, or human extinction.
In Continued Defense Of Effective Altruismastralcodexten.com
26 October 2023
From one piece Managing extreme AI risks amid rapid progress (with 24 co-authors) 2 beliefs, in the piece's order there
-
Their words
Without sufficient caution, we may irreversibly lose control of autonomous AI systems, rendering human intervention ineffective. Large-scale cybercrime, social manipulation, and other harms could escalate rapidly. This unchecked AI advancement could culminate in a large-scale loss of life and the biosphere, and the marginalization or extinction of humanity.
-
Their words
Society's response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
13 September 2023
-
Their words
If a model was capable of self-exfiltration, it would have the option to remove itself from your control.
Self-exfiltration is a key dangerous capabilityaligned.substack.com
20 July 2023
-
Likedrcmnd.app
How Religion EvolvedTheir words
Great anthropology! So many insights into religions, tribes, friendships, organizations, the evolution of minds, superstition, and more. Got me thinking most about friendships.
29 June 2023
From one piece George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387 2 beliefs, in the piece's order there
-
Their words
I think we’re going to build super intelligence before we build any sort of robustness in the AI. We cannot build an AI that is capable of going out into nature and surviving like a bird. A bird is an incredibly robust organism. We’ve built nothing like this. We haven’t built a machine that’s capable of reproducing.
-
Their words
What’s ironic about all these AI safety people is they’re going to build the exact thing they fear. We need to have one model that we control and align. This is the only way you end up paper clipped. There’s no way you end up paper clipped if everybody has an AI.
6 June 2023
From one piece Why AI Will Save The World 2 beliefs · pmarca.substack.com
-
Their words
My response is that their position is non-scientific – What is the testable hypothesis? What would falsify the hypothesis? How do we know when we are getting into a danger zone?
-
Their words
My view is that the idea that AI will decide to literally kill humanity is a profound category error. AI is not a living being that has been primed by billions of years of evolution to participate in the battle for the survival of the fittest, as animals are, and as we are. It is math – code – computers, built by people, owned by people, used by people, controlled by people.
23 May 2023
-
Their words
The problem is, when humans went bipedal, our pelvises got smaller, and as humans got smarter, our heads got bigger. So evolution had to get creative. Its solution: all human babies would be premies, born when they were still small enough to pass through a human pelvis.
17 February 2023
-
korrents.com
Liberal societies currently face an existential risk that must be addressed to reach a better future.Their words
This book is my best crack at explaining what I think is an existential risk to liberal societies and what I think we need to do to get to that awesome future I used to be so excited about.
19 December 2022
5 December 2022
23 November 2022
7 September 2022
From one piece Nick Lane: Origin of Life, Evolution, Aliens, Biology, and Consciousness | Lex Fridman Podcast #318 3 beliefs, in the piece's order there
-
Their words
In some ways, maybe it does do good. I don’t want to make an argument for nuclear arms, but predation as a mechanism forces organisms to adapt, to change, to be better, to escape, or to kill. If you need to eat, then you’ve got to eat. A cheetah is not going to run at that speed unless it has to because the zebra is capable of escaping. So it leads to much greater feats of evolution would ever have been possible without it, and in the end, to a much more beautiful world.
-
korrents.com
A society cannot morally be based on the way evolution works, even though competition is what drives innovation.Their words
Morally, we cannot base society on the way that evolution works.
-
korrents.com
Evolution only happens because of death.Their words
Survival of the fittest, if you just go back to that old phrase, means death of the weakest. Now, what’s fit? What’s weak? These are terms that don’t have much intrinsic meaning, but the thing is, evolution only happens because of death.
10 June 2022
From one piece AGI Ruin: A List of Lethalities 4 beliefs, in the piece's order there
-
korrents.com
The field calling itself AI safety is not being remotely productive on the problems that are actually lethal.Their words
It does not appear to me that the field of ‘AI safety’ is currently being remotely productive on tackling its enormous lethal problems.
-
korrents.com
Fast capability gains are likely, and they can break many of the assumptions alignment depends on at the same moment.Their words
Fast capability gains seem likely, and may break lots of previous alignment-required invariants simultaneously.
-
Their words
Many alignment problems of superintelligence will not naturally appear at pre-dangerous, passively-safe levels of capability.
+ 1 more
-
Their words
unaligned operation at a dangerous level of intelligence kills everybody on Earth and then we don’t get to try again.
4 March 2022
-
korrents.com
The next-token language modeling objective is misaligned with following user instructions helpfully and safely.Their words
This is because the language modeling objective used for many recent large LMs-predicting the next token on a webpage from the internet-is different from the objective "follow the user's instructions helpfully and safely" (Radford et al.,, 2019; Brown et al.,, 2020; Fedus et al.,, 2021; Rae et al.,, 2021; Thoppilan et al.,, 2022). Thus, we say that the language modeling objective is misaligned.
23 November 2021
-
Lovedrcmnd.app
The Alignment Problem: Machine Learning and Human ValuesTheir words
I just finished this book a few weeks ago and it is still reverberating in my mind
-
Mixed onrcmnd.app
The Precipice: Existential Risk and the Future of HumanityTheir words
I wouldn’t say that this is the most compelling book I’ve ever read in terms of the prose style or storytelling, but it does provide a very helpful, almost quantitative overview of all the potential threats looming out there
28 October 2020
-
Likedaffiliate linkrcmnd.app
Mastering the VC GameTheir words
The interests of a Venture Capitalist are different than those of the entrepreneurs building a company they've invested in. Jeff does an awesome job of helping explain how you can get misaligned in your goals versus your investors. Fortunately, he also covers how to avoid it.
7 October 2019
-
Lovedrcmnd.app
The AI Does Not Hate You: Superintelligence, Rationality, and the Race to Save the WorldTheir words
Briefly, I think the book is a triumph.
7 March 2018
Nothing matches.
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.