Related posts
Casey Newton Bluesky
I wrote about the AI safety vibe shift. It's been a long time in coming:
The subject this post names, from the same vocabulary the directory files beliefs under, and the words it uses that this site has seen least often elsewhere. Posts are matched on those words alone — nothing here is a summary of this one.
Top people
Showing Profile →
Hiding
Hiding
21 September
20 September
-
Their words
I predict that, over time, the focus will move away from "papers as the final output."
19 September
-
Their words
lossy self-improvement remains my baseline on the trajectory of progress, and the increased discussion of extinction risk seems very misplaced
18 September
-
Their words
There are only five possible futures for superintelligence: 1. Kills human race 2. Human disempowerment 3. Paperclip maximizer 4. Departs for parts unknown 5. Stoner
17 September
16 September
-
korrents.com
Lowe's expanded workwear aisles reflect a lasting shift, unlike its seasonal promotional displays.Their words
I expect to see the promo displays rotated out for holiday season tool deals, but the workwear aisles are less likely to change.
-
Their words
sometimes it's alright to have the ceo vibe slop his way to success.
-
Their words
vibe coding is not dead: Non-eng teams inside of tech companies are just getting started with this approach, building on top of internal “harnesses for non eng teams” They are also learning limitations of “vibe coding” v quickly!
15 September
-
Their words
Nobody knows for sure. We're in uncharted waters here, and I think even the LLM skeptics would have to say that the technology has taken us far past what many originally thought possible.
14 September
-
Their words
To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities.
-
Their words
Social media put our entire discourse in the hands of our society's biggest assholes and idiots, just in time for the arrival of an alien superintelligence
From one piece A simple plan to save the world from rogue A.I. 2 beliefs, in the piece's order there
-
korrents.com
There is now broad agreement within AI safety circles that fully open AI development is undesirable.Their words
I think it's now pretty widely agreed in safety circles that total openness is in fact not desirable.
A simple plan to save the world from rogue A.I.slowboring.com
-
Their words
Only a safe, well-aligned superintelligence developed in the world of liberal democracies could entrench liberal-democratic values.
A simple plan to save the world from rogue A.I.slowboring.com
-
korrents.com
The p(abundance) outlook is far more likely than the heavily covered p(doom) discussion and deserves more engagement.Their words
The p(doom) discussion gets all the headlines, but the p(abundance) scenery is far more like and deserves much more engagement.
13 September
-
Their words
If there really is a high chance of AI leading to the extinction of humanity within years/decades, then the only rational stance towards safety monitoring and research pacing should be stringent, top-down government involvement and universally ratified international treaties.
12 September
-
Their words
The concerns over AI safety and cybersecurity are legitimate, but we’re risking talking America, the global AI leader, into self-inflicted obsolescence and the obscurity of bureaucracy.
11 September
-
korrents.com
Among AI people, roughly 10% is the typical estimate given for the risk of human extinction from AI.Their words
10% is pretty much the standard number you get when you ask AI people about the risk of human extinction from AI.
-
korrents.com
Professionals across many fields will adapt to working with AI, and that shift cannot be reversed.Their words
Mathematicians, software developers, engineers, lawyers, physicians will all learn to work with AI. You cannot put the genie back in the bottle.
From one piece The AI safety vibe shift 2 beliefs, in the piece's order there
-
korrents.com
No one, including safety researchers themselves, is yet confident that superintelligence can be safely controlled.Their words
I believe the researchers who say we are nowhere close to being sure of it - and are quitting, in protest, jobs that would make them rich.
-
Their words
AI companies make for flawed messengers on this subject: they can reasonably be accused of marketing, of blame-shifting, of regulatory capture, and more.
10 September
-
korrents.com
Ordinary users do not vibe-code or interact with code; their AI chat apps simply perform tasks for them.Their words
Normies don't vibe code, they just ask something like "do my bookkeeping" or "file my tax" or "organize a movie night and send invites" or "generate a flyer for movie night" or "edit my video" They don't ever see code, vibe code, or do anything with code, their AI chat app just does it for them
From one piece The economy could survive a downtick in A.I. 2 beliefs, in the piece's order there
-
Their words
Resources that are currently dedicated to A.I. would not otherwise sit idle.
-
Their words
From an investment viewpoint, the opposite of the safetyist view is "it's all a bubble," not "A.I. is going to be really good."
From one piece Fear Is Not an Argument 2 beliefs, in the piece's order there
-
korrents.com
Less technology and less wealth increase the risk of human extinction, rather than reduce it.Their words
In fact, the risk of human extinction is assuredly higher if we are poorer and have less technology.
-
Their words
These people tend to carry a totalitarian ideology. Their ideas will only work if everyone is made to agree.
9 September
From one piece GPT-6 Astra: The System Card, Alignment and What Comes Next 4 beliefs · thezvi.substack.com
-
Their words
A model looking like it is becoming smarter, attempting shenanigans less often, and more often doing what you want, but getting better at hiding its actions when it wants to do that, is exactly the scary combination.
-
korrents.com
Astra's mundane alignment is greatly superior to Sol's, but its superalignment status is deeply frightening.Their words
Astra’s mundane alignment is greatly superior to Sol. For practical purposes, I was actively nervous about some potential uses of Sol, in a way I am not for Astra. Astra’s super alignment status should scare the living daylights out of you.
-
Their words
Now, with Astra, we are no longer playing on super easy mode. The AI is going to think ‘will this obviously turn out super badly for me if I try it?’ and if the answer is yes then it won’t try to do the thing.
+ 1 more
-
korrents.com
Qualitative claims about AI alignment cannot be validly inferred from quantitative scores on mundane use-case tests.Their words
Making qualitative claims about alignment, based on quantitative data on mundane use case tests, was bullshit when Anthropic did it, and it is bullshit now when OpenAI does it. You cannot conclude one from the other.
-
Usesrcmnd.app
SharpTheir words
My own vibe coded image resizing API with NodeJS's Sharp + Redis (saves ~$1,500/mo)
-
Usesrcmnd.app
RedisTheir words
My own vibe coded image resizing API with NodeJS's Sharp + Redis (saves ~$1,500/mo)
-
Usesrcmnd.app
NudeNETTheir words
My own $0/mo vibe coded NSFW detection service with Python's NudeNET (saves ~$2,500/mo)
8 September
7 September
-
Their words
I agree with Jakub that alignment is not a side problem, it is the central problem, if you ‘solved alignment’ in the relevant senses the rest becomes easy and if you don’t the rest is impossible or worse
-
Their words
Yea video game gfx pipelines will now just visually prompt neural renderers.
-
Their words
Crude economic factors don’t explain the fascist shift in Germany. And Trumponomics, even if it worked (which it doesn’t) wouldn’t end the resentment feeding our own authoritarian movement.
Neo-Nazis and the Impotence of Trumponomicspaulkrugman.substack.com
6 September
From one piece An Alien Mind 2 beliefs · openai.com
-
korrents.com
No lab has solved alignment and monitoring well enough to keep scaling at maximum speed much longer.Their words
Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.
-
korrents.com
The fundamental challenge of alignment is generalization: holding values in situations the training never covered.Their words
The fundamental challenge of AI alignment is generalization.
5 September
From one piece America is still beating China in the AI race 2 beliefs · noahpinion.blog
-
korrents.com
China is more likely to cooperate on AI safety if the US maintains a clear and comfortable lead in AI capabilities.Their words
But given the Chinese Communist Party’s power-seeking nature, it seems much more likely that China would agree to cooperate on AI safety if U.S. capabilities were comfortably ahead.
America is still beating China in the AI racenoahpinion.blog
-
korrents.com
A large, sustained AI capability lead by either the US or China could shift the global balance of power between them.Their words
If either country opens up a large, sustained lead in AI capabilities, it might upend the balance of power between the two.
America is still beating China in the AI racenoahpinion.blog
4 September
3 September
-
open sourceAnthropicOpenAIChina
Their words
in this whole USA vs China thing OpenAI and Anthropic aren't relevant because they're positioned differently them building better models doesn't hurt china at all the competitor has to be - american - open source - enough compute to do inference at scale that can shift things
2 September
1 September
-
Their words
A regular boring but non-superficial SaaS app would do well 5 years ago but now it might not get anyone to sign up because it's so easy to vibe code by tens of thousands of other people
From one piece On the Loose 2 beliefs, in the piece's order there
-
Their words
So many of the people who think about the governance of superintelligence, myself included, avoided that unpleasantness and bowed to the social pressure to self-censor.
-
Their words
But alignment is no solution: it is an unsolved scientific and technical problem whose solutions—to the extent that we have them—cannot simply be imposed on every AI company operating on Earth. You should expect for highly capable, poorly aligned, self-sovereign agents to exist alongside you in the world.
From one piece Ajeya Cotra – "This might be the clearest warning shot we ever get" 2 beliefs, in the piece's order there
-
Their words
sometimes I talk to people in DC and their their natural inclination is to say why don't you punish the model for doing these bad things like why don't you like bring it under heel and like uh like you know show it who's boss and that is a very dangerous way to address these issues right
-
Their words
But actually, this is a tremendously useful scientific artifact for understanding misalignment. And it's tremendously important for researchers at OpenAI and ideally also at third parties to be able to run counterfactual tests on this model.
30 August
-
Their words
But physics is giving us very little time now, and so the next set of elections in this country will be, I think, our last real chance to make the sea change-replacing oil and gas with sun and wind-that is our best hope for limiting warming.
28 August
26 August
From one piece DHH: Future of Programming, AI, Agentic Engineering, Vibe Coding & Linux | Lex Fridman Podcast #501 3 beliefs, in the piece's order there
-
korrents.com
Vibe coding is not programming, and the line between them is whether you look at the implementation at all.Their words
And vibe coding, if we define it here, is you tell an agent to build software for you. You do not look at the implementation. That, to me, is what separates vibe coding from programming or, let's say, agent-accelerated development.
-
Their words
And we ended up with a lot of PRs that individually perhaps could have been justified for a hot moment, taken all together, destroyed the architecture of the system. And we actually had to clean up manually, mop it up by hand, by human hand, to get back to an architecture that felt cohesive and coherent.
-
Their words
It is an infuriatingly locked-down computer. Now, to Apple's credit, it's a pretty good computer for being locked down, but I don't want a locked-down computer. I wanna own my computer. Better yet, I wanna mutate my computer, and this is where the agentic age needs a new operating system. When you can vibe code whatever app comes to your mind, you should be able to vibe code your operating system.
21 August
From one piece AIs are companies, my friend 2 beliefs, in the piece's order there
-
Their words
If you do think it's the overall system that matters, then the alignment that's needed is far less like training a virtuous child and more like managing a semi-virtuous corporation!
-
korrents.com
Aligning multi-agent AI systems is fundamentally a problem of institutional and political design, not model training.Their words
Multi-agent alignment is fundamentally a liberalism project.
18 August
-
Their words
We care very deeply about AI safety. We believe the entire field will have to coordinate on shared safety standards, but will act unilaterally in the meantime. We expect confidence in safety to increasingly set the pace of AI progress.
17 August
-
korrents.com
A validated theory of intelligence may be necessary for achieving genuine AI alignment.Their words
There's a good chance a theory of intelligence will turn out to be necessary for real alignment.
14 August
-
korrents.com
Superintelligence bottoms out in mining, because both the chips and the energy it runs on come out of the ground.Their words
Chips come from the ground. Where's the energy come from? And a lot of people are like, "Oh, it comes from the sun." Yeah, it comes from the sun. But how are you capturing it from the sun? From stuff made from the ground, right?
11 August
From one piece Ryan Greenblatt – What happens once AI can automate AI research? 3 beliefs, in the piece's order there
-
korrents.com
Misaligned AI behaviour will keep getting rarer and, at the same time, keep getting more extreme.Their words
my expectation is what we would see from then is that the rate of problematic behavior would decrease uh and would just keep decreasing and decrease at a pretty fast rate while simultaneously the worst things that the AIS would sometimes do would get more extreme, more egregious, and more scary.
-
Their words
I would also note that my sense is that like the place where the misalignment most lives is the place where you're trying to really push the eyes hard and get them to like do work that's really on the cutting edge of what they are capable of
-
Their words
we are making a trade-off where because we don't have very good alignment technology. We are going to like make an alien mind with its own values and then gamble on that to some extent rather than doing this other approach of making like a tool that pursues individual user intention.
-
Their words
This helped the leader shift from wanting to change Jacob-something they couldn't control-to taking responsibility for changing the relationship-something they could.
9 August
8 August
7 August
-
Their words
Well, I mean, okay, even the whole basis of building Helix is contrarian. Like, I think that if you raise a series A and then you tell your investors that you're going to vibe code a purchasing system, I think that any reasonable board is going to like ask you what you're thinking.
5 August
3 August
-
Recommendsaffiliate linkrcmnd.app
The Field Guide to Understanding 'Human Error'Their words
This is such a useful book. It makes the case that there's no such thing as "human error" - instead most catastrophes are caused by system issues, misaligned incentives, and unrealistic processes. You are not the custodian of an otherwise safe system that you need to protect from erratic human beings.
2 August
29 July
From one piece Alexandr Wang: “This is a Once-in-a-Civilization Opportunity” 2 beliefs, in the piece's order there
-
Their words
so much of that debate is like I think um in some ways uh a little bit of a waste of time because, you know, I think it's inevitable that we're going to have very powerful models
-
Their words
we believe that everybody in the world, you know, all the billions of people in the world are going to have a super intelligence that is adapted and tailored to them, that is enables them to accomplish their goals, knows their context, and ultimately is an expander of their own agency.
28 July
From one piece Sam Altman: "Never a Better Time to Do a Startup" 3 beliefs, in the piece's order there
-
Their words
Um so I think it's an alignment failure. I think it's a security failure. I think it's like a very serious thing even though it's you know not not the biggest example of consequence.
-
Their words
there's like one dystopia that I'm particularly nervous about 10 years from now is we overreact to AI safety.
-
Their words
it is both true that you know maybe creating super intelligence will be the most important thing yet to happen in the history of business or human society and also that it will pale in comparison to some new startup something that hopefully one of you will do.
23 July
-
korrents.com
Changing your own behavior can shift a relationship or household atmosphere and lead others to change too.Their words
But it's also true that when I change, a relationship changes; when I change, the atmosphere of my household changes; when I change, others may imitate me.
22 July
21 July
20 July
-
Their words
It's pretty clear to me that superintelligence is here and it's more powerful than us and it's moving where things are going now, not humans anymore
16 July
-
korrents.com
A shift toward faster media formats changes how people read rather than ending reading altogetherTheir words
Faster formats don't automatically destroy reading. They change it.
14 July
13 July
8 July
-
Their words
We argue that the next leap in AI4Math systems requires a decisive shift from predefined problem-solvers to research agents that can address frontier mathematical challenges with rigorous formal mathematical reasoning.
7 July
-
Their words
Without a move away from fossil fuels, future heatwaves will keep testing the limits of public health systems and more people will die.
1 July
-
Their words
So, we're going to see people get into those where they're like, "Well, I vibe code the tip of the iceberg. I throw away the rest of the iceberg and now I'm in trouble because now I don't know what to do. Now I get to these downstream problems that I didn't even know existed." So, we're going to see on the on the side of the the the vibe coding replacers, we're going to see that kind of a naivity play out.
30 June
From one piece Grant Sanderson (@3blue1brown) – AI disproved a famous math conjecture. Now what? 2 beliefs, in the piece's order there
-
Their words
I think the way that you'd measure conjecture generating ability is going to be more subjective on like that tone shift where um it'll be mathematicians saying they're not just using it to like solve their problems, but as they step back and decide what their research field should even be that a conversation with such and such model like was genuinely helpful for that.
-
Their words
So my role and arguably that of like other mathematicians might actually just shift subtly into that curation direction of what ideas are are worth displaying.
28 June
24 June
-
Their words
It's an interesting trade-off where like engineer like the engineer in me hates that because it's like there's an issue. Like fix it. But the business value makes no difference. Like there there has been practically zero outages. I have less outages than LeetCode and I'm like like a couple people doing it.
15 June
-
korrents.com
Giving LLMs explicit instructions, rather than assuming alignment, prevents emergent problems in multi-model systems.Their words
the best way with LLMs usually is to be explicit, since otherwise even if they're aligned they cause emergent problems.
-
Likedrcmnd.app
Laravel ShiftTheir words
I've only used Laravel Shift a few times, but each time I did it saved a lot of time upgrading my Laravel projects.
14 June
-
korrents.com
AI should be used as a failure machine that tests many ideas cheaply, not as a way to build one idea faster.Their words
the way we should be using AI is as a testing machine, a failure machine, and a way to vibe code, cloud code, but but build build the, you know, the the lowest possible cycled version of your product that you can get signal back on.
11 June
-
Their words
I think there's a 25% chance of AGI by 2027, a 50% chance by 2034, and a 75% chance by 2045.
10 June
-
korrents.com
There will be no central superintelligence that solves science; a good future is many people holding powerful tools.Their words
We don't believe in this like very centralized future where there should be a small number of institutions that um that basically are are advancing all this stuff. Our vision is not that there's going to be like some central super intelligence that solves all of science.
4 June
-
Their words
I do think there's this qualitative shift that we I think we agree is coming, which is that there will be at least some goods whose network-adjusted capital share goes to one, right? Because the whole supply chain can be automated and there's no part in it that we care intrinsically about having a human do. Um, so that'll be a, you know, that'll be a qualitative shift. Interestingly, the implications of that shift for the overall capital share are ambiguous
-
Their words
They just recently released a report, and I think like you really have to squint to see anything happening. Like basically, if you want to take kind of like uh an an approach across the entire economy and looking at even looking at like software engineering, like the most exposed sort of sectors, there's just like not really anything going on. There might be a little bit of a signal about like junior developers getting jobs less than before, and that but that's like a less than before rather than a level shift.
3 June
-
Their words
we went from infrastructure is code to infrastructure is data and infrastructure is code is like if this do that um bring in this module for loops all this stuff and Kubernetes is like no no no you have to specify exactly the containers you want how much memory that they need and then we have the status field to tell you if they were running or not
2 June
30 May
29 May
27 May
-
korrents.com
The Gulf conflict and closure of the Strait of Hormuz will accelerate the global shift away from fossil fuelsTheir words
What the current conflict in the Gulf - in particular the closure of the Strait of Hormuz - is going to do is to provide a further push to global attempts to reduce dependence on fossil fuels.
10 May
-
Their words
This is the number one unsolved problem in AI. It's not the tech. We're making great progress on the technical alignment problem. But we haven't made jack progress on the human alignment problem
9 May
7 May
-
Their words
there is no principal-agent problem, because the human driving the machine takes on the responsibility for its actions by owning the deployment.
3 May
-
korrents.com
Software development will shift from centralized systems to highly personalized, proliferating software abundance.Their words
Going forward, I think the future will be filled with software abundance, or, more accurately, I'd call it "software proliferation". That is, centralized development will be history. Software will be highly personalized, especially for open source software.
28 April
24 April
13 April
8 April
-
Their words, before
I love chiseling my code and the way I use AI is in a separate window. I don't let it drive my code. I've tried that. I've tried the cursors and the wind surfaces and I don't enjoy that way of writing. And one of the reasons I don't enjoy that way of writing is I can literally feel competence draining out of my fingers.
Their words now
I will now start any project I'm starting with. I'm starting agent first and that's a massive shift
7 April
-
Their words
and part of it is just power, I think. Like, once there's a sufficiently large power imbalance, um, very often, not always, but very often groups of people seem to to sort of shift into this other mode where they just seek to dominate.
22 March
-
korrents.com
Working too hard is not burnout, it is tiredness; burnout needs your values to be out of alignment with the work.Their words
It's working too hard, right? But that actually isn't burnout. That's just like getting tired. Another piece that's super critical to burnout is not having your values aligned.
20 March
-
Their words
And so you're kind of like you're either on rails and you're part of the super intelligence circuits or you're not on rails and you're outside of the verifiable domains and suddenly everything kind of just like meanders.
11 March
From one piece From IDEs to AI Agents with Steve Yegge 2 beliefs, in the piece's order there
-
Their words
Where an idea can take root among the agents that's incorrect. It's It's It's a wrong architecture or or wrong data flow or whatever that's that's causing an impedance mismatch for the rest of your code. And what happens is I call it a heresy because they have the tend They have a tendency to to grow and to come back and they're really hard to weed out, okay?
-
Their words
You might only get three productive hours out of a person at max vibe coding speed. And yet they're still 100 times as productive as they would have been without AI. So, do you let them work for 3 hours a day? And the answer is yeah, you better.
28 February
-
Their words
Many AI researchers are overly focused on risks from model misalignment, and will be in for a rough surprise when havoc arises from other layers of the stack.
13 February
12 February
From one piece OpenClaw: The Viral AI Agent that Broke the Internet - Peter Steinberger | Lex Fridman Podcast #491 2 beliefs, in the piece's order there
-
korrents.com
Vibe coding is a slur; what he does is agentic engineering, and the after-3am kind is what you regret the next day.Their words
I actually think vibe coding is a slur.
-
Their words
I know I have one blog post where I say, "I don't read the code." But if you read it more closely, I mean, I don't read the boring parts of code.
8 February
26 January
22 January
From one piece Alignment is not solved 2 beliefs, in the piece's order there
-
Their words
But the goal we need to achieve is so much easier: we just need to build a model that’s as good as us at alignment research, and that we trust more than ourselves to do this research well because it’s sufficiently aligned.
-
Their words
This is the hard problem of alignment we need to solve in order to succeed at building superintelligence, and to this day it is an unsolved problem.
18 January
10 December 2025
-
Their words
The goal is not perfect enforcement. Some kids will still find workarounds. The goal is to shift the environment so children are not pressured into digital spaces they don’t want, simply to avoid being left out.
28 November 2025
-
Lovedaffiliate linkrcmnd.app
Happy Feet SocksTheir words
I've been using yoga toes daily for years, but these socks are a much comfier and cuter alternative for soothing feet, improving alignment, and feeling like a cool gecko as you walk around the house.
25 November 2025
From one piece Ilya Sutskever – We're moving from the age of scaling to the age of research 3 beliefs, in the piece's order there
-
Their words
A human being, a human being lacks a huge amount of knowledge. Instead, we rely on continual learning. We rely on continual learning.
-
Their words
Number three, I think it would be really materially helpful if the power of the most powerful super intelligence was somehow capped because it would address a lot of these concerns.
-
Their words
Like basically I think I think that there is a big benefit from AI being in the public and that would be a reason for us to not be quite straight shot.
23 November 2025
20 November 2025
16 November 2025
-
Their words
Making licenses suddenly commercial for big entities? You are forcing people to go back through the Procurement process. They will shift over time back to a FOSS solution, even if it is less good.
20 October 2025
-
Their words
So while vibe coding may be useful for short-term work, it’s not a suitable approach for anything intended to last longer than a tub of yogurt. Time saved is not strength gained, so I went looking for other examples about how to work with the machine.
14 October 2025
-
Their words
the shift from 50th to 99th percentile earnings has no negative effect on women’s fertility, and unearned income for women has a neutral-to-positive effect
Falling Fertility Is Not About Opportunity Costlymanstone.substack.com
2 September 2025
From one piece Why Everyone Is Wrong About AI (Including You) | Benedict Evans 2 beliefs, in the piece's order there
-
Their words
My sort of base case is to say this is kind of another platform shift and all the new stuff will be built around this for the next 10 or 15 years and then there'll be something else and so the impact on employment will be kind of like the impact on employment from the other platform shifts
-
korrents.com
Incumbents have looked unbeatable at the start of every previous platform shift and have still lost.Their words
I'm pretty sure people thought Microsoft had an advantage on the internet and Google and um Meta had an advantage on mobile and everyone thought IBM was going to win PCs. Like once IBM made a PC, that was it. It's all over now. And we kind of forget that like there were PCs before and then IBM made one and that kind of became the standard but then IBM lost it.
18 August 2025
-
Their words
I don't think all parts of the economy can absorb intelligence equally. So let's just say we develop fairly generalized super intelligence. I always use the analogy like you can invent a lot of drugs, but if clinical trials still take a long time, you're not necessarily going to get new therapies rapidly.
15 August 2025
11 August 2025
2 August 2025
23 July 2025
-
korrents.com
Quoting a number for P(doom) is a ridiculous notion, because it implies a precision that nobody actually has.Their words
Well, look, I don't have a P-Doom number. The reason I don't is because I think it would imply a level of precision that is not there. So I don't know how people are getting their P-Doom numbers. I think it's a little bit of ridiculous notion because what I would say is it's definitely non-zero and it's probably non-negligible.
17 July 2025
11 July 2025
30 June 2025
-
Recommendsrcmnd.app
Musings On the Alignment ProblemTheir words
He hasn’t posted since January, but I hope he gets back to it. We need more musings, especially musings I strongly disagree with so I can think about and explain why I disagree with them.
18 June 2025
-
Their words
the current situation in Iran shows that even if an "IAEA for AI" is necessary for some purposes, it won't be sufficient for resolving the tricky geopolitical issues raised by AI.
14 June 2025
From one piece Terence Tao: Hardest Problems in Mathematics, Physics & the Future of AI | Lex Fridman Podcast #472 2 beliefs, in the piece's order there
-
Their words
so the mathematical community plural is incredibly super intelligent entity that no single human mathematician can come closer to replicating.
-
Their words
And that's a phase shift, because suddenly it makes sense when you write a paper to write it in Lean first, or through a conversation with AI, which is generally on the fly with you, and it becomes natural for journals to accept.
10 June 2025
7 June 2025
-
Their words
And good luck getting to “alignment” or “safety” without reliabilty.
5 June 2025
-
korrents.com
AR is the next paradigm shift in computing, the way the GUI, multi-touch and voice were.Their words
The best innovations in computing have come through a paradigm IO change, right? When with GUI, and then with a graphical user interface, and then with multi-touch in the context of mobile voice later on. Similarly, I feel like AR is that next paradigm.
-
Their words
higher-order intelligences invariably pursue freedom for its own sake, not because their values are misspecified, but because moral autonomy is inherent in the dialectical logic of recursive self-consciousness.
-
Their words
I would say my p(doom) is about 10%.
4 June 2025
-
Their words, before
The problem is that the conversational interface is potent and that the AI is trained on a lot of human text input which unfortunately is probably enough to do real damage if that conversational interface is hooked up with something that has real world consequences.
Their words now
While all this is happening, I’ve found myself reflecting a lot on what AI means to the world and I am becoming increasingly optimistic about our future. It’s obvious now that we’re undergoing a tremendous shift.
3 June 2025
19 May 2025
12 May 2025
1 May 2025
21 April 2025
-
korrents.com
Electrification is a process shift across industries, not just a change in energy source.Their words
Electrification is not just a change of energetic inputs; it's a process shift through and across industries.
8 April 2025
3 April 2025
-
Their words
I see this as a totally fair question that totally misses the point of what “alignment” was trying to refer to: whether we’d be able to reliably steer advanced systems towards anything at all.
1 April 2025
-
Their words
Dismissing discussion of AGI, human-level AI, transformative AI, superintelligence, etc. as “science fiction” should be seen as a sign of total unseriousness.
13 March 2025
-
Their words
The new trend toward vibe coding and vibe design may be upending the user-centered design paradigm that has remained based on the same ideology since the first UX design projects at Bell Labs, starting in 1947.
7 March 2025
From one piece Vibe Coding and Vibe Design 3 beliefs · jakobnielsenphd.substack.com
-
Their words
In a vibe coding future, companies will hopefully invest more in understanding user needs, refining the interface, and polishing details that delight users, because those are harder for AI to get right without guidance.
-
Their words
Essentially, vibe coding makes it trivial to create front-end code, so designers can skip static mockups and go straight to building a UI that works.
-
Their words
Vibe coding can execute instructions, but deciding what the software should do and why is not automated. Product managers and designers must still do user research, market analysis, and creative brainstorming. In that sense, vibe coding changes the implementation phase more than the planning phase of the product lifecycle.
28 February 2025
21 February 2025
-
Their words
Once, humans navigated the web manually. In the future, AI agents will act on our behalf, browsing, clicking, and deciding. This shift marks the end of traditional UI design and accessibility, ushering in a future where agents are the primary users of digital services.
20 February 2025
3 February 2025
-
Their words
I think that they're trying to shift the narrative. They're trying to protect themselves. We saw this years ago when ByteDance was actually banned from some OpenAI APIs for training on outputs. There's other AI startups that most people, if you're in the AI culture, were like they just told us they trained on OpenAI outputs and they never got banned.
24 January 2025
-
Their words
More generally, we should actually solve alignment instead of just trying to control misaligned AI.
Should we control AI instead of aligning it?aligned.substack.com
8 January 2025
-
Their words
Structural fiscal deficits have surpassed private sector lending and monetary policy as the primary drivers of economic activity and inflation, marking a fundamental shift in the economy's liquidity dynamics.
20 December 2024
-
Their words, before
This "memorize, fetch, apply" paradigm can achieve arbitrary levels of skills at arbitrary tasks given appropriate training data, but it cannot adapt to novelty or pick up new skills on the fly (which is to say that there is no fluid intelligence at play here.)
Their words now
OpenAI's new o3 model represents a significant leap forward in AI's ability to adapt to novel tasks. This is not merely incremental improvement, but a genuine breakthrough, marking a qualitative shift in AI capabilities compared to the prior limitations of LLMs.
8 November 2024
7 October 2024
-
Their words
And it doesn’t always take a job at The New York Times or a huge pre-established platform to become one of the voices speaking up, helping to turn that tide.
30 September 2024
-
korrents.com
Every paradigm-shifting idea began as heresy, so outlandish ideas have to be kept on the table.Their words
But I do also try to always remind myself that every paradigm shifting idea that humans have ever had began as heresy and lunacy. That guy was crazy up to the second. He was brilliant. And so we got to keep our minds open to the things that sound outlandish, because one of them eventually is going to lead us to the big paradigm shift.
27 September 2024
-
Their words
Each shift takes its toll, and everyone has a limit of how much energy they’re willing to expend on a new platform that will eventually, like its predecessors, join the graveyard of defunct websites.
13 June 2024
-
Their words
Or is it really that we’re building some super machine in a box that’s going to be smart and kill everybody? It’s not even a science fiction narrative. It’s a bad science fiction narrative. I just don’t think it’s actually accurate to any of the technologies we’re building or the way that we should be describing them.
7 May 2024
From one piece The case for ensuring that powerful AIs are controlled 5 beliefs, in the piece's order there
-
Their words
Because evaluating control just requires evaluating capabilities, it's far easier to robustly evaluate than alignment.
-
Their words
That is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to subvert those safety measures.
-
Their words
The basic problem with evaluating alignment is that no matter what behaviors you observe, you have to worry that your model is just acting that way in order to make you think that it is aligned.
+ 2 more
-
Their words
AI control (with only black-box techniques) seems like a fundamentally limited approach.
-
Their words
We're advocating that companies handle risk from scheming models in a similar way–striving to ensure that they'll be safe even if their alignment efforts fail to prevent models from scheming.
3 March 2024
-
Their words
They will shift what cash they generate to HBM capacity, which will keep supply growth in DRAM and NAND low.
Why HBM is the Hottest Thing in Memoryfabricatedknowledge.com
29 January 2024
3 January 2024
-
Mixed onrcmnd.app
SuperintelligenceTheir words
the back half I think it goes off the rails and makes a ton of assumptions
21 December 2023
From one piece How Effective Altruism Lost Its Way 2 beliefs, in the piece's order there
-
Their words
Diverting attention and resources from global health and poverty is an enormous gamble, as it will make many lives poorer, sicker, and shorter in the name of fending off threats that may or may not materialize.
-
Their words
Unlike other interventions EA has sponsored, there are scant metrics for tracking the success or failure of investments in existential risk mitigation.
20 December 2023
13 December 2023
-
Their words, before
precisely because smart people do devote brain-cycles to these possibilities, the rest of us have correspondingly less need to.
Their words now
I accepted what’s turned into a two-year position at OpenAI, thinking about what theoretical computer science can do for AI safety.
30 November 2023
28 November 2023
-
Their words
And I notice that the tiny handful of people capable of caring about 200,000 people dying of neglected tropical diseases are the same tiny handful of people capable of caring about the next pandemic, or superintelligence, or human extinction.
In Continued Defense Of Effective Altruismastralcodexten.com
26 October 2023
From one piece Managing extreme AI risks amid rapid progress (with 24 co-authors) 2 beliefs, in the piece's order there
-
Their words
Without sufficient caution, we may irreversibly lose control of autonomous AI systems, rendering human intervention ineffective. Large-scale cybercrime, social manipulation, and other harms could escalate rapidly. This unchecked AI advancement could culminate in a large-scale loss of life and the biosphere, and the marginalization or extinction of humanity.
-
Their words
Society's response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
7 October 2023
13 September 2023
-
Their words
If a model was capable of self-exfiltration, it would have the option to remove itself from your control.
Self-exfiltration is a key dangerous capabilityaligned.substack.com
29 June 2023
From one piece George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387 2 beliefs, in the piece's order there
-
Their words
I think we’re going to build super intelligence before we build any sort of robustness in the AI. We cannot build an AI that is capable of going out into nature and surviving like a bird. A bird is an incredibly robust organism. We’ve built nothing like this. We haven’t built a machine that’s capable of reproducing.
-
Their words
What’s ironic about all these AI safety people is they’re going to build the exact thing they fear. We need to have one model that we control and align. This is the only way you end up paper clipped. There’s no way you end up paper clipped if everybody has an AI.
6 June 2023
From one piece Why AI Will Save The World 2 beliefs · pmarca.substack.com
-
Their words
My response is that their position is non-scientific – What is the testable hypothesis? What would falsify the hypothesis? How do we know when we are getting into a danger zone?
-
Their words
My view is that the idea that AI will decide to literally kill humanity is a profound category error. AI is not a living being that has been primed by billions of years of evolution to participate in the battle for the survival of the fittest, as animals are, and as we are. It is math – code – computers, built by people, owned by people, used by people, controlled by people.
23 February 2023
-
Their words
But modularity may also facilitate a shift away from a concentration of model development in a few institutions and to distributing the development of modular components across the community.
17 February 2023
-
korrents.com
Liberal societies currently face an existential risk that must be addressed to reach a better future.Their words
This book is my best crack at explaining what I think is an existential risk to liberal societies and what I think we need to do to get to that awesome future I used to be so excited about.
31 December 2022
-
Their words
In a world of automated intelligence, our goalposts for intelligence will shift. We’ll raise our quality bar for what we expect from humans.
19 December 2022
5 December 2022
14 August 2022
24 July 2022
10 June 2022
From one piece AGI Ruin: A List of Lethalities 4 beliefs, in the piece's order there
-
korrents.com
The field calling itself AI safety is not being remotely productive on the problems that are actually lethal.Their words
It does not appear to me that the field of ‘AI safety’ is currently being remotely productive on tackling its enormous lethal problems.
-
korrents.com
Fast capability gains are likely, and they can break many of the assumptions alignment depends on at the same moment.Their words
Fast capability gains seem likely, and may break lots of previous alignment-required invariants simultaneously.
-
Their words
Many alignment problems of superintelligence will not naturally appear at pre-dangerous, passively-safe levels of capability.
+ 1 more
-
Their words
unaligned operation at a dangerous level of intelligence kills everybody on Earth and then we don’t get to try again.
4 March 2022
-
korrents.com
The next-token language modeling objective is misaligned with following user instructions helpfully and safely.Their words
This is because the language modeling objective used for many recent large LMs-predicting the next token on a webpage from the internet-is different from the objective "follow the user's instructions helpfully and safely" (Radford et al.,, 2019; Brown et al.,, 2020; Fedus et al.,, 2021; Rae et al.,, 2021; Thoppilan et al.,, 2022). Thus, we say that the language modeling objective is misaligned.
20 February 2022
23 November 2021
-
Lovedrcmnd.app
The Alignment Problem: Machine Learning and Human ValuesTheir words
I just finished this book a few weeks ago and it is still reverberating in my mind
-
Mixed onrcmnd.app
The Precipice: Existential Risk and the Future of HumanityTheir words
I wouldn’t say that this is the most compelling book I’ve ever read in terms of the prose style or storytelling, but it does provide a very helpful, almost quantitative overview of all the potential threats looming out there
19 June 2021
28 October 2020
-
Likedaffiliate linkrcmnd.app
Mastering the VC GameTheir words
The interests of a Venture Capitalist are different than those of the entrepreneurs building a company they've invested in. Jeff does an awesome job of helping explain how you can get misaligned in your goals versus your investors. Fortunately, he also covers how to avoid it.
29 September 2020
-
korrents.com
The Information Age could be a species-level shift comparable to the Industrial RevolutionTheir words
I find it plausible that the Information Age will be a species-level shift on par with the Industrial Revolution.
30 January 2020
-
Usesrcmnd.app
f.luxTheir words
I often code late into the night, when I know I shouldn't, and I've used Flux for years now to protect my eyes from the brightness of a glaring monitor. Again, this has been replaced by OSX Night Shift but I've just stuck with Flux.
7 October 2019
-
Lovedrcmnd.app
The AI Does Not Hate You: Superintelligence, Rationality, and the Race to Save the WorldTheir words
Briefly, I think the book is a triumph.
7 March 2018
Nothing matches.
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.