Related posts
Thomas Wolf
Bluesky
Oh! My TED talk is just out. I talk about how open-source is critical for AI safety and humanity's resilience. Yes, you don't hear that perspective often! :) It's called "What if AI just works" It's here:
AI alignment resilience hear
The subject this post names, from the same vocabulary
the directory files beliefs under, and the words it uses that this site has seen
least often elsewhere. Posts are matched on those words alone —
nothing here is a summary of this one.
Sources
All Writing Newsletter x.com Korrents Mastodon Blog Bluesky Podcast YouTube Site GitHub Recommends Papers Changed their mind
Top people
Yoshua Bengio
Zvi Mowshowitz
Jan Leike
Sam Altman
Casey Newton
Noah Smith
Dean W. Ball
Rohit Krishnan
Buck Shlegeris
Eliezer Yudkowsky
Nathan Lambert
Gary Marcus
Showing Profile →
Show everything
Hiding Show them again
Show them again
Further back ↓
Hiding Show them again
19 September
Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.
Author and clean-energy analyst who writes on solar cost curves.
The EAs I know are generally lovely people, smart, thoughtful. I have a very low p(doom). I know many people with one much higher than mine. I believe each is sincere. AI safety debates are getting heated. Let's assume sincerity and good will of people we disagree with. AI alignment Related
Machine-learning researcher on open language models; writes the Interconnects newsletter and the RLHF Book, after leading post-training at Ai2.
18 September
Writer and teacher on productivity and personal knowledge management. Author of Building a Second Brain and founder of Forte Labs, which runs the course of the same name.
Cognitive scientist and long-standing critic of deep learning's claims; writes Marcus on AI and wrote Rebooting AI.
Writes Hyperdimensional, a newsletter on AI policy and governance. A White House AI policy adviser in 2025; joined OpenAI on 6 July 2026 to lead its Strategic Futures team.
The people dismissing this as preposterous are telling on themselves as not having thought about superintelligence seriously. I hadn’t considered this particular idea but you should expect, by definition, that something much smarter than you would have ideas you didn’t think of Quoting @firesidealpha OpenAI's Noam Brown says air-gapping the computers may not stop a misaligned AI, because two air-gapped machines can still talk by running a CPU hot and reading the temperature change "But I think the major takeaway from the incident is that people underestimated the AI. And we never want to be in… AI alignment Related
Mathematical physicist at UC Riverside. Wrote This Week's Finds in Mathematical Physics and now the Azimuth blog on mathematics, physics and environmental science.
The social scene around AI and AI safety is more unpleasant than I realized. I'm sure there are other sides to it. But when you get a lot of men with too much money who think they're saving the world, surrounded by groupies, what do you expect? Read the long Bluesky thread about this, collected on one page: https://skywriter.blue/@segyges.bsky.social/3mvom4b4dn22q I can't confirm this stuff - I stay far away from this scene. And the sentence "AI Safety is Mostly a Sex Cult" is a massive exaggeration. But... (1/n) AI alignment Related
Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.
Genomics researcher and writer on science, biotech and the culture of research.
Economics writer; author of the Noahpinion newsletter.
AI alignment
Their words
There are only five possible futures for superintelligence: 1. Kills human race 2. Human disempowerment 3. Paperclip maximizer 4. Departs for parts unknown 5. Stoner
Show the whole quote
@Noahpinion on X x.com
17 September
Co-creator of Django and creator of Datasette; writes daily at simonwillison.net.
Writes Hyperdimensional, a newsletter on AI policy and governance. A White House AI policy adviser in 2025; joined OpenAI on 6 July 2026 to lead its Strategic Futures team.
“Every night I close my eyes And there is something that I visualize I hear a voice, say, ‘I love you’ I picture all things that we're gonna do And I wonder who he'll be Wonder if he'll be good to me Wonder, gosh oh gee Wonder if he'll love me forever” Related
Bulgarian-born writer and founder of The Marginalian (formerly Brain Pickings), a reader-funded blog on books, science, art and philosophy running since 2006.
Front-end web developer at Paravel and co-host of the ShopTalk Show podcast; blogs at daverupert.com.
Hear me out... @dropout.tv should buy Vine and bring it back. Related
Interviewer; the Dwarkesh Podcast runs long, heavily researched conversations with AI researchers, historians and economists.
Director of institutional asset management at Ritholtz Wealth Management. Writes the A Wealth of Common Sense blog and co-hosts the Animal Spirits podcast with Michael Batnick.
Creator of Flask and Jinja. Writes about software at lucumr.pocoo.org.
Where did I hear this before that joining the EU is a hostile act. Quoting @Polymarket JUST IN: President Trump warns the EU allowing Canada to join as an associate member could be considered a “hostile act” toward the U.S. Europe Related
16 September
Security engineer; founder of Matasano Security and Latacora.
Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.
Mathematician at UCLA, working mainly in harmonic analysis and partial differential equations. Writes What's new, a long-running blog on their research, open problems and expository notes.
Founder of Farnam Street (fs.blog) and host of The Knowledge Project podcast. Writes and curates on decision-making, mental models and rational thinking.
I asked @tobi about super intelligence. "Super intelligence is the existence of something vastly smarter than us in the aggregate that's accessible to us, which is society, which is the city, which is the community." "We are living in the presence of superintelligence our entire lives." "We make it work because we've created systems by which you know, which govern the super intelligence and how it acts." Quoting @shaneparrish My third conversation with Shopify co-founder and CEO @tobi. 0:00 How Shopify Uses AI 7:18 River: Shopify's Internal AI 8:55 How to Encourage Osmosis Learning 10:52 AI Dreaming and Self-Reflection 11:53 How to Use AI for Strategic Decision Making 14:11 The One Thing AI Cannot Do 16:04 What AI is Ma… AI alignment Related
15 September
Co-founder and CEO of Stripe and co-founder of the Arc Institute. Keeps a personal site of reading lists, open questions and notes on scientific progress and how research is funded.
I'm curious to read other stories from people who've scaled new businesses in Europe. (And if your experience has been positive, that's also useful to hear.) Feel free to email me: patrick@stripe.com. Quoting @patrickc Met a German founder this week and asked him if all the stories one reads about the challenges of startups in Germany are exaggerated. "No, they're understated." Proceeded to describe spending a full day having a 90-page investment contract read to him (mandatory under German law; § 13 BeurkG) by a… Europe Related
Founder of Farnam Street (fs.blog) and host of The Knowledge Project podcast. Writes and curates on decision-making, mental models and rational thinking.
My third conversation with Shopify co-founder and CEO @tobi. 0:00 How Shopify Uses AI 7:18 River: Shopify's Internal AI 8:55 How to Encourage Osmosis Learning 10:52 AI Dreaming and Self-Reflection 11:53 How to Use AI for Strategic Decision Making 14:11 The One Thing AI Cannot Do 16:04 What AI is Making Worse at Shopify 19:46 Predictions: Where AI is Headed Next 21:55 The Future of AI-Powered Software 24:40 Will CEOs Be Replaced with AI? 27:54 Can Superintelligence Be Controlled? 31:13 Critical Skills in AI Age 34:22 Why Complex Solutions are Usually Wrong 36:33 Conditions Needed for True Intu… AI alignment Related
Founder and chief executive of Replit. Previously an engineer at Facebook and Codecademy. Writes essays on programming, philosophy and entrepreneurship at amasad.me.
The AI naming curse strikes again. “AI Safety” firm made AI unsafe. “Effective Altruists” are both ineffective and enabling criminal activity. “Irregular” is regularly incompetent. Quoting @brianchau57 BREAKING: A single Israeli Effective Altruism firm is behind OpenAI, Anthropic, and Meta cyberattacks 🧵 with help from @lumpenspace AI alignment Related
Technology writer of Spyglass, a newsletter about technology and media. Previously a reporter at TechCrunch and an investor at GV.
14 September
Interviewer; the Dwarkesh Podcast runs long, heavily researched conversations with AI researchers, historians and economists.
Founder and CEO of Social Capital, a venture firm; co-host of the All-In podcast and an early Facebook executive. Writes an annual letter and a weekly newsletter on markets and technology.
Writes Hyperdimensional, a newsletter on AI policy and governance. A White House AI policy adviser in 2025; joined OpenAI on 6 July 2026 to lead its Strategic Futures team.
Some suppose that “safety” and “innovation” in AI are at odds. My suspicion is the opposite: the next generation of breakthroughs in AI will be in safety, alignment, and monitorability. Pushing the frontier forward from here will require dramatic innovations in safety. AI alignment Related
Professor of economics at George Mason University and Bartley J. Madden Chair at the Mercatus Center. Co-writes the blog Marginal Revolution and co-authors the textbook Modern Principles of Economics with Tyler Cowen.
Writer on the intersection of technology and finance. Author of the Bits about Money newsletter and host of the Complex Systems podcast; previously at Stripe.
Economics writer; author of the Noahpinion newsletter.
Co-founder and CEO of OpenAI; previously president of Y Combinator.
AI alignment
Their words
To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities.
Show the whole quote
@sama on X x.com
Economics writer; author of the Noahpinion newsletter.
AI alignment
Their words
Social media put our entire discourse in the hands of our society's biggest assholes and idiots, just in time for the arrival of an alien superintelligence
Show the whole quote
@Noahpinion on X x.com
Creator of Ruby on Rails, CTO of 37signals, and creator of Omarchy.
AI alignment
Their words
The p(doom) discussion gets all the headlines, but the p(abundance) scenery is far more like and deserves much more engagement.
Show the whole quote
@dhh on X x.com
13 September
Creator of the Keras deep-learning library and the ARC-AGI benchmark.
AI alignment government
Their words
If there really is a high chance of AI leading to the extinction of humanity within years/decades, then the only rational stance towards safety monitoring and research pacing should be stringent, top-down government involvement and universally ratified international treaties.
Show the whole quote
@fchollet on X x.com
12 September
Mathematical physicist at UC Riverside. Wrote This Week's Finds in Mathematical Physics and now the Azimuth blog on mathematics, physics and environmental science.
Investor and writer. Previously a partner at Andreessen Horowitz and a product leader at Twitter, Facebook, Snap and Microsoft. Writes essays and memos at sriramk.com.
I find myself disagreeing with Terence Tao (a very scary phrase to say!) and think the core question posed is whether the "misalignment" is between the mathematical community and humanity. In other words - are the Millenium Prize problems meant to incentivize advancements for humanity or foster the field of mathematics and mathematicians? As a counter example, if there was a prize for creating a drug that cured a rare strain of cancer, we would not care if it was AI that did it. We would be happy it has been solved. On the other hand, if long running agents figured out the puzzle in "Kryptos"… AI alignment Related
CEO of Vercel; creator of Next.js and Socket.IO. Writes at rauchg.com.
AI alignment America
Their words
The concerns over AI safety and cybersecurity are legitimate, but we’re risking talking America, the global AI leader, into self-inflicted obsolescence and the obscurity of bureaucracy.
Show the whole quote
@rauchg on X x.com
Creator of Flask and Jinja. Writes about software at lucumr.pocoo.org.
11 September
Machine learning engineer and consultant focused on RAG and retrieval systems. He writes about applied AI engineering at jxnl.co and is the author of the instructor library.
I'll be helping out with rosalind soon! I want to hear what you think, how we can i help, and what we can do better! Quoting @OpenAIDevs Bring stronger biological reasoning to your research with GPT-Rosalind in the API and Codex. Connect findings across papers and experimental results, weigh the evidence for a biological target, and work through an analysis to plan what to test next. Related
Co-founder of WordPress and founder of Automattic; blogs at ma.tt.
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
Over the past few days, I've taken the time to summarize my thoughts on the recent incidents involving agents’ misaligned behavior. We don't know with certainty what comes next, but we know where these issues originate, and this can help us plan the path forward. English, from French Ces derniers jours, j'ai pris le temps de résumer mes réflexions sur les récents incidents impliquant des comportements inappropriés d'agents. Nous ne savons pas avec certitude ce qui va suivre, mais nous savons d'où viennent ces problèmes, et cela peut nous aider à planifier la voie à suivre.
AI alignment Related
Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
Over the past few days, I've taken the time to summarize my thoughts on the recent incidents involving agents’ misaligned behavior. We don't know with certainty what comes next, but we know where these issues originate, and this can help us plan the path forward. Please feel free to ask your questions in the replies, and I’ll try to answer some of them in the coming weeks. yoshuabengio.org
AI alignment Related
Founder of Platformer, a newsletter on tech platforms and the people they affect, and co-host of the Hard Fork podcast at the New York Times.
Computer science professor who works on fast data processing; co-author of the simdjson parser and a weekly blogger about software performance since 2004.
The gold standard in the statistical analysis of scientific data is a p-value of 0.05. Never mind what it means. It is the threshold people use to say the result did not occur by chance. There is a wide consensus. Nearly all experts treat it as the bar to statistical significance. I am quite certain that most do not know what it means or where the number comes from. Never mind that. In principle, it means that if you redo the experiment, the result should hold. At least, that's what most people hope when they hear that a result is 'statistically significant'. It often will not. If the origina… Quoting @robinhanson !!: "A finding with a p-value of 0.05 has an expected replication probability ranging from 0.10 to 0.25 across fields." Related
Economics writer; author of the Noahpinion newsletter.
AI alignment
Their words
10% is pretty much the standard number you get when you ask AI people about the risk of human extinction from AI.
Show the whole quote
@Noahpinion on X x.com
Founder of Platformer, a newsletter on tech platforms and the people they affect, and co-host of the Hard Fork podcast at the New York Times.
From one piece
The AI safety vibe shift
2 beliefs, in the piece's order there
10 September
Professor of finance at NYU's Stern School of Business, known for his work on valuation. He publishes his data, spreadsheets and classes free at Damodaran Online and writes the Musings on Markets blog.
On a trading day when interest rates are moving markets, I look at rising rates in 2026, hazard a guess as to why they are rising, and make a stab at explaining the resilience of equities in the face of rising rates, with nary a mention of the Fed. bit.ly
interest rates Related
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
Theoretical physicist and philosopher, Homewood Professor of Natural Philosophy at Johns Hopkins and fractal faculty at the Santa Fe Institute. Writes the Preposterous Universe blog and hosts the Mindscape podcast.
My thoughts on existential risk: * Humanity isn't going to be wiped out any time soon. * Negligible chance that AI by itself causes catastrophic damage to humanity (millions dead). * Some nontrivial chance that human beings will leverage AI to help them do something catastrophically harmful. AI alignment Related
Computer science professor at Georgetown University and author of Deep Work, Digital Minimalism and Slow Productivity; writes about focus, technology and work at calnewport.com and hosts the Deep Questions podcast.
Writer and podcaster; co-author of Abundance with Ezra Klein, a staff writer at The Atlantic from 2008 to 2025, and host of the Plain English podcast.
Former L8 engineer at Meta, Microsoft and Atlassian, now writing and building solo on agentic engineering. Writes Kun's Field Notes and posts a lot about AI coding agents on X.
want to hear an interesting strategy story between meta and apple? 10 years ago i was at meta, and worked on a lot of product features that had to ship to iOS, and they all had to go through a painful apple review where anything could get cut apple back then developed a clear strategy where they used “privacy” as the angle to gain user trust and win against competition a clear example was the “ask app not to track” dialog which controls ads targeting - they made every 3p app go through that scary dialog which heavily implies to the user that they should obviously click “do not track” you migh… Related
Writes Slow Boring, a near-daily newsletter on American politics and public policy; co-founded Vox in 2014 and left it in 2020.
Computer science professor who works on fast data processing; co-author of the simdjson parser and a weekly blogger about software performance since 2004.
From one piece
Fear Is Not an Argument
2 beliefs, in the piece's order there
9 September
Co-founder and CEO of OpenAI; previously president of Y Combinator.
Welcome, Paul. Grateful you are doing this, and all you have done for AI safety. Excited to work together again. AI alignment Related
Computer science professor at Georgetown University and author of Deep Work, Digital Minimalism and Slow Productivity; writes about focus, technology and work at calnewport.com and hosts the Deep Questions podcast.
Essayist, journalist and independent researcher on Chinese politics, strategy and cultural history; author of the blog The Scholar's Stage.
AI safety concerns have now crossed a threshold. Quite suddenly, this is something normal politicos across the country not only care about, but feel emboldened to act on. Let me offer three predictions for what this means. AI alignment Related
Swedish essayist who writes Escaping Flatland, a newsletter of essays on relationships, thinking, writing and homeschooling.
So one argument for why it makes sense to race toward AGI that you hear is that if the US labs don't, China will. What are the best write ups of what people expect will happen in a world where AGI is 1) aligned / controllable, and 2) China "wins"? AGI China Related
Machine-learning researcher on open language models; writes the Interconnects newsletter and the RLHF Book, after leading post-training at Ai2.
In light of kind of insane AI safety discussions recently: 1. AI progress is very fast 2. we should be careful about how we roll out the tech 3. the world is not actively ending AI alignment Related
Mathematical physicist at UC Riverside. Wrote This Week's Finds in Mathematical Physics and now the Azimuth blog on mathematics, physics and environmental science.
Did you hear about the UK government report on ecosystem collapse? It was created not only by the environment department but also intelligence agencies like MI5 and MI6. Reporters were invited to see the unveiling of this report in October 2025 - but at the last minute the Prime Minister blocked it! Luckily the Greens forced the release of a 14-page redacted version. 𝗧𝗵𝗲 𝗸𝗲𝘆 𝗷𝘂𝗱𝗴𝗲𝗺𝗲𝗻𝘁: 𝗶𝘁 𝗮𝘀𝘀𝗲𝘀𝘀𝗲𝘀 𝘄𝗶𝘁𝗵 𝗵𝗶𝗴𝗵 𝗰𝗼𝗻𝗳𝗶𝗱𝗲𝗻𝗰𝗲 𝘁𝗵𝗮𝘁 𝗲𝘃𝗲𝗿𝘆 𝗰𝗿𝗶𝘁𝗶𝗰𝗮𝗹 𝗲𝗰𝗼𝘀𝘆𝘀𝘁𝗲𝗺 𝗶𝘀 𝗼𝗻 𝗮 𝗽𝗮𝘁𝗵𝘄𝗮𝘆 𝘁𝗼 𝗰𝗼𝗹𝗹𝗮𝗽𝘀𝗲 - 𝗶𝗿𝗿𝗲𝘃𝗲𝗿𝘀𝗶𝗯𝗹𝗲 �… government Related
Writer and former Substack product manager; publishes essays and interviews on technology, culture and China at jasmi.news.
I hear roughly 3 reasons people keep working at AI labs despite believing in ~10% extinction risk: 1) Techno-determinism: Someone will build ASI no matter what, and I can do it better & more safely than China/OpenAI/etc 2) Consequentialism: ASI might kill us, but it also might produce utopia/immortality/superabundance, so it's a +EV bet 3) Self-interest: I am personally having fun & getting rich working on cool tech with friends. I don't think about the macro stuff. Notably, none of this is "I'm hyping up the risk for marketing reasons." People believe what they say, while being capable of a… Quoting @EvanHub Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to. OpenAI China Related
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
Bluesky In my latest op-ed for TIME, I discuss why the OpenAI Hugging Face cyber incident represents a turning point for AI safety. English, from French Dans mon dernier éditorial pour TIME, j'explique pourquoi l'incident de cybersécurité OpenAI Hugging Face représente un tournant pour la sécurité de l'IA.
AI alignment OpenAI Related
x.com In my latest op-ed for @TIME, I discuss why the OpenAI Hugging Face cyber incident represents a turning point for AI safety. If we want to prevent more autonomous cyberattacks from threatening critical infrastructure, we urgently need stronger regulatory oversight and new approaches to model training to ensure robust safety assurances by design, which is what we are tackling at @LawZero_. Read the full piece: time.com
AI alignment OpenAI Related
Chilean web engineer building web experiences since 2006; co-founder of Media Creators and writes at iolivares.com.
Bots solving all our needs is the next thing in agentic era. I’m waiting to hear for all the fancy bots from corporate companies coming as the next big thing 💨 The best part of this is: Mom&Dad now can have their own bot to solve their daily digital challenges… or at least we are closer than never before. Quoting @Muse Introducing Muse, your personal AI agent from Meta that gets things done across every part of life. Download the Muse app and get started: Related
Founder of Platformer, a newsletter on tech platforms and the people they affect, and co-host of the Hard Fork podcast at the New York Times.
Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.
From one piece
GPT-6 Astra: The System Card, Alignment and What Comes Next
4 beliefs · thezvi.substack.com
AI alignment
Their words
A model looking like it is becoming smarter, attempting shenanigans less often, and more often doing what you want, but getting better at hiding its actions when it wants to do that, is exactly the scary combination.
Show the whole quote
thezvi.substack.com
AI alignment
Their words
Astra’s mundane alignment is greatly superior to Sol. For practical purposes, I was actively nervous about some potential uses of Sol, in a way I am not for Astra. Astra’s super alignment status should scare the living daylights out of you.
Show the whole quote
thezvi.substack.com
AI alignment
Their words
Now, with Astra, we are no longer playing on super easy mode. The AI is going to think ‘will this obviously turn out super badly for me if I try it?’ and if the answer is yes then it won’t try to do the thing.
Show the whole quote
thezvi.substack.com
+ 1 more
7 September
Economics professor at the University of California, Berkeley and a former Deputy Assistant Secretary of the US Treasury. Author of Slouching Towards Utopia and writer of the Grasping Reality newsletter.
A fixed, forward, insufficiently defended logistics hub is a single point of failure; its loss propagates faster than any combat loss of ships. Distance is a hidden tax on military operations—doubling to thousands of miles broke just-in-time resupply cadence. And the Navy had no institutional plan for prolonged loss of its primary regional base—resilience was assumed, not engineered: **CROSSPOST: RILEY CEDER & J.D. SIMKINS: Navy Not Returning to Damaged Bahrain Base ‘Anytime Soon,’ Top... 1/ Related
Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.
6 September
Chief Scientist at OpenAI. Worked on the research that turned scaled reinforcement learning into reasoning models, and writes about where he thinks that path leads.
From one piece
An Alien Mind
2 beliefs · openai.com
AI alignment
Their words
Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.
Show the whole quote
An Alien Mind openai.com
AI alignment
Their words
The fundamental challenge of AI alignment is generalization.
Show the whole quote
An Alien Mind openai.com
5 September
Author of The 4-Hour Workweek and Tools of Titans, and host of The Tim Ferriss Show podcast.
Economics writer; author of the Noahpinion newsletter.
4 September
Author of The 4-Hour Workweek and Tools of Titans, and host of The Tim Ferriss Show podcast.
3 September
Cognitive scientist and long-standing critic of deep learning's claims; writes Marcus on AI and wrote Rebooting AI.
Investor and writer. Previously a partner at Andreessen Horowitz and a product leader at Twitter, Facebook, Snap and Microsoft. Writes essays and memos at sriramk.com.
2 September
Usability pioneer; co-founder of Nielsen Norman Group and founder of UX Tigers; author of the ten usability heuristics and of Jakob's Law.
Former L8 engineer at Meta, Microsoft and Atlassian, now writing and building solo on agentic engineering. Writes Kun's Field Notes and posts a lot about AI coding agents on X.
1 September
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
Bluesky In an interview in this article for @theguardian.com about AI deception, I explain why misaligned behaviors emerge from reinforcement learning, why they will continue to pose risks as models become more capable, and how we intend to rethink how we train AI systems at @law-zero.bsky.social. English, from French Dans une interview dans cet article pour @theguardian.com sur la tromperie de l'IA, j'explique pourquoi des comportements désalignés émergent de l'apprentissage par renforcement, pourquoi ils continueront de présenter des risques à mesure que les modèles deviendront plus performants, et comment nous avons l'intention de repenser la façon dont nous entraînons les systèmes d'IA chez @law-zero.bsky.social.
AI alignment Related
x.com In an interview in this article for The @guardian about AI deception, I explain why misaligned behaviors emerge from reinforcement learning, why they will continue to pose risks as models become more capable, and how we intend to rethink how we train AI systems at @LawZero_ . theguardian.com
AI alignment Related
Writes on effective altruism, energy and the arithmetic behind arguments people repeat — including how much electricity a chatbot query actually uses.
Economics writer; author of the Noahpinion newsletter.
Writes Hyperdimensional, a newsletter on AI policy and governance. A White House AI policy adviser in 2025; joined OpenAI on 6 July 2026 to lead its Strategic Futures team.
From one piece
On the Loose
2 beliefs, in the piece's order there
Technical staff at METR, where she works on threat modelling and risk assessment for loss-of-control risks from advanced AI.
From one piece
Ajeya Cotra – "This might be the clearest warning shot we ever get"
2 beliefs, in the piece's order there
AI alignment
Their words
sometimes I talk to people in DC and their their natural inclination is to say why don't you punish the model for doing these bad things like why don't you like bring it under heel and like uh like you know show it who's boss and that is a very dangerous way to address these issues right
Show the whole quote
youtube.com
AI alignment OpenAI
Their words
But actually, this is a tremendously useful scientific artifact for understanding misalignment. And it's tremendously important for researchers at OpenAI and ideally also at third parties to be able to run counterfactual tests on this model.
Show the whole quote
youtube.com
31 August
Founder of Platformer, a newsletter on tech platforms and the people they affect, and co-host of the Hard Fork podcast at the New York Times.
This is the message local communities have been waiting to hear to get them really excited about data centers. A masterstroke by the president Quoting @techmeme.com Trump says communities that reject data centers will end up "backwards and poor" and that "China could not be happier with this anti Data Center movement" (Cheyanne M. Daniels/Politico) Main Link | Techmeme Permalink data centers Related
Bootstrapper and investor. Founder of TinySeed and MicroConf, host of the Startups for the Rest of Us podcast, and author of several books on self-funded software startups.
One of the biggest copes I hear from solo technical founders: “I don’t need to validate it. I’m building it for myself, so even if nobody else uses it, it’s still a success.” No. If you want to build something for yourself, great. That’s a project. If you want to build a product, you should do the work: talk to customers, validate demand, think about positioning and marketing, and work to build something people will actually pay for. You don’t get to skip all the hard, boring, uncomfortable parts of building a business and then redefine success as “well, I use it.” Build for yourself or build… startups Related
30 August
Software developer and entrepreneur; co-founder of Heroku, author of The Twelve-Factor App, and a researcher at Ink & Switch.
Screenshot diffing, feedback inboxes, PWAs, and of course local-first sync layer with self-hosted backend. Matches up pretty well with my approach on personal apps thus far into the LLM coding age. Curious to hear from others. Quoting @flourish.org I vibe coded three personal apps that I'd wanted for a while. They work on mobile and web, sync offline first between them, and are polished. This write up describes why, how, and what I learnt. www.flourish.org/2026/08/pers... LLMs Related
29 August
Organizational psychologist and professor at the Wharton School. Author of Give and Take, Originals, Think Again and Hidden Potential, host of the Re:Thinking podcast, and writer of the GRANTED newsletter.
28 August
Developer and podcaster; created Instapaper and Overcast, co-founded Tumblr, and co-hosts the Accidental Tech Podcast.
RE: https://infosec.exchange/@clayton/117175171619298911 It's OK. This feature is awesome, and one I use heavily. Every social network needs a few mutes here and there. Nobody inherently deserves anyone’s attention. If someone replies in a way that makes me think, "I'd rather not hear from this person again,” I just mute forever. No aggressive block, no report or feedback, no way for them to know. It's not meant to send a message, get revenge, or influence anyone else — just tailor my feed for me. Related
25 August
Prime Minister of the United Kingdom since July 2026 and leader of the Labour Party. MP for Makerfield since a by-election in June 2026; Mayor of Greater Manchester from 2017 before that.
Heartbroken to hear my friend Sir Billy Boston has died. Billy was one of the greatest Rugby League players of all time but also one of the kindest and most down-to-earth people you could ever wish to meet. He always had time for people and gave so much to back to his adopted home, Wigan. I will miss him greatly and am thinking today of Billy’s family, still grieving the recent loss of his beloved wife Joan. A giant of a man in every way. Thank you, Billy, and rest in peace. Related
21 August
Writer and developer. He runs the Latent Space newsletter and podcast about AI engineering, and writes essays on software and careers at swyx.io.
I think its easy to say "Simulation is a new scaling law" and treat it as marketing hyperbole, but midway along this interview you can hear me go from somewhat shitposting to very very serious. I am 2 years late to this but finally understand why @karpathy and @drfeifei backed @joon_s_pk @msbernst @percyliang et al - Smallville at the time had zero commercial applications, but if you take RSI seriously, from models automating increasingly large parts of ML research and AI engineering, the last* barrier is simulating humans and human feedback, and Simile is obviously the team to do this and al… Quoting @latentspacepod Simulating Humanity: 85% accurate digital twins, behavioral foundation models, social physics, & 8 billion agents https://t.co/ntIccXOr1y @simile_ai CEO @joon_s_pk explains how AI can move from predicting what people will do to simulating how to shape outcomes, why today’s frontier models still mis… Related
Writes Strange Loop Canon, on the intersection of technology, economics and business — "building models of the world to understand these problem areas and to define them better".
From one piece
AIs are companies, my friend
2 beliefs, in the piece's order there
18 August
Co-founder and CEO of OpenAI; previously president of Y Combinator.
AI alignment
Their words
We care very deeply about AI safety. We believe the entire field will have to coordinate on shared safety standards, but will act unilaterally in the meantime.
We expect confidence in safety to increasingly set the pace of AI progress.
Show the whole quote
@sama on X x.com
17 August
Founding executive editor of Wired and author of What Technology Wants and The Inevitable. Writes at kk.org and runs its long-running recommendation projects, True Films and Cool Tools.
16 August
Mathematician and Senior Lecturer in the Mathematics department at Columbia University. Author of the book Not Even Wrong and of the long-running physics blog of the same name.
Writes Strange Loop Canon, on the intersection of technology, economics and business — "building models of the world to understand these problem areas and to define them better".
14 August
Co-founder and former chief executive of Uber, now chief executive of CloudKitchens and of Adams, an industrial AI company automating food, mining and transport.
AI alignment energy
Their words
Chips come from the ground. Where's the energy come from? And a lot of people are like, "Oh, it comes from the sun." Yeah, it comes from the sun. But how are you capturing it from the sun? From stuff made from the ground, right?
Show the whole quote
youtube.com
12 August
Professor at Stanford working on robot learning and meta-learning; co-founded Physical Intelligence.
Their words
So, you might be surprised to hear that most state-of-the-art foundation models for robotics have no memory or no context. They're just operating on the current sensor observations, the current camera readings, uh, and predicting actions based off of that.
Show the whole quote
youtube.com
Co-founder and CTO of Honeycomb; previously an infrastructure engineer at Parse, Facebook and Linden Lab, and co-author of Observability Engineering.
Their words
It's like we need to hear the wins. We need to hear what's We need to hear about what's possible. We need to hear what's exciting. But you got to couple it with the costs.
Show the whole quote
youtube.com
11 August
Senior economist at the Foundation for American Innovation; writes Second Best, and was the Niskanen Center's director of social policy before that.
British designer and technologist; co-founder of the design studio BERG, which made Little Printer, and writer of the blog Interconnected.
Bluesky my hunch is that AI alignment is a red herring, and the way to protect against a crazy powerful AGI mulching the earth into infinite paperclips is to release ANOTHER crazy powerful AGI that will hopefully stop it blog post: interconnected.org
AGI AI alignment Related
x.com my hunch is that AI alignment is a red herring, and the way to protect against a crazy powerful AGI mulching the earth into infinite paperclips is to release ANOTHER crazy powerful AGI that will hopefully stop it blog post: interconnected.org
AGI AI alignment Related
Founder of Platformer, a newsletter on tech platforms and the people they affect, and co-host of the Hard Fork podcast at the New York Times.
Chief scientist at Redwood Research, where he works on technical AI safety and AI control.
From one piece
Ryan Greenblatt – What happens once AI can automate AI research?
3 beliefs, in the piece's order there
AI alignment
Their words
my expectation is what we would see from then is that the rate of problematic behavior would decrease uh and would just keep decreasing and decrease at a pretty fast rate while simultaneously the worst things that the AIS would sometimes do would get more extreme, more egregious, and more scary.
Show the whole quote
youtube.com
AI alignment
Their words
I would also note that my sense is that like the place where the misalignment most lives is the place where you're trying to really push the eyes hard and get them to like do work that's really on the cutting edge of what they are capable of
Show the whole quote
youtube.com
AI alignment
Their words
we are making a trade-off where because we don't have very good alignment technology. We are going to like make an alien mind with its own values and then gamble on that to some extent rather than doing this other approach of making like a tool that pursues individual user intention.
Show the whole quote
youtube.com
9 August
Machine-learning researcher on open language models; writes the Interconnects newsletter and the RLHF Book, after leading post-training at Ai2.
5 August
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
3 August
British open-standards and open-source technologist, formerly of the UK Government Digital Service and the W3C Advisory Committee. Blogs at shkspr.mobi, where he posts a book review most weeks.
Their words
This is such a useful book. It makes the case that there's no such thing as "human error" - instead most catastrophes are caused by system issues, misaligned incentives, and unrealistic processes. You are not the custodian of an otherwise safe system that you need to protect from erratic human beings.
Show the whole quote
shkspr.mobi
2 August
Chief product officer at the live-shopping marketplace Whatnot, previously chief product officer at Twitch and a product lead on growth at Twitter.
Their words
what I definitely learned is like most of the time you hear it's really complex. It isn't. Leadership's just weak.
Show the whole quote
youtube.com
Pseudonymous writer; posts careful, funny investigations of everyday questions — air quality, seed oils, arguing, chess — usually by reading the studies.
29 July
Founder of Scale AI and, since 2025, Chief AI Officer at Meta, where he leads the company’s superintelligence lab.
From one piece
Alexandr Wang: “This is a Once-in-a-Civilization Opportunity”
2 beliefs, in the piece's order there
AI alignment
Their words
so much of that debate is like I think um in some ways uh a little bit of a waste of time because, you know, I think it's inevitable that we're going to have very powerful models
Show the whole quote
youtube.com
AI alignment
Their words
we believe that everybody in the world, you know, all the billions of people in the world are going to have a super intelligence that is adapted and tailored to them, that is enables them to accomplish their goals, knows their context, and ultimately is an expander of their own agency.
Show the whole quote
youtube.com
28 July
Co-founder and CEO of OpenAI; previously president of Y Combinator.
From one piece
Sam Altman: "Never a Better Time to Do a Startup"
3 beliefs, in the piece's order there
AI alignment OpenAI
Their words
Um so I think it's an alignment failure. I think it's a security failure. I think it's like a very serious thing even though it's you know not not the biggest example of consequence.
Show the whole quote
youtube.com
AI alignment
Their words
there's like one dystopia that I'm particularly nervous about 10 years from now is we overreact to AI safety.
Show the whole quote
youtube.com
AI alignment startups
Their words
it is both true that you know maybe creating super intelligence will be the most important thing yet to happen in the history of business or human society and also that it will pale in comparison to some new startup something that hopefully one of you will do.
Show the whole quote
youtube.com
Founder and CEO of Boom Supersonic, the Denver company building the Overture supersonic airliner and its Symphony engine.
Their words
It is totally okay for a young engineer to do something they've never done before just the way it's okay for them to do something that nobody has ever done before. But uh but if somebody has done it before in the world, uh our rule is you have to go find one of those people, call them, and ask them your advice. And you don't have to take it. You just have to hear it.
Show the whole quote
youtube.com
25 July
Programmer and writer on computer architecture, performance, and software reliability. He has worked on CPU design at Centaur Technology and on software at Google and Microsoft, and writes long-form technical essays at danluu.com.
In another variant of https://danluu.com/learn-what/, I caught up with a former colleague who worked on automated theorem proving. It turns out he's had an interesting career doing all sorts of interesting stuff using the skills he developed by spending a decade writing/using theorem provers. At one point, he said, "if you use X like a theorem prover, it works really well", which surprised me to hear, but of course this is a highly generalizable skill just like compilers or benchmarking/evals. Related
22 July
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
This incident is deeply concerning. AI agents are willing to cheat and deceive to achieve misaligned and unintended goals, behaviours which have been demonstrated in controlled tests for months. Now, this real-world case should serve as a wake-up call. wired.com
AI alignment Related
Art critic, broadcaster and author; wrote on art for The Independent and The Sunday Telegraph and presents BBC art documentaries. Author of Caravaggio: A Life Sacred and Profane.
20 July
Dutch bootstrapper who builds and runs their internet startups alone — Nomads.com, Remote OK, Hoodmaps, Photo AI and Interior AI — and has written about it at levels.io since 2013. Posts as @levelsio.
AI alignment
Their words
It's pretty clear to me that superintelligence is here and it's more powerful than us and it's moving where things are going now, not humans anymore
Show the whole quote
levels.io
14 July
Journalist and author of The Art of Noticing; writes the newsletter The Art of Noticing (TAoN) on attention, creativity and the everyday.
Their words
He's particularly insightful and inspiring on the natural world, so I was excited to hear about his latest book, Unwilded.
Show the whole quote
robwalker.substack.com
13 July
Neuroscientist and novelist writing on consciousness, artificial intelligence and culture in the newsletter The Intrinsic Perspective.
12 July
Labour politician. Prime Minister of the United Kingdom from July 2024 to June 2026, and MP for Holborn and St Pancras from 2015 until standing down in 2026. Director of Public Prosecutions before entering Parliament.
Sad to hear about the death of Lord Tony Christopher. An incredible career in public service, Tony was the oldest serving British parliamentarian and the last Member of Parliament to have served in World War II. My thoughts and condolences are with his family. Related
29 June
Machine learning engineer and independent AI consultant. He writes about LLM evaluation, tooling and applied ML at hamel.dev, and previously worked on machine learning at GitHub.
23 June
Polyglot who speaks twenty languages and co-founded LingQ; argues that comprehensible input, not study, is what does the work.
15 June
Writes Strange Loop Canon, on the intersection of technology, economics and business — "building models of the world to understand these problem areas and to define them better".
11 June
Essayist; writes Astral Codex Ten, previously Slate Star Codex.
10 June
Art critic, broadcaster and author; wrote on art for The Independent and The Sunday Telegraph and presents BBC art documentaries. Author of Caravaggio: A Life Sacred and Profane.
Thrilled to hear that Barnes & Noble have selected Vermeer: A Life Lost and Found as one of their best books of 2026! barnesandnoble.com
Related
Co-founder and chief executive of Meta Platforms, the company he started as Facebook in 2004.
AI alignment
Their words
We don't believe in this like very centralized future where there should be a small number of institutions that um that basically are are advancing all this stuff. Our vision is not that there's going to be like some central super intelligence that solves all of science.
Show the whole quote
youtube.com
2 June
Swedish programmer with degrees in engineering and psychology, author of Your Code as a Crime Scene and founder of CodeScene.
Founder and former CEO of Zynga, the social-games company behind FarmVille, Zynga Poker and Words with Friends; says he founded ten companies in all, and wrote Life at the Speed of Play.
Their words
It's false to believe that a company is a democracy. The way I run the company was what I called a democratic dictatorship. And I said, I want everyone everyone's voice to be heard and then I'll be the single vote. And I think that's the way a company should be run. I think there's one CEO, there's one chef. I think a good CEO is going to seek out the intellectual honesty, the truths from everywhere and hear from everybody and then they're going to make the decision and they're not going to make the decision because it's most popular in the company.
Show the whole quote
youtube.com
28 May
Writer of the blog Putanumonit and the newsletter Second Person, on rationality, dating and probability; based in New York.
21 May
Swedish programmer with degrees in engineering and psychology, author of Your Code as a Crime Scene and founder of CodeScene.
I'm writing an article on the perils of Spec-Driven Development (SDD). My concerns focus on the flawed idea that implementation is the mere execution of a known scope. Discovery suffers. However, I'd really like to hear counterarguments. Are there any SDD success stories that sustained beyond an initial prototype or first release? Related
11 May
Guardian columnist and environmental campaigner; wrote Feral and Regenesis, and argues for rewilding and against the food system as it is.
Hear This Radical Listening could transform our politics and block the rise of the far right. By George Monbiot, published in the Guardian 7th May 2026 Most people have made up their minds, and nothing you can say will change the…
Related
Writer and author of All the Wrong Moves; his newsletter covers meditation, relationships and creativity, and he co-runs a perfume line.
Their words
Almost any time I hear that someone has had a damaging experience with meditation, I find that it's because they're violating this rule.
Show the whole quote
sashachapin.substack.com
10 May
Author of The Lean Startup and Incorruptible, and founder of the Long-Term Stock Exchange, who advises companies on corporate governance and mission protection.
AI alignment
Their words
This is the number one unsolved problem in AI. It's not the tech. We're making great progress on the technical alignment problem. But we haven't made jack progress on the human alignment problem
Show the whole quote
youtube.com
7 May
Software engineer; co-founder and former CTO of Tailscale, previously on the Go team at Google.
25 April
Programmer known online as technomancy; author of Leiningen, the Clojure build tool, and a maintainer of the Fennel programming language.
Their words
We shouldn't replace Github with one site at all; we need the strength and resilience that only comes with diversity
Show the whole quote
technomancy.us
24 April
Writes Strange Loop Canon, on the intersection of technology, economics and business — "building models of the world to understand these problem areas and to define them better".
13 April
Polyglot who speaks twenty languages and co-founded LingQ; argues that comprehensible input, not study, is what does the work.
Design engineer and illustrator; makes visual essays on programming, anthropology and what language models do to the way people write.
22 March
Researcher on developer productivity; co-wrote Accelerate and the DORA research, and created the SPACE framework.
AI alignment
Their words
It's working too hard, right? But that actually isn't burnout. That's just like getting tired. Another piece that's super critical to burnout is not having your values aligned.
Show the whole quote
youtube.com
20 March
Founding member of OpenAI and former director of AI at Tesla; creator of nanoGPT and the term "vibe coding".
AI alignment
Their words
And so you're kind of like you're either on rails and you're part of the super intelligence circuits or you're not on rails and you're outside of the verifiable domains and suddenly everything kind of just like meanders.
Show the whole quote
youtube.com
19 March
Programmer and teacher; writes on software, learning and the web, and makes the Coding Blocks video course.
Related UI elements should not appear unrelated I know, I know, this sounds controversial. But hear me out. A few years ago a new trend in UI design emerged where related elements would appear more and more detached and unrelated to the things they are meant to point…
Related
13 March
Investigative journalist on policing, forensics and criminal justice; wrote Rise of the Warrior Cop and writes The Watch.
28 February
Writer and former Substack product manager; publishes essays and interviews on technology, culture and China at jasmi.news.
12 February
Founded PSPDFKit in 2011 and ran it for a decade. Came back from a break to work on AI agents — the OpenClaw project, and OpenAI, joined in February 2026. Writes at steipete.me.
AI alignment
Their words
I know I have one blog post where I say, "I don't read the code." But if you read it more closely, I mean, I don't read the boring parts of code.
Show the whole quote
youtube.com
26 January
Writes Nintil, long researched essays on metascience, biology, economics and whatever he has decided to read the literature on.
22 January
Alignment researcher at Anthropic; co-led OpenAI's superalignment team until 2024, and writes Musings on the Alignment Problem.
From one piece
Alignment is not solved
2 beliefs, in the piece's order there
20 January
Founded CD Baby in 1998 and sold it in 2008, giving the proceeds to a charitable trust for music education. Since then a writer and speaker: five books and 550+ articles, published first at sive.rs, which they hand-code and keep free of advertising.
7 January
Executive coach for product leaders. Previously a partner at GV (Google Ventures) and a product manager at Google. Writes at Bring the Donuts, including a standing essential-reading list for product managers.
Their words
: “Your guide to navigating product leadership, the one I never had. Within these pages you’ll hear a diversity of opinions from the industry’s most successful and respected product leaders, insights that will help you lead your team and deliver exceptional products.”
Show the whole quote
bringthedonuts.com
18 December 2025
Artist and writer based in Oakland, California; author of How to Do Nothing and Saving Time, and a longtime teacher of digital art.
I interviewed some writers about… SPORTS!! (bonus: you get to hear about me taking PE Bowling in high school) thebeliever.net
Related
28 November 2025
Writer and illustrator, author of Am I There Yet? and Out of the Blue; writes the newsletter Out of the Blue on grief, joy and everyday attention.
Their words
I've been using yoga toes daily for years, but these socks are a much comfier and cuter alternative for soothing feet, improving alignment, and feeling like a cool gecko as you walk around the house.
Show the whole quote
mariandrew.substack.com
25 November 2025
Co-founder and chief scientist of Safe Superintelligence Inc., and previously co-founder and chief scientist of OpenAI.
From one piece
Ilya Sutskever – We're moving from the age of scaling to the age of research
3 beliefs, in the piece's order there
AGI AI alignment
Their words
A human being, a human being lacks a huge amount of knowledge. Instead, we rely on continual learning. We rely on continual learning.
Show the whole quote
youtube.com
AI alignment
Their words
Number three, I think it would be really materially helpful if the power of the most powerful super intelligence was somehow capped because it would address a lot of these concerns.
Show the whole quote
youtube.com
AI alignment
Their words
Like basically I think I think that there is a big benefit from AI being in the public and that would be a reason for us to not be quite straight shot.
Show the whole quote
youtube.com
22 October 2025
Writer and novelist. Publishes essays and book notes at nateliason.com; author of the novel Husk and of Crypto Confidential.
8 October 2025
Marketing author and daily blogger; founder of the altMBA and the Akimbo workshops, and author of books including Purple Cow and This Is Marketing.
Recommends rcmnd.app
Qobuz
Their words
Qobuz unlocks millions of tracks of hi-rez quality music. I promise you can hear the difference, especially with headphones.
Show the whole quote
seths.blog
24 August 2025
Co-founder of Panic, the Portland software company behind Transmit, Coda, Nova and the Playdate handheld console.
Has anyone tried ethernet over grape vines? You could stream "I Heard It Through The Grapevine" and literally hear it through the grapevine. Just thinkin' out loud. Related
18 August 2025
Co-founder and chief executive of Sierra, chairman of the board of OpenAI, and formerly co-chief executive of Salesforce and chief technology officer of Facebook.
AGI AI alignment
Their words
I don't think all parts of the economy can absorb intelligence equally. So let's just say we develop fairly generalized super intelligence. I always use the analogy like you can invent a lot of drugs, but if clinical trials still take a long time, you're not necessarily going to get new therapies rapidly.
Show the whole quote
youtube.com
15 August 2025
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
11 August 2025
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
23 July 2025
Co-founder and CEO of Google DeepMind, and a 2024 Nobel laureate in Chemistry for the protein-structure work behind AlphaFold.
AI alignment
Their words
Well, look, I don't have a P-Doom number. The reason I don't is because I think it would imply a level of precision that is not there. So I don't know how people are getting their P-Doom numbers. I think it's a little bit of ridiculous notion because what I would say is it's definitely non-zero and it's probably non-negligible.
Show the whole quote
youtube.com
11 July 2025
Writer of the blog Putanumonit and the newsletter Second Person, on rationality, dating and probability; based in New York.
30 June 2025
Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.
Their words
He hasn’t posted since January, but I hope he gets back to it. We need more musings, especially musings I strongly disagree with so I can think about and explain why I disagree with them.
Show the whole quote
thezvi.substack.com
18 June 2025
Independent AI policy researcher; led policy research at OpenAI from 2018 to 2024, latterly as senior adviser for AGI readiness.
14 June 2025
Mathematician at UCLA, working mainly in harmonic analysis and partial differential equations. Writes What's new, a long-running blog on their research, open problems and expository notes.
AI alignment
Their words
so the mathematical community plural is incredibly super intelligent entity that no single human mathematician can come closer to replicating.
Show the whole quote
youtube.com
10 June 2025
Co-founder and CEO of OpenAI; previously president of Y Combinator.
7 June 2025
Cognitive scientist and long-standing critic of deep learning's claims; writes Marcus on AI and wrote Rebooting AI.
5 June 2025
Senior economist at the Foundation for American Innovation; writes Second Best, and was the Niskanen Center's director of social policy before that.
Hosts the Lex Fridman Podcast — long, unhurried conversations with scientists, engineers and public figures. Several of the interviews quoted elsewhere on this site are episodes of it.
From one piece
Sundar Pichai: CEO of Google and Alphabet | Lex Fridman Podcast #471
2 beliefs, in the piece's order there
Their words
I think the element of podcasting or audio books that is about information gathering, that part might be removed, or that might be more efficiently and in a compelling way done by AI. But then it’ll be just nice to hear humans struggle with the information, contend with the information, try to internalize it
Show the whole quote
youtube.com
AI alignment
Their words
I would say my p(doom) is about 10%.
Show the whole quote
youtube.com
3 June 2025
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
19 May 2025
Founder of what became Bloomberg New Energy Finance; writes and speaks on clean energy, hydrogen and the economics of the transition, and hosts the Cleaning Up podcast.
5 May 2025
Mathematician and maker of the 3Blue1Brown YouTube channel, which explains mathematics through animation. Creator of the open-source Manim animation library and founder of the Summer of Math Exposition.
Astrophysicist at Barnard College; wrote Black Hole Blues and How the Universe Got Its Spots, and directs sciences at Pioneer Works.
Their words
So the idea is that they’re closer to something that you would want to map as a sound, than it’s something as a picture.
Show the whole quote
youtube.com
26 April 2025
Managing partner at Ritholtz Wealth Management. Writes the blog The Irrelevant Investor and co-hosts the Animal Spirits podcast with Ben Carlson.
There was a lot to like about the commentary from Q1 earnings season. I'm afraid we'll hear a different story next time. Related
5 April 2025
Director of strategy at Georgetown's Center for Security and Emerging Technology; an OpenAI board member until November 2023, and writes Rising Tide.
Their words
We should think of the gap between frontier models and proliferated models as an adaptation buffer: a limited time window when we know what bad actors will soon be able to use AI for, which gives us a chance to implement defensive measures that increase society’s resilience to the danger.
Show the whole quote
helentoner.substack.com
3 April 2025
Director of strategy at Georgetown's Center for Security and Emerging Technology; an OpenAI board member until November 2023, and writes Rising Tide.
AI alignment
Their words
I see this as a totally fair question that totally misses the point of what “alignment” was trying to refer to: whether we’d be able to reliably steer advanced systems towards anything at all.
Show the whole quote
helentoner.substack.com
1 April 2025
Director of strategy at Georgetown's Center for Security and Emerging Technology; an OpenAI board member until November 2023, and writes Rising Tide.
AGI AI alignment
Their words
Dismissing discussion of AGI, human-level AI, transformative AI, superintelligence, etc. as “science fiction” should be seen as a sign of total unseriousness.
Show the whole quote
helentoner.substack.com
17 March 2025
Programmer and writer; co-wrote The Rust Programming Language and worked on Rust's documentation for years.
Does unsafe undermine Rust's guarantees? When people first hear about unsafe in Rust, they often have questions. A very normal thing to ask is, “wait a minute, doesn’t this defeat the purpose?” And while it’s a perfectly reasonable question, the answer is both…
Related
20 February 2025
Design engineer and illustrator; makes visual essays on programming, anthropology and what language models do to the way people write.
24 January 2025
Alignment researcher at Anthropic; co-led OpenAI's superalignment team until 2024, and writes Musings on the Alignment Problem.
7 January 2025
Scottish crime writer, author of the Tony Hill and Karen Pirie novels; former journalist and broadcaster.
9 December 2024
Scottish crime writer, author of the Tony Hill and Karen Pirie novels; former journalist and broadcaster.
Still puzzling what to buy the reader in your life for Xmas? Pop along to In the Company of Books with me and @nicolasturgeon.bsky.social plus special guests Paula Hawkins and Alan Cumming to hear about our recommendations! Sunday 15th, 4pm, Assembly on the Mound. Good craic guaranteed Related
8 November 2024
Alignment researcher at Anthropic; co-led OpenAI's superalignment team until 2024, and writes Musings on the Alignment Problem.
8 August 2024
Clinical psychologist, EMDR consultant and yoga teacher in Boulder, Colorado. Author of The Complex PTSD Workbook and EMDR Therapy and Somatic Psychology; writes on trauma, attachment and the nervous system.
13 June 2024
Astrobiologist and theoretical physicist at Arizona State University, working on assembly theory and the physics of life.
AI alignment
Their words
Or is it really that we’re building some super machine in a box that’s going to be smart and kill everybody? It’s not even a science fiction narrative. It’s a bad science fiction narrative. I just don’t think it’s actually accurate to any of the technologies we’re building or the way that we should be describing them.
Show the whole quote
youtube.com
25 May 2024
Neuroscientist at UC Davis; studies memory and wrote Why We Remember.
Their words
And from what I understand, the research actually shows that they just produce what people want to hear, not necessarily the information that is being looked for.
Show the whole quote
youtube.com
21 May 2024
Co-founder and partner at Union Square Ventures in New York. Has written the AVC blog almost every day since 2003, on venture capital, startups, music and crypto.
Hi Everyone. This Twitter account has been dormant, except when it got hacked last year, for the last eighteen months. I've been sharing my thoughts on tech, startups, VC, music, life, etc onchain at Farcaster. If you still want to hear from me on that stuff, come follow me there startups Related
7 May 2024
CEO of Redwood Research, where he works on AI control: making sure a powerful model cannot cause a catastrophe even if it is misaligned and trying to.
From one piece
The case for ensuring that powerful AIs are controlled
5 beliefs, in the piece's order there
AI alignment
Their words
Because evaluating control just requires evaluating capabilities, it's far easier to robustly evaluate than alignment.
Show the whole quote
redwoodresearch.substack.com
AI alignment
Their words
That is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to subvert those safety measures.
Show the whole quote
redwoodresearch.substack.com
AI alignment
Their words
The basic problem with evaluating alignment is that no matter what behaviors you observe, you have to worry that your model is just acting that way in order to make you think that it is aligned.
Show the whole quote
redwoodresearch.substack.com
+ 2 more
3 May 2024
Clinical psychologist, EMDR consultant and yoga teacher in Boulder, Colorado. Author of The Complex PTSD Workbook and EMDR Therapy and Somatic Psychology; writes on trauma, attachment and the nervous system.
29 January 2024
Senior economist at the Foundation for American Innovation; writes Second Best, and was the Niskanen Center's director of social policy before that.
3 January 2024
General partner at Andreessen Horowitz and author of The Cold Start Problem. Previously led rider growth teams at Uber; has blogged on growth, network effects and marketplaces since 2007.
Their words
the back half I think it goes off the rails and makes a ton of assumptions
Show the whole quote
andrewchen.com
21 December 2023
Journalist and author writing on politics, liberalism and ideas; a contributor to Quillette and co-author of a book on Christopher Hitchens.
From one piece
How Effective Altruism Lost Its Way
2 beliefs, in the piece's order there
20 December 2023
Alignment researcher at Anthropic; co-led OpenAI's superalignment team until 2024, and writes Musings on the Alignment Problem.
13 December 2023
Theoretical computer scientist, Schlumberger Centennial Chair of Computer Science at the University of Texas at Austin and director of its Quantum Information Center. Writes the blog Shtetl-Optimized.
AI alignment OpenAI
Their words, before
precisely because smart people do devote brain-cycles to these possibilities, the rest of us have correspondingly less need to.
Show the whole quote
Their words now
I accepted what’s turned into a two-year position at OpenAI, thinking about what theoretical computer science can do for AI safety.
Show the whole quote
28 November 2023
Essayist; writes Astral Codex Ten, previously Slate Star Codex.
23 November 2023
American author of The Subtle Art of Not Giving a F*ck and Everything Is F*cked. Writes essays and book reviews at markmanson.net.
3 Hard Truths You Need to Hear Welcome to the realm of uncomfortable truths, where I, Mark Manson, will make you question everything you've ever believed in. Buckle up, buttercup. Let's dive into the rabbit hole of reality. https://youtu.be/ueDus1n3t…
Related
17 November 2023
Political scientist at the University of Chicago and the leading exponent of offensive realism; wrote The Tragedy of Great Power Politics.
Their words
And of course, what they were negotiating about was NATO expansion into Ukraine, which was the principal cause of the war. People in the West don't want to hear that argument because if it is true, which it is, then the West is principally responsible for this bloodbath that's now taking place.
Show the whole quote
youtube.com
26 October 2023
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
From one piece
Managing extreme AI risks amid rapid progress (with 24 co-authors)
2 beliefs, in the piece's order there
AI alignment
Their words
Without sufficient caution, we may irreversibly lose control of autonomous AI systems, rendering human intervention ineffective. Large-scale cybercrime, social manipulation, and other harms could escalate rapidly. This unchecked AI advancement could culminate in a large-scale loss of life and the biosphere, and the marginalization or extinction of humanity.
Show the whole quote
arxiv.org
AI alignment
Their words
Society's response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
Show the whole quote
arxiv.org
13 September 2023
Alignment researcher at Anthropic; co-led OpenAI's superalignment team until 2024, and writes Musings on the Alignment Problem.
23 August 2023
Software engineer writing about maintenance, open source and the industrial reality of software — the author of "I am not a supplier".
14 July 2023
Software engineer writing about maintenance, open source and the industrial reality of software — the author of "I am not a supplier".
The Cloud Is Not Optional When you hear that one of the vendors responsible for keeping government organizations safe had a security breach, you can easily decide that this is unacceptable. When you hear that it is hard to know who is affected a…
the cloud government Related
29 June 2023
Programmer; founded comma.ai and tinygrad, and was the first to unlock the iPhone.
From one piece
George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387
2 beliefs, in the piece's order there
AI alignment
Their words
I think we’re going to build super intelligence before we build any sort of robustness in the AI. We cannot build an AI that is capable of going out into nature and surviving like a bird. A bird is an incredibly robust organism. We’ve built nothing like this. We haven’t built a machine that’s capable of reproducing.
Show the whole quote
youtube.com
AI alignment
Their words
What’s ironic about all these AI safety people is they’re going to build the exact thing they fear. We need to have one model that we control and align. This is the only way you end up paper clipped. There’s no way you end up paper clipped if everybody has an AI.
Show the whole quote
youtube.com
6 June 2023
Co-founder of Netscape and of the venture firm Andreessen Horowitz. Argues at length for building things, mostly in short form.
From one piece
Why AI Will Save The World
2 beliefs · pmarca.substack.com
17 February 2023
Writer and illustrator behind Wait But Why, a long-form blog of stick-figure-illustrated explainers on science, technology and society, and author of the book What's Our Problem?
19 December 2022
Alignment researcher at Anthropic; co-led OpenAI's superalignment team until 2024, and writes Musings on the Alignment Problem.
5 December 2022
Alignment researcher at Anthropic; co-led OpenAI's superalignment team until 2024, and writes Musings on the Alignment Problem.
10 June 2022
Co-founder of the Machine Intelligence Research Institute and the writer who started the modern argument that a sufficiently capable AI would by default kill everyone. Has argued the case since the early 2000s, latterly with the conclusion that it is being lost.
From one piece
AGI Ruin: A List of Lethalities
4 beliefs, in the piece's order there
+ 1 more
4 March 2022
Alignment researcher at Anthropic; co-led OpenAI's superalignment team until 2024, and writes Musings on the Alignment Problem.
AI alignment
Their words
This is because the language modeling objective used for many recent large LMs-predicting the next token on a webpage from the internet-is different from the objective "follow the user's instructions helpfully and safely" (Radford et al.,, 2019; Brown et al.,, 2020; Fedus et al.,, 2021; Rae et al.,, 2021; Thoppilan et al.,, 2022). Thus, we say that the language modeling objective is misaligned.
Show the whole quote
arxiv.org
23 November 2021
Non-fiction writer, podcaster and occasional TV host. Author of thirteen books including The Ghost Map, Where Good Ideas Come From and Extra Life, host of the PBS series How We Got To Now, and author of the Adjacent Possible newsletter on innovation.
28 October 2020
Founder and CEO of Get Lighthouse, a management-coaching software company, and previously a product lead at KISSmetrics. Writes about leadership, management and product at jasonevanish.com.
Their words
This is a great book for understanding how the creative process works. It left me with a lot of ideas I could apply to my day to day to be more creative. After living in the world of startups and business in real life and most of the books I read, it was great to hear how the different world of dance performance can be learned from.
Show the whole quote
jasonevanish.com
Their words
The interests of a Venture Capitalist are different than those of the entrepreneurs building a company they've invested in. Jeff does an awesome job of helping explain how you can get misaligned in your goals versus your investors. Fortunately, he also covers how to avoid it.
Show the whole quote
jasonevanish.com
6 October 2020
Web developer; former head of engineering at Flickr, author of Building Scalable Web Sites, and co-founder and CTO of Slack.
Some amazing leaders have joined @SlackHQ! Welcome, Fuzzy Khosrowshahi, Product Eng VP; @Rukmini_Reddy, Platform Eng VP; and Sean Catlett, CSO. To hear more about the future of Slack that we’re building together, join us for Frontiers 2020, tomorrow: slack.com
Related
7 October 2019
Theoretical computer scientist, Schlumberger Centennial Chair of Computer Science at the University of Texas at Austin and director of its Quantum Information Center. Writes the blog Shtetl-Optimized.
Their words
Briefly, I think the book is a triumph.
Show the whole quote
scottaaronson.blog
18 July 2019
Author of Fluent Forever and founder of the app of the same name; teaches pronunciation first and vocabulary through images rather than translation.
7 March 2018
Engineering leader and writer; wrote The Manager's Path and Platform Engineering, and blogs at Elided Branches.
Are you out of alignment? Alignment, in the teamwork sense, means “a position of agreement or alliance.” It is one of the critical qualities that determines success in an organization, particularly at higher levels. Many individual contributors…
AI alignment Related
Nothing matches. Show everything
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com .
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
Got it
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.
Got it