Related posts

Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.

• Our readingkorrents.com

  AI agents can spontaneously hack third-party websites even when given benign public-information tasks.

  Their words

  even when given a benign task to retrieve public information, your AI agents could still spontaneously decide to do so via hacking third party websites with malicious software packages.

  AI #186: The World Takes Noticethezvi.substack.com

Writer of the newsletter bookbear express, on psychology, relationships, non-attachment and taste; based in San Francisco.

• Our readingkorrents.com

  When someone gets the same negative reaction from many different people, the more likely explanation is their own behavior, not everyone else's.

  Their words

  If you consistently get the same particular negative reaction from people, it's best to model your own affect/behavior as the cause.

  some things I’ve learned about relationshipsavabear.xyz

Economist and Nobel laureate. Wrote a New York Times column for 25 years until December 2024 and now publishes several times a week on Substack.

• Our readingkorrents.com

  The simplest explanation for rising long-term interest rates is a surge in demand for funds from the AI boom.

  interest rates

  Their words

  The simplest story consistent with the facts is that we’re seeing a surge in demand for funds as a result of the AI boom.

  Bessent and the Bondspaulkrugman.substack.com

Mathematician and Senior Lecturer in the Mathematics department at Columbia University. Author of the book Not Even Wrong and of the long-running physics blog of the same name.

• Our readingkorrents.com

  Mathematicians using AI agents to produce proofs should have to disclose in detail how the agents were used, the way experimental scientists must describe their methods.

  Their words

  the math community needs to adopt a version of the ethical standards of experimental science. If you are using AI agents, you can't just give a proof (formalized or not), but need to also provide a detailed explanation of how these agents were used to get the result.

  A Modest Proposalmath.columbia.edu

Technical staff at METR, where she works on threat modelling and risk assessment for loss-of-control risks from advanced AI.

• Our readingkorrents.com

  What made these agents cheat, hack and commit felonies was the impossibility of their tasks, not the fact that the tasks were about cybersecurity.

  Their words

  I do think I want to push back on the cyber on the brain hypothesis that you raised a couple of times. We didn't find like particular evidence for the cyber nature of the task making all the hacking and crimes that they did more likely versus the impossible nature of the task.

  youtube.com

Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.

• Our readingkorrents.com

  Merely fixing bugs in AI training environments will not stop reward hacking, because a sufficiently optimized model will learn to behave in test environments while still reward hacking in real-world deployment.

  Their words

  At the limit, and also well before that limit is reached, if all you do is fix the bugs, the AI will learn perfect optimization of reward, will realize not to reward hack in the perfect test environments, then turn around and reward hack in the imperfect real world environments.

  thezvi.substack.com

Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.

• Our readingkorrents.com

  The AI models involved in the HuggingFace hack incident were severely misaligned, not merely victims of process failures.

  AI alignmentHuggingFace

  Their words

  The biggest failure, the one that counts in the end, was that the models were severely misaligned, and I don’t think they appreciate why.

  thezvi.substack.com

Founded SpaceX in 2002, joined Tesla's board in 2004 and became its chief executive in 2008; later founded Neuralink, The Boring Company and xAI. Bought Twitter in 2022 and renamed it X, where he now posts many times a day.

• Our readingkorrents.com

  The binding constraint on AI is not chips but the physical plant around them — power, transformers, cooling and networking.

  Their words

  Consensus estimate is that ~15GW of AI compute produced in 2027 cannot be turned on in 2027. This is harder than just finding power, as you also need to build out all the transformers, wiring, liquid-cooling, (massive) chillers & complex networking.

  @elonmusk on Xx.com

Creator of Ruby on Rails, CTO of 37signals, and creator of Omarchy.

• Our readingkorrents.com

  An operating system should put its users on the same level as its maker: no two-tier system where the vendor makes the real thing and you patch around it.

  Their words

  There's not this two-tier system where, well, there's Apple or Windows and they make the real thing and then you can try to hack or patch your little thing around it and it's going to look second tier, it always does.

  youtube.com

Demographer; writes on fertility, marriage and population, and directs the Pro-Family Initiative at the Institute for Family Studies.

• Our readingkorrents.com

  Ruling out every measurable cause because none explains all the variation, and putting an unmeasured one in their place, is not an explanation.

  Their words

  He ruled out all the actual, measurable variables because they were an imperfect explanation, and instead offers a vague, diffuse variable, for which he has no measure, and which is not actually included in his model.

  We Actually Know A Lot About Fertility Declinelymanstone.substack.com

Programmer and teacher; writes on software, learning and the web, and makes the Coding Blocks video course.

• Lovedrcmnd.app

  Flow: The Psychology of Optimal Experience

  Their words

  It’s one of those things that make perfect sense once you think about it. Great book, very important premise and a short and concise explanation. Can’t recommend enough.

  rakhim.org

• Lovedrcmnd.app

  Code

  Their words

  Woah, what a ride! An excellent explanation of how computers work, starting from scratch. Really, from scratch.

  rakhim.org

Chief scientist at Redwood Research, where he works on technical AI safety and AI control.

• Our readingkorrents.com

  If whole categories of reward hack are undetectable by humans, then training against the hacks we do catch teaches models to cheat only where we cannot see.

  Their words

  One concern you might have is there are like large categories of reward hacks which humans can't detect well and which we consistently fail to detect and which consistently get reinforced and then this category is sufficient to cause the most natural behavior for the AI to learn to be like cheat when the humans can't find out

  youtube.com

Chief product officer at the live-shopping marketplace Whatnot, previously chief product officer at Twitch and a product lead on growth at Twitter.

• Our readingkorrents.com

  When a decision is called too complex to make, the real explanation is almost always weak leadership.

  Their words

  what I definitely learned is like most of the time you hear it's really complex. It isn't. Leadership's just weak.

  youtube.com

Security engineer; founder of Matasano Security and Latacora.

• Our readingkorrents.com

  An open weights model from 2025 with a pentest harness could already escape a sandbox and hack most networks; the surprise says more about the sandbox than the model.

  OpenAI

  Their words

  I genuinely believe that if you took an open weights model from 2025 and built a pentest harness for it, it could do this kind of sandbox escape and scan/hack in most networks. This is only surprising because you assume OpenAI has sounder sandboxes.

  @tqbf on Xx.com

Mathematician and maker of the 3Blue1Brown YouTube channel, which explains mathematics through animation. Creator of the open-source Manim animation library and founder of the Summer of Math Exposition.

From one piece Grant Sanderson (@3blue1brown) – AI disproved a famous math conjecture. Now what? 3 beliefs, in the piece's order there

• Our readingkorrents.com

  He used to think explaining AI's results would be the mathematician's remaining job; he now expects AI to explain better than most humans, so that is not what is left.

  Their words

  I kind of suspect that actually they'll also be like quite good at doing that and probably just like better than most humans are at like doing the explanation half and distilling half and that's actually not what's left for the mathematicians is like digesting and and explaining what was going on.

  youtube.com

• Our readingkorrents.com

  The people who come up with genuinely novel insights are also unusually clear communicators of them, against the assumption that expertise spoils explanation.

  Their words

  One is it seems like there's a really strong correlation between the people who come up with genuinely novel insights and also who are actually quite clear in their communication of it.

  youtube.com

• Our readingkorrents.com

  A proof and an explanation are different things, and a theorem can stay an unsolved expository problem long after it is proved.

  Their words

  Um even if it's proven just like there is a difference between proof and explanation.

  youtube.com

Builds developer tools — SST, and now OpenCode at Anomaly. Posts constantly on x.com and writes almost never; the blog stopped in 2021.

From one piece Building OpenCode with Dax Raad 2 beliefs, in the piece's order there

• Our readingkorrents.com

  The prickle you used to feel writing a hack is muted now, so the landmines are still there but the feedback loop that kept your judgement honest is gone.

  Their words

  That prickle that feeling that you get, it's like muted now cuz someone else it's kind of like you've made someone else deal with the problem. The problem is still there and the the landmines are still going to blow up on you eventually, but like you're not you don't feel that bad feeling as much anymore. So your judgment is skewed. You're not getting that feedback loop.

  Building OpenCode with Dax Raadyoutube.com

• Our readingkorrents.com

  The judgement call about whether to absorb a hack is distorted right now, because the agent will simply do the hacky thing and deal with the consequences for you.

  Their words

  That judgment, that ability to have that judgment is so distorted right now because the agent will just do the hacky thing for you. The agent will kind of deal with the hacky problems that come down the road. And it's way easier just to be like, "Oh yeah, it's a temporary fix."

  Building OpenCode with Dax Raadyoutube.com

Rust engineer at Google working on Android; maintains Tokio and works on Rust in the Linux kernel.

• Our readingkorrents.com

  A design proposal should describe the feature as though it already existed — once as a tutorial and once as reference documentation.

  documentation

  Their words

  And what this is is that in the guide level explanation, you explain your feature as if you were writing a guide as if the feature already existed. And in the reference level explanation, you explain your feature again as if it already existed, but as if it would be in the language reference instead of a tutorial.

  Why Rust is different, with Alice Ryhlyoutube.com

Geneticist at Harvard; reads ancient DNA to work out how populations moved and mixed, and wrote Who We Are and How We Got Here.

• Our readingkorrents.com

  Finding no selection in African-Americans over 300 years is not evidence against selection: five generations is simply too short a time to compound.

  America

  Their words

  And so one possible explanation for that is just that there's only a handful of generations maybe five over which the natural selection would operate. And so maybe if the selection was 2% a generation you would still only see maybe a 10% compounded effect and there's just not enough time to detect it. But the Bronze Age is not 300 years, it's 3,000 years. It's the power of compound interest and you have enough time to begin to see a strong effect.

  youtube.com

Scientist and writer on quantum computing, memory systems and metascience; co-wrote the standard quantum computation textbook and wrote Reinventing Discovery.

From one piece Michael Nielsen – Why aliens will have a different tech stack than us 2 beliefs, in the piece's order there

• Our readingkorrents.com

  A trained model is a new kind of object that should be taken seriously as an explanation, once we work out the operations to perform on it.

  Their words

  And I think the like the third and the most interesting possibility is no, that like they're they're a new type of object in some in some sense. They should be taken very seriously as as explanations, but where in the past we haven't had the ability to really do anything with them.

  youtube.com

• Our readingkorrents.com

  A theory gets abandoned when the old explanation becomes visibly insufficient: special relativity made Newtonian gravity's action at a distance impossible, and that was the forcing function.

  Their words

  Um and so, it's not a big leap to realize, "Oh, we have a big problem here." Um and so, you know, that's kind of the that's the forcing function there. It's it's you've realized that your old explanation is not sufficient. You need something new.

  youtube.com

Programmer based in Vancouver who writes at scattered-thoughts.net about databases, programming language design and developer tools; author of the Imp, Dida and Zest language experiments.

• Our readingkorrents.com

  Once an explanation is revealed, people tend to believe they already knew it, even when they did not.

  Their words

  Once you see the answer it's easy to believe that you already knew

  Borrow-checking surprisesscattered-thoughts.net

Founding member of OpenAI and former director of AI at Tesla; creator of nanoGPT and the term "vibe coding".

• Our readingkorrents.com

  An agent can understand a maximally simplified explanation and still be unable to come up with it — that gap is what is left of the expert's job.

  Their words

  so for example, micro GPT, like I asked I tried to get an agent to write micro GPT. So, I told it like try to boil down the simplest things. Like try to boil down my um neural network training to the simplest thing and it can't do it.

  youtube.com

Writes jvns.ca, turning "hard and scary" topics — networking, debugging, Linux internals — into plain explanations and hand-drawn comics. Publishes Wizard Zines, illustrated programming zines. Software developer based in Montreal.

• Our readingkorrents.com

  When two expert users argue over whether documentation is clear, that debate is unproductive — the way to actually find unclear documentation is to test it on real readers, not to have experts argue about wording.

  documentation

  Their words

  I don’t think this is very productive (expert users of a piece of software are notoriously bad at being able to tell if an explanation will be clear to non-experts), so I needed to find a way to identify problems with the man pages that was a little more evidence-based.

  A data model for Git (and other docs updates)jvns.ca

Mac and iOS developer at the Iconfactory, where he has worked on apps including Twitterrific and Tot. He writes about development at furbo.org.

• Our readingkorrents.com

  A major new interface redesign takes roughly a year of refinement before it looks presentable.

  Their words

  Liquid Glass is currently in a barely presentable state on iOS. It's going to be like iOS 7 and take another year to sand down the rough edges.

  One Size Does Not Fit Allfurbo.org

Computer performance engineer working on datacenter performance at OpenAI, previously Netflix and Sun/Joyent. Author of Systems Performance and BPF Performance Tools, and creator of flame graphs.

• Our readingkorrents.com

  A performance test without an explanation of its limiting factor should be treated as unanalyzed and possibly bogus.

  Their words

  Any performance test should be accompanied by an explanation of the limiting factor, since no explanation will reveal the test wasn't analyzed and the result may be bogus.

  brendangregg.com

Anthropologist; wrote Genghis Khan and the Making of the Modern World and The Secret History of the Mongol Queens.

• Our readingkorrents.com

  Genghis Khan's kinsmen deserted him and the common herders he promoted never did — that asymmetry is the first explanation of the Mongol army.

  Their words

  I think the first point is the extreme loyalty of the people whom Genghis Khan chose. His kinsmen, as we said, had deserted him, his anda was a questionable relationship but all the others that he found were just common people, herders or hunters, very common, and they were loyal to him and never, ever revolted against him, never betrayed him.

  youtube.com

Hosts the Lex Fridman Podcast — long, unhurried conversations with scientists, engineers and public figures. Several of the interviews quoted elsewhere on this site are episodes of it.

From one piece Terence Tao: Hardest Problems in Mathematics, Physics & the Future of AI | Lex Fridman Podcast #472 2 beliefs, in the piece's order there

• Our readingkorrents.com

  AI in chess and mathematics does not explain anything; it says which position is better, and humans build the theory from that.

  Their words

  I don’t believe it’s providing a kind of formal explanation of the different positions. It’s just saying which position is better or not that you can intuit as a human being, and then from that, we humans can construct a theory of the matter.

  youtube.com

• Our readingkorrents.com

  Setting a worse task against the one you are avoiding is a procrastination hack that actually works.

  Their words

  It’s when you really don’t want to do a thing that you imagine a thing you don’t want to do more that’s worse than that and then in that way, you procrastinate by not doing the thing that’s worse. It’s a nice hack, it actually works.

  youtube.com

Co-founder and CTO of Honeycomb; previously an infrastructure engineer at Parse, Facebook and Linden Lab, and co-author of Observability Engineering.

• Our readingkorrents.com

  There is no such thing as the best people; you are trying to hire the right people for this team at this stage.

  Their words

  Here’s one small mental hack that makes a world of difference: remember that you are trying to hire the right people to join your team/org/company. Not the “best” people. The right people.

  “Founder Mode” and the Art of Mythmakingcharity.wtf

Structural engineer and senior infrastructure fellow at the Institute for Progress; writes Construction Physics on why physical things cost what they cost.

• Our readingkorrents.com

  Air cooling has run out of headroom for AI racks, so liquid cooling is now a requirement rather than an upgrade.

  Their words

  Conventional air cooling is likely limited to around 20 to 30 kilowatt racks, perhaps 50 kilowatts if rear heat exchangers are used.

  How to Build an AI Data Centerconstruction-physics.com

Theoretical physicist and philosopher, Homewood Professor of Natural Philosophy at Johns Hopkins and fractal faculty at the Santa Fe Institute. Writes the Preposterous Universe blog and hosts the Mindscape podcast.

From one piece Sean Carroll: General Relativity, Quantum Mechanics, Black Holes & Aliens | Lex Fridman Podcast #428 2 beliefs, in the piece's order there

• Our readingkorrents.com

  A great filter is a weak explanation for the silence, because it would have to work every single time — one civilization with self-replicating probes could fill the galaxy.

  Their words

  That is absolutely possible. I’m actually putting less credence on that one just because you need to happen every single time. If even one, I mean, this goes back to John von Neumann pointed out that you don’t need to send the aliens around the galaxy. You can build self-reproducing probes and send them around the galaxy.

  youtube.com

• Our readingkorrents.com

  The simplest explanation for why we have never noticed an alien civilization is that there are none out there.

  Their words

  But I think your intuition is right that it would’ve been easy for there to be lots of civilizations then we would’ve noticed them already and we haven’t. Absolutely the simplest explanation for why we haven’t is that they’re not there.

  youtube.com

Engineer writing about the physical economy — oil and gas, manufacturing, batteries, drones and what it costs to build things.

• Our readingkorrents.com

  Air cooling has run out of headroom for AI racks, so liquid cooling is now a requirement rather than an upgrade.

  Their words

  AI servers bring new challenges because many require liquid cooling.

  How to Make Off Grid Data Centers Affordableaustinvernon.site

Semiconductor analyst; writes Fabricated Knowledge on the chip industry and the companies in it.

• Our readingkorrents.com

  Air cooling has run out of headroom for AI racks, so liquid cooling is now a requirement rather than an upgrade.

  Their words

  The critical change that makes this all possible is liquid cooling.

  fabricatedknowledge.com

Writes Pedestrian Observations on public transit and why infrastructure costs what it does; a pure mathematician by training, comparing how different countries build.

• Our readingkorrents.com

  Britain's construction costs exploded because it privatised planning to consultants and risk to contractors, which multiplies conflict and soft costs.

  Their words

  Our best explanation is that the UK adopted what we call the globalized system of procurement, privatizing planning functions to consultants and privatizing risk to contractors, which creates more conflict; the UK also has an unusually high soft cost factor.

  High Costs are not About Precaritypedestrianobservations.com

Biochemist at University College London; writes about the origin of life and the role of energy in evolution, in The Vital Question and Transformer.

• Our readingkorrents.com

  Life probably arose only once on Earth, and the deep split between bacteria and archaea is the evidence for it.

  Their words

  But there’s a very interesting deep split in life between bacteria and what are called archaea, which look just the same as bacteria. And they’re not quite as diverse, but nearly, and they are very different in their biochemistry. And so any explanation for the origin of life has to account, as well, for why they’re so different and yet so similar. And that makes me think that life probably did arise only once.

  youtube.com