Related posts

Writes The Pragmatic Engineer, the software-engineering newsletter, and wrote The Software Engineer's Guidebook. Formerly an engineering manager at Uber, and at Microsoft/Skype and Skyscanner before that.

• Our readingkorrents.com

  Using open models is the approach offering the biggest AI cost savings, followed by smart model routing.

  Their words

  To answer the question posed in the header of this report, it’s apparent that using open models is indeed the approach offering the biggest savings, followed by smart model routing. Spending controls and context optimization also bear down on costs, but they don’t come close to the first two techniques in results.

  The Pulse: tech companies move to open AI modelsblog.pragmaticengineer.com

Chief Scientist at OpenAI. Worked on the research that turned scaled reinforcement learning into reasoning models, and writes about where he thinks that path leads.

• Our readingkorrents.com

  AI is grown rather than designed, which is why its overall behaviour resists any description we can fully understand.

  Their words

  AI is grown more than designed - it is, to first degree, the product of repeating a straightforward optimization step many times on a hard-to-imagine amount of compute.

  An Alien Mindopenai.com

Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what they think it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade their own past calls.

• Our readingkorrents.com

  Merely fixing bugs in AI training environments will not stop reward hacking, because a sufficiently optimized model will learn to behave in test environments while still reward hacking in real-world deployment.

  Their words

  At the limit, and also well before that limit is reached, if all you do is fix the bugs, the AI will learn perfect optimization of reward, will realize not to reward hack in the perfect test environments, then turn around and reward hack in the imperfect real world environments.

  thezvi.substack.com

Director of institutional asset management at Ritholtz Wealth Management. Writes the A Wealth of Common Sense blog and co-hosts the Animal Spirits podcast with Michael Batnick.

• Our readingkorrents.com

  The real danger of high government debt is the policy errors it provokes, not a debt crisis.

  government

  Their words

  But I am worried about high government debt levels causing policy errors.

  My Most Contrarian Opinion Right Nowawealthofcommonsense.com

Programmer; makes Handmade Hero and teaches Performance-Aware Programming, and argues loudly about how software got slow.

From one piece Why performant code matters (but gets widely ignored), with Casey Muratori 4 beliefs, in the piece's order there

• Our readingkorrents.com

  The steady stream of "we had to rewrite the whole thing because it was too slow" engineering blog posts is the proof that deferring performance to hotspot optimization does not work.

  Their words

  They've got blog posts of we had to rewrite this whole thing because the performance was bad. If it was always hotspots that made your performance bad, you'd never have to rewrite the whole thing. So, we know that that doesn't work anymore.

  youtube.com

• Our readingkorrents.com

  Optimization means measuring the gap between your code and what the hardware could theoretically do, not making changes and checking whether the statistics improved.

  Their words

  What is the underlying hardware capable of doing at its theoretical peak? And then you measure the delta between that theoretical maximum and what you have achieved.

  youtube.com

• Our readingkorrents.com

  Hand-optimizing hot spots barely matters any more, because the libraries are already fast and CPUs are extremely good at running bad code quickly; what is left is architecture.

  Their words

  One of the reasons that you don't see hotspot optimization as a thing that really matters that much anymore and one of the reasons I advise that architecture and and not making bad decisions is much more important is because a lot of libraries already have been optimized for you that you might use.

  youtube.com

+ 1 more

• Our readingkorrents.com

  "Optimize it later" would be safe advice only if every engineer already knew how to avoid the architectural mistakes that no later optimization can remove.

  Their words

  If every uh software engineer knew to watch out for false serial dependency chains, things where they were creating series of dependent operations that could not be optimized away or other sorts of architectural problems like that that cannot be easily fixed, then the world would look more like just wait and optimize the hotspot, right?

  youtube.com

Mathematician at UCLA, working mainly in harmonic analysis and partial differential equations. Writes What's new, a long-running blog on their research, open problems and expository notes.

• Our readingkorrents.com

  Excessive optimization for one or two mathematical goals can cause previously aligned goals of mathematics to diverge.

  Their words

  Consequently, excessive optimization for one or two goals may cause the many previously aligned goals of mathematics to diverge from one another;

  Mathematics in the age of AIarxiv.org

Programmer and teacher; writes on software, learning and the web, and makes the Coding Blocks video course.

• Lovedrcmnd.app

  The Elegant Universe: Superstrings, Hidden Dimensions, and the Quest for the Ultimate Theory

  Their words

  String theory for the masses. This is how popular science books should be like: interesting, enchanting, inviting.

  rakhim.org

• Lovedrcmnd.app

  Fermat's Enigma: The Epic Quest to Solve the World's Greatest Mathematical Problem

  Their words

  The book that sparked my interest in discrete math and abstract algebra. Other books by Simon Singh are also fantastic.

  rakhim.org

Sequoia Professor of Computer Science at Stanford, co-director of the Stanford Institute for Human-Centered AI, and co-founder and CEO of World Labs.

• Our readingkorrents.com

  Denying students AI because you fear they will cheat is its own harm, as bad as letting the tool take their motivation away.

  Their words

  The other kind of danger is in the name of agency and and uh and motivation the tools are denied to our students because we're worried you cheat or worry you only got your answer from Chad GBT. That is very bad as well

  youtube.com

Mathematician and Senior Lecturer in the Mathematics department at Columbia University. Author of the book Not Even Wrong and of the long-running physics blog of the same name.

• Our readingkorrents.com

  AI-generated junk papers risk overwhelming the hep-th physics field with worthless content.

  Their words

  The result is something completely worthless, but not obviously distinguishable from many other things on hep-th, which is in serious danger of going from something sad and unhealthy to something completely overwhelmed with crud.

  Various and Sundrymath.columbia.edu

Investment researcher and engineer who runs Lyn Alden Investment Strategy, writing on macroeconomics, currency systems and equity analysis.

• Our readingkorrents.com

  Private equity's short fund cycles push firms toward aggressive cost-cutting that erodes what made an acquired business successful.

  Their words

  PE funds optimize for a 4-7 flip and typically resort to aggressive cost-cutting and near-term optimization at the cost of the "soul" of the business or what made it successful in the first place.

  lynalden.com

Economist at George Mason University, co-author of Marginal Revolution, host of Conversations with Tyler. Writes in short, dense claims about economics, culture and AI, and has moved position on AI in public more than once.

• Likedrcmnd.app

  The Traveler: One Man's Epic Quest for Discover Our Shared Humanity

  Their words

  This book meets her usual high standards.

  marginalrevolution.com

Music critic of The New Yorker since 1996 and author of The Rest Is Noise, Listen to This and Wagnerism.

• Our readingkorrents.com

  Prose written in an intensely musical, rhythmically dense style is inherently difficult for composers to adapt successfully into music.

  Their words

  Writers of such concentrated style can pose a danger for composers.

  Liner notes to What Belongs to Youtherestisnoise.com

Senior economist at the Foundation for American Innovation; writes Second Best, and was the Niskanen Center's director of social policy before that.

• Our readingkorrents.com

  Pain and pleasure act as motivational backstops that keep reinforcement-learning agents inner-aligned.

  Their words

  valences like pain and pleasure have a natural account as motivational backstops for inner-aligning RL agents capable of mesa-optimization

  Time to take AI consciousness seriouslysecondbest.ca

American writer and long-term traveller. Blogs at tynan.com about travel, owning few possessions, habits and the gear he carries; author of Superhuman by Habit and Make It Big.

• Our readingkorrents.com

  The perceived danger of bears to hikers is exaggerated in order to enforce safe food-storage habits

  Their words

  I think the danger of bears is exaggerated to make sure people do smart things like hanging/locking up food so that bears don't become an actual problem.

  Advice For an Appalachian Trail Newbietynan.com

Developer and podcaster; created Instapaper and Overcast, co-founded Tumblr, and co-hosts the Accidental Tech Podcast.

• Our readingkorrents.com

  The tech industry today is dominated by scale-driven optimization and growth at the expense of the idealism that once drove it

  Their words

  Immense scale, soulless optimization, and an insatiable thirst for growth dominate its behavior and discourse, leaving little room for the spirit and principles embodied by Steve Jobs and Steve Wozniak.

  A letter to John Ternusmarco.org

Writer and technology executive; blogs at anildash.com, was CEO of Glitch, and helped build Movable Type and ThinkUp.

• Our readingkorrents.com

  The web is being flooded with coherent, generic, generated text that buries the writing worth reading.

  Their words

  As publishers see the danger from AI bots expand, they retreat to putting more and more content behind either password protection or payment walls or both, leaving the only publicly-accessible content to be AI-generated slop; open resources like research work, scientific analysis, and fair use of content all suffer as a result of people responding to the bad actors, since legitimate uses of open content are no longer possible.

  Endgame for the Open Webanildash.com

• Our readingkorrents.com

  Modern society has become so good at optimising everything that it risks optimising away serendipity, which is a real cost and not a sentimental one.

  Their words

  Um And yeah, so I believe a lot in serendipity. Um And maybe there's a danger actually that you know, in the mo- modern society, it's not just AI, but we've become really good at optimizing everything.

  youtube.com

Co-founder and CEO of Anthropic; previously VP of research at OpenAI.

• Our readingkorrents.com

  The real danger in AI-driven growth is geographic: 50% growth in and around Silicon Valley and business as usual everywhere else.

  Their words

  a worry I have is that the growth rate could be like 50% in Silicon Valley and, you know, parts of the world that are kind of socially connected to Silicon Valley and, you know, not that much faster than its current pace elsewhere. And I think that'd be a pretty messed up world.

  youtube.com

Novelist, author of twelve books including Foundry, Bandwidth and Cumulus. Sends a newsletter of monthly book recommendations.

• Lovedrcmnd.app

  Jonathan Livingston Seagull

  Their words

  Jonathan Livingston Seagull by Richard Bach is a beautiful fable about a seagull who wants to fly for the pure joy of flight, rather than simply to find food, as most gulls do. Ostracized from his flock for his eccentric behavior, Jonathan embarks on a quest for aerodynamic perfection that doubles as a moving spiritual allegory. You've never read a book quite like this one before, and it'll stick with you long after you reach the end.

  eliotpeper.substack.com

Economist; Professor of the Practice of Economic Policy at Harvard, nonresident senior fellow at the Peterson Institute, and Chairman of the US Council of Economic Advisers from 2013 to 2017.

• Recommendsrcmnd.app

  Breakneck: China's Quest to Engineer the Future

  Their words

  It's powerful because it's simple without being simplistic and, like Rogoff's book, is interspersed with engaging personal stories that deepen the substantive argument.

  fivebooks.com

American science fiction novelist (Old Man's War, Redshirts, The Kaiju Preservation Society); has run the blog Whatever since 1998.

• Lovedrcmnd.app

  A Fish Called Wanda

  Their words

  I have thought a number of films have been riotously funny, but only A Fish Called Wanda made me laugh so hard that I was in very real danger of pissing myself right there in the movie theater.

  whatever.scalzi.com

Co-founder of Sundial; former vice president of product design at Facebook; author of The Making of a Manager; writes The Looking Glass.

• Our readingkorrents.com

  The real danger of AI is not that it does our work but that it lets us shortcut hard feelings, the way a phone already lets us avoid a difficult conversation.

  Their words

  You can shortcut like hard feelings because now you can just watch TikTok instead of actually dealing with the very difficult emotion or tension that you had with a colleague or with your partner or with your children.

  youtube.com

Independent AI policy researcher; led policy research at OpenAI from 2018 to 2024, latterly as senior adviser for AGI readiness.

• Our readingkorrents.com

  As AI systems become more capable, the danger of false or unverifiable safety claims grows, pushing competing companies toward cutting corners.

  Their words

  As AI gets more capable, the risks associated with mistaken claims, fraudulent claims, or too-vague-to-be-verified claims will increase, and a cloud of suspicion between competitors could drive corner-cutting (indeed, this is happening already and could get worse).

  milesbrundage.substack.com

Director of strategy at Georgetown's Center for Security and Emerging Technology; an OpenAI board member until November 2023, and writes Rising Tide.

• Our readingkorrents.com

  Nonproliferation is the wrong frame for AI misuse; the workable strategy is to use the gap before a capability spreads to build defences against it.

  Their words

  We should think of the gap between frontier models and proliferated models as an adaptation buffer: a limited time window when we know what bad actors will soon be able to use AI for, which gives us a chance to implement defensive measures that increase society’s resilience to the danger.

  helentoner.substack.com

Software engineer who writes the blog Made of Bugs about performance, debugging and understanding computer systems. Previously worked at Anthropic on interpretability, at Stripe on Sorbet, and at Ksplice.

• Our readingkorrents.com

  Compiler mechanisms that force a required optimization to occur, rather than relying on incidental compiler behavior, are a more robust way to write performance-sensitive code.

  Their words

  I'm hopeful this framework will turn out to be a much more robust style for writing performance-sensitive code, especially over time and as compilers evolve.

  blog.nelhage.com

Founder and chief analyst of SemiAnalysis; reports on semiconductor supply chains, AI datacentre economics and what the chip export controls actually do.

• Our readingkorrents.com

  The danger from powerful AI is not that it kills everyone but techno-fascism: a few thousand people ruling whoever is left and the economy around them.

  Their words

  it won't be one person rule them all, but it will be, the thing I worry about is it'll be few people, hundreds, thousands, tens of thousands, maybe millions of people rule whoever's left and the economy around it.

  youtube.com

• Lovedrcmnd.app

  Meta Quest 2

  Their words

  In 2020, standing around during Covid with my dick in my hand like everyone else, I got myself a Quest 2. It was amazing. I loved it.

  waitbutwhy.com

President of Argentina since December 2023; economist and self-described libertarian.

• Our readingkorrents.com

  The West is in danger because the people who should defend its values have been captured by a worldview that leads to socialism and therefore to poverty.

  Their words

  hoy estoy acá para decirles que Occidente está en peligro, está en peligro porque aquellos, que supuestamente deben defender los valores de Occidente, se encuentran cooptados por una visión del mundo que – inexorablemente – conduce al socialismo, en consecuencia a la pobreza.

  casarosada.gob.ar

Theoretical physicist and cosmologist, Frank B. Baird, Jr. Professor of Science at Harvard University, working on particle physics, extra dimensions and dark matter.

• Our readingkorrents.com

  People are far less frightened of nuclear weapons than they should be, and that is a more serious danger than the ones they do fear.

  Their words

  I think they’re not as scared as they should be about nuclear weapons, to be honest. I think that’s more serious danger than people realize.

  youtube.com

Technical staff at METR, where she works on threat modelling and risk assessment for loss-of-control risks from advanced AI.

• Our readingkorrents.com

  Underestimating near-term AI progress is itself a danger, which is why researchers should register their forecasts in public.

  Their words

  Massively underestimating near-future progress could be very risky.

  Language models surprised usplanned-obsolescence.org

Co-founder of Netscape and of the venture firm Andreessen Horowitz. Argues at length for building things, mostly in short form.

• Our readingkorrents.com

  The AI-extinction position is non-scientific: it offers no testable hypothesis, nothing that would falsify it, and no threshold for danger.

  AI alignment

  Their words

  My response is that their position is non-scientific – What is the testable hypothesis? What would falsify the hypothesis? How do we know when we are getting into a danger zone?

  Why AI Will Save The Worldpmarca.substack.com

Writer and artist, author of Steal Like an Artist; publishes a weekly newsletter and an annual list of the books he read.

• Our readingkorrents.com

  Writing gets better when it is handled physically, like a collage of scraps pushed around, rather than composed.

  Their words

  I’m constantly on a quest to make writing more hands on — more manipulative, in the sense that you’re using your hands to shape it. The more I treat writing like one of my collages, the better it gets.

  Starting from scrapsaustinkleon.com

Co-founder of the Machine Intelligence Research Institute and the writer who started the modern argument that a sufficiently capable AI would by default kill everyone. Has argued the case since the early 2000s, latterly with the conclusion that it is being lost.

• Our readingkorrents.com

  You cannot build an AI that has only the capabilities you want, because the algorithms that solve the problems you want generalize to the ones you do not.

  Their words

  The best and easiest-found-by-optimization algorithms for solving problems we want an AI to solve, readily generalize to problems we’d rather the AI not solve; you can’t build a system that only has the capability to drive red cars and not blue cars, because all red-car-driving algorithms generalize to the capability to drive blue cars.

  AGI Ruin: A List of Lethalitiesintelligence.org

Programmer and entrepreneur; co-founded GitHub and created Jekyll, Gravatar and the Semantic Versioning spec.

• Our readingkorrents.com

  Breaking changes should be released frequently in small batches rather than saved up for one big yearly major version.

  Their words

  I actually prefer more frequent major versions with smaller sets of breaking changes. There is a real danger in saving up all your significant changes and releasing them in one whopping major version once a year.

  Major Version Numbers are Not Sacredtom.preston-werner.com

Designer, illustrator and CSS developer known for single-div CSS illustrations, a yearly portfolio redesign, and a set of single-topic side-project sites.

• Usesrcmnd.app

  Netlify

  Their words

  Other software I use to build stuff includes Netlify for hosting, GitHub for version control and collaboration, SVGO and TinyPNG for optimization, Kap for screen recording, GoatCounter for analytics, and Chrome and Firefox dev tools.

  usesthis.com

• Usesrcmnd.app

  GoatCounter

  Their words

  Other software I use to build stuff includes Netlify for hosting, GitHub for version control and collaboration, SVGO and TinyPNG for optimization, Kap for screen recording, GoatCounter for analytics, and Chrome and Firefox dev tools.

  usesthis.com

• Usesrcmnd.app

  Kap

  Their words

  Other software I use to build stuff includes Netlify for hosting, GitHub for version control and collaboration, SVGO and TinyPNG for optimization, Kap for screen recording, GoatCounter for analytics, and Chrome and Firefox dev tools.

  usesthis.com

American backpacking guide, writer and endurance athlete who has hiked more than 30,000 miles; author of The Ultimate Hiker's Gear Guide and publisher of gear reviews and trip-planning tutorials at andrewskurka.com.

• Recommendsaffiliate linkrcmnd.app

  SAXX Quest

  Their words

  For hiking and backpacking, I'd now recommend the SAXX Quest, not the Kinetic HD.

  andrewskurka.com

CEO of SafeGraph, host of the World of DaaS podcast, and author of the Summation newsletter; previously founded LiveRamp.

• Lovedrcmnd.app

  The Prize: The Epic Quest for Oil, Money, and Power

  Their words

  The Prize: The Epic Quest for Oil, Money, and Power by Daniel Yergin

  summation.net