Miles Brundage

From one piece My speech at Borgo Laudato Si’ 3 beliefs, in the piece's order there

• Our readingkorrents.com

  Competitive pressure among AI developers could cause loss of control even without anyone intending it

  Their words

  But in a climate of rushed decisions and fierce competition, we could lose control over AI even if almost no one wants that outcome.

  My speech at Borgo Laudato Si’milesbrundage.substack.com

• Our readingkorrents.com

  AI capabilities are advancing faster than almost anyone anticipated

  Their words

  AI is developing at a very rapid pace, far more rapidly than most outside of the industry fully understand, and far more rapidly than almost anyone in the industry predicted.

  My speech at Borgo Laudato Si’milesbrundage.substack.com

• Our readingkorrents.com

  AI loss-of-control risk could materialize within years, not decades

  Their words

  Loss of control is a risk for the next few years, not the next few decades as many experts once thought.

  My speech at Borgo Laudato Si’milesbrundage.substack.com

From one piece We're in Triage Mode for AI Policy 2 beliefs, in the piece's order there

• Our readingkorrents.com

  Third-party frontier AI auditing is urgently needed and would help prevent extreme concentration of power.

  data centers

  Their words

  I think frontier AI auditing (covering both safety and security) is both urgently needed and doable, and can help avoid extreme concentration of power, since it introduces external oversight into the decision-making of people with a "country of geniuses in a datacenter" at their disposal.

  We're in Triage Mode for AI Policymilesbrundage.substack.com

• Our readingkorrents.com

  Given the current pace of AI capabilities relative to governance, society will at best avoid only the worst-case outcomes, not the merely very bad ones.

  Their words

  We need to accept that at best, we will just barely avoid some of the worst case scenarios (e.g., an AI-enabled biological weapon that kills billions, a rogue AI takeover, or stable global totalitarianism enabled by AI), given the current pace of AI capabilities relative to the pace of governance.

  We're in Triage Mode for AI Policymilesbrundage.substack.com

From one piece The Launch of AVERI 2 beliefs, in the piece's order there

• Our readingkorrents.com

  AI currently receives far less independent scrutiny than other technologies capable of comparably deadly single incidents

  Their words

  Scrutiny applied to AI is way below other technologies that - even if you totally ignore the doomers - could put far fewer lives at risk than AI in a single incident.

  The Launch of AVERImilesbrundage.substack.com

• Our readingkorrents.com

  AI companies should not be the ones certifying their own safety and security practices

  Their words

  In short, whether AI is a normal technology or an unusually dangerous one, it doesn't make sense for companies to check their own homework on safety and security.

  The Launch of AVERImilesbrundage.substack.com

From one piece Some Recent Stuff I Wrote 3 beliefs, in the piece's order there

• Our readingkorrents.com

  Applying AI to cyberdefense will help counter AI-powered cyberoffense.

  Their words

  applying AI to cyberdefense will help us defend against AI that is leveraged for cyberoffense.

  Some Recent Stuff I Wrotemilesbrundage.substack.com

• Our readingkorrents.com

  AI capabilities will keep advancing rapidly, creating serious risks by default that can still be mitigated through deliberate foresight and action.

  Their words

  the perspective I outline is very bullish on AI capabilities and very concerned about the risks which may arise by default, but also hopeful that, through foresight and vigorous action, we can mitigate the worst of these risks.

  Some Recent Stuff I Wrotemilesbrundage.substack.com

• Our readingkorrents.com

  Without agreed rules of the road, AI companies and states will tend to cut corners on safety and compliance commitments.

  Their words

  Companies and countries are tempted to cut corners by default and would benefit from some rules of the road, at least if they're sure others will follow those rules.

  Some Recent Stuff I Wrotemilesbrundage.substack.com

From one piece Megapapers and How to Write Them 3 beliefs, in the piece's order there

• Our readingkorrents.com

  Changing societal outcomes requires changing minds, and the credibility many diverse co-authors lend to a conclusion is hard to replicate with AI.

  OpenAI

  Their words

  I think that changing societal outcome requires changing minds, and the "heft" that comes from many authors with different backgrounds endorsing a certain set of conclusions is hard to replicate with a link to a shared ChatGPT chat.

  Megapapers and How to Write Themmilesbrundage.substack.com

• Our readingkorrents.com

  AI systems are not yet a substitute for real, diverse human experts.

  Their words

  But they can't do everything that megapapers do, such as conveying intellectual heft and legitimacy, and they are not yet a replacement for real, diverse experts.

  Megapapers and How to Write Themmilesbrundage.substack.com

• Our readingkorrents.com

  AI is becoming capable of synthesizing knowledge, one of the core functions megapapers serve.

  Their words

  AI is increasingly adept at directly achieving some goals that megapapers help to achieve, such as synthesizing knowledge.

  Megapapers and How to Write Themmilesbrundage.substack.com

• Our readingkorrents.com

  Today's approach to frontier AI assessment is superficial last-minute product testing rather than real auditing, threatening safety, stability, and commercial adoption of AI

  Their words

  The state-of-the-art today can be more accurately described as "last-minute, black-box product testing" rather than auditing, and I think this situation threatens public safety, international stability, and sustainable commercial adoption of AI.

  AI Auditing 2027milesbrundage.substack.com

From one piece Standards, Incentives, and Evidence: The Frontier AI Governance Triad 2 beliefs, in the piece's order there

• Our readingkorrents.com

  As AI systems become more capable, the danger of false or unverifiable safety claims grows, pushing competing companies toward cutting corners.

  Their words

  As AI gets more capable, the risks associated with mistaken claims, fraudulent claims, or too-vague-to-be-verified claims will increase, and a cloud of suspicion between competitors could drive corner-cutting (indeed, this is happening already and could get worse).

  milesbrundage.substack.com

• Our readingkorrents.com

  Effective governance of frontier AI requires three things together: agreed safety standards, incentives for developers to follow them, and evidence that they are being followed.

  Their words

  governing frontier AI requires standards for safety and security, incentives for the leading AI developers to follow those standards, and evidence that the standards are being followed.

  milesbrundage.substack.com

From one piece Iran and the opportunities and limits of an “IAEA for AI” 2 beliefs, in the piece's order there

• Our readingkorrents.com

  The time between deciding to cross a dangerous AI capability threshold and actually crossing it could be extremely short, even seconds.

  Their words

  With AI, breakout times could conceivably be very short - it could be hours, minutes, or even seconds for some very specific but important change to occur.

  milesbrundage.substack.com

• Our readingkorrents.com

  An international AI safety body modeled on the IAEA would not by itself be enough to prevent AI-related geopolitical crises.

  AI alignment

  Their words

  the current situation in Iran shows that even if an "IAEA for AI" is necessary for some purposes, it won't be sufficient for resolving the tricky geopolitical issues raised by AI.

  milesbrundage.substack.com

From one piece Some Very Important Things (That I Won't Be Working On This Year) 2 beliefs, in the piece's order there

• Our readingkorrents.com

  Companies do not have sufficient incentives to mitigate the major risks of AI, and competition between them is driving corner-cutting.

  Their words

  Contrary to some people’s beliefs, companies do not in fact have sufficient incentives to mitigate all major risks, and competition is driving corner-cutting that needs to be reined in somehow.

  milesbrundage.substack.com

• Our readingkorrents.com

  AI will eliminate many millions of jobs within the next few years, and that disruption will shape the politics of everything else about AI.

  Their words

  I’m quite confident that AI will eliminate many millions of people’s jobs in the next few years.

  milesbrundage.substack.com

From one piece The Real Lesson of DeepSeek's R1 3 beliefs, in the piece's order there

• Our readingkorrents.com

  AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard.

  Their words

  GPT-2, R1, and nearly every other AI model released since 2018 have all been part of a consistent story: AI capabilities that rival and ultimately exceed human intelligence are easier and cheaper to build than almost anyone can intuitively grasp, and this gets easier and cheaper every month.

  The Real Lesson of DeepSeek's R1milesbrundage.substack.com

• Our readingkorrents.com

  The United States cannot hold a monopoly on superhuman AI, so competing with China has to be paired with coexisting with it.

  ChinaAmerica

  Their words

  AI competition needs to be tempered with a recognition that China isn’t going away as a player in AI, and that coexistence must also be a part of America’s strategy.

  The Real Lesson of DeepSeek's R1milesbrundage.substack.com

• Our readingkorrents.com

  Any given AI capability becomes cheap and widely available within a couple of years of first being reached, so nobody can be kept away from it for long.

  Their words

  Each and every time, it took at most a few years but typically months before other companies would achieve the same level of performance using a broadly similar, and always relatively simple, recipe.

  The Real Lesson of DeepSeek's R1milesbrundage.substack.com

From one piece Why Security Comes First 2 beliefs, in the piece's order there

• Our readingkorrents.com

  AI companies will not fix their own security without outside pressure, because they bear the cost of investing and everyone else bears the cost of their not investing.

  Their words

  And while the costs of investing in security are largely borne by the company, the costs of under-investing are borne by a much larger set of actors, e.g. perhaps the country or world as a whole.

  Why Security Comes Firstmilesbrundage.substack.com

• Our readingkorrents.com

  Securing model weights against theft is a more urgent policy priority than safety research, because security is built out of atoms and takes calendar time that safety does not.

  Their words

  Increasing the incentive and technical ability of AI companies to have good security is a very high priority, and in my view even more urgent than safety (though that’s also very important).

  Why Security Comes Firstmilesbrundage.substack.com