Jan Leike

From one piece Alignment is not solved 4 beliefs, in the piece's order there

• Our readingkorrents.com

  Making a capable model malicious is about as easy as making it aligned, because the same levers work in both directions.

  Anthropic

  Their words

  In fact, making an evil version of Claude that’s just as smart and agentic would be pretty easy.

  Alignment is not solvedaligned.substack.com

• Our readingkorrents.com

  Recursive self-improvement has already started, because AI research itself is now being automated.

  Their words

  We are starting to automate AI research and the recursive self-improvement process has begun.

  Alignment is not solvedaligned.substack.com

• Our readingkorrents.com

  We do not have to align superintelligence directly; we have to build a human-level automated alignment researcher we trust more than ourselves.

  AI alignment

  Their words

  But the goal we need to achieve is so much easier: we just need to build a model that’s as good as us at alignment research, and that we trust more than ourselves to do this research well because it’s sufficiently aligned.

  Alignment is not solvedaligned.substack.com

+ 1 more

• Our readingkorrents.com

  Alignment of today's models is going well enough to look solvable, while aligning models we can no longer understand remains unsolved.

  AI alignment

  Their words

  This is the hard problem of alignment we need to solve in order to succeed at building superintelligence, and to this day it is an unsolved problem.

  Alignment is not solvedaligned.substack.com

• Our readingkorrents.com

  Building AI that can actually be trusted is the goal; containing an AI known to be misaligned is not a substitute for it.

  AI alignment

  Their words

  More generally, we should actually solve alignment instead of just trying to control misaligned AI.

  Should we control AI instead of aligning it?aligned.substack.com

• Our readingkorrents.com

  Human oversight of AI behaviour is already intractable at the volume models produce, so the overseers have to be models too.

  Their words

  Therefore it’s simply intractable to meaningfully oversee all but a tiny fraction of our models’ behavior with actual humans.

  Two alignment threat modelsaligned.substack.com

From one piece Self-exfiltration is a key dangerous capability 2 beliefs, in the piece's order there

• Our readingkorrents.com

  The dangerous capability that matters most for misalignment is whether a model can exfiltrate its own weights, because that is the one that removes it from your control.

  AI alignment

  Their words

  If a model was capable of self-exfiltration, it would have the option to remove itself from your control.

  Self-exfiltration is a key dangerous capabilityaligned.substack.com

• Our readingkorrents.com

  Once model weights leave their owner's control the loss is irreversible, which is why securing them is the precondition for governing AI at all.

  Their words

  Moreover, model exfiltration is likely impossible to reverse.

  Self-exfiltration is a key dangerous capabilityaligned.substack.com