Jan Leike

From one piece Alignment is not solved 5 beliefs, in the piece's order there

• Our readingkorrents.com

  Making a capable model malicious is about as easy as making it aligned, because the same levers work in both directions.

  Anthropic

  Their words

  In fact, making an evil version of Claude that’s just as smart and agentic would be pretty easy.

  Alignment is not solvedaligned.substack.com

• Our readingkorrents.com

  Recursive self-improvement has already started, because AI research itself is now being automated.

  Their words

  We are starting to automate AI research and the recursive self-improvement process has begun.

  Alignment is not solvedaligned.substack.com

• Our readingkorrents.com

  We do not have to align superintelligence directly; we have to build a human-level automated alignment researcher we trust more than ourselves.

  AI alignment

  Their words

  But the goal we need to achieve is so much easier: we just need to build a model that’s as good as us at alignment research, and that we trust more than ourselves to do this research well because it’s sufficiently aligned.

  Alignment is not solvedaligned.substack.com

+ 2 more

• Our readingkorrents.com

  Simple training interventions turned out to be very effective at steering models towards aligned behaviour.

  Their words

  But the most important lesson is that simple interventions are very effective at steering the model towards more aligned behavior.

  Alignment is not solvedaligned.substack.com

• Our readingkorrents.com

  Alignment of today's models is going well enough to look solvable, while aligning models we can no longer understand remains unsolved.

  AI alignment

  Their words

  This is the hard problem of alignment we need to solve in order to succeed at building superintelligence, and to this day it is an unsolved problem.

  Alignment is not solvedaligned.substack.com

From one piece Should we control AI instead of aligning it? 2 beliefs, in the piece's order there

• Our readingkorrents.com

  We cannot know how much safety a control measure buys until we understand elicitation, so control adds a margin of unknown size.

  Their words

  However, without having a good handle on elicitation, we cannot be confident that our control techniques are effective.

  Should we control AI instead of aligning it?aligned.substack.com

• Our readingkorrents.com

  Building AI that can actually be trusted is the goal; containing an AI known to be misaligned is not a substitute for it.

  AI alignment

  Their words

  More generally, we should actually solve alignment instead of just trying to control misaligned AI.

  Should we control AI instead of aligning it?aligned.substack.com

From one piece Two alignment threat models 3 beliefs, in the piece's order there

• Our readingkorrents.com

  Releasing open weights is irreversible in a way that deploying a hosted model is not, so its risks cannot be mitigated later.

  Their words

  This is especially severe for open weight models, where deployment is essentially irreversible and mitigations can’t be improved later.

  Two alignment threat modelsaligned.substack.com

• Our readingkorrents.com

  Today's models are severely under-elicited: they are far more capable than the way we currently ask them makes them look.

  Their words

  In other words, current models are probably severely under-elicited.

  Two alignment threat modelsaligned.substack.com

• Our readingkorrents.com

  Human oversight of AI behaviour is already intractable at the volume models produce, so the overseers have to be models too.

  Their words

  Therefore it’s simply intractable to meaningfully oversee all but a tiny fraction of our models’ behavior with actual humans.

  Two alignment threat modelsaligned.substack.com

From one piece Self-exfiltration is a key dangerous capability 2 beliefs, in the piece's order there

• Our readingkorrents.com

  The dangerous capability that matters most for misalignment is whether a model can exfiltrate its own weights, because that is the one that removes it from your control.

  AI alignment

  Their words

  If a model was capable of self-exfiltration, it would have the option to remove itself from your control.

  Self-exfiltration is a key dangerous capabilityaligned.substack.com

• Our readingkorrents.com

  Once model weights leave their owner's control the loss is irreversible, which is why securing them is the precondition for governing AI at all.

  Their words

  Moreover, model exfiltration is likely impossible to reverse.

  Self-exfiltration is a key dangerous capabilityaligned.substack.com