Jan Leike

Beliefs Korrents

• Korrents What they believe 16 beliefs — each backed by an exact quote.

  Each is a korrent — compiled by korrents.com, not by them: the one-line wordings are korrents', the quotes are theirs.

  Recent

  • Alignment of today's models is going well enough to look solvable, while aligning models we can no longer understand remains unsolved.

    This is the hard problem of alignment we need to solve in order to succeed at building superintelligence, and to this day it is an unsolved problem.

    Alignment is not solved Said 22 Jan 2026

    Full record — everyone who holds this →

    AI alignment

  • Simple training interventions turned out to be very effective at steering models towards aligned behaviour.

    But the most important lesson is that simple interventions are very effective at steering the model towards more aligned behavior.

    Alignment is not solved Said 22 Jan 2026

    Full record — everyone who holds this →

  • We do not have to align superintelligence directly; we have to build a human-level automated alignment researcher we trust more than ourselves.

    But the goal we need to achieve is so much easier: we just need to build a model that’s as good as us at alignment research, and that we trust more than ourselves to do this research well because it’s sufficiently aligned.

    Alignment is not solved Said 22 Jan 2026

    Full record — everyone who holds this →

    AI alignment

  Show 13 more

  • Recursive self-improvement has already started, because AI research itself is now being automated.

    We are starting to automate AI research and the recursive self-improvement process has begun.

    Alignment is not solved Said 22 Jan 2026

    Full record — everyone who holds this →

  • Making a capable model malicious is about as easy as making it aligned, because the same levers work in both directions.

    In fact, making an evil version of Claude that’s just as smart and agentic would be pretty easy.

    Alignment is not solved Said 22 Jan 2026

    Full record — everyone who holds this →

    Anthropic

  • Building AI that can actually be trusted is the goal; containing an AI known to be misaligned is not a substitute for it.

    More generally, we should actually solve alignment instead of just trying to control misaligned AI.

    Should we control AI instead of aligning it? Said 24 Jan 2025

    Full record — everyone who holds this →

    AI alignment

  • We cannot know how much safety a control measure buys until we understand elicitation, so control adds a margin of unknown size.

    However, without having a good handle on elicitation, we cannot be confident that our control techniques are effective.

    Should we control AI instead of aligning it? Said 24 Jan 2025

    Full record — everyone who holds this →

  • Human oversight of AI behaviour is already intractable at the volume models produce, so the overseers have to be models too.

    Therefore it’s simply intractable to meaningfully oversee all but a tiny fraction of our models’ behavior with actual humans.

    Two alignment threat models Said 8 Nov 2024

    Full record — everyone who holds this →

  • Today's models are severely under-elicited: they are far more capable than the way we currently ask them makes them look.

    In other words, current models are probably severely under-elicited.

    Two alignment threat models Said 8 Nov 2024

    Full record — everyone who holds this →

  • Releasing open weights is irreversible in a way that deploying a hosted model is not, so its risks cannot be mitigated later.

    This is especially severe for open weight models, where deployment is essentially irreversible and mitigations can’t be improved later.

    Two alignment threat models Said 8 Nov 2024

    Full record — everyone who holds this →

  • Once model weights leave their owner's control the loss is irreversible, which is why securing them is the precondition for governing AI at all.

    Moreover, model exfiltration is likely impossible to reverse.

    Self-exfiltration is a key dangerous capability Said 13 Sept 2023

    Full record — everyone who holds this →

  • The dangerous capability that matters most for misalignment is whether a model can exfiltrate its own weights, because that is the one that removes it from your control.

    If a model was capable of self-exfiltration, it would have the option to remove itself from your control.

    Self-exfiltration is a key dangerous capability Said 13 Sept 2023

    Full record — everyone who holds this →

    AI alignment

  • Increasing language model size does not inherently improve following of user intent.

    Making language models bigger does not inherently make them better at following a user's intent.

    Training language models to follow instructions with human feedback (with 19 co-authors) Said 4 Mar 2022

    Full record — everyone who holds this →

  • Large language models are not aligned with their users.

    For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users.

    Training language models to follow instructions with human feedback (with 19 co-authors) Said 4 Mar 2022

    Full record — everyone who holds this →

    LLMs

  • Fine-tuning with human feedback is a promising direction for aligning language models with human intent.

    Even though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent.

    Training language models to follow instructions with human feedback (with 19 co-authors) Said 4 Mar 2022

    Full record — everyone who holds this →

  • The next-token language modeling objective is misaligned with following user instructions helpfully and safely.

    This is because the language modeling objective used for many recent large LMs-predicting the next token on a webpage from the internet-is different from the objective "follow the user's instructions helpfully and safely" (Radford et al.,, 2019; Brown et al.,, 2020; Fedus et al.,, 2021; Rae et al.,, 2021; Thoppilan et al.,, 2022). Thus, we say that the language modeling objective is misaligned.

    Training language models to follow instructions with human feedback (with 19 co-authors) Said 4 Mar 2022

    Full record — everyone who holds this →

    AI alignment

  Beliefs others hold too

  • Recursive self-improvement has already started, because AI research itself is now being automated. 2 hold this

    We are starting to automate AI research and the recursive self-improvement process has begun.

    Alignment is not solved Said 22 Jan 2026

    Full record — everyone who holds this →

  • Once model weights leave their owner's control the loss is irreversible, which is why securing them is the precondition for governing AI at all. 2 hold this

    Moreover, model exfiltration is likely impossible to reverse.

    Self-exfiltration is a key dangerous capability Said 13 Sept 2023

    Full record — everyone who holds this →

  All 16 beliefs →