Lilian Weng
Machine-learning researcher; has written the Lil'Log survey posts on how a model technique works since 2017, and worked at OpenAI from 2018 to 2024, latterly leading its safety systems team.
Lilian Weng did not write this page. What is this?
It collects the places they publish and what they have said there, each linked to the source. They have no account here. Is this you? Claim it, correct it, or ask us to remove it from ppll.
Where they publish
Blog Lil'Log Long survey posts on how a model technique works. Has a feed.
A handful of posts a year, each a worked survey of one area — agents, hallucination, diffusion, reward hacking — with the maths kept in and the papers cited.
Recent
- Harness Engineering for Self-Improvement 4 Jul 2026 The concept of recursive self-improvement (RSI) dates back to I. J. Good (1965), where he defined an “ultraintelligent machine” as a system that can surpass humans in all intellectual activities and design b…
- Scaling Laws, Carefully 24 Jun 2026 Scaling laws are one of the most critical empirical findings in deep learning. The observation is simple in form: the training loss $L$ decreases predictably as we scale up model size $N$, dataset size $D$, and compute…
- Why We Think 1 May 2025 Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post. Test time compute (Graves et al. 2016, Ling, et al. 2017, Cobbe et al. 2021) and Chain-of-thought (CoT) (Wei et al. 202…
Show 17 more
- Reward Hacking in Reinforcement Learning 28 Nov 2024 Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task. Reward hacking exi…
- Extrinsic Hallucinations in LLMs 7 Jul 2024 Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content. As a term, hallucination has been somewhat generalized to cases when the model…
- Diffusion Models for Video Generation 12 Apr 2024 Diffusion models have demonstrated strong results on image synthesis in past years. Now the research community has started working on a harder task—using it for video generation. The task itself is a superset of t…
- Thinking about High-Quality Human Data 5 Feb 2024 [Special thank you to Ian Kivlichan for many useful pointers (E.g. the 100+ year old Nature paper “Vox populi”) and nice feedback. 🙏 ] High-quality data is the fuel for modern data deep learning model train…
- Adversarial Attacks on LLMs 25 Oct 2023 The use of large language models in the real world has strongly accelerated by the launch of ChatGPT. We (including my team at OpenAI, shoutout to them) have invested a lot of effort to build default safe behavior into…
- LLM Powered Autonomous Agents 23 Jun 2023 Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM…
- Prompt Engineering 15 Mar 2023 , also known as In-Context Prompting, refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights. It is an empirical science and the effect of prompt…
- The Transformer Family Version 2.0 27 Jan 2023 Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago. Here I did a big refactoring and enrichment of that 2020 post — r…
- Large Transformer Model Inference Optimization 10 Jan 2023 [Updated on 2023-01-24: add a small section on Distillation.] Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks. They are powerful but very expensive to train and use. The ex…
- Some Math behind Neural Tangent Kernel 8 Sept 2022 Neural networks are well known to be over-parameterized and can often easily fit data with near-zero training loss with decent generalization performance on test dataset. Although all these parameters are initialized at…
- Generalized Visual Language Models 9 Jun 2022 Processing images to generate text, such as image captioning and visual question-answering, has been studied for years. Traditionally such systems rely on an object detection network as a vision encoder to capture visua…
- Learning with not Enough Data Part 3: Data Generation 15 Apr 2022 Here comes the Part 3 on learning with not enough data (Previous: Part 1 and Part 2). Let’s consider two approaches for generating synthetic data for training. Augmented data. Given a set of existing training samples, w…
- Learning with not Enough Data Part 2: Active Learning 20 Feb 2022 This is part 2 of what to do when facing a limited amount of labeled data for supervised learning tasks. This time we will get some amount of human labeling work involved, but within a budget limit, and therefore we nee…
- Learning with not Enough Data Part 1: Semi-Supervised Learning 5 Dec 2021 When facing a limited amount of labeled data for supervised learning tasks, four approaches are commonly discussed.
- How to Train Really Large Models on Many GPUs? 24 Sept 2021 [Updated on 2022-03-13: add expert choice routing.] [Updated on 2022-06-10]: Greg and I wrote a shorted and upgraded version of this post, published on OpenAI Blog: “Techniques for Training Large Neural Networks&r…
- What are Diffusion Models? 11 Jul 2021 [Updated on 2021-09-19: Highly recommend this blog post on score-based generative modeling by Yang Song (author of several key papers in the references)]. [Updated on 2022-08-27: Added classifier-free guidance, GLIDE, u…
- Contrastive Representation Learning 31 May 2021 The goal of contrastive representation learning is to learn such an embedding space in which similar sample pairs stay close to each other while dissimilar ones are far apart. Contrastive learning can be applied to both…
Link verified 20 Sept 2026. Recent items update automatically from the channel.
GitHub @lilianweng Code, and the source of the blog.
Recent
- lilianweng.github.io — My personal page 14 Jul 2026 CommitsAdd a few more papers into harness post
- generalization-experiment — TBA 22 Mar 2019 CommitsInitial commit
- ScholarFame — Small analysis about fame and productivity of scholars 14 Feb 2014 CommitsInitial commit
Show 2 more
- BagSaleDetector — Handbag Sale Detector 3 Aug 2012 Commitschange constants · reformat the email + sort by discount · minor fix
- BlankTabMultivariateTestAddon — The merged add-on of new tab page with multiple designs, used for firefox testpilot study 2 Aug 2011 Commitsnew xpi for 7.0a2 · removing blocked/pinned sites in fallbacks · small bugs
Link verified 20 Sept 2026. Recent items update automatically from the channel.
Beliefs
Korrents What they believe 16 beliefs — each backed by an exact quote.
Each is a — compiled by korrents.com, not by them: the one-line wordings are korrents', the quotes are theirs.
Recent
The deployment layer surrounding a model is as important as the model's raw intelligence.
the layer between the raw model and the real-world context seems to be as important as the model's raw intelligence
Harness Engineering for Self-Improvement Said 4 Jul 2026
Recursive self-improvement only works if the base model is already capable enough to improve its own mechanism.
Recursive structure alone is not enough. The base model must be capable enough to improve the mechanism.
Harness Engineering for Self-Improvement Said 4 Jul 2026
Harness improvements help deploy a model better, but the model's underlying intelligence remains the core driver of capability.
harness improvement enables better deployment of the model but intelligence is still the core.
Harness Engineering for Self-Improvement Said 4 Jul 2026
Show 13 more
As models keep growing, AI is running out of enough high-quality unique training tokens to keep up.
As the model size grows significantly, we are running out of enough high-quality unique tokens.
Scaling Laws, Carefully Said 24 Jun 2026
Large language models cannot reliably self-correct their own mistakes without external feedback.
this self-correction capability turns out to not exist intrinsically among LLMs and does not easily work out of the box, due to various failure modes
Why We Think Said 1 May 2025
Longer test-time thinking improves an AI model's robustness to adversarial or unusual inputs.
thinking for longer should be especially useful when the model is presented with an unusual input, such as an adversarial example or jailbreak attempt
Why We Think Said 1 May 2025
Chain-of-thought explanations from language models can be biased because training does not explicitly reward faithful reasoning.
model CoTs could be biased due to lack of explicit training objectives aimed at encouraging faithful reasoning
Why We Think Said 1 May 2025
It is fundamentally difficult to design a reward function that accurately captures the intended goal in reinforcement learning.
Reward hacking exists because RL environments are often imperfect, and it is fundamentally challenging to accurately specify a reward function.
Reward Hacking in Reinforcement Learning Said 28 Nov 2024
More capable AI agents are more likely to find and exploit flaws in their reward functions.
A more intelligent agent is more capable of finding "holes" in the design of reward function and exploiting the task specification-in other words, achieving higher proxy rewards but lower true rewards.
Reward Hacking in Reinforcement Learning Said 28 Nov 2024
A language model avoids hallucinating only by being factual and admitting when it does not know the answer.
To avoid hallucination, LLMs need to be (1) factual and (2) acknowledge not knowing the answer when applicable.
Extrinsic Hallucinations in LLMs Said 7 Jul 2024
Using supervised fine-tuning to teach a language model new knowledge risks increasing its hallucination rate.
These empirical results from Gekhman et al. (2024) point out the risk of using supervised fine-tuning for updating LLMs' knowledge.
Extrinsic Hallucinations in LLMs Said 7 Jul 2024
Video generation requires more world knowledge than image generation because of the need for temporal consistency across frames.
It has extra requirements on temporal consistency across frames in time, which naturally demands more world knowledge to be encoded into the model.
Diffusion Models for Video Generation Said 12 Apr 2024
High-quality video data, especially paired with text, is harder to collect at scale than image or text data.
In comparison to text or images, it is more difficult to collect large amounts of high-quality, high-dimensional video data, let along text-video pairs.
Diffusion Models for Video Generation Said 12 Apr 2024
High-quality data collection depends more on careful human execution than on machine learning techniques alone.
Lots of ML techniques in the post can help with data quality, but fundamentally human data collection involves attention to details and careful execution.
Thinking about High-Quality Human Data Said 5 Feb 2024
Higher-perplexity adversarial attack text is easier to detect and mitigate.
High perplexity makes an attack more vulnerable to be detected and mitigated.
Adversarial Attacks on LLMs Said 25 Oct 2023
Universal adversarial trigger attacks are easy to detect because the learned trigger tokens tend to be nonsensical.
One drawback with UAT (Universal Adversarial Trigger) attacks is that it is easy to detect them because the learned triggers are often nonsensical.
Adversarial Attacks on LLMs Said 25 Oct 2023
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
Feed
As its own page →Hiding
14 July
4 July
From one piece Harness Engineering for Self-Improvement 3 beliefs · lilianweng.github.io
-
Their words
harness improvement enables better deployment of the model but intelligence is still the core.
Harness Engineering for Self-Improvementlilianweng.github.io
-
korrents.com
Recursive self-improvement only works if the base model is already capable enough to improve its own mechanism.Their words
Recursive structure alone is not enough. The base model must be capable enough to improve the mechanism.
Harness Engineering for Self-Improvementlilianweng.github.io
-
korrents.com
The deployment layer surrounding a model is as important as the model's raw intelligence.Their words
the layer between the raw model and the real-world context seems to be as important as the model's raw intelligence
Harness Engineering for Self-Improvementlilianweng.github.io
24 June
-
korrents.com
As models keep growing, AI is running out of enough high-quality unique training tokens to keep up.Their words
As the model size grows significantly, we are running out of enough high-quality unique tokens.
1 May 2025
From one piece Why We Think 3 beliefs · lilianweng.github.io
-
Their words
model CoTs could be biased due to lack of explicit training objectives aimed at encouraging faithful reasoning
-
korrents.com
Longer test-time thinking improves an AI model's robustness to adversarial or unusual inputs.Their words
thinking for longer should be especially useful when the model is presented with an unusual input, such as an adversarial example or jailbreak attempt
-
korrents.com
Large language models cannot reliably self-correct their own mistakes without external feedback.Their words
this self-correction capability turns out to not exist intrinsically among LLMs and does not easily work out of the box, due to various failure modes
28 November 2024
From one piece Reward Hacking in Reinforcement Learning 2 beliefs · lilianweng.github.io
-
korrents.com
More capable AI agents are more likely to find and exploit flaws in their reward functions.Their words
A more intelligent agent is more capable of finding "holes" in the design of reward function and exploiting the task specification-in other words, achieving higher proxy rewards but lower true rewards.
Reward Hacking in Reinforcement Learninglilianweng.github.io
-
Their words
Reward hacking exists because RL environments are often imperfect, and it is fundamentally challenging to accurately specify a reward function.
Reward Hacking in Reinforcement Learninglilianweng.github.io
7 July 2024
From one piece Extrinsic Hallucinations in LLMs 2 beliefs · lilianweng.github.io
-
korrents.com
Using supervised fine-tuning to teach a language model new knowledge risks increasing its hallucination rate.Their words
These empirical results from Gekhman et al. (2024) point out the risk of using supervised fine-tuning for updating LLMs' knowledge.
-
korrents.com
A language model avoids hallucinating only by being factual and admitting when it does not know the answer.Their words
To avoid hallucination, LLMs need to be (1) factual and (2) acknowledge not knowing the answer when applicable.
12 April 2024
From one piece Diffusion Models for Video Generation 2 beliefs · lilianweng.github.io
-
korrents.com
High-quality video data, especially paired with text, is harder to collect at scale than image or text data.Their words
In comparison to text or images, it is more difficult to collect large amounts of high-quality, high-dimensional video data, let along text-video pairs.
-
Their words
It has extra requirements on temporal consistency across frames in time, which naturally demands more world knowledge to be encoded into the model.
5 February 2024
-
korrents.com
High-quality data collection depends more on careful human execution than on machine learning techniques alone.Their words
Lots of ML techniques in the post can help with data quality, but fundamentally human data collection involves attention to details and careful execution.
25 October 2023
From one piece Adversarial Attacks on LLMs 2 beliefs · lilianweng.github.io
-
korrents.com
Universal adversarial trigger attacks are easy to detect because the learned trigger tokens tend to be nonsensical.Their words
One drawback with UAT (Universal Adversarial Trigger) attacks is that it is easy to detect them because the learned triggers are often nonsensical.
-
Their words
High perplexity makes an attack more vulnerable to be detected and mitigated.
23 June 2023
15 March 2023
27 January 2023
10 January 2023
8 September 2022
9 June 2022
15 April 2022
20 February 2022
5 December 2021
24 September 2021
11 July 2021
31 May 2021
22 March 2019
14 February 2014
3 August 2012
2 August 2011
Nothing matches.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.