Christopher Olah
Interpretability researcher at Anthropic, which he co-founded; before that at OpenAI and Google Brain, where he wrote the explanations of neural networks most people learned them from.
Christopher Olah did not write this page. ppll compiled it from their own words, each linked to the source. Is this you? · Report a mistake
Latest GitHub
What they believe
-
Sharing credit generously benefits everyone because it encourages future collaboration.
Credit isn't zero sum. It is often in everyone's benefit to be generous with credit, because it creates an incentive for others to help in the future.
colah.github.ioSaid 30 May 2019This on korrents
-
Deep learning will be understood very differently in thirty years than it is today.
I think it is very likely that, reflecting back in 30 years, we will see deep learning very differently.
colah.github.ioSaid 3 Sept 2015This on korrents
The bold line is korrents’ wording; the quote is theirs. These 2: what they wrote before what they said aloud, one per source, newest first.
Feed
As its own page →Hiding
4 November
18 May
4 January 2024
9 January 2021
-
Lovedrcmnd.app
The Structure of Scientific RevolutionsTheir words
Kuhn’s “The Structure of Scientific Revolutions” is one of my favorite books, and you can get an audio book!
-
Recommendsrcmnd.app
You and Your ResearchTheir words
Chris comment: This is a great question. See also Hamming's famous "You and Your Research" talk.
30 May 2019
From one piece Collaboration and Credit Principles 3 beliefs · colah.github.io
-
Their words
Remember that you are likely overestimating your own contributions.
-
korrents.com
Failing to credit someone who deserves it does more harm than mistakenly crediting someone who doesn't.Their words
the harm of not crediting someone who deserves it is often greater than the harm of crediting someone who doesn't deserve it.
-
korrents.com
Sharing credit generously benefits everyone because it encourages future collaboration.Their words
Credit isn't zero sum. It is often in everyone's benefit to be generous with credit, because it creates an incentive for others to help in the future.
20 November 2018
30 December 2017
19 November 2017
10 March 2017
13 January 2016
6 September 2015
3 September 2015
From one piece Neural Networks, Types, and Functional Programming 2 beliefs · colah.github.io
-
Their words
The known connection between geometry, logic, topology, and functional programming suggests that the connections between representations and types may be of fundamental significance.
-
Their words
I think it is very likely that, reflecting back in 30 years, we will see deep learning very differently.
31 August 2015
From one piece Calculus on Computational Graphs: Backpropagation 2 beliefs · colah.github.io
-
korrents.com
An idea can look obvious in hindsight only because the insight that makes it obvious was itself the hard discovery.Their words
It also wasn't obvious that derivatives were the right way to train them. Those are only obvious once you realize you can quickly calculate derivatives. There was a circular dependency.
-
Their words
That's the benefit of hindsight. Once you've framed the question, the hardest work is already done.
27 August 2015
From one piece Understanding LSTM Networks 2 beliefs · colah.github.io
-
Their words
LSTMs are explicitly designed to avoid the long-term dependency problem. Remembering information for long periods of time is practically their default behavior, not something they struggle to learn!
-
Their words
In theory, RNNs are absolutely capable of handling such "long-term dependencies." A human could carefully pick parameters for them to solve toy problems of this form. Sadly, in practice, RNNs don't seem to be able to learn them.
30 April 2015
30 March 2015
16 January 2015
From one piece Visualizing Representations: Deep Learning and Human Beings 2 beliefs · colah.github.io
-
Their words
The problem is that in dimensionality reduced plots of high-dimensional data, position doesn't explain the data points.
-
korrents.com
Understanding a machine learning model and understanding the data it operates on are inseparable tasks.Their words
Understanding data and understanding models that work on that data are intimately linked.
9 October 2014
13 July 2014
Nothing matches.
Beliefs
Korrents What they believe 11 beliefs — each backed by an exact quote.
Each is a — compiled by korrents.com, not by them: the one-line wordings are korrents', the quotes are theirs.
Recent
Sharing credit generously benefits everyone because it encourages future collaboration.
Credit isn't zero sum. It is often in everyone's benefit to be generous with credit, because it creates an incentive for others to help in the future.
Collaboration and Credit Principles Said 30 May 2019
Failing to credit someone who deserves it does more harm than mistakenly crediting someone who doesn't.
the harm of not crediting someone who deserves it is often greater than the harm of crediting someone who doesn't deserve it.
Collaboration and Credit Principles Said 30 May 2019
People tend to overestimate their own contributions to collaborative work.
Remember that you are likely overestimating your own contributions.
Collaboration and Credit Principles Said 30 May 2019
Show 8 more
Deep learning will be understood very differently in thirty years than it is today.
I think it is very likely that, reflecting back in 30 years, we will see deep learning very differently.
Neural Networks, Types, and Functional Programming Said 3 Sept 2015
The connection between representations and types may be fundamentally significant, akin to links among geometry, logic, topology and functional programming.
The known connection between geometry, logic, topology, and functional programming suggests that the connections between representations and types may be of fundamental significance.
Neural Networks, Types, and Functional Programming Said 3 Sept 2015
Once a problem is correctly framed, the hardest part of solving it is already done.
That's the benefit of hindsight. Once you've framed the question, the hardest work is already done.
Calculus on Computational Graphs: Backpropagation Said 31 Aug 2015
An idea can look obvious in hindsight only because the insight that makes it obvious was itself the hard discovery.
It also wasn't obvious that derivatives were the right way to train them. Those are only obvious once you realize you can quickly calculate derivatives. There was a circular dependency.
Calculus on Computational Graphs: Backpropagation Said 31 Aug 2015
Standard recurrent neural networks are theoretically capable of representing long-term dependencies but fail to learn them from data in practice.
In theory, RNNs are absolutely capable of handling such "long-term dependencies." A human could carefully pick parameters for them to solve toy problems of this form. Sadly, in practice, RNNs don't seem to be able to learn them.
Understanding LSTM Networks Said 27 Aug 2015
LSTM architectures are designed so that retaining information over long time spans is their default behavior rather than something they must struggle to learn.
LSTMs are explicitly designed to avoid the long-term dependency problem. Remembering information for long periods of time is practically their default behavior, not something they struggle to learn!
Understanding LSTM Networks Said 27 Aug 2015
Understanding a machine learning model and understanding the data it operates on are inseparable tasks.
Understanding data and understanding models that work on that data are intimately linked.
Visualizing Representations: Deep Learning and Human Beings Said 16 Jan 2015
In a dimensionality-reduced visualization of high-dimensional data, the raw position of points carries no meaning on its own.
The problem is that in dimensionality reduced plots of high-dimensional data, position doesn't explain the data points.
Visualizing Representations: Deep Learning and Human Beings Said 16 Jan 2015
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
Recommends rcmnd
rcmnd What they recommend 2 things — each backed by an exact quote.
Compiled by rcmnd.app, not by them: the quotes are theirs, and each links to the post it was copied from.
Recent
Recommends You and Your Research
Chris comment: This is a great question. See also Hamming's famous "You and Your Research" talk.
colah.github.io 9 Jan 2021
Loved The Structure of Scientific Revolutions
Kuhn’s “The Structure of Scientific Revolutions” is one of my favorite books, and you can get an audio book!
colah.github.io 9 Jan 2021
What others say about The Structure of Scientific Revolutions →
Where they publish
Blog colah's blog Long illustrated essays on how neural networks work.
The essays that taught a generation what an LSTM, an embedding or a feature visualisation is — few and far between now, and still the reference.
Link verified 20 Sept 2026.
x.com @ch402 Posts, mostly on interpretability.
Link verified 20 Sept 2026. Recent items update automatically from the channel.
GitHub @colah Code, and the source of the blog.
Recent
- colah.github.io 4 Nov 2026
- surfcad — Surface-Oriented CAD Library for Python 4 Jan 2024
- NN-Topology-Post — A blog post exploring a connection between neural networks and topology 20 Nov 2018
Show 11 more
- Conv-Nets-Series — A series of blog posts on convolutional neural networks and their generalizations. 30 Dec 2017
- Visualizing-Representations 19 Nov 2017
- Visualizing-Deep-Learning — A series of blog posts on visualizing deep learning. 10 Mar 2017
- Motivated-Topology — A topology textbook with a hubristic title 13 Jan 2016
- group-conv-post 6 Sept 2015
- LabeledTensor — Experiment with improving NumPy syntax 30 Apr 2015
- NN-Group-1-Code — Code for a discussion group I'm running 30 Mar 2015
- Fanfiction-Graphs-Post — A blog post on analyzing fanfiction through graphs! 9 Oct 2014
- NLP-RNNs-Representations-Post — A blog post using word embeddings and RNNs to explain representations. 9 Oct 2014
- Gathered-Correspondences — Email exchanges 13 Jul 2014
- DL-Types-FP-posts — Blog post exploring connection between Deep Learning, Types and Functional Programming 13 Jul 2014
Link verified 20 Sept 2026. Recent items update automatically from the channel.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.