Christopher Olah

From one piece Collaboration and Credit Principles 3 beliefs, in the piece's order there

• Our readingkorrents.com

  People tend to overestimate their own contributions to collaborative work.

  Their words

  Remember that you are likely overestimating your own contributions.

  Collaboration and Credit Principlescolah.github.io

• Our readingkorrents.com

  Failing to credit someone who deserves it does more harm than mistakenly crediting someone who doesn't.

  Their words

  the harm of not crediting someone who deserves it is often greater than the harm of crediting someone who doesn't deserve it.

  Collaboration and Credit Principlescolah.github.io

• Our readingkorrents.com

  Sharing credit generously benefits everyone because it encourages future collaboration.

  Their words

  Credit isn't zero sum. It is often in everyone's benefit to be generous with credit, because it creates an incentive for others to help in the future.

  Collaboration and Credit Principlescolah.github.io

From one piece Neural Networks, Types, and Functional Programming 2 beliefs, in the piece's order there

• Our readingkorrents.com

  The connection between representations and types may be fundamentally significant, akin to links among geometry, logic, topology and functional programming.

  Their words

  The known connection between geometry, logic, topology, and functional programming suggests that the connections between representations and types may be of fundamental significance.

  colah.github.io

• Our readingkorrents.com

  Deep learning will be understood very differently in thirty years than it is today.

  Their words

  I think it is very likely that, reflecting back in 30 years, we will see deep learning very differently.

  colah.github.io

From one piece Calculus on Computational Graphs: Backpropagation 2 beliefs, in the piece's order there

• Our readingkorrents.com

  An idea can look obvious in hindsight only because the insight that makes it obvious was itself the hard discovery.

  Their words

  It also wasn't obvious that derivatives were the right way to train them. Those are only obvious once you realize you can quickly calculate derivatives. There was a circular dependency.

  colah.github.io

• Our readingkorrents.com

  Once a problem is correctly framed, the hardest part of solving it is already done.

  Their words

  That's the benefit of hindsight. Once you've framed the question, the hardest work is already done.

  colah.github.io

From one piece Understanding LSTM Networks 2 beliefs, in the piece's order there

• Our readingkorrents.com

  LSTM architectures are designed so that retaining information over long time spans is their default behavior rather than something they must struggle to learn.

  Their words

  LSTMs are explicitly designed to avoid the long-term dependency problem. Remembering information for long periods of time is practically their default behavior, not something they struggle to learn!

  Understanding LSTM Networkscolah.github.io

• Our readingkorrents.com

  Standard recurrent neural networks are theoretically capable of representing long-term dependencies but fail to learn them from data in practice.

  Their words

  In theory, RNNs are absolutely capable of handling such "long-term dependencies." A human could carefully pick parameters for them to solve toy problems of this form. Sadly, in practice, RNNs don't seem to be able to learn them.

  Understanding LSTM Networkscolah.github.io

From one piece Visualizing Representations: Deep Learning and Human Beings 2 beliefs, in the piece's order there

• Our readingkorrents.com

  In a dimensionality-reduced visualization of high-dimensional data, the raw position of points carries no meaning on its own.

  Their words

  The problem is that in dimensionality reduced plots of high-dimensional data, position doesn't explain the data points.

  colah.github.io

• Our readingkorrents.com

  Understanding a machine learning model and understanding the data it operates on are inseparable tasks.

  Their words

  Understanding data and understanding models that work on that data are intimately linked.

  colah.github.io