Musician and writer; answers readers' questions weekly at The Red Hand Files, and fronts Nick Cave and the Bad Seeds.

• Our readingkorrents.com

  What moves a listener is not the song alone but a human being straining against their own limits, which is why a machine can write a good song and never a great one.

  Their words

  What we are actually listening to is human limitation and the audacity to transcend it. Artificial Intelligence, for all its unlimited potential, simply doesn’t have this capacity.

  theredhandfiles.com

Financial journalist; chief financial correspondent at Axios and host of the Slate Money podcast, previously a blogger at Reuters and author of The Phoenix Economy.

• Likedrcmnd.app

  Google Docs

  Their words

  Now, almost everything happens in Google Docs, except for when I write straight into the Axios or Substack CMS. (Both of which are lovely, WYSIWYG, very simple and easy to use.)

  usesthis.com

One of the four authors of the 2018 paper "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", at Google AI Language, the affiliation printed on the paper.

From one piece BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2 beliefs, in the piece's order there

• Our readingkorrents.com

  Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context.

  Their words

  Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.

  arxiv.org

• Our readingkorrents.com

  Current techniques restrict pre-trained representation power because standard language models are unidirectional.

  Their words

  We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.

  arxiv.org

One of the four authors of the 2018 paper "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", at Google AI Language, the affiliation printed on the paper.

From one piece BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2 beliefs, in the piece's order there

• Our readingkorrents.com

  Current techniques restrict pre-trained representation power because standard language models are unidirectional.

  Their words

  We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.

  arxiv.org

• Our readingkorrents.com

  Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context.

  Their words

  Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.

  arxiv.org

One of the four authors of the 2018 paper "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", at Google AI Language, the affiliation printed on the paper.

From one piece BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2 beliefs, in the piece's order there

• Our readingkorrents.com

  Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context.

  Their words

  Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.

  arxiv.org

• Our readingkorrents.com

  Current techniques restrict pre-trained representation power because standard language models are unidirectional.

  Their words

  We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.

  arxiv.org

One of the four authors of the 2018 paper "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", at Google AI Language, the affiliation printed on the paper.

From one piece BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2 beliefs, in the piece's order there

• Our readingkorrents.com

  Current techniques restrict pre-trained representation power because standard language models are unidirectional.

  Their words

  We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.

  arxiv.org

• Our readingkorrents.com

  Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context.

  Their words

  Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.

  arxiv.org

Statistician and Chief Scientist at Posit (RStudio). Author of the ggplot2, dplyr and tidyverse R packages and of books on R programming and data science.

• Recommendsrcmnd.app

  The Algorithm Design Manual

  Their words

  To me, this book is an illustration of the power of names. Today, in the era of Google, if you know the name of something, you can find out about it with a simple search. But if you don't know of what you're looking for, it suddenly becomes much harder to find it. Having in the back of your head the names of common algorithms that help you solve problems is really powerful.

  fivebooks.com

London-based front-end architect, web developer evangelist and cloud consultant; Microsoft MVP. Writes at praveen.science and answers on Stack Overflow.

• Usesrcmnd.app

  Google Chrome

  Their words

  Google Chrome: What's a Web Developer without Google Chrome browser? The first thing I do with Safari is to get Google Chrome and install the latest version.

  blog.praveen.science

Associate Professor at MIT CSAIL, leading the Programming Systems Group; works on programming systems that manipulate uncertainty, from neural networks to unreliable hardware.

• Our readingkorrents.com

  Pruning cuts the parameters of a trained network by over 90% without costing accuracy, so inference gets smaller and faster for free.

  Their words

  Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy.

  arxiv.org

Chief AI Scientist at Databricks, after its acquisition of MosaicML, where he was a founding team member; known for the Lottery Ticket Hypothesis from his MIT PhD (ICLR 2019 Best Paper).

• Our readingkorrents.com

  Pruning cuts the parameters of a trained network by over 90% without costing accuracy, so inference gets smaller and faster for free.

  Their words

  Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy.

  arxiv.org

Alignment researcher; developed reinforcement learning from human feedback at OpenAI, founded the Alignment Research Center, and now works at the US AI Safety Institute.

From one piece Takeoff speeds 4 beliefs, in the piece's order there

• Our readingkorrents.com

  A slow takeoff is significantly more likely than a fast one: AI that is nearly as powerful will have transformed the world before the incredibly powerful kind arrives.

  Their words

  I currently think a slow takeoff is significantly more likely.

  Takeoff speedssideways-view.com

• Our readingkorrents.com

  Technologies are almost always preceded by slightly worse versions of themselves, and there is no reason for AI to be the exception.

  Their words

  Things are usually preceded by worse versions, even in cases where there are weak reasons to expect a discontinuous jump.

  Takeoff speedssideways-view.com

• Our readingkorrents.com

  Incredibly powerful AI will arrive into a world that is already being transformed and already alarmed, not into a world like today's.

  Their words

  In particular, this means that incredibly powerful AI will emerge in a world where crazy stuff is already happening (and probably everyone is already freaking out).

  Takeoff speedssideways-view.com

Philosopher and historian of existential risk; wrote Human Extinction, and coined "TESCREAL" with Timnit Gebru for the cluster of ideologies they argue underpins the AI industry.

From one piece Why superintelligence is a threat that should be taken seriously 2 beliefs, in the piece's order there

• Our readingkorrents.com

  Never project human mental properties onto an artificial intelligence; the difference between the two architectures is the reason to worry, not a reason not to.

  AI alignment

  Their words

  One of the most commonly emphasized points in the AI safety literature is that one must never anthropomorphize AI—that is, don’t project human mental properties onto artificially intelligent systems.

  thebulletin.org

• Our readingkorrents.com

  A superintelligence needs no consciousness, emotions or malice to be dangerous; a goal system slightly misaligned with ours is enough.

  AI alignment

  Their words

  A superintelligence whose goal system is even slightly misaligned with ours could, being far more powerful than any human or human institution, bring about human extinction for the very same reason that construction workers routinely slaughter large populations of ants.

  thebulletin.org

Game designer and programmer, co-founder of id Software and co-creator of Doom, Quake and Wolfenstein 3D. Now runs Romero Games in Galway, Ireland.

• Usesrcmnd.app

  REAPER

  Their words

  I use a ton of software. Here's a list: Xcode, Pages, Numbers, Keynote, Sublime Text, Corona SDK, Unity, Final Cut Pro, Versions, Perforce, VMWare Fusion, Messages, REAPER, Transmit, Coda, Dropbox, Skype, Dashlane, Slack, Terminal, Assembla, Airmail, Spark Mail, Google Docs, Pixelmator, Pixen, TexturePacker, Particle Designer, Tiled, iTunes + Music + Match, Screenium, Steam, Twitter, Facebook.

  usesthis.com

• Usesrcmnd.app

  Unity

  Their words

  I use a ton of software. Here's a list: Xcode, Pages, Numbers, Keynote, Sublime Text, Corona SDK, Unity, Final Cut Pro, Versions, Perforce, VMWare Fusion, Messages, REAPER, Transmit, Coda, Dropbox, Skype, Dashlane, Slack, Terminal, Assembla, Airmail, Spark Mail, Google Docs, Pixelmator, Pixen, TexturePacker, Particle Designer, Tiled, iTunes + Music + Match, Screenium, Steam, Twitter, Facebook.

  usesthis.com

Chief Scientist at Google DeepMind and Google Research, at Google since 1999, where he co-created MapReduce, Bigtable, TensorFlow and the TPU.

• Usesrcmnd.app

  emacs

  Their words

  For writing code, I use emacs and Google's internal distributed build system (a version of which was open sourced as Bazel) and our version control system

  usesthis.com

• Usesrcmnd.app

  LaTeX

  Their words

  For technical papers, I and my coauthors usually use LaTeX. For giving presentations, I almost exclusively use Google Slides.

  usesthis.com

• Usestheir ownrcmnd.app

  Google Slides

  Their words

  For giving presentations, I almost exclusively use Google Slides.

  usesthis.com

Founder and chief executive of Replit. Previously an engineer at Facebook and Codecademy. Writes essays on programming, philosophy and entrepreneurship at amasad.me.

• Recommendsrcmnd.app

  Rationality: From AI to Zombies

  Their words

  luckily I was aided by a few books that I recommend to anyone who wants to come to terms with this

  amasad.me

Programmer and writer on computer architecture, performance, and software reliability. He has worked on CPU design at Centaur Technology and on software at Google and Microsoft, and writes long-form technical essays at danluu.com.

• Mixed onaffiliate linkrcmnd.app

  Site Reliability Engineering

  Their words

  A description of how Google handles operations.

  danluu.com

Associate Professor at MIT EECS and Distinguished Scientist at Google DeepMind; first author of the Deep Residual Learning (ResNet) paper, among the most-cited papers of the century.

• Our readingkorrents.com

  Depth is not free: past a point, deeper neural networks are harder to train rather than simply better.

  Their words

  Deeper neural networks are more difficult to train.

  arxiv.org

Computer vision researcher and former Chief Scientist of Megvii, previously at Microsoft Research Asia; co-author of ResNet and Faster R-CNN. Died in 2022; this page reflects his complete public record.

• Our readingkorrents.com

  Depth is not free: past a point, deeper neural networks are harder to train rather than simply better.

  Their words

  Deeper neural networks are more difficult to train.

  arxiv.org

Interpretability researcher at Anthropic, which he co-founded; before that at OpenAI and Google Brain, where he wrote the explanations of neural networks most people learned them from.

• Our readingkorrents.com

  Standard recurrent neural networks are theoretically capable of representing long-term dependencies but fail to learn them from data in practice.

  Their words

  In theory, RNNs are absolutely capable of handling such "long-term dependencies." A human could carefully pick parameters for them to solve toy problems of this form. Sadly, in practice, RNNs don't seem to be able to learn them.

  Understanding LSTM Networkscolah.github.io