Related posts

Machine-learning researcher on open language models; writes the Interconnects newsletter and the RLHF Book, after leading post-training at Ai2.

• Our readingkorrents.com

  Open models will end up serving a long-tail of specialized uses while closed models retain the most valuable applications.

  Their words

  open models are still incredibly useful, but fill a long-tail ecosystem relative to the closed counterparts that have monopoly ownership stakes in the most valuable areas like knowledge work collaboration, drug discovery, SWE, etc.

  Teaching Everyone to Fish for Tokensinterconnects.ai

AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).

• Our readingkorrents.com

  As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context.

  LLMs

  Their words

  As reasoning models and agent workflows keep more tokens around (for longer), KV-cache size, memory traffic, and attention cost quickly become the main constraints, and LLM developers are adding a growing number of architecture tricks to reduce those costs.

  magazine.sebastianraschka.com

Economist at George Mason University, co-author of Marginal Revolution, host of Conversations with Tyler. Writes in short, dense claims about economics, culture and AI, and has moved position on AI in public more than once.

• Likedrcmnd.app

  Preparing for War: The Making of the Geneva Conventions

  Their words

  A very good look at the negotiations behind the Geneva Conventions of 1949 and just how driven by national self-interest they were, including colonial motives from the major colonial powers, who wished to retain stronger rights to put down uprisings.

  marginalrevolution.com

Head of growth at Chess.com, previously growth and monetization lead at Grammarly and Duolingo and a product manager at Google.

• Our readingkorrents.com

  Retention is what makes a consumer subscription business possible: without it the whole burden falls on getting a stranger to pay on day one.

  Their words

  Yeah, I mean user retention is gold for consumer subscription companies. If you don't retain your users, then a lot of the onus is on getting them to pay on like day one. That's super hard, right?

  youtube.com

Creator of the Keras deep-learning library and the ARC-AGI benchmark.

• Our reading nowkorrents.com

  The memorize-fetch-apply paradigm behind LLMs can reach arbitrary skill given training data, but it cannot adapt to novelty or acquire new skills on the fly.

  LLMsOpenAI

  Their words, before

  This "memorize, fetch, apply" paradigm can achieve arbitrary levels of skills at arbitrary tasks given appropriate training data, but it cannot adapt to novelty or pick up new skills on the fly (which is to say that there is no fluid intelligence at play here.)

  Their words now

  OpenAI's new o3 model represents a significant leap forward in AI's ability to adapt to novel tasks. This is not merely incremental improvement, but a genuine breakthrough, marking a qualitative shift in AI capabilities compared to the prior limitations of LLMs.

Neuroscientist at UC Davis; studies memory and wrote Why We Remember.

From one piece Charan Ranganath: Human Memory, Imagination, Deja Vu, and False Memories | Lex Fridman Podcast #430 2 beliefs, in the piece's order there

• Our readingkorrents.com

  You only retain what you were pushed to your limit on: if learning feels like coasting, you are not learning.

  Their words

  So that’s why I think this is a thing that I come back to over and over again, is that you will retain information better if you’re constantly pushing yourself to your limit. If you are feeling like you’re coasting, then you’re actually not learning

  youtube.com

• Our readingkorrents.com

  Spacing study out beats massing it: half an hour today and half an hour a week from now is retained longer than two half-hours in one day.

  Their words

  if I just try to do half-an-hour and then later in the day I do half-an-hour, I won’t retain that information as long as if I do half-an-hour today and half-an-hour one week from now. So doing that extra spacing should help me retain the information better.

  youtube.com

Experimental psychologist and author of the blog Experimental History, on psychology, science reform and creativity.

• Our readingkorrents.com

  Learning a technical skill turns less on memorising its parts than on accepting that you will feel stupid and carrying on anyway.

  Their words

  The students who ultimately succeed in learning R are not the ones who force themselves to memorize functions or do a bunch of coding drills.

  experimental-history.com