Creator of the Keras deep-learning library and the ARC-AGI benchmark.

• Our reading nowkorrents.com

  The memorize-fetch-apply paradigm behind LLMs can reach arbitrary skill given training data, but it cannot adapt to novelty or acquire new skills on the fly.

  LLMsOpenAI

  Their words, before

  This "memorize, fetch, apply" paradigm can achieve arbitrary levels of skills at arbitrary tasks given appropriate training data, but it cannot adapt to novelty or pick up new skills on the fly (which is to say that there is no fluid intelligence at play here.)

  Their words now

  OpenAI's new o3 model represents a significant leap forward in AI's ability to adapt to novel tasks. This is not merely incremental improvement, but a genuine breakthrough, marking a qualitative shift in AI capabilities compared to the prior limitations of LLMs.

Cognitive scientist and long-standing critic of deep learning's claims; writes Marcus on AI and wrote Rebooting AI.

• Our readingkorrents.com

  Large language models do not reason formally: their performance collapses as a problem is made bigger, in the way a calculator's never does.

  LLMs

  Their words

  Another manifestation of the lack of sufficiently abstract, formal reasoning in LLMs is the way in which performance often fall apart as problems are made bigger.

  garymarcus.substack.com

American science-fiction writer, author of the collections Stories of Your Life and Others and Exhalation, and an essayist on artificial intelligence for The New Yorker.

• Our readingkorrents.com

  A language model is not using language at all, because language requires an intention to communicate.

  LLMs

  Their words

  But a large language model is not a writer; it’s not even a user of language. Language is, by definition, a system of communication, and it requires an intention to communicate.

  Why A.I. Isn't Going to Make Artnewyorker.com

Machine-learning researcher; has written the Lil'Log survey posts on how a model technique works since 2017, and worked at OpenAI from 2018 to 2024, latterly leading its safety systems team.

From one piece Extrinsic Hallucinations in LLMs 2 beliefs, in the piece's order there

• Our readingkorrents.com

  Using supervised fine-tuning to teach a language model new knowledge risks increasing its hallucination rate.

  LLMs

  Their words

  These empirical results from Gekhman et al. (2024) point out the risk of using supervised fine-tuning for updating LLMs' knowledge.

  Extrinsic Hallucinations in LLMslilianweng.github.io

• Our readingkorrents.com

  A language model avoids hallucinating only by being factual and admitting when it does not know the answer.

  LLMs

  Their words

  To avoid hallucination, LLMs need to be (1) factual and (2) acknowledge not knowing the answer when applicable.

  Extrinsic Hallucinations in LLMslilianweng.github.io

Co-founder and CEO of Perplexity, an AI answer engine; previously a research scientist at OpenAI and a PhD student at UC Berkeley.

• Our readingkorrents.com

  Once a language model reads the results, search can trade precision for recall, because the model does not care that the right link came ninth.

  LLMsGoogle

  Their words

  In Google, even though we call it 10 blue links, you get annoyed if you don’t even have the right link in the first three or four. The eye is so tuned to getting it right. LLMs are fine. You get the right link maybe in the 10th or ninth. You feed it in the model. It can still know that that was more relevant than the first.

  youtube.com

Astrobiologist and theoretical physicist at Arizona State University, working on assembly theory and the physics of life.

• Our readingkorrents.com

  A large language model is the genetic system for language: it stores the dynamics of a structure that until now lived only across human brains.

  LLMs

  Their words

  I think large language models are kind of like the genetic system for language in some sense, where it’s allowing an archiving that’s highly dynamic.

  youtube.com

Theoretical physicist and philosopher, Homewood Professor of Natural Philosophy at Johns Hopkins and fractal faculty at the Santa Fe Institute. Writes the Preposterous Universe blog and hosts the Mindscape podcast.

• Our readingkorrents.com

  A language model's apparent mind is mostly our own bias: it predicts text, and leverages our evolved habit of attributing intentionality to anything that acts human.

  LLMs

  Their words

  What a large language model is trying to do is to predict text. That’s what it does. And it is leveraging the fact that we human beings for very good evolutionary biology reasons, attribute intentionality and intelligence and agency to things that act like human beings.

  youtube.com

Research scientist at Meta in Berlin working on multilingual models and evaluation; led the multilingual team at Cohere and was a research scientist at Google DeepMind before that.

• Our readingkorrents.com

  Retrieval-augmented generation is the most practical approach for reducing LLM hallucinations.

  LLMs

  Their words

  Retrieval-augmented generation (RAG; Lewis et al., 2020), which conditions on the LLM's generation on retrieved documents is the most practical paradigm IMO.

  Command R+ruder.io

Research scientist at Meta in Berlin working on multilingual models and evaluation; led the multilingual team at Cohere and was a research scientist at Google DeepMind before that.

• Our readingkorrents.com

  Available online text data is becoming a limiting factor for AI training, and repeating it yields diminishing returns.

  LLMs

  Their words

  recent LLMs are reaching the limits of text data online and repeating data eventually leads to diminishing returns

  NLP Research in the Era of LLMsruder.io

Founder of Amazon in 1994 and of the space company Blue Origin in 2000.

• Our readingkorrents.com

  Large language models are discoveries rather than inventions, closer to looking through a telescope than to engineering an aircraft.

  LLMs

  Their words

  It's interesting to me that large language models in their current form are not inventions, they're discoveries. The telescope was an invention, but looking through it at Jupiter, knowing that it had moons, was a discovery.

  youtube.com

Research scientist at Meta in Berlin working on multilingual models and evaluation; led the multilingual team at Cohere and was a research scientist at Google DeepMind before that.

From one piece EMNLP 2023 Primer 3 beliefs, in the piece's order there

• Our readingkorrents.com

  The rise of closed-source large language models makes preserving an open culture of sharing knowledge and software important for NLP.

  LLMs

  Their words

  In light of the increasing number of closed-source LLMs, it is important to continue to promote an open culture of sharing knowledge, data, and software, from which the NLP community has benefited greatly.

  EMNLP 2023 Primerruder.io

• Our readingkorrents.com

  Large language models remain limited in non-English settings, making further multilingual development an important research direction.

  LLMs

  Their words

  LLMs are still limited in non-English settings and that making LLMs more multilingual is an important direction.

  EMNLP 2023 Primerruder.io

• Our readingkorrents.com

  Full fine-tuning of large language models has become too costly, making parameter-efficient fine-tuning methods necessary.

  LLMs

  Their words

  Full fine-tuning of LLMs has become prohibitive and requires parameter-efficient methods instead.

  EMNLP 2023 Primerruder.io

Theoretical physicist and cosmologist, Frank B. Baird, Jr. Professor of Science at Harvard University, working on particle physics, extra dimensions and dark matter.

• Our readingkorrents.com

  Large language models are generalisations of what we already have; the breakthrough that would let AI discover things is still waiting to happen.

  LLMs

  Their words

  Large language models are more or less generalizations of stuff that we have. There’s still breakthroughs in AI waiting to happen, and maybe they are happening and maybe they’ll be good, maybe not, but that’s not quite the same.

  youtube.com

Programmer; founded comma.ai and tinygrad, and was the first to unlock the iPhone.

From one piece George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387 2 beliefs, in the piece's order there

• Our readingkorrents.com

  AlphaGo and MuZero are far more impressive than large language models, because they play beyond the best humans while the models write middle-school essays.

  LLMs

  Their words

  I think things like MuZero and AlphaGo are so much more impressive because these things are playing beyond the highest human level. The language models are writing middle school level essays and people are like, wow, it’s a great essay.

  youtube.com

• Our readingkorrents.com

  Future language models will be smaller and loop on themselves with a retrieval system, which is what will finally let them cite their sources.

  LLMs

  Their words

  I think future LLMs are going to be smaller, but are going to run looping on themselves and are going to have retrieval systems. And the thing about using a retrieval system is you can cite sources, explicitly.

  youtube.com

Designer and engineer; formerly principal research engineer at GitHub Next; writes at wattenberger.com.

• Our readingkorrents.com

  Chatbots are a terrible interface for large language models, because a text box has no affordances.

  LLMs

  Their words

  Hopefully I've convinced you that chatbots are a terrible interface for LLMs. Or, at the very least, that we can add controls, information, and affordances to our chatbot interfaces to make them more usable.

  Why Chatbots Are Not the Futurewattenberger.com

Alignment researcher at Anthropic; co-led OpenAI's superalignment team until 2024, and writes Musings on the Alignment Problem.

• Our readingkorrents.com

  Large language models are not aligned with their users.

  LLMs

  Their words

  For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users.

  arxiv.org