What public figures publish and believe, in their own words.
About this feed
Highlights: posts that did unusually well for the person who wrote them, everything
they published at length, each release and new project, and every belief — at most
two a day from anyone. Day by day, newest
day first; within a day, the people with the most beliefs on this site come first. Nothing
else orders it. Show everything instead.
The quoted blocks are what people actually said; a beneath one is
the belief those words support, in korrents' wording. Nobody here wrote their own page.
Top people are the people in this feed with the most beliefs on this site, then the
most here. Choose an area and the row leads with the people whose beliefs are about it;
tap a face for their feed.
I think it's actually the opposite which is that everyone's kind of using the same data which is you need such an enormous amount of generalized text that the amount that Google has or that meta has is not actually enough to move to be a kind of fundamental difference in what you can train with.
You see this for the music now. You can generate new music. It could generate new stuff that you wouldn't know. For an LLM variance is bad. originality is is is a lower score. So what's the feedback loop for original but good?
there's no apparent equivalent in LLM right now there's no reason why the LMS get better because more people use them now that may come you have that open AI and people have doing memory where it remembers what else you've asked, but that seems more like a switching cost than a network effect.
I initially started making Anki (well, Mochi) cards to keep track of Google DeepMind's LLM lineage—Gopher, Chinchilla, Gato, PaLM, Sparrow, Meena, LaMDA, Bard, Gemini, etc.
This means that working on neural networks is NOT getting us closer to AGI, except indirectly.
Their words now
It’s become very difficult for me to maintain the belief in the stupidity of ChatGPT when every time I laugh at it, it ends up ridiculing me 6 months later.
The success of LLMs can thus be seen as vindicating semantic inferentialism against earlier, symbolic approaches to AI that tried and failed to explicate the rules of ordinary language using formal logic.
Worse, as the latest Apple papers shows, LLMs may well work on your easy test set (like Hanoi with 4 discs) and seduce you into thinking it has built a proper, generalizable solution when it does not.
What the Apple paper shows, most fundamentally, regardless of how you define AGI, is that LLMs are no substitute for good well-specified conventional algorithms.
the text embeddings across LLMs appear to largely converge on a "universal geometry" despite differing architectures, parameter counts, and training sets.
The problem is that the conversational interface is potent and that the AI is trained on a lot of human text input which unfortunately is probably enough to do real damage if that conversational interface is hooked up with something that has real world consequences.
Their words now
While all this is happening, I’ve found myself reflecting a lot on what AI means to the world and I am becoming increasingly optimistic about our future. It’s obvious now that we’re undergoing a tremendous shift.
LLMs can write a large fraction of all the tedious code you’ll ever need to write. And most code on most projects is tedious. LLMs drastically reduce the number of things you’ll ever need to Google.
I think there's this whole idea, I call it a denial of attention. I think there's an entire attack vector that's going to be happening. We're using LLMs to generate fake bug reports, fake all these things to just actually effectively to demotivate and hurt open source maintainers.
And so if you don't have those hard skills and you're not ultimately the driver at the end of the day, you're going to really find some hard times, and your ability to progress will be directly bound to how good the LLMs are.
It doesn't actually care about the craft. But when you work with an intern or you work with somebody else, they care. When they factor something, they actually go over and go, "Oh yeah, this is actually kind of bad. I'm going to come back to that." They finish this, they go back over here and they make this even better. They actually care about the thing itself.
This is a command line tool with plenty of plugins that lets you prompt different models. Think of it as a command-line version of Open WebUI. It’s particularly useful for quick scripting and basic automation.
Overall, though, this experience reinforced my belief that the tooling and interface design around LLMs is lagging way behind the actual capabilities, and is an area of active experimentation and development, even aside from any future model improvements.
There’s still plenty to worry about with respect to the environmental impact of the great AI datacenter buildout, but a lot of the concerns over the energy cost of individual prompts are no longer credible.
The Rattle Bag edited by Seamus Heaney and Ted Hughes - In an age of LLM-generated poetry and machine learning curation (even coming from yours truly sometimes), it's refreshing to have real people who love poetry curate it and show you what you need to read to touch grass.
But they said, "Look, in the end, actually the only way to actually get a real brain in the VAT is actually to have a brain in a body." And it could be a robot body, but you still need a brain in the body. So I don't think LLMs will get there because they can't. You really need to be embedded in a world, at least that's the E-four idea.
This "memorize, fetch, apply" paradigm can achieve arbitrary levels of skills at arbitrary tasks given appropriate training data, but it cannot adapt to novelty or pick up new skills on the fly (which is to say that there is no fluid intelligence at play here.)
Their words now
OpenAI's new o3 model represents a significant leap forward in AI's ability to adapt to novel tasks. This is not merely incremental improvement, but a genuine breakthrough, marking a qualitative shift in AI capabilities compared to the prior limitations of LLMs.
Another manifestation of the lack of sufficiently abstract, formal reasoning in LLMs is the way in which performance often fall apart as problems are made bigger.
But a large language model is not a writer; it’s not even a user of language. Language is, by definition, a system of communication, and it requires an intention to communicate.
In Google, even though we call it 10 blue links, you get annoyed if you don’t even have the right link in the first three or four. The eye is so tuned to getting it right. LLMs are fine. You get the right link maybe in the 10th or ninth. You feed it in the model. It can still know that that was more relevant than the first.
What a large language model is trying to do is to predict text. That’s what it does. And it is leveraging the fact that we human beings for very good evolutionary biology reasons, attribute intentionality and intelligence and agency to things that act like human beings.
Retrieval-augmented generation (RAG; Lewis et al., 2020), which conditions on the LLM's generation on retrieved documents is the most practical paradigm IMO.
It's interesting to me that large language models in their current form are not inventions, they're discoveries. The telescope was an invention, but looking through it at Jupiter, knowing that it had moons, was a discovery.
In light of the increasing number of closed-source LLMs, it is important to continue to promote an open culture of sharing knowledge, data, and software, from which the NLP community has benefited greatly.
Large language models are more or less generalizations of stuff that we have. There’s still breakthroughs in AI waiting to happen, and maybe they are happening and maybe they’ll be good, maybe not, but that’s not quite the same.
I think things like MuZero and AlphaGo are so much more impressive because these things are playing beyond the highest human level. The language models are writing middle school level essays and people are like, wow, it’s a great essay.
I think future LLMs are going to be smaller, but are going to run looping on themselves and are going to have retrieval systems. And the thing about using a retrieval system is you can cite sources, explicitly.
Hopefully I've convinced you that chatbots are a terrible interface for LLMs. Or, at the very least, that we can add controls, information, and affordances to our chatbot interfaces to make them more usable.
For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.