hi kids, gramps here. learn your algorithms and datastructures, cause SOTA frontier models will still fuck you up bad behind your back, especially if you don't read your code anymore.
the largest tech companies (at the top 1%) aren't using fewer tokens, they're just being cost conscious and moving to cheaper, still very capable open models running vs SOTA model usage
I have said for many years now that the thing I'm worried about with the models is a Black Monday type scenario, where many algorithms work with each other and get us into weird basins of actions.
Sometimes I get super nervous that someday the US government is really going to fuck with the web. Like they are going to decide they just don't have enough control over it and are going to get really heavy-handed somehow.
It's true that Meta's rivals do largely employ the same cocktail of video formats, recommendation algorithms, and push notifications to continuously derail their users' attention.
One of the few books that successfully brings CS into everyday life. We need more books like this. An audiobook is also available, but this one is better on paper.
So we conjectured that the lack of data was a huge part of the reason that's the lack of progress in AI. So we took a departure from everybody else who are really focusing only on algorithm and said that we need data. we need data to drive these algorithms.
Your your models can be leaked, algorithms can be replicated, but hundreds of millions of miles of fully autonomous operations in the real world, backed by evidence-grade evaluation and publicly audited proof, that is much, much more difficult to replicate.
If you think about our large scale models today, they probably see a thousand times as much data as a human does by the age of 18. Yet, the human by the age of 18 is better in a lot of things and, you know, on par uh with those frontier models that have seen way more data. So could you come up with much more data efficient systems that can learn continuously learn from their own actions?
And then I was like, okay, can you come up with an algorithm that is better than the algorithms that I came up with or that anybody else came up with and go ahead and like look at all the published work and synthesize that and then try to come up with something novel and it it's not able to do it. And I can give it a lot of time and it's it's still not able to do it.
And DSA interviews were never the best for that. Well, thinking, sure. But in terms of like does that skill translate to what you're doing on the job? It never really translated to that. It was more about evaluating like does somebody think?
I'm not going to spend four interviews going through and asking somebody data structures and algorithm stuff. I just have them do work that might be similar to something I'd give them on the job or even just have a conversation with them. See how they think. Like can they think through trade-offs? I don't even care about what answer they give me to a problem. I care about like what's like why did they say that?
This is a super exciting release - Claude Fable 5 is the same underlying model as Mythos but with added safeguards. The benchmarks are great and it's SOTA on everything by a margin but I'll add that *qualitatively* also, this is a major-version-bump-deserving step change forward
99% of people who you'd want to read your essay probably don't know you exist, or don't give a fuck, or wouldn't be bothered to engage even if they read it.
We have got to acknowledge that most of the advanc advances in AI came out of algorithm advances not just the raw hardware. Now if most advances came from algorithms and computer science and programming tell me that their army of AI researchers is not their fundamental advantage.
Given the simplicity and speed of the algorithms in this post and the increasingly small deltas between successive algorithms, perhaps we are nearing an optimal solution.
Well, there's nothing in them which will cause it to generalize. Well, the gradient descent will cause them to find a solution to the problems they've seen. And if there's only one way to solve them, you know, they they'll do it. But there are many ways to solve it. Some which generalize well, some which generalize poorly. There's nothing in them in the algorithms that will cause them to generalize well.
What the Apple paper shows, most fundamentally, regardless of how you define AGI, is that LLMs are no substitute for good well-specified conventional algorithms.
I predict (with confidence bordering on arrogance) that whatever machine-learning progress can be made will correlate extremely closely with the quality of the data around any given problem
I'm excited about what this means for the open-source community and research, with the gap between closed-source and open-weight models closing and SOTA-level conversational models being more easily accessible.
It just predicts what you are likely to re-Tweet and like, and linger on. That’s what all these algorithms do. It’s what Tik-Tok does, it’s what all these recommendation engines do. And it turns out that the thing that you are most likely to interact with is outrage. And that’s a quirk of the human condition.
Songs arise out of suffering, by which I mean they are predicated upon the complex, internal human struggle of creation and, well, as far as I know, algorithms don’t feel. Data doesn’t suffer.
The best and easiest-found-by-optimization algorithms for solving problems we want an AI to solve, readily generalize to problems we’d rather the AI not solve; you can’t build a system that only has the capability to drive red cars and not blue cars, because all red-car-driving algorithms generalize to the capability to drive blue cars.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.