one of the things that made Steve Jobs, Steve Jobs, was how in command he was of the facts - yes, even those sprinkled with reality distortion dust! - on stage.
But alignment is no solution: it is an unsolved scientific and technical problem whose solutions—to the extent that we have them—cannot simply be imposed on every AI company operating on Earth. You should expect for highly capable, poorly aligned, self-sovereign agents to exist alongside you in the world.
I mean I think like if you look at uh my colleagues work on say alpha fold that was a very specific model for uh protein folding and it was highly successful um and was able to really handle that domain quite well so that all of a sudden you now have this amazing tool and model that can give you answers to questions about proteins and their structure um really effectively um but it's not a general model it's a very specific one and there are other I domains where that kind of approach can work really well. Uh maybe in material science or chip design or things like that that uh will enable you to leverage the capabilities of a very accurate but but niche model uh to do things that are hard today.
Everything is a trade-off. It's not really In engineering, there's no correct answer like there is in math or science. Engineering is about the best solution at the time.
The result is something that's beautiful, sharp, critical and lingering. Long after I closed the cover, I found myself mulling over the delicate ways that Lai raised the contradictions, sorrows and beauty of queer love, racial identity, camaraderie, self-control, and self-indulgence. Lai's characters have no answers, only questions that can never be fully resolved. Instead, these questions are the defining puzzles, defeats and triumphs of their lives.
And so you sort of you've you've pursued very similar ideas. It's been very successful in one case and it's been completely and utterly unsuccessful in the other case. And I think I mean a priori, you can't tell which of these is the thing to do. And you actually need to do both.
And one of the things I noticed is for 25 years we've kind of had the answers. Somebody comes to us and says we have too many bugs or like, all right, well, here's how you write tests. Oh, I can't write tests. Well, here's how you design so you can write tests. It's just kind of press play uh on the recorder. And the thing that's changed is at this moment nobody knows the answers to anything.
I don't think that human preference is just an equation that people can just like package nicely and like, "Hey, ask people like which which of these two answers people would prefer?" It's very very personal, very culturally dependent, geographically dependent, age dependent.
they they're like, "Yeah, you know, I use Cursor, and I I ask it questions sometimes, and I'm really impressed with the answers, and then I review its code really carefully, and then I check it in." And I'm like, "Dude, you're going to get fired, and you're one of the best engineers I know."
What you want to do is say what made you cancel? In other words, what about the product or situation or whatever caused the cancellation? Just phrasing it that way, you get much better results.
Only 17% of those who got no correct answers on historical development changes thought the world would be better off. 62% of those with more than five correct answers did.
And that's the thing, it's the pool of data, and I think that's what people miss. We as paleontologists get caught up on single superlative specimens and then try and treat them as a silver bullet almost.
You can have AI give a lot of context, but one of our important design goals though, is when you come to Google Search, you are going to get a lot of context, but you're going to go and find a lot of things out on the web. So that will be true in AI mode, in AI overviews, and so on.
Now that we have the answers, it all looks very obvious, and mostly straightforward.
Their words now
I should have been pointing out red flags starting back at WWDC last year, and I am embarrassed and sorry that I didn’t see what should have been very clear to me from the start.
And humans are actually very good at reading or judging between two things versus... This goes back to the core of what RLHF and preference tuning is that it's hard to generate a good answer for a lot of problems, but it's easy to see which one is better.
And these reasoning behaviors emerge naturally. So these things like, "Wait, let me see. Wait, let me check this. Oh, that might be a mistake." And they emerge from only having questions and answers.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.