foundation is ostensibly a sci-fi series about empires, psychohistory, and hard technology, but really it's about individuals with extreme agency and puzzle-solving ability having wildly outsized impact on a universe that would otherwise just sit there waiting to decay
The only way to avoid extreme power concentration is to ensure we have multiple independent providers of frontier AI models, including open-source options.
I think one one thing that um feels especially concerning to me about this whole incident is that this might be the clearest warning shot we ever get for loss of control.
the other extreme is if we're serious about exploring the universe and going to the stars, we are going to have to adapt ourselves to that environment. We're not going to export Earth with us everywhere we go.
my expectation is what we would see from then is that the rate of problematic behavior would decrease uh and would just keep decreasing and decrease at a pretty fast rate while simultaneously the worst things that the AIS would sometimes do would get more extreme, more egregious, and more scary.
Third, raw daily data is the wrong tool for this question. Time of observation changes and the 1980s switch to MMTS sensors both suppress modern hot day counts relative to the past, and the raw and homogenized series diverge almost exactly when the instrument transition happened.
They excel at making the dull delightful. There are a lot of big and serious facts to get through, but they have a gift for communicating them simply and eloquently. I just wish they'd've come to a different conclusion.
what I don't like about it I didn't like about it then and still don't like about it is it's not defensible. Nobody's going to say I'm not agile. Oh no, I prefer rigid development. Oh, I prefer inflexible development. No, everybody's going to say that they're agile, which extreme doesn't have that problem.
And I wouldn't be surprised if you know 6 months or a year from now, the model is able to do zero shot an entire poker solver, basically my entire PhD thesis in one go.
Um I think another similarity is I mean, the core notions uh behind Agile and extreme programming are solid and good, but a huge snake oil industry appeared around it, the Agile industrial complex as I like to refer to it. Um and that will happen. That is happening with AI right now, and it's often hard to see the difference between where is the snake oil and where is the real stuff.
So, I see a a big trend is the re-soloing of programming. Where a big part of extreme programming is creating a safe social environment for basically antisocial people. Not just asocial, antisocial. And the when I think about the degree of interaction on an XP team, people aren't talking to each other hours a day and happy to be doing so because it's set up for that to be a positive experience.
none of us wanted to go through that extreme but lots of time when you are under a lot of pressure and no time to react other than just to survive that scale that keep on coming at you, you have to make uh decision that increase uh speed and velocity because speed and velocity allow us to build quick enough to survive
I I I I do feel that, you know, fully autonomous one-shot approaches are not the right approach for these problems. I mean, I I think you you'll get a lot more mileage out of the interplay between between humans collaborating with these tools.
Um we are seeing a lot fewer sort of pure AI solutions now where um they are just one shots the problem. Um so so there was there was a month where that happened and and that has stopped.
I think frontier AI auditing (covering both safety and security) is both urgently needed and doable, and can help avoid extreme concentration of power, since it introduces external oversight into the decision-making of people with a "country of geniuses in a datacenter" at their disposal.
Like any addictive consumer product that routinely exposes users to graphic sex, extreme violence, and anonymous sexual predators, social media in its current form causes a variety of harms to people of all ages. But we allow adults (often defined as age 18) to make decisions that are bad for them.
My one little bit, the one little bit of of fundamental uncertainty even on long time scales is this thing about tasks that aren't verifiable. Like, planning a mission to Mars, like, uh you know, doing some fundamental scientific discovery like like CRISPR, like, you know, writing a writing a novel. Hard to hard to verify those tasks.
I’ve got a GitHub actions thing that runs a piece of software I wrote called shot-scraper that runs Playwright, that loads up a browser in GitHub actions to scrape that webpage and turn the results into JSON, which then get turned into an atom feed, which I subscribe to in NetNewsWire.
I’ve got a GitHub actions thing that runs a piece of software I wrote called shot-scraper that runs Playwright, that loads up a browser in GitHub actions to scrape that webpage and turn the results into JSON, which then get turned into an atom feed, which I subscribe to in NetNewsWire.
Like basically I think I think that there is a big benefit from AI being in the public and that would be a reason for us to not be quite straight shot.
If you go to look about how psychologists think about learning, there's nothing like uh imitation. Maybe there are some extreme cases where humans might do that or appear to do that, but there's no basic animal learning process called imitation.
The biggest misunderstanding is somehow that a group of people or even just one person runs the entire Chinese economy. It is far from the reality. It is a very complex, large economy, and even if there is an extreme form of political centralization, the economy is totally decentralized.
I think the first point is the extreme loyalty of the people whom Genghis Khan chose. His kinsmen, as we said, had deserted him, his anda was a questionable relationship but all the others that he found were just common people, herders or hunters, very common, and they were loyal to him and never, ever revolted against him, never betrayed him.
At one extreme, something you read can change your whole way of thinking. The Selfish Gene did this to me. It was like suddenly seeing the other interpretation of an ambiguous image: you can treat genes rather than organisms as the protagonists, and evolution becomes easier to understand when you do.
So, trying to establish a base on Mars it's going to be so hard that it is not even going to be close to being self-sufficient for a couple a century at least. So, it's not like a backup plan now, we have to solve the problem of climate change, we have to deal with that.
In the early days, the DE1 was considered "unforgiving" by many. I didn't mind, as the DE1's ability to control shot parameters more than made up for its early challenges.
Somehow we’ve embarked on the biggest project in history even though it has no articulable purpose, offers no benefits, and will cost taxpayers more than a good-sized war.
So, it wouldn’t surprise me any longer if we found life on Mars and it had a genetic code that was not very different to the genetic code that we have here without it just being transferred across, there’s some inevitability about the whole of the beginnings of life, in my view.
in 1985, um my colleagues and I came up with a description of how people actually make life-and-death decisions under extreme time pressure and certainty. And we called it the recognition prime decision model. And it's a blend of intuition plus analysis. And we found we studied firefighters. We found that most fire that accounted for almost 90% of the tough decisions firefighters made.
When you take venture funding, you sign up for a rocket ship ride that will either take you to the moon or to crash-land painfully back on earth. Those are the only two choices. And both rides tend to require heavy extraction of value from the customer.
One of the reasons I've never cared for crypto currencies is that the associated utopia of trustless society had zero appeal to me. I don't think the world is better off by erasing the need to trust in our transactional counterparts, so turning these transactions into pure computing always struck me as a regression.
Their words now
But wherever this leads us next, it's clear to me now that I was too hasty to completely dismiss crypto on the basis of all the things wrong with it at the moment. Instead of appreciating the fundamental freedom to transact that it's currently our best shot at protecting.
The short answer is that elevating the fulfilment of humanity’s supposed potential above all else could nontrivially increase the probability that actual people – those alive today and in the near future – suffer extreme harms, even death.
Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.
Ultimately, however, one-shot, or even sometimes zero-shot, seem like the fairest comparisons to human performance, and are important targets for future work.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.