But alignment is no solution: it is an unsolved scientific and technical problem whose solutions—to the extent that we have them—cannot simply be imposed on every AI company operating on Earth. You should expect for highly capable, poorly aligned, self-sovereign agents to exist alongside you in the world.
But the problem is these agents are just naturally pretty sloppy and they're naturally pretty spiky in their capability profiles. And so you wouldn't necessarily even if you noticed a weird error that it made, you wouldn't necessarily jump to the conclusion that it was like because of some sort of malign crazy conspiracy.
I am very much opposed to the view that the AIs are sentient, or might be sentient. I view that as a category error , and the chances of it being true are vanishingly small.
Linux conquered everything else. All the AI infrastructure that everyone runs off, it's all running on Linux. All the systems, all the servers, everything is Linux. So it was kind of a curiosity that the desktop and the personal computers we were using really hadn't been captured, but clearly it was just waiting for this moment. Linux spent the time from '91 to now waiting for agents to fully flourish as an end user operating system.
This is such a useful book. It makes the case that there's no such thing as "human error" - instead most catastrophes are caused by system issues, misaligned incentives, and unrealistic processes. You are not the custodian of an otherwise safe system that you need to protect from erratic human beings.
Um, and that's a very very useful general technique is you know inference time compute to perform search over plausible ways of solving the problem that can get much much higher performance or much more reliability in longunning agent flows.
I think a lot of the reason people are skeptical of these is because they've been burned by things like case and UML and all these other miracle solutions that were forced on them by people who wanted them to use it no matter what.
It becomes insufferable like as a mathematician because you you would basically be like I'm every single time I see one of these I kind of don't know if it's worth my time even if 99 out of 100 of them are right.
There may be a lot of AI things that's going on right now, right? Uh eventually some of these things will consolidate, some will go under, some will become really awesome solutions and all that stuff. And so, but the the market will sort it out.
An agent has no such learning ability. At least not out of the box. It will continue making the same errors over and over again. Depending on the training data it might also come up with glorious new interpolations of different errors.
There isn't this cumulative process which is, uh, sort of built up interactively. Um, it it it seems to be a lot more trial and error and just repetition, brute force, um, you know, which can see it scales and it can work amazingly well in in certain contexts, um, but yeah, this this idea this this sort of building up cumulatively from, um, from partial progress is kind of is what's still not quite there yet.
Um we are seeing a lot fewer sort of pure AI solutions now where um they are just one shots the problem. Um so so there was there was a month where that happened and and that has stopped.
the marketing metrics the finance department has faith in and demands from marketing are not those metrics which are most conducive to building brand and customer value over time. They're the metrics which are most conducive to selling tech solutions to the companies.
Further, the fact that there is no such thing as a ‘Dark Ages’ and that a term we used for source survival has escaped containment and been rendered meaningless by a bunch of poorly read self-congratulatory dolts is, in fact, what we call ‘settled academic fact’.
Similarly, AI translation will prevent you from learning to speak another language, AI summaries will block you from really understanding the arguments laid out in a book, and AI analyses of your personal problems will make it harder for you to think through solutions to your own problems.
If you go to look about how psychologists think about learning, there's nothing like uh imitation. Maybe there are some extreme cases where humans might do that or appear to do that, but there's no basic animal learning process called imitation.
But again, the people who love to solve problems are the ones who are just so enabled right now and I think everyone starts there. No one is falls in love with solutions at the beginning of their careers.
And that's the thing, it's the pool of data, and I think that's what people miss. We as paleontologists get caught up on single superlative specimens and then try and treat them as a silver bullet almost.
will the error rate ever be controllable or manageable will you ever get to a model that knows when it's wrong which to me seems like given a stat statistical system seems like a contradiction in terms
It's also worth noting that chat isn't the only way to integrate AI in software products and increasingly agent-based applications outperform chat-only solutions. So expect things to keep changing.
Bitcoin is not going to catch on as a charity. In order for spending it to catch on persistently at scale (i.e. not just billions of dollar-equivalents in annual global medium-of-exchange volume, but trillions), it has to solve problems for spenders and/or recipients that other solutions are not doing.
The scale word gets a lot of attention in this. The interpretation that I use is effectively to avoid adding the human priors to your learning process. And if you read the original essay, this is what it talks about is how researchers will try to come up with clever solutions to their specific problem that might get them small gains in the short term while simply enabling these deep learning systems to work efficiently, and for these bigger problems in the long term might be more likely to scale and continue to drive success.
But what happens when every company can just invent their own business logic really cheaply and quickly? You stop using platform SaaS, you start building custom tailored solutions, you change them really quickly.
Humans being awful drivers leads to a massive increase in vehicle costs and weight, makes cars more dangerous to people outside of them, and imposes significant costs on infrastructure like roads.
is this idea that we form these internal models at particular points of high prediction error or points of, I believe also points of uncertainty, points of surprise or motivationally significant periods. And those points are when it’s maximally optimal to encode an episodic memory.
China built half of the world’s ships (by gross tonnage) in 2022, while the US had 0.2 percent of capacity: in practice, this meant that while China builds hundreds of new ships a year, the US builds three to five.
The trouble with Xi Jinping is that he is 60 percent correct on all the problems he sees, while his government’s brute force solutions reliably worsen things.
“lots of anonymous people just invented through trial and error and adaptation” just doesn’t cut it for me — I’ve never found such stories to be true upon closer investigation.
My view is that the idea that AI will decide to literally kill humanity is a profound category error. AI is not a living being that has been primed by billions of years of evolution to participate in the battle for the survival of the fittest, as animals are, and as we are. It is math – code – computers, built by people, owned by people, used by people, controlled by people.
Their solutions, such as pasture-fed meat, with its massive land demand , are impossible to scale without destroying remaining wild ecosystems: there is simply not enough planet.
Although his training is not in economics, Richard Reeves draws on economic research, as well as research from the fields of sociology, psychology, and political science, to tell an under-told story. The book explores the problems faced by boys and men, the roots of those problems, and possible policy solutions.
Another book I wanted to mention in this context is a fairly obvious choice, Antonio Damasio's Descartes' Error, which is a foundational book about the self, in particular because it reconnects emotion with rationality and the body.
Does part of your work include designing, architecting, building, or thinking about how systems work? This book is a snarky look at all the reasons systems fail. The author has clearly lived a life in systems design and has no qualms about calling out all the obvious and not so obvious reasons an complex or complicated system struggles. My main qualm was it offered few solutions while pointing out problems.
Human error is blamed for over 90 percent of industrial and automobile accidents. It is the leading cause of aviation accidents, and medical error is reported to be the third-largest cause of death in the entire United States. Horrifying? Yes, but why do we label it “human error”? It is design error.
Information, once freely shared, is a Pandora’s box that cannot be closed again. It is very hard for me to imagine a world where we transition back to paying for access to information; these are pre-digital solutions to modern problems.
It is by questioning the obvious that we make great progress. This is where breakthroughs come from. We need to question the obvious, to reformulate our beliefs, and to redefine existing solutions, approaches, and beliefs. That is design thinking.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.