One potential upshot of this: a more fragmented media environment but a much more media-aware population will lead to more small-scale, meme terrorism.
They see what needs to happen but don’t want to be the bad guy. I’m happy to just make the call if I agree with the premise. Saves enormous amounts of meetings and change management etc.
This concept is still alive in present-day Japan, where cherry blossoms are viewed with the bittersweet knowledge that their beauty lasts only for a short while.
In this case, however, the agents did not copy their weights, attempt to procure replacement compute, or take other steps that would be rational to take if their objective was to survive shutdown. So while the agents in the OpenAI-Hugging Face Incident were rogue, they were not truly sovereign.
even in this incident we saw there was a lot of pressure um as a result of this incident to stop doing cyber security evaluations and I really don't think that stopping doing evaluations and like sort of blinding ourselves to the result of evaluations is the right reaction to this problem.
I think one one thing that um feels especially concerning to me about this whole incident is that this might be the clearest warning shot we ever get for loss of control.
The reality is quite the opposite; a country that can create a national health care system is more aware of its own existence, because of the experience of solidarity which enables everyone to thrive thanks to everyone else.
it was only about 15 years ago I started beginning to be aware that there may be a an important biology underlying motivation and when that biology goes wrong, it can lead to loss of motivation and even pathological apathy.
It'd be pretty annoying if they just start like spontaneously getting notifications during meetings that like, "Oh, it thought that you were thinking about an Uber, therefore it decided to summon summon two for you." And so you'll probably want these to be pretty explicit.
when AIs are extremely extremely capable my view is that those AIs will be harder to align than current systems. So for current systems, we have this feedback loop where we basically like we create an AI. We do some evaluations on it. We see that it has some kind of messed up behavior that we can kind of quickly understand. Then we like can like go look in training and be like, "Oh, the these training environments led to this problematic behavior. Let's like tweak that training data. Let's introduce some additional training data to like correct this other issue and then move forward from there." But in a regime where the AIs are extremely situationally aware, very very very very capable and um you know uh we don't necessarily understand what they're doing, this feedback loop breaks down.
The only argument as far as I'm aware for why you would want product management to be a specialist function is really it's a trade, not a qualification.
Um so I think it's an alignment failure. I think it's a security failure. I think it's like a very serious thing even though it's you know not not the biggest example of consequence.
But it recognizes that calling an intelligent, self-aware, emotionally complex being "property" does not answer the question of whether that being has an interest in freedom.
And I think, you know, most people when they talk about aging, they're so worried about dementia and, you know, Alzheimer's disease as a specific form, but balance is even more important, I think. And most people aren't even aware of their balance.
Because if you're those three things, if you got fire in your belly, you learn quickly, and you're self-aware, you can kind of get good at anything eventually.
More data for better comparisons are good. Now everybody has to do it, and regulators and the public have to learn to look at only the data, and not individual incidents
the investor community writ large has slowly become aware of and believes it strongly in increasing returns and power laws. And so over time, if they all believe that, they're going to be more willing to invest on the come and take risk.
So I I don't do one-on- ones. My friend Bing told me that Jeff Bezos wouldn't ever do one-on ones because it created politics and it was a huge waste of his time. I was like, I love that. No one ones.
I I definitely think I'm so aware of the fine line between success and failure, especially on your first company. And and and it's so it pains me how much founders and es especially I see it in men, not all men. And I have four sisters and a bunch of daughters. But I would say I see it in a lot in men and and friends, college friends, people I've grown up with that if they had an initial failure, if they had failures, they get attached to it and they start feeling defined by it
AI today it's just starting to to become aware of you know the existence of of of language services and agents today like to use grep and awk and whatever you know to to find all the places where you reference a certain thing but it's not semantic search, right?
So, you you can't look at any given scientific achievement purely in isolation and give it an objective grade without being aware of the context both in the the past and the future. And so it it it may never be something that you can just reinforcement learn the same way that that you can for much sort of more localized problems.
Scrutiny typically either jumps several points on any 10-point scale after an incident that kills a bunch of people, or increases very slowly over decades.
Scrutiny applied to AI is way below other technologies that - even if you totally ignore the doomers - could put far fewer lives at risk than AI in a single incident.
This one only gets better with age. Although it’s only mentioned briefly, this is where Andy Grove first introduced OKRs to the world. His practical advice about meetings, especially the importance of 1-on-1s, inspired
If you are in regular meetings with a hardware vendor as a customer (or potential customer) you can accomplish a lot by providing firm and tough feedback, particularly with Intel today.
I personally hate meetings because a significant percent of meetings when done poorly don’t serve a clear purpose. But that’s a meeting problem, that’s not a communication problem.
When I have conversations with breached companies, my messaging is crystal clear: be transparent and expeditious in your reporting of the incident and prioritise communicating with your customers.
So I do think it's really hard to have an original thought. We are social creatures. We encounter the same situations again and again. And so it's really hard. You're born into these traditions of thinking and being and knowing. And most people are never going to question them, and most people are never going to become aware of them.
Whatever founder mode consists of, it's pretty clear that it's going to break the principle that the CEO should engage with the company only via his or her direct reports.
It’s just this idea that people have about the past can be very useful if it brings you happiness in the present, but if it narrows your worldview in the present, you’re not aware of those biases that you have, it can be toxic either at a personal level or at a collective level.
On the other hand, the manager-free experiments I’m aware of (e.g. holacracy at Medium and GitHub, or “Choose Your Own Work” at Linden Lab) have all been quietly abandoned or outgrown.
Instead of meetings, I used Workplace posts (Meta’s internal version of Facebook Group posts) to share thoughts, start discussions, and make announcements.
So, experts are highly aware of mistakes. But people who are journeymen, many of them stay as journeymen because they they want to move on and forget about their mistakes.
Programming, of course, is forgetting, but we need to at least try to be aware of the costs of the abstractions we choose and consider who it is that ends up being forgotten.
A strategically aware intelligence can choose its visible outputs to have the consequence of deceiving you, including about such matters as whether the intelligence has acquired strategic awareness
Part of what I do for work is record videos so they let me buy a new webcam: Logitech BRIO. I am very happy with it! Works well for streaming and video meetings.
I use Calendly to schedule podcast interviews and other meetings. So much nicer than trying to work out a time to chat by going back and forth in an email thread.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.