I guess our best defense right now is dependency cooldowns - giving new package releases a few days before upgrading to them, in the hope that supply chain attacks like this will be spotted by someone else.
Right now OpenAI is extremely dependent on CoT monitoring, everyone else depends on it quite a lot as well, and it looks like it may not last much longer, and that Astra already is on the edge of steganographic capabilities and can do substantial obfuscation if and only if it thinks you would think it is up to no good.
Even if everyone tries as hard as is plausible, Chain of Thought monitoring will probably never be easier than it is now and will get harder over time. In a year, chances are very high it will not be able to serve the function it is currently being asked to serve.
If this is only due to gains in capabilities, that is extremely bad news, and it means CoT monitoring is unlikely to survive for another year unless we find a way to actively improve it, and it might not last six months.
But keep those methods you use to investigate things and monitor things very separate from the methods you use to generate reward which is something that AI companies including open AAI have held up as a principle especially in the case of avoiding putting training pressure on the chain of thought. So you might have monitors that read the agents chain of thought in order to alert you if something is going wrong somewhere but you don't train the agents with the outputs of that monitor.
And you have to have a robust and resilient supply chain. You cannot just depend on one or two player in different geographical. And so, I think, you know, the more and more people going to realize making in United States is critical.
I do think there's this qualitative shift that we I think we agree is coming, which is that there will be at least some goods whose network-adjusted capital share goes to one, right? Because the whole supply chain can be automated and there's no part in it that we care intrinsically about having a human do. Um, so that'll be a, you know, that'll be a qualitative shift. Interestingly, the implications of that shift for the overall capital share are ambiguous
there's a sense in which nothing's yet been completely automated. If you look at the network-adjusted factor shares of a good, which is to say you look down the supply chain and say not just like the final step, but how much of that is done by capital and labor, but what went into the machines that can automate that final step. You'll find that labor's adding a lot of value down the supply chain.
So, I think there is a world where it is concentrated, in which case it's going to be really hard to index AGI. There is another world where it is not It's electricity, then like basically every company has access to AGI. So, you just buy you use buy the index. So, like, you know, Nigeria just needs to buy the index.
As we move from, you know, hey, [clears throat] these companies are selling tokens where they provide the entire uh reasoning chain and all that to uh selling automated, you know, white collar work, right? Automated software engineer, send them the request, they give you the result back and there's a bunch of thinking on the back end that they don't show you. The ability to distill out of American models into Chinese models will be harder.
Um in general the semiconductor supply chain has not right it's lived through the booms and bust and uh we can talk a bit more about it but basically no one you know some players as of very recently have like woken up but in general no one really sees demand for 200 gawatts a year of AI chips or you know trillions of dollars of spend a year in the semiconductor supply chain.
Yeah, I think the biggest bottleneck is compute and for that the longest lead time supply chains are not power or data centers. They're actually the semiconductor supply chain themselves, right? It switches back from being power and data center uh as a major bottleneck to chips.
oh 50 gigawatts of economic you know sort of capex in in the data center and what gets built on top of that in terms of tokens is even larger right it might be hundred billion dollars worth of AI value into the supply chain is held up by this $1.2 two billion dollars worth of tooling that simply just cannot expand its supply chain quickly.
Um, just shipping out all the engineers and blowing up the fabs means China has a stronger semiconductor supply chain than the rest of the world, right?
Nicolas Niarchos, The Elements of Power: A Story of War, Technology, and the Dirtiest Supply Chain on Earth is a good treatment of minerals issues as they relate to the Congo today.
And I think the United States could probably do that in 2 years or less if you started today. Like it's currently 11:00, so like we're going to start cutting checks by noon.
But, China mainly dominates these markets because it has produced a long-term industrial strategy for these technologies and has honed an optimised, modern supply chain as a result.
That’s because America has an amazing consumer supply chain, one of the best, if not the best in the world, but this is totally different from having an industrial supply chain.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.