We could not possibly know that such a model is aligned. If that is all it takes to get a generation ahead of Astra, and we are willing to move this fast, these pauses in training are not going to end up meaning very much, time is even shorter than we knew
But given the Chinese Communist Party’s power-seeking nature, it seems much more likely that China would agree to cooperate on AI safety if U.S. capabilities were comfortably ahead.
Why are we surprised? Even if you're a good programmer, if you finish a job and you ask your also very good peer to review it, you're gonna end up with better code. Of course you're gonna end up with better code. So build that into your process.
Don't try to anticipate anything. You will literally go crazy. Because even the smartest brains in the business cannot anticipate what two model hops from here is going to look like. It's an absolute waste of time, and you will develop an AI psychosis trying to deduce what two years from now is gonna look like. Focus on right now, and right now is the most incredible time to be into computers.
I think you're gonna have a hard time coping with the new reality if the only thing you loved about programming was the mechanical bits of putting the right logical constructs together to produce something other people told you to produce. Because that mechanical process is under threat. If you are, as you just mentioned, excited about building things, I don't think you're under threat at all.
Countries and peoples have a right to set an immigration policy, and this notion that we shouldn't have any borders or, or even worse than that, it's all just a blank slate, that all peoples are just the same, and we can take people from one place of the earth and we can place them in another place on the earth and everything's just gonna work out hunky-dory. Empirically not true.
And so basically everything that we can verify reasonably well with some feedback loop, the AIS are doing pretty well on. And that's sufficient to make AR and D go quite fast and to continue. But there's some parts of of developing uh aligned and safe AIs that are more subtle, hard to check, depend on, you know, detailed in the weeds things.
Structure and Interpretation of Computer Programs (SICP) by Gerald Jay Sussman and Hal Abelson. SICP is my all-time favourite programming book. It's also where I learned about the power of wishful thinking when coding. SICP uses wishful thinking as a design tool: write code as if the abstraction already existed. Pretend. Then, once the ideal abstraction has taken shape, you go ahead and implement those functions. This meta-level is what made SICP so valuable. Ultimately, it's a book that teaches how to think about code and problem solving. I'm tempted to even say that SICP is more a work of art than a pure coding book, but I won't go there. A beautiful book.
Maybe I would pick something again that's that's in the category hard and boring because that's usually a category that is a little bit easier to actually find people that will appreciate when you solved something. If you pick something that is fun, even if it's hard, you're gonna have a very tough time, especially in a time where people can just prompt things into existence.
However, looking ahead, I still do not believe that future AI (say, in 15 years) will be based on the LLM stack. I believe it will necessarily have to move closer to its optimal, final form -- symbolic learning. Obviously this is a risky and contrarian belief -- the safe bet would be LRMs. But let's see.
Yeah, I mean I think it's really having incredibly good taste in what you ask your agents to work on, right? That is the the crux of you know from my background uh a research problem. You know, a researcher can have all the tools and all the techniques, but often most of the battle is what problem are you gonna spend your time on?
Know where the money is coming from. Know where you plug into that. And like if you're just asking those questions, you're actually already like steps ahead of all of the other like fledgling perspective mathematicians.
And then I was like, okay, can you come up with an algorithm that is better than the algorithms that I came up with or that anybody else came up with and go ahead and like look at all the published work and synthesize that and then try to come up with something novel and it it's not able to do it. And I can give it a lot of time and it's it's still not able to do it.
It shouldn't It shouldn't be shelved next to How to Win Friends and Influence People because it's a manual not of how to gain power, but of how to keep power.
this is the rule is really simple. only make deposits. Never make withdrawals. That's it. Because you're going to make withdrawals by accident sometimes because you're gonna make mistakes, but you never intentionally make a withdrawal.
That's why I don't think accelerationist steamrolling will work. The only way this conflict resolves peacefully is some kind of grand bargain: a flagship policy plan that directly addresses the public's top fears about AI and meaningfully redistributes the gains.
Where I get fired up, and this ties back to the AI discussion, is when that's turned into this meme, that programmers no longer have to be competent. I mean the AI is gonna figure it out. The generators is gonna figure it out. I don't need to know SQL, active record is gonna abstract it away from me. No, no, no dude, hold up. The path here is competence.
Their words now
really good programmers are currently more valuable than ever because they're the ones who are able to get the most out of the AI acceleration.
For years, it was faster to mock up software than to ship it. Designers stayed "ahead" of engineering with prototypes. Now AI coding agents make development so much faster that the loop has flipped.
And so this loop, this cycle, is gonna go on and on and on. It kinda comes down to basically intelligence is gonna scale by one thing, and that's compute.
You know, a lot of people would say, "You know AI is gonna completely destroy software. We don't need software anymore. We don't even need tools anymore." That's ridiculous.
Which is, you know, you’re the game designer who’s playing God, and players are the ants in your ant farm, and you want to see what they’re gonna do, which is not the correct way to be a good multiplayer designer.
Small studios are the future of gaming. The big studios basically acquire the small studios for new IP and ideas, and the small studios grow in. The really compelling, new, innovative ideas are gonna come out of small studios.
That is the best prop artist in the industry. That’s who’s gonna show up on our doorstep, so when they show up here, we should treat them like the best prop artist in the industry instead of starting from a place of doubt and cynicism.
I actually think profitability happens when you underestimated the amount of demand you were going to get and loss happens when you overestimated the amount of demand you were going to get because you're buying the data centers ahead of time.
Write notes in your own words, connect them to existing notes, and let ideas emerge from the connections. I wrote a whole blog post about this, so go ahead and read it. Highly recommended.
So if you’re very fond of somebody or you love somebody and they die, it’s kind of infantile to whine about it ever after. Because what did you think was gonna happen? Either you or them.
Decades of grading data; standardized test scores; cross-sectional, longitudinal, observational, and experimental studies; along with many other types of ancillary and convergent evidence, ultimately tell the same story: education can raise the absolute performance of most students modestly, but it almost never meaningfully reshuffles the relative distribution of ability and achievement
it's a luxury good like you're not paying the penalty for that the people who embrace it and think that they can just ask AI for everything or they can just look everything up but they don't have to understand the copy pasta that they slam into their project they're the ones who going to suffer
Their words now
Where I get fired up, and this ties back to the AI discussion, is when that's turned into this meme, that programmers no longer have to be competent. I mean the AI is gonna figure it out. The generators is gonna figure it out. I don't need to know SQL, active record is gonna abstract it away from me. No, no, no dude, hold up. The path here is competence.
I do not believe that analytical skills are the missing ingredient in thinking about the future. Rather, I believe that imagination about the future and a theory of change that helps to describe it clearly, are what is needed to look ahead in a more compelling way.
So Tesla hasn’t found a different, better way to bring driverless technology to market. Waymo is just so far ahead that it’s dealing with challenges Tesla hasn’t started thinking about.
Installed generators cost ~$800/kW, and the data center capital cost, including servers, is ~$40,000/kW, so adding 1% more makeup compute capacity is money ahead of purchasing generators.
I expect that the delta between 5 and 4 will be the same as between 4 and 3 and I think it is our job to live a few years in the future and remember that the tools we have now are going to kind of suck looking backwards at them and that’s how we make sure the future is better.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.