The subjects this post names, from the same vocabulary
the directory files beliefs under. Posts are matched on those words alone —
nothing here is a summary of this one.
the jev release is giving react major version drop vibes. just a massive crashing wave, everyone rethinking fundamental stuff and thinking through the implications of a novel new primitive.
I do not think those running the government testing understand what is going on, and believe they are reacting basically on vibes and what various trusted people tell them.
every week i see a new benchmark that ranks claude code last
and everyone pats themselves on the back for not using claude code and being "smarter"
and it's still #1 and still growing faster than most of the other things on the list
this has been going on for a year
We’re positively reinforcing AI for success on benchmarks, including impossible benchmarks , then negatively reinforcing it for getting caught cheating.
Google, which led on benchmarks not that long ago, has fallen behind where it now counts: it has no leading frontier model and it has nothing close to Codex and Code.
the problem with training models on maintainability is like the cost function of bad architecture and bad program design can't be evaluated by running the unit test because it hits you 3 to 6 months later
I think the way that you'd measure conjecture generating ability is going to be more subjective on like that tone shift where um it'll be mathematicians saying they're not just using it to like solve their problems, but as they step back and decide what their research field should even be that a conversation with such and such model like was genuinely helpful for that.
Um so if you say okay well we're going to instead of just running this model once we're going to run it five times and take the best of the five responses or like ask a judge which one it thinks is best then you can get much higher scores than that model. And so it's really easy to make something that looks a lot better on paper but is actually not better once you control for the amount of test time compute.
But what we're seeing today with the modern models is that 5.5 and other models can think for if you scaffold them reasonably well, can think for weeks even um before having performance plateau on some of these benchmarks. And so, the point at which they plateau is simply too far out to reasonably test.
my claim is the proper way to evaluate the models now is you either have some kind of budget for the benchmark whether it's tokens or cost or time or whatever or you plot the performance as a function of the amount of test time compute that's going into the model
I think the reason why it doesn't show up as so much better on the benchmarks is because the benchmarks are being presented, the benchmark results are being presented in the wrong way. They're not controlling for the amount of test time compute that is being used on that benchmark question.
And so you kind of end up in this this bad equilibrium where everybody kind of knows that it's a bad equilibrium, but like nobody wants to break out. And I I felt like, okay, well, if I just hopefully come out and say like, look guys, let's all recognize that we're in a bad equilibrium and let's move to this different equilibrium where we're we're plotting things with an X-axis
In the English-language arena, Britten sets the standard for handling writers of inborn musical power-the likes of Shakespeare, Donne, Blake, Keats, Hopkins.
This is a super exciting release - Claude Fable 5 is the same underlying model as Mythos but with added safeguards. The benchmarks are great and it's SOTA on everything by a margin but I'll add that *qualitatively* also, this is a major-version-bump-deserving step change forward
I had already put both laptops through my benchmark gauntlet, which revealed one theme: the Mac is faster (in most cases), more efficient, quieter, built better, has a much nicer display, and costs much less.
And so I think it's it's really important uh when when we think about benchmark progress to think about it from that perspective, which is benchmarks rise on problems that we've framed that we can articulate, that we can score. And there's a lot of work that's human work that uh it it can't be scored until you write it down
So, here I actually think it's even stronger. The thing with C++ is that if you make a mistake, right, in in JavaScript, maybe you take down your server, which is already bad enough, but in C++ when you make a mistake there, now it's actually a security vulnerability most of the time.
So that's unfortunate, and that's why some environments like Go, for example, has has Go routines and green threads, which are really language emulated lightweight threads that kind of do what I'm talking about, but but at a much lower cost. But you avoid the function coloring. So there is a bunch of different things, but but you know, but for an environment that already exists like JavaScript or like C# and and the Windows event loop and and whatever, this this was the right solution.
And we we full well knew that there was absolutely zero chance that we would appeal to the JavaScript ecosystem with a proprietary programming language licensed from Microsoft. No. No one was going to come. It had to be open source. There was just no two ways about it, right?
So, if we're checking 99% instead of 100%, well, heck, that's better than the 0% that JavaScript checked, right? And it gives you like language features that no other languages can provide because they can't get to 100%.
As the software industry is largely driven by trends and fashion, React's "cool factor" led to a mass amnesia regarding how to make normal, well-designed web apps made of HTML with a sprinkling of scripting on top and everything became one big tangled, brittle mass of Javascript that completely keeled over when scripting wasn't available.
Nvidia's computing stack is the best performance per TCO in the world, bar none. Nobody can demonstrate to me that any single platform in the world today has better performance TCO ratio. Not one company.
none of us wanted to go through that extreme but lots of time when you are under a lot of pressure and no time to react other than just to survive that scale that keep on coming at you, you have to make uh decision that increase uh speed and velocity because speed and velocity allow us to build quick enough to survive
we have a very clear definition and expectation of what it is at the staff engineer level because we benchmark ourselves to all the great company out there Google, Facebook and all that
We show that equilibrium generically occurs at neither the Harberger nor Glaeser-Luttmer benchmark. Cost-minimizing suppliers drive allocations to vertices, not interiors. Corners are not an assumption but an outcome about what cost-minimizing suppliers choose. The correct benchmark is corners, not random, and corners generate qualitatively different welfare properties: losses far larger than either efficient or random distributions, and discontinuous jumps from small parameter perturbations.
To be clear, speed in itself does not mean labor markets and employment won’t eventually recover, it just implies the short-term transition will be unusually painful compared to past technologies, since humans and labor markets are slow to react and to equilibrate.
elements, JavaScript reigns supreme. SvelteKit's efficiency and reactivity make it my framework of choice, and TypeScript's type annotations bring a welcome layer of confidence to my codebase.
I used to think it required more churn to get progress to stay on the leading edge of new stuff. And I wrote this before I experienced the indignity of the 2010s in the JavaScript community.
But the metaverse and 3D gaming in general needs something that's rather more powerful, more safe, more scalable, and more capable than JavaScript because the metaverse is actually a more difficult technical problem than a webpage.
It's just that Zig, C, C++, all those languages that were being tested, they're all LLVM backends, right? That's the one that actually turns the thing into the executable part. And if there's a variation in speed, it just means in one language you didn't quite express what you are supposed to correctly.
It's easy to get impressive-looking results if you're comparing against a poorly-tuned baseline, and that observation turns out to explain a surprising fraction of supposed improvements.
As a Python and JavaScript programmer my favorite models right now are Claude 3.7 Sonnet with thinking turned on, OpenAI’s o3-mini-high and GPT-4o with Code Interpreter (for Python).
So, taking also any benchmark that is derived from competition and saying this is where we should be is also so dangerous because it might not even be applicable depending on how you define the metric.
Before this book I used to think that Javascript is "bad" or a "hacky" language. This book changed my perspective, understanding some tradeoffs and some remarkable strengths of this weakly typed language that is starting to take over the world. Note that this book is now dated
when you send someone a link to a post, they can't read it without loading the entire Mastodon frontend app into their browser, which will is a monster React codebase that will break if they have scripting disabled and also can take up to 30 seconds on older hardware.
My hot take is that JS has the lowest bar of entry to building and being productive (a good thing) but one of the highest bars of any language and stack to building high quality, stable, and reliable software. Very few devs have the expertise to pull off the latter
Yeah, it is interesting because, when people react against you by saying you are being arrogant about this, 99.999% of the time, all they mean is I disagree. That’s all they really mean, right?
One thing that drives me crazy about the React ecosystem, and more specifically "tech influencers" and "thought leaders" in the space, is the infantilization of the developers using and working in it. I’m tired of reading takes like TypeScript generics, mapped types, etc should only be needed and used by library authors for most use cases. Or today’s discourse; Don’t use `useCallback’, ‘useMemo’, and React.memo. If you’re building anything beyond a simple CRUD app or a an incredibly focused app with few features, you _will_ need these features.
RSC is the future of React. The React team has made this very clear and we are lucky to be in touch with their amazing team members to help us along this path.
I have argued that the probability of a bad equilibrium is only marginally influenced by the level of debt, but can be much reduced by a contingent rule making the primary balance react to an increase in debt service.
Multipliers are likely to vary a lot over time and space, but the bulk of the evidence is that they are different from zero, positive for spending, negative for taxes, and that they are stronger when monetary policy does not or cannot react to fiscal policy.
Eric Schmidt, Jonathan Rosenberg and Alan Eagle, Trillion Dollar Coach: The Leadership Playbook of Silicon Valley’s Bill Campbell: I understood Bill Campbell was a behind-the-scenes guy in Silicon Valley, but I had no idea just how influential he was. Bill Gurley of Benchmark noted “I would argue that Bill has had a bigger impact on Silicon Valley than any other single person simply because his reach was so amazingly wide.” That’s a story I want to read.
Did you know, the tests you write with react-testing-library look almost identical to the tests you write with vue-testing-library. Magic things happen when you don't test implementation details. :)
Visual Studio Code is the go-to editor for some complex languages like React using JSX or TypeScript and I finally agreed with Wes Bos that VS Code is better for development of React JS.
JavaScript allows us to mask network latency altogether. Applying this as a design principle should even remove most spinners or “loading” messages from your applications.
For text editing that is not specifically related to programming the Mac and iOS, I rely heavily upon BBEdit, where I do most of my Python, HTML, JavaScript and PHP editing.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.