https://t.co/5MvjBuyABJ excellent blog by OSR's Ed Humpherson on the importance of looking at the variability in health metrics rather than just their average - and the tools available for doing this
"Carbon intensity" is a greenwashing term: It measures pollution per barrel, allowing oil companies to claim climate progress even as rising production drives up total overall emissions.
All those things you can easily measure in isolation, and you can also optimize for them quite locally. But these local optimizations do not produce global optimums, and the fewer of us are looking at the output, the less it matters.
There is no reason NOT to have every single email you get routed through an agent / LLM / API to be scored and approved or rejected And then the approved emails ordered by importance It's crazy this isn't a feature of Gmail, Fastmail etc AI could permanently fix email!
If anything, I think that the book has gained very substantially in relevance, as more of our social structures have come to rely on coordination rather than violence.
Facts and Fallacies of Software Engineering by Robert L. Glass. In essence, this is a book about an industry that refuses to learn. That was true 25 years ago when this book was published, and it's probably twice as true today. (Just think about all the AI adoption metrics being rolled out — back to productivity mistaken for lines of code produced, only more elaborate. And expensive). What I like about this book is that Glass doesn't present anything new. Quite the opposite, actually. Rather, it's about research lessons that we all should know, but tend to forget. Ever had to do an estimate, or plan according to a requirements spec? Or maybe you thought that enough eyeballs make all bugs shallow? Then this book is for you. A great work by a fantastic author.
Multi-Paradigm Design for C++ by James O. Coplien. Twenty years after reading this, the core techniques are still with me. These are commonality analysis, where the purpose is to identify families of systems, and variability analysis, which focuses on capturing the domain parameters that vary. In essence: the foundation of great software design.
that your model is really table stakes, but eval and metrics, that's your most important. That's your strategic moat. So, build your eval before you build your technology.
there are real downstream impacts to people's businesses that often come from just like a lack of nuance in understanding metrics, particularly averages.
something we learned from YC uh was that the importance of focusing on very concrete easy to explain customer problems. Like it's it's very easy to to hallucinate or to you know imagine some customer problem that's not actually something viscerally felt by a person who would pay money.
Uh, business are uh, the time to revenue for new companies incorporated with Atlas is declining. And so, by all the kind of objective metrics we can look at, uh, it seems to be a better time than ever to start a business.
So we've always told computer scientists from the very beginning that really it's really important to specify what it is, what's the software that you're writing is trying to accomplish before then going and writing it. And so now we actually have agent-based systems that can do the writing, but the importance of specifying what what it is you want has actually gone up because before you'd be handing it off to a very intelligent human who maybe has context or can ask you follow-up questions.
In a world of AI with agents operating across multiple systems, wanting source of truth data, the importance of having preferred paved paths that get the most of the benefits and produce some guardrails so we can make sure we're doing good work, common infrastructure, common paved paths, solving problems once with a core set of capabilities becomes more important.
since 1970, natural forcings have been roughly flat, internal variability averages out over the longer period, and greenhouse gases explain essentially all of the observed warming.
And so I think you are right that lean is maybe overrated on the side of the importance of it being used as a VR environment for any kind of like just progress in math generally. But I I I definitely wouldn't write it out of the story.
but the thing about AI is that, you know, all of many of these things we were talking about how important they were and how valuable were and trying to persuade people of the importance of them. Yes, even the internet. That may sound surprising, but there were people who would weren't thinking that was important. Um but AI, there's kind of no argument about how important it is. People can't can I mean, you cannot put blinkers on to deny the importance of this thing.
Because Enthropic saw it before Google. And then Google had Nano Banano and Gemini 3 which caused their user metrics to skyrocket and leadership at Google was like oh and then they started making the statement of we have to double compute every is it 6 months or I don't remember the exact number that they said.
This one only gets better with age. Although it’s only mentioned briefly, this is where Andy Grove first introduced OKRs to the world. His practical advice about meetings, especially the importance of 1-on-1s, inspired
Product managers need a solid foundation in statistics to be metrics-driven. This classic book is a lively and fun book will leave you smarter and more skeptical.
the marketing metrics the finance department has faith in and demands from marketing are not those metrics which are most conducive to building brand and customer value over time. They're the metrics which are most conducive to selling tech solutions to the companies.
one of the one of the ways in which my thinking has been changing is that I now place more importance on AI being deployed incrementally and in advance.
we literally at some point had a team that would that was called blockers and they just went and one by one struck them down and each time we saw uh improvement in retention, improvement in activation the metrics for as we addressed each one you could literally see the change in the graph.
Because it's like okay we got these numbers and the numbers go up. It's like no, the fact that you still have to choose which things you look at is an art, not a science. And your interpretation of if the number went up 5%, is that good? Is that not good? Is also an interpretation and is an is an art, not a science.
We live in the era of the symbolic executive, when "being good at stuff" matters far less than the appearance of doing stuff, where "what's useful" is dictated not by outputs or metrics that one can measure but rather the vibes passed between managers and executives that have worked their entire careers to escape the world of work.
So most people, when they advertise new context window increase, they talk a lot about finding the needle in the haystack sort of evaluation metrics and less about whether there’s any degradation in the instruction following performance. So I think that’s where you need to make sure that throwing more information at a model doesn’t actually make it more confused.
Accessibility has failed as a way to make computers usable for disabled users. My metrics for usable design are the same whether the user is disabled or not: whether it’s easy to learn the system, whether productivity is high when performing tasks, and whether the design is pleasant — even enjoyable — to use.
Unlike other interventions EA has sponsored, there are scant metrics for tracking the success or failure of investments in existential risk mitigation.
And I notice that the tiny handful of people capable of caring about 200,000 people dying of neglected tropical diseases are the same tiny handful of people capable of caring about the next pandemic, or superintelligence, or human extinction.
I spoke with @mariamenounos about the link between emotions and illness, and the importance of focusing on mental health, on her Heal Squad podcast. To listen to the full episode, please follow: https://t.co/XPIKAvi8tH
Initial massive exaggeration of the importance of an effect gives way to more realistic, much smaller effects (which might still be useful) after people start doing better studies.
Not only are the traditional staples of classic value investing (readily discernable quantitative measures of cheapness in the here-and-now) no longer likely to produce a sustainable edge on their own, but the world has gotten more complex, with many more dynamics that can drive a decoupling of near-term metrics from valuation, both to the positive and negative.
If you want to learn the basics of Customer Development in only an hour or two, this is the book to read. This is a book you can give to any member of your startup and have them understand the importance of and how to implement customer development in your company.
If you ever needed convinced of the importance of why starting with a small experiment is really important this is the book to convince. With interesting stories from Pixar and amongst others, it makes a compelling case.
As generative drug analoging grows in importance, it's going to be crucial for people to make the entire training set available in detail when such work is published.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.