Holden Karnofsky
Co-founded GiveWell and led Open Philanthropy; writes Cold Takes, on the most important century and how to think about it.
Holden Karnofsky did not write this page. What is this?
It collects the places they publish and what they have said there, each linked to the source. They have no account here. Is this you? Claim it, correct it, or ask us to remove it from ppll.
Where they publish
Blog Cold Takes His blog on transformative AI, philanthropy and reasoning. Has a feed.
Recent
- Good job opportunities for helping with the most important century 18 Jan 2024 I wrote ~2 years ago that it was hard to find concrete ways to help the most important century go well. That’s changing.
- What does Bing Chat tell us about AI risk? 28 Feb 2023 Early signs of catastrophic risk? Yes and no.
- How major governments can help with the most important century 24 Feb 2023 Governments could be crucial in the long run, but it's probably best to proceed with caution.
Show 12 more
- What AI companies can do today to help with the most important century 20 Feb 2023 Major AI companies can increase or reduce global catastrophic risks.
- Jobs that can help with the most important century 10 Feb 2023 People are far better at their jobs than at anything else. Here are the best ways to help the most important century go well.
- Spreading messages to help with the most important century 25 Jan 2023 For people who want to help improve our prospects for navigating transformative AI, and have an audience.
- How we could stumble into AI catastrophe 13 Jan 2023 Hypothetical stories where the world tries, but fails, to avert a global disaster.
- Transformative AI issues (not just misalignment): an overview 5 Jan 2023 An overview of key potential factors (not just alignment risk) for whether things go well or poorly with transformative AI. https://www.cold-takes.com/transformative-ai-issues-not-just-misalignment-an-overview/
- Racing through a minefield: the AI deployment problem 22 Dec 2022 Push AI forward too fast, and catastrophe could occur. Too slow, and someone else less cautious could do it. Is there a safe course?
- High-level hopes for AI alignment 15 Dec 2022 A few ways we might get very powerful AI systems to be safe.
- AI Safety Seems Hard to Measure 8 Dec 2022 Four analogies for why "We don't see any misbehavior by this AI" isn't enough.
- Why Would AI "Aim" To Defeat Humanity? 29 Nov 2022 Today's AI development methods risk training AIs to be deceptive, manipulative and ambitious. This might not be easy to fix as it comes up.
- Beta Readers are Great 5 Sept 2022 Back in January, I posted a call for "beta readers": people who read early drafts of my posts and give honest feedback. The beta readers I picked up that way are one of my favorite things about having started Cold Takes…
- The Track Record of Futurists Seems ... Fine 30 Jun 2022 We scored mid-20th-century sci-fi writers on nonfiction predictions. They weren't great, but weren't terrible either. Maybe doing futurism works fine.
- Nonprofit Boards are Weird 23 Jun 2022 With great power comes, er, unclear responsibility and zero accountability.
Link verified 20 Sept 2026. Recent items update automatically from the channel.
Beliefs
Korrents What they believe 16 beliefs — each backed by an exact quote.
Each is a — compiled by korrents.com, not by them: the one-line wordings are korrents', the quotes are theirs.
Recent
The track record of thoughtful people predicting the long-run future is mediocre rather than hopeless, which is enough to take such predictions seriously.
They weren't infallible oracles, but they weren't blindly casting about either.
The Track Record of Futurists Seems ... Fine Said 30 Jun 2022
Letting one calculation take over a giving decision is disturbing, and there is still a strong case for it if you want your ethics to be principled and about other people.
These ideas can be disturbing and off-putting, but I think there is also a strong case for them, for those who wish their ethics to be principled and focused on the interests of others.
Defending One-Dimensional Ethics Said 15 Feb 2022
Ethics based on common sense and the conventions of the time has a horrible track record.
Ethics based on "common sense" seems to have a horrible track record.
Future-proof ethics Said 2 Feb 2022
Show 13 more
The best opportunities to do good are the ones almost nobody is paying attention to.
Indeed, I think some of the best opportunities to do good in the world come from working on issues that aren't yet widely recognized as huge moral issues of our time.
Future-proof ethics Said 2 Feb 2022
Counting every being that can suffer, and applying consistent moral weights, yields conclusions its own defenders find uncomfortable and cannot fully endorse.
Combining these three pillars yields a number of unusual, even uncomfortable views about ethics. I feel this discomfort and don't unreservedly endorse this approach to ethics.
Future-proof ethics Said 2 Feb 2022
Any harm can in principle be outweighed by a small enough benefit delivered to a large enough number of people.
In theory, any harm can be outweighed by something that benefits a large enough number of persons, even if it benefits them in a minor way.
Future-proof ethics Said 2 Feb 2022
Almost everything anyone believes rests on trusting other people, and there is no alternative to that.
I think it's completely reasonable to form the vast majority of one's beliefs based on trust like this. I don't really think there's any alternative.
Minimal-trust investigations Said 23 Nov 2021
Occasionally suspending your trust in everyone and digging into one question as deeply as you can is the single most formative thing you can do for how you think.
Minimal-trust investigation is probably the single activity that's been most formative for the way I think.
Minimal-trust investigations Said 23 Nov 2021
A giving approach that ranks options only by estimated expected value, with no preference for better-grounded estimates, is flawed.
I feel that any giving approach that relies only on estimated expected-value – and does not incorporate preferences for better-grounded estimates over shakier estimates – is flawed.
Why we can’t take expected value estimates literally (even when they’re unbiased) Said 18 Aug 2011
Give where there is strong evidence that donations do a lot of good, rather than where weak evidence suggests they might do far more.
And we generally prefer to give where we have strong evidence that donations can do a lot of good rather than where we have weak evidence that donations can do far more good
Why we can’t take expected value estimates literally (even when they’re unbiased) Said 18 Aug 2011
Ranking charities by their estimated expected value alone removes any reward for being transparent about how they actually work.
EEV doesn’t seem to allow rewarding charities for transparency or penalizing them for opacity: it simply recommends giving to the charity with the highest estimated expected value, regardless of how well-grounded the estimate is.
Why we can’t take expected value estimates literally (even when they’re unbiased) Said 18 Aug 2011
A world where altruists followed expected-value estimates would be one where nearly all of them helped distant strangers instead of the people around them, and that world would be worse.
In such a world, it seems that nearly all altruists would put nearly all of their resources toward helping people they knew little about, rather than helping themselves, their families and their communities. I believe that the world would be worse off if people behaved in this way, or at least if they took it to an extreme.
Why we can’t take expected value estimates literally (even when they’re unbiased) Said 18 Aug 2011
The moment one cause looks far better than every other on thin information is exactly the moment it most needs sceptical investigation.
Yet it seems that when people are valuing one action far above others, based on thin information, this is the time when skeptical inquiry is needed most.
Why we can’t take expected value estimates literally (even when they’re unbiased) Said 18 Aug 2011
The more an argument asks you to do, the more evidence you should require before doing it.
The more action is asked of me, the more evidence I require.
Why we can’t take expected value estimates literally (even when they’re unbiased) Said 18 Aug 2011
An ungrounded estimate that promises an enormous payoff should be discarded rather than discounted, because the size of the claim is itself the reason to distrust it.
An ungrounded estimate making an extravagant claim ought to be more or less discarded in the face of the “prior distribution” of life experience.
Why we can’t take expected value estimates literally (even when they’re unbiased) Said 18 Aug 2011
The deeper anyone digs into a charity's cost-effectiveness estimate, the more unwarranted optimism they find, so the right prior on giving is a sceptical one.
Giving well seems conceptually quite difficult to me, and it’s been my experience over time that the more we dig on a cost-effectiveness estimate, the more unwarranted optimism we uncover.
Why we can’t take expected value estimates literally (even when they’re unbiased) Said 18 Aug 2011
Beliefs others hold too
The best opportunities to do good are the ones almost nobody is paying attention to. 2 hold this
Indeed, I think some of the best opportunities to do good in the world come from working on issues that aren't yet widely recognized as huge moral issues of our time.
Future-proof ethics Said 2 Feb 2022
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
Feed
As its own page →Hiding
18 January 2024
28 February 2023
24 February 2023
20 February 2023
10 February 2023
25 January 2023
13 January 2023
5 January 2023
22 December 2022
15 December 2022
8 December 2022
29 November 2022
5 September 2022
30 June 2022
-
Their words
They weren't infallible oracles, but they weren't blindly casting about either.
23 June 2022
Nothing matches.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.