Diederik P. Kingma
Research scientist at Anthropic; co-creator of the Variational Autoencoder and, with Jimmy Ba, of the Adam optimizer, two of the most widely used tools in deep learning.
Diederik P. Kingma did not write this page. ppll compiled it from their own words, each linked to the source. Is this you? · Report a mistake
Latest x.com
What they believe
-
Stochastic optimization can be done well using only adaptive estimates of the gradient's lower-order moments.
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.
arxiv.orgSaid 22 Dec 2014This on korrents
The bold line is korrents’ wording; the quote is theirs. These one: what they wrote before what they said aloud, one per source, newest first.
Feed
As its own page →Hiding
28 February
30 October 2025
15 October 2025
1 October 2025
3 June 2025
25 April 2025
15 April 2025
2 December 2024
8 October 2024
1 October 2024
1 July 2024
7 May 2024
1 May 2024
10 April 2024
3 December 2022
1 December 2022
30 November 2022
28 November 2022
27 November 2022
9 November 2022
5 November 2022
Nothing matches.
Beliefs
Korrents What they believe 4 beliefs — each backed by an exact quote.
Each is a — compiled by korrents.com, not by them: the one-line wordings are korrents', the quotes are theirs.
Recent
Stochastic optimization can be done well using only adaptive estimates of the gradient's lower-order moments.
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.
Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv) Said 22 Dec 2014
An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train.
The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters.
Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv) Said 22 Dec 2014
The same optimizer should handle objectives that move under it and gradients that are noisy or sparse.
The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients.
Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv) Said 22 Dec 2014
Show 1 more
An optimizer's hyper-parameters should mean something a practitioner can reason about, and should rarely need tuning.
The hyper-parameters have intuitive interpretations and typically require little tuning.
Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv) Said 22 Dec 2014
Beliefs others hold too
Stochastic optimization can be done well using only adaptive estimates of the gradient's lower-order moments. 2 hold this
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.
Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv) Said 22 Dec 2014
An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train. 2 hold this
The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters.
Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv) Said 22 Dec 2014
The same optimizer should handle objectives that move under it and gradients that are noisy or sparse. 2 hold this
The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients.
Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv) Said 22 Dec 2014
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
Where they publish
x.com @dpkingma Notes on generative modeling and deep learning research.
His own account, active since his PhD years, in his own voice.
Recent
- Obviously I strongly support Anthropic's position here. 28 Feb 2026
- 1X seems to have the right approach to developing safe humanoid home robots. Developing full autonomy will require lots of in-distribution demonstration data, so this launch, mostly tele-operated, makes a lot of sense. I expect such robots to be ubiquitous in 5-10 years. 30 Oct 2025
- I really liked this interview, where Geoff manages to explain the essential bits for a general audience. 15 Oct 2025
Show 16 more
- Think, people. Like many other technologies, AI technology has applications that are a net negative, unless well regulated. Don't support the anti-regulation lobby and don't support the people and the companies behind it. 1 Oct 2025
- It's already the case that people's free will gets hijacked by screens for hours a day, with lots of negative consequences. AI video can make this worse, since it's directly optimizable. AI video has positive uses, but most of it will be fast food for the mind. 3 Jun 2025
- Doing ICLR'25 test of time award with @jimmybajimmyba in a few mins 25 Apr 2025
- Thank you! See you guys in Singapore next week 🥳 15 Apr 2025
- 👇 Great work led by Yushun (@ericzhang0410) introducing Adam-mini, a version of Adam that, surprisingly, reduces Adam's memory requirement by 50% (!), without negatively affecting convergence rates. Please read Yushun's thread for details! 2 Dec 2024
- Great blogpost by Ruiqi (and other GDM ex-colleagues), clearly explaining the the connection between flow matching and diffusion models. Super happy they took the time to explain this topic, there's confusion on this topic, I think many will find this quite valuable! 2 Dec 2024
- Congrats to @geoffreyhinton for getting the Nobel! His impact is immeasurable, very much deserved. 8 Oct 2024
- Personal news: I'm joining @AnthropicAI! 😄 Anthropic's approach to AI development resonates significantly with my own beliefs; looking forward to contributing to Anthropic's mission of developing powerful AI systems responsibly. Can't wait to work with their talented team, including a number of great ex-colleagues from OpenAI and Google, and tackle the challenges ahead! 1 Oct 2024
- The recording of our talk for the ICLR'24 test-of-time award (with @wellingmax) is now available online: https://t.co/uY5tG2aCUd Biggest live audience I've ever spoken to, with >2000 attendees 😅. But it was a lot of fun! 1 Jul 2024
- Thanks to the ICLR Award Committee! And thank you for the kind words, Max! You were the perfect Ph.D. advisor and collaborator, kind and inspiring. I really couldn't have wished for better. 7 May 2024
- What are, according to you, the most interesting or most under-appreciated applications of VAEs? Please share below. Links appreciated. (I'm building a list for future reference.) 1 May 2024
- I've had private beta access to Udio and it's wild. Really good music generation, significantly better than the alternatives. Keep an eye in them. 10 Apr 2024
- Funny how the replies reveal that many people can't actually shape rotate in their mind 16 Mar 2024
- Happy to see that our work on diffusion model objectives (https://t.co/0qJRyangWX) is starting to get noticed, with e.g. Stable Diffusion 3 (https://t.co/vHxqwLH0eY) building on our result that the flow matching objective can be simply understood as a special case of diffusion. 11 Mar 2024
- Our paper on diffusion model objectives got accepted at NeurIPS'23 as an oral :-) @RuiqiGao will present our work at 10am CST. (Unfortunately I'm unable to be there myself due to getting covid) I'll post the slides below, feel free to reply with questions or comments. 13 Dec 2023
- It found it interesting to re-read this post by @MoAlQuraishi from 10 years ago (2013). https://t.co/VRzQWeUpVU He was mostly right, except that the development of AI into a mass-market technology started earlier than he expected. 10 Dec 2023
Link verified 9 Sept 2026. Recent items update automatically from the channel.
Mastodon @dpkingma@sigmoid.social The same, on Mastodon.
His secondary account on the ML-focused instance sigmoid.social.
Recent
- “Adam can converge without any modification on update rules” https://arxiv.org/abs/2208.09632 Proves that (vanilla) Adam is theoretically justified without any modification. Presented at NeurIPS'22. By Yushun Zhang, Congliang Chen, Naichen Shi, Ruoyu Sun, Zhi-Quan Luo. Also provides suggestions for tuning hyperparameters beta1 and beta2. 3 Dec 2022
- Pros and Cons of SF, according to chat.openai.com. 1 Dec 2022
- A poster with eight tablets for animations. Wild. Full-sized rollable OLED screens as posters in the near future? #NeurIPS #NeurIPS22 30 Nov 2022
Show 4 more
- Somebody should make bottom-loading kitchen ovens. Could be wall- or ceiling-mounted. Much more energy efficient, and your face wouldn't get blasted every time you open the oven door. Also, kid safe. 28 Nov 2022
- I'll be at NeurIPS in New Orleans from Tuesday to Saturday. Who is going? Let me know if you have a poster presentation I should definitely hit up! Looking forward to catching up! 27 Nov 2022
- This might be blasphemy over here, but to support longer/deeper discussions on Mastodon, I think it'd be nice if Mastodon would implement options for non-chronological lists of posts, e.g. similar to reddit's default 'hot" view. Could be as simple as sorting threads based on most recent reply. This could be opt-in and can co-exist with the current chronological view. 9 Nov 2022
- Just migrated to sigmoid.social, an AI-focussed Mastodon server. Thanks for setting it up @thegradient! Migration was a breeze. 5 Nov 2022
Link verified 9 Sept 2026. Recent items update automatically from the channel.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.