Related posts
Andrej Karpathy
GitHub
build-nanogpt — Video+code lecture on building nanoGPT from scratch
nanogpt lecture scratch
The the words it uses that this site has seen
least often elsewhere. Posts are matched on those words alone —
nothing here is a summary of this one.
Sources
All Writing x.com Bluesky Newsletter Korrents GitHub Blog Mastodon Recommends Site
Top people
Sebastian Raschka
Arnold Kling
Timothy Snyder
Balaji Srinivasan
Kit Yates
Lenny Rachitsky
Rakhim Davletkaliyev
Lex Fridman
Andrej Karpathy
Pieter Levels
Matt Birchler
Daniel Lemire
Showing Profile →
Show everything
Hiding Show them again
Show them again
Further back ↓
Hiding Show them again
19 September
Dutch bootstrapper who builds and runs their internet startups alone — Nomads.com, Remote OK, Hoodmaps, Photo AI and Interior AI — and has written about it at levels.io since 2013. Posts as @levelsio.
🌎 Now redesigning the https://t.co/HGCLKS5BD6 profile trips globe from scratch I've always wanted to add space trips, so starting with the moon here which you can add as a future trip, also adding Mars etc. Other trips also need to work here though like going into orbit like @satofishi did on @SpaceX's Dragon capsule which will be more in reach for more people hopefully pretty soon! Quoting @levelsio ✨ Finally built a timeline for https://t.co/HGCLKS5BD6 now I always wanted to make my site a bit more social, because I had profiles and trip overlaps and you could kinda see where other people went, but not something like this So on this new page you can see people near you, where everyone's trave… Related
Mathematical biologist; Senior Lecturer in the Department of Mathematical Sciences at the University of Bath and author of The Maths of Life and Death and How to Expect the Unexpected.
x.com So excited to be back speaking in this fantastic lecture theatre at the Royal Institution. But this is a lot of books to sign after the talk. My hand is going to be sore. Related
Bluesky So excited to be back speaking in this fantastic lecture theatre at the Royal Institution. But this is a lot of books to sign after the talk. My hand is going to be sore. Related
Economist; writes In My Tribe, and wrote The Three Languages of Politics.
Designer, product manager, YouTuber and developer who has written about technology at Birchtree since 2010. He makes the Quick Reviews and Quick Notes apps and the A Better Computer YouTube channel.
Their words
app developers don't have to start from square one and reconceptualize their apps for larger displays because a lot of them have already done that by supporting the iPad.
Show the whole quote
birchtree.me
17 September
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
16 September
Computer science professor who works on fast data processing; co-author of the simdjson parser and a weekly blogger about software performance since 2004.
American historian of Central and Eastern Europe and the Holocaust; author of Bloodlands, On Tyranny and On Freedom, and writer of the newsletter 'Thinking about...'.
15 September
Economist; writes In My Tribe, and wrote The Three Languages of Politics.
13 September
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
Reasoning from scratch round 3: This time, I cover generating a verifier for... a) ...evaluation (base model versus any future model improvement) b) ...the reinforcement learning with verifiable rewards (RLVR) training later on 00:00 Introduction 01:21 Four approaches to LLM evaluation 07:20 Verifiers and reinforcement learning with verifiable rewards 10:52 Notebook setup and dependencies 13:43 Section 3.1 Building a math verifier 18:57 Section 3.2 Loading a pre-trained model to generate text 24:34 Generating and displaying model answers 29:23 Section 3.3 Implementing a wrapper for easier tex… LLMs Related
Marketing author and daily blogger; founder of the altMBA and the Akimbo workshops, and author of books including Purple Cow and This Is Marketing.
When does it become a speech? One person isn’t a lecture, it’s lunch. Three people is office hours. Six people is a seminar. At some point, we shift from interaction to performance–but the people are the same people, the very ones that were happy to…
Related
11 September
Economist; writes In My Tribe, and wrote The Three Languages of Politics.
Co-founder and CEO of 37signals, makers of Basecamp and HEY; co-author of Rework, Remote and It Does Not Have to Be Crazy at Work.
A really lovely GRID VIEW is coming to Basecamp 5 shortly. We had a grid view in 4, but it wasn't any good and I didn't want to drag it into 5. Had we done that, we'd have called it done and never revisited. By intentionally leaving it out, we allowed pressure to build to bring it back. And with no legacy to battle or revamp, we were able to return to the drawing board and make something so much better, from scratch. Coming to a Basecamp near you. Soon. Related
10 September
Artist and developer who makes esoteric software, games and tools aboard a sailboat as part of Hundred Rabbits with Rekka Bellum; creator of the Uxn virtual machine and the Orca livecoding environment.
My CD-ROM of Proust's Contre Sainte-Beuve comes with a shareware version of Cailleux's Lecture. #theLibrary Related
9 September
Mathematician at UCLA, working mainly in harmonic analysis and partial differential equations. Writes What's new, a long-running blog on their research, open problems and expository notes.
Javier Gomez-Serrano will give a special lecture at Harvard this Friday on recent developments in fluid mechanics: cmsa.fas.harvard.edu
Related
Writer of Lenny's Newsletter and host of Lenny's Podcast, covering product, growth and career in tech. Previously a product lead at Airbnb. Also an angel investor and advisor.
I asked Grok Bot product lead @romanugarte_ why they built a new stand-alone app vs. building it into Cursor "It was not obvious at all. We saw what some of our competitors doing this, adding new tabs for each new form factor. People can feel that this was not a single, consistent vision of the way that work should work. It's kind of a shipping your org chart style thing that users are reacting negatively to. And so we decided, let's just start completely from scratch." Quoting @lennysan Grok @Bot is the hottest new AI product in the world right now. I'm absolutely hooked. It all started a few months ago as a tiny team holed up in a separate part of the office, building from scratch. Four weeks after the first line of code, the whole company was obsessed. Three weeks later, it laun… Related
American historian of Central and Eastern Europe and the Holocaust; author of Bloodlands, On Tyranny and On Freedom, and writer of the newsletter 'Thinking about...'.
8 September
Writer of Lenny's Newsletter and host of Lenny's Podcast, covering product, growth and career in tech. Previously a product lead at Airbnb. Also an angel investor and advisor.
Grok @Bot is the hottest new AI product in the world right now. I'm absolutely hooked. It all started a few months ago as a tiny team holed up in a separate part of the office, building from scratch. Four weeks after the first line of code, the whole company was obsessed. Three weeks later, it launched to the world. Less than a month in, millions of Grok Bots are already helping people get work done. Roman Ugarte (@romanugarte_) incubated, launched, and now leads product for Grok Bot. This is the first time he's told the story: 🔸 Why they started fresh instead of building it into Cursor 🔸 T… Related
6 September
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
Reasoning from scratch round 2: In this video, I cover the text generation process in LLMs and KV caching (to prepare the base model before adding reasoning techniques in the upcoming ones). 00:00 Introduction and reasoning model demo 01:55 How to work through the book 05:00 Chapter 2 overview 08:25 Checking PyTorch and hardware support 10:26 Apple silicon and MPS caveats 15:00 Cloud GPU options 16:08 Tokens and tokenization 18:20 Qwen3 and the Reasoning From Scratch package 23:05 Encoding and decoding text 26:24 Downloading weights and selecting a device 31:01 Loading the pretrained Qwen3 mo… LLMs Related
Economist; writes In My Tribe, and wrote The Three Languages of Politics.
4 September
Economist; writes In My Tribe, and wrote The Three Languages of Politics.
2 September
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
A lot of hype around OpenAI's Astra model here on my timeline today. Apparently, this goes back to a new article from The Information, which said Astra is a "recurrent depth or looped transformer". It's always interesting to read about new or different approaches (including rumors about what the closed labs may be up to), but let's debunk this a bit. About 2 months ago, I shared the architecture details of Nanbeige, for example, where "Nanbeige4.2-3B is pretrained from scratch on 28T tokens with a Looped Transformer that reuses the layer stack to increase capacity without adding parameters."… OpenAI Related
American historian of Central and Eastern Europe and the Holocaust; author of Bloodlands, On Tyranny and On Freedom, and writer of the newsletter 'Thinking about...'.
1 September
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
30 August
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
A little video that - explains the relationship between conventional LLMs and reasoning models (and agents), - philosophizes a about "from scratch" approaches, - and explains how to install Python & PyTorch requirements with uv. LLMs Related
29 August
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
Thanks everyone for all the nice feedback on Build a Reasoning Model (From Scratch) so far! I am also flattered that 2 book clubs are discussing it. I look forward to join for live Q&As on Thu, Sep 3, at 10 am (and 2 pm CT). Please join us & bring your questions! Quoting @sophiamyang Join The AI Book Club for a live conversation with @rasbt about his new book, Build a Reasoning Model (From Scratch)! 📅 Sep. 3, 10 AM CT 🌐 Online We'll talk about how reasoning models work, how to build them, and take questions from the community. Related
28 August
Developer and podcaster; created Instapaper and Overcast, co-founded Tumblr, and co-hosts the Accidental Tech Podcast.
RE: https://mastodon.social/@martimw/117174328129113589 From scratch. Making custom models has turned 3D-printing from a family novelty into a full-blown craft and life-improving hobby for me. I started in TinkerCAD, which can get you VERY far, VERY quickly. Now I make most models in Onshape, since I've built up the skills to be pretty fast with it, too. I sometimes print stuff I find on Makerworld, but usually the problems I'm trying to solve are too specialized. Related
26 August
American historian of Central and Eastern Europe and the Holocaust; author of Bloodlands, On Tyranny and On Freedom, and writer of the newsletter 'Thinking about...'.
24 August
Creator of the Keras deep-learning library and the ARC-AGI benchmark.
Their words
If you're 17 (or any age) and you want to learn to build LLMs from scratch, read chapters 15-16 of Deep Learning with Python
Show the whole quote
x.com
23 August
Pulitzer Prize-winning reporter and author of The Power of Habit, Smarter Faster Better and Supercommunicators. Writes for The New Yorker.
You probably waste hours staring at a blank screen trying to invent a completely original idea from scratch. But data shows that 90% of the most creative breakthroughs are just common concepts stolen from different industries and mixed together in a new way. You can generate brilliant ideas instantly using the Intellectual Middleman method. Here is how it works: 🧵 Related
Co-founder of Y Combinator and Viaweb; essayist at paulgraham.com.
LLMs
Their words
Someone asked what I'd do if I were 17. I'd learn how to build LLMs from scratch, and then train ones as powerful as I could with whatever hardware I could get access to.
Show the whole quote
@paulg on X x.com
22 August
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
19 August
Writes The Pragmatic Engineer, the software-engineering newsletter, and wrote The Software Engineer's Guidebook. Formerly an engineering manager at Uber, and at Microsoft/Skype and Skyscanner before that.
If you’ve ever opened Chrome DevTools, or optimized a page for Core Web Vitals, you’ve used software built by @addyosmani.bsky.social. We start from when he built a web browser from scratch, at 16, through his 14 years at Google, to what he sees working (and not working) building software with AI: Google Related
American historian of Central and Eastern Europe and the Holocaust; author of Bloodlands, On Tyranny and On Freedom, and writer of the newsletter 'Thinking about...'.
18 August
US science and technology adviser to the president, director of the White House Office of Science and Technology Policy; previously US chief technology officer and chief operating officer of Scale AI.
Their words
The thing about executive orders, which is a little secret, is when administration changes, you can just revoke the executive order and and start from scratch again. Um when something is a law, you you can't do that.
Show the whole quote
youtube.com
16 August
Programmer and teacher; writes on software, learning and the web, and makes the Coding Blocks video course.
Their words
Having read this after LOTR, it was fun but underwhelming. Starting from scratch, it’s definitely better to start with The Hobbit.
Show the whole quote
rakhim.org
Their words
Woah, what a ride! An excellent explanation of how computers work, starting from scratch. Really, from scratch.
Show the whole quote
rakhim.org
15 August
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
1 August
Swedish designer and programmer. Designed early Spotify, worked at Facebook and Figma, and created the Inter typeface.
Built a Playbit app that implements Stable Audio 3 completely from scratch in C and WebGPU compute shader. Fun little toy; a spin-off from my Dream Together tooling! Related
20 July
Polyglot who speaks twenty languages and co-founded LingQ; argues that comprehensible input, not study, is what does the work.
Their words
If you are starting a language completely from scratch, authentic media is just noise. You’ll absorb very little. You need highly repetitive, simple input.
Show the whole quote
blog.thelinguist.com
15 July
Founder of HumanLayer; wrote 12-Factor Agents, on how to build AI agents that hold up in production.
1 July
Self-taught home cook and food writer who has run the Smitten Kitchen recipe site since 2006; author of four cookbooks.
30 June
Economist at the Hoover Institution; wrote The Fiscal Theory of the Price Level and blogs as The Grumpy Economist.
2 June
Artist and writer based in Oakland, California; author of How to Do Nothing and Saving Time, and a longtime teacher of digital art.
a new essay adapted from my Earth Day lecture at the College of Idaho - inspired by my friend Tom, the Santa Cruz Mountains, and James Cameron's 1989 film, The Abyss: longreads.com
Related
29 May
Hosts the Lex Fridman Podcast — long, unhurried conversations with scientists, engineers and public figures. Several of the interviews quoted elsewhere on this site are episodes of it.
Their words
One of my favorite lecture series he has given is The Evidence for Modern Physics where he breaks down the experiments that validate some of the weird laws of physics we have, and what it would take to validate even the weirder ones.
Show the whole quote
x.com
28 May
Professor of international affairs at Johns Hopkins SAIS and co-author, with Abraham Newman, of Underground Empire: How America Weaponized the World Economy.
12 February
Hosts the Lex Fridman Podcast — long, unhurried conversations with scientists, engineers and public figures. Several of the interviews quoted elsewhere on this site are episodes of it.
coding agents
Their words
It’s actually how good they are at programming is almost a burden in their ability to empathize with the system that’s starting from scratch. It’s a totally new paradigm of, like, how to program. You really, really have to empathize.
Show the whole quote
youtube.com
22 January
Robert Gray Dodge Professor of Network Science at Northeastern University; discovered scale-free networks and, with Reka Albert, proposed the Barabasi-Albert model.
Speaking today at 10am EST/4pm EU about our Nature on String Theory and the Brain and the Lognormal Connectome. ...at the U of Cambridge Making connections- brains and other complex systems lecture, Sign up here to watch here: https://t.co/lxigtmZDjJ Photo by Peter Puklus Related
15 December 2025
American author of Liar's Poker, Moneyball, The Big Short and The Undoing Project, and host of the podcast Against the Rules.
30 November 2025
Austrian developer. Created the libGDX game framework, and more recently the pi coding agent, which moved with them to Earendil — Armin Ronacher's company — in April 2026. Writes at mariozechner.at.
12 November 2025
Founding member of OpenAI and former director of AI at Tesla; creator of nanoGPT and the term "vibe coding".
5 October 2025
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
11 September 2025
Machine learning engineer and consultant focused on RAG and retrieval systems. He writes about applied AI engineering at jxnl.co and is the author of the instructor library.
6 September 2025
AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).
13 August 2025
Co-founder of Panic, the Portland software company behind Transmit, Coda, Nova and the Playdate handheld console.
Who else does the involuntary giant yawn / nose scratch at the precise moment Face ID pops up every time thus causing it to fail Related
30 March 2025
Professor at Carnegie Mellon working on self-driving-car safety; writes Safe Autonomy on how autonomous vehicles are and are not shown to be safe.
22 March 2025
Programmer and streamer; was an engineer at Netflix, now teaches and argues about editors, Vim and how people learn to program.
Their words
I highly recommend starting from scratch if you want to understand a concept. You don't know how an HTTP server works, create a TCP socket, learn how to parse HTTP.
Show the whole quote
youtube.com
18 March 2025
Distributed-systems safety researcher; runs the Jepsen consistency-testing project and wrote the Riemann monitoring system.
2 February 2025
General partner at Andreessen Horowitz from 2013 to 2018 and CTO of Coinbase from 2018 to 2019; co-founder of Counsyl and Earn.com. Wrote The Network State (2022), free in full at thenetworkstate.com, and founded the Network School.
Their words
The Knowledge: How to Rebuild Our World From Scratch, How Innovation Works: And Why It Flourishes in Freedom, How to Invent Everything: A Survival Guide for the Stranded Time Traveler, A Connecticut Yankee in King Arthur's Court, Principles for a Changing World Order
Show the whole quote
balajis.com
Their words
The Knowledge: How to Rebuild Our World From Scratch, How Innovation Works: And Why It Flourishes in Freedom, How to Invent Everything: A Survival Guide for the Stranded Time Traveler, A Connecticut Yankee in King Arthur's Court, Principles for a Changing World Order
Show the whole quote
balajis.com
Their words
The Knowledge: How to Rebuild Our World From Scratch, How Innovation Works: And Why It Flourishes in Freedom, How to Invent Everything: A Survival Guide for the Stranded Time Traveler, A Connecticut Yankee in King Arthur's Court, Principles for a Changing World Order
Show the whole quote
balajis.com
+ 2 more
Their words
The Knowledge: How to Rebuild Our World From Scratch, How Innovation Works: And Why It Flourishes in Freedom, How to Invent Everything: A Survival Guide for the Stranded Time Traveler, A Connecticut Yankee in King Arthur's Court, Principles for a Changing World Order
Show the whole quote
balajis.com
Their words
The Knowledge: How to Rebuild Our World From Scratch, How Innovation Works: And Why It Flourishes in Freedom, How to Invent Everything: A Survival Guide for the Stranded Time Traveler, A Connecticut Yankee in King Arthur's Court, Principles for a Changing World Order
Show the whole quote
balajis.com
29 January 2025
Executive coach for product leaders. Previously a partner at GV (Google Ventures) and a product manager at Google. Writes at Bring the Donuts, including a standing essential-reading list for product managers.
19 January 2025
Growth advisor and operator who has led growth at Miro, Amplitude, Dropbox and SurveyMonkey.
Their words
And number one thing that whenever you have an initiative or whenever you have a metric that you need to go and move, do not start from scratch ever. Do not start from scratch.
Show the whole quote
youtube.com
4 January 2025
Design engineer and illustrator; makes visual essays on programming, anthropology and what language models do to the way people write.
17 November 2024
Member of technical staff at Anthropic. He has led ML/AI teams at Amazon, Alibaba and Lazada, and writes about LLMs, recommender systems and engineering at eugeneyan.com.
3 April 2024
Writer and engineer behind the blog Melting Asphalt, and co-author with Robin Hanson of The Elephant in the Brain.
responses here were awesome. thanks everyone! some ppl asked for my recs: — “The Great Courses” (high-quality lecture series recorded in the studio) — particularly the history courses — all-time fav is “Big History” by David Christian — any linguistics course by John McWhorter Quoting @kevinsimler can y'all recommend some audiobooks for me? (i have audible credits i need to use) Related
9 November 2023
Founded SpaceX in 2002, joined Tesla's board in 2004 and became its chief executive in 2008; later founded Neuralink, The Boring Company and xAI. Bought Twitter in 2022 and renamed it X, where he now posts many times a day.
Their words
Because you think of how many electric motors are made in the world. There’s like tens of thousands, hundreds of thousands of electric motor designs. None of them were suitable for a humanoid robot, literally none.
Show the whole quote
youtube.com
26 October 2023
Software engineer and a long-time technical lead of the Go programming language; author of plan9port and of the research.swtch.com essays.
28 September 2023
Co-founder and chief executive of Meta Platforms, the company he started as Facebook in 2004.
Their words
So that's a lot of what I think we need to nail for the creators, and that's why that one is actually a much harder problem, I think, than starting with new characters that you're creating from scratch.
Show the whole quote
youtube.com
30 September 2022
Swedish software developer writing about tooling, working from home and running a software business at seastone.io.
No one said “we have to rebuild this from scratch” without someone saying “we’ll fix that … later” a thousand times before. Related
28 January 2022
Ancient and military historian; writes A Collection of Unmitigated Pedantry, long series checking what history actually says against how popular culture tells it.
Their words
the system of social organization that characterized the ancient world practically vanished and would have to be redeveloped centuries later.
Show the whole quote
acoup.blog
25 January 2022
Co-founder of Netscape and of the venture firm Andreessen Horowitz. Argues at length for building things, mostly in short form.
startups
Their words
the reality is the kids that make new things work from scratch. It actually turns out that they actually have been deep in the domain for a long time. Almost every case, they've been thinking hard about the problem that they're trying to solve actually for in in a lot of cases for many years.
Show the whole quote
youtube.com
21 June 2021
Founding member of OpenAI and former director of AI at Tesla; creator of nanoGPT and the term "vibe coding".
1 March 2020
Indian software engineer working on Android and React; writes about his setup and side projects at ishubhamsingh.dev.
Nothing matches. Show everything
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com .
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
Got it
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.
Got it