Monitoring COT and communication between agents is also getting more difficult as networks get bigger and models become more advanced, according to @ConnorTabarrok, policy lead at Equistamp, a startup that provides safety evaluations for AI projects.
The subject this post names, from the same vocabulary
the directory files beliefs under, and the words it uses that this site has seen
least often elsewhere. Posts are matched on those words alone —
nothing here is a summary of this one.
The difference between software engineering at large corporations and small ones will increase. Start-ups adopting practices of Google will look even sillier than before, because they can now build so much faster and with so much less constraints.
Technologies are not neutral, but political. Money is indeed one of the incentives, but not the only one. A mixture of millennialism, eugenics, search for utopia and for a spread of the specie beyond the confines of the Earth, are all elements we find among the founders’ views.
In the most early adopting tech pioneering place that's Silicon Valley new startups aren't even building software anymore And it's debatable if anyone will actually need a custom harness or it won't just be generically offered by the AI frontier companies So software is mostly dead and hardware it is
One thing that's under appreciated on the website of the demos is the Stanford demo where Ben showed uh anywhere between 3 to 25 images you can reconstruct that entire Stanford quad. But the thing is we had to show it from aerial view. But every single input image is been standing on the ground taking a picture from the ground. So, everything you see are generated but according to the laws of reconstruction.
there's still a lot of startups in this batch that are not shipping fast enough. And so obviously there's variation in shipping speed. It's not just the rate at which you can produce things. You have to think of these ideas first, right?
even in this incident we saw there was a lot of pressure um as a result of this incident to stop doing cyber security evaluations and I really don't think that stopping doing evaluations and like sort of blinding ourselves to the result of evaluations is the right reaction to this problem.
But from their perspective, they've just been trained for millions of subjective years to do as well as they possibly can on these evals. In many cases, the only way in which they've been able to perform well on that training is explicitly by cheating, right?
Met a German founder this week and asked him if all the stories one reads about the challenges of startups in Germany are exaggerated. "No, they're understated." Proceeded to describe spending a full day having a 90-page investment contract read to him (mandatory under German law; § 13 BeurkG) by a notary that then charged €30,000. That was for his first company. His second company, needless to say, was not incorporated in Germany.
I think it's all about having regulatory clarity and ease for all sizes of of companies. Um and if we can work with Congress to do something like that, I think that would be that would be the biggest boon for for for startups.
if if DC is in a vacuum isn't hearing anything from the startup ecosystem or even from big tech ecosystem or even from financial services or all these different industries, we can't make the best decisions.
The most successful skill is if you don't just have one, but you have variety and you can switch between them and you can use context to switch between those.
And the type of risk that we are asking investors to take is um, we know the physics works. We know that there's infinite demand and how we go from here to there is technology execution. And guess what? Venture capital in the United States is the best at underwriting tech risk of anywhere in the world.
when AIs are extremely extremely capable my view is that those AIs will be harder to align than current systems. So for current systems, we have this feedback loop where we basically like we create an AI. We do some evaluations on it. We see that it has some kind of messed up behavior that we can kind of quickly understand. Then we like can like go look in training and be like, "Oh, the these training environments led to this problematic behavior. Let's like tweak that training data. Let's introduce some additional training data to like correct this other issue and then move forward from there." But in a regime where the AIs are extremely situationally aware, very very very very capable and um you know uh we don't necessarily understand what they're doing, this feedback loop breaks down.
Facts and Fallacies of Software Engineering by Robert L. Glass. In essence, this is a book about an industry that refuses to learn. That was true 25 years ago when this book was published, and it's probably twice as true today. (Just think about all the AI adoption metrics being rolled out — back to productivity mistaken for lines of code produced, only more elaborate. And expensive). What I like about this book is that Glass doesn't present anything new. Quite the opposite, actually. Rather, it's about research lessons that we all should know, but tend to forget. Ever had to do an estimate, or plan according to a requirements spec? Or maybe you thought that enough eyeballs make all bugs shallow? Then this book is for you. A great work by a fantastic author.
Many startups are racing to create foundational world models, but I think the first ones to succeed will likely be the platforms in control of this data bank.
And so speed determines success and failure and speed is determined by infrastructure. This is driven by really boring sounding things like how well do your purchasing and recruiting and spending processes work. This is as important as how well do you understand understand the object level technical content of the thing that you're building.
And one of the harder lessons as a startup founder, one of the harder things I think to to really deal with is the fact that you cannot delegate your judgment. As as the CEO, you must always make decisions that make sense to you, no matter how much momentum or inertia alternatives seem to have.
You know, in hindsight, I think that um that was a a poor intuition. Uh it's been pretty robustly and reliably the case over many decades in Silicon Valley has a surfeit of opportunities.
But now, people know that, well, the risk of the status quo is actually extremely high. And so, even if there's risk in doing all the new things, well, this path also looks pretty dangerous. And so, I really think there's never been a better time for startups to to sell um and to have their products get adopted at, you know, pretty meaningful scale right out of the gate.
So, I um yeah, I think maybe maybe a better way of saying it is 20 20 years ago that whole lean startup thing was uh was almost the only thing to do because of capital available and you didn't have AI that made, I don't know, spinning up an organization with many different potentialities and capabilities so much easier, whereas now I think you can start these much more aggressive and ambitious things up front.
many of the companies that were most successful over the last 10 years, so many of them are are very anti-lean startup, right? Uh whether it's, you know, the labs themselves or Anduril or um yeah, you you you you can go down the list. A lot of them have this characteristic.
Yeah, I mean I think uh sometimes it's uh a product that you build that might have access to particular kind of data that the underlying model might not the a general model. So it might be you're building something to help users organize all their own personal information and the model won't necessarily have access to that. And so there you can have a big advantage because all of a sudden your model has visibility or your product has visibility into important data.
And now I actually think with the power of agents um and AI broadly speaking, it's much closer to Goliath versus Goliath. Like I think but maybe the startup is like a Mecca Goliath that is like vastly enhanced by the power of agents and AI and you know, the the large companies are the sort of like more traditional Goliath, so to speak. But I think that startups now like if you properly embrace AI agents and um figure out the way to leverage their strengths in the most like ambitious ways, you can easily outcompete incumbents.
I think concentration of power has basically been bad in every moment of human history um to varying degrees of course but I have a real spirit and I think this is part of the startup spirit of thinking that the world, the economy, society is the best off when power is very widely distributed
it is both true that you know maybe creating super intelligence will be the most important thing yet to happen in the history of business or human society and also that it will pale in comparison to some new startup something that hopefully one of you will do.
In fact, if if we are right that AI is going to be such a big change, startups will be much more important to making sure that the power of this technology gets widely distributed throughout the economy and society and is not just concentrated in a few companies or models.
one of the most important things we discovered along the way is it's extremely difficult to iterate with outside suppliers, particularly outside suppliers in aerospace, which are let's just say I have no kind words for them. So we chose to build our own machine shop. And now we can go from a digitally designed engine part to a prototype part in about 24 hours.
the the big lesson is that for me is technology is changing all the time, and so long as you're able to confront the reality, so long as you are able to learn, the technology itself actually doesn't matter.
one of the things I've always believed believed in is what makes great companies is a unique perspective about the world that you deeply believe in. It's not so much the technology, it's not so much uh the market even.
For the first time, I ran all potential winners through Pangram, which is an AI-writing detector.1 So according to the machine (and to my ear), all of the winners are 100% human-written.
You think this is crazy low but ~5% ownership probably the most common final % most VC funded startups will have when they work out, especially when you have a co-founder
It's actually very difficult because, yeah, you would have to the only way to to really do the evaluations is then delay the model release cycle. Um and you know there's a lot of competitive pressure right now to not do that.
I think like in startups the term is agency. Like somebody who's high agency who's just going to get things done, who's never going to like say no to something. I think like that attitude is really important of like okay, if I don't know something, I'll just learn it.
And usually I like the customer is hyper scale. They have the scale. If they like what you have, they're willing to pay millions of dollars next few years. And even giving some warrant is worth it because you have a big one customer, you can scale.
if if we're if we're too ambitious and we're at the outset and too ambitious and visionary about the product we want to build, then we will probably miss product market fit because we won't start at a small enough humble enough place
in the Peter Teal sense it's almost a moral arbitrage because there's something in our gut as a product you you became a founder an entrepreneur because you wanted to go be an innovator and so it can feel like a beatdown that your path to innovation starts with copying other people's work
I think part of the theory is like if you're building tools that are this complicated, you kind of want to have a 10 to 15 year time horizon on on building out these efforts.
I think the the whole industry bends towards youth for that reason and because it's a hustle business like there there's there's always a rock you haven't looked under and you know age brings children and homes and and and other requirements you get tied to and responsibilities and you're just not able to go spend 80 hours a week studying YouTube like you just can't.
I I definitely think I'm so aware of the fine line between success and failure, especially on your first company. And and and it's so it pains me how much founders and es especially I see it in men, not all men. And I have four sisters and a bunch of daughters. But I would say I see it in a lot in men and and friends, college friends, people I've grown up with that if they had an initial failure, if they had failures, they get attached to it and they start feeling defined by it
The way I think of the abyss is it's this place that we go to as founders and entrepreneurs after our thing, after it dies or it's bought or it's over for whatever reason. It's this amorphous place that we are in our life that has no structure.
did you know that according to the legal documents you yourself signed, your company's literal charter that you have right now, and this is not some hypothetical future thing, you've already put in motion, a rule that says you have a fiduciary duty to say yes in this situation.
According to Harvard Law School, among venturebacked companies that have the standard best practices set up that you got from your lawyer, okay, only 20% of founders are still the CEO 3 years after going public.
so many of the best practices that your lawyers, your bankers, whoever advisers you have, they're going to be pushing best practices on you that are younger than the trees in your local park.
I call it the force that no one controls but everyone obeys that tends to drag organizations down into mediocrity to the point that we lose control of them. Now, sometimes we lose control of them because we get fired.
The Lean Startup helps you build a valuable company. Incorruptible is about how and why to protect it, keeping a company mission-driven over the long term instead of letting it rot from the inside.
Nor nor did I I would say my mistake is I didn't deeply internalize that they they really had no other options that that that a VC would never put in 510 billion of investment into an AI lab with the with the hopes of it turning out to be anthropic. And so that was my miss.
The link between venture capital and evangelical Christianity was closer than I thought. They're not just analogous; they deliberately cross-pollinate.
I believe that data - real-world data, mostly human-generated, validated, and cleaned - is the only reliable moat we have as software founders in the near and mid-term future.
At Anthropic, we don't build for the model of today, we build for the model of six months from now. And that's still my advice to founders that are building on LLMs.
I see no evidence that I'm dragging people along with me. I feel like each each book feels like another startup and that I've got to go out and make it happen almost as if I've not written one.
Because when you believe in something, when you become obsessed with something, you sacrifice yourself to it. And that sacrifice actually feels like pain and stress and doubt. Not fun.
investors at many tech companies, including most on the large cap list, have given up their corporate governance rights, often voluntarily (through the acceptance of shares with different voting rights), to founders and top management in these companies
I just realized the only 2 countries left with actual substantial startup activity now are literally only the US and China The rest of the world can't really do startups, doesn't have the funding, can't grow them and it's more like performative hobby projects for their governments Which might tell us where the future wealth will be concentrated in the world
The panel had a generally positive view of Braintrust, highlighting its clean UI and structured approach to evaluations. The tool’s emphasis on human-in-the-loop workflows was a significant strength.
I think this situation, this historic wrong that’s been done is, put simply, is just a gigantic PR mistake for France. There’s no entrepreneur that sees, that aspires to be the next Pavel Durov to create the next Telegram, sees this and wants to operate in France after seeing this.
I have written forewords for Amir in the past on two of his prior books, Ecosystem Arabia: The Making of a New Economy and Venture Adventure: Startup Fundraising Advice from Top Global Investors, both of which I recommend.
And it struck me how the most well-known brands have stood for one clear thing. Like they have a clear position. And so in order for superhuman to be memorable, I believed that we needed to occupy a clear position that was unique and which was available and which reinforced our product strategy.
I knew that our competition was not going to be startups. It was incumbents. And I also knew that incumbents generally struggle with speed because by definition they have massive scale and usually entrenched architecture.
I think that they're trying to shift the narrative. They're trying to protect themselves. We saw this years ago when ByteDance was actually banned from some OpenAI APIs for training on outputs. There's other AI startups that most people, if you're in the AI culture, were like they just told us they trained on OpenAI outputs and they never got banned.
I really believe that the founder led growth is not being popularized enough that you do not need growth teams until you actually can start running experiments on your user base
Many software investors eschew hard tech startups because of their capital intensity, but it’s hard to deny that huge returns are possible in hard tech: just consider SpaceX.
Fascinating subject. Countries are made of stories. Kings didn’t need their subjects to agree, but nations do. So to build a nation, they need to make a story that helps people feel a shared identity, nationalism, and what distinguishes them from their neighbors. Back-creating a history. Founders of Israel did this brilliantly.
The Social Network is substantially made up, more a source for vibes rather than a source for facts. Even the vibes fail to cohere with reality. And yet it convinced many proto-founders to put in YC applications.
You can set out to build a good business and it’s still fine. Maybe the long-term business model of Perplexity can make us profitable in a good company, but never as profitable in a cash cow as Google was. You have to remember that it’s still okay.
Almost everyone in the class said that the days were moving by slower. Then I would say, “Okay, so do you feel like the weeks are passing by slower, faster, or the same?” The majority of them said that the weeks were passing by faster. According to the laws of physics, I don’t think that makes any sense, but according to memory, it did because what happened was people were doing the same thing over and over in the same context.
In particular, suppose that our control evaluations directly estimate a less than 1% chance of catastrophe if our untrusted AIs are scheming, then the actual risk conditional on scheming is probably more like 5% to 20% due to the potential for failures in the evaluation.
Employees don't want what you want. Customers don't want what you want. Uh you know, that that one of the challenges of the whole stock option thing is entrepreneurs and founders think that other people will be as motivated by owning part of the company as they are. They are not. Not even close.
And oftentimes it's more important to them to have the public perception that they're good directors so they get the next best deal. If they have a reputation for taking on management too aggressively, word will get out in the small community of founders and they'll miss the next Google.
The only interesting problem is dramatically reducing the cost of access to orbit, which is, if you can do that, you open up a bunch of new endeavors that lots of start-up companies everybody else can do. One of our missions is to be part of this industry and lower the cost to orbit, so that there can be a renaissance, a golden age of people doing all kinds of interesting things in space.
According to my numbers, in the United States there will be a net of 3.24 million new low-skill jobs, but there will also be 4.7 million fewer people in the labor force.
Despite evaluations, we cannot consider coming powerful frontier AI systems "safe unless proven unsafe". With current testing methodologies, issues can easily be missed. Additionally, it is unclear if governments can quickly build the immense expertise needed for reliable technical evaluations of AI capabilities and societal-scale risks. Given this, developers of frontier AI should carry the burden of proof to demonstrate that their plans keep risks within acceptable limits.
When you take venture funding, you sign up for a rocket ship ride that will either take you to the moon or to crash-land painfully back on earth. Those are the only two choices. And both rides tend to require heavy extraction of value from the customer.
A successful startup has about five years until they become a new incumbent. Um and and they actually start to behave like an incumbent. That's rational. Like of course that's rational. Like they've now built something worth defending.
the reality is the kids that make new things work from scratch. It actually turns out that they actually have been deep in the domain for a long time. Almost every case, they've been thinking hard about the problem that they're trying to solve actually for in in a lot of cases for many years.
Free software is designed to be used commercially, but you have to do it correctly. This is a resource which is made available to companies who want to exploit it, but they must do so according to the terms of the licenses.
It is the kind of book you will keep by your desk and pull out from time to time to figure out how to approach an issue or to help one of your senior leaders figure out how to do that.
The fast route — venture capital funded — is going to impose constraints on your business that will ultimately make it difficult to remain true to your open-source mission.
Selling is hard. Building a repeatable model with a team that you attempt to rapidly scale is even harder. Fortunately, Mark Roberge, who was one of the very first employees at HubSpot shares the processes and frameworks he used to build a sales machine. If you like the principles behind Lean Startups, then his data- and experiment-driven approach to sales will be very appealing.
It goes much further and deeper into how to approach actually doing Lean in your business than Eric Ries's The Lean Startup book. It also has some awesome case studies showing how lean can apply to any industry. See my full review here.
This is Steve Blank's textbook to starting a company. It's the Four Steps to the Epiphany, expanded and much more readable. If you want to deep dive into the Lean methodology, this is the book to read.
This book was written before the Lean Startup movement, but espouses many of the same concepts. If you feel like you're starting at zero in understanding what to do to become customer driven, this is a good place to start.
Because you radically overestimate the likelihood that your startup will succeed and radically overestimate the portion of the pie that will be allocated to you if the startup succeeds.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.