without metadata prompting when you add lower quality data from 80% data to 100% data the performance actually decreases which is perhaps not too surprising because you're adding lowquality data to your data mixture whereas with the metadata prompting the performance actually increases when you add that lowquality data
If you count all the permutations, you can write all the tests you want. It is impossible to cover all of these and not break things from time to time. It is infinitely harder to evolve software that has users.
If you keep adding deadlines without also checking whether the workload itself is sustainable, you risk what I call Obliger-rebellion-the point where an Obliger who has met, met, met, and met expectations suddenly snaps and refuses to meet any more.
we we kind of have coarse level uh recursive self-improvement already. And the fact that every time you use it, it improves the markdown files. Uh every time you use it, it updates its uh long-term memory.
Or the perfect way to go through feedback and leveling and compensation. But every time we saw that and we added more process, we spent more time without getting better outcomes.
the genie runs out of g runs itself out of options it can't make further forward progress and so I'll wipe it away start over I won't try and tweak I'll start over and say all right well if I implement things in a different order. If I implement with this markdown file or if I implement it with this commit hook, we collectively need to try absolutely everything.
I know through my experiences that even the most advanced artificial intelligences don’t have adequate enough insights to allow one to blindly follow them and that unique human understanding and insights are still invaluable, and that that is especially true in investing where value-added is a zero-sum game (so that, when it comes to adding value, what is widely known is of little value.)
there's a sense in which nothing's yet been completely automated. If you look at the network-adjusted factor shares of a good, which is to say you look down the supply chain and say not just like the final step, but how much of that is done by capital and labor, but what went into the machines that can automate that final step. You'll find that labor's adding a lot of value down the supply chain.
I'm watching people write these huge markdown files to give the agent context. How about you write documentation to give me context so I can have fully working examples.
As reasoning models and agent workflows keep more tokens around (for longer), KV-cache size, memory traffic, and attention cost quickly become the main constraints, and LLM developers are adding a growing number of architecture tricks to reduce those costs.
And the reason they came, I think, is absolutely because of the the better tooling. And I think we were totally right there that like adding a an erasable type system and then using that to enable great tooling is really where the pro where the programmer productivity boost is realized.
The more ants you put in the puzzle, the faster the solution. But the more humans you add, the worse. Unless the humans are very carefully aligned, this is the key lesson for organizational design.
There is no perfect SAK, but the Compact is the best SAK. And this is not a "smartest Kardashian" scenario. Over the years I have genuinely loved carrying the Compact. Adding the pocket clip and the excellent PDW scales has made it one of my most frequently carried items.
if somebody, you know, is standing behind the dessert table and is replenishing, restocking the desserts and keeps kind of you know, adding adding new ones in, it may turn out that you know, a little bit later a much better desserts appear.
we're trying to crack the next frontier which is how we get that level of productivity increase and output building new features on top of a code base that are older,
had we freeze time, that piece of code could be decomposed in a matter of 3 to 6 months. But it took us 2 years to do that because as we peel out a piece of code, the business keep on going forward, right?
It used to be that you have documentation for other people who are going to use your library, but like you shouldn't do that anymore. Like you should have instead of HTML documents for humans, you have markdown documents for agents.
The robot arm base case saves some hourly labor at the expense of adding more skilled labor, and can easily be negative if it needs reprogramming more than a few times per year.
You already bought that phone, why should Apple be adding a 30% junk fee to all commerce you do, and why do they selectively apply it to some things and not others? I've always viewed this as deeply abusive and that it shuts down the competitive engine that once fueled the app and software economy.
The scale word gets a lot of attention in this. The interpretation that I use is effectively to avoid adding the human priors to your learning process. And if you read the original essay, this is what it talks about is how researchers will try to come up with clever solutions to their specific problem that might get them small gains in the short term while simply enabling these deep learning systems to work efficiently, and for these bigger problems in the long term might be more likely to scale and continue to drive success.
And the history of NLP and language processing instruction, tuning and tasks per language model used to be like one language model did one task, and then in the instruction tuning literature, there's this point where you start adding more and more tasks together where it just starts to generalize to every task. And we don't know where on this curve we are.
Installed generators cost ~$800/kW, and the data center capital cost, including servers, is ~$40,000/kW, so adding 1% more makeup compute capacity is money ahead of purchasing generators.
I would heavily discourage any startup that was thinking about starting as a nonprofit and adding a for-profit arm later. I’d heavily discourage them from doing that. I don’t think we’ll set a precedent here.
it will take time for deep learning systems and training data to mature enough to be useful for the truly value-adding task of predicting safety and efficacy in humans
New land can be brought under cultivation (or cultivated more intensively) of course, but marginal gains decrease rapidly (because the best land is cultivated first and because adding more labor to already cultivated land, while it can increase harvests, is less efficient than cultivating new land) and there is a fairly hard ceiling on total production of this sort that was, for the most part, fairly low.
I now suspect that adding a source of hydraulic resistance under the puck that is non-negligible compared with the espresso puck itself is desirable to reduce the impact of channels because it slows down the flow of water where it would tend to become otherwise too fast and as a consequence less even across the surface of the coffee bed.
So there are sharp limits to how much extra range an army can get by adding carrying capacity; in practice about a month and a half’s logistics buffer is the maximum that’s possible and even doing that is expensive.
With each rule and added complexity you make the system less human and less fun. You make it a Computer Scientists rube goldberg machine while sterilizing it of all the joy of life.
Adding a dependency is a serious decision which requires consensus within the team, an audit of the new dependency, an understanding of its health and long-term prospects, and an ongoing commitment to re-audit them and be prepared to change course as necessary.
If it’s your code that gets you over the finish line for every project, you aren’t providing multiplicative value for the team, you’re providing the additive value of your work as an engineer.
For a free Markdown editor (on Mac), I can't recommend MacDown highly enough. It has a simple interface with an editor and preview pane and it just works.
my current favorite notes app. Can grow and get as complex as you need it to be, but for simple notes, the markdown and code highlight support is fantastic.
Adding new folks to a team disrupts that team’s gelling process, so I’ve found it much easier to have rapid growth periods for any given team followed by consolidation/gelling periods where the team gels.
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.