Hillel Wayne

So, it's hard to tell how much of like the loss of the past few years was AI versus the end of like zero interest rate policy and like the post-COVID crash. And I think it's more the latter, but like again, LLMs are still getting better.

I think it is making it more popular. I don't know if it'll make it go mainstream, but it's definitely making it a lot more popular. It's bringing it from maybe like 0.1% to 0.3% which is huge.

And often I found with my clients I have to tell them like it's doing a good job at generating the actual design, but in actually expressing what the design is supposed to do, it cannot do that yet. You have to do that part yourself.

As a general thing we've seen like to get good results you have to already know how to get good results without it. It just helps you get good results faster.

I love formal methods, but I think it's a fairly niche tool for most people and I think like property-based testing is in general going to be useful for more people.

I think a lot of the reason people are skeptical of these is because they've been burned by things like case and UML and all these other miracle solutions that were forced on them by people who wanted them to use it no matter what.

I think the case of TLA+ and most, not all, but most formal methods, they shine the most in highly computational domains, where most of the problems are highly technical and not like business embedded.

And I wonder sometimes if that is the reason people don't recognize the use of math in software engineering is because the math they do need is not the math they've been exposed to.

But I think it is more useful for most developers to have an exposure to like what math has in the various fields versus just going all in every single field when they see them, right? You've got to know what's available to know what's most useful for you. And most math will not be useful for you.

I think a large part of the problem of why it's hard for us is because you don't get a lot of practice. Usually when you have a race condition in a system, you find out months later and then you try a fix and you find out weeks later after that if the fix actually worked.

Then the actual system might still have bugs, but we can iron out the issues in the abstraction such that we don't actually build them in the real system.

but I can tell you with first hand experience nuclear power plants do not care about this stuff. They're actually just fine with with with thorough testing.

when you start talking about like most interesting domain problems, you have to pull in so much context that basically even writing what the function is supposed to do becomes a nightmare. The imperative program you write that will get correct 99% of the time is probably good enough to use in almost all cases.

I think this project and writing about it and thinking about it has firmly moved me from the camp of we are definitely not to we probably are.

And that kind of compiling of information about the materials is something other fields do that we don't do. An analogy that I would think of in software would be something like a 500-page book on how to version an API.

while we are a lot better at iterating than other fields, we're worse at the planning part. Like we still need to do some kind of planning before we iterate and we just aren't as good as those other fields.

Yeah, I interviewed like 20 people on this. I think all 20 mentioned version control as the thing they wish they had in their old field.

Software is kind of unique in having the third kind of the practitioner conference where we are just meeting to get better at what we do. We also are really the only kind to really focus heavily on like open source in making our knowledge freely available.

And the first thing he pointed out to me was that they had their Agile revolution in 1960.

The core tension of engineering is between how expensive it is to make a mistake and how quickly you can iterate. The faster you can iterate, the less planning you need to do before you iterate, and the more expensive it is, the more planning you need to do.

If I had to summarize what I found in general, I'd put it like this. Everybody hates waterfall.