ExFed

One of my favorite papers! On a similar note, I recently started reading A Philosophy of Software Design by John Ousterhout. Although it’s a lot more recent (2018), I’d argue it’s required reading in light of the LLM hype craze.

Absolutely, the author needs to be able to reason about their changes, no matter what. However, the reason why I think the two situations are fundamentally different, though, is that it’s a lot easier to validate the existence of features than it is the non-existence of bugs or malicious behavior. The biggest risk to removing code is breaking preexisting features, whereas the biggest risk to adding code is introducing malicious behavior.

Agreed. I have a sense that, eventually, development communities will figure out etiquette and policies to govern LLM usage. But how do you enforce that kind of policy? Right now, it’s essentially a judgement call by the maintainers. It’s hard to catch sneaky LLM usage.

On the other hand, I think there are objectively good ways to use LLMs for software:

• High-level design and planning
• Technical Research (although this tends towards the most popular tech)
• POCs & rapid prototyping
• “Textbook” solutions
• TDD Red/Green development (where the LLM generates failing tests based on the high-level spec, and the programmer writes the implementation)

25kLOC delta in a single PR should be cause for instant rejection

Not to pick at nits, but it would be VERY different if it was 1k lines added and 24k lines removed. There’s something extremely satisfying about removing 10k+ lines of unnecessary code.

I’m the cloud infrastructure business we like to call that the 9mm test. It’s quite effective.

Whoa you gotta put an NSFW tag on images like these.