Martin Fowler thinks that working with LLMs represents as fundamental a change in software development as the shift from assembly code to higher-level languages like Ruby and Fortran was. But unlike those transitions, this isn't just a jump up in abstraction level. Martin writes:

When I wrote a Fortran function, I could compile it a hundred times, and the result still manifested the exact same bugs. Large Language Models introduce a non-deterministic abstraction, so I can't just store my prompts in git and know that I'll get the same behavior each time. As my colleague Birgitta put it, we're not just moving up the abstraction levels, we're moving sideways into non-determinism at the same time.

I think that how much of an impact this has will depend on where you use LLMs in your software development process. When you're working with an LLM to create code, but then committing the results to be used, the line of non-determinism stops well within your view. On the other hand, if you're using prompts to write code and not looking at the output, then you'll have to sort out what systems you need to provide accountability that the final result meets your requirements, even if you're not able to inspect every line of code.