Karpathy on LLMs as “people spirits”

Andrej Karpathy calls the first paradigm of programming, traditional code, Software 1.0. He calls the weights in a neural net Software 2.0. Software 3.0 is where the neural nets become programmable with English. In order to succeed, we’re going to have to be good at using all three of these paradigms together.