"understand the human language, write programs, and find bugs in a complex code base"
LLMs do none of those things. "LLMs are language models, and their superpower is fluency. It’s this fluency that hacks our brains, trapping us into seeing them as something they aren’t."