This pretty much mirrors my experience so far. LLMs are trained in the semantics of experts, but only experts can actually wield them correctly. It allows us to work with the shapes in our heads (shapes the AI understands now) rather than having to convert them into a collection of things expressible in a programming language (which the AI can now do mechanically).

The next frontier will likely be some kind of AI-first IR that contains high level abstractions you can reshape and reason about with the AI (rather than creating documents - which are lower compression) before it commits to an implementation language.

Thats very interesting view. I actually had some experience with that kind of dense language for specifying when I start testing using two llms to code: one to instruct the coding agent and the agent itself. The instructor start creating quite dense english that sometimes I could not easily understand, but seems to make the coding agent pretty efficient in understanding the requirements.

This doesn't resonate:

Consider this prompt:

Make sure that, whenever a job runs, we can tell where it started from, who triggered it, what settings were passed in, and what changed later. In the future, if something breaks, we want to be able to trace it back and understand what happened.

That is fine.

But an expert can say:

Jobs should persist provenance metadata."

That only works if the model is trained, specifically the way you want, to understand that second sentence. If not, any model could work with the first sentence, but not with the second.

You've crated a need for expert training at the model level (which is insanely expensive to create and maintain) rather than accessible natural language discussions that work on any model and understood by anyone. Denser isn't "better" because the words have more power.

Would love some critique of my thoughts.