"Enough context" is doing a lot of work in your argument. What is that context? At some point it includes architectural intent, performance constraints, team conventions, future extensibility assumptions ... i.e., exactly the stuff that isn't in the spec and that constitutes the engineering judgment we're discussing. If you force fold all of that into "context," then sure, but now you've just moved the goalposts from spec to "spec plus everything else the developer knows," which is then like a 1:1 map from spec to code.

On inferring how components relate from training data: that gives you statistical priors over topologies which is a good to have sure, but it is not same a justified topology selection for a specific system.