The book "Peopleware" discusses this.
https://www.amazon.com/Peopleware-Productive-Projects-Tom-De...
Basically, open floor plans are cheap and you can maximize metrics like $/person-sqft.
Other factors:
* "everyone else does it"
* Open floor plans seem like a good idea when you first hear them; the downsides are not immediately obvious. Verbal reasoning, no matter how flawed, almost always wins.
* The decision makers tend to be blissfully unaware of the dynamics of knowledge work because their work is typically interrupt-driven. Thus, they don't see any problem putting 50 programmers in a cramped and echoey gymnasium.
* Even if the decision maker is fully aware, their boss might not agree to a higher-cost office plan. Actual dollars will almost always beat non-metrics like "compounded employee productivity" or "time and money not wasted due to mistakes caused by people operating in a distracting environment".
--Yes, in theory you could design an experiment, but that would delay the decision and probably require spending money. And the outcome would probably be misinterpreted: the open floor people will be louder and will appear to be working harder. (But are they actually more productive or are they just scrambling to fix all the bugs they created?)