> That model’s usage was well within our budget ($68, about 4kWh of energy use / 365 grams of carbon emissions).

The energy cost is literally 1% of the total cost. For context, 4kWh of energy would drive you about 15 miles in an EV, about half of the average person's driving miles. It's boiling 10 gallons of water.

With the talk of AI Data Center's impact on the world, you'd think this would be 10x to 100x the amount of energy in order to get the effects they're using here.

My takeaway: the AI data center buildout is an overbuild probably at least as large as the fiber buildout that left us with so much dark fiber. If not even bigger. The only thing that will save the economy is the inability of NVIDIA and chip fabs to produce enough chips to match the buildout planned.

> Nonetheless, it’s a good reminder to be careful with model selection and with agentic patterns. We could have achieved similar results for most likely 5x less cost with not that much more effort. Lessons learned! We need to budget for this, and be more careful. Could have seen it coming, but now we know.

I don't get it. Why was it wrong? Which one would have been better? What was the lesson and how could you have foreseen it?

Edit: I found a linked article that mentions the inference provider who does the measurements.