A deep dive into using the seldom used multipart/x-mixed-replace content-type for streaming combined text, audio and JSON from an LLM API.

It was a fun experiment that I think could be useful for many projects. It seems highly likely that as a tech community we are going to be building a lot of software on top of LLM APIs. It also seems highly likely that more LLMs will become multi-modal AND that no matter the speed improvements there will be many use-cases where you want to stream the data to your application rather than wait for the full response.