I'd like to get the raw LLM outputs and choose the next token myself. Being able to go back and discard the last few tokens without losing prefix optimization (with appropriate billing) would also be very useful.

The APIs I looked at are all way too high-level. I can go open-source and run things on my own hardware, but that would rule out all the best models.