Context: We have a few microservices, each with their own MySQL databases that communicate with each other over HTTP 1.1. A few synchronous operations that constitute a single transaction require writes across two different microservices.

The system can end in states where one of the microservices completes it part of the transaction but the other does not and there is no straightforward way to do rollbacks.

How do you solve this problem?