I've seen lazy devs ignore the underlying issue, and then just push over and over broken code and wait 10-15 min for CI to push new version to the evironment. It's maddening.
Our Docker Compose local dev setup started to break down once we had to model more complicated production behavior locally – things like table-specific Postgres roles for audit logs and dynamically provisioned databases per-clinical-trial. We were drifting toward a bespoke Bash mess to keep dev and prod in sync.
Our core idea was instead to embrace Terraform in the local dev environment too. We were already using Terraform heavily in prod, and Terraform's robust provider ecosystem meant that we could e.g. substitute Docker containers for RDS and MinIO for S3 without deviating too far from our production configuration.
The really fun part is how we use Terraform to handle dynamic provisioning, which we need for isolated, per-clinical-trial databases. The way we do it in prod is by giving each clinical trial its own, isolated Terraform state, stored in a cloud storage bucket. By writing an equivalent local Terraform config for this provisioning step, we enable the app to run the same `terraform apply` command as it does in prod to locally to spin up a new database, with the individual state for that new db stored in a local MinIO bucket... which is itself created by the original `terraform apply` that sets up the initial local dev infrastructure.
Altogether, Terraform gives us a super high-fidelity local environment that lets us test complex application behavior and infrastructure logic without the full overhead of spinning up a local k8s cluster (which is what I imagine the next best alternative might be?). It's readable, declarative, and required no new tooling on our side since we were already using Docker and Terraform anyways.
Curious to hear how other folks are managing complex local dev setups, especially if you're not on Kubernetes!