Ask HN: Self-hosted application devs how do you handle deployment downtimes?

3 points by CloseChoice 2 years ago | 4 comments

Hi guys, I am currently developing an application on a small Hetzner server. I implemented a CI workflow using Github Actions but every time I deploy I have a small downtime (approx. 2mins) and therefore I wonder: How do you handle this? Do you care about the downtime? Do you have common (announced) downtimes in which you deploy?

stephenr 2 years ago |

For `$CLIENT` everything is redundant so there are two instances (on two VMs) running the app. During most deployments, one goes offline to update; when it's done it comes back online and the other goes offline.

For deployments that include a database migration, both go offline to maintain data integrity, but one stays online until the actual deployment is ready to run, to reduce the window where both are offline.

Due to the majority of deployments being the former type (ie zero downtime), and those that have deployments being optimised to reduce the downtime, we don't bother with "scheduled" maintenance like you're talking about.

If it's something especially complex, `$CLIENT` has sometimes announced the outage to customers ahead of time on social media, but this isn't very common.

sgt 2 years ago |

There are many solutions for this. Many are complex and will probably take time to implement right now, and you just want to focus on your app in the next days.

So if you want something super simple that will cause no downtime while switching over to a new version, just run two instances of your service, one on e.g port 9000 and the other on 9001.

At any time your proxy (nginx or traefik etc) will use only one of these.

Once you upgrade the version for the inactive one, you change the port used in nginx and reload it. Then the new version will be used.

fragmede 2 years ago |

Why would you go down for deploys? That's just wrong. One way around it is to have a load balancer that points at the old one until the new one is ready and able to serve requests, and only then does it switch over.

t312227 2 years ago |

hello,

idk what "deploying" means in your case.

but in general: try to keep the old deployment online as long as possible and switch to the new one after validating it ...

eg.

1. deploy to a new lets call it "location" - which could be a new directory, a new set of container-images, versioned packages for your linux distro (service managed by systemd) etc...

2. run & validate the newly deployed service-version

3. switch - by certain means, heavily depended on your setup

this should keep the downtime within fractions of a second - if this is feasible for your application.

but in the end a viable solution depends on your actual deployment-process :)

just my 0.02€