Cloud-Based Cron Jobs

8 points by isnotaname 1 year ago | 3 comments

isnotaname 1 year ago |

Cron jobs don’t scale well when you’re on Kubernetes or in distributed systems — you end up with duplicates, failures, and a mess. We built Schedo to solve this and much more: a cloud scheduler that just works, without extra complexity. It’s designed to handle scale and reliability without you having to juggle locks, queues, or overkill third-party tools.

SamInTheShell 1 year ago | |

Are you saying cronjobs in Kubernetes doesn't scale well? Specifically this core resource here is what I'm wondering about: https://kubernetes.io/docs/concepts/workloads/controllers/cr...

I just don't see what doesn't "scale" exactly here. Been using k8s for 7 years, never had a problem that wasn't my own fault. Even for parallel processing, I feel like there's so many options and patterns to use because it depends on the task the job does.

lenvl 1 year ago | | |

I'd say here, that's may not be about the scalability itself. For me, it's more about convenience. I have been using k8s jobs for a while now.

for some reason, the company I work for wants to fix bad architectural decisions with k8s jobs :)

We had a bunch of issues with k8s jobs, but you are right none of them were k8s fault. But having a need to monitor them, and dump the logs to Loki or DD, made debugging quite complex.

If you need to restart the job devs would go to DevOps slack channel, for the job to be fired manually, etc.

IMO, k8s jobs are more for devops & infra team tasks, not for developers. The biggest upset is the need to spin up the containers for the job to be executed. We put out jobs inside services, so you could call script that has a db conn and it does the job, but we need to isolate rabbitmq connections to not accept messages while the service is in "cron-job" mode.

I know this is more our problem, than the issue with jobs, but still. I have never seen cron jobs implemented with no issues.