Ask HN: GitHub Actions as CI with self-hosted runners

2 points by cimaster 1 year ago | 2 comments

We’re considering switching from Jenkins to GitHub Actions as our primary CI system. We’re planning to run self-hosted runners on the same infrastructure we’ve been using with Jenkins.

So far, my experience with GitHub Actions has been positive—honestly, using anything other than Jenkins feels like an improvement. That said, I’ve run into a few challenges, particularly around the interface and some missing features compared to more traditional CI systems. For instance, there’s no clear way to view a queue of pending jobs across the repository, triggering a build for a historical commit isn’t straightforward (unless you set up dispatch parameters manually), and there’s no built-in test result tracking.

Managing self-hosted runners also requires setup and monitoring, whereas Jenkins provides a lot of this out of the box.

For context, we’re a fairly typical company with around 150 developers and a few dozen repositories.

Has anyone here been through a similar transition? Any advice or insights?

eberkund 1 year ago |

I assisted a past company with transition from Jenkins to GitHub Actions and some of the things you mentioned also came up for us although we did not self-host. It's been a while but I'll try to comment to the best of my recollection.

I think Jenkins makes it easier to support some less than ideal patterns because of how customizable it is.

Tracking the % of test failures across builds and normalizing failing tests rather than treating even one test failure as a critical issue that blocks the build. For test coverage we had to introduce a 3rd party tool like CodeCov or Coveralls.

I have rarely found it necessary to trigger CI workflow for a past commit. Even without dispatch parameters, you can rerun failed jobs from the web UI. If you are using workflows to deploy artifacts then I found it more straightforward to separate the testing and generation of artifacts from their deployment. So instead of building a testing, building and deploying a container all in one workflow you would instead store your artifacts in a central location and then specify the tag that you wish to deploy without rebuilding.

I also found that because we were self-hosting Jenkins and additional instances to run CI are relatively affordable there was very little emphasis put on performance and efficiency so over the years this had led up to almost 1 hour builds per commit which added up when multiplied across many developers and open PRs. I found that by splitting some repos and consolidating others we could bring this down substantially.

Also I am not sure if you're aware but there are some 3rd party managed GitHub Actions providers out there which make work as a compromise for you between hosting your own and using the runners provided by GitHub.

crohr 1 year ago | |

For me the main issue with managed GitHub runners is their low performance compared to the cost. In the context of developing my own tool for managing self-hosted runners, I maintain a benchmark here [1] that compares GitHub vs third party providers.

https://runs-on.com/benchmarks/github-actions-cpu-performanc...