The new pricing model for Travis CI(blog.travis-ci.com) |
The new pricing model for Travis CI(blog.travis-ci.com) |
I've always found "build minutes" to be a little bit of a vendor-favored pricing model. I really love wanting to do a release, and watching my CI provider take three minutes to pull down a 30MB docker image, or "npm install" running at dialup speeds. All while they're billing you per minute -- they make money by not investing in their infrastructure! I'd prefer to pay per byte transferred and CPU instruction executed -- if they make the hardware or network faster, the price stays the same, but they can do more work with their infrastructure. And if you schedule less work, the price for you goes down.
But, it's simply not done, and that's kind of sad because slow CI is probably the biggest engineering time killer in existence. Other than Hacker News ;)
The strategy is 100% to not improve the infrastructure, fire all the developers to save costs, and milk money from existing customers for as long as possible.
Travis bought out https://news.ycombinator.com/item?id=18978251
Staff laid off the next month https://news.ycombinator.com/item?id=19218036
I got our CI pipeline from 90 minutes to 10 minutes by running parts in parallel and applying lots more hardware.
The cost is essentially nothing ($0.10 / build) compared to the time developers spend waiting.
I currently use & recommend buildkite because they offer a preconfigured cloudflare template that does all of the hard parts for you.
We had Jenkins, but it required so much engineering time to manage, secure, upgrade, etc. Then when we hit scaling limits as our team grew, scaling it out to multiple machines took more time and cost a significant amount as we had to have capacity for peak time, which when all engineers are in one timezone is a significant peak compared to, say, the weekend.
We moved to CircleCI and while I have many frustrations with it, parallelism and ability to speed up the pipeline with minimal development overhead are not really frustrations I have. The cost is also minimal compared to the developer time, and while we're getting "less for our money", because we only pay for the active time, it's actually cheaper for us than Jenkins was just for hardware rental, let alone developer time managing Jenkins.
I can completely see how a different org with different constraints, different deployments, clouds, strategies, provisioning, distribution of engineers, etc, could come to the conclusion that you did – that BYO is better, but I think it does depend on so much.
[1]: https://yourbase.io
If you're at the size where slow CI negatively affects your projects, then you're big enough to own your own CI (at least the build agents).
Remember that one-man projects don't need CI, and that CI for small (n<5) teams is almost never the bottleneck. These SaaS CI providers really target the open-source / small-team market and it makes sense that they wouldn't optimize for larger-scale operations.
Even a single project with 20-minute build times is enough to slow down or frustrate development.
At the same time, I would not easily justify spending time managing CI infrastructure with my team of 6-10 people.
Things may have changed since then, but the last time I self-managed build agents, it often lead to build jobs being tightly coupled to the build agent and installed software versions. With a docker-based CI system, you are forced to have everything specified in code, making it much more maintainable.
Additionally, hosted CI allows me to do 100 parallel builds on Linux, MacOS and Windows. Perhaps this is a niche use case, but I saved a lot of time and reduced build times by an order of magnitude by switching from self-hosted CI on Windows and MacOS to a hosted solution.
Strongly disagree. When I joined my current company, as an early engineer on a fairly new product, we had an 8 minute deploy time and I see that as a fairly critical component to our rapid iteration cycle that was critical to the company at that stage.
We had 3 engineers and had no time to spend on owning our own CI.
The open source Python library I maintain has over 30 instances in the test matrix of Python version x platform x implementations. I don't even have a Windows dev box handy. TravisCI and Appveyor take care of that for me.
I think you vastly underestimate how much stuff people want to fit into CI and how quickly it turns into a big blob. I work as a freelancer helping not-quite-startups-anymore with things like CI speedup and tuning the database queries emitted by their ORM. You know, things where it's easy to build up technical debt.
It's not uncommon for a team of 3-4 to build so many tests and add so many linters and whatnot that CI takes more than an hour. Often, some basic love can bring it down to ~5 minutes but many teams are so focused on building new features that they will not take time to sharpen their tools.
Basically they significantly cut the FOSS usage by instead offering free credit and then reviewing the projects on case-by-case basis if you run out of it.
It's a big change, but honestly, I don't know how they could they provide free computation for any public repo no questions asked as they have been up until now.
Source: https://mailchi.mp/3d439eeb1098/travis-ciorg-is-moving-to-tr...
After travis-ci got bought and layed off a lot of key staff I did think the day would come where they will no longer will be the defacto choice for open source and I think this news confirms that day being today :-)
OSS projects are eligible for a free Gold plan on GitLab.com and can benefit from more CI minutes: https://about.gitlab.com/solutions/open-source/
If you are planning to continue using GitHub, you can also integrate with GitLab CI/CD for your OSS project: https://about.gitlab.com/solutions/github/
EDIT: Travis seems to take 10 credits per minute, with 1000 free credits per month. So it seems to be the least generous tool with 100 minutes per month vs 400 (GitLab CI) and 2000 (GitHub Actions).
I find this pay it forward model better than any other option.
It's 100% open source and there's almost no income (some github sponsors) for it. I guess welp we'll have to switch away, or have to constantly send "ask" tickets to get free credits :/
Many of us will be in this position.
(GitLab supports using its CI/CD on a non-GitLab repo just fine, but it can cause some initial confusion.)
Personally, I really like that Travis offers a variety of architectures; I’m currently running binutils on it: https://travis-ci.com/github/saagarjha/binutils-gdb. I suspect this might be a substantial portion of the CI it sees, and it’s been great that it’s been free so far. I am unsure if it will still stay up with these changes. But, I’m sure running these kinds of builds can’t be cheap at all.
Moving off Travis to something like GitHub Actions will cost a significant amount of time and the opportunity cost is ridiculous.
This along with the Docker Hub limits makes a very strong case for GitHub's strategy.
What if you defined your GitHub actions to call out to shell commands? Then there's very little that's specific to GH actions.
Actions are nice but personally I'd rather have my CI as generic as possible, especially for the reason you mentioned which is to be able to run things locally if needed.
The only thing I sometimes miss from Travis was the ability to run "debug" builds, but most of the time this was necessarily only because of how omnibus their VMs were and oddities of pathing.
[1] https://about.gitlab.com/devops-tools/github-vs-gitlab/#comp...
That’s why they are turning off free foss builds.
Since their acquisition they have just been going downhill. I've been planning to move all my projects to Github Actions for a while. This is the "drop that overlfows the glass" as we say in Spanish, I'm putting everything aside tomorrow to migrate all my public and private projects to Github Actions.
Based on MacStadium's public pricing, which is $150/month for the base model of the latest Mac Mini, this cannot be cheap for Travis, especially given they offered (past tense) Mac OS builds for free.
Not surprised to see them start charging more, but it's coming at exactly the wrong time given that GitHub (with Actions) is now offering a product that's a fair bit better, and it's still free.
On my repositories, setting up NPM now takes around 120 seconds instead of 20 seconds a month ago, time of npm run build has been increased from 7 seconds to 20 seconds...
The same build that took 2 minutes last month takes 4-5 minutes today
Now Apple looks like a legacy platform similar to Intel.
Then they moved the repository replication to the paid plan (20$/month/user).
For a simple 10 person team, that's 200$ a month. Exclusively to get access to repository replication !!
I have my own runners, so there's no reason to pay for the paid plan. I just end up hacking around some replication script.
See https://medium.com/@PedroGomes/mirror-repository-to-gitlab-f...
It looks like "pull" replication is available on the $4/mo. plan, per their documentation.
It's on qemu, not that fast but it works.
They also differentiate heavily on how they work. With GitLab, if you for example want to run jobs on ephemeral KVM virtual machines, you would have an agent on host machine (or multiple of them), which would then receive a job, spawn VM, execute commands in it, and at the end terminate it.
With GitHub, you would have to spawn VMs in advance, and launch their agent in each of them. When agent dies, you will have to manually kill the VMs, and spawn a new one.
This means that there is no easy way to have VMs spawn on demand, or to spawn different VMs with different configurations depending on job label.
> CI/CD is a part of both the open source GitLab Community Edition and the proprietary GitLab Enterprise Edition
https://about.gitlab.com/stages-devops-lifecycle/continuous-...
As I start to use more advanced features of CI/CD, I might get a better idea of how they compare. I am aware that GitHub Actions has a marketplace but I don't really want to set up my builds that way. I would rather write the steps myself, pulling in docker images and packages from package managers like npm, PyPI, and apt as needed.
The step from a docker-based build to a proper build agent is a small one. From there, running your CI yourself on a cloud provider is not particularly hard and at size will quickly be cheaper than having an intermediary.
Most people do not need CI until you have a separation of concerns (i.e. code from credential management) that are managed by different people/teams, and therefore all of the decisions cannot be made in a single room.
I do see being able to run test matrices across multiple OS or device options as a reason for smaller teams to adopt CI early.
https://docs.github.com/en/free-pro-team@latest/actions/host...
- Oftentimes the staging of the CI build can be improved. Devs often set up CI so that linters must pass _before_ actual tests are run. Run them in parallel instead and fail the whole run if the linters don't pass. This is even more important if there are multiple linters (perhaps for different sections of the codebase) and they all get applied serially before any of the tests start.
- Obviously, split up your tests as well so they can run in parallel. If you have a project containing both JS and backend tests, don't wait for one to start on the other. Many "bigger" languages also have something akin to parallel_tests (https://github.com/grosser/parallel_tests) that let you quickly set up multiple databases to separate transactions etc. It also provides tooling to remember the output of previous runs and uses that to equalize the parallel tracks of subsequent runs as much as possible.
- Cache as much as possible. This is a wider topic, but dependencies, docker layers and static assets can all be cached and correctly using this alone can hugely cut CI time. You don't want to know how many projects I've seen that don't have this set up correctly (or at all).
- Longer running projects can have hundreds of database migrations and applying them all to an empty database can take minutes. Big frameworks like Rails can dump the schema for you in a way that you can load in a second instead. Have a separate job that runs in parallel and applies all the migrations then verifies the output against the schema, all the other jobs load in the schema and use that.
> You have 1,000 credits left - these will begin counting down automatically as soon as you run your first build.
> You can use your credits to build on both private and open-source repositories using Linux, macOS, and Windows OS.
> 1,000 credits will be replenished automatically monthly. Additional Credits purchase is not available for Free Plan.
Even if the email is accurate, which is the better scenario, though, 1000 credits is 100 minutes of Linux build time (or 50 on Windows, 20 on Mac). Which is not much for any but the smallest projects.
For us, it was a bare metal machine where we had to email a sales rep to get a machine added, then spend ~2 hours setting up firewall stuff with semi-manual Ansible scripts. Add to that minimum contract terms and difficulty cleaning machines, and it was a pain to manage.
Conversely, if you've got a reliable autoscaling solution of some sort, and your build manager is capable of poking that as necessary to scale up and down (possible with Jenkins, but hard), then this could be really easy to do and BYO may be feasible.
Having a CI provider give us ~unlimited pay-as-you-go capacity that needed no management on our end and was always a clean environment, that was worth a lot to us in engineering time.
That'll do it. I'm using the Buildkite elastic stack, which took me about 20 minutes to start using and 4-5 hours to dial in to ideal settings (eg adding IAM to allow deploys from agents, getting the right size spot instances etc).
(no idea beyond "could", but it does at least seem plausible)