Spotify getting into enterprise IT tools(techcrunch.com) |
Spotify getting into enterprise IT tools(techcrunch.com) |
For a small startup, poking around in the gcp/aws console works well enough. Maintaining another console layer on top of that feels like one more thing that could break.
If you’re a big company with a team that could support developer platform tools, you probably have some sort of portal system hacked together and you’ll never be able to change because somebody has a critical dependency on the old system
It might not be a massive market, but there are a few interesting companies building this kind of tools and they sell it
We were early adopters of Backstage, back in 2020. We've migrated almost all of our portals and "meta-infrastructure" to Backstage, and it's certainly not perfect, and not everyone has moved over, but the majority of our teams are in one service catalogue and people can much more easily find service owners, dependencies and more.
It's been a great tool and a pretty big internal success story for us.
But this move from Spotify tells me that they are no longer stewards of an open source project, they now use open source as a loss-leader for their paid offerings.
Which is totally fine, but it will change how we look at that relationship.
> Libraries, websites, ML models — you name it, Backstage knows all about it
Spotify should spin off Backstage as a separate company, providing the initial IP and funding and being the first client.
Many companies, like Uber, created a lot of useful infra, though they failed to monetize it. They could take a cut in the Chronosphere (M3) and Temporal (Cadence).
Much better than laying off people, though origin companies would need to be smart, without being too greedy.
Though the model is unproven.
Amazon/AWS?
AWS worked, but the Fire phone turned out to be a write-off.
We're looking at it for an internal platform serving ~50 engineers today, spanning mobile, backend, enterprise integrations, and ML teams. The hope is it'll make it easier to spin up new things "the right way" without e.g. ML PhDs having to learn Helm and K8s, plus other goodies like nice docs.
The problem you end up having is that (for example) Google docs is really good for writing and commenting on docs, and a general purpose tool is never going to be as good as Google docs for docs. You could write a "Google Docs" plugin and just link out to a project's google doc but then you're just a link aggregator which offers limited usefulness.
Backstage is built on a modular, plug-in based architecture that allows engineers to layer-up their developer portal to meet their own needs.
https://backstage.io/docs/getting-started/configure-app-with...
Additionally, check out what it takes to stand up a standalone backstage server. You used to have to clone their repo if I remember right, but now they give you a starter template via npx and leave it up to you to put all of the pieces together. They don't even ship a docker image to get you started.
https://backstage.io/docs/getting-started/
Frankly, the architecture of Backstage is ill-conceived and a hassle to operate. If you have the budget their are vendors that will deal with all of this for you. I think the concept of Backstage is fine, but the execution leaves a lot to be desired.
A plugin architecture like Wordpress or Grafana is much more manageable for an operator. In fact, a set of plugins for Wordpress could probably get you an equivalent solution, but it's PHP, so you know, ew. I'd love to see an alternative to backstage that gets it right.
With plugins focused around projects/deployments/metrics.
Which sounds actually interesting.
- you have all your micro services catalogues in one place, with a fairly decent data model (services are grouped into systems, they are owned by teams, they have declared dependencies on each other)
- you have documentation for each service that you can easily find
- you have a template catalogue that allows you to easily create new services. You need a full-stack app? Here, recommend option is Java + React, use this template. You need a new ML system? Here’s what you should use, click “create” and we’ll bootstrap it for you
- plus a few more, like customizable home page
If you are looking for a way to start cataloguing and standardizing services across your org, the developer portal (whether Backstage or other solution) is a good thing to have
I'm sure you used stuff like confluence but it was always a terrible way to search across code bases and projects to understand how they worked.
Backstage was interactive documentation. I can see what projects we have, I can click on one, see who is maintaining it, what their communication is, what their various perf/benchmark scores were. It was nice.
Apparently it's even grown beyond documentation needs from what I can tell and it provides platform tooling now.
While I don't know your exact situation, I'm currently at company of your size engineers and stop torturing your Developers with Helm is off the shelf advice I would give. See if Kustomize would get the job done and be honest. I've found Devs can handle Kustomize a ton easier since kubectl kustomize > testoutput.yaml is alot easier for them to parse.
Helm is great if you have a bunch of different things to launch with a ton of different options but most companies don't do that. They need HPA/Deployment/Service/Ingress which is easily handled by Kustomize.
Any good resources you can point me to on how Kustomize might be easier for us?
Creating software templates can be a large investment. It would be great if there was a software template marketplace. The backstage contributors took a stab at this by creating a template repo that has not been updated in about a year. https://github.com/backstage/software-templates
If you do invest in creating a template, be sure to get plenty of stakeholder feedback so the thing you make is something that actually gets used.
We have a lot of in-house tooling and Backstage allows us to move big chunk of UIs into one place - it helps us keep UIs consistent, and makes it easier for everyone to find the tools we have. However with a lot of older, more mature in-house tooling it’s been a struggle to migrate to their data model, introduce their software templates etc.
For an org with ~50 engineers I’d recommend something more off-the-shelf like Cortex, Port, or OpsLevel
If we start looking at it again, I'd try a simpler use-case, but definitely interested in trying it again.
(Another thought: "short" and "long" terms are variable for different teams and different types of services. Scheduling should take that into account.)
Admittedly, however, I've not been through it at a particularly large organization. I'm sure there is exponentially metastasizing complexity (human perhaps even more than technical) once the process involves more people than can fit in a room.
However, biggest is it's just YAML patching. So you don't have this {{if such}} littered throughout your project that people can't easily pick up. Values is biggest reason I see people want Helm but if it's all env variables, kustomize lets you put in .env file it turns into config map and injects into deployments.
It's possible Kustomize doesn't fix your problem esp if Developers have decided it's not their problem. I just find Devs find Kustomize much easier.