Distributed system is slower than a laptop(codegood.co) |
Distributed system is slower than a laptop(codegood.co) |
Except the company probably approved a budget for AWS or another cloud provider, and basically gave a blank check to developers to deploy whatever is needed. So developers are going to just deploy MSK or whatever is trendy, instead of trying to get the most throughput from the servers they got from IT.
That happens at every level, from individual developers through to project leads and CTOs.
Consultancies are often no better. Choosing technologies that require substantial or highly specialised skill sets seems almost routine. I’m looking at you, Kubernetes.
I’m not entirely innocent here either. I owe a decent portion of my mortgage to MuleSoft consulting. That said, I don’t think I ever pretended it was always the best solution. Even while working directly for MuleSoft, my recommendation in probably half of the engagements was some variation of: ‘You’re using the wrong technology for this.’
But by then, an executive had usually tied their reputation to the project and the platform, commitments had been made, and changing course had become politically harder than continuing.
And so we persist.
In my experience, the best technology choices are boring ones. There’s still a large area of immature technology you can get creative with (like Backstage or Port for software catalogs and setting up a nice golden path”), but the meat and potatoes of development work should be a boring choice, that follows a well-tread path within a large ecosystem of developers.
There are exceptions, but they’re not for the majority of organisations.
P.D: the business world is the one that needs AND tries everything, and most attempt to implement something is much worse than the simpler previous thing.
For most that I know and most I bet (even without knowing) add "high availability" or "warm standby" WILL CERTAINLY lead to worse availability.
In fact, the best setup for most people, and consider things from today (that you will see, not that different from mainframe days!):
- Single server on a *nix LTS/immutable distro with just the RDBMS + App backend, on decent hardware. Fast, lots of ram or whatever optional
- A true reliable backup system
- A decent network setup, ideally a VPN one for connect everybody, with good firewall, SSL and that is
More complex than this and is 90% certain is much worse, other 8% until the dude that do it and care leaves.
McSherry does a lot of interesting work on making monotonic/incremental distributed systems efficient (e.g. Differential and Timely Dataflow). Those kinds of systems scale much more linearly.
(modern server hardware and operating systems are also surprisingly reliable nowadays, which makes it harder to reach breakeven with a distributed design)