Ask HN: Resources for engineers new to system design I came across a twitter thread on resources for engineers new to system design* - there are some great recommendations there. What would (or do) you recommend that first-time system designers read? * https://twitter.com/HenryR/status/1094033131399565313?s=19 |
As an engineer new to system design, I found the whole book to be gold. It gave me the vocabulary to continue learning more on my own.
I'd consider its level of depth somewhere in the middle between specialist books and 10k foot overview books. I recommend it to anyone that has been a software developer or DBA for 5+ years, as I think they'd get the most value out of it.
https://github.com/donnemartin/system-design-primer
Also, you might want to check out 'Web scalability for startup engineers' by Artur Ejsmont.
This class covers topics on the engineering of computer software and hardware systems. Topics include techniques for controlling complexity; strong modularity using client-server design, operating systems; performance, networks; naming; security and privacy; fault-tolerant systems, atomicity and coordination of concurrent activities, and recovery; impact of computer systems on society.
[1] https://ocw.mit.edu/courses/electrical-engineering-and-compu...
[2] https://www.youtube.com/playlist?list=PL6535748F59DCA484
Reading high scalability and watching talks about how Netflix, Google, Spotify and friends deal with their massive loads can be entertaining, but for the vast vast majority of us it is only that: entertainment.
Google's solution is almost certainly not right for you. Netflix's solution is almost certainly not right for you.
I've never had the distinction to work on a project that large, but I imagine that even if you are this scale their solutions are still not useful, because at that scale and complexity everyone's needs are different.
If the ideas sound familiar, that's because they're basically just good software design. Except if you don't design things well from the start, you have to either kill it off or spend years fixing it.
I think a lot of this could be the exactly right mindset for many of us. Sure, there's something to be said for an MVP, but with some experience in designing composable abstractions, you can create an MVP that scales up to real workloads in half the time of your competitors, who basically have to rebuild their thing from scratch.
Butler Lampson "Hints for Computer System Design"
But for truly first-time system designers that haven't had much exposure to systems thinking before, you might find [1] useful. Donella Meadows[2] focused on environmental sciences and economics, so the specific examples/anecdotes mentioned may not be of interest to you. But she is also well known for her work in systems thinking, and the mental frameworks and general systems thinking principles sprinkled throughout are just as applicable to designing complex software systems as they were to her work. And make it significantly easier to evaluate the practical resources others have provided and interpret them in context to your own needs.
[1] http://donellameadows.org/archives/leverage-points-places-to...
Two decades ago, when I was learning to code, the career path for engineers was to eventually become a "software architect". The architect was to be the god among mortals who would "design" large systems and dictate how these things fit together.
Fast-forward a few years and it turned out the "architects" were the biggest waste of of time and money. The best system designs came from the low-level engineers who were actually building the individual components.
In my humble opinion, the best way to learn big system design is to just put in your 10,000 hours of coding. The principles necessary for multi-thread concurrency are not so different from multi-datacenter concurrency. I suspect there are 1,000s of subtle design patterns that one can perhaps never fully articulate.
Just my $0.02. Good luck!
Incorporate ecological and cybernetic principles in your designs.
Microservices are the prime example of thing you should never do at small scale. They're a solution to an organizational problem of organizing large teams. Typically each team has a single microservice... and then 5-person companies will go all microservices and support 20 services with one team, and then get confused why it's not going well.