Building for the 99% Developers(future.a16z.com) |
Building for the 99% Developers(future.a16z.com) |
Not to mention, this sort of company needs to do a careful evaluation if the chosen product and especially support for it is likely to be around for a while. Many got bitten hard by Angular's demise - and now imagine this for something like a database that's supposed to hold your entire corporate data.
I think this favors stacks and frameworks where the business logic of a domain is already reflected, at least partially.
This might explain why some open source frameworks persist and thrive even while their "tech" is deemed obsolete / deficient...
I mean it’s not wrong, we all live in that real world (my job is literally migrating off stuff) and recognize everything the author says.
It’s just that I don’t see those myths being myths in the first place.
In fact, thinking that the 1% is any different is perhaps the biggest myth in the article.
Sure they have distributed crap to handle and uptime commitments to go by, but I don’t expect the life of these people to be much different
Having new joiner stating on 2nd day that the whole system has to be moved to microservices on graphql with blue/green continuous deployment to solve all past and future problems is a real problem, it does exist.
Some people seem to fall into this trap described in the article of reading some blog post on some approach and fixating on it in religion-like mindset. It makes it hard to unwind ideas they throw into business people's minds because they all sound superficially great. You have to break it down to first principles and go through it, explain why things make sense for 10k+ engineering teams but don't for our dozen people team.
They will call current architecture legacy from the start and that sticks with business people. The reality is often the other way around - big players would love to be able to run systems using those simple, straightforward arrangements: single version for all services, ability to have few minute downtime to upgrade system offline, single monorepo, single database – but they simply can't because they have thousands of people working on it, running at massive scales across the globe.
When you say those things people get very defensive and it's hard to keep dialog on. Because those new things could actually work very well, but ie. not as replacement for everything but to create some satellite services, maybe for things that crystalized over years, unlikely to change and can be extracted as dedicated service, maybe graphql makes sense for admin section where the f/e team wants to experiment more etc.
I'd say:
- be rational
- analyze from first principles
- use N-order thinking
- be open minded - for new tech and ~legacy~ current tech
- judge on simplicity as one of main criteria
- stop calling tech you're currently running without already available alternative "legacy" - it's your "current" tech
- monoliths are not "legacy", they are desired if possible, split if they'd simply not work or you organically grew into maturity level where it start to make sense
- same with single database
- same with single versioning
- same with single monorepo
- use your own business reality as base for evaluation, not somebody else's reality
what does this mean?
Stackoverflow maybe sometimes.
When I became a Product Manager, I started voicing opinions on all these forums.
This one hits really hard and I'd love to see more of that content.
I’ve been shipping (as opposed to coding), for my entire adult life, and I’ve learned (the hard way, of course) that “ship” is always at least a couple of clicks back from “bleeding edge.”
I’m in the home stretch of an app that I’ve been developing for the last year and a half, or so.
The backend is written in PHP, and the frontend is a “classic” UIKit/Storyboard/MVC app, as opposed to a SwiftUI/Combine/MVVM project.
It’s fairly ambitious, and, when I started, I was not confident that “the bleeding edge” would work (it might have, but I didn’t know that it would). There was no question that the classic patterns would work, so I picked them.
It’s coming along great; far better than I had originally envisioned the project. It is ultra-high-quality, fully native, with only one small external (meaning that I didn't write it, myself) dependency (the backend has zero dependencies), has many capabilities that have only appeared in the last couple of OS releases, is easily localized, conforms to multiple device configurations, dark mode, accessibility features, etc.
I read (here), about a well-known app that had been a highly successful classic native app (probably written in ObjC) that was supposed to be rewritten in SwiftUI, but the project failed, and was eventually shipped in Electron. That’s kind of my worst nightmare. I suspect the developers had to put clothespins on their noses, for much of the project.
I will be releasing software, using more cutting edge tech, but all in good time. It’s unlikely to be “cutting edge,” by the time I use it. I like to give the stack enough time to smooth off the rough edges, before I rely on it for ship.
I know that many fairly well-known applications for Apple platforms, are still written in Objective-C. Even Apple still uses it, for some of its internal tooling.
I will admit that one gamble I took, was jumping on the Swift bandwagon, almost immediately, but there were multiple signals that it was not another OpenDoc[1], and that I could trust it. I’m fairly conservative about bandwagons. It’s earned me more than a few sneers (especially since I’m an older chap), but -and this bears repeating-, I’ve spent my entire adult life, delivering finished software. It’s not always been great software, and it has not always been commercially successful, but it has all been “finished.”
There was a post here, some time back, where the author challenged the reader to mention three projects in their career that they had finished. My comment was that I could mention thirty, and point to the repos. This was met with incredulity, which shocked me, as I have known many developers, far more productive than I. I guess times have changed.
[0] https://vickiboykis.com/2019/05/10/it-runs-on-java-8/
[1] https://en.m.wikipedia.org/wiki/OpenDoc (Full disclosure. I was very much a “bandwagon” guy, back then, and even took an OpenDoc course, from Apple’s DU).
Just use Rails.
look just because you're not Lebron James or Michael Jordan doesn't mean you can't go to your local basketball court and enjoy a game of pick up.
But the same way you don't conflate what you need to do to be one the of the best people to ever play a game to play at all...
your software engineering team doesn't need to feel completely useless because they aren't immediately using <insert new hot tool> .
Everything in software engineering from side project to unicorn startup service is about tradeoffs.
Tradeoffs happen because of constraints.
I don't expect a consulting shop to build code the same way an open source project does or a publicly traded company that has been profitable for 10 or 20 years.
I'm hoping you can read in this the need to be cognizant about manpower tradeoffs.
So yes I fully appreciate this as someone who has consulted, worked a public companies, and is now at a rapidly growing startup...
Just because your team isn't meeting some imaginary ideal for some other form of team with different constraints doesn't mean you can't ever strive for any ideal or give up...
you just have to be realistic about trade offs...
my favorite example of this is thinking about a system like rails vs early react (loooong next.js or even before the context system) or a lot of things built in the python web ecosystem.
With rails you have scaffolds and endless convenience methods that let you be productive quickly even as a small team or shop...
with early react or python web you were largely left to your own devices out side of the core flows they had solved (painting updates to the dom with react or some rudimentary crud stuff with django or flask) ...
Because they were built by and more importantly FOR different types of teams.
Rails by the relatively slender (and famously against VC type scaling) 37 signals consulting crew and react by facebooks behemoth engineering team.
Of course they are going to build for their own trade offs...
why would you not?
(I intially opened this rant with this final piece thata I thought better of starting with but I still stand on the feeling...
The is the biggest piece of curmudgeonly nonsense I've ever read... its like someone training an ml model on last 5 years of HN comments and churning out a click bait blog post)
I'd never bother suggesting it though:
- no one likes being told they're small when they think they're big
- it's bad for devs careers (Resume-Driven Development)
- as TFA points out - it's not what FAANG are doing so I'd lack legitimacy
- Avoid if specifically mentions big companies as users: Facebook, Twitter, Google, these have as many devs as needed to throw at the most trivial tasks, and then some. I am in a team of 3 that need to get shit done, not chase package manager and compiler throw-ups. Tell me your language is used by a one person show to serve millions of people and then I'll listen.
- Avoid if the developer base is mostly young people: They are energetic and they don't mind complicating the shit out of something, because they can figure it out now and probably down the road. I'm old, and I just want things to work and I want to understand how it all works without spending a week and wrestling with tooling. Not all out-of-college folks are the same but most are (and I used to be one, so there).
- Avoid kitchen sinks: borrowing ideas sparingly is fine - throwing in every new feature in another language you come across makes for a soup, not a tool. A good heuristics is the rate at which features are added. Look for logarithmic trend.
I've seen a lot of suspect technologies in my career but GraphQL is definitely the worst
I've been using GraphQL for a little while now and I've had nothing but good experiences (although my use case might not be the most common) so I'm interested in knowing what makes you think so poorly of it.
Most developer's resumes would be vastly enhanced by delivering a lean, technically successful product to end users, or 5.
A lot of developers think playing with cool technology makes them better developers for some reason.
Which isn't true.
What makes them better developers is knowing what that technology does, and choosing the appropriate solution.
And building consensus, managing personalities and driving execution. By being a reliable, high quality human being.
Which is very different to knowing how to operate the API for that solution.
It worked poorly, took months to build, and lost data. I implemented a python script that read from twitter firehose and wrote directly to redshift. It was about 50 lines long and did the same thing, and I deployed it to two existing prod servers that were not network bound.
There was a few more cases like the on-shore database identity reconciliation being inefficient, so the company wanted to leverage cloud auto scaling. Well, if you have an exponential time algorithm and you put it into the cloud, it will still be exponential time and cloud auto-scaling will be extremely expensive, and also introduce latency between distributed system components, and indeed it was. The offshore teams initial logic was only that, cloud is auto scaling so can accommodate any performance the business logic needed.
Sometimes people just view tech/cloud as lego blocks to glue together, based on the product sheet marketing, and do not have enough basic understanding of computer science to understand performance, or understand what a simple solution would look like
“Grinding LeetCode” (tm r/cscareerquestions) doesn’t make you a better developer. But it does get you in the top paying companies. Well I didn’t. But that’s a different story.
On the other side of the fence in corp dev where pattern matching is the norm, employers aren’t going to want to hear that you used Sqllite.
Try getting a job as a dev with vanilla JS and SQLite skills, as opposed to React and Microservice skills. What a business needs on a technical level doesn't matter when it comes to finding jobs.
I've come to appreciate that the bleeding-edge really means you'll bleed over it. In the search for "magic" to make my job easier, it often falls apart of the edge cases. So to just get stuff done, I'll trend towards lower-level software even if it means more boilerplate.
Also, there's a great idea that seems very worthwhile. I don't recall the source or exact phrasing: you only have 1 innovation point for a project. Everything in your stack should be familiar to you (and you know the pros, cons, and issues) but you're allowed ONE new magic/helper/tool. That limits the blast radius of all this new-fangled stuff and gives you room to try out new tools.
This is exactly what 90%+ of people working on frameworks/languages don't get. Somewhere in their education/career they learnt that repetition is bad. DRY everything. And then they follow that religiously. Pragmalism (minimal pragmatism) is sorely missing.
- Does it work?
- Can it be explained to coworkers/usable?
- Does it solve the problems I'm having?
- Does it seem like it's going to be around in the next 5 years (weaker version: do I need to worry about it being around in the next 5 years?)
- Can I debug issues with it relatively easily?
Turns out that good rules of thumbs are ones that look at stuff on the merits, instead of trying to find catchy third-order effects that you see in projects that rub you the wrong way.
These might be ok heuristics, but it's pretty limiting to not use some tech just because a big company uses it.
(Edit: For the record I think msh is the correct one, it’s GP’s reasoning that is bananas)
An example could be - you're arguing with somebody in another team at similar level. If you convince them - you win, as a first order consequence, you may think: great. But if you dig deeper you may realize that doing things your way may not be so great for you in the long run; ie. if you have to deal with passive-aggressive attitude or don't have 100% support of that team. So selfishly, 2nd order thinking - maybe you should let go and you'll win long run.
I would've sworn it was postgres. Just shows how much attention I pay to anything that isn't business logic
"Sweaty Startups"
- Lawn work
- Accounting/Bookkeeping
- Power washing
- House cleaning
And a single content site monetized by affiliate/ads
When I want to create a new site I ssh in and type:
sudo ee site create yourdomain.com --wp --letsencrypt
And I have a deployed wordpress site and ssl certificate.
Does React specifically mention big companies, have a young developer base and is a "kitchen sink"? I have no idea.
Is Python a "bad" language? What about Go? C++ kind of seems like a kitchen sink, is it "bad"?
I really have no idea how to figure out if these rules apply to any languages/frameworks.
It's next to impossible to hold an engineering team together to even KTLO if you don't have some project which grow's their careers.
So one might not get in due to lacking .NET skills, but the other enterprise architecture and soft skills will be just as relevant.
Then it is up to the person to sell themselves.
Source, I have been jumping across .NET, Java, C++, native and Web for almost 30 years now.
Couldn't you... just fib? And learn it in a weekend?
It is relatively bad compared to the alternatives (plain good old http apis)
GraphQL has allowed my company to keep our network traffic very lean while evolving a product very quickly and to have very few issues in communicating between our mobile application and backend teams. But once again, I can see how our requirements are not everyone else's.
Hand-building a reactive (lowercase r) table with sorting, filtering, etc. that's performant at large data volumes to keep your stack "simple", would be a nightmare to maintain and wouldn't allow other developers to bring prior experience with them when they later have to maintain that code.
This environment would necessitate "reinventing the wheel but so much worse" for anything other than the absolute most trivial applications.
Until I saw a something by Steve Ballmer (I think), explaining that their strategy was to provide tools for those "99% developers". The ones who don't read HN, the ones who don't code for fun. Everyone for whom coding is something that gets in the way of their real objective.
No wonder they produced bad code! But that's not the point: they're solving something else. They're not spending effort making bug-free maintainable code, they've got more important things to do. They're doing the accounting for their small company, point of sale for the shops in their city, all the kind of things that may not be glamorous but that's all around us (for better or for worse)
So while I puttered around in my self-congratulatory ivory tower of unix, Microsoft understood what the real world needed, for the 99% of developers...
> Here's my point: yes, the whole Internet runs on Unix philosophy. But businesses sure don't. The big problem comes up in my description of pipelines up above: they only parse about 99% correctly, which is fine for your idiotic comments about YouTube videos, but pretty nasty when you mangle critical business data. And when the business dudes get involved, they'd rather do anything than mangle their critical business data. You hear me? Anything!
> If it keeps the data from getting mangled, they'd happily sacrifice searchability. Or developer hours. Or the ability to use off-the-shelf software. Or millions of dollars in licensing fees (because at least they'll accurately know how many millions).
> Pay attention, because that attitude is exactly why Windows is so strong, why the vast majority of developers prefer to develop on Windows, and why the vast majority of users prefer to use Windows. It may be gross, and adding new features or libraries may be a lot like stabbing yourself repeatedly with a fork (which, tragically, is not itself included in Win32), but Windows works consistently. So does SOAP. But not Unix or microformats or REST.
> It would be awesome if someone could find a way to satisfy both camps (the Internet people and the Enterprise people) at once. Then maybe one or the other set of technologies could finally die. But I'm not counting on it. Until then, one or the other technology is effectively dead, but which one it is depends who you are.
From: Novel, yes; widespread, no (2006), https://apenwarr.ca/log/20061201
I have no idea that windows works consistently (?!) where unix (linux) or REST (?) doesn't.
I like the idea of what they're trying to get to, but sometimes it helps to cite examples.
imo, LARGELY the reason that "windows=business" is from historical lock-in effects. I, as a linux developer, still get ".xsl" files on the regular, who have each gone through 4-5 different business people before me.
windows is largely a "suite" of software, from OS, to Office, to even Internet Explorer. Business types often didn't (and perhaps still don't?) know what separates Word from calc.exe - it's the entire computer interface, it works, and it's what others are using. Only a big "everything-in-one-package" could hope to upset them, and even then might have a hard time doing it.
I don't think I'll ever try to purposely run much windows software again, but I do give them kudos for what they've achieved and continue (for now) to achieve.
but to say it works consistently!!?
Was that ever true?
...right?
Windows isn't popular because its good at anything its popular because windows it meets the bare minimum and has a huge marketing budget unix doesn't.
Also, the idea that enterprise Windows business programmers were more into data integrity than the UNIX "Internet" programmers runs contrary to my experience seeing those two cultures encounter each other in the 2000s. In my experience, the former took human intervention for granted to handle unexpected or even routine-but-unusual cases, and the latter understood that their job was to build systems that could operate properly without humans constantly picking data up off the floor and dusting it off. The Windows programmers I worked with were great at creating rough-and-ready GUIs to help human employees do their jobs, honestly truly great at the "make users awesome" aspect of programming, but writing software that worked well enough to run without a human operator was just something they had never had to do. They couldn't understand why you were worried about some weird case that would mangle 0.1% of records, because in their mind that was the kind of thing that an expert user fixes up in Excel every Friday before the reports run over the weekend. A customer sends a payment after we've written off their debt? Never heard of that happening, but if it does, accounts receivable can escalate to Liz and she'll figure it out. February 29th? I think they know better than to run imports on a day like that, but to be safe, we'll remind them.
Those mushy "Internet" programmers inherited a legacy that was rooted in a requirement to be robust against nuclear war[1]. The mind boggling difference in ambition between surviving armageddon and "they know better than to run the software on February 29th, lol" made for a huge difference in culture and made me look up to the "Internet" programmers as intellectual role models while respecting the business-oriented Windows programmers as scrappy, context-sensitive, tactically oriented commandos.
[1] I'm not sure to what extent this was real, but the belief was real.
Take the POS example. You're helping real people be more efficent, and you've litterally got better things to do than be a software purist ( no offense, have same problem ). People are relying on you.
Meanwhile, serving the hospitality environment where multiple devices might be ran around a restaurant, and end up down a beer garden, all the whilst being expected to have every order on every device, because a table may call a wandering staffer, whilst having spotty AF network connectivity, processing trillion of transactions a year.
Suddenly you have a real life use case where your DS&A knowledge is useful, a real reason to be efficent on the wire, the device, and the server, and improvements in architecture can remove bad states, bugs, and improve the lives and fortunes of real people.
If being a 1% dev means profiling people and serving them ads, or writing pure code that nobody uses, then I'm quite happy in the 99%.
Disclaimer: I'm a CTO of one of those POS startups. :)
I didn't read the article the same way as many here did - my take was that he was talking about the 99% of developers who don't need the newest/leading-edge technologies because they are not solving for those problems (massive scaling for example), because the business they are working with/for simply will never need to worrying about that.
I didn't take it to mean that some of those same developers solving those '99%' problems, are not individually top 1% developers - you can be a top 1% developer imo, and still used tried-and-tested technology that is consider 'old' or boring.
Using the right tools, does not make one a top 1% developer; a top 1% developer will shine no matter what tools that need to use; they just use them better.
Just look at the default contents of .NET6 vs everything else out there and it doesn't even seem fair anymore.
We build a product for financial institutions that serves interfaces to multiple classes of devices and integrates with 15+ different 3rd party systems. It has to address concerns of multiple lines of business and regulatory regimes simultaneously.
Guess how many non-Microsoft dependencies we need to get this job done? I'll give you a hint- it's less than 5.
Any sort of ideological hang-ups are dashed to the rocks in my mind when I see how amazed the customer is with the final result. All of that principled idealism melts away after receiving their sentiment.
They shaped the world to their liking and now it looks like a match. duh.
While tech is (allegedly) evolving as fast as ever, the problems to be solved often do not. Sure, scale is always an issue and you want to reduce maintenance. But how many project do you have that have to scale to millions of users?
> Visual Basic, Access
Very true, but those tools have evolved as well. Sure, everyone laughs about people not using SAP or Salesforce and instead use a self-made CRM. I have seen terrible things here, but also good ones with more features than both other tools could provide. And it made working far more efficient. Of course Access is only used as a front end today, but the fact that there are no real alternatives is a certificate of failure of those that criticise these tools the most (I did so as well). And it is surprisingly adaptable to new feature requests.
Well, fine. They feel that way, but it isn't really true. Their going to be the ones maintaining that software for a while and even a basic Google like "Visual Basic best practices" would save them mountains of time in the long run.
And Visual FoxPro. Perhaps the most underrated tools for SME. And possibly one of the reason why CRM and ERP have a hard time breaking into those market.
And this is going to be rant-ish..
Then somewhere along the line came Google. Yes I absolutely blame them ( In terms of mentality ). Instead of Microsoft which at the time was the ultimate evil company, providing tools ( not even programming or code, but "tools" ) to 99% of developers. We got Google, representing "real nerds" on the opposite side of spectrum telling you these technology are utter piece of crap and ours are the greatest. The proper way to do it. Embrace complexity to its maximum. And of course a lot of people were sold. Nerd could finally get rid of all the ugliness from computing. Visual Basic? Ugh. Learn a proper programming language. PHP? ROFL.
And of course there is Resume Driven Development. The actual technology in itself doesn't matter. What matter is if MAMAAN are using it. So you have a better chance or joining them.
There is no fun in computing anymore, not for the 99% developers, and even more so for those who only wants to be 99% developers.
Too smoothly. We got 20% layoffs for doing a good job! Cost savings.
Unless you're happy that all those 99% developers have created towers of COBOL that run the world and they can't touch or upgrade.
The most laidback, humorous and mature developers I've met were Cobol developers. (probably older people are like that haha)
I think there is really not much to add to this, I just wanted to say thank you for this, a lot of people need to read this and wake-up call, that markets hammer on your. The true feedback loop has no feelings, no compliments - just pure and direct feedback by either using the product, or not, irregardless whether its Kubernetes, Solana or <script>
Most companies, not just FAANGs, use Linux extensively and this is not out of masochism.
There can be a niche for the product creation stage. But how does a company scale in MS's market? Sooner or later, it's a Slack vs MS Teams situation.
Like MS did use their existing sales channels to stop slack growing, but if slack at least had feature parity with Teams then it would have been a harder sell.
Maybe a majority don’t, and my experience is definitely biased. But i’ve met many programmers of diverse backgrounds, and a lot are very passionate and research the newest and best technologies on their own. They probably don’t read Hacker News, but I think “hacker” programmers are more common than you realize.
Some (Excel) were wildly successful while also introducing anti-patterns. Some (Access) were just anti-patterns.
Not 99% of developers, but 99.99% of problems that need to be solved.
I think it's happening because successful projects focus on the product delivering on the reason it exists(i.e. facilitate file transfer in the most intuitive way possible) and to do that quickly as possible for the lowest cost possible engineers hack together solutions derived from the existing tools without focusing on the technical or ideological perfection. Also, they cannot focus on the technical stuff anyway because to needs and direction change all the time, therefore it's not possible to crystallise an optimised and well built solution anyway.
What Microsoft or other companies do is to built versatile tools that that engineers can bend and stick together to accomplish the tasks in hand very quickly and if it looks like the thing is here to stay they can design and implement an optimised and elegant solution later on. At first it may look ugly but it is usually an original work done by domain experts who are exploring technical solutions to the problem in hand. Once the solution is found, experts in computer programming and architecture can step in and make it elegant but that last step is not needed for the wast number of solutions.
An example for this is UK's covid case tracking early on. Apparently they quickly implemented a central Excel spreadsheet that would collect the CSV data sent from the test centres. Unfortunately, the solution they implemented was way too hacky and they lost data once they reached the limits of the spreadsheet format they have chosen. Had they chosen a better format, their solution would have worked up until much higher scale and once they have a better understanding of the nature of data collection etc. then they could have implemented a clean sheet solution with "perfect" code and scalability, maybe years later for the future pandemics and write blog posts about the enormous challenges and their ingenious solutions. They couldn't have start by firstly building the perfect data collection solution because they wouldn't know how things will pan out and if they tried to force their way(i.e. mandatory formats by the tests centres) it would have been too big of a projects.
Your choice of the UK's Covid tracking spreadsheet debacle to illustrate your point is curious, to put it politely.
As The Register [1] (and many others) pointed out at the time, it was the wrong tech choice, executed slowly and at huge cost, by the wrong people with little knowledge of better choices, and directed by management and government leaders with track records of lying and incompetence.
> They couldn't have start by firstly building the perfect data collection solution because they wouldn't know how things will pan out and if they tried to force their way(i.e. mandatory formats by the tests centres) it would have been too big of a projects.
I have a different view on every part of this sentence!
[1] https://www.theregister.com/2020/10/05/excel_england_coronav...
The things big tech companies do are usually to avoid scaling problems. In particular scaling on the engineering side. The above can scale to billions on the user side if needed but 10k engineers constantly updating and messing with a traditional SQL schema doesn't work.
You'd be crazy to try to replicate something like the big tech companies database storage systems when you're just starting out. You don't have 10k engineers. You may think "but i want to scale to a billion users". Well you still don't have those problems. The scaling problems big tech companies have are scaling to 10k engineers.
There's many apps that need say, a 100GB database in 2022. Those apps also needed 100GB database in 2007, when horizontal scaling was the hottest thing around.
Nowadays however, Moore's law has steadily overtaken an order of magnitude of use cases from 2007.
Maybe 1/10th of the 2007 use cases still need that kind of enormous, big tech scaling.
BINGO! This really resonated with me. It's a good reminder that developer social media is going to have some of the same things going on as regular social media, like unrealistic comparisons to idealised snapshots.
This. We are constantly presented with new and shiny things. And then, barely a year later, there is the next ideal technology. And the next, and the next, and the next.
We have seen this with everyhing: Languages, coding standards, paradigms, editors, organisational methodology, interview methodology, frameworks, databases, security, design basics,...the list is endless.
But here is some truth:
C is still written. As is pure JS. Imperative/Prodecural code never went away. Websites on servers we can physically touch are still deployed. People still write working implementations for almost everything under the sun from scratch. Information-Dense, slim, efficient interfaces that get jobs done are still there. As are monolithic architectures. Good 'ol RDBMS still work great. bash is still the dominant shell. grep/cut/awk are still great for analysing logs. Water still flows downhill.
"new" and "better" are 2 different words, with different meanings. Something that's new could be better. But he fact that it's new, in itself, doesn't automagically make it better. And even if its better in one scenario, its doesn't have to be better in all scenarios.
> a lot of what most “developer influencers” say is fairly aspirational. Their own companies don’t necessarily do things as smoothly as they preach to others.
There's a pride element there: developers don't want to admit that they work in less-than-stellar conditions.
Some developer talks also double as recruiting events, and you're going to turn people off when you mention that the test suite takes a day to run and requires the use of a fax machine.
Suddenly, you see Kubernetes clusters of 2 to 4 vCPUs VMs/Nodes and containers with limits to 1,000 milicore (or 1 vCPU), and then the team solves the performance problem with dozens, sometimes hundreds of replicas for one particular microservice. Many developers don't even understand the impact the JVM has when running on a single core (yeap... you get Stop the World GC - aka Serial GC - by default).
And then, the dev team decides to move to a new language because of performance and cost issues. And that by itself just brings many other problems.
All they had to do was keep the same amount of CPU and memory, but less VMs and replicas, with more vertical resources. Depending on the workloads and the system, it is even possible to reduce the cost.
All this is caused by the push in the industry for companies to go Cloud Native.
Whether this distinction is relevant to you depends on where you sit. If you are a startup selling developer tools, by all means think about the 99% developers, but also know that a lot of them are in environments that don't spend a lot on tools and are averse to exploring new technologies. If you are a developer in one of these environments, well, the standard advice on Hacker News is already "You're not Google."
I feel like some of this is a bit fatalistic. The 99% can follow a DevOps playbook if they realize its value, and it's cheaper to have DevOps than to not have DevOps, in the anything-more-than-short timeframe. The 99% can certainly have test coverage standards for new code. Somehow, we moved from a world where the 99% didn't use source control, and now they do! Some technologies and practices are so impactful, you should aspire to them no matter what you're environment is (e.g. code review, CI/CD).
The 99% concept also hides a lot of important details, as it's defined by exclusion. The choices you need to make for an early stage startup, a mature WordPress shop for small businesses, and a legacy mainframe team in a F50 are as different from each other as they are from FAANG.
SQLite can be used for 95% of real world use cases.
And whenever WAL2 + BEGIN CONCURRENT get merged into main, it will be able to handle 99% use cases.
And don’t fool yourself into believing you’re not in that 1%.
While the insights on developer influencers was sharp, the article itself felt like more of a reaction than an evaluation.
I’ve worked in many legacy companies. The rationale for staying on the tech stack they have, and the approach they take to DevOps, is not particularly well reasoned. Often they are experiencing painful consequences due to their adherence to old design patterns.
I wanted each paragraph to be more diagnostic, frankly more reminiscent of the wide ranging and interesting debate we find here (especially when the old guard shows up and speaks with real authority and wisdom on how the problems they solve don’t map to Kubernetes, etc.).
I’d really welcome counter-arguments to the point I’m making, so I’ll frame it this way:
This felt like the same kind of “playing to the crowd” that dev influencers do, just a different crowd.
My approach - I could not care less about what FAANG does. Due to their scale and org structure they are solving problems which 99% of mere mortal businesses will never face. I am not a luddite and am constantly looking for new things that can make my development easier. But I consider those from ROI point of view as I am vendor with clients and I want to make money, not waste it. Coolness factor, fashion, corporate propaganda / indoctrination mean zilch to me.
React makes sense for FB's bloated, complex web app. Your CRUD app will do fine with far simpler dev tools. Even vanilla JS does the job in most cases.
99% developer feels compelled to learn and adopt what is trendy because that is what the client/market/boss demands; but it is you the 'engineer' who has to tell the client/boss how that bridge is to be built. When was the last time you went to a doctor and said: hey doc I have this, give me that medicine.
Allow me to introduce to you: pharma advertising in the US
And that would be what? Exactly?
I did a personal project last year that was vanilla HTML/CSS/Js, with some templating in Go (I know it's templating isn't the best, comparatively). By the end, I wish I would have done it in React. I ended up re-inventing the wheel for a lot of things, and ended up with something that was decidedly messier. All while attempting to be more simplistic.
I wish Web Components would have taken off. When Polymer first launched, I was totally on board. I loved the idea of writing vanilla frontends, and using Web Components where I needed extended functionality (vs a full framework). But, it didn't, and unless your site is super simple, and mostly static, React et al. are the best ways to create frontends.
Instead Go was selected, templating HTML by hand, and assumedly doing nearly everything else by hand well. That is not the optimal solution for a simple CRUD app’s implementation, and isn’t reflective of anything but poor engineering.
My physician friend fires patients for doing this kind of stuff. Usually they demand unnecessary x-rays and CT scans.
A huge portion of the industry will be 10, 20, even 30 years behind and that's fine. I don't think there's any myths about this.
Serverless technologies help us build scalable, highly-available, and performant cloud applications that run by default in multiple availability zones and multi-region with minimal additional effort. We leverage GitHub Actions to drive our CI/CD process and automated testing. Instead of focusing on managing servers and cloud infrastructure, us simple 99% developers, those who get passed up by FAANG and Leetcode grinder interviews, can build solid applications that provide value to our customers and generate nice profit margins. The 99% can deliver apps that can perform similar to the 1% with far less complexity.
I also strongly believe that severless is an overloaded term and everyone understands something different from it. Some people consider this only as a FaaS but I think that it's any cloud service, that can auto-scale to infinity, scales to zero, pay-per-use and managed. I frankly believe Serverless should be regarded as a paradigm rather than a technology advancement. here's a very nice blog about it: https://ben11kehoe.medium.com/serverless-is-a-state-of-mind-...
pi's article on this seems relevant here:
I have seen a lot of people try this and fail because they aren’t building a personalized kit, they are putting together popular dependencies. That is radically different.
When you build your own kit it is tiny and extremely portable. You dig into the nuts and bolts to see what works and what plays well together. Since it’s all your personal preferences you maximize on productivity every time you use it. The best part is that you only have to build it once. Then after it’s a few hours of maintenance every six months as your requirements and preferences evolve.
99% of all developers had enough todo with their work, they don't have time to write shiny articles about the used technology.
This should make it clear that many great articles on many great technologies are written by a few, most of which are at an academic level and have less to do with the practice of 99%. But everybody thinks that using this tools that way is normal, it isn't.
Generally observed that the less you see engineering as an ideal state or set of standards, and rather as a living and breathing organism in your company, the easier your life will be. If you're ever working on a 100% refactor of something to a new framework or system, chances are you're not focused enough on the top line problems at your company. Which is related to https://rkg.blog/desperation-induced-focus.php
I was too passionate about coding to butcher it in this way so I left the professional sector and focused on open source (thankfully I have enough side income to do this). It's mental torture to come to work every day, working in an industry which is supposedly all about logic and problem solving and be constrained by what is essentially religious dogma when applied in context.
10 years ago, it was such fun to be in the software industry as a developer. The company directors would trust you to choose any stack/framework you wanted or even build your own lightweight in-house framework (or no framework at all). It's no longer the case.
I’ve written far more fun code in a testing role than I ever could in a dev role :)
Certainly not for me personally, nor for the 10 or so companies I've worked for as a freelancer over the last 10 years. Among them are some - real - market leaders as well as small and fine companies that are extremely demanding when it comes to self-used technology (including development tools, of course).
Just look at who the owners of github (most people know it) and especially gitlab (most people don't) are.
More generally, just the single arbitrary picked difference between one company producing something tangible and another producing a pure software product should be eye-opening to anyone who has had both types of experiences. As for my career path, I've worked in instrumentation, avionics, cars, and Embedded, among others. Each with a very different culture, even among themselves, not to mention compared to software companies.
On the other hand, you're gonna want to rebuild that ASAP in something that has legs for longevity's and paying down tech debt's sake. You won't potentially have an edge over any competitors by using their same tech stack.
Most people need to get some data from a database to a web page and back again, to paraphrase dhh (I think).
This does not require a huge amount of architecture or infrastructure in most cases, even at scale. The engineering challenges should be elsewhere, not in this relatively simple and solved problem.
Thing is, I see all the times web frameworks that would be used in those cases, but improve in many directions almost irrelevant for it. Performance? Ability to do same webapps in yet another boring scripting language? Very specific and rigid abstractions and fancy tools that provide for very little flexibility? Fancy reactive approaches that manage to turn the codebase inside out, create hard and undebuggable problems out of the easy ones, and sometimes provide random subset of guarantees of various degrees of usefulness (I’m actually a fan of FRP, just not the misuse of it)?
Somehow Django, with its clear admin system and other out-of-the-box QoL modules remains as close to the most practical and useful approach as one can imagine. (Still even people who use it manage to reimplement its parts for no reason. Every sufficiently big webapp contains a bad, buggy, incoherent implementation of 90% of Django)
There are incredible amounts of money and effort and time to be solved by creating frameworks and abstractions that help with the glorious use-case of “huge, annoying, evolving in all the wrong directions essentially CRUD app”. Yet we’re not even stuck with Django - I see webdev regressing into horrifying piles of JS…
I'd say it's so close to every single case, thinking about the rest is best characterised as daydreaming.
Scale horizontally and you're often constraining and or complicating your design/architecture, but you can handle outages far better, if not seamlessly. It's much harder to get right, so it's usually best for more senior teams that have specific need for it.
Or (another Greg Young reference) - optimize for the ability to delete your code.
Indeed. It’s often the case that new is worse, and the authors and influencers pushing the new don’t know it yet, or sometimes they do but they downplay that knowing it will take time to get better, and underestimate how long. The reason old defaults to better is simply because it’s been used more, it automatically solves more problems because by the time it’s old it’s been built to solve more problems and it’s weaknesses have been fixed or patched or shimmed or worked around. Old isn’t automagically better either, of course, and things can get too old and too shimmed, but if the new thing is solving the same problems and being compared to something well tested and used by many, it’s very unlikely to be better until it starts getting old.
Maybe we should take a minute to ask: better for who? Better for customers is different than better for programmers. New is both more fun and easier to maintain. New is better for programmers. Old is better for customers, it’s more stable and changes more slowly.
There is something pretty important to be said for being an early participant in a project, programmers who are involved in building and integrating new things automatically become more productive than programmers who get hired to maintain old things. I’ve seen this first-hand in several different ways, the most stark of which was selling a company to programmers better than me, but took years to become productive because they didn’t know how everything worked and didn’t want to break anything.
There is also something to be said for replacing old with new. I’ve seen first hand teams tear down old engines because they felt crufty to build new ones, with the promise that it was going to be fast and easy and that they learned their mistakes the first time. Two completely separate companies launched into a 1 year rewrite that ended up taking 5 years, costing tens of millions, and making a good deal of the same mistakes over again before they were done. Both cases were failures to evaluate how well things were working in the old system, they were blinded to the overall success by the long list of rather minor problems.
But maintainability is not an end in itself. The software also has to do what it is intended to do. That is way more important than how the code looks, in my (current) view. Therefore I write some pretty sloppy code. The reason not to write "perfect code" is also that lot of code gets thrown away at some point. Then all the effort that went into polishing it was wasted.
Linting I only use for adding/correcting a file header, order or methods or other things that it can fix automatically.
Fun problem a few days ago: a slight design flaw allows a class of bug that a typechecker can't currently catch. Fixing the design flaw involves updating 25,000 lines across a few thousand files. Only 6 of the 25,000 lines demonstrated the bug (~1 in 4,000).
If we wait two months we can build a tool to catch the errors (and many others like it) without updating the code. The design flaw stays, but its impact is nullified.
Not that I think stateless is inherently superior, just that containerizing and scaling out sort of assumes moving state out of the application server and into redis, postgres, mongo, elastic, kafka, rabbitmq, …
As the OP points out, GC in java-land isn't worth writing home about on 1vCPU. When you throw a 16GiB heap at it though, with 16vCPUs then you'll unlock some of the really interesting optimizations and garbage collectors that the showcase the JVM in an advantageous light.
So it's not that the JVM can't scale horizontally - that's a matter of system architecture - it's that scaling it as 100 tiny nodes is far less impactful than having 5 large nodes.
The problem is using JVM without understanding some basic properties of it. Scaling is a matter of how you programmed it, not the language used.
That's right: PHP, baby. Ugly, hacky, sticky, oozing PHP.
Oh woe is me, etc etc, but you know what? It makes the customers money, which makes the company money, which makes me money. I've gotten used to eating and living indoors, so this is a good thing.
It's not serverless, there is no GraphQL, it's not within a mile of the nearest Rust compiler and it could not be implemented in golang even in my wildest fever dreams, with or without generics. There isn't even any machine learning!
We need to keep in mind the trillions of lines of legacy code out there that is still being maintained, refactored and rewritten, every day, that is not a new product or a groundbreaking new paradigm. It's just the internet, and the hacky PHP and smooth perl5 that keeps it running.
So yeah, 99% might be an exaggeration, but if I was making a new IDE (or whatever) today, I'd sure as hell target WordPress before I started making up my own buzzwords. I can't say for sure if that's because of my perspective or because of some objective data, but I do know that it would be a product with a hell of a lot more customers.
* it’s possible the market isn’t lucrative enough for a VC funded SV company; perhaps other models in other locations might make it more cost effective to serve that market
* putting the profit motive aside: I think there is a lot to learn from understanding how the 99% of developers work and how they may be enabled by tooling to make them more productive. The solutions to the problems faced by that group may actually help build better technologies.
We run a few Github bots & a function that updates a Route 53 record on serverless. (Security didn't want to give permission to R53; "too much, too broad"; a lambda that exposed only the necessary action to the service that required it was the compromise). But it's all extremely low-frequency stuff with no or little state, where the costs of a VM would far exceed the costs of "serverless". It was the right tool, for those jobs. (& it's usually niche stuff… I'm trying to think if I've ever worked somewhere with something core on Lambda or the like…)
But also got tons of VMs. Lots of VMs. Probably too many VMs.
Will their requirements ever scale to a point where it actually makes sense for them to be deployed as serverless services?
And even that. Let’s treat scaling issues latter. Create something that can be scaled horizontally dumbly, like a monolith you can run on n servers so you can scale without loosing money and then scaling can become your problem.
And that’s probably never if you are b2b like a majority of companies, since your usage depends and can be predicted from your sales team performance.
How would you gain professional experience in something that is out of your current scope?
For example, if a frontend web developer wants to pivot to backend using serverless.
Serverless allows anyone to be scale for minimal up-front cost and only pay for the usage they really need.
So glad that FAANG attract all this rare talent so us no-FAANG engineers can make a buck.
The given examples are really weird to me because they're some of the few things from gigantic companies that actually work properly.
The biggest single trick I learned was to use 1 sqlite connection instance for the entire lifetime of the application. You can add orders of magnitude more throughput with this path. SQLite serializes writers by default so dont duplicate the effort. You can even use the RETURNING keyword to grab insert keys without needing to lock over LastInsertRowId.
We also are close to 100% of business logic being executed via SQL queries as well.
Would you mind sharing how you implement/use 1 connection?
(From mine, it is an explicit requirement that we weather such events. I keep trying to keep global outages from our platforms off the requirements list…; so, IDK, perhaps your requirements allow different a different approach.)
Nevertheless, checkout https://litestream.io - given that SQLite is just a file, it’s incredibly easy to replicate. Failover is coming in the next release, which is already beta.
Also checkout this Expensify blog post on how they use SQLite. It’s scales amazingly well. https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-q...
EDIT: I don’t understand, why the downvote?
Every few years in Seattle, there's a big snowstorm, and everything just shuts down. We live.
If a service goes down once in a blue moon, I could catch up my email, spend some time morale building with my team, or, god forbid, just go home because work isn't that important.
Now if they said postgres on RDS can be used for 95% of real world use cases, that I could get behind.
Our roadmap has replication being done at the application level (i.e. business logic is aware of other nodes and policy around replication). Those specific entities which absolutely must survive would be synchronously replicated to additional witnesses on the network.
Note that this is about non-repudiation, not operational uptime. If we cared about always on operations, we would take a radically different path.
Abstractly, I think the role of ensuring business data survives some incident is not something to be pushed down to the database layer. When you pull this concern into business logic, you can produce far more robust solutions.
Most businesses seem to handle downtimes okay; some money[1] is lost, maybe.
[1] And if the business feels that the expected (negative) value of the downtime is less than the cost of mitigating it, they simply (correctly) ignore it. For example, most businesses don't build a separate highway to their building in the event that a traffic jam causes fewer customers to show up. The expense is not worth the savings.
There is good middleware like activeMQ that can manage transactions in the meantime if your DB is on holidays.
A but of planning makes you very resilient against threats.
Woah. There are some gotchas. It still won't support as much concurrency as row-level MVCC. But it'll still be a big improvement.
https://www.sqlite.org/cgi/src/doc/wal2/doc/wal2.md
https://www.sqlite.org/src/doc/begin-concurrent/doc/begin_co...
...because complexity pays people's salaries...
You are absolutely right. Unfortunately this puts us in a minority in the CV-driven-development era.
There are numerous opposite examples when I see that this new tech, lib, tool actually saves me time and money. I pay quite a few dollars for software tooling. Well if the tool does not offer perpetual license it is a no go for me then.
This is all that matters to me. On desktop for example I skipped moving to that .NET bandwagon and stayed with Delphi for my GUI desktop products. They worked 20 years ago and they work the same now. Single 10MB self updating exe with zero deployment issue. And free from numerous limitations imposed by UWP. Competitor is 1GB package with crapload of problems and every update turns a nightmare for customers. In my case all the time is spent creative stuff that brings me new customers / money instead of feeding someone else. Sure it costs me few hundred a year but that is peanuts.
As a non power user Windows is consistent enough.
Obviously not for power user who often stray from the green usage path of Windows.
Sort of like comparing working on a nodejs http server project on Windows vs working on a project that extensively uses winapi.
Very rarely is the data stored in the Access file itself.
Pretty much any tech choice (including Excel done sensibly) would work for those volumes.
(ETA: Of course, easy to spin things the other direction and read Microsoft's offer as a bully move "join us or we'll build a better you without you". The reality is somewhere in the middle of the two extreme perspectives.)
It just boggles the mind, but fair enough seems like a marketing failure rather than a technology one.
If you want to back that up with static types Ruby > 3.0 provides that out of the box too.
What is it you think it's lacking?
This is extremely common, in my experience. (Like it's a default tendency of nature.)
So, the VMs example was my previous employer. An yep, not really any multi-app VMs. There were some that did do a couple of things, but it wasn't great: it meant the deployment and dependencies of anything sharing a VM were all interdependent … and often under specified. Painful.
It's why Kubernetes exists, really. (Which is funny how much hate it & Docker seem to get on HN.) In my current employ, we do use k8s, and while much runs on it, and it's nice, we still haves some single-service VMs. I'd like to move them all into k8s if at all possible, but it is not always possible. Or it's not always that time gets dedicated to it.
Logical database backups (I'd point to mysqldump/mydumper & myloader if we were talking about mysql), combined with physical snapshots & copies of the binary files.
500 GiB is a lot, but not too out there these days.
Longer term, we are looking at synchronous log replication so that no business data would ever be lost on a primary outage. Having an availability problem is acceptable as long as we can bring everything back up in a reasonable amount of time.
Of course your competitors will do the same, you can't scale up an organization of top 1% professionals, so you don't really need them to compete in the market but you do really need them to solve a lot of problems. Just that today those problems aren't being solved in most places.
Is that assessment accurate to reality? How would we know if the bottleneck for most business is "my employees aren't good enough to solve hard technical problems" or "most of our problems aren't hard technical problems"?
What are some of these problems exactly, that me and my extremely large toolkit of cool things I can curl from Github can't solve?
Because I'm not a genius.
But I'm real good at stapling together some useful crap.
OSes, compilers, game engines, distributed process frameworks, etc.
Which as you'll note, are infrastructure things. Things which need 1/100th the manpower (at most!) to maintain.
While everyone else is busy churning user-facing apps.
You don't need to do that stuff, you career can be extremely successful without ever going near those things, especially if you are not interested in them.
Or how about high performance? Google famously knows that every millisecond of delay strongly impacts user retention and bounce rate, yet you wouldn't know it browsing many "99%" services. Achieving consistently low end-to-end latency is really hard regardless of scale, it's so easy for performance issues to creep in from every corner of your stack and accumulate.
These are hard problems and IMO they are rarely solved well by just stapling together some useful crap.
Ah, you mean grid/utility computing. But as you point out that's broader than FaaS.
Once it is sloppy, it does not get easier to make code unsloppy as time goes by. Of course, "not sloppy" does not mean perfect; but it does mean understandable, testable, maintainable and extensible.
> understandable, testable, maintainable and extensible.
Those are all adjectives, just like "sloppy" is. So we must each make the decision as to how much effort we will invest in making code MORE understandable, MORE testable, MORE maintainable and MORE extensible.
If the value of any of those properties is 0, then the code is definitely sloppy.
Limited horizontal replication is not the same as PAXOS.
When folks talk about scaling up for stateful resources (database), this typically includes having stand-by replicas to which one can fail over within seconds.
Scaling up the DB and scaling out stateless compute nodes is more than enough for many, many use cases. Having a DB machine with one 1TB RAM isn't anything special these days.
How fatal[1]? Not all outages are fatal. Scale vertically and you'll have some downtime if any node between (and in) your user and your system goes down. For 999 out of a thousand businesses, it won't be fatal.
If your business is so fragile that the first 24 hour outage kills it, then it would have been killed anyway in the near future.
[1] I know, this is liking asking "how pregnant?"...
Setting up something like a read replica on RDS (or frankly even a DB you manage yourself is pretty trivial, especially if you're using it solely for availability and not actually reading from it.
Come now, is there any reason to throw those insults?
Litestream: What are the durability guarantees? (Are there any?) what would make me pick that over RDS or Cockroach? "given that SQLite is just a file, it’s incredibly easy to replicate" — sorry, that just doesn't click with me. What about a file is "inherently easy to replicate"? I would not know how to implement something that wouldn't be O(n) with the filesize. Ideally you'd just sync up changes, but how to get those? Normal DBs with streaming WALs (RDS/PG/MySQL) or Raft-based replication (Cockroach) have good answers there.
Expensify: I skimmed the article, but AFAICT, the article talks about scaling QPS to a single SQLite file, but nothing about replication/durability.
On my previous job, I had been working for 15 years on the development of a complex business system. It included desktop apps, mobile apps, webs, on-premises and clouds. Throughout the years, we have introduced many then cutting-edge technologies for new products within the system. Some technologies before they were cool. But, the fine-working-already-done products, we kept supporting with the original technology for the lifetime of the product.
The point is that many new tools and technologies bring a very limited value to the finished working products.
Now, I am a maker of the new development tools. So, I am eager to push them to the world, but wouldn't like to be perceived as an "architecture astronaut". Your opinion helps in understanding how and why engineers choose new tools and technologies.
>"Now, I am a maker of the new development tools."
This is a part where I spend money. Good tools are very valuable as they directly save me time / money.
>"The point is that many new tools and technologies bring a very limited value to the finished working products."
Even for new ones. For example my servers are modern C++. In theory I should be using Rust / Go for new ones if I listen to a chorus. Guess what modern C++ works just fine for me and produces stellar results hence no reason for me to switch. I do some toy projects with new languages / tech to get a grip and be aware just in case.
The user won't care about intricacies, but they've heard of the words "network" and "support" and they'll understand the gist of the problem.
What distro of linux / unix should you go for? FreeBSD? Fedora? Arch? It probably doesn't matter that much as they all do the same thing.
2. You probably notice macs more because of their uniform branding, compared to a mish-mash of different looking other laptops.
3. Is it not enough to watch and listen to a keynote without pulling out a laptop?
The prevalence of MacBooks at tech conferences is in my experience a very US-centric thing. I rather associate ThinkPads with tech conferences.
I have been a windows user for a long time and recently started working on a mac to write some iphone code. It is agonizing, but presumably not because there is any problem with macs, but just because the systems are different. Using the cmd key instead of the ctrl is the most obvious difficulty, but I can't say that makes macs better or worse.
One thing that really got me while using the mac is I was adding a file to my project and I double clicked on a folder to navigate into it. Coming from a pc this is what you would do. You also do that on a mac in other places. But here it inserted the folder into my project. I won't get into what happened when I deleted the folder from my project.
Using a system you are not familiar with sucks, whether it is a mac or a pc.
Other than this, is there something I don't know that makes developing on a mac so much better?
From 1995 until 2010, at least? For sure.
It was a crushing majority, even.
4a. Dual booted, is it running macOS, Linux, Windows, *BSD?
4b. Is it running a VM (or more than one)?
4c. Is it used as a very expensive terminal with SSH to a different *nix OS or a remote desktop system?
At various times, all those have been true of mine.
Strongly recommend reading this response by Terry Lambert on getting OSX to POSIX compliance.
https://www.quora.com/What-goes-into-making-an-OS-to-be-Unix...
I could, I guess, tackle them. But I've got an appreciation that what's already there is already pretty great.
I suppose that's maybe the answer to my question. You don't need to be 1% to solve a problem as well as the solutions that already exist.
You need to be the 1% to improve on the current state of the art.
Which in a commercial environment, as you can imagine, is exceptionally rare. They are more about finding unsolved problems than iterating on problems already solved by FOSS.
But in general I agree with you, the crushing majority of developers out there won't work on those things and won't ever even need to.
It's also a source of not always deserved elitism from the people that work on those things, and I definitely dislike that (there's enough elitism in this field, as-is).
I think there's a lot of people who are highly, highly motivated to work on tough problems with competent co-workers.
Those emotions are less driven by the domain itself, and more about the fact that those people have succeeded in finding meaningful work with co-workers of appreciable worth.
The tricky thing I find is finding that. Far too many regular projects are focused on mobilizing the dumbest blocks of wood towards a goal. Without actually investing in the human beings doing the work.
I think once we regain a culture of actually investing in people who aren't in the top 10% in their field, things can change. Currently though we do not have that culture.
Pay an extra $$$ for an outage scenario that occurs once a month, and doesn't materially impact our users? Hell no.
Spend XXX developer hours and/or $$$ on hardware to make the site 50% faster, instead of adding new features our users pay for? Hell no.
And let's be honest, for touting speed, Google's awfully slow and bloated these days.
What a perfect demonstration of "Those scenarios are rare if you have gotten used to what 99% can and can't do and therefore just dismiss many solutions without thinking since you know the 99% can't do those thing."
Yeah I wonder why these orgs always find these XXX hours & $$$ costs cost-prohibitive even though the benefits have proven to be substantial in cutting edge tech industries. It's almost like they typically dismiss smarter and more cost-effective technical solutions superior to naive "throw hours and hardware at it" approaches, because 99% can't do those things.
Should ask Reddit.
And yes, it's a sad state of affairs, but an hour of downtime in healthcare isn't all that uncommon. Convincing administrators to ~triple their hosting costs to avoid the occasional outage is pretty hard to do.
I mean, I recently watched a documentary on the 2003 east coast power outage, and one of the primary causes was the single computer in charge of issuing alerts (in an alert-based workflow system) going down.
This is because SQLite is just a file, not a network connection or named pipe. Obtaining and releasing file handles is a non-trivial task that takes significantly longer than reusing the same one.
Now, PHP doesn't share memory between requests so there's no way to re-use the same connection across requests. That's just a sad fact about PHP and I don't think there's a reasonable way around it other than not using PHP.
PS: I haven't used PHP in over a decade so my knowledge might be outdated.
no linux support on those games right? then there would be no option to use linux.
Developing something for like Sony or Nintendo might be more OS agnostic in the developer's seat, but then it relies on the tools working on those OS's.
Windows is never really a choice, it's more "I choose X, which only works reliably on windows"
I studied the spreadsheet applications design history to Create my class History of Tech Design [0] and got a totally new perspective on how good the excel team was at getting things right.
[0] https://workflowy.com/s/strate-history-of-te/a4ID6kKtznLwQC7...
And later they expanded into applications as we know today and built monopolies for email, texts, presentations, spreadsheets and browsing the information superhighway. All prior art, indeed, near zero innovation.
The accountants love it.
Why didn't they charge ahead and achieve dominance in the 90s?
The point is that when someone asks "As one of the 99%. What are some of these problems exactly, that me and my extremely large toolkit of cool things I can curl from Github can't solve?", the answer is "so you can build a cost effective system that doesn't take down the east coast power grid because a single computer goes down".
Yeah, reddit is notorious for its downtime issues. They're notorious because it's actually rather unusual in the consumer space to be so bad, they kind of suck compared to the majority of their high tech competitors.
That depends on the execution model/PHP SAPI. mod_php and php-fpm use long-lived PHP processes and so can support persistent database connections depending on the configuration.
https://stackoverflow.com/questions/39753/connection-pooling...
Or not using SQLite for this task?