Postgres isn't slow, your storage is(clickhouse.com) |
Postgres isn't slow, your storage is(clickhouse.com) |
Do you want to spend thousands of dollar upfront for that DB server with sweet sweet NVME SSDs, of which your application might not even use 1% before the app get scraped?
sell a lie, blame the victim?
Kestrel (AspNetCore) + SQLite + NVMe all on the same machine is where things start to get really spicy in my experience. Imagine handling an entire form submission that mutates database state in tens to low-hundreds of microseconds. Total wall clock time required to service any HTTP request can be well under a single millisecond. If your average cpu time per request is 30uS, you could approach half a million per second on a 16-32 core machine.
Since I’m a freelance programmer, I mainly work on small-to-medium-sized projects, so I don’t know much about this kind of infrastructure programming.
There. Fixed it for you.
Too many times I see people making the same avoidable mistakes time and time again.
They get stuck in the dream of "portable" schemas and don't use their brain to design a database-specific schema.
They have no clue how to write SQL queries (or again, have stupid dreams of "portable" queries). As a result performance suffers.
Then they come to your desk complaining the databse is slow, and insisting that they absolutely MUST have a Redis caching layer.
I'm sick and tired of people using databases as blackbox dumping grounds. If slow storage was always the problem, my life would be far less stressful. Since the prescription would be easier for the doctor to write. ;)
On the other side if dev(ops) does not know that local storage is faster than network/cluster filesystems…
Happy to still be here among my fellow humans. Hackernews the last bastion it feels like.
Can we somehow filter out slop from the front page? That would be great :).
And then you push it to production and get completely different results
Statistics change everything. Queries which produced beautiful plans in local and staging environments breakdown when given production statistics, some users get the same plans you did, others get nasty nested loops, indexes don't get used, and debugging and fixing it requires poking around in prod statistics which is never a good place to be. Don't even get me started on how RLS impacts query optimisation
Postgres is great but the authors refusal to add plan hints is a decision that comes to bite me at least a few times a year
Sure. But to be clear: I am not denying storage is a genuine problem for some people.
I am just saying in the majority of cases it's probably not "the storage". For most people "the storage" will be quite far down the list of likely causes of a "slow" database.
Throw all the Enterprise SSDs or other storage solutions you like at a shitty schema and it will still struggle under load (or sometimes just struggle full stop).
Lots of things in PGSQL benefit from thinking about how your use case is manipulating the underlying B-tree, if you work in performance sensitive stuff the devil is really in the details. And having to rebuild indexes that have decayed over time is painful.
Optimizing column layout has been enough for me to leave OLAP workloads in Postgres a few times when I otherwise would've had to spin up something dedicated for them.
Erm. Isn't this kind of a schema specific question ?
I mean, sometimes I see schemas that are so goddam awful that the answer to your question would be "anything, literally anything".
Other times its the old story of failing to use or misconfiguring indexes.
And then, yes you end up in the realms of Postgres specific stuff. What you do there clearly depends on what you're looking at.
But the point is, I could probably count on one hand the number of times the problem has been "the storage" or "the database". Most of the time its "the developer". :D
But using detectors to label so people can choose seems entirely reasonable to me even with a risk of false positives, and I think a lot of people who are generally pro-AI would feel that's fine.