It's 2026, Just Use Postgres(tigerdata.com) |
It's 2026, Just Use Postgres(tigerdata.com) |
You better hope you dont have any big indexes or your writes will queue and you start getting "database is locked" errors. Good luck queueing writes properly at the application layer without rebuilding a full distributed queue system / network interface / query retry system on top of SQLite.
I really really really wish SQLite or PGLite supported multiple writers, especially if the queries are not even touching the same tables.
SQLite is designed for application databases.
Postgres is designed for client-server.
It's not about "power" (what does that even mean?), it's about totally different design choices that are intended for different purposes. It's an architecture question, not a "power" or "visitors" question.
Would you except "Just use postgres for client-server" as a reasonable change?
Again, huh?
Client-server vs locally embedded does not map to "many bulldozers vs 1 shovel".
SQLite and Postgres aren't even really competitors. They serve extremely different use cases. I'm not aware that "most" people are picking Postgres when they "need" SQLite. What are you even basing this on? SQLite is already an extremely popular choice.
I agree that managing lots of databases can be a pain in the ass, but trying to make Postgres do everything seems like a problem as well. A lot of these things are different things and trying to make Postgres do all of them seems like it will lead to similar if not worse outcomes than having separate dedicated services.
I understand that people were too overeager to jump on the MongoDB web scale nosql crap, but at this point I think there might have been an overcorrection. The problem with the nosql hype wasn't that they weren't using SQL, it's that they were shoehorning it everywhere, even in places where it wasn't a good fit for the job. Now this blog post is telling us to shoehorn Postgres everywhere, even if it isn't a good fit for the job...
What I like about the "just use PostgreSQL" idea is that, unfortunately, most people don't use Redis well. They are just using it as a cache, which IMHO, isn't even equivalent to scratching the surface of all the amazing things Redis can do.
As we all know, it's all about tradeoffs. If you are only using Redis as a cache, then does the performance improvement you get by using it out weight the complexity of another system dependency? Maybe? Depends...
Side note: If you are using Redis for caching and queue management, those are two separate considerations. Your cache and queues should never live on the same Redis instance because the should have different max-memory policies! </Side note>
The newest versions of Rails have really got me thinking about the simplicity of a PostgreSQL only deployment, then migrating to other data stores as needed down the line. I'd put the need to migrate squarely into the "good problems" to have because it indicates that your service is growing and expanding past the first few stages of growth.
All that being said, man I think Redis is sooooo cool. It's the hammer I am always for a nail to use on.
They may be its point, but I frankly didn't see much use in the wild. You might argue that then those systems didn't need Redis in the first place and I'd agree, but then note that that is the point tigerdata makes.
edit: it's not about serious uses, it's about typical uses, which are sad (and same with Kafka, Elastic, etc, etc)
> Tell me you don't understand Redis point is data structures without telling me you don't understand Redis point is data structures.
regardless of the author, I think slop of that sort belongs on reddit, not HN.
Im also curious about benchmark results.
If you follow this advice naively, you might try to implement two or more of these other-kind-of-DB simulacra data models within the same Postgres instance.
And it’ll work, at first. Might even stay working if only one of the workloads ends up growing to a nontrivial size.
But at scale, these different-model workloads will likely contend with one-another, starving one-another of memory or disk-cache pages; or you’ll see an “always some little thing happening” workload causing a sibling “big once-in-a-while” workload to never be able to acquire table/index locks to do its job (or vice versa — the big workloads stalling the hot workloads); etc.
And even worse, you’ll be stuck when it comes to fixing this with instance-level tuning. You can only truly tune a given Postgres instance to behave well for one type-of-[scaled-]workload at a time. One workload-type might use fewer DB connections and depend for efficiency on them having a higher `work_mem` and `max_parallel_workers` each; while another workload-type might use many thousands of short-lived connections and depend on them having small `work_mem` so they’ll all fit.
But! The conclusion you should draw from being in this situation shouldn’t be “oh, so Postgres can’t handle these types of workloads.”
No; Postgres can handle each of these workloads just fine. It’s rather that your single monolithic do-everything Postgres instance, maybe won’t be able to handle this heterogeneous mix of workloads with very different resource and tuning requirements.
But that just means that you need more Postgres.
I.e., rather than adding a different type-of-component to your stack, you can just add another Postgres instance, tuned specifically to do that type of work.
Why do that, rather than adding a component explicitly for caching/key-values/documents/search/graphs/vectors/whatever?
Well, for all the reasons TFA outlines. This “Postgres tuned for X” instance will still be Postgres, and so you’ll still get all the advantages of being able to rely on a single query language, a single set of client libraries and tooling, a single coherent backup strategy, etc.
Where TFA’s “just use Postgres” in the sense of reusing your Postgres instance only scales if your DB is doing a bare minimum of that type of work, interpreting “just use Postgres” in the sense of adding a purpose-defined Postgres instance to your stack will scale nigh-on indefinitely. (To the point that, if you ever do end up needing what a purpose-built-for-that-workload datastore can give you, you’ll likely be swapping it out for an entire purpose-defined PG cluster by that point. And the effort will mostly serve the purpose of OpEx savings, rather than getting you anything cool.)
And, as a (really big) bonus of this approach, you only need to split PG this way where it matters, i.e. in production, at scale, at the point that the new workload-type is starting to cause problems/conflicts. Which means that, if you make your codebase(s) blind to where exactly these workloads live (e.g. by making them into separate DB connection pools configured by separate env-vars), then:
- in dev (and in CI, staging, etc), everything can default to happening on the one local PG instance. Which means bootstrapping a dev-env is just `brew install postgres`.
- and in prod, you don’t need to pre-build with new components just to serve your new need. No new Redis instance VM just to serve your so-far-tiny KV-storage needs. You start with your new workload-type sharing your “miscellaneous business layer” PG instance; and then, if and when it becomes a problem, you migrate it out.
Just. Use. Postgres.
> Why This Matters Now: The AI Era
> This isn’t just about simplicity. AI agents have made database sprawl a nightmare.
or:
> “But Specialized Databases Are Better!”
> Let’s address this head-on.
I agree with your point, Raja, but if I wanted an LLMs take on it, I’d ask an LLM.
GPTZero gives this a 95% chance of being entirely AI-generated. (5% human-AI mix, and 0% completely original.)
But I could tell you that just by using my eyes, the tells are so obvious. "The myth / The reality, etc."
If I wanted to know what ChatGPT had to say about something, I would ask ChatGPT. That's not what I come here for, and I think the same applies to most others.
Here's an idea for all you entrepreneur types: devise a privacy-preserving, local-running browser extension for scanning all content that a user encounters in their browser - and changing the browser extension icon to warn of an AI generated article or social media post or whatever. So that I do not have to waste a further second interacting with it. I would genuinely pay a hefty subscription fee for such a service at this point, provided it worked well.
But for non relational data, I prefer something simpler depending on what the requirements are.
Commenters here are talking "modern tools" and complex systems. But I am thinking of common simpler cases where I have seen so many people reach for a relational database from habit.
For large data sets there are plenty of key/value stores to choose from, for small (less than a mega byte) data then a CSV file will often work best. Scanning is quicker than indexing for surprisingly large data sets.
And so much simpler
yes pg is awesome and it’s my go to for relational databases. But the reason why mongo or influx db exists is because they excel in those areas.
I would use pg for timeseries for small use cases, testing. But scaling pg for production time series workloads is not worth it. You end up fighting the technology to get it to work just because some lame person wanted to simplify ops
I'd never advocate for a new oracle install. But, I'd likely defend an existing one. I've seen how effective pl/sql can be in complex environments. Rewriting all that sql just because "oracle man bad" (or whatever) is a huge fucking ask of any rational business owner.
I've had engineers want to talk about syncing it to MySQL using some custom plumbing so that they can build a reporting infra around their MySQL stack, but it's just another layer of complexity over having code just use Microsoft's reporting services.
I'll add, having finance people with Excel really like being able to pull data directly from MSSQL, they do not like hearing about a technican's python app.
eg Python, react... very little OCaml, Haskell, etc.
Then use Postgres until you can’t.
yea this is me. postgres is actually insane for how much is offered at checks notes free.99.
_however_ we are probably due for like. I don't know a happy configurator type tool that has reasonable presets and a nice user friendly config tool that helps people get going without sidequesting for a year on devops/dbadmin expertise. that isn't even a favored outcome imo, you just get pretty lukewarm postgres-deployers who are probably missing a bunch of important settings/flags. my team mates would probably shit themselves in the face of postgres configs currently, they are absolute rats in the code but good and proper deployment of postgres is just a whole other career-arc they haven't journeyed and a _lot_ of organizations don't always have a devops/dbadmin type guy readily available any time you want to scrap together an app who's just going to wait for your signal to deploy for you. or said devops/dbadmin guy is just.. one guy and he's supporting 500 other things. not saying the absence/failing to scale teams with such personnel is right, it's just the reality and being up against workplace politics and making the case to convince orgs to hire a bigger team of devops/dbadmin guys involves a lot of shitty meetings and political prowess that is typically beyond an engineers set of capabilities, at least below the senior level. any engineer can figure out how to deploy postgres to something, but are they doing it in a way that makes an orgs security/infra guys happy? probably not. are they prepared to handle weird storage scenarios (log or temp space filling grinding server to a halt) and understand the weird and robust ways to manage a deployment? probably not.
I suspect it not being open source may prevent a certain level of proliferation unfortunately.
Have you looked into it?
Supabase helps when building a webapp. But Postgres is the powerhouse.
Postgres won as the starting point again thanks to Supabase.
The argument, as in this blog, is that a single Postgres stack is simpler and reduces complexity. What’s often overlooked is the CAPEX and OPEX required to make Postgres work well for workloads it wasn’t designed for, at even reasonable scale. At Citus Data, we saw many customers with solid-sized teams of Postgres experts whose primary job was constant tuning, operating, and essentially babysitting the system to keep it performing at scale.
Side note, we’re seeing purpose-built technologies show up much earlier in a company’s lifecycle, likely accelerated by AI-driven use cases. At ClickHouse, many customers using Postgres replication are seed-stage companies that have grown extremely quickly. We pulled together some data on these trends here: https://clickhouse.com/blog/postgres-cdc-year-in-review-2025...
A better approach would be to embrace the integration of purpose-built technologies with Postgres, making it easier for users to get the best of both worlds, rather than making overgeneralized claims like “Postgres for everything” or “Just use Postgres.”
Start with Postgres for everything, then use custom solutions where you are running into the limitations of Postgres.
I take it as meaning use Postgres until there's a reason not to. ie build for the scale / growth rate you have not "how will this handle the 100 million users I dream of." A simpler tech stack will be simpler to iterate on.
Cloud providers will rent you enormous beasts of machines that, while expensive, will remain cheaper than rewriting for a migration for a long time.
You can get a few hundred TB of flash in one box these days. You need to average over 1 MB of database data per user to get over 100 TB with only 100 million users. Even then, you can mostly just shard your DB.
I personally think reading manuals and tuning is a comparably low risk form of software development.
Using new technology is probably too risky unless Postgres is not capable
Oh no, not a company hiring a team of specialist in a core technology you need! What next, paying them a good wage? C'mon, it's so much better to get a bunch of random, excuse me, "specialized" SaaS tools that will _surely_ not lead to requiring five teams of specialists in random technologies that will eventually be discontinued once Google acquires the company running them.
OK but seriously, yeah sometimes "specialized" is good, though much less rarely than people pretend it to be. Having specialists ain't bad, and I'd say is better than telling a random developer to become a specialist in some cloud tech and pretending you didn't just end up turning a - hopefully decent - developer into a poor DBA. Not to mention that a small team of Postgres specialists can maintain a truly stupendous amount of Postgres.
I queried said database... it was slow.
I looked to see what indexes they had set up... there were none.
That team should have just used postgres and spent all the time and money they poured into this fancy database tech on finding someone who knew even a little bit about database design to help them.
And TBH, developers are pretty dumb not to realize that the tech tools monoculture is a way for business folks to make us easily replaceable... If all companies use the same tech, it turns us into exchangeable commodities which can easily be replaced and sourced across different organizations.
Look at the typical React dev. They have zero leverage and can be replaced by vibe coding kiddies straight out of school or sourced from literally any company on earth. And there are some real negatives to using silver bullet tools. They're not even the best tools for a lot of cases! The React dev is a commodity and they let it happen to them. Outsmarted by dumb business folks who dropped out of college. They probably didn't even come up with the idea; the devs did. Be smarter people. They're going to be harvesting you like Cavendish.
Good luck maintaining such code. As we know from constant whining here on HN ("Hooks are too complicated..." "I don't understand useEffect"), React's model feels slightly more complex than other frameworks' models.
I can even argue that the more popular React is, the more React developers are needed.
That being said, while I think Postgres is “the right tool for the job” in many cases, sometimes you just want (relative) simplicity, both in terms of complexity and deployment, and should use something like SQLite. I think it’s unwise to understate simplicity, and I use it to run a few medium-traffic servers (at least, medium traffic for the hardware I run it on).
(I run a medium-sized Postgres deployment and like it, but I don't feel like it's a cost-effective solution to every database problem.)
PostgreSQL is good enough to get to medium sized with nearly every use case. Once you are there, you have the use case and the test data to test any alternative for it well, rather than trying to guess beforehand what you actually need.
The advice is basically "PostgreSQL is probably good enough for whatever you're building now, and you should only look for other solution once you are big enough that it stops being that"
It would be awesome to have easy clustering directly built-in. Similar to MongoDB, where you tell the primary instance to use a replica set, then simply connect two secondaries to primary, done.
This is very hard to fix and requires significant architectural changes (like Yugabyte or Neon have done).
Argument is that partitioning almost never happens, because of network reliably rerouting traffic.
I've never really regretted waiting to move to a new tool, if we already had something that works. Usually by doing so you can wait for the fads to die down and for something to become the de facto standard, which tends to save a lot of time and effort. But sometimes you can in fact get value out of a specialized tool, and then you might as well use it.
Huh, apparently this is controversial, based on the score ping-ponging up and down! I'm not really sure why though. Is it because of the reference to MariaDB?
Redis is many times faster, so much that it doesn't seem comparable to me.
A lot of data you can get away with just caching in-mem on each node, but when you have many nodes there are valid cases where you really want that distributed cache.
https://www.postgresql.org/docs/current/sql-createtable.html...
But it's perfect HN bait, really. The title is spicy enough that folks will comment without reading the article (more so than usual), and so it survives a bit longer before being flagged as slop.
PGlite would be perfect if only it allowed multiple writer connections. SQLite is ok but I want PG extensions and I want true parallel multi-writer support!
At some point you end up with binary columns and custom encoded values, to save space by reducing row count. Kind of doing away with the benefits of a DB.
It compresses the indexes pretty well but don't forget to turn off postgres' own copy on write mechanism
I really like having some JSON storage because I don't know my schema up front all the time, and just shoving every possible piece of potentially useful metadata in there has (generally) not bit me, but not having that critical piece of metadata has been annoying (that field that should be NOT NULL is NULL because I can't populated it after the fact).
I've fixed absolutely terrifying replication issues, include a monster split brain where we had to hand pick off transactions and replay them against the new master. We've written a binlog parsing as an event source to clear application caching. I can talk to you about how locking works, when it doesn't (phantom locks anyone?), how events work (and will fail) and many other things I never set out to learn but just sort of had to.
While I'd love to "just use Postgres" I feel the tool you know is perhaps the better choice. From the fandom online, it's overall probably the better DBMS, but I would just be useless in a Postgres world right now. Sorta strapped my saddle to the wrong start unfortunately.
btw, big fan of postgres :D
First, to use Postgres for all those cases you have to learn various aspects of Postgres. Postgres isn't a unified tool which can do everything - instead it's a set of tools under the same umbrella. As a result, you don't save much from similarly learning all those different systems and using Postgres only as a RDBMS. And if something isn't implemented in Postgres better than in a 3rd party system, it could be easier to replace that 3rd party system - just one part of the system - rather than switching from Postgres-only to Postgres-and-then-some. In other words, Postgres has little benefits when many technologies are needed comparing with the collection of separate tools. The article notwithstanding.
Second, Postgres is written for HDDs - hard disk drives, with their patterns of data access and times. Today we usually work with SSDs, and we'd benefit from having SSD-native RDBMSes. They exist, and Postgres may lose to them - both in simplicity and performance - significantly enough.
Still, Postgres is pretty good, yes.
Tiger Data offers continuous aggs via hypertable but they need to be configured quite granularly and they're not super flexible. How are you all thinking about that when it comes to postgres and aggregations?
I hope all of them donate to the PostgreSQL community in the same amount they benefit from it
Look. In a PostgreSQL extension, I can't:
1. extend the SQL language with ergonomic syntax for my use-case,
2. teach the query planner to understand execution strategies that can't be made to look PostgreSQL's tuple-and-index execution model, or
3. extend the type system to plumb new kinds of metadata through the whole query and storage system via some extensible IR.
(Plus, embedded PostgreSQL still isn't a first class thing.)
Until PostgreSQL's extension mechanism is powerful enough for me to literally implement DuckDB as an extension, PostgreSQL is not a panacea. It's a good system, but nowhere near universal.
Now, once I can do DuckDB (including its language extensions) in PostgreSQL, and once I can use the thing as a library, let's talk.
(pg_duckdb doesn't count. It's a switch, not a unified engine.)
I had added Spacey to my codebase for one of its features and found that just doing the work in the db was near instant and my containers would not run out of ram. I want to get back to Spacey for more involved nlp work but right now the db "just works". I think Oracle is nicer but postgres does what I need for a lot less money!
https://github.com/johnwatson11218/LatentTopicExplorer/commi...
In many cases, it is going to save you time by having less infra and one less risk while you're getting started. And if you find yourself outgrowing the capabilities of Pg then you look for an alternative.
1. Downtime doesn't matter. 2. Paying someone else (eg. AWS) to manage redundancy and fail-over.
It just feels crazy to me that Postgres still doesn't have a native HA story since I last battled with this well over a decade ago.
What you're describing would probably have been equally possible with Postgres from 20 years ago, running on an average desktop PC from 20 years ago. (Or maybe even with SQLite from 20 years ago, for that matter.)
Don't get me wrong, Postgres has gotten a lot better since 2006. But most of the improvements have been in terms of more advanced query functionality, or optimizations for those advanced queries, or administration/operational features (e.g. replication, backups, security).
And hardware has gotten a lot better too. As TFA writes: it's 2026.
That being said, Oracle Database SE2 is $17,500 per core pair on x86, and Enterprise is $47,500 per core pair. XE has hard limits on size and limits on active CPUs. XE also does not get patches; if there is a critical vulnerability, it might be years before an upgrade is released.
Nobody would deploy Oracle Database for new systems. You only use this for sunk costs.
Postgres itself has a manual that is 1,500 pages. There is a LOT to learn to run it well, comparable to Oracle.
For simple things, SQLite is fine. I use it as my secrecy manager.
Postgres requires a lot of reading to do the fancy things.
> XE has hard limits on size and limits on active CPUs
So no, it can't be used for free, you will pay for that with your time to keep it in usable state. We had to use it as our dev dbs because our customers used it and the size limit was a huge PITA. We mostly use MSSQL as dev db now because the other half of customers use that (we have to support both in the end) and it is way worse in every other way but there is no size limit for dev use.
In a more charitable interpretation, maybe the parent is talking about sub-100ms total round trip time for an API call over the public internet.
So many times when trying to just go for simplicity with SQLite it takes me like one working day until I run up against enough annoyances to where resolving those is more work than setting up the "set up and forget" postgres instance.
Granted, this is for personal stuff... but "Postgres packaged for low maintenance" is present in a lot of OS package managers! Even for smaller data analysis work SQLite perf leads _loads_ to be desired (once had QGIS struggling with a sqlite DB... pg made everything mostly instant. Indices etc... but stuff I _couldn't easily get with sqlite_)
If SQLite works for you that's great, I do think it's worth it for people to _try_ to do simple pg setups to understand just how painful it is to use pg (for me: not that high)
Second, when I need a local save file. Sometimes small local apps are better served by a save file and they save file might as well have an extensible format that I can update as I go. This is more rare but still can be useful.
The first use case is very powerful. A temporary SQL database that can be blown away with zero trace of it is great. And the ability to run complex queries on it can really help.
But 99% of the time I just use Postgres. It works, it has sane defaults, it is crazy extensible, and it has never not met my needs, unlike Oracle or MySQL.
``` uv run --with marimo marimo run --sandbox ```
and you’re ready to go.
i personally like postgres myself for just about all use cases that must be shared with others (app with more than one client that might be providing CRUD updates or anything really that demands a central data store). ive used sqlite a couple times with WAL to try and make a small app shared between 2-3 people who all would contribute updates thru it but it wasnt ideal. for postgres so many features/extensions its concurrent writes are fast as hell and if you just want to one-shot a solution then you cant go wrong, but it's ofc not the same as sqlite setup.
i think a lot of the pain with postgres is just learning to effectively be a knowledgeable db admin of sorts. its somewhere between being a competent devops guy and a dbadmin expert all in one. if you're actually doing some kind of production deployment it is kinda scary hoping you've got everything set up right. even supabase which makes this whole process trivial to get going requires an understanding of not-always-understood security premises that just make things spooky.
lot of words to say i dont get much out of these discussions tbh. theres just too many use cases and variables in everyones working/hobby lives im not sure that there is a proverbial bottom to any of it. some will use sqlite and some will use postgres and some will use some weird thing no ones heard of because they're afraid to rawdog sql and just want immediate graphql capability to be the main mode of data retrieval. some will show up here and talk about why you need redis in the middle.
its too much noise so i just keep using postgres because its free and boring and fast. end of the day i just want to make stuff people can use. it's a hard endeavor to do well alone, if you dont have a team of other experts who can help you put all the puzzle pieces together on how to deploy things the right way and also add pieces like redis or whatever... it's just a lot. it's hard to find where to get started. sqlite is the only solution that really by nature of what it is seems to champion the lonely developer, but the tradeoffs are big if you're trying to make something that should get used by many people.
Having to look up and spend time fixing permissions every time itself makes using Postgres for simple uses difficult for me but if you're using it ad hoc, any tips?
And Turso: https://turso.tech/
1) A single person, doing a ton of heavy lifting, leaves, or worse, turns it over, or silently sells out to a nefarious person.
2) A severe security vulnerability is found. If everyone is using postgres, everyone is vulnerable. Bonus points if the vulnerability is either not publicly disclosed or it is hard to fix.
3) Commercial/Government interests heavily influence and push the project into places that could make it vulnerable in any given way. This is absolutely a thing.
4) AI. No clarification here. Just use your imagination, with recent news regarding FFMPEG and other projects in mind.
There are two reasons why emailing us is better:
First, the negative consequences of a false allegation outweigh the benefits of a valid accusation.
Second and more important: we'll likely see an email sooner than we'll see a comment, so we can nip it in the bud quickly, rather leaving it sitting on the front page for hours.
Thanks!
Why not let your audience decide what it wants to read?
I say this as a long time HN reader, who feels like the community has become grumpier over the years. Which I feel like is a shame. But maybe that's just me.
That’s largely how these voting sites work.
Them saying: "What you get: pgvectorscale uses the DiskANN algorithm (from Microsoft Research), achieving 28x lower p95 latency and 16x higher throughput than Pinecone at 99% recall" is marketing unless they give how you'd replicate those numbers.
Point being: this could have been written by an LLM, because it doesn't represent any work-done by Tiger Data.
So what did the VACUUM-free, undo-based MVCC storage engine project stall? https://wiki.postgresql.org/wiki/Zheap
Why is there no InnoDB for PostgreSQL?
(Maybe OrioleDB will avoid a similar fate.)
(we maintain OrioleDB) Mostly because the TAM API isn't mature enough yet. Hopefully we can upstream more patches so that it's possible
> undo-based MVCC storage engine project stall?
From what I could gather, it ran out of steam simply because of the difficulty of the task. There is a lot of work involved to get the requisite patches into core and the community are (correctly) cautious
OTOH, if and only if you design your schema to exploit MySQL’s clustering index (like for 1:M, make the PK of the child table something like (FK, some_id)), your range scans will become incredibly fast. But practically no one does that.
The major thing I advocate for is don't pick a NOSQL database to avoid relational dbs, only to try and do a bunch of relational work in NOSQL that would have been trivial in an RBDMS. Postgres can even power graph query results which is great.
You can achieve that with Postgres as well if you accept duplicating the data by adding an index with an include clause containing all the non-key columns you want to return in the SELECT clause. This way, you'll get fast index-only scans.
With MySQl variants like percona xtradb setup can go from 1 instance to cluster to geo replicating cluster with minimal effort.
While vanilla postges for an equivalent setup is basically pulling teeth.
Once upon a time, logical replication wasn't a thing, and upgrading major versions was a nightmare, as all databases in the chain had to be on the same major version. Upgrading big databases took days because you had to dump and restore. The MVCC bloat and VACCUM problem was such a pain in the ass, whereas with MySQL I rarely had any problems with InnoDB purge threads not able to keep up with garbage collecting historical row versions.
Lots of these problems are mitigated now, but the scars still sometimes itch.
Just use memcache for query cache if you have to. And only if you have to, because invalidation is hard. It's cheap, reliable, mature, fast, scalable, requires little understanding, has decent quality clients in most languages, is not stateful and available off the shelf in most cloud providers and works in-clusetr in kubernetes if you want to do it that way.
I can't find a use case for Redis that postgres or postgres+memcache isn't a simpler and/or superior solution.
Just to give you an idea how good memcache is, I think we had 9 billion requests across half a dozen nodes over a few years without a single process restart.
Run benchmarks that show that, for your application under your expected best-case loads, using Redis for caching instead of PostgreSQL provides a meaningful improvement.
If it doesn't provide a meaningful improvement, stick with PostgreSQL.
I like Redis a lot, but for things in the start I'm not sure the juice is always worth the squeeze to get it setup and manage another item in teh stack.
Luckily, search is something that has been thought about and worked on for a while and there's lots of ways to slice it initially.
I'm probably a bit biased though from past experiences from seeing so many different search engines shimmed beside or into a database that there's often an easier way in the start than adding more to the stack.
But As soon as you go outside Postgres you cannot guarantee consistent reads within a transaction.
That’s usually ok, but it’s a good enough reason to keep it in until you absolutely need to.
I argue popularity is insufficient signal. React as tech is fine, but the market of devs who it is aimed at may not be the most discerning when it comes to quality.
Here is how I did it with SQLite, which was problematic with anything but basic types.
https://www.linuxjournal.com/content/sqlite-extraction-oracl...
They left much out, so many important things that I learned later, as I saw harmful things happening.
The very biggest thing is "nologging," the ability to commit certain transactions that are omitted from the recovery archived logs.
"You are destroying my standby database! Kyte is explicit that 'nologging' must never be used without the cooperation of the DBA! Why are you destroying the standby?"
It was SSIS, and they could never get it under control. ALTER SYSTEM FORCE LOGGING undid their ignorant presumption.
It has always felt to me like devs will gravitate towards doing the opposite of what makes sense for their DB. If they have a Document DB, they'll try to use it relationally. If they have a relational database, they'll shove everything into a JSON column. Then in both cases, they complain that it's slow.
Same with MySQL and many other "traditional" databases. It tends to work out because these failures are rare and you can get pretty close with external leader election and fencing, but Postgres is NOT easy (likely impossible) to operate as a CP system according to the CAP theorem.
There are various attempts at fixing this (Yugabyte, Neon, Cockroach, TiDB, ...) which all come with various downsides.
[1]: Someone tried it with Patroni and failed miserably, https://www.binwang.me/2024-12-02-PostgreSQL-High-Availabili...
I understand that reading the title and probably skimming the article makes it a good jumping off point for a comment thread. I do like the HN comments but I don't want it to be just some forum of curious tech folks, I want it to be a place I find interesting content too.
The flexible typing is the biggest WTF to me, especially because it necessitates insane affinity rules[0]. For example, you can declare that a column is of type “CHARINT” (or “VARCHARINT”, for that matter), and while that will match the rule for TEXT affinity (contains the string “CHAR”), it also matches the rule for INTEGER affinity (contains the string “INT”), and since that rule matches first, the column is given INTEGER affinity. "FLOATING POINT" maps to INTEGER since it ends in "INT", and "STRING" maps to NUMERIC since it doesn't match anything else.
Then there are the comparison rules (same link). NULL < NULL, INTEGER || REAL < TEXT < BLOB - but those may be altered at comparison time due to type conversion. Hex values as strings get coerced to 0 as INTEGER, but only if they're in the SQL text, not if they're stored in a table. Finally, no conversion takes place for ORDER BY operations.
This is particularly galling considering that most of sqlite3's display types (this is `markdown`) don't visually differentiate between string-types and numeric-types - I manually added the strings on rows (by PK) 2 and 4 to assist the explanation.
sqlite> CREATE TABLE foobar (id INTEGER NOT NULL PRIMARY KEY, b BLOB NOT NULL);
sqlite> INSERT INTO foobar (b) VALUES (10), ('10'), (0xA), ('0xA');
sqlite> SELECT id, b, 15 > b, '15' > b, 0xF > b, '0xF' > b FROM foobar ORDER BY b;
| id | b | 15 > b | '15' > b | 0xF > b | '0xF' > b |
|----|----- |--------|----------|---------|-----------|
| 1 | 10 | 1 | 1 | 1 | 1 |
| 3 | 10 | 1 | 1 | 1 | 1 |
| 4 | '0xA' | 0 | 1 | 0 | 1 |
| 2 | '10' | 0 | 1 | 0 | 0 |
Clearly, it's possible to reduce the risk to the point where many companies are willing to accept it, but it's still a problem and comes with high operational costs. And for some use cases (like finance), even a small risk of undefined database behavior or lost writes is unacceptable.
The future are distributed databases with consensus, and unfortunately, Postgres isn't there yet.
If your network is down, your clients can't connect DB too, and nothing works. State of network partitioning is harder to achieve, its more like if connection between two datacenters is down where DB replicas live, which is more rare because of all global connection redundancy.
> If your network is down, your clients can't connect DB too, and nothing works
This sounds simple, but can only be achieved with high certainty using distributed consensus.
With 100 million DAU, you're often going to have problems with this rate unless you batch your commits. With 100 million user accounts (or MAU), you may be fine.
I'm sure we have different capabilities and constraints, but I am unaware of any fundamentally different approaches to indexes.
I don't know enough to comment yet but will go read about it.
The big problem for me from running DB on Btrfs is that when I delete large dirs or files (100GB+), it locks disk system, and Db basically stop responding on any queries.
I am very surprised that FS which is considered prod grade having this issue..
Very solid and no such issues.
This is well within my budget, but it sounds like there might be room for improvements?
memcached clients also frequently uses ketama consistent hashing, so it is much easier to do load/clustering, being much simpler than redis clustering (sentinel, etc).
Mcrouter[1] is also great for scaling memcached.
dragonfly, garnet, and pogocache are other alternatives too.
redis i/o is multithreaded, it's just the command loop that's single-threaded. If all you're doing is SET and GET of individual key-value pairs, every time I've seen a redis instance run hot under that sort of load, the bottleneck was the network card, never the CPU.
I ... actually think scaling redis for simple k-v storage is already pretty easy so I dunno that that's much of a concern?
mcrouter ... damn I haven't thought about mcrouter in at least 10 years.
It's my job to read HN posts and comments all day, every day, and these days that means spending a lot of time evaluating whether a post seems LLM-generated. In this case the post seems LLM-generated or heavily LLM-edited.
We have been asking the community not to publicly call out posts for being LLM-generated, for the reasons I explained in the latest edit of the comment you replied to. But if we're going to ask the community that, we also need to ask submitters to not post obviously-LLM-influenced articles. We've been asking that ever since LLMs became commonplace.
> I say this as a long time HN reader, who feels like the community has become grumpier over the years. Which I feel like is a shame. But maybe that's just me.
We've recently added this line to the guidelines: Don't be curmudgeonly. Thoughtful criticism is fine, but please don't be rigidly or generically negative.
HN has become grumpier, and we don't like that. But a lot of it is in reaction to the HN audience being disappointed at a lot of what modern tech companies are serving up, both in terms of products and content, and it doesn't work for us to tell them they're wrong to feel that way. We can try, but we can't force anyone to feel differently. It's just as much up to product creators and content creators to keep working to raise the standards of what they offer the audience.
That's false. The overwhelming sentiment of the community is that HN should be free of LLM-generated content or content that has obvious AI fingerprints. Sometimes people don't immediately realize that an article or comment has a heavy LLM influence, but once they realize it does, they expect us to act (this is especially true if they didn't realize it initially, as they feel deceived). This is clear from the comments and emails we get about this topic.
If you can publish a new version of the post that is human-authored, we'd happily re-up it.
Feels like a net negative for the HN community.
In other words, there are people and situations where it makes sense to use something else. But most people believing they're in that category are wrong.
I always get frustrated by this because it is never made clear where the transition occurs to where you are doing something special enough. It is always dismissed as, "well whatever it is you are doing, I am sure you don't need it"
Why is this assumption always made, especially on sites like HackerNews? There are a lot of us here that DO work with scales and workloads that require specialized things, and we want to be able to talk about our challenges and experiences, too. I don't think we need to isolate all the people who work at large scales to a completely separate forum; for one thing, a lot of us work on a variety of workloads, where some are big enough and particular enough to need a different technology, and some that should be in Postgres. I would love to be able to talk about how to make that decision, but it is always just "nope, you aren't big enough to need anything else"
I was not some super engineer who already knew everything when I started working on large enough data pipelines that I needed specialized software, with horizontal scaling requirements. Why can't we also talk about that here?
Rather my point was that people, including myself, have a tendency to believe they're in an exceptional case when they're actually not. And thus will see discussions on sites like this and assume that's what they need to do. And of course they don't understand the tradeoffs and wind up not realizing they're actually making things harder for themselves.
The classic example is scaling issues where people, again including myself, assume they have exotic scaling needs simply because it's larger than anything they've seen before. When in fact by objective measures what they have is something that could run perfectly fine on 20 year old hardware and bog standard techniques.
No I don’t. I’ve never used the thing so I don’t know when it’ll come in useful.
If they don’t, the trade off for NotPostgres is such that it’s justifiable to force the engineer to run their own benchmarks before they are allowed to use NotPostgres
Here's my opinion: just use postgres. If you're experienced enough to not when I say that, go for it, the advice isn't for you. If you aren't, I'm probably saving you from yourself. "Reasonable scale" to these people could mean dozens of inserts per second, which is why people talking vagueries around scale is madenning to me. If you aren't going to actually say what that means, you will lead people who don't know better down the wrong path.
I've sat in on meetings about adding auth rate limiting, using Redis, to an on-premise electron client/Node.js server where the largest installation had 20 concurrent users and the largest foreseeable installation had a few thousand, in which every existing installation had an average server CPU utilisation of less than a percent.
Redis should not even be a possibility under those circumstances. It's a ridiculous suggestion based purely on rote whiteboard interview cramming. Stick a token_bucket table in Postgres.
I'm also not convinced that thinking about reasonable scale would lead to a different implementation for most other greenfield projects. The nice thing about shoving everything into Postgres is that you nearly always have a clear upgrade path, whereas using Redis right from the start might actually make the system less future-proof by complicating any eventual migration.
> That's false. The overwhelming sentiment of the community is that HN should be free of LLM-generated content or content that has obvious AI fingerprints.
Yeah it is indeed, and for good reason: why would I spend time reading something the author didn't spend time thinking through and writing?
It's not that people don't like Postgres articles (otherwise, the upvotes would be much lower), but once you read a bit of the article, the LLM stench it gives off is characteristic. You know: Standard. LLM. Style. It's tiresome. Irksome. Off-putting.
What I'm wondering is, if LLMs are trained on "our" (in the wider sense of the word) writing style, and spew it back at us, what data set was it that overused this superficial emphatic style to such a degree, that it's now overwhelmingly the bog-standard generative output style?
Don’t get me wrong—I’m a huge huge fan of Postgres. I’ve worked at Postgres companies for a decade, started a Postgres company, and now lead a Postgres service within a company! But I prefer being real rather than doing marketing hype and blanket love.
"Just use postgres" is the snappy phrase you're looking for.