All you need is Postgres until you scale into TBs of data. We use Postgresql as a durable workflow engine, vector search, time-series data, BM25 search, OLTP/OLAP engine, and a queue. It's basically the only dependency we have for https://lobu.ai

The main benefit is centralizing all the data in one place so we don't need to worry about copying data in between multiple systems. Once something becomes the bottleneck, you can eventually migrate to a purpose specific tool to scale out.To be honest, LISTEN/NOTIFY in my opinion is the most fragile part of PG but it's fine as start until you scale out.

Armin Ronacher's `absurd` is an implementation of durable workflows for postgres:

https://lucumr.pocoo.org/2025/11/3/absurd-workflows/

https://github.com/earendil-works/absurd

https://earendil-works.github.io/absurd/

I've not used it, but it's worth comparing to other options

I want to dig into this "free" workflow_error.sql. I'll assume 1024 byte workflow job descriptors, and the article's steady state of 10,000 jobs per second.

Possibility one: There is one index on the table, and it is the created_at TS. This query has to scan 10,000 jobs/sec * 60 seconds * 60 minutes * 24 hours * 31 days * 1024 bytes / job = 25,543 GB.

A KV store would scan exactly that much.

Possibility two: The primary key is refined to (state, timestamp). Assume a 1% failure rate. Now, we "only" scan and return 255 GB. A key value store would scan exactly that much. (This is probably the right physical design).

Possibility three: The primary key is (timestamp), and there's a secondary index on state. I guess we do an index join, where one side of the join is 25,543 GB, and the other side is one unsorted bucket with 255GB * number of months the system has been in operation in it.

A KV store wouldn't let you express that.

Now, what other ad hoc queries are we supposed to efficiently support over a one month lookback? Also, what does PG do if you tell it to scan 25TB at the same time as it's inserting 10MB/sec at 10K TPS? How is vacuuming configured?

My dream is, instead of separating data storage, state machines, valid state constraints, and the logic that transitions between valid states, we can actually unify these into some kernel of app state. Honestly, Postgres already has a lot of these capabilities, but I don’t see an obvious story on the app or product level, providing provably correct sets of states that apps can transition between, and which they can automatically expose to clients in informative ways (this user can like this post, but not edit). It looks colored Petri net shaped to me, but I don’t yet see a simple app state paradigm in the same way that the database has obvious successful boundaries.

Curious to know experience of people using DBOS and Temporal.

I have used Temporal in the past, works really good, my only problem with it was some limits on request payload or event sizes, created some inconveniences to us when building solutions. It also enforces good engineering practices, but sometimes you don't want to write special logic if your CSV file is larger than 2Mb, upload it to S3, pass link, then download it in the workflow.

What is your experience with DBOS? How does it compare to Temporal in terms of operational complexity, feature parity and anything else

I completely get the concept and agree - this is great way to build this kind of durability in a workflow system.

That said, my gamer-brain wants to call this "Save-scumming at scale." Which is to say, a lot of people already know that this approach works, but maybe they haven't made the connection to abstract CS stuff.

Another strategy that can be used to build robustness is to build your workflow out of idempotent operations. That can be useful for situations where the workflow state is too large to back up. Instead, you just run the job from the top and it's a bunch of no-ops until you start making progress again.

Conductor OSS does this quite well https://docs.conductor-oss.org/devguide/ai/index.html

https://github.com/agentspan-ai/agentspan which is essentially an agentic SDK layer for Conductor can convert any of your langgraph, openAI, vercel, or ADK agent and makes it durable and adds orchestration with no code changes.

At Estuary, we have an in-house Rust crate [1] for building scale-out durable actors / FSMs in Postgres. It powers all async activity in our control plane -- slews of fine-grain scheduled actions, complex change propagation through data-flow topologies, reliable alert and email delivery, and more -- at hundreds to thousands of state transitions per second (today). It's been a wonderful pattern to build on, and is all of three source files.

Here's a an example computing a Fibonacci sequence (very inefficiently, with lots of spawned sub-tasks and message passing) [2]

[1] https://github.com/estuary/flow/tree/master/crates/automatio... [2] https://github.com/estuary/flow/blob/master/crates/automatio...

I do love Postgres and DBOS.

I also recently started experimenting with https://github.com/earendil-works/absurd which is also Postgres and even simpler than DBOS. Their comparison is a great read:

https://earendil-works.github.io/absurd/comparison/

But for operational reasons I've started using sqlite for durable workflows instead. Porting the database concepts from either DBOS or absurd PG to SQLite is remarkably easy these days. A small polling loop instead of notify/listen feels fine for smaller workloads.

Since DBOS doesn't support Rust, we implemented a very minimal Rust version of this at https://github.com/tensorzero/durable. It has been quite stable and extensible but of course you need to be very careful with the SQL implementations. Hope this is interesting to readers here.

Continuously amazed by what you can do with few tools, as long as Postgres is a part of your toolkit.

I recently developed a distributed queue and it works really great - benchmarks great too, with no race conditions or conflicts. I used SKIP LOCKED so that workers can compete safely.

You can also have multiple workers across nodes avoid conflict by using session wide mutexes i.e. pg advisory lock.

As someone who uses dbos.dev, restate.dev, cf workflows here is a snippet from our Agents.md:

  Restate.dev:
    for payment integrations on northflank since its faster than cf workflows, independent of cf and its downtime and self-hostable vendor-lock-in free,
  Cloudflare workflows:
    for non critical stuff like csv/pdf report generations since it's very cheap.
  DBOS.dev:
    for workflows that need atomic messaging tied to a postgres db transaction for 100% reliabilty/durabilty(for example populating a materialized row or sending out critical email/push to a merchant).

DBOS and Restate are similar on surface but Restate requires a central "orchestrator" which has pros and cons but makes it easy to build with serverless workers on cf/vercel.

It also has VirtualObject which is a nice vendor-lock-in-free OSS alternative to CF's single threaded DurableObject.

Where DBOS absolutely shines is

1) Atomic messaging in the same db tx as your business logic via dbos.enqueue_workflow! This is often the most brittle part of any solution and doing it atomically and durably with same tx that ran your business logic drastically reduces lots of complexity.

2) Since DBOS stores workflow state in db it should be easy to build dashboard for observability from metabase/looker(I wish restate exposed its rocksdb instance so it could be hooked up to metabase).

This feels like the sort of architecture that starts clean and then gradually grows most of the things a workflow-native system already has. I've seen systems like this, seen companies that are built out of this idea, and built small systems like this over time.

Once you need retries, backoff, timeouts, cancellation, versioning, visibility, task routing, rate limits, leases, heartbeats, stuck-worker detection, replay/debugging semantics, workflow migration, fanout/fanin, long timers, audit trails, and operator tooling, the “just use a database” story becomes “build a poor copy of a workflow engine plus a bunch of workers.” pretty quick.

That may still be a good tradeoff for many applications, especially if Postgres is already the core operational dependency. But the comparison shouldn’t be “database vs overcomplicated orchestrator.” It’s more like “what complexity do you want to own, and what do you want to buy / offload to a professional system?”

Having inherited a few of these - you tend to home-grow an ad-hoc version of many of the existing OSS tools, but with less of the patterns baked in.

Not sure where the NIH ends and where you're actually better off with a supported orchestration approach. I suppose if you expect your program to be around a while (or need advanced features), maybe think about using something a bit more battle tested?

I feel it's way too hand wavy on consistency and correctness. My opinion as someone who've implemented marketing workflows that breaks all the time (and tons of painful lessons).

Strong correctness guarantee is something that should not be undermine. Even more important than availability.

The examples on the website is simple but heavily undermines the importance of correctness. Anyone who implement similar pseudo-code directly will eventually suffer from data correctness issue in crashes.

  @DBOS.workflow()
  def checkout_workflow(items: Items):
      order = create_order()
      reserve_inventory(order, items)
      payment_status = process_payment(order, items)

      if payment_status == 'paid':
          fulfill_order(order)
      else:
          undo_reserve_inventory(order, items)
          cancel_order(order)

We have a durable queue built into postgres to handle some complex notification-ish logic. It's worked excellently and while there are services various cloud providers would love to sell us to do that it's extremely cheap to run.

For that particular usage, the volume we process and business criticality make it a good choice for inventing here - but for other durable processes we just use off the shelf tools since the cost of maintenance would quickly outstrip the value.

Postgres is a great tool to use and far more powerful than most people give it credit for - but there's always the balance of in-house maintenance vs. paying rent for someone else's solution.

Isn't this Just Oban from elixir? :)

We work on disk log based architecture for workflows at Unmeshed (https://unmeshed.io/) which helps it to scale at a fraction of the cost of traditional workflow systems that are based on expensive databases.

Postgres is not cheap to run in the cloud at scale. We went for the cheapest infra, which is basically the disk storage.

Every time I am surprised to see the promises of the cheap durability without mentioning costs of running dirable postgres, which might be not easy or cheap.

Citing CockroachDB as an example of scaling Postgres made me spit out coffee. Was this LLM-written?

Convex has a workpool component that gives the ability to compose big complicated flows in an understandable way, and give you realtime updates on status of various pieces: https://www.convex.dev/components/workflow

I have only used two databases, SQLite and Postgres, depending on if the database needs to be bundled with the application. They both feel like magic. Even though I am not a religious person, I recognize the value in acknowledging a higher power.

How do you incorporate secrets in this kind of implementation? Stored in db?

The "everything can be done in Postgres" crowd is crazy. It is like a religion at this point.

how is this compared to hatchet?

Haven't used DBOS but use Temporal at current job and used it at previous job as well so I have about 1.5 years under me now. I also run it at home to handle some home automation tasks that aren't super time sensitive (the latency of workflows isn't super bad, but I wouldn't use one for something that is triggered by a motion event in my house unless we're talking about a timeout to turn something off after inactivity).

I really like running a thin rest API in front of it inside your vpc or k8s cluster or whatever to help with event driven triggers so that they don't have to worry about Temporal auth and checking workflow status if there is any decision making around that. This helps keep your event as logic-free as possible.

Let me give a vague example: you have some sort of db trigger, and this trigger either acts directly or puts the event on a queue, your handler calls the thin rest api with the necessary event details, rest API can make the decision if this starts a workflow, signals an existing one, or ignores it (the pattern for this can vary based on the situation, but SignalWithStart is common for me or just dropping if the event is not worthy of starting a workflow and no workflow for that <ItemYouCareAbout> exists).

Then the parent/child workflow ability is very valuable when you need to orchestrate different self-contained behaviors for a single object's lifecycle, with cancellability when an external factor changes the trajectory of an object.

Long, vague story short, I find it very powerful and easy to work with and has really helped move lifecycle logic out of APIs where things can easily become riddled with debt and precarious to manage. I agree with you that it helps follow more best-practices instead of just throwing logic some place that seems easy but becomes a hidden trap later.

I thought Temporal was overly complex, but as you said the best part is it does enforce good engineering practices.

Then I tried their Cloud offering and was appalled at their pricing. I burned through the $1,000 free credits before I even got something to production. Didn't want to bother with running a local Temporal, either.

Best solution is to just take inspiration from their architecture and then do it yourself in Postgres, IMO.

They've just released an external storage approach to solve the large payload issue. I don't 100% love it (it's bolted on, not an intrinsic part), and it's an early release right now - but you can consider this effectively solved for now.

we're using dbos for ai gen workflows and processing video files. understanding how to migrate from celery took time, but for our case it was worth it.

I run a large on-prem temporal setup - throwaway acct as they will likely out me.

Temporal is, in my opinion having run it in prod for over a year - poorly designed, slow and ridicliously heavy infra wise.

If you're doing anything non-trivial (say, 200+ events/workflow) and you need to run only a couple hundred of them concurrently all day, you're going to spend millions on infra, and it's still going to absolutely suck.

Try running their own benchmarks, the numbers are pathetic.

Their sales team is also absolutely appalling and desperate.

From a Developer standpoint, the SDK is quite nice though.

Don't get trapped into nexus, and if the sales team call you make sure legal is in the room.