Preamble: there was a discussion on Twitter about pricing mechanisms for vector DBs, so I jotted down some thoughts. The reason for posting it on HN is that I would love to get additional thoughts.

Most native vector databases[1] are actually vector search engines (e.g., one can do vector search with ClickHouse, which would be more of a "vector warehouse") relying on memory[2] for persistency and things you want from your vector search engine are a healthy[3] tradeoff between scale and speed. In a SaaS solution, this means that CPUs can be shared and allocated accordingly based on usage in a cluster[4]. Disk is -somewhat- neglectable and can be captured in overall pricing, but memory is crucial because it's the thing that keeps your database persistent (i.e., you get a response in ms any time you query, no cold starts).

For pricing, this boils the options down to roughly three variations;

1. Charge for CPUs and memory used. Or;

2. Create a box (aka VM, node, pod, etc). Or;

3. Charge for the lowest denominator (a single vector dimension).

Based on customer feedback, what most people want to know is simple: "with my dataset and use case (throughput, QPS, etc), how much will I pay per month?"

In the first case, the user needs to do the math. They need to know that different algorithms in combination with different embeddings sizes (e.g., mpnet-base's 768 vs Cohere's 1024 vs OpenAI's 1536, etc) have a different memory footprint and that CPUs have a different impact on performance. In the second case, spending increases in a staircase-like manner. Again, the user has to do the math, often something like: 1 box takes x-embeddings of size-y with a QPS/throughput of z. If you have (x, y, z)+1, you need another box. I.e., it can get expensive quickly because you're scaling over multiple axes. Charging for the lowest denominator (a single dimension) means that you can go to the most granular level in pricing.

At Weaviate, we've chosen the latter because the only three things a user needs to know are: embedding size of the model, data objects stored per month, and queries per month.

Is this optimal? For some, it is; for others, it isn't, so we keep refining the approach. But I still think that taking the abstraction away and not running the risk of overcharging by simply adding boxes or by letting your customers figure it out (i.e., letting them do the math) is the right approach.

PS: For hybrid-SaaS, this is different because our customers pay for the infra themselves. Here you have a simple markup for additional services and the software on top per hour.

[1] This is why we called Weaviate a vector search engine initially because that's what it does.

[2] This is all based on the current algorithms used in the vector DBs; the moment this moves from memory to disk (i.e., a full CRUD, disk-based ANN index), this opens up the door for completely different pricing models.

[3] With healthy, I mean things like not sacrificing throughput for speed. Etc.

[4] This is added value in a SaaS solution; you can leverage economies of scale (the more customers, the more efficient usage of resources.

user:	rolandtannous
created:	July 8, 2023
karma:	2
about:	Strategy and Digital Transformation consultant. Founder at Gravity Think. http://gravitythink.com Building Chory https://chorypm.com