How we built a serverless SQL database(cockroachlabs.com) |
How we built a serverless SQL database(cockroachlabs.com) |
The answer is "one". If you have less than 10k req/s you shouldn't even start to think about multiple DB servers or migrating from bog-standard MySQL/MariaDB or Postgres.
I will never understand this obsession with "scaling". Modern web dev seriously over-complicates so many things, it's not even funny anymore.
What happens when that database loses some data? Do you want an up-to-the second backup, or point-in-time recovery? Or are you OK restoring last night's backup? Distribution isn't only about scale, it's also about durability.
What happens when you need to run an expensive business process ad-hoc? Do you want it to be easy to scale out reads, or to export that data to an analytics system? Or are you OK building something else to handle that case? Distribution isn't only about scale, it's also about flexibility.
What happens when you want to serve customers in one market, and make sure that their data stays local for regulatory compliance reasons or latency? Are you OK with having separate databases? Distribution isn't only about scale, it's also about locality.
When you are big enough to worry about the other issues, you surely are big enough to handle the requirements in-house. I see the dependence on some specific companies as the bigger threat to reliability.
I find that it's not often that people grasp that distribution is about availability. It's obvious when you say it, but for a long time my own intuition was that distribution is about mostly durability or consensus protocols to provide total order across multiple machines. Yet these build together into availability.
In fact, I first noticed this distinction when reading Brian M. Oki's seminal 1988 paper on Viewstamped Replication, the work that would pioneer the field of consensus—a year before Paxos but with an intuitive protocol essentially identical to Raft. The surprising thing is that today many of us might have titled the paper something about "consensus" or "total order" (which it practically invented, and which was the major breakthrough, at least how to do this in the presence of network partitions) but that he titled it "Viewstamped Replication: A New Primary Copy Method to Support Highly-Available Distributed Systems".
I did a short intro talk to Viewstamped Replication (and particularly why FTP or nightly backups or manual failover are not a solution): https://www.youtube.com/watch?v=_Jlikdtm4OA
The talk is followed by interviews with Brian M. Oki and James Cowling (authors of the 1988 and 2012 papers respectively).
Serverless is not just about auto scaling up from 1 to n, it's about autoscaling down from 1 to 0.
If Cockroach provides a robust SQL DB at a marginal cost of ~$0/mo for small request volumes, that is a real value add over running your own pg server.
Not having to deal with administration or backups is another big value add.
This offering looks like it compares very nicely to say running an rds instance with auto backups enabled.
In your k8s example, by running a k8s cluster when you don't need one, a cost you pay is the overhead.
In the Cockroach serverless case, costs that come to mind include vendor lock-in when you evolve a pattern of production traffic that is hard to migrate to other solutions, and security and compliance challenges due to the virtualized instances running on shared clusters. In many cases these tradeoffs may be worthwhile. My point is that looking at it only in the dimension of scaling up and down, doesn't tell the whole story. The OP doesn't talk about tradeoffs, so in the comment section we must.
But with any service without a constant workload (I’d wager almost all services besides prototypes that get no users) you’re going to have to literally scale that one machine, by replacing it with a bigger machine. When you have 50 users you’re not going to be paying for some yy.24xlarge. You’ll start with something much more affordable. When the service grows to 50,000 users, you certainly won’t be at “Facebook scale”, but that t3.small isn’t going to cut it. Should your service ever decline, it’d be nice to scale that machine down to save on costs.
At a previous job, we spent many human hours continually ratcheting up the size of our Postgres machine a few times a year. Not only did this take non-trivial engineering hours and mind-space, it also caused maintenance downtime due to the limitations of traditional DBMSs.
Self-managed CockroachDB eliminates the downtime needed to scale. To handle a more intense workload, add machines. If you want to vertically scale each machine, that can be done without downtime too.
CockroachDB Serverless takes this a step further by scaling up and down to suit the demands of a highly dynamic workload, while minimizing costs.
Maybe what looks like an mega-scale obsession to you is actually a bunch of people trying to avoid the common headaches of managing a moderately sized, dynamic service.
The interviewer responded "When that matters, I won't even be managing the person whose problem that is."
The following questions are likely to come up
1) My t3.xl DB is down, how much bigger can I make it?
2) My r3.24xl DB can only handle 100 TPS and now my site is down, what can I do?
3) My 2x r3.24xl DB cluster costs a lot of money, Are other solutions cheaper?
4) My latency is high, are other solutions faster?
For someone who hasn't dealt with these questions before, these will become long and painful lessons with massive material impacts to the business.
It's appealing to use Dynamo as it takes the magic out of scaling. It's appealing to use serverless RDBMS as you don't have to think about it anymore unless it has high costs/latency.
The answer is very clear-cut:
Work with professionals.
[0]: https://github.com/hasura/graphql-engine/issues/678
[1]: https://github.com/cockroachdb/cockroach/issues/28296
[2]: https://www.cockroachlabs.com/docs/v21.1/stream-data-out-of-...
[0] https://web.archive.org/web/20090306191715/http://www.my-idc...
A lot of successful businesses start with things that are not scalable, and it is a strength, not a weakness. If you start a social network for instance, you can't beat Facebook at its own game. You have to do something that Facebook can't do because it is too big. Scalability problems will be tackled as you grow.
Among the many things Facebook can't do is running its service on a single database. It makes things much harder on them, thankfully, you are much smaller than Facebook and you can. Take that advantage.
Now, if you setup your DB using public cloud's flavor of PostgreSQL... That's a different story.
It’s amazing that this a killer and not standard feature, but here we are!
My question is similar, which is, is CockroachDB going to have an equivalent RDS proxy so that apps can handle traffic spikes and not have to deal with problems with DB connection pools
At least if cost and scaling to zero is important to you. Dynamodb addresses the cost issue, but it's so painful to use, and is unsuitable for a lot of use-cases. Aurora Serverless takes 30 seconds to cold-start, so it's also usually a non-starter, other than for batch type workloads.
So great work, but I have a couple of questions:
Any plans to support AWS privatelink? I played around with this a bit, and it seems you need to connect to it over the internet which isn't always ideal.
Will there be a limit on how many free databases you can create? I think there are some valid use-cases where one might want to create a bunch of them, but I would be scared to do this at the risk of being kicked off your platform for abuse.
Regarding a cluster limit, we currently allow up to 5 clusters per customer account. I'd like to hear what kind of use-cases you have in mind for having a lot more clusters. One I've thought about is CI runs, where you'd want dozens or hundreds of temporary clusters running at once, in order to run your testing in parallel.
In a similar vein to how you have made cockroach multi-tenant, not too long ago I worked on building a multi-tenant sass version of a business intelligence app. The app uses a relational DB, initially we used separate schemas on the same db cluster, but we had problems with noisy neighbors, as well as concerns about it's security.
We later opted to run dedicated database clusters for each tenant, however it greatly increases the marginal cost, and makes it difficult to provide a free tier of service, which is a valuable way to gain new customers.
What I can't really understand is why would someone use the dedicated cluster in AWS at that price.
Google says that Cloud Spanner should get you around 7,000 read queries per second and 1,800 write queries per second for $650/mo. If a simple indexed read is 1 RU, $650 on Cockroach Serverless would get you around 2,500 reads/second. Of course, I think it's completely reasonable for a Serverless option to cost a bit more given that you'd need to over-provision Cloud Spanner (even if you smartly increased/decreased the amount of compute allocated based on demand).
Planet Scale charges $15 per 100M rows read and $15 per 10M rows written. If an RU is a row read, then Cockroach Serverless would be $10 per 100M rows read. If a write takes 10 RUs, Cockroach would cost $10 per 10M rows written. Both of those would be less than Planet Scale's cost - but it's possible that a row read will cost more than 1 RU. Let's say that an indexed lookup of a row costs 5 RUs. Then Cockroach Serverless starts costing 3.3x more than Planet Scale.
AWS DynamoDB charges $1.25 per million write request units and $0.25 per million read request units. If I can get 10M reads from Cockroach Serverless for $1 and 4M reads from DynamoDB for $1, Cockroach's pricing looks pretty good. Of course, if I need 5 RUs to do an indexed read, the pricing doesn't look as good anymore.
I do respect Cockroach Labs somewhat ambiguous description here. Planet Scale's $15 per 10M rows written feels like something that could become bad. What if I define hundreds of indexes on the table? What if I'm inserting very large blob/text columns that are 50MB in size? Likewise, what if I index no columns and end up forcing a full table scan, but only 10 rows are returned? Do they consider that I "read" 10 rows or "read" all the rows in the table? If it's the former, I'm putting a lot of strain on their system without paying for it. If it's the latter, I'm going to just define indexes that might not be worth it if I were paying for the IO needed to do all that writing.
Still, it would be nice if Cockroach Labs offered some indication of what could be accomplished with 1 RU. "An indexed read of 1 row or an indexed read of a few rows in sequence; for example, 'SELECT * FROM people WHERE age > 18 ORDER BY age, name LIMIT 10' where there exists an index on (age, name)." That would let me know what to expect. "A write of a row under 4KB in size with no secondary indexes will cost 3 RUs; expect secondary indexes to increase the cost by 1 RU each" would give me an idea of what's going on.
I think there are definitely cases that one can't easily enumerate. For example, "UPDATE people SET age = 18 WHERE EXISTS (SELECT * FROM legacy_info WHERE people.id = legacy_info.people_id AND legacy_info.is_adult = true)". That's potentially going to require lots of stuff that's harder to predict. However, at this point I don't know if an indexed read of a row costs 1 RU or 10 RU. If an indexed read of a single row costs 1 RU, if I read 10 rows sequentially will that mean 10 RUs or will 1 RU have enough IO to cover that since they're sequential (or will the billing just over-charge since there's tangibly more rows and that's easy to explain)?
I think a decent amount of the value depends on the pricing and it's hard to judge that right now.
One thing I will note is that the storage seems expensive. It's slightly cheaper than Planet Scale, but a lot more than the $0.30/GB of Cloud Spanner or the $0.23/GB of FaunaDB or $0.25/GB of DynamoDB. I've been wondering a bit about the storage pricing of Planet Scale since $1.25/GB seems expensive. Cockroach Serverless is coming in at $1/GB which also seems expensive compared compared to alternatives. If Cloud Spanner is basically offering a third the price, is the "Serverless" flexibility worth it given that Spanner can be scaled up/down pretty easily in very granular increments.
Actually, one thing that could be useful might be noting how many request units per month one of the dedicated instances would have. A 2 vCPU CockroachDB instance costs $350-400. Would that be 4 billion request units per month (assuming you were fully utilizing the box)? Would it be more like 15 billion request units per month since you're presumably paying a premium for the Serverless flexibility?
Being open on how we solve hard problems is the way to build our collective knowledge as a developer community. Certainly CockroachDB itself has benefited enormously from all that has gone before and been published in the open.
We have developed just this[1] except using a MariaDB storage engine (MyTile) we've written. You can serverless run queries against TileDB arrays without spinning up a MariaDB instance. You can run any type of query MariaDB supports (joins, aggregates, CTE, etc). I've linked the basic documentation and an example notebook below[2]. You can run SQL queries from python/R or even JS or curl. We support a number of data return formats, i.e. arrow and JSON to facilitate use cases.
I'll also mention that we have a number of public example dataset[3] in TileDB cloud, such as the NYC taxi data used in this notebook[4], which you can explore!
[1] https://docs.tiledb.com/cloud/api-reference/serverless-sql
[2] https://cloud.tiledb.com/notebooks/details/TileDB-Inc/Quicks...
[3] https://cloud.tiledb.com/explore/arrays
[4] https://cloud.tiledb.com/notebooks/details/TileDB-Inc/tutori...
I have a database now with 40 Tb - this would cost $40k a month just to store!
I launched a test cluster, and the RUs are continuously increasing without me having connected to the cluster yet. At this rate of RU climb, the cluster would use over 8mil of the available 10mil RUs in a month without me touching it.
Coming from AWS, one of the most difficult aspects of using Aurora is guessing how many I/Os will be used for different workloads. It would be a shame to introduce this complexity for CockroachDB Serverless, especially if the RUs are impacted by internal cluster operations that aren't initiated by the user.
One thing that may not be clear - you get 10M RUs for free, up front, but you also get a constant accumulation of 100 RU/s for free throughout the month. That adds up to >250M free RUs per month. This ensures that your cluster is always accessible, and that you never truly "run out" of RUs - at most you get throttled to 100 RU/s.
I hear you on the difficulty of understanding how your queries map to RUs. SQL queries can be enormously complex and differ by multiple orders of magnitude from one another in terms of their compute cost. That's why we built a near real-time dashboard that shows you how quickly you're consuming RUs. You can run your workload for a few minutes and then check back on the dashboard to see how many RUs that workload consumed.
Nice use of K8s here and overall a great post! This is not related to CockroachDB but related to kube and cgroup. I am wondering if you guys have faced this infamous CPU throttling issue[0] when you guys were doing the metering and limiting.
First of all, CockroachDB Serverless is available on AWS, and should integrate quite well with that ecosystem, including with Serverless functions offered by AWS Lambda.
Here are a few advantages of CockroachDB Serverless that Aurora will struggle to match (note that we're still working on Serverless multi-region support):
1. Free-forever tier. We offer a generous "free forever" tier that doesn't end after a month or a year. As the blog post outlines, our architecture is custom-built to make this economical.
2. No ceiling on write scalability. Even non-Serverless Aurora runs into increasing trouble as the number of writes / second increases past what a single machine can handle. CockroachDB just keeps going. We've had multiple high-scale customers who hit Aurora limits and had to move over to Cockroach to support business growth.
3. True multi-region support. Aurora only allows read-only, stale replicas in other regions, while CRDB allows full ACID SQL transactions. If you want to move into other regions of the world and have latency concerns or GDPR concerns, CRDB is custom-built to make the full SQL experience possible.
4. No Cloud lock-in. Perhaps this is not a concern for you company, but many companies don't like getting completely locked in to a single Cloud provider. CockroachDB works on multiple cloud providers and doesn't have a monetary interest in locking you in to just one.
5. Online schema changes. CockroachDB supports operations like adding/removing columns, renaming tables, and adding constraints without any downtime. You can perform arbitrary schema changes without disturbing your running application workloads. SQL DDL "just works".
6. Cold start in an instant. CockroachDB clusters automatically "scale to zero" when they're not in use. When traffic arrives, they resume in a fraction of a second. Compare that to Aurora, where you need to either have a minimum compute reservation, or you need to endure multi-second cold starts.
7. Great support. We've got a friendly Slack room where you can get free support and rub shoulders with fellow CockroachDB users, as well as CockroachDB folks like myself. We also have 24/7 paid support for deeper problems you might encounter.
Taken altogether, CockroachDB can go wherever your business needs it to go, without all the constraints that traditional SQL databases usually have. Do you want thousands of clusters for testing/development/tiny apps at a reasonable cost? Could your business take off and need the scale that CRDB offers? Could your business need to expand into multiple geographic regions? Are some of your workloads erratic or periodic, but still should start up instantly when needed? It's not just about what you need now, but what you may need in the future. It makes sense to plan ahead and go with a database that has "got you covered" wherever you need to go.
What would be your company's reasons for wanting this available in self-hosted CRDB? What kinds of use cases would it address for you?
Or would users face connection limits at some upper bound until the old function connections get spun down?
[1] https://www.cockroachlabs.com/docs/v21.1/example-apps.html
[1]https://www.cockroachlabs.com/docs/cockroachcloud/quickstart...
Is the open source version viable for small hobby projects?
Now you get into the issues of tracing, request failures and retries, payload size (only json? really..) etc.
But I do agree that a super optimized sql for a a specific purpose has it's performance benefit
Our site is a bit out of date — we’ve got a new marketing site coming shortly, and we’ve gathered an all-star engineering team to take this to the next level. Stay tuned :)
(Or better yet if you want a unified data stack for your team, please get in touch… we can load data from 100+ sources into versioned Splitgraph images, or we can index and proxy to existing data sources with federated querying).
We have opted for YugabyteDB table spaces to solve the isolation problem. We allocate a number of dedicated tablet servers to a tenant so that their db objects are close to each other. YugabyteDB executes queries only on those tablet servers where the data relevant to the query is. A resource heavy tenant doesn’t have an impact on other tenants. It’s not really “serverless” but does seem to solve the problem at a cost.
Tenants receive complete databases with grant permissions so they can do as they please.
Because YugabyteDB uses Postgres 11 under the hood, we have an additional sanity layer in form of a Postgres process utility hook which prevents the tenant from escaping the sandbox.
There's no technical reason why CockroachDB Serverless couldn't support that scenario - it's more of a product/business/sales question. I think we'd be very open to talking with companies that have that kind of problem to solve.
Further, once you're experiencing the types of success that demands a superior solution the priority to invest in such a solution is clear.
It isn't "lock-in" - there's enough experience and capability in the market to solve those problems _once you have to_. Solutions like this let you decide whether the right time is at start-up or scale-up.
40Tb is a ton-o-data! Snowflake sounds like a good fit.
I like their product offering - I'd be interested in using them for hobby projects, but not for work atm.
I'm only half joking there.
MySQL and Postgres are massively more widely-used than Cockroach and Spanner, broadly very successfully. It's entirely feasible to run it with high uptime.
what does it mean ?
- When the bridge collapses I'll be managing someone else.
- When they find out this airplane has critical design flaws I'll be managing someone else.
- When this software I'm working on is hacked I'll be managing someone else.
I personally interview a lot of people and if they start proposing microservices or k8s or anything trendy like that (before having context), I consider it a negative point.
When hiring, I want someone to take a look at a busines problem, break it down into a smaller pieces. Most of the times, the most important engineering work is coming up with the right data models/data structures.
So yeah, maybe the architecture I have now wont scale up. But at least it'll get the business going. Years later when there are more resources, the software could be rewritten or whatever.
Also, you'r example mentioned "software being hacked". I never said undermine security. Security should always be taken seriously. Security is not a premature optimization. Scalability is.
There are plenty of situations where decent security is a premature optimization too. For example prototypes or proofs-of-concept that are only intended to demonstrate or benchmark a capability. The sort of thing where, even if you did insist on making it 'secure' the username/password would be admin/admin.
What you said all sounds reasonable. Sure, don't build something scalable if you've figured out you want something quick to prove your market and you're ok with tossing it all away or if you'll never need the scale. These are not the same thing. But this will be somebody else's problem? I don't want that guy on my team... Something someone told me a long time ago, write code like the guy that takes it over from you is a psychopath and has your address.
"Doesn't scale" doesn't imply "inadequate."
Overbuilding is waste.
When a bridge fails it'll just fail. When software runs into a scaling limit it'll degrade and fall on its arse constantly and be absolutely terrible as long as it takes for the software team to completely rework their entire architecture often having to learn completely new technologies.
The real risk you're incurring by deploying a prototype to production until traction is demonstrated and scaling becomes an issue is that deferring the design of the 'real' system risks succumbing to second-system syndrome and prematurely trying to make it a 'platform'.
Roughly, the first version just has to work, the second has to scale, and the third (assuming you get there that soon) is the one that starts needing to be refactored/generalized into various reusable subsystems.
It has to handle the projected load, with some safety margin.
"projected until when?" you ask?
Well, your projections will be wrong. And, your needs will change, which means that you're going to be changing the system anyway.
That's where judgment comes in.
Suppose that each customer is worth $10/month and that the "not scalable" version can handle 1M customers. That's at least $5M MRR when the system is at 50%.
If the scalable version takes significantly longer to develop, you might choose to put it off until you have more money/resources, especially since there will probably be other changes. (And, you don't know where the actual bottlenecks are.)
I don't know the relevant details about reddit but the assumption that the early reddit people could have easily built something more scalable yet there are tech reasons why the later people, with far more resources, can't.
As to the assumption that one knows the important bottlenecks 2-3 orders of magnitude in advance, that's just wrong.
"late answers are wrong answers" isn't just for real-time. That applies to products as well as signals.
Technical debt is not necessarily a bad thing.