Multiple Digital Ocean services down(status.digitalocean.com) |
Multiple Digital Ocean services down(status.digitalocean.com) |
dig b1.eu-central-1.storage.railway.app +short
s3.eu-central-1.wasabisys.com.
eu-central-1.wasabisys.com.
Object Storage is currently in Priority Boarding, our beta program. We can and will definitely do better, document it and add it to the subprocessor list. I'm really sorry about the current lack of it. There was another important project that I had to do between the beta release of buckets and now. I'm oncall this week, but will continue to bring Buckets to GA next week. So, just to give this context. There's no intentional malevolence or shadiness going on, it's simply because there's 1 engineer (me) working on it, and there's a lot of stuff to prioritize and do.
It's also super important to get user feedback as early as possible. That's why it's a beta release right now, and the beta release is a bit "rushed". The earlier I can get user feedback, the better the GA version will be.
On the "simply a wrapper for wasabi buckets" - yes, we're currently using wasabi under the hood. I can't add physical Object Storage within 3 weeks to all our server locations :D But that's something we'll work towards. I wouldn't say it's "simply" a wrapper, because we're adding substantial value when you use Buckets on Railway: automatic bucket creation for new environments, variable references, credentials as automatic variables, included in your usage limits and alerts, and so on.
I'll do right by you, and by all users.
These are all low-traffic projects so it's more cost effective to just throw on the smallest LB than spend the time setting it up myself.
> This incident affects: API, App Platform (Global), Load Balancers (Global), and Spaces (Global).
My DOs are working fine as well.
and they're doing just spectacular
maybe also GCP, hetzner, akamai
Did your high availability system survive this outage?
CDN is great for peak-load, latency reductions, and cost - but not all sites depend on it for scale 24/7
I imagine a cache layer cannot be that easy to spin up - otherwise why would they outsource it?
They wouldn't survive DO losing a DC, they're not so mission critical that it's worth the extra complexity to do that, and I don't recall DO losing a DC in the past 10 years or so.
They did stay up during this outage, which was apparently mostly concentrated on a different product called the 'global load balancer', which ironically is exactly the extra complexity I mentioned to in theory survive a DC outage.
Keep in mind these are "important" in the sense that they justify $100/mo on infra and monitoring, but not "life critical" in that an outage is gonna kill somebody or cost millions of bucks an hour. Once your traffic gets past a certain threshold, DO's costs don't scale that well and you're better off on a large distributed self-managed setup on Hetzner or buying into a stack like AWS.
To me their LB and DB products hit a real sweet spot -- better reliability than one box, and meaningfully less work than setting up a cluster with floating IP and heartbeats and all that for a very minimal price difference.
Common example: you can absolutely serve static content from an S3 bucket worldwide without using a CDN. It will usually scale OK under load. However, you're going to pay more for egress and give your customers a worse experience. Capacity isn't the problem you're engineering around.
For a site serving content at scale, a CDN is purpose-built to get content around the world efficiently. This is usually cheaper and faster than trying to do it yourself.
The thing is with edge capacity is that you massively overbuild on the basis that;
It's generally a long ISH lead time to scale capacity (days not minutes or hours).
Transit contracts are usually 12-60 months
Traffic is variable, not all ports cover all destinations
Redundancy
So if you are doing say 100Gbps 95%ile out of say London then you will probably have at least 6+ 100Gb ports, so you do have quite a bit of latent capacity if you just need it for a few hours.