DDoS Protection(docs.digitalocean.com) |
DDoS Protection(docs.digitalocean.com) |
I get that someone has to pay for this, but if I had 50 servers here and they all got shit on, I'd be on the hook for a lot of money through no fault of my own.
Then I thought, what a great way to tax the spammers / hackers that use DO!
I would gladly pay the $5 ddos fee to hammer the ips that keep trying to login to my wordpress sites.
just kidding, I would not actually do that.. but the thought is pleasant.
I've used DO many times and I am a fan btw - just looked back at their pricing page a couple days ago considering spinning up a droplet to self host a git thing.
Also didn’t see what their policy on tweaks are and or expectation on mitigating a more advanced attack.
I.e. DNS, NTP floods are low hanging fruit but it doesn’t take much nowadays to do something more custom.
> The Incident:
Beginning at 17:10 UTC, May 9th, multiple DigitalOcean customers experienced Droplet network outages due to an action on Droplets by an automated mechanism. This mechanism has been in place at DigitalOcean since 2019. It helps us ensure that any potentially compromised Droplet seen participating in an outbound Denial of Service attack is quickly taken offline. This is in place to assist in protecting all DO customers by ensuring we have a network focused on delivering legitimate traffic at speed and scale, unencumbered by illegitimate traffic. When triggered, this mechanism suspends networking capabilities on the Droplet or Droplet-based services temporarily to allow the owner to investigate the issue. Users are informed via a support ticket and email that details the paths to recovery. This incident was triggered by an unannounced data change made by a third-party, which DigitalOcean uses to assist in analyzing traffic flow and metrics, as well as detecting malicious traffic patterns.
Due to this mechanism constantly running and no changes being made directly by DigitalOcean, our teams were delayed in beginning an incident response. After multiple reports from customers that they believed the notification of outgoing Denial of Service attacks from their Droplets were false positives, an internal incident was declared to investigate the issue and start remediation efforts.
After a thorough investigation by the DigitalOcean Security and Networking teams, the root cause was discovered to be an erroneous change made by a third-party service that reports data on traffic. Contact was established with the third-party, and they confirmed a change had been made. Investigation began on their side, and they confirmed there was a bug causing bad data to be returned from their API.
Remediation of this incident was done through multiple paths. Complete resolution was achieved once the third-party rolled back the change that was made, which was causing bad data to be reported to DigitalOcean systems. Before that rollback was able to be put in place, DigitalOcean took direct action to take the automated mechanism that disables Droplet networking offline, given the suspected bad data. The support teams also worked throughout this incident to directly address customer tickets and re-enable networking on impacted Droplets.
Some people prefer all-in-one solutions, otherwise things like AWS wouldn't exists. Others prefer to spread out their eggs. One is not absolutely right and the other absolute wrong.
https://docs.aws.amazon.com/waf/latest/developerguide/ddos-s...
What happened to the DDoS Open Threat Signaling (RFC 8811) protocol? Do any of the many service providers, most of which include some sort of DDoS protection, use this system at all?
Do you only have to pay if you have been exposed to an attack?
You pay for it at a price of 20% of your plan's price. If you have a $25/mo plan, it'll be a $5 add-on. You can enable/disable it so that it's only on when you want it and it is pro-rated in that case. If you want to respond to an attack for 2 weeks of the month, it'll cost $2.50. So you can manually turn it on if you're under attack and only pay for that time, but if you want it to automatically handle an attack you have to pay for the month.
I think most people would just turn it on if they were a likely attack target. You don't want your site to go down and either get paged or find out about it hours later. If you're an unlikely attack target or a site that can stand some downtime, you could leave it off and just enable it when under attack.
It's like insurance. Do you buy it after an accident or before?
Otherwise, DDoS is usually volumetric: send you more packets than will fit on your network interface. The only prevention is larger interfaces, but 1Gbps of DDoS was readily available when I was dealing with it in 2018ish and it was pretty clear that people were using the site I ran as a test target for DDoS as a service (always exactly 90 seconds of junk traffic on our www, very rarely a real service host, etc). There was a recent crackdown on DDoS as a service, but I'd be surprised if 10Gbps isn't easily available now.
If you’re hosting with a provider, your maximum factor will be how much your provider will “tank” for you.
Otherwise harden your ports, drop anything via IPTables, turn on NOTRACK. Better but more advanced would be to use tc (traffic control) to drop bad packets before they enter the net filter lifecycle
Not that I advocate for ddos - I think it's a terrible thing, and I've been through it a few times.
DO and OVH are my most blocked ip blocks on several servers. I also get some hetzner and aws and microsoft blocks a lot among others sure.
Interesting that I just launched a brand new WP on a brand new domain, and in less than 24 hours half of the hack attempts are from DO ips.
You could lecture me about reporting and blah blah. I've been down those roads spending literal months doing that. With DO's cheap boxes and rotating IPs it's not worth it, I just block the entire CIDR every time, today it's 157.245.0.0/16 and 174.138.0.0/17
If DO was serious about stopping these abuses they would offer a WP plugin or opt-in setting that could check data from wordfence and similar and easily see which of their boxes are being used to hack into sites, all this could be automatic, without the form filling and delays that are required.
If it is an unimportant service you just suffer the DDoS or switch IPs.
Or you use a front end on a VPS that does have DDoS and use a IPv6 tunnel or tail scale to connect to your actual service.
I proxy through Fly.io or AWS Global Accelerator when I can’t use Cloudflare for TCP as a hobbyist.
(Why only three protocols and such strict limits, years after launching? You won’t cannibalize Spectrum by allowing more hobbyists to use it, you’ll market it for use at scale - like Argo Tunnel’s trycloudflare)
You don't have to be a big company to own a block. An ipv4 block is <$10k and the price is on a downtrend now that covid is over. It is risky as a site if you don't own your IP block because false abuse complaints can be sent to the ASN you are renting an IP from and can result in downtime. If you own the IP, the abuse reports for it go to you to handle.
>won't just drop non-Cloudflare traffic
Not all sites do this. Also they likely have other stuff running on the machines that are accessible without cloudflare.
Can you elaborate?