Cloudflare took down our website(robindev.substack.com) |
Cloudflare took down our website(robindev.substack.com) |
Also this sounds like an online gambling site of questionable legality, knowingly serving customers in jurisdictions where it's illegal, so I can't say I have too much sympathy, and I feel like Cloudflare effectively fired them as a customer when they realized what they were up to.
In defense of Cloudflare, the sys ops engineer should have understood the situation and knew they were misusing Cloudflares services. They decided to play hard ball by bringing up the fact they were thinking of leaving. And we have no history of the multiple phone calls they had with Cloudflare.
Combine it with the stories I hear about Sales, the numerous other PR fumbles already mentioned in this thread, and the months I’ve personally waited (while on a paid plan!) for ticket responses only to get cookie cutter responses is, quite frankly, embarrassing.
CloudFlare puts in a good front, and their products seem decent, but they really have questionable business practices that should make anyone think twice before using them.
(Amount owed by customer at end of month times the probability of on time full payment) minus Cost of providing service to customer for one month = profit
Since this is an online casino, could the risk of late/under/no payment be quite high?
But the casino still decided to stretch the penny and alas, whoever at Cloudflare was in charge got quite upset their extortion-tactic failed. So they decided to resolve it the American way and kick them out with zero warning - ouch! How fascinating.
I myself like using Cloudflare as it's quite affordable to setup and use. Makes me sad to know they have to resolve to tactics like this to finance their service. Well, at least I don't work in dubious businesses that violate TOS so perhaps I can at least wish for a graceful termination when my Enterprise bill is due.
Better yet, configure CloudFlare through terraform, so all your config exists in your own repo all the time. It also helps day to day since it's not that hard to accidentally flip some switch in the dashboard.
But yeah, do research alternatives. CF has too much power already and will either ignore issues, destroy you, or pay lawyers to protect people trying to get you murdered, depending on their mood. There are better options.
> captc
I'm 80% confident HN tried to hide this link. It's the fastest downhill I've noticed on here, and I've been lurking and commenting for longer than 10 years.
Does HN has stake in cloudflare?!
more than a coincidence
Sorry to be “that guy,” but, you’re serving 4 million people at a casino and paying $250 a month for shared multi tenant infra, and you’re SURPRISED you have problems? Really?
To be honest, I’m glad these sorts of businesses get kicked off Cloudflare because it causes problems for others sharing the same IP space and infra. I’ll let someone else with experience discuss how many times a day the network would see a hacking or DDoS attempt against the online casino, which is by far the favorite target of hackers. But in general, I just don’t want any of my infra touching the same stuff as these guys.
Like another person here, I am assuming that Cloudflare ops told someone “tell these guys to get their own IPs and upgrade,” and then the message went to Cloudflare’s (utterly lousy!!!) sales people to try to fix before shutdown, and then it all turned into the mess we see here.
The true moral of the story, I think, is, if you’re running an online casino on a shoestring budget, expect bad things to happen to you. Of all kinds.
I’m guessing they aren’t doing that well and are looking for revenue to cover the holes.
If OP’s business was in fact illegal, CF should have stated it. Now it seems CF is an evil sales driving monster. A monster that grew so big it thinks it can do whatever it likes.
The sad part is that, assuming OP is not leaving out critical parts, multiple people play parts of this evil machine. I’ve seen how sales people think. But this is next level toxic culture. The second customer threats of leaving for the competition, they freak out and pull a bigger lever to destroy them. And the fact that a company allows this to happen…
I would never do business with CF. Good thing i don’t right now. Cause i will definitely take it elsewhere.
> This could arguably be seen as a violation of the Cloudflare TOS, as they wrote above.
And the very next paragraph begins with:
> In any case, we receive >95% of our traffic through the main domain that’s been unchanged since our founding, and were happy to resolve this issue in whatever way...
And then they complain about paying up?
The only issue I see here is around the aggressiveness on the CF side. But, I was not in those meetings and the way above reads tells me that I might have been slightly mad so perhaps the CF was just taking it out on them?
Anyway. I don't think this is a CF foul.
So they did a good thing taking it down, no?? Addiction as a businessmodel is not that cool
Sounds like a pretty abhorrent website
Before that CF was high and mighty on the "free speech" horse.
There's an old spat between CloudFlare and Malwarebytes where MB was threatening to block all of CloudFlare because they wouldn't remove literal malware. The argument being that running a reverse proxy "isn't hosting", and should be treated differently, even though to literally anyone else there's no difference between an origin server and a proxy.
CloudFlare is just sketchy as all get out, IMO.
Putting my biases on the table: I think CloudFlare shouldn't have hosted Kiwi Farms in it's current state, because I don't think hosting dox should be legal. A website that hosts dox is not engaging in speech, it is engaging in censorship. Hell, in the EU, it's already illegal to host dox, the US just needs to pass a privacy law comparable to that of the GDPR. Kiwi Farms is legal in the US purely for the same reason why the CIA/FBI/NSA can legally buy advertising data from Google and Facebook.
However, any reasonably competent person can see that recentralization of the Internet is a Bad Thing™, and that this is precisely what Cloudflare wants.
Likewise, we know that aggregating our data through a for-profit company that's based in the United States means that collected data is reasonably in the hands of the NSA, which makes their DNS-over-HTTPS scheming suspect.
Just like what happened with the company in this post, we have plenty examples of them abusing their position to extract money from both legitimate companies, like this one which is aware of their legal obligations in various countries, and scammers and spammers alike, who Cloudflare are more than happy to host indefinitely in the name of "free speech".
Their lack of clear communication, their broken abuse reporting, their continued claims that they don't "host" all show them to be antagonistic towards anyone negatively impacted by their facilitation of illegal activity.
Cloudflare is an evil company that just happens to be better (but not great) at hiding it than other evil companies.
Bait-and-switch seemed to be the most common pattern, plus crazy-high prices once you’re on the “switch” side of things.
But their sales team was so uniquely uninterested in our business that I never had to find out first hand.
It's an inevitable outcome, as long as there is nothing done against the big threat actors: government-run APTs from China, Russia, North Korea and Iran, government-tolerated scammers (India, Turkey), rogue actors in our governments' security services (e.g. Pegasus), ordinary criminals mass-hacking vulnerable devices and selling access to them to be abused for DDoS'ing for less than the cost of a coffee at Starbucks... it's a wild west, and people are hiding themselves behind the largest giants they can find: Cloudflare, Akamai, AWS, Azure and GCP.
Saying it's inevitable makes you seem like a Cloudflare apologist, which unfortunately we see way too often here on ycombinator. Has anyone refuted my suggestion that Cloudflare is knowingly evil? No. Have they downvoted because my information is incorrect? Also, no, or if they think I'm incorrect, they haven't bothered pointing out how.
People want to like the things they choose, and this, unfortunately, is where Cloudflare is cleverer than other large, evil companies.
But they keep doing it. Just not KiwiFarms but other websites. I've reported it to them, they claim they are only a proxy (that's not true, they are also the registrar and DNS). Nothing was done.
Fundamentally, the OP might be involved in something scummy or at least against Cloudflare's TOS. But if that's the case, if you have a customer who is violating your TOS, you don't hit them up and say "pay me an extra $119k a year and I'll look the other way". You say "here are your violations, fix them and prove to me that you fixed them, or pay for plan X which has terms under which they are not violations."
The way Cloudflare handled this is completely inappropriate and even if it wasn't their intention, makes it seem very strongly like extortion. Two wrongs don't make a right, and OP's business being possibly shady does not give Cloudflare license to extort them.
Trying enforce "reasonable behavior" by suing is a massive money pit that might yield nothing at all.
Scammer does recon on victim. Notices they use CF. Use high pressure sales tactic to get them to pay a hefty sum up front or else lose access.
But as you read on, I see company did their own DD and followed up directly with CF executives and teams. Confirmed account is locked at CF.
In this case, CF is acting scammy.
I wonder if they are having liquidity issues thus the push for high pressure sales tactics and blackmail.
Pretty standard behavoir from both sides with room for improvement. They should have been more clear about what's going on and you should have been more insightful what they wanted.
In my opinion they acted to fast and not really cooperative, but if you wouldn't have declined the initial offer and started to figure out why they offer it and what options there are, you would have came out with a better deal than 10k/month and likly without 1 year upfront payment which would have given you the time needed to transition to another service.
455 points, 3hrs old, but on the 2nd page of HN. What's up with the algo?
I guess it's due to general negative sentiment towards casinos, which may be understandable but doesn't (in my biased opinion) change anything about CF's behaviour in this post. I would have left it out but it's necessary in order to provide the full context.
* Cloudflare operates at a scale where its caching saves a lot of bandwidth, which saves ISPs money, which makes Cloudflare an attractive partner for peering and co-location.
* CDN is a platform on top of which Cloudflare can offer a lot of additional services that used to be expensive dedicated middleboxes.
[1] Cloudflare decrypts your traffic, reads it, and then forwards it. They see all encrypted data going to and from your website, in plaintext.
1. Its quite possible thar CF having this site on some multi-tenant infrastructure could be threatening. Not unreasonable at least to ask them to have their own IP block.
2. If thats the issue then a clear explanation should have been provided. Routing to sales is inexcusable. Someone isnt being transparent.
3. If it’s a pure cost / revenue issue then say that, set a deadline and negotiate. This is bad karma and even though CF is clearly the market leader, what they do isnt rocket surgery. Not worth it.
- a companies ops team identifies a tenant that is too heavy/burdensome for multi-tenant infra and is causing issues. These issues can cost a serious amount of money if you factor in dev/ops times to resolve, other customers impacted, etc. Certainly more than what a hypothetical single multi-tenant customer could be paying
- they escalated internally and need the tenant moved to enterprise asap to resolve
- the only reason the tenant was on multi was because sales sold them the wrong thing, so now it's on sales to explain how to fix this
- improper handling internally results in this landing only on sales, with no backup, and with their task being to get them to take enterprise
- when the customer refuses enterprise they go "we've tried nothing and we're all out of ideas"
Again, this is totally speculation and I'd hope CF has more mature practices than this but this is a scenario I've seen before in much smaller orgs.
Given that the article is an online casino that seems to be using potentially ToS violating domain rotation, and that they pay so little per month for apparently millions of users, I for one will not form an opinion on CF based on this article before CF has a chance to defend itself.
$120k will never be enough, price hike is incoming for renewal.
One Question: For the Web site for my startup, I have the ASP.NET code running so ASAP will be getting into to a business account with my ISP, IPs, domain names, DNS, etc., at least for the Alpha Test.
So far, my intention is to host my own Web server. I've heard of CloudFlare, how they can help stop DDOS attacks, etc. but so far have hope not to use them.
Question: How realistic is it for me just to host my own Web server and, e.g., avoid any chances of problems with CloudFlare, the Cloud, VPNs, etc.?
Thanks!
Imagine getting banned by cloudflare or some other cloud provider....
"Small SaaS banned by Cloudflare after 4 years of being paying customer"
https://news.ycombinator.com/item?id=34639212
Also:
> I'm a SysOps engineer at a fairly large online casino
Oh no, a casino losing the trust of its customers? Those places are normally so scrupulous!
Just FYI some countries ban casino domains/ips that are not licensed to operate even when its "just a landing page that says sorry not available"
I hope that’s not the case, because that would allow for bad behavior by reps trying to manufacture end-of-quarter sales.
EDIT: why the down votes?
Holy shit.
In that light you can see why Sales would be sent as the messenger. But I agree they shouldn't have been involved. Sending a T&S representative would have been better. But then again it looks like from screenshot #2 that they kind of did that? They just didn't have a direct call with T&S.
Of course, it is easy to identify the IP addresses of the well-known VPNs, so it's not rocket science, but it does mean that popular VPNs will no-longer give you out-of-region access.
And for some reason Cloudflare's the bad guy.
In fact, the company vision and values (as the team grew) may have changed over the time, but originally it seems it was somewhat of a different spirit (and closer to a data collection network).
OP runs a casino/gambling site. Gambling is a regulatory mess (I have spent far too long dealing with this as an RNG supplier), and so it's very hard to comply with every jurisdiction, and each one needs you to prove compliance to operate in that jurisdiction.* Gaming companies spend a lot on compliance and tracking, but since the internet is the internet, it's pretty hard to enforce perfectly, so some countries and ISPs take this into their own hands.
Due to that, IPs hosting gambling and gaming sites often get regionally blocked by internet providers or otherwise flagged as hosting illegal content. Those regional blocks consequently affect the reputation score of the IP, and if you are a traffic aggregator like Cloudflare, can cause other customers to have issues. One of the most aggressive and annoying regulatory environments for gambling companies is the US, so it's very possible Cloudflare has had some trouble due to gambling use of their IPs in states in the USA.
Cloudflare wanted them to use the BYOIP features of the enterprise plan, and did not want them on Cloudflare's IPs. The solution was to aggressively sell the Enterprise plan, and in a stunning failure of corporate communication, not tell the customer what the problem was at all. The message from Cloudflare should have been "Enterprise plan + BYOIP or ban, and maybe we'll work with you on price" but it was instead "you would really like the Enterprise plan."
*As an aside - we're lucky in that respect being a tech supplier with relatively uniform rules, but our customers (the gaming companies) get the short end of the stick here.
FYI, we also fully block users from the US (due to regulations).
My problem here is mainly the unprofessional communication and huge mess of mixing "compliance" with sales, without giving any clear information or options. And then the removal of our account without warning while we were still talking to them.
Generally, low-reputation IP addresses are associated with scams, spammers, and other similar things. Gaming somehow gets lumped into this bucket in some jurisdictions, but that hurts you worldwide (similar with other "sin businesses" like porn). These blacklists get published (I think there's some parts of BGP that make this happen, but I'm not quite sure what the mechanism is), and being on any one of them hurts your traffic everywhere because it becomes suspect.
I agree with you that this mix of compliance, engineering, and sales is gross. If this was the issue, they should have just told the OP.
The most (in)famous case is China's GFW which banes IPs all the time. Yes, other websites often get accidentally blocked, but they don't care. Moreover, you can't even communicate with them because there are no official legal regulations. This is something what any CDN or cloud providers have to deal all the time.
A demand for 120k upfront or else bad stuff happens is by no reasonable definition "selling", aggressive or otherwise
BYOIP isn’t just expensive—if your content is bad for IP reputation the time-to-flagging of your IPs is going to be way shorter on BYOIP than on shared IPs due to there being less dilution. And that’s without getting into the challenges around rotating/renting IPs on a continual basis.
I do agree that CF did not communicate that well or professionally—if the sales emails are the only comms that happened here.
Many gambling companies are fine just doing BYOIP or running dedicated hosting infrastructure that is on providers who are explicitly running hosting for that industry (although they are moving to cloud). There is a good reason this separate infrastructure exists. In general, I would not assume they are rotating IPs: this is not a scam, it's a business, and they are largely fine with being blocked in places where they can't legally operate.
Is there much of a market for that? I thought random.org had it all sewn up.
I think we are only one of ~3 TRNG suppliers who have been audited. Many games don't use a TRNG, though.
Since it uses atmospheric noise, you can also influence the numbers from random.org by transmitting radio waves in the area nearby - the operator of random.org has mentioned that there's so much RF activity that he is concerned about whether the bits are still random. A final issue is that they are also so low-volume that they probably can't get enough test data for the required audits (which can be a lot of data).
To underscore the volume question: Random.org used to have a running count of bits generated. The counter wasn't monotonic (before it broke in ~2015-2019), but the peak value I saw when I checked archive.org was about 250 GiB total since 1998 (that was in 2015). That is one quarter of the size of our "light" qualification test ("heavy" is 16 TiB). The RNG auditors also take O(100) megabytes for each audit, which would be a significant fraction of random.org's output.
I don't think it's a huge market, but state-run lotteries around the world need good random number generators for games without physical balls (like Keno for example).
I've talked with people that have created RNGs (rather than buying off-the-shelf solutions) and it sounds like soul-crushing work - mostly due to dealing with the government regulators that need to give final approval before the RNG can actually be put into production.
There are seals on the hardware, any modification must be approved, you must certify that the payout is the expected one, ...
That is just a story they have made up. They don't know why Cloudflare shut their account down. I reckon the Fastly "reason" is likey a red herring.
One thing I've learned to be wary of on the job is "do you need help?" That phrase is often code for "You are not performing up to our expectations. This is your first and only warning. Get in shape or get out."
You start using the service and don't pay a lot, so you make plans around a certain level of expenses. Then out of the blue you receive an "urgent" email from a sales representative and suddenly you have to go from $20 or $250 to $thousands right away.
Obviously it's not in CF's interest to keep a customer that doesn't pay enough, but dropping a "bomb" on the customer and make them feel like they're about to be kicked out from the service makes the customer lose trust on CF.
CF can probably match Fastly's price. If they had acted differently in this and other similar situations, they could keep the customer, be paid more, trust wouldn't be affected, and there would be no bad PR here.
Since the CF management that posts on HN usually say this is not supposed to happen, perhaps someone needs to sit down and look at the incentives sales reps have? Even if you don't care about the customers, this is affecting the CF brand a lot.
Why on earth any company would jump from $250 to $10k per month unless they had a gun to their heads? Even if their revenue is to the millions/billions (which most likely is considering the nature of their business). They work for their own profit, not Cloudflare's.
We only have one side of the story here, but it's not the first time I've seen posts/comments about these emails from Cloudflare and the messy communication that follows. As a business customer, I really hope I don't have to deal with any of this.
You get what you pay for.
The offending parsers were rewritten in Rust (https://github.com/cloudflare/lol-html), as well as WAF, image optimization, and a few others. Nginx is being replaced with a custom cache server.
New implementations are using either the Workers platform, or are written in Rust or Golang.
There is just no reason they would suspect that they were going to lose the deal to Fastly at this moment. They were very much the default winner.
Extortion or not, I just can't fathom that they ragequit the deal at this moment, because they were about to win it.
It therefore seems likely that after looking into it they disqualified it as a business category which is against their TOS or whatever.
Or that the enforcement and sales teams have very similar, overlapping triggers for engagement, etc.
Cloudflare's behaviour here was shitty and this is not the only report. By all means their reputation is very generous free tier and a horrible experience in paying.
BUT seriously who ragequits a winning deal? Another comment summed this up - the attention caused them to take a look and realize they don't support shady-ish casinos, possibly (seeming to) evade US legislation, etc.
The first sales email is from a Cloudflare with “Gaming Division” in their email signature, so they were clearly aware of the nature of the customer’s business. Moreover, it seems they have an entire department dedicated to serving the gaming market.
How is that something worth going viral over? Salespeople get fired all the time for not meeting their sales goals. Engineers similarly get fired all the time for not meeting their productivity goals. If you don't do your job well, don't expect to keep it.
And if I recall correctly, in this particular case, she was a green employee who hadn't even made a single sale yet! What more obvious of a layoff target is there than that? Would you keep a green unproven salesperson over a proven veteran salesperson who's landed 9 figures in sales?
Especially in a world where people pick up their whole lives and relocate for jobs. Recent joiners aren't getting any sustainable kind of severance either. The idea is if you're hiring them you have a minimum commitment to support their success.
Yes she was an obvious fire, but it's also the organization's fault. Enterprise deals also take way longer to close than that...
All that said, salespeople can and do move jobs a lot. I'm sure she'll be fine.
Asked for a little time, they said fine and we moved much of the bandwidth usage to a couple of dedicated servers on OVH I think.
Never heard from them after that.
We are doing about 3TB
2-Your business is probably very profitable, and $300 a month is very cheap compared to the potential hassles they could face working with such a business.
3-I find it very inappropriate to dox business representatives and show names when you have carefully hidden any information regarding yourself and haven't even disclosed your company name.
After all they can choose with whom they want to do business. They gauged what price they could ask you, factoring in how profitable your business is and how noisy and painful it might be to work with you. It sucks but this is the downside of SaaS/PaaS.
As far as I can see, the author was careful to redact their domain from all screenshots.
Attorneys love it when people put everything in writing like this.
If a country A decides to block twitter.com but forgets to ban x.com which remains available ... is Twitter engaging in illegality / violation of CDN terms of service?
Couple this with the fact you have a new rep every 6 months and you get some pretty annoying nag service for the entire duration of your contract.
I have no idea why Cloudflare would ask you to use these two features. SSL for SaaS is only useful if you want to add domains and certificates via API.
I have had my fair share of negative experience with Cloudflare but this is next level bad. Unfortunately companies can chose who they want to do business with but it shouldn't be like this.
The Google Cloud situation and all these little happenings, including the proliferation of Gen AI into everything, make me long for the days when companies had their mainframes onsite, in closets or separate rooms, away from CDNs and cloud networks. It seems like a better idea to use these cloud networks as a separate off-site backup rather than for primary use.
I’d love to learn more about what will happen next in this saga. I’ve seen a post where a Cloudflare exec has posted here on HN before. They probably won’t say anything for legal reasons, but what repercussions can Cloudflare expect for this? Will they be, or can they be, sued for this downtime and the related expenses?
When big cloud goes down, you get a few days of credit. That's it.
What's the Google Cloud situation?
[1] https://news.ycombinator.com/item?id=40313171
CF was more than happy to help, but .25k wasn’t a number that solved all the problems.
https://igamingbusiness.com/legal-compliance/legal/cloudflar...
"We'd like to talk to you about an enterprise plan."
"No thanks, I'm fine with the free plan."
"Based on your traffic, we'd like to talk to you about an enteprrise plan."
"Is there a traffic limit on the free plan?"
"No, there is no limit. But based on your traffic, we require you to get an enteprirse plan."
[Gives up and gets an enterprise plan]
[6 months later]
"Based on your traffic, we'd like to talk to you about up'ing your enteprise plan to a new monthly pay."
"Is there a cap to traffic in our current plan? I don't see that in our terms."
"No, there is no cap to traffic in cloudflare plans, but based on your traffic, we're going to require you to pay more per month than you are currently paying."
"OK, can you tell me the traffic limit in our current or new plan? So I know what I'm paying for and when I'm approaching it?"
"No. But you need to pay more."
[Wash, rinse, repeat, every 6-12 months]
It seems like while cloudflare technically does not charge for egress, in fact for large egress it's just a game of chicken between the customer and a salesperson every 6-12 months, with the salesperson trying to figure out the most they can manage to get without losing the customer? I mean, I guess that is standard for enteprise sales, but I think usually you at least have some terms to know what you've got for how long without a renegotiation?
Shocking how often "gatekeepers" fall to the temptation.
10TB/80TB at 120k/yr, either way, Cloudflare is taking you for a ride.
If you aren't self hosting, you're really doing it wrong.
It's not like you can have your domain/DNS somewhere else and point to Cloudflare IPs (to not put DNS and CDN in same basket). Cloudflare does not allow that setup.
You can't protect your website from your DNS provider or hosting provider suddenly kicking you off. You are going to be offline for a couple of days.
Sure you can. Colocate in two or three places. You're your own DNS provider and your own hosting provider. If one of your colocation companies doesn't like what you're doing for whatever reason, you use the other two until you replace that provider.
Yet everyone in "ai ai ai ai" is buying up Nvidia/CUDA like there is no tomorrow and then pooping on AMD for even trying to do anything.
History loves to repeat itself.
Secondly I'm a little confused why they would require you to pay a year upfront? I would like to hear from cloudflare as to why they required this? It's pretty fair for them to ask you to pay a year in advance because of the risk that you carry as a gambling company.
Cloudflare needed you to have to enterprise plan to remove liability from them. It's not even a big request, they have specific pricing plans for a reason.
They know this
It was at the top of HN, then quickly buried to #20-#30. It is now at #27, being a hour old with 318 points.
Caching, detecting+modifying headers, routing traffic, ...
Also; if not registering domains on CF does anyone else do at-cost or otherwise super cheap pricing?
Actually had a sales call with Cloudflare in the last month and I got some bad vibes from the whole experience. We did not end up going with them.
Such as:
- Unmetered DDoS protection (i.e. no absurd base fee for it existing)
- Unmetered rate limiting (protection against cost attacks on the next)
- Reasonably priced object storage (i.e. not more expensive than numbers listed here https://blog.cloudflare.com/aws-egregious-egress)Very typically free = actually "fair use".
Where it gets murky is when this becomes a shotgun sales tactic.
I'll be they are now paying Fastly a lot closer to $10k/month than $250/month
What seems interesting to me is just what the loophole is and how many other business are also on the radar for this drastic pricing change. Are there other goodwill discounts Cloudflare is ready to start collecting on, or does the gambling site represent a unique situation?
And let's be honest, if a big provider wants to offer cached versions of pages, they probably should have a way to purge those files in case there's a problem (eg: malware).
You're putting words into my mouth.
Adding Challenges, TLS fingerprinting and Rate Limiting is possible on just about every major CDN platform to be honest. I guess with CF it's more "ootb" though, where you don't really have to think too much about policies - but at the same time, you can't go as granular in those policies (e.g layered) as some others.
Them very aggressively highlighting the BYO IP feature and then even suggesting third parties to rent IPs from strikes me as a significant detour from their normal “script” (having dealt with their AU sales team before).
> We do have multiple domains that mostly act as mirrors to our main domain. We have these for a few reasons. One is that since we are a casino, we have different regulatory requirements we need to comply with in many countries.
Evasion:
> Another is that we use them to target different global user groups and affiliates and track conversions long-term. This also means that if a country DNS-blocks our main domain, a secondary domain may still be available.
This is more like one gang hitting up another for "protection" payments. I had to laugh when they called it "Trust & Safety".
It's impossible to say what's going on since it's an anonymous post with no details.
Maybe it's all 100% true.
Maybe there are some key details being left out. Wouldn't be the first time I've seen one of those outrage posts that seriously misrepresented things.
Whatever the case, obviously the author is not an unbiased party. These posts do well because "zomg Cloudflare bad!", and maybe they are, but I sure as fuck don't trust some casino guy either.
That being said, I doubt that's the core issue in this case.
If you think something your client wants could explode into a liability, you can turn them away or you can just make sure their bill covers your exposure.
If it's a legally questionable service, there's likely to be plenty of abuse contact, or they're going to be a big target of crime, they're going to end up paying more. This is the same reason why some industries (eg porn sites) have always paid more for card processing.
Their business was causing IP reputation damage and all plans but the enterprise BYOIP plans share the same IP pool.
Essentially it was "use your own IP pool and pay us for the cost of maintaining that pool for you or GTFO".
This wasn't just a normal sales rep hitting them up. This was trust and safety (i.e. the moderation team) coming to them with a compromise that would allow them to stay on the platform. They chose against that and were dragging their feet.
The timeline of the article also really makes this clear. This wasn't over the course of 24 hours. This started a full 4 weeks prior with sustained back and forth. They only included a few images of emails from the discussions but the article makes clear that there was more discussion happening.
And to quote the article. After receiving the ultimatum, they got an entire extra week to deliberate.
> We managed to buy a week of time by letting it escalate to our CEO and CTO and having them talk directly with Cloudflare.
Then finally when they told CF that they were just buying time while looking to move elsewhere, CF dropped their act of goodwill and the moderation team resumed the moderation action they would have taken in the first place had this been a smaller account.
----
So yeah it sounds bad from the snippets but this was basically "hey you are a big customer and you are breaking rules we would normally ban anyone else for but if you can compensate us we'll spend the labor hours and infra to let you keep operating in your own little quarantine box.". So this really should be seen as an act of goodwill rather than malice.
Risk can be mitigated, especially if you take care to know what the risk is, but risk mitigation and the salaries of the risk mitigation teams are not free.
The answer of "no, we will not host you unless you pay us enough money to hire people to make sure we're not breaking laws by hosting you" makes plenty of sense, and an online casino that is likely dubiously legal in many countries is definitely a place where you might use that answer.
I'd also expect there are cases where Cloudflare enter into enterprise agreements with customers, get a good hard look at exactly what's happening, and then tear up the agreement and walk away.
If they can indeed stop providing services to a casino, why cannot they shutdown a website spreading pro-war propaganda, or a website selling illegal services ?
It means they are making editorial choices, instead of just being the technological provider and being a neutral "internet pipe".
Not sure it's really in their best interest to self-police in the end, as they could lose their DMCA safe harbor provision ?
Because their main network all uses one big IP address pool and the blocks by various regions/countries against their site were probably not just DNS blocks but also IP address blocks.
So they now have an account whose activity is getting their IPs banned in countries where they operate.
So they told the account owners they needed to pay for an enterprise account and a dedicated IP address pool maintained by cloudflare. That's why CF kept talking about BYOIP in the emails.
i.e. "Pay for us to build you a quarantine with your own IP pool or leave ASAP"
This.
That said, we're seeing this across so many platforms, from datacenters to social network sites.
They blew their safe harbor provisions years ago and yet remain untouched despite this.
It smells like the "problem" was detected by automation, but instead of being able to reach anyone technical to work through it, you can only call sales teams.
In my opinion it's one racket vs another.
Well HN is the unofficially official Cloudflare Support forum. I think we will hear from them soon. From past experience normally their response time for anything Cloudflare on HN is within 2-3 hours.
Edit: "How do you know?" -- I don't know it's actually what happened, but when switching to enterprise, you don't go from 10% margin to 98% margin. The added costs actually represent added budget for the provider to deal with your "special case". ALL enterprise pricing tiers are disguised consulting contracts.
i.e. it's a $10k fee for maintaining the infrastructure for a quarantine around their services
Except the "place" isn't Mom and Pop's bodega, it's a casino dodging countries blocking its main domain.
Looks like they COMPLY with regulatory interest, to me.
"In order to comply with tax regulations and donor laws, we had to structure our activities in order to make it possible for political donations to be classified as regular consulting income".
Let's not try to find reasons to harm the messenger and stick to the facts -- a paying customer was suddenly extorted for hundreds of thousand of dollars out of nowhere.
My 2c: It’s scummy that CF did this. It looks like they were disingenuous about the severity of the violations and used it as an excuse to get more $$$ from an already paying customer to make the manufactured problem go away.
My guess: Their account fell out of the non-enterprise TOS for some reason which is being obscured in the post (probably domain rotation related). Their T&S team proposed moving to enterprise for a custom resolution. OP's company refused, their account was purged because they had gotten several warnings about it.
I'm sure this sounds frustrating to the average HN dev who runs a legitimate startup with cloudflare on top and is now biting their nails worried to death about what will happen to them. But "online casino" immediately raised a million alarm bells in the post.
Since we already left Cloudflare the only reason I finished writing this article is to warn others. I think it's still relevant to many companies regardless of what you think of casinos, since very unprofessional sales tactics (unprofessional as in business threatening) seem common place with them. Do look at the linked other posts and comments here from other people affected that don't have anything to do with casinos.
I'm happy to answer questions as well.
It doesn't seem so, so there is at least a valid reason for Cloudflare to keep them as a customer as they're not violating the laws where they have their business in.
The 'customer support of last resort' genre is common and not usually a good fit for HN [1]. If people feel this story is unusually relevant and interesting, I'm not sure I agree—long experience has taught us that one-sided articles like this nearly always leave out critical information—but I also don't mind yielding in an occasional specific case, so I've rolled back the penalties on this thread.
The issue from our point of view is not about story X or company Y—it's a systemic one: the most popular genres of submission (especially the rage-inducing ones) get massively over-represented by default, so countervailing mechanisms are needed [2] if we're to have a space for the more intellectually curious stories that the site is meant for.
[1] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
[2] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
I do admit that I originally drafted this article as a "customer support of last resort", since that seems to work well for CF specifically. But it's too late for that anyways by now - the problem is "resolved" by fire and we don't plan to move back.
I purely posted it now as a precautionary tale for other people because of all the pain it has caused us. So the audience is tech people in most companies of small size that will hit more traffic at some point in the future.
They're already off Cloudflare, I would see this story more as "Dealing with tech company X is a business risk" cautionary tale.
Moderator action seems unlikely because it’s still on the second page.
If a thread is too interesting, it gets penalized, can't have too many people commenting on an exciting topic
(3) Mh, I don't think this is doxxing and didn't expect having names would be a big problem. I've just updated the screenshots anyways and censored the names of the representatives.
Cloudflare of course chooses who they want to do business with, but they also pride themselves in being neutral.
OP is lucky CloudFlare even gave them 24 hours. I’m not going to dig through the their TOS or anything but I’m going to guess that you need to have an Enterprise contract to be a business of certain categories like banks/crypto, pornography, and gambling, which explains why they were being connected with a sales team.
OP mentions lost customer trust…but Cloudflare doesn’t want or need OP to trust them. $250 a month isn’t enough to deal with a business like that.
I did quickly search the TOS for the word "gambling" and did not find it.
What the hell? That's way more than AWS costs, 90% of which would be egress fees. And cloudflare has done a lot of marketing to rightfully call out those egress fees as far too high.
Even on the enterprise plan they don't really start to talk to you about traffic until you are like 3x over your contracted traffic for a couple of months.
It sucks, it feels like they are competing against themselves because they don't have clear pricing or limits.
That can’t be right. I’ve hit 10+ TB within a few weeks on free tier and everything was fine
If you have a contract with them then they can't arbitrarily choose who they do business with. OP would presumably have a chance at a lawsuit against cloudflare here, the success of which would depend on how well cloudflare argued the ToS violation. A lawsuit might not be worth pursuing here, but this isn't a case of "it can't be helped".
Is this true? We are at 3TB and growing so I'm slightly concerned
For 3k/month you can get a good quality 10Gbps link. That's 3.2PB with a P.
Cloudflare controls supply and demand, which, by definition of the law, should be classified as extortion.
It meets the definition of a RICO "enterprise". The question is, will anyone bring it up for judicial review?
After hearing about these sorts of "discussions" from other colleagues, I certainly talked about using their services.
And then I realized that I had to hand them over my DNS? Uhh, no. It could have been "set nameserver to ours in your DNS console".
And also there was the recent SSL spoofing they're doing even with DNS with no hosted websites. And they charge money to send a revocation.
The whole thing is a hot yipes!
>And also there was the recent SSL spoofing they're doing even with DNS with no hosted websites. And they charge money to send a revocation.
What's your threat model here? That cloudflare will go rogue and... MITM your users? Can't they do that even if they're not in charge of your DNS? Even if you point an A record to them, that's enough to get a certificate via an ACME http-01 challenge[1].
[1] https://letsencrypt.org/docs/challenge-types/#http-01-challe...
You just need to configure the nameservers and that's it. That's how I do it for mine.
... that's how it works? They give you the nameservers to use, you set your domains up with them.
You can register a domain through them, but don't have to.
The folks at CF could have been less obtuse in handling the matter. But at the end of the day this is an online casino breaking ToS and they got spanked.
Fwiw this was some years ago and we moved most of our stuff away from them in response. I didn't get the feeling that this was malicious from their side, more like growing pains / mediocre support people / etc. But the end result was the same as you describe, except we chose not to pay up.
EDIT: more context: I shared this story on HN once before, jgrahamc responded with “please email me”, we did but it didn't move the needle. This further convinced me that CF just has a lot of stuff going on and something weird about our traffic made them error out. My suspicion is that the enterprise plan was supposed to make it internally defensible to pour more engineering resources into our case, but they were never explicit about that which made us worry enough to not do it.
I think that a large reputable business like CF should be clearer about stuff like this. That said, as someone running an API business, I also hold some sympathy for “customer does something weird an unexpected, it’s hitting a limit we didn't even know we had, srsly now what?”. The answer to that should be “work together with the customer to get to the bottom of things, customer might need to make changes too”. They didn't do that, which disappointed us, but I can relate to the situation nonetheless.
We’re still a CF customer, just not for this part of our offering.
this makes it sound like the limit is automatic or applies non-discriminately to customers, but my first instinct is that this was manually set by someone, maybe the sales reps again?
The bandwidth caps and all included features were clearly spelled out in the entetprise contract and when we went over, they didn't push for a contract renegotiation unless the overage lasted like 3+ months. And we frequently got new features included in for free.
In fact, recently they asked to renegotiate the contract due to some obsolescence and we ended up significantly dropping the bill as a result. Kind of backfired on them, I wonder if the account manager is kicking herself for this.
Perhaps the stories I have heard are from people with particularly bad/aggressive sales reps, or who are particularly bad negotiators on their side.
I will say, though, that the free plan is marketted as without traffic/bandwidth limits, and has no traffic limits in it's terms of service, no? If it is possible to abuse it with an amount of bandwidth, rather than this being a "feeling", wouldn't it be more clear and transparent and respectful to just make it clear in the terms?
Only if the cost of supporting the depreciated feature was less than the delta.
I still encounter people who refuse to believe that CF bandwidth isn't really free, when you can easily demonstrate that it's not by just observing who uses them. If their bandwidth truly was free and unlimited with no catch whatsoever then every bandwidth giant like Imgur would use CF, but they don't. Imgur uses Fastly, probably because it's cheaper than CFs "free".
With published thresholds they’re less able to upsell someone just shy of the limit without publicly changing the tiers. Doing that has the potential to upset existing customers who are over the new limit all at once, while also providing intel to competitors looking to undercut them.
IMHO it's just the price finding model that CF has adopted, I expect in the future they'll release limit numbers... unless they decide not releasing numbers is more profitable (i.e. the used car sales pricing model)
I'd guess that the cost of switching/cost compared to other alternatives/cost compared to business value/revenue, remained sustainable for the customer, who didn't want to deal with a switch.
In truth, this is kind of how "enterprise sales" has always worked? The salesperson trying to figure out the biggest price that won't lose the customer? But additionally having unclear terms and unclear length of contract (or really no contract locking in your terms/payment) is definitely in the vendor's favor...
That's the thing that gets me about all types of subscriptions / pay walls. You have my attention momentarially, make your best pitch as to why paying you is in my interest.
I think the lesson here is to be as provider agnostic as possible and have a backup plan in case your current provider decides they don’t like you anymore or they just delete all your data just because they can.
I was just curious to see what a landing page of a RNG supplier looked like, how do you even do sales for such a thing? With 3 players I guess it's just something you know in the industry and those partnerships are likely long-lived, right?
niches like this fascinate me for some reason!
I guess I'll see in a while if this was also just a sales tactic from Cloudflare or not.
Right, I'm not arguing that they're guilty, just that the legal system doesn't operate off pure black and white rules like "if you have two seemingly unrelated domains then you're trying to evade bans".
>and they give a traffic number that proves they're not doing "domain rotation".
Are you talking about the part where they said "95% of our traffic through the main domain"? Without additional context behind that number it's a stretch to claim that "proves" they're not trying to evade bans. For instance if their country is banned in country A, but country A is a small country that only makes up 3% of their overall traffic, they can confidently claim "95% of our traffic through the main domain" while still theoretically using alternate domains for ban evasion purposes.
I have more good things to say about OVH then bad. I do wish their edge firewall worked against internal traffic too however.
And you are still unprotected from your DNS registrar kicking you out, directing your domain to some "customer terminated" page until you can find another registrar, and have new NS records propagate (days).
Let me summarize: businesses do silly and sometimes stupid things for irrational reasons, or for reasons they never care to divulge. To avoid what happened that led to this discussion, the most suitable solution is to not be at the whims of companies that don't communicate well.
The legal requirements of domain registrars are clearly spelled out, unlike the TOS from, say, Cloudflare which leaves tremendous amounts to the imagination. These are not the same at all.
The same thing is true for IP reputation, just without an external official complaining. If other CF customers are negatively impacted by one customer’s action, CF isn’t violating safe harbor by booting that customer or passing on the costs of mitigating that impact. That’s just running a business, not exercising editorial review of hosted content in violation of safe harbor provisions.
I'm not saying cloudflare can't do it, I'm just saying it's wrong.
The only questions that come to mind: how do you know? If that was the case why didn't they tell the customer?
If you had legal presence in EU then new Digital Services Act[1] might a help for you. I am not sure if you could sue them based on that law, but you could maybe lodge a complaint.
https://www.wolftheiss.com/insights/digital-service-act-explained-new-obligations-for-online-businesses-and-other-digital-services/
https://blog.cloudflare.com/digital-services-actNot in response to the way Cloudflare came at them, anyway.
The new contract did put limits on some things we didn't have formal limits on before (like number of DNS queries), but aligned with our current usage, so our bill didn't go up.
I've been meaning to probe renegotiating pricing because we've been on this billing tier for probably a decade, and in the end some things we were able to negotiate down, and some rearchitect on the tech side to drop the usage by a staggering amount. We're still working out exactly what that amount is, I have several more weeks before renewal.
I think it’s weird to accept an enterprise contract without explicit terms…
Gambling however, you constantly have government auditors and the tax office crawling up your literal arse to make sure you don't cheat the gamblers, or worse, the government out of their money. And in some cases, add the mob or other criminals on top who also want their cut.
For components in the path of money flow (payment processors, RNGs, hosting, etc.), it's similar.
To me it looks like https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_pr... is entirely the wrong email to send in the situation and if you are as old as I am and come from where I come from, you will have flashbacks to "reading between the lines" of the party daily in the 1980s. The real content is at the bottom:
> As we have a very short window to report back to Trust & Safety team, please let me know if you can make time tomorrow
Big red flashing lights: the right questions are 1) why is T&S involved at all 2) What are their concerns which forces such a hurried deadline? 3) What are the consequences of missing this deadline.
The right email would start with something like this:
> Providing services to your business constitutes serious legal risk to Cloudflare. We are happy to work with you in the future if you are buying an Enterprise plan. As we need to commit significant resources to accommodate you, we need an annual commitment. Otherwise, with much regret we need to terminate our services provided to you as it is our right per Terms on date/time. ("We may at our sole discretion terminate your user account or Suspend or terminate your use or access to the Service at any time, with or without notice for any reason or no reason at all.")
> This plan would also include these features:
Companies take risks if the reward is considered good enough. In this case, that reward is income from the customer (who can still be dropped if the hit pieces start getting published).
For me, my corporate tax professional errs on the side of well tested strategies or those that the IRS has ruled on administratively, and that is perfect for my risk tolerance.
As I said above, we're happy to make exceptions for the occasional thread that is genuinely of interest, but sometimes the process for idenitfying those is "apply standard penalties -> generate howls of protest -> eventually find out about community pushback (e.g. in this case someone emailed hn@ycombinator.com) -> restore post".
I understand all the complexities here. Have worked closely with a Trust and Safety team before (real estate portal). But if that business was good enough to exist on the CF network for X years, it should be given time to move if it's no longer an agreeable business partner, that makes it a worthy story.
1. It's probably not RICO[1]
2. Are businesses under any obligation to take down shady businesses (eg. DDoS services that are ostensibly stress testing services) absent a court order?
[1] https://web.archive.org/web/20180305062824/https://www.popeh..., specifically sections "Wait. Isn't the defendant the enterprise?" and "So what's "racketeering activity"?"
This guy ran a DNS for years to prove it until he disappeared. Lots of nazi websites, ddoxing sites, crime networks, conspiracy sites, ransomware groups and russian misinformation campaigns that he uncovered.
Honestly I don't see another way to gather the data necessary for this otherwise. You have to have the DNS data to be able to imply intent.
And Akamai is very expensive.
Was there any indication that the "mass layoffs" were planned 90 days in advance?
That timing shows that it's not just implementing headcount and budget reductions, which are somewhat defensible if still tragic. It was instead a forced turnover, which in some cases can be a route to wage suppression.
Apparently the person in question was fired within 3 months of being hired[1]. If this is true it seems like a stretch to characterize it as "forced turnover" or "wage suppression".
If you're a public company and you go from healthy to mass layoffs within a single quarter then your investors (and employees) should be a lot more concerned.
1. According to the CEO the layoffs in question were for "~40 sales people out of over 1,500 in our go to market org"[1]. Is that really a "mass layoff"?
2. Did you not remember that the layoffs coincided with reversal in macroeconomic conditions? Specifically, the reversal from "inflation is transitory" to "inflation is persistent and the central bank will hike interest interest rates".
Not really nice to leave out halve of the info :)
If you can‘t land a single customer in 6 months under these circumstances, you‘re likely just not a good fit.
You can even google my name to confirm this info. This is pretty much exactly like it went when I started.
They also were clearly lying in those email messages: The second email says that domain rotation is strictly forbidden, but a few days later in the third email they're explicitly selling features for rotating domains more effectively.
And sorry, but a company selling "we'll override the Trust and Safety team if you pay us $$$" is absolutely unacceptable. There are only two options, both bad. Either they're not running a real TnS operation, but just pretend-staff one in order to run these kinds of shakedown operations. Or they're running a real TnS team that found a real problem but are letting sales people override the TnS team's honest judgement.
It's called "extortion"
You put yourself in a bad spot. We can either kick you out or work (for a price) to help you.
Extortion ? Hardly. Nobody work for free, you know.
Without more details about the environment, it is a 50/50 call.
They did. Repeatedly. You see it mentioned in the few emails the OP chose to share. But they also didn't share the other communication they had over the month long discussion they had with cloudflare.
> BYOIP also costs nothing to produce.
That's not really accurate. Cloudflare is entirely built around one big unified anycast network. If you want to provision an entirely separate network that maintains all the same features they are using from the main cloudflare network, that's going to require provisioning a lot of cpu time and routing table slots at a lot of different sites plus whatever admin and engineer overhead comes from maintaining this quarantined service.
>Here's the features you get with enterprise
And
>If you do not use byoip with your service you cannot be a customer, and also pay us $120k upfront while we do not tell you why, it's only a 40x price increase. You have 24h.
As a network engineer I'm well aware of what it costs to add prefixes to what you announce over bgp, some miniscule additional CPU overhead, likely unmeasurable. Even slow mips processors could process full table bgp.
Anycast/unicast changes nothing here. Anycast simply means that the same prefix is announced in multiple different PoPs.
puts on tin foil hat and looks around nervously
They are valuable in keeping people aware of what’s going on. But only to a point and people will endlessly argue over where that point is and scream censorship and over ups by corporate interests.
It gets rather grating after a while. Even more so when I suspect an article is omitting key facts to generate the attention they want.
I haven’t flagged this article but I can easily see why people would.
Whoever runs that team really, really gets off on being withholding, as Buster Bluth would say.
They say it's because if such teams don't operate under secret protocols, the "bad guys" will discover the loopholes in them. But I rather suspect that this has more to do with evading legal liability.
The most unprofessional thing CF did in my view was cutting off the customer's service so abruptly. But we have to bear in mind here we're only seeing one side of the story. And again, online casino, violating ToS, using CF's platform to circumvent blocks that were being placed on their website. Potentially to circumvent laws and so forth. That custom quote they received from CF could be pricing in a lot of things, including legal risk.
There's just a lot we don't know here, this isn't a typical customer and the idea that they got cut off abruptly because they told CF they were shopping around is entirely speculation by the post author.
Note, this is pure conjecture, I’m just well aware from my own engineering experience that stuff can break under varied load in all kinds of unexpected ways. A large part of the work of an infrastructure business is going “woa shit I hadnt expected that we could fail in that way too” and then building infrastructure to be able to handle that case. You simply can’t predict everything your customers are going to throw at you. I think this was what happened + not sufficiently knowledgeable/experienced support. But I admit that I’m really just guessing.
The alternative would be that CF purposefully dropped 10% of our traffic to convince us to upgrade to enterprise, and despite our bad experience, I don’t believe they’re that kind of business. And if they were they handled it very badly because it took them 3 weeks of feet-dragging to even bring up the upsell.
In support of your theory particular is I don't think enterprise sales "ragequits" a conversation when the customer is mid-evaluation based simply on the idea that they are considering multiple options.
Why would they walk away at this point, let alone ban the customer.
From the write-up I bet CloudFlare had it as a "60% to close" in their CRM at this moment. It doesn't make sense for them to drop the ban hammer in this moment.
PS: explanation or not, this is deeply shady behaviour from CloudFlare. Just perhaps a little less so.
> Why would they walk away at this point, let alone ban the customer.
It wasn't just that they were considering multiple options. Looking at the timeline, this was about a month after their initial soft gloves approach/enforcement action and they drug their feet the entire way through it.
Once CF got to the top of the leadership chain at their company and it was clear that all the relevant decision makers were involved in the conversation but were unwilling to pay, they just folded their cards, resumed the initial enforcement action, and moved on with their day.
If this was a small account they probably wouldn't have even blinked twice with just striking down the user for causing reputation harm and violating TOS but since they were a large account CF clearly went out of their way to meet with them multiple times and try to find a solution. But after a month of little to no progress while the account continues causing reputational harm and is unwilling to budge, they just called it quits and moved on.
If there is a TOS issue I’m not listening to a sales pitch on it. You better tell me what the issue is upfront in the first email instead of dicking around with the commission based workers. Like very low level stuff here imo
I don't see an unwillingness to fix TOS issues anywhere. Just an unwillingness to buy the enterprise tier. Those should not be treated the same way!
It doesn't matter the reasoning - its the execution wherein lies the issue - this is an extortionary business practice plain and simple.
By the way, it appears gambling sites are fine on CF [1].
[1] https://community.cloudflare.com/t/using-the-services-for-on...
CF calls and says there is a problem with domains. They want to push an enterprise plan. Customer wants to solve problem, dropping domains, making changes. CF says, only enterprise plan will remedy the situation.
There is obviously a sales script involved.
“get back to Trust & Safety"
Heard that story several times, it's always another team, e.g. "Licensing" that need to be satisfied, or that if you don't pay up, that team will be off the leash. Also heard the pay-for-a-year-upfront for several large vendors who pull this. The reason is, some sales reps need to make numbers, so they shake the tree and see who falls down:
"Cloudflare has absolutely no information on when they will force you into custom billing, but when they start "urgently" needing to talk to you you're probably not going to get out until you have a juicy custom contract with them."
this customer is damaging Cloudflare IP reputation which hurts other customers. Cloudflare can either fire the customer to protect other customers using Cloudflare IPs, or force this customer to use their own IPs and damage/manage their own IP reputation.
unfortunately this is expensive and OP is mad they can't do their legally fraught gambling operation on Cloudflare's addresses for free
If a custom IP is going to be mandatory, they need to say that and give a deadline, at the very least.
As a CF customer, I am happy that Cf is preventing another business from damaging mine.
This is directly contradicted by the contents of the article, perhaps you should re-read it.
>$120k up front for one year of Enterprise
Doesn't sound like a reputation problem.
They could've explained the problem ("your gambling business is a problem for our IP reputation") and offered a solution ("we can switch you over to BYOIP so this won't be a problem"), but instead they sent in an army of sales reps that demanded an upfront payment for a product tier that they only needed one small part of, to the point of sales people pretending to be part of other teams.
It makes business sense to kick out casinos, but OP got fucked over by Cloudflare's shitty practices.
But only after getting ~US$100k up front first, just because you can.
We may at our sole discretion suspend or terminate your access to the Websites and/or Online Services at any time, with or without notice for any reason or no reason at all. We also reserve the right to modify or discontinue the Websites and/or Online Services at any time (including, without limitation, by limiting or discontinuing certain features of the Websites and/or Online Services) without notice to you. We will have no liability whatsoever on account of any change to the Websites and/or Online Services or any suspension or termination of your access to or use of the Websites and/or Online Services.
For everyone else, this clause is pretty much standard for all SaaS services. Take your pick. If you don’t want this level of service with any vendor you have to sign an enterprise contract where termination procedures are agreed upon more intentionally by both parties.
> if a country DNS-blocks our main domain, a secondary domain may still be available
>Note that 80TB is the number they tried to sell us, I don’t know if it is accurate since they removed all our access to historical analytics.
I mean you dont need accurate Data but surely most would know by heart their traffic in rough figures? Or am I the old dog where every new Web Dev are so used to Cloud and Serverless they have no idea what they are using?
That's why the org took heat online for it.
I'm not sure why you think she was fired/assessed on the basis that she wasn't able to close enterprise deals within 90 days. The same tweet seems to refute this by saying "we can often tell within 3 months or less of a sales hire [...] whether they’re going to be successful or not". Presumably they're looking at various indicators (eg. size/composition of the sales pipeline, reviewing her sales calls and/or emails) and using that to predict performance.
Also here you go: https://en.m.wikipedia.org/wiki/Grok
That’s a very cynical take.
Nonetheless, have a wonderful Sunday evening!
My understanding: When complex knowledge is absorbed through deep immersion.
I was surprised to see to grok as a synonym for (? a possibly very superficial flavour of) to understand.
Would have loved to know why, from a linguistic perspective, GP used that word in this context.
I spend a lot of time wondering if the Emperor is wearing any clothes.
It would be completely different for a small project of course, but once you're counting in TBs... it's less important.
As far as exact volume of QPS or TB/month or whatever, I really couldn't say.
They weren’t protecting you or any of their customers. This is a mafia style shakedown
My guess would be that they were saying that Cloudflare's HN comment on the incident was to complain about google not cleaning up after the incident.
> the particular way you expressed your point heavily implies it.
You can't turn around the burden of proof this easily. Saying "the particular way you expressed it" doesn't give you license to make things up about a comment that is an inch above yours.
>the CEO’s strategy was to turn up here and criticise Google for not deindexing quickly enough
This wording implies that the CEO deflecting on HN was their strategy for responding to this problem, not that the CEO deflected on HN in addition to admitting fault elsewhere. Typically 'strategy' is used to refer everything that they planned to do, not a single action.
That is all I wrote and all I meant.
https://news.ycombinator.com/item?id=13718752#13721644
I don't much care if words are being put in my mouth but I do point it out.
But then again as Maya Angelou said:
"When someone shows you who they are, believe them the first time."
The CEO replied to someone asking about the services they were working with and complained about Google taking longer than the others. Maybe we're reading his comment in a different way, but to me there's a big gap between what he did and having a "strategy" to blame Google.
Cloudflare's CTO (jgrahamc) was on that thread too and didn't spend his time criticising Google. He wasn't hiding or saying "look over there instead!".
So I don't see any strategy from the CEO, CTO or the company to criticise Google or to ignore the fact that CF had f'ed up. Pointing out that Google was slow to remove cached pages is, in my view, a valid criticism.
Now, if you said that they had a strategy to minimise the problem, then I'd agree with you.
He was specificity responding to someone complaining it's still in Google's cache, by stating that "The caches other than Google were quick to clear [..] I agree it's troubling that Google is taking so long.".
By leaving out this context, and phrasing it a "strategy", is not a fair paraphrasing. These bits matter. Two things can be true at the same time: 1) Cloudflare messed up, and 2) Google is very slow to deal with this, and also messed up. Indexing all of the web comes with some responsibilities.
Please apply the same standards to yourself that you impose on others with such aggression and hostility.
... the CEO’s strategy was to turn up here and criticise Google for not deindexing quickly enough.
Jeez, this place.
I don’t know about you, but my customer trust is at an all time low, and I’m seriously considering at least moving all my registered domains off CloudFlare.
Any customer or potential customer who reads about this incident may have their trust in Cloudflare reduced, and rightfully so in my opinion. They have the legal right to terminate the relationship without reason or warning, but exercising that right in this context hurts their reputation.
If that was the problem, this issue wouldn't be relevant to most people.
When you switch to "they can terminate anyone", and they act this rashly and unexpectedly, that means anyone needs to live in fear.
So nobody is going to live in fear when cloudflare has the exact same TOS policy as everyone else.
Being a gambling website makes you an outsized target for attacks.
OP basically wanted to run DraftKings from the small business plan and CF understandably didn’t care to deal with a loss-leading customer like that.