Ask HN: Has anyone leveraged GDPR to overturn automated bans?

146 points by ndmrs 4 years ago | 74 comments

There are many headlines of people getting their <FAANG/Big Company> accounts banned and losing access to a lot of important documents or services. Often, trying to talk to support on any of these companies is akin to talking to a wall.

However, GDPR has a clause stating that "The data subject shall have the right not to be subject to a decision based solely on automated processing". Which would mean that any EU/EEA citizen should have the right to have the decision reviewed by a human.

Has anyone successfully overturned a banned account using this method?

lambada 4 years ago |

From memory I believe FAANG etc all _claim_ that appeals you lodge are reviewed by a human.

Now if you don’t believe them then you’d need to take them to court and show why you think that’s not the case.

Which I guess means my question is why don’t you believe them and how likely is it that they are lying when they claim thy appeals are reviewed by a human?

tyingq 4 years ago | |

There was a recent example with Google Drive where it explicitly disabled any way to appeal. I was able to reproduce the issue where it was flagging files that consisted of a single byte, sometimes followed by \r\n or \n.

Here's the HN story: https://news.ycombinator.com/item?id=30060405

Screenshots of trying to "appeal" (Request a review) from when I recreated the issue show pretty clearly there is no human involved: https://imgur.com/a/5YHQtLi

This wasn't an account ban, so I don't know how well it fits the GDPR language. Though I'd be surprised if this was somehow the only "fully automated account action" FAANG type companies are doing.

Someone 4 years ago | | |

I don’t see how you get from Google’s statement “Was taken down for legal reasons and cannot be appealed“ to “no human was involved”.

jeffbee 4 years ago | | |

Are you suggesting that Europe has established a fundamental human right to have Google provide free static hosting services?

CodesInChaos 4 years ago | |

I think the better question is what a "human review" entails. I assume they have some kind of "human review" in there, but no meaningful human review.

toomuchtodo 4 years ago | |

> Which I guess means my question is why don’t you believe them and how likely is it that they are lying when they claim thy appeals are reviewed by a human?

Why would we believe them? It's Google's responsibility to prove their assertion, versus regulators taking them for their (not so good) word. The default should be the assumption that the corporation is being dishonest.

ben_w 4 years ago | | |

If you’re taking them (or anyone else) to court, isn’t the burden of proof on you?

BLanen 4 years ago | |

> Reviewed by a human

Can just mean some low-paid Amazon Mechanical Turk worker clicked on "Yes".

lamontcg 4 years ago |

We seriously need an Internet Bill of [Personal] Rights and get it into law and use it against the FAANGs. Europe at least seems to be trying, along with California sometimes.

chrisin2d 4 years ago | |

Agreed. We're overdue for a Magna Carta for our new era, lest we be absolutely ruled by the ever-growing myriad of algorithms and models that govern our participation in society and economy.

from 4 years ago | | |

It upsets me that we've ceded control of a nontrivial part of our lives to a bunch of opaque risk scoring algorithms. No one knows how they work and even if they did they couldn't tell you because that could "help the bad guys". My mom just called me the other day and told me she got locked out of her Google account after trying to reset her password. Who knows what tripwire she accidentally walked into (maybe has to do with the fact that it's a big household that probably has 10 Google accounts on the same IP address). You can find it in a bunch of other places too. Discord will give you vague error messages when you try to sign up with a VOIP number that magically go away when you use a regular number. "Card processing error" when you use a prepaid card. Of course ReCaptcha won't even let you fill out a form if you have an IP address that has ever been associated with known undesirables.

charlieyu1 4 years ago | |

For every proponent of a bill that protects user rights, there will be three opponents who claim censorship is needed to protect the public, to stop misformation, etc

throwawaygamma6 4 years ago |

Yes. I once got my account permanently locked at a well known service provider when I simply tried to make a payment for the first time. Support wasn't useful and all they could do was tell me that I somehow violated their Terms of Service for committing "fraudulent patterns" over and over again.

I could have and maybe should have just let it go, but it really got under my skin. I first tried out of band approaches to contacting somebody there. I didn't reach anybody, and you quickly realize how everybody else on the Internet just assumes you must either be lying or not telling the full story. Maybe it's just acceptable losses while doing business at scale.

So I finally just emailed them a polite GDPR request containing some spiel about Article 15(h), how I have the right to request my personal data, and also have the right to correct any inaccuracies in it, which must be the case since I committed no such fraudulent actions. I also requested a full list of all their data subprocessors, which I couldn't actually find listed anywhere on their site.

I'm not a lawyer, and I don't know if my request hit all the right notes or not. But literally one hour later, I got my account unlocked with a personal apology.

For what it's worth I also let them know that I'm not really looking to circumvent their systems, and I'm sure they have to deal with a lot of bad actors. But there really needs to be a better way to reach somebody to fix things when automated systems go wrong.

I also have the feeling that this approach would fall on deaf ears for big FAANGs, and there really needs to be some high profile ruling to put the fear in them.

foxfluff 4 years ago | |

> I didn't reach anybody, and you quickly realize how everybody else on the Internet just assumes you must either be lying or not telling the full story.

I have observed the same. When I evaluate service providers, I'm curious to know how they handle dispute with customers.. it's quite depressing to see that on most online forums, it usually goes straight into victim blaming. You must have violated the TOS, you must be doing something sketchy, you're not telling the whole story, you're just holding a grudge so get over it, you're just entitled, etcetra. There's very little sympathy, and no giving benefit of the doubt.

> But literally one hour later, I got my account unlocked with a personal apology.

Congrats! This is a lovely anecdote, thank you so much for sharing.

Nextgrid 4 years ago |

The problem is that the GDPR is pretty much not enforced. See https://ruben.verborgh.org/facebook/ where the author tries to get all his data from Facebook - the case hasn't moved since 3 years now.

The regulators are useless (especially the Irish one which seems happy to shield big tech scum from having to comply with the law) which confirms my own experience raising complaints with the ICO (the UK privacy regulator).

delroth 4 years ago |

Not speaking for my employer, but the actual quote from GDPR is:

> The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.

Emphasis mine. This would not include the vast majority of automated bans. It's more meant as a way to prevent e.g. automated police action via algorithmic selection.

foxfluff 4 years ago | |

I agree that it probably does not include the vast majority of automated bans.. but I'd prompt anyone interested to read the relevant guidelines to understand what might be in scope as far as legal effects or "significant effects" are concerned; it goes well beyond profiling by authorities, and commercial data controllers are far from exempt.

One example of a legal effect is cancellation of a contract. Examples of significant effect include automatic refusal of an online credit application, and e-recruiting practices without any human intervention.

Advertising is in scope too: "For example, someone known or likely to be in financial difficulties who is regularly targeted with high interest loans may sign up for these offers and potentially incur further debt."

Pricing is in scope too: "Automated decision-making that results in differential pricing based on personal data or personal characteristics could also have a significant effect if, for example, prohibitively high prices effectively bar someone from certain goods or services."

Finally, there's an example of profiling reducing a credit card limit. "This could mean that someone is deprived of opportunities based on the actions of others."

Anecdotally, getting kicked out of my email account has had far bigger effects on me than being rejected my credit card application.

https://ec.europa.eu/newsroom/article29/items/612053/en

lmkg 4 years ago | |

This is correct. The Article 22 rights are the most narrowly-restricted of any data subject rights granted by GDPR. It only applies to things that are a Very Big Deal, like prison sentences, voting rights, or eligibility for government services.

While not tested by the courts, there is a plausible argument that "similarly significantly affects him or her" might apply to bans that impact your ability to earn a living. So streamers getting banned from YouTube, or AdWords bans for businesses where that's their main source of revenue. Bans that are lower-stakes than that get harder to justify under Article 22.

leobg 4 years ago |

There’s also the right to be forgotten. How can they ban you if they have to delete all information that can be used to identify you?

Even hashes of your email address or payment data should be something you should be able to request they must delete.

teraflop 4 years ago | |

Even under the GDPR, the right to be forgotten is not absolute or unconditional.

If a person revokes consent for their personal data to be used, the data must be deleted if "there is no other legal ground for the processing". But if a data processor has an overriding "legitimate interest" in storing data about you, then they have legal grounds to do so without your consent. The details of this will vary depending on the situation (and the jurisdiction) but, for example, fraud prevention is explicitly called out as a legitimate interest.

https://law.stackexchange.com/questions/37882/google-adwords...

hsbauauvhabzb 4 years ago |

Aside from using imap to backup my mail, what else should I do to help mitigate an arbitrary ban? I’ve had a gmail account for 20 years since 12 year old me got caught up in invite fomo. I’ve since moved to other providers but still there’s a fair amount tied into my account currently.

Mostly I’m scared of ‘multifactor’ where email access is considered a form of identity, but I’m not sure what else

blibble 4 years ago |

getting anything sensitive data out of large companies with the GDPR seems to be impossible unless you want to resort to lawyers

I was trying to get my matchmaking data out of Activision Blizzard and they flat out refused, saying my data was their property

their exact response was:

> "the information requested are trade secret and/or intellectual property needed to preserve our game integrity"

I complained to the regulator, who agreed with my assessment, but to enforce it I'd have to go to court

seems the GDPR is basically useless

rognjen 4 years ago | |

That's a very interesting situation for two reasons:

1. Arguably your matchmaking data is someone else's as well. Meaning, they'd be potentially exposing other people's data to you.

2. Arguably you don't own the matchmaking data. You only own the initial request for matchmaking. The end result is actually a product of their proprietary algorithm. You didn't generate it.

Perhaps it might be a good idea getting in to touch with a privacy campaigner, or if the European equivalent of ACLU exists, and have them test this in court because it affects two different and important aspects.

jdavis703 4 years ago | |

I hear people saying laws that require police reports, police enforcement or interactions with the court are useless. For people like yourself who feel this way, what alternatives do you propose?

blibble 4 years ago | | |

what's the purpose of a regulator if they agree with you and can't do anything to enforce the law?

it's privacy theatre, nothing more

advisedwang 4 years ago |

GDPR Article 22 (the rule you refer to) also has exceptions:

> Paragraph 1 shall not apply if the decision...is necessary for...performance of, a contract between the data subject and a data controller

Which I can see applying as they probably have something in the ToS to enforce here.

It also allows automated decision making to comply with EU law. I don't know EU copyright law well enough, maybe Google has a responsibility to take down that data under copyright law and so this exception applies too.

yccs27 4 years ago | |

The ToS does not constrain the company! The agreement does not stipulate that the platform _has_ to enforce ToS, so this is not a necessary action to perform the contract.

ilamont 4 years ago |

The data subject shall have the right not to be subject to a decision based solely on automated processing

Lots of leeway for FAANG/BigCo management to wriggle out of that one. "Sure, Jones in Legal gets an email notification every time an account is banned and has the option to review it."

I can only imagine the lobbying and "negotiation" that takes place to have legislators water down the requirement for real human beings to review or respond to such bans.

sealeck 4 years ago | |

I doubt that would hold legally speaking because that would essentially be purely automated data processing.

stubish 4 years ago |

Are these actual GDPR takedowns causing accounts to be banned, or are these via the internal copyright enforcement systems implemented so that actual legal GDPR requests don't have to be sent (with all the strings attached to those). ie. Banned because of GDPR (giving you rights) or banned because of violating Terms of Service (giving you no rights in almost all TOS)?