Big Data Is Not Truth

cameronh90 12 years ago |

This is true of any statistics, and I wasn't aware that anyone believed that big data was going to somehow eliminate subjectivity from the world. More data and faster processing means more opportunities to analyse and a bigger sample size, but as the adage goes, there are lies, damn lies and statistics. The article seems like a bit of a straw man.

And regarding myth 3 and 6: you can't really hold these views simultaneously. If you can opt out, then that's a built in form of discrimination. Whether that's actually relevant probably depends on your domain.

wavefunction 12 years ago | |

Many of the vocal cheerleaders have probably little experience or training in statistics, and so such nuance is beyond the scope of their understanding. Sure this seems self evident to those of us with the exposure, but think about all the folks who stop at college algebra.

yummyfajitas 12 years ago |

Before Big Data disappears into the background as another fact of life, Ms. Crawford said, “We need to think about how we will navigate these systems. Not just individually, but as a society.

http://xkcd.com/1215/

kmm 12 years ago | |

There should be an xkcd about reposting xkcd when you can't add something meaningful to the conversation yourself.

sliverstorm 12 years ago | | |

It's true it isn't high-effort. That said, referencing xkcd (if not done solely for the humor) can be a nice way of succinctly presenting an idea that you would have made a mess of explaining yourself.

bradleysmith 12 years ago | | |

I agree with you. It takes nearly no effort to digest, and xkcd has a strong selection of 'debate-memes' to choose from on making intelligent points.

kmm 12 years ago | | |

I know and I sympathize. It's just that xkcd is so condescending sometimes and I wanted a good way to let that know.

srl 12 years ago | | |

Almost: http://xkcd.com/307/

pdpi 12 years ago |

The whole piece reeks of FUD to me.

What I would gather from taking the article at face value is: According to Myths #2 and #3, big data is classist, racist and sexist. According #5 and #6, Big Data is actually Big Brother. #1 points out that Big Data isn't The Next Big Thing, because it's actually old news.

There's plenty of genuine problems with Big Data, but the article seems to be more concerned with playing on people's fears regarding it than actually providing any concrete arguments about its failings.

wmil 12 years ago | |

FUD is probably the wrong word.

Quentin Hardy has spent his life in the publishing industry. He's likely a strong verbal thinker.

That's exactly the sort of person who hates statistical reasoning.

Thrymr 12 years ago |

Data (of whatever size) is never truth. It is a means of testing hypotheses.

mtowle 12 years ago |

It's articles like this one that make me wonder why anyone is submitting pieces from the NYT, et al. Their audience is simply not HN's audience.

mark_integerdsv 12 years ago |

...but the hype cycle is and articles like this one are merely a product thereof. I really feel that opinion pieces ought to come from someone whose opinion is or should be highly regarded in a given field.

mathattack 12 years ago |

Very light article. Seems like they put up a strawman of a marketers over-simplified view of the world, and then attacked it.

Most serious practitioners realize this is tough business.

zxcdw 12 years ago |

Privacy? There is none! Deal with it.