HN: The Anti-Science Crowd?

7 points by thelettere 4 years ago | 15 comments

Why does every post having to do with a scientifically testable question almost invariably have top comments which are purely anecdotal rather than even attempting to point at empirical evidence?

This is particularly noticeable in anything having to do with quality of life or lifestyle questions, including the current front page question about anti-depressants. Indeed that relatively upvoted post goes so far as to ask purely for anecdotes - neatly dismissing nearly a century of evidence on the placebo effect, the known wide variance of reactions to any intervention and a million other documented factors that render anecdotes useless. Indeed, as far as HN is concerned, we may as well be in the dark ages, taking a poll on how HNers respond to leeches.

On a site users often describe as a kind of oasis from the nonsense perpetuated on the rest of social media, this seems rather baffling and suggests a need for - at minimum - some self-reflection.

bjourne 4 years ago |

You might be interested in this classic article: https://ieeexplore.ieee.org/document/675631

The author argued "yes, definitely", but most computer scientists answered "nah, too much work." People fond of $new language like to claim that if just all major software was rewritten in their favorite language life would be so much easier and bugs and security issues would be so much fewer. So you ask, "how do you know?" They answer: "It is obvious." "Have you tested it?" "There is no point, cause it is OBVIOUS!" "But if it is obvious, shouldn't it also be easy to test?" "We have NO TIME for that and if you don't see that $new language is better than $old language you are an IDIOT! Troll someone else!!" Or they'll argue that tests doesn't prove anything cause it depends on the context (like programmer skill, familiarity with $old and $new language, etc). Essentially, their hunch carries more weight than any quantitative data you could ever collect...

So much of software engineering is just shiny new thing after shiny new thing. No one knows whether the shiny new stuff is better than the old stuff or not. Even the book that introduced TDD & DP readily admitted that the project in which the development method was tested failed! Yet everyone adopted TDD because it was so "obvious" it was better.

FeaturelessBug 4 years ago |

I would imagine it's because many people have come to recognize that both forms of data have value. I might up vote a comment that was purely anecdotal that I found interesting even if that comment was something I didn't agree with or that I felt directly conflicted with well documented research. Additionally believing that any group of people online or in person is somehow intellectually superior or immune to basic human bias is a trap that will catch you again and again. Every human- no matter how smart, educated, or well read is fallible in a multitude of ways. I feel that the incidences of this increase exponentially when you get us into a social group.

thelettere 4 years ago | |

Have value how? How is it not in fact the very opposite of valuable (in cases where there is some empirical knowledge)?

I enjoy anecdotes occasionally as well, but here's a thought: if all you have is personal experience (i.e. you have no familiarity with the accumulated knowledge on the subject) - maybe you should listen rather than speak. Speaking then is akin to spitting in the face of the countless individuals who spend their lives investigating these questions, acting as if they and their work basically does not exist. It muddies the waters - particularly in light of our cognitive biases - and often misinforms where it should enlighten.

How is that moving the conversation forward? If this were a literal watercooler, than that's one thing - but this is forever archived material. The written word has always historically had a higher bar than spoken word (a tradition that the internet has undermined, and to our detriment) - and for a reason.

If we're all just here to entertain each other before we pass into oblivion, then that makes some sense. But the dominant narrative here is not of that kind, but of a belief in progress and the future.

If that is in fact an genuine wide-spread belief here, then these kind of threads are in service to the exact opposite of such beliefs. I'm not saying that anecdotes have no place, but only in the context of some conversance with the knowledge base - or in cases where the question has not been tested or is untestable.

Unless of course we want to just drop the pretenses, embrace our irrationality and start regular threads on medicinal leeches, communications with the dead, and phrenology. But somehow I don't see that happening anytime soon.

DiggyJohnson 4 years ago | | |

Personal experience is the medium of life itself. I’m surprised to hear you give it so little space in the discussion. I weigh personal experience (especially multiple anecdotes from credible sources) about as high as I weigh expert opinion. Just a question: are you a young person? I’ve just seen the experts be pants on head wrong many times, and that has an effect. I’ve also specifically been burned by overconfident doctors, so add that to the important anecdote pile.

If the science doesn’t align with the anecdotes (on any topic that isn’t partisan politics), then that makes me want to investigate the science, not toss out the anecdotes.

ComradePhil 4 years ago |

If you can't test it, because you need a degree, a license, an approval, access to huge funds, it is not "scientifically testable". Once in a while, when "science" is put to test, it has failed. A lot of the times, when irregularities are pointed out, people are deplatformed, discredited, have their licenses taken away, forced to resign. Stop repeating that nonsense.

There's only marketing material crafted to secure funds or profits. We only have anecdotes and stories that we can rely on... and only when it comes from people we know.

version_five 4 years ago |

If people just wanted stats or references, there are other places for that. They (we) come to a forum to discuss stuff, so anecdotes have value. Plus in general they are easier and more entertaining to process.

I'll also point out that your question has an anecdote in it about a current front page post. This is not a bad thing, you use it to illustrate your point, same as happens in comments.

(And, your headline implies without any evidence that your anecdotes make HN readers "anti-science". This is not good)

thelettere 4 years ago | |

There are no studies on the empiricism of HN comments, in which case anecdotes and personal observations are all we have.

And I'm not making some obtuse argument, but a common sense one which is readily recognizable by anyone who frequents HN. Are you contesting it?

DiggyJohnson 4 years ago | | |

You would need to see a study on the empiricism of HN comments before you cede that these discussions are valuable for the people to choose to engage in them? You’re holding discussion to the rigor of scientific publication. Have you completely squashed your internal agent of curiosity?

version_five 4 years ago | | |

FYI, your downstream comment is "dead", only people with showdead on can see it - I think this is because of the fourth last word you used. Don't ask me how I know, but variations of this word automatically kill posts

tcj_phx 4 years ago |

There's a replication crisis in science, in that most published literature can't be replicated. In medicine every patient is unique. It's hard to be scientific when no experiment with humans is repeatable.

With regards to the 'current front page question about anti-depressants': at least the mental health industry aspires to be helpful. Depression is mostly solved, except for how big business can make $$$ from simple fixes for depression.

thelettere 4 years ago | |

Not "most". Half of the published research in Ioannidis's ground-breaking analysis couldn't be replicated. Which means half could be (edit: numbers which roughly match a more recent analysis: https://elifesciences.org/articles/71601).

Which is why single studies should never be taken seriously - which is nothing new and frankly common sense. Thankfully there are recent reviews and meta-analysis readily available on most subjects, so that shouldn't even be a temptation.

Thank God for science, am-I-right?

phendrenad2 4 years ago |

I can't wait for the top comments here to be purely anecdotal rebuttals of your analysis!