Relevant article on similar lack of reproducibility in cancer research.

http://www.reuters.com/article/2012/03/28/us-science-cancer-...

Summary: a team at Amgen discovers 47 of 53 "landmark" studies published in high-quality journals could not be reproduced. A team at Bayer did an internal review of programs they had initiated based on journal studies and found that less than a quarter of those findings could be reproduced.

Three very damning quotes: "Some authors [of the journal articles] required the Amgen scientists sign a confidentiality agreement barring them from disclosing data at odds with the original findings."

"'We went through the paper line by line, figure by figure,' said Begley. 'I explained that we re-did their experiment 50 times and never got their result. He said they'd done it six times and got this result once, but put it in the paper because it made the best story. It's very disillusioning.'"

"The problem goes beyond cancer. On Tuesday, a committee of the National Academy of Sciences heard testimony that the number of scientific papers that had to be retracted increased more than tenfold over the last decade; the number of journal articles published rose only 44 percent."

Academics are pressured to produce publications, not to produce science, and their studies are not always rigorous (not blinded to experimenters, etc.). People with high integrity and ability do produce good science that gets published, but unfortunately that appears to be the minority, even in highly prestigious journals.

So I'm curious: how would you fix it?

One thing the article mentions that might improve things is every journal dedicating one complete issue a year to reproducing the most influential studies of the year. Another could be getting a consortium of pharmas (who try to reproduce studies all the time, because if you're going to successfully make drugs you need the thing to work) to publish their internal data for the benefit of all. Does something like that exist?

user:	myleskeating
created:	August 14, 2014
karma:	1