Does history have a replication crisis?

Does history have a replication crisis?(ageofinvention.xyz)

43 points by salonium_ 2 years ago | 21 comments

dmvdoug 2 years ago |

In order to become a history teacher, I had to get a master’s degree in education. I was required to take a minimum of five graduate-level history classes in the history department as part of it. I was transitioning from being a lawyer. Professors would talk about checking citations (although not systematically) but it was more for the purpose of finding useful material you want to use, not to check how they are being used by authors. Ok, whatever. I was used to reading cited sources for the purpose of evaluating the argument they were purportedly supporting. But every single time I raised in a seminar that some citation did not bear the weight placed on it, or even simply failed to logically support what it purported to support, it was explained to me that I was missing the point.

After the second seminar of this happening, the chairperson of the history department called me into his office. He said he understood that I used to be a lawyer and thus was likely to be more drawn to or interested in arguing about things. He understood I was going to become a teacher, not a professional historian. But my approach to sources and citations was being complained about by history graduate students and the professors agreed. I was told that when reading secondary sources in seminars I should spend more time thinking about what is valuable about them and how they might be used in further research than scrutinizing the evidence and logic underpinning them. Anyway, it was a little insulting for me to think that I could legitimately critique such things in a useful way when I had not spent the years of work in the archives that the authors of those works did. He had called this meeting with me in order to be frank with me about the problems that the history department was having with me and to give me a “second chance” before talking to the education department where my degree would come from.

Suffice to say, I got the message: I obediently complied and shut my mouth in the remaining seminars. I got my degree and was certified and am now a history teacher at an urban public high school. I’m happy with my life. And I’ll never think about professional history/historians with the same fundamental deference to expertise as I did before I got a glimpse of it up close.

defrost 2 years ago | |

As a degree mathematician who has done a little contract work for both historians and lawyers and may have occasionally regarded law as the study of first order logic and theatrics (fun sledge, amusement rather than disrespect intended) I have to ask whether the history crowd had a valid point wrt to strength of reference in supporting an argument.

If five decades onwards one were to write a History of Our Times (the Central North American Edition) it would be correct to, say, point out that claims were made in the media and courts regarding a stolen election and to then cite court filings, articles in prominent newspapers, and perhaps videos from both network and youtube archives.

It's fair to say that such claims formed the zeitgeist of our recent times for a substantial portion of the population affected and that any citation failed to logically support what it purported to support.

This is the "History is what happened" argument not the "History makes sense" position.

I freely acknowledge I've slid past the arguments made in this thread source, that current history isn't being rigourous with open access to sources and annoted commentary of primary material - but I note that the crafting of alternative histories can be coincident with primary events.

sdwr 2 years ago | | |

The election is something that distance makes clearer, although 50 years might not be enough time. Think of the history of the roman senate. Truth is barely talked about, the narrative revolves around factions, power, and outcomes.

Call it post-truth, call it machiavellian, but being on top is a social game, not a practical one.

dmvdoug 2 years ago | | |

> If five decades onwards one were to write a History of Our Times (the Central North American Edition) it would be correct to, say, point out that claims were made in the media and courts regarding a stolen election and to then cite court filings, articles in prominent newspapers, and perhaps videos from both network and youtube archives. It's fair to say that such claims formed the zeitgeist of our recent times for a substantial portion of the population affected and that any citation failed to logically support what it purported to support.

If you argued claims were made in the media regarding a stolen election and cited a variety of newspapers and blogs where people talk about how DT thought he won the election because of 10,000 dead people voting in GA or whatever, so he should be President not Biden, how would those citations fail to logically support your argument that claims were made in the media regarding a stolen election?

alsetmusic 2 years ago | |

About five years ago I asked myself whether I wanted to pursue a career change out of tech. I’m always excited by history and I’m sad that more people don’t feel the same way. I considered becoming a history professor or teacher and it was the credentials required that stopped me.

I’ve never had the patience for school even though I love to learn. I make a good living in my field despite not having a CS degree like many of my peers. I commend you for taking it upon yourself to do what I did not. I hope you have a very successful teaching career, by which I mean that I hope you have at least one student discover the joy in learning history.

aidenn0 2 years ago | |

> Suffice to say, I got the message

And this is at least part of why I don't do well in academia (and, tbf, many other settings); I refuse to get the message.

dmvdoug 2 years ago | | |

I had a different objective: get my MAT (Master of Arts in Teaching), get certified in my state and get a job at a secondary school. They were threatening to throw a monkey-wrench at me. It wasn’t worth the trouble.

Had I been a history graduate student, on the other hand, I very well have fought that fight, because it represents a significant problem in the field of academic history.

dado3212 2 years ago |

This is kind of an open problem, especially in the Internet age. Any retraction that’s published will have far less reach than the original incendiary claim. And it’s simply not viable to follow that hundred-chain length sourcing tree for everything used in an analysis. Not sure that there’s a solution that’s actually implementable.

RetroTechie 2 years ago | |

Life's too short to check or even verify the many factoids you encounter in daily life. With:

Check = check cited sources yourself. And

Verify = re-do a study yourself. Outside of hard science like physics (and relatively simple experiments), in most cases that's not even possible. Especially history: original artifacts disappear, witnesses die & the places where things happened are changed.

Written records are just that - what was written down. Accurate or not.

This is true for ordinary people as well as scientists. Though I'd expect scientists to make a bit more effort (+ using field-specific knowledge to sort things out).

But news reports tend to be notoriously unreliable imho. Especially popular media or (yuk) social media.

If some science study is reported on, it's often enlightening to read that study yourself. And find it says very different things than the news reports. Or does not even say what's reported elsewhere. Bias, sensationalism & clickbait are a thing.

derbOac 2 years ago |

The problem he mentions happens in science too. It's interesting to see it manifest in history, but it's well-documented that retracted scientific papers continue to be cited as veridical at high rates long after they've been retracted. I've personally noticed papers being cited for things other than what they actually say as well (someone posted an article to HN not too long ago about a particularly amusing example of this but I can't recall it very well).

It doesn't really undermine what the author is saying, but I do think it might be one of these cases where errors in citations reveal something more fundamental about citation process in general. For example, I can also think of shifts in who gets cited on a point, as if something is a new discovery, when it's really more fads in who is prominent in a field, and people "discovering" something by virtue of reading it for the first time by a specific author. The common denominator is convenience or ignorance (not paying attention per the author), which has implications for many things, not just factual errors per se.

mike_hearn 2 years ago |

The article focuses on the fact that nobody checks citations, but that isn't a problem specific to history. It's more like a general issue with academia. Science papers have the exact same problem. I found during COVID many papers being widely cited in the press as the consensus of experts which had bogus citations in them.

But there are other issues that are actually specific to history. Some years ago I had fun writing a series of essays about these, through the framing of conspiracy theories about history.

Part 1 looks at claims that key historical sources might have been forged during the Renaissance.

https://blog.plan99.net/meta-historical-conspiracies-part-1-...

Part 2 looks at problems with astronomical dating methodologies.

https://blog.plan99.net/meta-historical-conspiracies-part-2-...

Part 3 looks at methods like radiocarbon dating and tree ring counting.

https://blog.plan99.net/meta-historical-conspiracies-part-3-...

The goal here was to describe other people's claims about historical "replication problems" for entertainment purposes rather than try and discern the truth, so I wouldn't argue they're all real. But the trend towards presenting history as scientific doesn't gel well with the culture in the academic humanities. As dmvdoug writes, historians don't really care about even basic things like whether citations are accurate, let alone more debatable things like the accuracy of radiocarbon dating or whether sources have plausible chains of custody. And as the story of Henry Cort demonstrates, historians will happily make up narratives from whole cloth if it advances their ideological agenda, yet nobody within the academy seems to care.

ta20230829 2 years ago |

> This lack of effective institutions or incentives was really brought home to me recently by the publication of a paper in the prestigious journal History & Technology by Jenny Bulstrode of UCL, in which she claimed that the inventor Henry Cort had stolen his famous 1783 iron-rolling process from Reeder’s iron mill in Jamaica, where it had been developed by 76 black metallurgists by passing iron through grooved sugar rollers. It was a widely-publicised paper, receiving 22,756 views — eleven times as many views as the journal’s next most most read paper, and frankly unheard of for most academic papers — along with a huge amount of press coverage.

It is pretty obvious that this became so widely spread because it agrees with a certain political viewpoint.

hgsgm 2 years ago |

Those who do not study history are doomed to replicate it.

Those who do study history are doomed to not replicate it.

nextlevelwizard 2 years ago |

Problem is that no one wants to pay you to replicate a study and if you do replicate a study out of your own pocket it is a kin to setting that money on fire career-wise since no one will care.

The whole peer review science scene is all about making money and getting famous. At worse reviewers are trying to nitpick and block their peers long enough to beat them to publication so they can have the claim to fame and thus get more grant money for their next project.

m0llusk 2 years ago |

HistoryNarrativeCheckerGPT maybe?