There are a lot of things to love in this piece from Christie Aschwanden about why retractions, studies that don’t hold up to reproduction, and even sub-fraudulent “p-hacking” do not mean that science is broken, but it is, simply, very hard. Among those things are the great visuals from Ritchie King – including a fun “p-hacking” demonstration tool.

For me, the real take home message goes beyond the “science is hard” catchphrase. Science isn’t just hard in the way implied by Tom Hanks’ Jimmy Duggan character in A League of Their Own:

It’s supposed to be hard. If it wasn’t hard, everyone would do it. The hard… is what makes it great.

Contrary to the rhetoric that would portray “science is hard” as an endorsement of success over a monumentally difficult task, this is not the point.

As Ashwanden addresses, science is hard because it is messy and complicated and requires a communal effort from members of a species that is only dubiously social outside of relatively narrow local groups.

If we’re going to rely on science as a means for reaching the truth — and it’s still the best tool we have — it’s important that we understand and respect just how difficult it is to get a rigorous result.

There are things like sampling variance and mistakes and uncontrollable environmental variables and resource limits and the fabled “orthologous methods” that inject all sorts of inconsistency and challenges into the textbook scientific method. This is why the great philosophers of science* spoke about disproof rather than proof, about independent reproducibility, about probability rather than certainty.

These issues do not indicate that science is broken. There simply is no other way it could work in the hands of mere humans. What may be broken is the way we perceive science. We need to understand that it is a gradual and a community effort. We need to understand that our mythos of science – of the great, usually in the stories, man performing a great experiment and making a great discovery – are almost always false summaries which are convenient and inspiring, but do not represent why science is truly hard.

*It is also why those who dismiss the philosophy of science as a waste of time – I’m looking at you Neil DeGrasse Tyson – deserve nothing but the most vigorous of side-eyes on that point.

Filed under: Follies of the Human Condition Tagged: Christie Aschwanden, FiveThirtyEight, Linkonomicon, retractions, Ritchie King, science, scientific method, Scientific Process