Sentences to ponder, scientific fraud edition

In 2015, a big team of researchers tried to redo 100 psychology studies, and about 60% failed to replicate.

This finding made big waves and headlines, and it’s already been cited nearly 8,000 times.

But the next time someone brings it up, ask them to name as many of the 100 studies as they can. My bet is they top out at zero. I’m basically at zero myself, and I’ve written about that study at length. (I asked a few of my colleagues in case I’m just uniquely stupid, and their answers were: 0, 0, 0, 0, 1, and 3.)

This is really weird. Imagine if someone told you that 60% of your loved ones had died in a plane crash. Your first reaction might be disbelief and horror—“Why were 60% of my loved ones on the same plane? Were they all hanging out without me?”—but then you would want to know who died. Because that really matters! The people you love are not interchangeable! Was it your mom, your best friend, or what? It would be insane to only remember the 60% statistic and then, whenever someone asked you who died in that horrible plane crash, respond, “Hmm, you know, I never really looked into it. Maybe, um, Uncle Fred? Or my friend Clarissa? It was definitely 60% of my loved ones, though, whoever it was.”

So if you hear that 60% of papers in your field don’t replicate, shouldn’t you care a lot about which ones? Why didn’t my colleagues and I immediately open up that paper’s supplement, click on the 100 links, and check whether any of our most beloved findings died? The answer has to be, “We just didn’t think it was an important thing to do.” We heard about the plane crash and we didn’t even bother to check the list of casualties. What a damning indictment of our field!

Here is more from Adam Mastroianni.