> If researchers preregister their study protocols and make their materials, code and data publicly available, then the research community can verify studies and build upon them. In this way, open science can reduce questionable research practices, or make them more detectable, realigning the incentive structures that traditionally underpin research culture (i.e., a refocus on quality over quantity). Openness can also help to reduce publication bias by making outputs more discoverable (even if a researcher does not publish a study, there is a record that it was conducted; see Ensinck & Lakens, 2023).
That all sounds great, but if somebody really wants to get a certain result then none of that will prevent them from fudging the data if necessary to get that result. I believe the reason psychology struggles with replication is because the topics being researched are "too important.". To use the example from the article, psychology textbooks talk about the stereotype threat effect, knowledge of negative stereotypes influencing people to perform poorly, without mentioning the dodgy evidence and inconsistent results. This is probably because the result is "too important" in a social/political sense, and the distortion exemplified by textbooks will continue no matter what structural changes are made to the field. If psychology were studying sea snails, where the stakes are low, then it would be a lot easier. But it studies people, the stakes are too high so distortion, fraud, etc are inevitable and any system created to minimize these will be subverted if not outright ignored when getting the "right" result is deemed necessary.
Interesting you describe it as being ‘too important’ for being a reason people lie and cheat in the social sciences. I wouldn’t put it so kindly.
Academia as a power structure is a lot about gaining and retaining power, this also encourages researchers to intentionally or unintentionally fabricate results, to make a name, to get cited, to get funding and prestige.
I do buy the fact that many individuals conducting the tests don’t have a firm enough grasp on basic statistics but it’s laughable that an industry that relies on proving its worth via these measures, failed for so long to put emphasis on the value of measuring things the right way. Especially with the so called peer review system.
The bigger fraud is that so many people decided to go into the field on the back of seeing results from these studies reported in the press and spent hundreds of thousands of $$ on what is largely being seen as snake oil.
> If researchers preregister their study protocols and make their materials, code and data publicly available, then the research community can verify studies and build upon them. In this way, open science can reduce questionable research practices, or make them more detectable, realigning the incentive structures that traditionally underpin research culture (i.e., a refocus on quality over quantity). Openness can also help to reduce publication bias by making outputs more discoverable (even if a researcher does not publish a study, there is a record that it was conducted; see Ensinck & Lakens, 2023).
That all sounds great, but if somebody really wants to get a certain result then none of that will prevent them from fudging the data if necessary to get that result. I believe the reason psychology struggles with replication is because the topics being researched are "too important.". To use the example from the article, psychology textbooks talk about the stereotype threat effect, knowledge of negative stereotypes influencing people to perform poorly, without mentioning the dodgy evidence and inconsistent results. This is probably because the result is "too important" in a social/political sense, and the distortion exemplified by textbooks will continue no matter what structural changes are made to the field. If psychology were studying sea snails, where the stakes are low, then it would be a lot easier. But it studies people, the stakes are too high so distortion, fraud, etc are inevitable and any system created to minimize these will be subverted if not outright ignored when getting the "right" result is deemed necessary.