"Why the reproducibility movement is not enough to salvage the trustworthiness of psychological science"

Date & time

4–5pm 20 October 2015

Location

PSYC G8

Speakers

Professor James Coyne, University Medical Center, Groningen, The Netherlands

Registered replication reports and reproducibility initiatives represent important milestones in the struggle against the untrustworthiness of psychological science. They generate publicity and provide quantitative data concerning the extent of the problems we face.   Yet their accomplishments should not be overestimated, and the limitations of organized replication efforts as a general strategy of reform should not go unnoticed.  Reproducibility initiatives are only suitable for some areas of research and risk overemphasis on the problems concentrated in these areas. Large bodies of clinical and policy-oriented research do not lend themselves so readily to efficient replication.  Reproducibility initiatives favor collaboration with original investigators over insistence on greater transparency in original reports. They potentially further ghettoize replications, protecting the existing publication practices of vanity journals.

Institutional agenda and the incentives they provide for publishing bad science remain intact and resistant to reform. Questionable publishing practices (QPPs) need to be given attention along with questionable research practices (QRPs).  Some of the effort being put it into replications would better be directed to insisting on enforcement of existing commitments to preregistration of studies, reporting standards, data sharing, transparent reporting of all findings, and appeal processes for rejections of manuscripts. The “Pottery Barn rule” (you break it, you buy it) should be more strictly enforced.  But an expanded Pottery Barn rule is warranted so that the trustworthiness of findings can more readily be assessed without formal replication, and untrustworthy findings can be more readily corrected or eliminated from the literature.

Updated:  24 November 2017/Responsible Officer:  Director, RSP/Page Contact:  Web Admin, RSP