Replicability, Robustness, and Reproducibility in Psychological Science

被引:252
|
作者
Nosek, Brian A. [1 ,2 ]
Hardwicke, Tom E. [3 ]
Moshontz, Hannah [4 ]
Allard, Aurelien [5 ]
Corker, Katherine S. [6 ]
Dreber, Anna [7 ]
Fidler, Fiona [8 ]
Hilgard, Joe [9 ]
Struhl, Melissa Kline [2 ]
Nuijten, Michele B. [10 ]
Rohrer, Julia M. [11 ]
Romero, Felipe [12 ]
Scheel, Anne M. [13 ]
Scherer, Laura D. [14 ]
Schoenbrodt, Felix D. [15 ]
Vazire, Simine [16 ]
机构
[1] Univ Virginia, Dept Psychol, Charlottesville, VA 22904 USA
[2] Ctr Open Sci, Charlottesville, VA 22903 USA
[3] Univ Amsterdam, Dept Psychol, NL-1012 ZA Amsterdam, Netherlands
[4] Univ Wisconsin, Addict Res Ctr, Madison, WI 53706 USA
[5] Univ Calif Davis, Dept Psychol, Davis, CA 95616 USA
[6] Grand Valley State Univ, Psychol Dept, Allendale, MI 49401 USA
[7] Stockholm Sch Econ, Dept Econ, S-11383 Stockholm, Sweden
[8] Univ Melbourne, Sch Biosci, Parkville, Vic 3010, Australia
[9] Illinois State Univ, Dept Psychol, Normal, IL 61790 USA
[10] Tilburg Univ, Metares Ctr, NL-5037 AB Tilburg, Netherlands
[11] Univ Leipzig, Dept Psychol, D-04109 Leipzig, Germany
[12] Univ Groningen, Dept Theoret Philosophy, NL-9712 CP Groningen, Netherlands
[13] Eindhoven Univ Technol, Dept Ind Engn & Innovat Sci, NL-5612 AZ Eindhoven, Netherlands
[14] Univ Colorado, Anschutz Med Campus, Aurora, CO 80045 USA
[15] Ludwig Maximilian Univ Munich, Dept Psychol, D-80539 Munich, Germany
[16] Univ Melbourne, Sch Psychol Sci, Parkville, Vic 3052, Australia
基金
澳大利亚研究理事会; 美国国家卫生研究院; 荷兰研究理事会; 美国医疗保健研究与质量局;
关键词
replication; reproducibility; robustness; generalizability; research methods; statistical inference; validity; theory; metascience; REGISTERED REPLICATION REPORT; PUBLICATION DECISIONS; STATISTICAL POWER; SOCIAL-PSYCHOLOGY; EFFECT SIZE; TESTS; BIAS; CONSEQUENCES; TRANSPARENT; CREDIBILITY;
D O I
10.1146/annurev-psych-020821-114157
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
Replication-an important, uncommon, and misunderstood practice-is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress.
引用
收藏
页码:719 / 748
页数:30
相关论文
共 50 条
  • [1] Reproducibility and replicability in science
    不详
    [J]. ENVIRONNEMENT RISQUES & SANTE, 2019, 18 (05): : 433 - 433
  • [2] REPLICABILITY, REPRODUCIBILITY, AND ROBUSTNESS - COMMENTS ON COLLINS,HARRY
    CARTWRIGHT, N
    [J]. HISTORY OF POLITICAL ECONOMY, 1991, 23 (01) : 143 - 155
  • [3] Promoting Reproducibility and Replicability in Political Science
    Brodeur, Abel
    Esterling, Kevin
    Ankel-Peters, Joerg
    Bueno, Natalia S.
    Desposato, Scott
    Dreber, Anna
    Genovese, Federica
    Green, Donald P.
    Hepplewhite, Matthew
    de la Guardia, Fernando Hoces
    Johannesson, Magnus
    Kotsadam, Andreas
    Miguel, Edward
    Velez, Yamil R.
    Young, Lauren
    [J]. RESEARCH & POLITICS, 2024, 11 (01)
  • [4] Reproducibility and replicability in science: A Sisyphean task
    Borges, Renee M.
    [J]. JOURNAL OF BIOSCIENCES, 2022, 47 (01)
  • [5] Reproducibility and replicability in science: A Sisyphean task
    Renee M Borges
    [J]. Journal of Biosciences, 2022, 47
  • [6] Reproducibility and replicability of science and thoracic surgery
    Lawton, Jennifer S.
    [J]. JOURNAL OF THORACIC AND CARDIOVASCULAR SURGERY, 2016, 152 (06): : 1489 - 1491
  • [7] The three R's of scientific integrity: Replicability, reproducibility, and robustness
    McIntosh, Robert D.
    Chambers, Christopher D.
    [J]. CORTEX, 2020, 129 : A4 - A7
  • [8] A better approach for dealing with reproducibility and replicability in science
    Nichols, James D.
    Oli, Madan K.
    Kendall, William. L.
    Boomer, G. Scott
    [J]. PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2021, 118 (07)
  • [9] Identifying and Overcoming Threats to Reproducibility, Replicability, Robustness, and Generalizability in Microbiome Research
    Schloss, Patrick D.
    [J]. MBIO, 2018, 9 (03):
  • [10] Reproducibility and Replicability
    Sweedler, Jonathan V.
    [J]. ANALYTICAL CHEMISTRY, 2019, 91 (13) : 7971 - 7972