Tom Hardwicke

Tom Hardwicke

Research Fellow

University of Melbourne



Publications


Preprints / Stage One Registered Reports

Hardwicke, T. E., Schiavone, S., Clarke, B., Vazire, S. (accepted in principle). Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary [Stage One Registered Report]. Peer Community in Registered Reports.
[preregistration]

Frank, M. C., Braginsky, M., Cachia, J., Coles, N., Hardwicke, T. E., Hawkins, R., Mathur, M., & Williams, R. (in preparation). Experimentology: An open science approach to experimental psychology methods. MIT Press.
[open access book]

Fernández, L. M., Hardwicke, T. E., & Vadillo, M. A. (preprint). Retracted papers clinging on to life: An observational study of post-retraction citations in psychology. PsyArXiv.
[preprint] [data] [analysis-code]

2024

Dulitzki, C., Crane, S. M., Hardwicke, T. E., & Ioannidis, J. P. A. (in press). Expanding the Data Ark: An attempt to make the data from highly cited social science papers publicly available. Royal Society Open Science.
[preprint] [data] [preregistration] [materials] [analysis-code] [reproducible-container]

2023

Hardwicke, T. E., & Vazire, S. (2023). Transparency is now the default at Psychological Science [editorial]. Psychological Science, 09567976231221573.
[paper]

Thibault, R. T., Kovacs, M., Hardwicke, T. E., Sarafoglou, A., Ioannidis, J. P., & Munafo, M. R. (2023). Reducing bias in secondary data analysis via an Explore and Confirm Analysis Workflow (ECAW): A proposal and survey of observational researchers. Royal Society Open Science, 10, 230568.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code]

Hardwicke, T. E., & Wagenmakers, E.-J. (2023). Reducing bias, increasing transparency, and calibrating confidence with preregistration. Nature Human Behaviour, 7, 15-26.
[paper] [preprint]

Hardwicke, T. E., Salholz-Hillel, M., Malički, M., Szűcs, D., Bendixen, T., Ioannidis, J. P. A. (2023). Statistical guidance to authors at top-ranked journals across scientific disciplines. The American Statistician. 77(3), 239-247.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

2022

Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Tzavella, L., Bendixen, T., Handcock, S. A. Köneke, V. E., Ioannidis, J. P. A. (2022). Post-publication critique at top-ranked journals across scientific disciplines: A cross-sectional assessment of policies and practice. Royal Society Open Science, 9, 220139.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container] [media1] [media2] [media3]

TARG Meta-Research Group & Collaborators [Thibault, R. T., Hardwicke, T. E., Clark, R. W. A., Pennington, C. R., Nilsonne, G., O’Mahony, A., Drax, K., Thompson, J., Munafò, M. R.] (2022). Discrepancy review: A feasibility study of a novel peer review intervention to reduce undisclosed discrepancies between registrations and publications. Royal Society Open Science, 9, 220142.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M., Rohrer, J., Romero, F., Scheel., A., Scherer, L., Schönbrodt, F., & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73, 719-748.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., & Ioannidis, J. P. A. (2022). Estimating the prevalence of transparency and reproducibility-related research practices in psychology (2014-2017). Perspectives on Psychological Science, 17(1), 239-251.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2021

Hardwicke, T. E., Szucs D., Thibault, R. T., Crüwell, S., van den Akker, O. R., Nuijten, M. B., & Ioannidis, J. P. A. (2021). Citation patterns following a strongly contradictory replication result: Four case studies from psychology. Advances in Methods and Practices in Psychological Science, 4, 1-14.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Bohn, M., MacDonald, K., Hembacher, E., Nuijten, M. B., Peloquin, B. N., deMayo, B., Long, B., Yoon, E. J., & Frank, M. C. (2021). Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: An observational study. Royal Society Open Science, 8, 201494.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2020

Hardwicke, T. E., & Goodman, S. N. (2020). How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey. PLOS ONE, 15, e0239598.
[paper] [preprint] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020). Calibrating the scientific ecosystem through meta-research. Annual Review of Statistics and its Application, 7, 11-37.
[paper] [preprint]

Ferrero, M., Hardwicke, T. E., Konstantinidis, E., & Vadillo, M. A. (2020). The effectiveness of refutation texts to correct misconceptions among educators. Journal of Experimental Psychology: Applied, 26, 411-421.
[paper] [preprint] [materials] [data] [analysis-code]

Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020). An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017). Royal Society Open Science, 7, 190806.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2019

Hardwicke, T. E., & Ioannidis, J. P. A. (2019). Petitions in scientific argumentation: dissecting the request to retire statistical significance. European Journal of Clinical Investigation, 49, e13162.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container] [comment1] [comment2] [comment3]

Nosek, B. A., Beck, E. D., Campbell, L., Flake, J. K., Hardwicke, T. E., Mellor, D. T., van ’t Veer, A. E., & Vazire, S. (2019). Preregistration is hard, and worthwhile. Trends in Cognitive Sciences, 23, 815-818.
[paper] [preprint]

Hardwicke, T. E., Frank, M. C., Vazire, S., & Goodman, S. N. (2019). Should psychology journals adopt specialized statistical review? Advances in Methods and Practices in Psychological Science, 2, 240-249.
[paper] [materials] [data] [analysis-code] [reproducible-container] [media]

2018

Hardwicke, T. E. & Ioannidis, J. P. A. (2018). Mapping the universe of registered reports. Nature Human Behaviour, 2, 793–796.
[paper] [materials] [data] [analysis-code] [reproducible-container] [preprint] [comment] [comment2]

Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., Lenne, R. L., Altman, S., Long, B., & Frank, M. C. (2018). Data availability, reusability, and analytic reproducibility: Evaluating the impact of a mandatory open data policy at the journal Cognition. Royal Society Open Science, 5, 180448.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container] [blog1] [blog2] [comment]

Hardwicke, T. E. & Ioannidis, J. P. A. (2018). Populating the Data Ark: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles. PLOS ONE, 13, e0201856.
[paper] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Henry Tessler, M., Peloquin, B., & Frank, M. C. (2018). A Bayesian decision-making framework for replication. Behavioral and Brain Sciences, 41, e132.
[paper] [preprint]

Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Hofelich Mohr, A., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018). A practical guide for transparency in psychological science. Collabra: Psychology, 4, 20.
[paper] [materials]

2017

Cristea, I. A., Naudet, F., Shanks, D. R., & Hardwicke, T. E. (2017). Post-retrieval Tetris should not be likened to a ‘cognitive vaccine’. Molecular Psychiatry, 23, 1972-1973.
[paper] [pdf] [data] [analysis-code]

2016

Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to acknowledge open practices: A simple, low cost, effective method for increasing transparency. PLOS Biology, 14, e1002456.
[paper] [preregistration] [materials] [data] [analysis-code]

Hardwicke, T. E. & Shanks, D. R. (2016). Reply to Walker and Stickgold: Proposed boundary conditions on memory reconsolidation will require empirical verification. PNAS, 113, E3993-E3994.
[paper] [pdf]

Hardwicke, T. E., Mahdi, T., & Shanks, D. R. (2016). Post-retrieval new learning does not reliably induce human memory updating via reconsolidation. PNAS, 113, 5206-5211.
[paper] [pdf] [materials] [data] [analysis-code] [comment] [media]

Vadillo, M. A., Hardwicke, T. E., & Shanks, D. R. (2016). Selection bias, vote counting, and money priming effects: A comment on Rohrer, Pashler, and Harris (2015) and Vohs (2015). Journal of Experimental Psychology: General, 145, 655-663.
[paper] [pdf] [data] [media]

Hardwicke, T. E. (2016). Persistence and plasticity in the human memory system: An empirical investigation of the overwriting hypothesis. PhD Thesis.
[thesis] [data] [analysis-code] [osf]

Before 2016

Baker, R., Dexter, M., Hardwicke, T. E., Goldstone, A., & Kourtzi, Z. (2014). Learning to predict: Exposure to temporal sequences facilitates prediction of future events. Vision Research, 99, 124-133.
[paper] [pdf]

Ludwig, C. J. H., Farrell, S., Ellis, L. A., Hardwicke, T. E., & Gilchrist, I. D. (2012). Context-gated statistical learning and its role in visual-saccadic decisions. Journal of Experimental Psychology: General, 141, 150-169.
[paper] [pdf]

Hardwicke, T. E., Jameel, L., Jones, M., Walczak, E. J., & Magis-Weinberg, L. (2014). Only human: Scientists, systems, and suspect statistics. Opticon1826, 16, 1-12.
[paper] [pdf]

Hardwicke, T. E. (2012). Biological Psychology. Psychology Learning & Teaching, 11, 112-113.
[paper]

Tom Hardwicke

Research Fellow

University of Melbourne



Publications


Preprints / Stage One Registered Reports

Hardwicke, T. E., Schiavone, S., Clarke, B., Vazire, S. (accepted in principle). Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary [Stage One Registered Report]. Peer Community in Registered Reports.
[preregistration]

Frank, M. C., Braginsky, M., Cachia, J., Coles, N., Hardwicke, T. E., Hawkins, R., Mathur, M., & Williams, R. (in preparation). Experimentology: An open science approach to experimental psychology methods. MIT Press.
[open access book]

Fernández, L. M., Hardwicke, T. E., & Vadillo, M. A. (preprint). Retracted papers clinging on to life: An observational study of post-retraction citations in psychology. PsyArXiv.
[preprint] [data] [analysis-code]

2024

Dulitzki, C., Crane, S. M., Hardwicke, T. E., & Ioannidis, J. P. A. (in press). Expanding the Data Ark: An attempt to make the data from highly cited social science papers publicly available. Royal Society Open Science.
[preprint] [data] [preregistration] [materials] [analysis-code] [reproducible-container]

2023

Hardwicke, T. E., & Vazire, S. (2023). Transparency is now the default at Psychological Science [editorial]. Psychological Science, 09567976231221573.
[paper]

Thibault, R. T., Kovacs, M., Hardwicke, T. E., Sarafoglou, A., Ioannidis, J. P., & Munafo, M. R. (2023). Reducing bias in secondary data analysis via an Explore and Confirm Analysis Workflow (ECAW): A proposal and survey of observational researchers. Royal Society Open Science, 10, 230568.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code]

Hardwicke, T. E., & Wagenmakers, E.-J. (2023). Reducing bias, increasing transparency, and calibrating confidence with preregistration. Nature Human Behaviour, 7, 15-26.
[paper] [preprint]

Hardwicke, T. E., Salholz-Hillel, M., Malički, M., Szűcs, D., Bendixen, T., Ioannidis, J. P. A. (2023). Statistical guidance to authors at top-ranked journals across scientific disciplines. The American Statistician. 77(3), 239-247.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

2022

Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Tzavella, L., Bendixen, T., Handcock, S. A. Köneke, V. E., Ioannidis, J. P. A. (2022). Post-publication critique at top-ranked journals across scientific disciplines: A cross-sectional assessment of policies and practice. Royal Society Open Science, 9, 220139.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container] [media1] [media2] [media3]

TARG Meta-Research Group & Collaborators [Thibault, R. T., Hardwicke, T. E., Clark, R. W. A., Pennington, C. R., Nilsonne, G., O’Mahony, A., Drax, K., Thompson, J., Munafò, M. R.] (2022). Discrepancy review: A feasibility study of a novel peer review intervention to reduce undisclosed discrepancies between registrations and publications. Royal Society Open Science, 9, 220142.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M., Rohrer, J., Romero, F., Scheel., A., Scherer, L., Schönbrodt, F., & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73, 719-748.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., & Ioannidis, J. P. A. (2022). Estimating the prevalence of transparency and reproducibility-related research practices in psychology (2014-2017). Perspectives on Psychological Science, 17(1), 239-251.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2021

Hardwicke, T. E., Szucs D., Thibault, R. T., Crüwell, S., van den Akker, O. R., Nuijten, M. B., & Ioannidis, J. P. A. (2021). Citation patterns following a strongly contradictory replication result: Four case studies from psychology. Advances in Methods and Practices in Psychological Science, 4, 1-14.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Bohn, M., MacDonald, K., Hembacher, E., Nuijten, M. B., Peloquin, B. N., deMayo, B., Long, B., Yoon, E. J., & Frank, M. C. (2021). Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: An observational study. Royal Society Open Science, 8, 201494.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2020

Hardwicke, T. E., & Goodman, S. N. (2020). How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey. PLOS ONE, 15, e0239598.
[paper] [preprint] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020). Calibrating the scientific ecosystem through meta-research. Annual Review of Statistics and its Application, 7, 11-37.
[paper] [preprint]

Ferrero, M., Hardwicke, T. E., Konstantinidis, E., & Vadillo, M. A. (2020). The effectiveness of refutation texts to correct misconceptions among educators. Journal of Experimental Psychology: Applied, 26, 411-421.
[paper] [preprint] [materials] [data] [analysis-code]

Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020). An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017). Royal Society Open Science, 7, 190806.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2019

Hardwicke, T. E., & Ioannidis, J. P. A. (2019). Petitions in scientific argumentation: dissecting the request to retire statistical significance. European Journal of Clinical Investigation, 49, e13162.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container] [comment1] [comment2] [comment3]

Nosek, B. A., Beck, E. D., Campbell, L., Flake, J. K., Hardwicke, T. E., Mellor, D. T., van ’t Veer, A. E., & Vazire, S. (2019). Preregistration is hard, and worthwhile. Trends in Cognitive Sciences, 23, 815-818.
[paper] [preprint]

Hardwicke, T. E., Frank, M. C., Vazire, S., & Goodman, S. N. (2019). Should psychology journals adopt specialized statistical review? Advances in Methods and Practices in Psychological Science, 2, 240-249.
[paper] [materials] [data] [analysis-code] [reproducible-container] [media]

2018

Hardwicke, T. E. & Ioannidis, J. P. A. (2018). Mapping the universe of registered reports. Nature Human Behaviour, 2, 793–796.
[paper] [materials] [data] [analysis-code] [reproducible-container] [preprint] [comment] [comment2]

Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., Lenne, R. L., Altman, S., Long, B., & Frank, M. C. (2018). Data availability, reusability, and analytic reproducibility: Evaluating the impact of a mandatory open data policy at the journal Cognition. Royal Society Open Science, 5, 180448.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container] [blog1] [blog2] [comment]

Hardwicke, T. E. & Ioannidis, J. P. A. (2018). Populating the Data Ark: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles. PLOS ONE, 13, e0201856.
[paper] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Henry Tessler, M., Peloquin, B., & Frank, M. C. (2018). A Bayesian decision-making framework for replication. Behavioral and Brain Sciences, 41, e132.
[paper] [preprint]

Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Hofelich Mohr, A., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018). A practical guide for transparency in psychological science. Collabra: Psychology, 4, 20.
[paper] [materials]

2017

Cristea, I. A., Naudet, F., Shanks, D. R., & Hardwicke, T. E. (2017). Post-retrieval Tetris should not be likened to a ‘cognitive vaccine’. Molecular Psychiatry, 23, 1972-1973.
[paper] [pdf] [data] [analysis-code]

2016

Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to acknowledge open practices: A simple, low cost, effective method for increasing transparency. PLOS Biology, 14, e1002456.
[paper] [preregistration] [materials] [data] [analysis-code]

Hardwicke, T. E. & Shanks, D. R. (2016). Reply to Walker and Stickgold: Proposed boundary conditions on memory reconsolidation will require empirical verification. PNAS, 113, E3993-E3994.
[paper] [pdf]

Hardwicke, T. E., Mahdi, T., & Shanks, D. R. (2016). Post-retrieval new learning does not reliably induce human memory updating via reconsolidation. PNAS, 113, 5206-5211.
[paper] [pdf] [materials] [data] [analysis-code] [comment] [media]

Vadillo, M. A., Hardwicke, T. E., & Shanks, D. R. (2016). Selection bias, vote counting, and money priming effects: A comment on Rohrer, Pashler, and Harris (2015) and Vohs (2015). Journal of Experimental Psychology: General, 145, 655-663.
[paper] [pdf] [data] [media]

Hardwicke, T. E. (2016). Persistence and plasticity in the human memory system: An empirical investigation of the overwriting hypothesis. PhD Thesis.
[thesis] [data] [analysis-code] [osf]

Before 2016

Baker, R., Dexter, M., Hardwicke, T. E., Goldstone, A., & Kourtzi, Z. (2014). Learning to predict: Exposure to temporal sequences facilitates prediction of future events. Vision Research, 99, 124-133.
[paper] [pdf]

Ludwig, C. J. H., Farrell, S., Ellis, L. A., Hardwicke, T. E., & Gilchrist, I. D. (2012). Context-gated statistical learning and its role in visual-saccadic decisions. Journal of Experimental Psychology: General, 141, 150-169.
[paper] [pdf]

Hardwicke, T. E., Jameel, L., Jones, M., Walczak, E. J., & Magis-Weinberg, L. (2014). Only human: Scientists, systems, and suspect statistics. Opticon1826, 16, 1-12.
[paper] [pdf]

Hardwicke, T. E. (2012). Biological Psychology. Psychology Learning & Teaching, 11, 112-113.
[paper]