B3615 - Research-on-research Blinded data analysis to improve the robustness and reproducibility of health research - 08/10/2020

B number: 
Principal applicant name: 
Robert Thibault | University of Bristol (United Kingdom)
Prof Marcus Munafo
Title of project: 
Research-on-research: Blinded data analysis to improve the robustness and reproducibility of health research
Proposal summary: 

Bias in scientific research can lead to research waste and useless or even harmful health care and policy interventions. To reduce bias, researchers commonly employ experimental designs that blind both participants and outcome assessors. Data analysts, however, are rarely blinded. Here, we propose to test whether blinding data analysts improves the reliability of published research findings. To execute the study, we will randomize researchers who request the Avon Longitudinal Study of Parents and Children (ALSPAC) dataset, and consent to partake in the research, to receive either data from only 10% of participants, or data from all participants. Researchers who receive 10% of the data will develop their analysis based on this subset and be asked to register their analysis plan. After registering their analysis plan, the full dataset will be provided. We call this “Blind Access”. Researchers who receive the full dataset will not be required to register their analysis. We call this “Standard Access”. We will then compare the reliability of the published findings between the Blind Access and Standard Access groups.

Impact of research: 
1. If we find that blinded data analyses are less biased than non-blinded analyses, this will be the first substantial empirical evidence to support blinded data analysis in health research. This finding would promote researchers to conduct blinded data analyses and in turn increase the robustness and reproducibility of published research findings. 2. We will garner information on how best to implement blinded data analysis. This will be useful information for data managers, including ALSPAC. 3. If we find that blinded data analyses and non-blinded data analyses are similarly biased, then we will have documented that this initiative is not necessarily worth pursuing.
Date proposal received: 
Saturday, 12 September, 2020
Date proposal approved: 
Tuesday, 15 September, 2020
Statistics/methodology, This project is not concerned with a disease or condition. We are conducting 'research on research' to improve the robustness and reproducibility across observational health research., Statistical methods, meta-research blind data analysis