In order for the public to have faith in the conclusions of scientists it is important that the methods they employ are robust and transparent. This is especially important for controversial topics with major implications for mental health. The public should rightly demand that such findings are not contingent on the beliefs of the scientists, their particular methods, computational quirks or simple accident. This is of particular relevance when total transparency is not possible because the data is sensitive.

This study addresses the question of robustness by taking a controversial question, “Is there an association between screen time and depression and anxiety?”, recruiting teams of independent data analysts and looking at how they answer the question using the same data, effectively ‘crowd-sourcing’ the data analysis.

The study will use the answers teams provide to this controversial question to answer further questions such as: “Do the methods used to answer the question influence the result and if so by how much?” and, “Do the beliefs and particular expertise of the analysts influence their results?”. To do this, we will use a statistical technique called a ‘multiverse analysis’, whereby reasonable alternatives to choices made by the teams, during the data analysis, are explored and recorded to see how sensitive the results were to the choices made.

To ensure the study is transparent we will investigate the use of anonymised data. Using anonymised data is controversial as the anonymisation process may erase important features of the data. This study will ask “Does the anonymisation process affect the results and if so by how much?”.

The answer to all these questions will help us understand how scientists arrive at answers to controversial questions and whether crowd sourced analysis and data anonymisation techniques can ensure findings are robust and transparent.

Finally, our study will challenge the teams to come up with interesting ways to visualise the answers to these questions in exchange for a prize. Visualising the teams’ answers, along with how robust they are, in a clear, accessible way will be important to help communicate complex results both for this study and in the future.