Introducing Doublethink: joint Bayesian-frequentist model-averaged hypothesis testing

This week Nick Arning, Helen Fryer and I released two related preprints describing a new method called Doublethink, and its application to identifying risk factors for COVID-19 hospitalization in UK Biobank:

Doublethink: Bayesian-frequentist model-averaged hypothesis testing

Doublethink enables joint Bayesian and frequentist hypothesis testing when there is model uncertainty by interconverting Bayesian posterior odds and classical (frequentist) p-values. It has broad implications because (i) it reveals connections between the Bayesian approach to model averaging and the classical approach to multiple testing, and (ii) it brings the benefits of Bayesian model averaging to classical statistics.
Doublethink addresses two fundamental problems in hypothesis testing:
  1. In classical tests, the statistical evidence that one variable directly affects an outcome generally depends on which other variables are assumed to directly affect it.
  2. In Bayesian tests, the statistical evidence that one variable directly affects an outcome depends on the prior assumptions.
These issues are addressed by computing p-values from Bayesian model-averaged posterior odds, which (1) account for model uncertainty and (2) are theoretically invariant to prior assumptions, assuming large sample sizes.
Doublethink simultaneously controls the frequentist family-wise error rate (FWER) and the Bayesian false discovery rate (FDR). It builds on Johnson's Bayesian tests based on likelihood ratio statistics, and Karamata's theory of regular variation.

Identifying direct risk factors in UK Biobank with Doublethink

We applied Doublethink to identify direct risk factors for COVID-19 hospitalization in UK Biobank. This is a well-studied problem but we took an 'exposome-wide' approach in which we evaluated whether 1,900 variables measured in the UK Biobank each affected the outcome. This is still an under-utilized approach in epidemiology, which usually focuses on candidate risk factors.
Exposome-wide approaches have potential benefits over candidate risk factor approaches, including:
  • The ability to discover unexpected results.
  • Stringent control for multiple testing.
  • Avoidance of bias in choosing candidate risk factors or deciding to publish.
However, we only studied the direct effects of variables on the outcome. This means we cannot make statements about the total (direct and indirect) effects of a variable, e.g. smoking, on the outcome, which are needed in applications like assessing potential interventions.
We identified individual variables and groups of variables that were 'exposome-wide significant' at 9% FDR and 0.05% FWER, after accounting for the direct effects of all other variables.

Comparing our results to over 100 published studies of COVID-19 in UK Biobank, we
  • Recapitulated several commonly reported direct risk factors, e.g. age, sex, and obesity.
  • Excluded others, e.g. diabetes, cardiovascular disease, and hypertension, which might be mediated through other variables that measure general comorbidity.
  • Identified some infrequently reported direct risk factors, both individually, e.g. lung infection, and as groups, e.g. constipation/urinary tract infection, which might reflect underlying kidney disease.
The ability to test groups of variables, which increases sensitivity, was one of the benefits of Doublethink's model-averaging approach. It is particularly helpful in large biobanks that measure thousands of variables, because correlation between variables is pervasive, and can dilute the significance of individual variables that measure similar phenomena, like the numerous types of deprivation index. It serves as a flexible alternative to pre-analysis variable filtering algorithms, while controlling the risk of false positives by pre-defining significance thresholds for all possible tests.
To read more, please check out the preprints here and here.

Presentation: Genome-wide Association Studies of COVID-19

An online recording of the talk about Genome-wide Association Studies of COVID-19 at the UK Biobank 2020 meeting on 23 June 2020. The full conference is also online.


The group’s research response to COVID-19

This is an update on the group's research response to the COVID-19 pandemic. As an infectious disease group we have been keen to contribute to the international research effort where we could be useful, while recognising the need to continue our research on other important infections where possible.

  • Bugbank. Thanks to a pre-existing collaboration between our group, Public Health England and UK Biobank, we were in a position to help rapidly facilitate COVID-19 research via SARS-CoV-2 PCR-based swab test results. Beginning mid-March, we worked to provide regular (usually weekly) updates of tests results, which were made available to all UK Biobank researchers beginning April 17th. This is one of several resources on COVID-19 linked to UK Biobank. Beginning in May we provided feeds to other cohorts: INTERVAL, COMPARE, Genes & Health and the NIHR BioResource. We provide updates on this work through the project website www.bugbank.uk. We have published a paper describing the dynamic data linkage in Microbial Genomics (press release). Key collaborators in this project are Jacob Armstrong (Big Data Institute) Naomi Allen (UK Biobank) and David Wyllie and Anne Marie O'Connell (Public Health England).


  • Epidemiological risk factors for COVID-19. Graduate student Nicolas Arning and I are developing an approach to quantify the effects of lifestyle and medical risk factors for COVID-19 in the UK Biobank that accounts for inherent uncertainty in which risk factors to consider. The new method employs the harmonic mean p-value, a model-averaging approach for big data that we published previously. We are in the process of evaluating the performance of the approach, comparing it to machine learning, and interpreting the results.

  • Antibody testing for the UK Government. Postdoc Justine Rudkin has been working in the lab with Derrick Crook, Sir John Bell and others to measure the efficacy of antibody tests for the UK Government. They have tested many hundreds of kits to establish the sensitivity and specificity of the tests to help evaluate the utility of a national testing programme. This work was crucial in demonstrating the limitations of early blood-spot based tests, and the credibility of subsequent generations of antibody tests. The work has been published in Wellcome Open Research.


Work on other infections that has continued during the lockdown. Postdoc Sarah Earle continues research into pathogen genetic risk factors for diseases including tuberculosis and meningococcal meningitis, while Steven Lin has continued to pursue work on hepatitis C virus genetics and epidemiology. Many of our close collaborators are infection doctors and they have of course been recalled to clinical duties. Laboratory work in the group has been severely disrupted, particularly several of Justine's Staphylococcus aureus projects. We are keen to pick up on those projects where we left off when the chance arrives.