Randomised controlled clinical trials are the gold standard approach to understanding the benefits of new medical treatments. These work by randomly allocating a patient recruited into the trial to receive either the new treatment or an existing or placebo treatment, and comparing the outcomes from the two groups of patients. The randomisation step is key to allowing a robust, fair and unbiased comparison of the two treatments.  

Recently, there is increasing interest in using alternative approaches based upon using data collected from electronic health records during routine medical care – so called Real World Data (RWD). One idea is, instead of a randomised trial, to run a single arm trial where every patient receives the new treatment and compare the outcomes from this group with outcomes with a RWD group of patients who have received the existing treatment – an external control arm. Directly performing these comparisons would lead to a biased result as, without the benefit of randomisation, the characteristics of the two groups of patients receiving the two treatments may be different which would also affect their outcomes. This makes it unclear whether to interpret any differences in outcomes as being due to the treatment or just a consequence of the underlying differences between the two groups of patients. 

Methods such as propensity scoring allow adjustment for the differences in patient characteristics between treatment groups, but these are imperfect and can still retain biases and cause additional levels of uncertainty.  We have developed a method which uses a set of historic studies to understand the impact of using an external control arm instead of a randomised control arm and incorporate the appropriate levels of bias and uncertainty into the analysis of a new study, giving more robust and trustworthy results.