Biomarkers in lung cancer screening: the importance of study design

In most randomised controlled trials of low radiation dose computed tomography (LDCT) screening for lung cancer, eligibility has been determined by age and tobacco smoking criteria[1, 2]. A number of multivariable risk prediction models have been developed that are more sensitive and specific, but are still heavily dependent on smoking and age[3]. Some have been used successfully in trials and pilot programmes and have yielded higher detection rates, although they may also select people with more comorbidities[4–6]. Using biomarkers to select people for lung cancer screening is an important area for research because it offers the possibility to identify those at high risk who aren’t eligible for screening by conventional criteria while avoiding further investigation in those who do not have malignant disease. This means that the impact of the intervention as a whole could be greater (more lives saved) and harms, due to the screening of low-risk people or those without cancer, could be avoided. Two biomarkers intended for use in lung screening are presented in this issue of the ERJ.

Sullivan et al describe the use of the “EarlyCDT-Lung”, a 7-autoantibody blood test to detect lung cancer which was evaluated in a randomised trial in Scotland (Early Lung Cancer Detection in Scotland, ECLS)[7]. Just over twelve thousand people who were at high risk of developing lung cancer by age and smoking criteria were randomised 1:1 to an intervention arm or usual care. In the intervention arm, participants were offered EarlyCDT-Lung, and those who were test-positive (≈10%; n=598) went on to have an initial chest X-ray followed by LDCT every 6 months for 24 months. The researchers report a lower proportion of late-stage cancers in the active arm, 33 of 56 (59%) compared with 52 of 71 (73%) in the control arm (HR 0.64 (95% CI 0.41, 0.99)). Seven patients with unclassified stage in the control arm were assumed to be diagnosed at late stage, whereas all in the intervention arm had known stage. This assumption favoured the primary outcome of the study, making the difference in late stage disease statistically significant. The authors reported sensitivity estimates 2 years after recruitment, for lung cancer overall and stratified by early/late stage disease.

The most important limitation in the research design, which the researchers acknowledge, was that neither the participants who tested negative by EarlyCDT-Lung, nor the participants in the control arm, were offered LDCT. This has three implications for interpreting the results presented. Firstly, the ability of LDCT screening to diagnose lung cancer at earlier stages is well-established, with 75% or more diagnosed at stage I/II[1, 2, 5]. Since LDCT screening was only offered to those who tested positive by the biomarker, it is not possible to determine whether EarlyCDT-Lung made any improvement to earlier detection. Indeed, a test generating a result at random would be expected to result in earlier diagnosis in the intervention group, simply by triggering a CT scan in a proportion of these participants.

Secondly, assessing overall sensitivity for lung cancer after only 2 years follow-up is likely to overestimate the sensitivity of the EarlyCDT-screening intervention (reported as 32%). This is because lung cancer has a sojourn time (interval between development and presentation of cancer) of 4–6 years[8] – indeed ‘catch-up’ of cancer presentations in the non-LDCT-screened group in the National Lung Screening Trial continued for 9 years[9]. Therefore, one would expect continued identification of lung cancers in the test negative group over subsequent years of follow-up, thereby progressively reducing the sensitivity (i.e. the proportion identified by the test). A more accurate estimate of the sensitivity of the EarlyCDT-Lung would have been obtained if all participants in the screened arm had received an exit LDCT, because LDCT detects many of these cancers. Furthermore, a strange aspect of the ECLS results is that more lung cancers were recorded in the control group (n=71) than in the intervention group (n=56). LDCT typically detects a 2.2-fold (NELSON), 2.4 (MILD) to 3.1-fold (LUSI) greater number of cancers in the LDCT arm compared with the no-screening control arm in the first 2 years[2, 10, 11]. In the ECLS, 71 lung cancers were detected over 2 years in the control arm. If LDCT had been offered to all intervention-arm participants we would expect at least 156 (2.2 × 71) lung cancers to be detected. Among these, approximately 117 (75%) would be expected to be stage I/II. Since only 12 participants had stage I/II lung cancer in the EarlyCDT positive group, the test sensitivity would be 10%. The ECLS reported sensitivity of 52% for stage I/II disease is, therefore, likely to be a substantial over-estimate and entirely due to the fact that only test positive partcipants had LDCT.

Finally, measuring sensitivity for early versus late stage disease (in the absence of screening for the test negative group) is flawed. Any comparison of sensitivities between these two groups is biased by the fact that the cancers in the test positive group will generally be early stage cancers detected by LDCT screening, whereas those in the test-negative participants will present at a later time-point and therefore likely to be more advanced stage. Thus, EarlyCDT-Lung must be compared directly with existing methods for selecting people for screening (and should include LDCT for test negative participants) before any conclusions about its utility can be drawn. At worst, it could cause considerable harm by excluding people at high risk who would be selected by currently available and validated multivariable models. It is noted that in ECLS 90% of participants were effectively excluded from LDCT by virtue of the negative blood test.

A number of other biomarkers for lung screening are in various stages of development[12]. However, biomarkers with promising early results have often failed in subsequent research [13, 14], and this is often explained by differences in design of studies between development and validation. For initial development studies, biomarkers are typically tested for differences between cases and controls. A common pitfall is that biomarkers are capable of distinguishing between cases and controls at the time of diagnosis, but fail when tested in pre-diagnostic samples, or fail in the setting of early-stage disease. Another pitfall is that cases and controls are often poorly matched for important factors, or even derived from separate populations. When later validated in a population of at-risk people, the test performs less well. Therefore, initial control populations should be derived from the same cohort as the patients with cancer and selected in a way that simulates the intended downstream use of the biomarker. For example, biomarkers intended for use in selection for screening should be tested in a group potentially eligible for screening. The biomarker should then be compared with the gold-standard method for the clinical application of interest, for example, to risk prediction models if intended for use in defining eligibility for screening.

Also in this issue of ERJ, Gaga et al report on a novel methylation-based biomarker, Lung EpiCheck, which shows a sensitivity of 85% for early stage lung cancer with a specificity of 64% when combined with other risk factors[15]. This new test is promising but, quite rightly, the authors recommend further prospective trials to establish how the test performs in populations eligible for screening. As the authors acknowledge, there were important differences in the cases and controls in the European dataset due to retrospective selection of cases and identification of controls from an outpatient population. Furthermore, the controls did not have follow-up or LDCT to look for early-stage lung cancer. In the single-centre Chinese validation set, healthy controls were used. Thus, the high AUC of 0.94 is likely a result of the marked difference between cases and controls and will reduce when the biomarker is used to select people for screening where cases and controls are from the same population. One important step will be to use the test alongside LDCT screening and compare the performance with that of currently validated risk models.

Biomarkers offer a promising approach to improve the impact of screening, because unlike conventional risk prediction models, they may be able to identify people who have high risk for lung cancer without concurrently elevated risk for competing causes of death related to age and smoking. However, it is important to establish whether a positive biomarker is also a marker for poor outcome, as biomarkers may detect more aggressive disease with a poor prognosis[16]. For example, some cell-free DNA tests may be correlated with more aggressive tumours[17–19]. Conventional cancer screening reduces mortality mainly by detecting less aggressive tumours (length time effect) so although detecting aggressive cancers early may improve treatment options, the impact on mortality may be less. Sullivan et al show that a positive Early CDT, based on measuring the immune response to lung cancer, in combination with LDCT showed a non-significant trend to reduced mortality. This may merit further study.

The ultimate aim is to have a biomarker that is inexpensive and can be used as the screening test, with LDCT being the downstream investigation, in a similar way to colorectal screening where the faecal immunochemical test or faecal occult blood test is followed by colonoscopy. It is important that the test identifies not only people who will develop disease, but specifically people who can benefit.

Many promising biomarkers are emerging,[20–22] but the onus is on the research community to test each one using robust methodology. Studies must be designed in such a way that the added contribution of the biomarker can be clearly identified and quantified, in direct comparison to the gold-standard method for the task that the biomarker purports to accomplish. This is an exciting and promising field, but the excitement must be met with carefully designed, definitive studies.