Multi-analyte assays and early detection of common cancers

The clinical use of biomarker assays to identify cancers, at a sufficiently early enough stage to enable a surgical cure, remains the “Holy Grail” of cancer diagnostics (1). Progress in various bio-assays, especially with liquid biopsies using molecular genetic markers (2), has excited the research community although their adoption in routine clinical practice remains elusive.

In January this year, Cohen and colleagues published the results of a multi-analyte assay (CancerSEEK) that combines 8 well established protein markers derived from the literature and a panel of over 1,000 mutant genetic variants identified from numerous cancer tissue samples (3). While the results of this study provide further support for a “proof of concept” for this approach (4), we remain unconvinced that the assay will advance the clinical management of oncology patients. On the positive side, the assay has a very high specificity (>99%) and has biological plausibility in utilising known tumour protein-based markers and free DNA variants that correlated with driver mutations from a small set of tumour tissues (3). However, several methodological issues remain obstacles to routine clinical use.

First, the performance statistics of sensitivity and specificity are meaningless when derived from case-control data (5) where cancer incidence, subject selection, survival effects, along with other sources of possible bias, may have considerable influence on the utility of the assay. To establish any sort of predictive utility, the assay must be tested in a cohort study, better reflecting the screening population of interest (6). This is particularly the case in the study by Cohen and colleagues as they did not include people with advanced stage cancers as part of their case group. Worse, the controls are likely to be a biased population and, along with the cancer cases, not representative of the intended screening population (5). Apart from gender (for breast and ovarian cancer in women) and smoking history (for lung cancer in current and former smokers), it is difficult to imagine just who would best represent the intended screening population. This is compounded by the fact that this assay purportedly identifies 8 common but distinct cancer subtypes. This has relevance to the assay’s performance in terms of positive and negative predictive values and thus its ability to provide clinically useful prediction information. When the incidence of the cancer is low in the screening population, the negative predictive rate will be high based on low incidence alone. The results of the Cohen study (3) require replication in the appropriate screening population of interest, with further evidence that assay variation and analytical utility is established for this multi-analyte assay. Such a replication would include the finalised “locked down” algorithm where the results of the individual biomarker assays would be combined to generate a positive or negative result or a composite score.

Second, it is concerning that the sensitivity of the test is dependent on the histological type of cancer and also its stage. The total number of cancer subjects (n=1,005) while large, is made up of 8 different cancer subtypes with very different biology and potential for over diagnosis (3). For the individual cancer types, the numbers are too small to really understand how CancerSEEK truly functions or would contribute in a meaningful way to clinical decisions. The description of the selection and characteristics of this cohort is limited. The sensitivity was greatest in aggressive cancers of the ovary and liver and least in breast cancer and lung cancers which are far more common. Moreover, the sensitivity of the assay was only 40% in stage I and 70% in stage II and III disease (3). These findings may speak to the variation in biology according to histological type and stage. Like developers of other molecular assays, there appears to be a blind faith of acceptance that tumours do not drastically alter their signature as they progress through different clinical stages, releasing cells, DNA or proteins into the circulation as part of this process (7-9). Without serial measurement of the CancerSEEK assay, it is unclear just how consistently the free DNA mutation detection component may alter the performance of the assay overall. A further issue is the reproducibility of the individual assay components where both the presence and detection level of each marker may be subject to considerable variation in both the clinical and laboratory setting. False positive and false negative rates for each biomarker must be considered.

Third, as the true performance of this assay, as demonstrated in a large prospectively collected cohort is unknown, it is hard to know how this assay will augment existing early detection methods such as CT/MRI screening (breast, ovary, liver, pancreas and lung) and endoscopic screening (oesophagus, stomach and colon). As any positive assay test will require imaging or endoscopic follow-up, it is questionable just how this assay will simplify or improve existing clinical pathways where regular imaging or endoscopy is already recommended. While the apparent specificity of CancerSEEK is high (>99%), the sensitivity is very variable, particularly where stage I cancers are concerned (3). To date targeted screening is based on gender, age and family history for many of these cancers (10). For breast and colon, routine genetic testing provides another means to target people outside the usual gender-age based screening populations (11). One aspect that the investigators are not able to model is the ratio of cancers detected at a surgically-resectable stage versus those without cancer who are false-positives on the assay (3). Although a 1% false positive rate sounds acceptable, it depends on the cancer incidence in the screening population where the assay is being used. For example, in the CT screening studies for lung cancer, the annual lung cancer diagnosis rate is less than 1% and only a fraction of these will translate into lung cancer deaths averted from early detection and treatment. We remain concerned that screening using this assay will not necessarily address the issues of over diagnosis and overtreatment (12). Issues of cost-effectiveness are equally unclear here. This is relevant as the assay has limited capacity to determine just where the cancer is present (organ or metastases) and just which tests are required to verify it (imaging, ultrasound, or endoscopy. The real utility of this test requires confirmation in several replication studies where the performance of the biomarker assay has been properly assessed in the screening populations of interest.

Fourth, we suggest that the methodology to develop a bioassay identifying those at risk of lung cancer, or lung cancer in its earliest stage is in danger of producing an entirely false signal through confounding (13,14). In the context of germ-line mutations, we have previously reported that undetected chronic obstructive pulmonary disease (COPD) is problematic in studies investigating biomarkers in lung cancer (15). This is because the prevalence of COPD is near always two-fold greater in lung cancer cases than the controls. Unless the case-control studies stratify on the presence of COPD, they will always be at risk of a classic confounding effect where their biomarker signal may represent an association with COPD and not lung cancer. Even in a stable assay like germline mutations inherited from birth (e.g., single nucleotide polymorphisms), the potential for mistaking an association with lung cancer instead of COPD is considerable. A similar issue applies for any of the expression-based biomarkers which may also be spuriously linked to lung cancer (instead of COPD) due to this fundamental difference in the prevalence of COPD in lung cancer case-control studies (16). We have recently shown that between 50–75% of lung cancer cases have underlying COPD, further complexity is added to the expression markers from chronic bacterial colonisation in the lung cancer, smoking effects and recent use of inhaled or oral corticosteroids. The last are standard treatments in patients with moderate to severe COPD and may alter gene expression. This means any lung cancer biomarker study must account for these factors before claiming their assay is actually identifying lung cancer rather than some other clinical phenotype associated with COPD.

Lastly we remain sceptical about the clinical utility of biomarkers like CancerSEEK in the context of distinguishing benign from malignant lesions. Blood based assays (17,18) are by their very essence a composite of blood markers from all parts of the body and those presumably cannot distinguish just which nodule of concern may be malignant and which nodules are actually benign. Such a distinction in lung cancer will have to come down to serial imaging examining volumetric changes. Even sensitive assays revealing the presence of tumour DNA will not help localise it, particular in lung cancer where multiple nodules may be present (19).

In summary, this CancerSEEK assay requires considerable validation in large prospective studies in order to confirm a clinical utility in the screening populations of interest and within the existing screening environment in which it is being considered.