Large Clinical Databases for the Study of Lung Cancer: Making Up for the Failure of Randomized Trials

Introduction

Although randomized controlled trials (RCT) represent the optimal study design for assessing the efficacy of clinical interventions, the practice of thoracic surgery has historically been significantly influenced by case series and single-institution observational cohort studies¹ due to the well-described difficulties of performing RCTs.² Researchers in the United States and Europe are increasingly utilizing large-scale population-based administrative databases^3-5 and clinical registries^6-10 to perform retrospective cohort studies to answer questions regarding the efficacy, cost and complications of interventions, to study quality improvement, to assess the treatment of rare conditions, and to evaluate national practice patterns. These studies can significantly enhance quantitative evidence regarding lung cancer treatment when performed by investigators who have clinical expertise in lung cancer with a thorough understanding of both the advantages and limitations of the database being utilized, through collaboration with experts in advanced statistical techniques.

Advantages of Analyzing Large Clinical Databases

Well-performed RCTs that minimize bias in patient selection provide the highest grade evidence to guide clinical practice. Although thoracic surgeons have performed extremely influential RCTs in the past,¹¹ many barriers to the use of RCT have generally limited the use of RCTs to investigate surgical interventions for lung cancer, including inadequate equipoise amongst surgeons and patients, complex institutional administrative requirements for RCT approval, difficulty in obtaining adequate, and inadequate research infrastructure in many programs.² Moreover, clinical trials involving thoracic surgical patients are commonly terminated early due to poor accrual.^12-14 As a result, most treatment guidelines for lung cancer surgery are based on experts' interpretations of data from case series and single-institution observational cohort studies.¹ The evidence from these study designs are more susceptible to bias and have lower “grades” than RCTs.

Well-done studies of large-scale population-based datasets can bridge the gaps in evidence that result from the dearth of data from RCTs. A major strength of clinical database studies is that their sample sizes are typically orders of magnitude larger than those of randomized controlled trials and single institution case series. For example, the National Cancer Data Base (NCDB) receives over one million cancer case reports annually⁶, and the NCDB contains data on more than 1.5 million patients with non-small cell lung cancer (NSCLC).¹⁵ The Medicare claims database has data on more than 45 million patients.¹⁶ The size of the available patient populations studied provide unprecedented statistical power to researchers and enables the performance clinically meaningful subset analyses that would be difficult to do with RCTs. For example, in the Lung Cancer Study Group (LCSG) randomized trial of lobectomy versus limited resection for T1 N0 NSCLC, investigators had found that limited resection was associated with increased locoregional recurrence for patients with tumors up to 3 cm in size but were unable to make any conclusions for smaller tumors due to lack of statistical power.¹¹ Two well-designed ongoing randomized trials – the Cancer and Leukemia Group B (CALGB) 140503 and the Japanese Clinical Oncology Group (JCOG) 0802 studies ^{17, 18} – are focusing on tumors smaller than 2 cm, but may not have sufficient power to evaluate smaller tumors or compare differences between wedge vs segmentectomy vs lobectomy due to sample size and patient accrual issues. With the statistical power that comes from large clinical datasets, investigators have been able to evaluate the impact of lobectomy vs limited resection on survival for patients with smaller tumors less than 2 cm^{19, 20} and less than 1 cm²¹ as well as evaluate the differences in long-term outcomes between segmentectomy vs lobectomy,²² wedge vs segmentectomy²³ and wedge vs anatomic resection.²⁴

Considering that the highest level of evidence is given when multiple RCTs are available for a clinical situation, studies of large clinical registries or administrative claims databases can perform the important function of confirming results from both RCTs as well as smaller single-institution studies. For example, a study of thoracoscopic lobectomy at our institution demonstrated a lower morbidity compared to thoracotomy.²⁵ A subsequent national study using the Society of Thoracic Surgeons (STS) Database found similar results.²⁶ Although both studies were propensity-matched analyses and although the single-institution study was of a prospectively-collected database with excellent follow-up, the multi-institutional STS database provided a sample size six times larger than the single-institution study and was far more influential despite having limitations of voluntary participation, limited auditing of submitted data, and lack of follow-up beyond 30-days.

In addition, analyses of large-scale datasets have helped provide insight into the surgical management of topics that can be challenging to study, such as stage IIIA(N2) NSCLC treatment. Although a randomized clinical trial provided important data on the role of surgery for N2 disease, the trial's results also raised several questions²⁷ which have yet to be definitively answered with another randomized trial. Population-based datasets allow further investigation in this topic whereas analysis of single-institution datasets are significantly limited due to both institutional biases and small sample sizes associated with the relatively rarity of this disease sub-stage.

Because analysis of large databases has data from real-world clinical practice, their results can be more generalizable than specialized single-institution studies or even a RCT, whose cohort may be healthier than the typical lung cancer patient population. In addition, large clinical databases facilitate development of risk-assessment tools (e.g. risk calculators), drive quality improvement and clinical governance, and help facilitate ongoing review of disease incidence, disease mortality, volume-outcome relationships and national trends in the use of procedures and disparities in health care.²⁸ Analyses from large databases can be used to screen various topics related to lung cancer, perhaps to identify which topics would be most appropriate for the investment of resources to conduct a large and expensive randomized trial.

Understanding and Overcoming the Limitations of Using Large Databases

All large-scale population-based databases have inherent limitations. Briefly, the STS and American College of Surgeons National Surgical Quality Improvement Program (NSQIP) databases are clinical registries with detailed co-morbidity and postoperative data but are limited in that they only have short-term outcomes.^{7, 9} Both the Surveillance, Epidemiology and End Results (SEER) Program and NCDB are cancer registries that have long-term survival data but are limited in that they do not have data on recurrence-free survival.^{6, 10} While the NCDB includes patient composite co-morbidity scores, it does not have detailed co-morbidity data and the SEER simply does not have any co-morbidity information.^{6, 10} The Medicare claims database and Nationwide Inpatient Sample (NIS) are administrative claims databases that rely on billing codes which may suffer from inaccuracies and variabilities in the coding; further Medicare data does not include patients younger than 65 years.^{4, 5} It is critically important that thoracic surgeons with clinical expertise in lung cancer be involved in the design of studies that use these datasets so that the limitations are both minimized and understood so that results can have important clinical implications. It is also imperative that thoracic surgeons foster collaboration with both biostatisticians and epidemiologists during these studies to achieve the highest possible impact.

Conclusion

Analyses of large-scale multi-institutional datasets are being increasingly performed by researchers to study lung cancer. These studies can provide important understanding of treatment at a population level and in many situations can enhance quantitative evidence regarding prognosis, efficacy of interventions, and disparities in treatment. Although these studies have limitations and cannot replace the gold standard of RCTs, appropriate use of these valuable datasets can enhance current evidence and also help direct future research endeavors.

Central Message.

The increasing use of large, multi-institutional datasets by researchers to study lung cancer has improved the understanding of treatments at a population level and in many situations can enhance quantitative evidence regarding prognosis, comparative efficacy of various interventions, and disparities in treatment.