Strengths and limitations of large databases in lung cancer radiation oncology research

Abstract

There has been a substantial rise in the utilization of large databases in radiation oncology research. The advantages of these datasets include a large sample size and inclusion of a diverse population of patients in a real-world setting. Such observational studies hold promise in enhancing our understanding of questions for which evidence is conflicting or absent in lung cancer radiotherapy. However, it is critical that investigators understand the strengths and limitations of large databases in order to avoid the common pitfalls that beset observational analyses. This review begins by outlining the data variables available in major registries that are used most often in observational analyses. This is followed by a discussion of the type of radiotherapy-related questions that can be addressed using such datasets, accompanied by examples from the lung cancer literature. Finally, we describe some limitations of observational research and techniques to mitigate bias and confounding. We hope that clinicians and researchers find this review helpful for designing new research studies and interpreting published analyses in the literature.

Introduction

The field of radiation oncology has witnessed a substantial increase in the publication of observational analyses using large healthcare databases. These datasets provide investigators with the tools to probe into questions that may not be feasible with prospective studies. The advent of drugs like immune checkpoint inhibitors and newer-generation targeted therapies, as well as enhanced radiotherapy and surgical techniques like intensity-modulated radiation therapy (IMRT), stereotactic body radiation therapy (SBRT), proton beam therapy, minimally invasive surgery, and robotic-assisted surgery, have led to a need to understand practice patterns, adherence to guidelines, healthcare disparities, and comparative effectiveness.

Common categories of databases include national cancer registries, claims-based datasets, national surveys, and hospital encounter data. A brief description of some of the major databases in the United States used in observational studies in lung radiation oncology follows below. Tables 1 and 2 summarize some of the key similarities and differences among them.

Table 1. Comparison of NCDB, SEER, SEER-Medicare, and HCUP—descriptive and demographic information.

Variables	NCDB	SEER	SEER-Medicare	HCUP (NEDS/NIS)
Number of patients	>34 million [2004–2015]	>10.4 million [1975–2016]	>2.3 million [1991–2015]	NEDS: >322 million [2006–2016]; NIS: >203 million [1988–2016]
Representation	Hospital-based	Population-based	Population-based	Population-based
Proportion of newly diagnosed cancers nationally	70%	35%	8%	N/A
Weighted	No	No	No	Yes
Ages represented	All ages	All ages	65+a	All ages
Demographic variables
Comorbidities	Charlson-Deyo score	No	Medicare claims	Discharge diagnoses and Elixhauser score
Race	Yes	Yes	Yes	NEDS: no; NIS: yes
High school degree	Yes	Yes	Yes	No
Insurance	Yes	Yes	Yes	Yes
City size	Yes	Yes	Yes	Yes
Facility type	Yes	No	Yes	Yes
Hospital size	No	No	Yes	Yes
Hospital region	Yes	Yes	Yes	Yes

^a, also includes patients with end-stage renal disease and on disability. NCDB, National Cancer Data Base; SEER, Surveillance, Epidemiology, and End Results; HCUP, Healthcare Cost and Utilization Project; NEDS, Nationwide Emergency Department Sample; NIS, National Inpatient Sample.

Table 2. Comparison of NCDB, SEER, SEER-Medicare, and HCUP—clinical, treatment, and outcomes information.

Variables	NCDB	SEER	SEER-Medicare	HCUP (NEDS/NIS)
Cancer data
AJCC staging	Separate clinical and pathologic	Combined clinical and pathologic	Combined clinical and pathologic	No
Histology	Yes	Yes	Yes	No
Anatomical site	Yes	Yes	Yes	Yes
Genetic markers	Yes, limited (not lung)	Yes, limited (not lung)	Yes, limited (not lung)	No
Metastatic locations coded	Bone, brain, liver, lung (at diagnosis)	Bone, brain, liver, lung (at diagnosis)	Yes, multiple (SEER codes and Medicare claims)	Yes, multiple (discharge diagnoses)
Treatment
Sequence	Yes	Yes	Yes	No
Surgery	Yes	Yes	Yes	No
Surgical margins	Yes	No	No	No
Number of lymph nodes dissected	Yes	Yes	Yes	No
Radiation therapy
Received	Yes	Yes (upon request)	Yes	Yes
Modality	Yes	No	Yes	No
Dose	Yes	No	No	No
Fractionation	Yes	No	Yesa	No
Chemotherapy
Received	Yes; includes single or multi-agent chemo	Yes (upon request)	Yes	Yes (very limited)
Type	No	No	Yes	No
Immunotherapy	Yes	No	No	Yes
Received	Yes	No	No	Yes
Type	No	No	No	No
Outcomes
Overall survival	Yes	Yes	Yes	In-hospital mortality only
Disease-specific survival	No	Yes	Yes	No
Disease-free survival	No	No	Yes (limitedb)	No
Length of stay	Yes	No	Yes	Yes
Cost	No	No	Yes	Yes
Complications	No	No	Yesc	Yesd

^a, variable not collected but inferred through Medicare claims; ^b, variable not collected but inferred through Medicare claims. This applies only for patients who receive treatment for their recurrence; ^c, complications available via Medicare claims data; ^d, complications available via discharge diagnoses, though it may be difficult to ascertain with certainty whether or not these diagnoses represent comorbidities or complications. NCDB, National Cancer Data Base; SEER, Surveillance, Epidemiology, and End Results; HCUP, Healthcare Cost and Utilization Project; NEDS, Nationwide Emergency Department Sample; NIS, National Inpatient Sample.

National Cancer Data Base (NCDB)

The NCDB is a joint quality improvement program of the Commission on Cancer (CoC) between the American College of Surgeons and the American Cancer Society (1). This nationwide oncology outcomes database encompasses more than 1,500 CoC programs and approximately 70% of all newly diagnosed cases of cancer in the U.S. (2). A portion of the dataset, known as the Participant User File, was made public in 2013 to researchers at programs accredited by the CoC, and is updated and released annually. Since then, the number of publications involving the NCDB has increased exponentially (3,4). In particular, the NCDB has been widely used to study patterns of radiation therapy as it includes granular data not encompassed other databases, including dose, fractionation, timing, modality, location, anatomical site, boost, and reason for no radiation. Extensive patient sociodemographic and clinical factors as well as facility characteristics are available. Variables like individual facility case volume can be extrapolated based on individual facility identification numbers.

Limitations of the NCDB include selection bias (5,6), lack of longitudinal treatment data, and lack of clinically relevant endpoints like complications, patient-reported outcomes (PROs), cause-of-death, local control, and disease-free survival. Furthermore, since the NCDB is hospital-based rather than population-based (not designed to be representative of the U.S. population overall), generalizability from this data is more limited than that of other databases.

Surveillance, Epidemiology, and End Results Program (SEER)

The SEER program of the National Cancer Institute is a widely used and authoritative source of information on cancer incidence, staging, treatment, demographics, and survival information (7). The program pools data from 19 geographic areas covering approximately 35 percent of the U.S. population. These areas are particularly chosen to be representative of the U.S. population and therefore have been used to answer important epidemiological questions such as cancer prevention and screening, healthcare disparities, effectiveness of public health interventions, and implementation of healthcare policy. Publications using the SEER database to study lung cancer have increased markedly from the 1980s to the 2010s (8).

SEER data must be interpreted carefully as there are some important limitations (9). These include unrecorded variables, underreported and incomplete adjuvant treatment data, variations in coding and reporting, and migration of patients between SEER registry areas. Multiple analyses of local SEER registries in comparison to Medicare claims, medical record review, or patient self-report have demonstrated that the under-ascertainment of radiotherapy within those SEER registries may range from 10–30% (10-12). In 2016, the SEER program eliminated the routine reporting of radiotherapy. This is now available by request, although SEER requires the investigator sign a data use agreement acknowledging the limitations of the radiotherapy variable. Radiotherapy data in SEER is further limited to receipt, modality, and sequence with surgery.

SEER-Medicare

The SEER-Medicare dataset was created from the linkage of the SEER database to Medicare claims (13). The Medicare dataset contains diagnostic and billing codes for services covered by Medicare including tests, procedures, office visits, admissions, medical equipment, hospice care, and prescription drugs (14). This database has been primarily used to study patients age 65 or older, although Medicare does cover individuals who are disabled or with end-stage renal disease. For persons age 65 and older, 94% have been linked to their Medicare enrollment file. Claims data provides more granular comorbidity data based on International Classifications of Diseases (ICD) coding. Treatment-related complications and disease recurrence, while not coded, can be inferred from health claims (15). While SEER includes radiotherapy data from the first course of therapy, Medicare claims supplements this with information on type of modality administered, timing, fractionation, and anatomical site of treatment. A major additional limitation of this dataset beyond those associated with SEER would be the possibility of making inaccurate clinical inferences based on what ultimately is administrative and billing data (14). Generalizability to patients under the age of 65 is also highly limited.

Healthcare Cost and Utilization Project (HCUP)

The HCUP, which is sponsored by the Agency for Healthcare Research and Quality, includes the largest collection of longitudinal hospital data in the U.S., with all-payer and encounter level information. The HCUP includes healthcare databases such as the Nationwide Emergency Department Sample (NEDS) and the National Inpatient Sample (NIS), both representing a 20% stratified sample of U.S. hospital-based emergency departments (ED) and discharges, respectively. Like SEER, these are meant to be representative of the U.S. population, and so national estimates can be made. Discharge data is identifiable through ICD data, similar to most claims-based datasets. Cancer-related studies using these datasets have characterized general ED visits (16), complications of treatment (17), oncologic emergencies (18), and outcomes from oncologic surgeries (19-22). Since data is limited to the ED visit or inpatient encounter, few studies have examined radiotherapy utilization, although studies investigating patterns of emergent in-house radiotherapy would be feasible.

Survey data

Survey datasets have been gaining currency amongst health researchers due to their incorporation of PROs, which most large cancer databases lack. Examples of such datasets include the SEER-Medicare Health Outcomes Survey (SEER-MHOS), the National Health Interview Survey (NHIS), and the National Health and Nutrition Examination Survey (NHANES), with the latter two maintained by the Centers for Disease Control and Prevention. The strength of these studies lies in their ability to collect data on the patient experience, as well as demographic, financial, preventative care, and detailed social information like personal habits (alcohol, tobacco, drug use), diet, exercise, and reproductive health. These datasets have been used sparingly to answer radiotherapy-related questions (23), although a project to analyze quality-of-life after surgery or radiotherapy for stage I lung cancer patients is currently underway (24). The primary limitation of survey data would be non-response bias, with modern response levels hovering around 70% and trending downward (25).

Research questions using large datasets in lung cancer

Large databases have many advantages in epidemiological research, with their strength primarily resting on larger sample size, inclusion of more diverse subsets of patients, and completed outcomes. Below are some research questions related to lung cancer radiotherapy that have been addressed by large datasets.

Adoption of novel treatments or technology

One of the strongest advantages of large datasets lies in studying practice patterns across a disease site. This can be particularly useful in the field of lung radiotherapy, as differences in practice exist between providers based on the available evidence. One common application of this question is in the adoption of new techniques or technologies, including SBRT for early-stage non-small cell lung cancer (NSCLC) as well as metastatic disease to the lungs. Given the high rates of local control seen in prospective studies evaluating the efficacy of SBRT (26,27) in early-stage NSCLC, multiple groups have used the NCDB (28-30) to explore how this has changed practice patterns in the past 10–15 years. Another example has been the advent of IMRT in the treatment of locally advanced NSCLC. A SEER-Medicare analysis noted that IMRT utilization increased from 0.5% in 2001 to 14.7% in 2007 for patients with stage III NSCLC (31).

Healthcare disparities

Given that randomized controlled trials (RCTs) tend to enroll patients who are young, white, male, and healthy (32,33), understanding treatment variations in groups often excluded from RCTs is important. This is one manner by which investigators can learn about healthcare disparities based on age, sex, race/ethnicity, median household income, education, insurance status, and geographic region. One NCDB study examined barriers to chemotherapy or radiotherapy utilization in small cell lung cancer (SCLC) and found that patients on government insurance were less likely to receive radiotherapy (34). Other NCDB studies have found that black race was associated with lower likelihood of receiving SBRT or surgery in stage I NSCLC or in receiving cancer-directed care in stage III NSCLC (35,36). These studies can help inform future efforts to improve access to care and reduce inequality in our healthcare system.

Adherence to national guidelines

Large databases can be especially helpful for studying practice patterns with regard to adherence to national guidelines, such as those from the National Comprehensive Cancer Network (NCCN) (37) or professional societies like the American Society for Radiation Oncology (ASTRO) or the American Society of Clinical Oncology (ASCO). A SEER-Medicare study looking at guideline concordance in lung cancer in the elderly found that only 45% of patients received treatment consistent with guidelines, with non-concordant care being associated with worsened overall survival (38). Another study found that two thirds of potentially eligible patients for surgery in early-stage SCLC were not receiving surgery (39). A recent SEER-Medicare study investigating adherence to surveillance guidelines after curative therapy for NSCLC found that only 61% of patients received routine imaging surveillance, and that patients treated with SBRT were more likely to have undergone recommended imaging (40). These studies can enhance our understanding of current gaps in healthcare delivery and quality.

Complications of therapy and quality-of-life

In addition to expanding our repertoire of curative and life-prolonging therapies, there has been an increased emphasis on providing high-value care and improving the patient experience. The quality of PROs in clinical trial protocols is often poor, partly due to logistic challenges as well as limited resources available for monitoring these outcomes (41,42). However, a recent RCT comparing routine surveillance to electronic symptom monitoring in patients with lung cancer found an overall survival benefit in the monitoring group, demonstrating the importance of actively collecting PROs (43). Claims-based datasets such as SEER-Medicare and encounter-based hospital databases like HCUP-NEDS or HCUP-NIS can be useful for studying toxicities and other aspects of the patient experience. One SEER-Medicare study quantified the treatment burden of patients with stage I lung cancer (44). Another study using the NIS evaluated the utilization of intensive care during terminal hospitalizations in patients with metastatic lung cancer (45). Further study into PROs can be carried out with survey datasets.

Comparative effectiveness

Perhaps the most exciting but controversial application for large databases is in the area of comparative effectiveness research (CER). RCTs represent the gold standard in CER to evaluate the efficacy of an intervention (46). However, there are many instances in which a prospective RCT cannot be performed, often due to high costs, small sample sizes, long time to completion, strict enrollment criteria, and limited external validity. The reasons for the latter are multifactorial such as low inclusion of elderly, indigent, uninsured, underinsured, and racial ethnic minority patients (47). Clinical trial enrollment has been suggested as a particular problem in lung cancer (48), given the high likelihood of poor performance status, need for emergent radiation, and patient refusal (49). Patients with lung cancer also tend to have a smoking history that could lead to comorbidities that render them ineligible for clinical trials. Another concern is that lung cancer patients appear to be more likely to receive first-line treatment with their local oncologist, while referrals to the academic center may be more likely for second-line or non-therapeutic studies (50). These issues with trial recruitment or enrollment in lung cancer open a potential avenue for large observational datasets to address questions in specific underrepresented populations that may not be possible to answer in a prospective study. In addition, it is also important to understand the effectiveness of certain interventions in the “real world” among all populations in all settings, which may provide information that complements efficacy data derived from the RCT setting.

Below are some of the most common questions in lung radiotherapy that have begun to be addressed by CER studies.

Surgery vs. SBRT for early-stage NSCLC

One area of controversy in early-stage NSCLC is the efficacy of surgery compared to SBRT. While surgical resection with either lobectomy or sublobar resection has historically been the standard of care in operable patients, data from modern SBRT trials show local control rates above 90% (51) and comparable to that of surgery. A meta-analysis of the randomized STARS and ROSEL trials comparing surgery to SBRT for operable stage I NSCLC demonstrated a survival advantage to SBRT (52), but both trials closed early due to poor accrual, leaving this study to be criticized as an underpowered post-hoc analysis. Multiple database studies have attempted to answer this question, with nearly all of them demonstrating superiority of surgery over SBRT (53-55). However, the main limitation of these studies lies in the selection bias against SBRT as many of these patients receive SBRT due to poor performance status or high comorbidity burden. This was partially accounted for in an NCDB study examining patients who were surgical candidates and refused surgery for SBRT, thereby excluding patients who were not healthy enough for surgery (56). This study also observed a survival improvement for patients undergoing surgery compared to SBRT. Despite efforts to minimize bias, it is indeed impossible to account for all unknown confounders that may exist in large databases.

Post-operative radiotherapy (PORT) for locally advanced NSCLC

PORT has been another area of considerable controversy ever since the PORT meta-analysis showed no improvement and potentially a detriment in survival using older radiotherapy therapy techniques. Given that radiotherapy techniques have improved significantly since then, allowing for lower scatter doses to critical organs-at-risk like the lung, heart, and esophagus, more recent SEER and NCDB analyses have explored similar questions, demonstrating a potential survival improvement with PORT in patients with N2 disease or positive margins (57-59). Based largely on this observational data, current NCCN guidelines support the use of PORT in this population. Prospective trials like Lung-ART are aiming to answer this question more definitively, but the hypothesis generated by these large-database studies has been practice-changing.

Photon vs. proton beam therapy for locally advanced NSCLC

While the current standard-of-care for lung cancer involves photon therapy, proton beam therapy offers a distinct dosimetric advantage that could potentially improve the therapeutic ratio for radiotherapy by delivering higher doses to the target and sparing normal tissues. Current RCTs like RTOG 1308 are underway comparing photon to proton therapy in NSCLC, although we will likely have to wait years before learning these results. In the meantime, one NCDB study observed a 5-year overall survival improvement with proton therapy among NSCLC patients of all stages (60). Further database studies can potentially highlight whether these results bear out in other contexts such as the post-operative setting, in which the advantage of proton therapy in avoiding toxicity may be the greatest.

Chemoradiotherapy and prophylactic cranial irradiation (PCI) in limited-stage SCLC

The standard-of-care for limited-stage SCLC has been combined chemotherapy with radiotherapy (CRT), followed by PCI. However, one of the pivotal RCTs published in 1992 that demonstrated superiority of CRT over RT suggested that this benefit may be limited to patients younger than 70 years (61). Recent NCDB analyses demonstrated that this benefit may extend to highly elderly patients selected in the modern era given improvements in radiotherapy technique (62,63). This may be a result of lower toxicities to the normal lung and heart, but the hypothesis generated by these large-database projects support using aggressive therapy even in populations that were not well-represented in RCTs. Additionally, a SEER analysis noted that elderly patients may also continue to have a benefit to PCI, a finding which had been shown in RCTs that primarily included younger patients (64).

Radiotherapy dose and fractionation in NSCLC

One of the biggest advantages of the NCDB is in its detailed collection of dose and fractionation data for patients undergoing radiotherapy. This enabled multiple studies comparing survival outcomes for patients treated with differing radiation regimens. Dose escalation remains a controversial topic in NSCLC. For early-stage NSCLC, several single-institution and multi-institution studies have suggested that SBRT with a biologically effective dose using alpha/beta of 10 (BED₁₀) of at least 100 or 105 is necessary for optimal local control and overall survival, but NCDB studies have suggested a potential survival improvement with an even higher BED₁₀ threshold (65,66). For locally advanced NSCLC, RTOG 0617 demonstrated inferior survival for patients receiving 74 Gy compared to 60 Gy (67), though many believe there still could be an advantage to some level of dose escalation when more stringent normal tissue constraints are met. This was explored by an NCDB study that found a survival benefit to dose escalation to 70 Gy but not beyond (68). There is certainly room for further study in this area such as for patients who are elderly or not candidates for chemotherapy.

Facility volume in NSCLC

Given that modern lung radiotherapy is commonly associated with significant technical demands in treatment planning, multidisciplinary coordination to ensure the reliability of concurrent treatment, severe toxicities, and substantial risk of recurrence and subsequent death, a high level of expertise may be necessary for providers treating lung cancer. Extensive literature shows that patients have improved outcomes when treated by high-volume surgeons and hospitals for oncologic resections, including video-assisted thoracoscopic lobectomy and robotic-assisted lobectomy (20,22). More recent data has examined the effect of the treating institution on outcomes following lung radiotherapy. A secondary analysis of the RTOG 0617 RCT in locally advanced NSCLC showed that receiving chemoradiotherapy at institutions with higher clinical trial accrual volume was associated with longer overall survival (69). NCDB analyses have also shown longer overall survival for patients receiving definitive chemo-RT for locally advanced NSCLC at facilities treating a high number of annual cases compared to those treated at lower-volume facilities (70), similar to findings noted regarding SBRT for stage I NSCLC (71).

Pitfalls and disadvantages of using large databases in lung cancer

Before embarking on a large database analysis, it is important to have a well-designed hypothesis and to be intimately aware of the strengths and limitations of the database at hand. For example, researchers interested in studying national disease incidence might choose to work with the SEER database, while others more interested in a radiotherapy dose escalation study may prefer the NCDB. The question should aim to fill a relevant gap in knowledge that prospective studies have not yet answered, and the researchers should consider what techniques they plan to use to minimize the effect of any potential inherent biases in the study question or the data itself prior to conducting the analysis.

Below are listed some of the most common design and analytical issues that befall such large database studies in lung cancer along with methods to address them.

Selection bias

Perhaps the most common and frequently cited criticism of large datasets is selection bias, an inherent result of non-random assignment. This stems from the idea that patients may be more likely to receive a particular treatment based on unmeasured confounders, such as performance status. Many patients who are elderly or medically unfit for surgery are more likely to be referred for radiotherapy. These patients may instead be more likely to die from comorbidities unrelated to their cancer. An additional missing variable in lung cancer is the location of lung tumors in relation to critical structures, which are often centrally located. Tumor location may influence which dose-fractionation schemes are chosen and therefore bias any comparisons of effectiveness among them. Therefore, in database comparisons of two management strategies or dose-fractionation regimens, investigators are often comparing two very different populations and may not be able to attribute the outcome of one group solely to the intervention.

Immortal time bias

In observational studies comparing adjuvant treatment to observation after surgery, patients who die soon after surgery will not have the opportunity to enter the adjuvant therapy cohort and will be much more likely to be included in the observation cohort. This would guarantee worse comparative survival for the observation arm than would otherwise be expected on an intention-to-treat analysis of an RCT. This is because the adjuvant therapy cohort has a certain time period of being “immortal” from early death after surgery, since those patients (by definition) must have lived a certain length of time to be included in that cohort. This type of bias is called immortal time bias (72), also known as guarantee time bias (73,74).

To account for this bias regarding time-dependent variables like survival or tumor response, investigators will often incorporate a landmark analysis, in which the patients who die a certain period soon after surgery are excluded from analysis. The optimal landmark to use for these analyses has remained elusive, as there is no standard definition for the period during which deaths would be classified as immediate postoperative mortality. Critics of stringent landmark corrections argue that disregarding the potential short-term survival benefit of PORT, particularly in aggressive cancers with poor prognosis, could cause a bias in the different direction. Therefore, a sequential landmark analysis method (selecting monthly landmarks from 1-6 months postoperatively) has been proposed as a sensitivity analysis to measure the robustness of the results from the primary analysis (72). Examples of landmark analyses in the lung radiotherapy literature are becoming increasingly abundant (58,75-78).

Strategies to address confounding

There are several techniques commonly employed to mitigate selection bias and address confounding in CER. One is stratification, in which patients can be grouped based on a covariate, such as comorbidity status. A second method is multivariable regression, in which all potential confounders are included in the multivariable model, and truly independent predictors will be revealed as statistically significant. A third method is propensity score matching (PSM), a statistical technique used to estimate the effect of an intervention or treatment by accounting for the propensity to receive a certain therapy based on the observed covariates (14). PSM has been applied often in the lung radiotherapy literature (53-56,60). Finally, instrumental variable analysis (IVA) is a technique originating from econometrics, but now appearing in epidemiological studies to account for confounding and mimic randomization. An instrumental variable is one that is strongly correlated with the treatment assignment but not with the outcome of interest. IVA has been used in multiple lung cancer database studies to account for geographic variation in radiotherapy utilization (79,80).

Even after employing techniques to reduce confounding, it is impossible to account for all potential confounders that might affect a CER study. One report found that even after multiple statistical adjustments it is still possible to obtain outcomes different from those of comparable RCTs (81). In a recent analysis, observational database studies from NCDB, SEER, and SEER-Medicare were matched with comparable RCTs and found that agreement between matched pairs occurred only 40% of the time without any variables that predicted stronger correlation (82). Criticisms of this study include the fact that the match criteria only included age and stage, and that CER studies are not designed to match inclusion and exclusion criteria RCTs perfectly, since they are typically intended to generate hypotheses that fill in gaps of knowledge that have not yet been answered by RCTs. While checklists and published guidelines for observational research, such as the STROBE guidelines (83), are helpful, it is imperative that researchers and journals continue to build and enforce rigorous standards for observational database studies.

Conclusions

Well-conducted observational studies using large databases hold much promise to enhance our understanding of lung cancer management. Given the need to deliver high-value care to patients from all segments of the population, observational analyses can help clarify ideal management strategies among certain subsets of patients as well as identify which patients may not be receiving optimal care. Furthermore, the advent of machine learning may help eliminate unknown confounders and improve the accuracy and predictions of models based off large datasets (84). While RCTs remain the gold standard by which we base our treatment decisions, observational analyses using large registries can provide important hypothesis-generating data, from which future practice-changing prospective trials can be built.