Temporal Trends in Clinical Evidence of 5-Year Survival Within Electronic Health Records Among Patients With Early-Stage Colon Cancer Managed With Laparoscopy-Assisted Colectomy vs Open Colectomy

Jue Hou; Rachel Zhao; Tianrun Cai; Brett Beaulieu-Jones; Thany Seyok; Kumar Dahal; Qianyu Yuan; Xin Xiong; Clara-Lea Bonzel; Claire Fox; David C Christiani; Thomas Jemielita; Katherine P Liao; Kai-Li Liaw; Tianxi Cai

doi:10.1001/jamanetworkopen.2022.18371

. 2022 Jun 23;5(6):e2218371. doi: 10.1001/jamanetworkopen.2022.18371

Temporal Trends in Clinical Evidence of 5-Year Survival Within Electronic Health Records Among Patients With Early-Stage Colon Cancer Managed With Laparoscopy-Assisted Colectomy vs Open Colectomy

Jue Hou ¹, Rachel Zhao ², Tianrun Cai ³, Brett Beaulieu-Jones ⁴, Thany Seyok ³, Kumar Dahal ³, Qianyu Yuan ⁵, Xin Xiong ¹, Clara-Lea Bonzel ¹, Claire Fox ⁶, David C Christiani ⁵, Thomas Jemielita ⁶, Katherine P Liao ^3,⁴, Kai-Li Liaw ⁶, Tianxi Cai ^1,^4,^✉

¹Department of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, Massachusetts

²Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada

³Division of Rheumatology, Inflammation, and Immunity, Department of Medicine, Brigham and Women’s Hospital/Harvard Medical School, Boston, Massachusetts

⁴Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts

⁵Department of Environmental Health, Harvard T. H. Chan School of Public Health, Boston, Massachusetts

⁶Merck & Co, Inc, Kenilworth, New Jersey

Accepted for Publication: April 26, 2022.

Published: June 23, 2022. doi:10.1001/jamanetworkopen.2022.18371

^✉

Corresponding Authors: Tianxi Cai, ScD, Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck St, Room 434, Boston, MA 02115 (tcai@hsph.harvard.edu).

Author Contributions: Drs Hou and Tianxi Cai had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Hou, Zhao, Tianrun Cai, Beaulieu-Jones, Fox, Jemielita, Liao, Liaw, Tianxi Cai.

Acquisition, analysis, or interpretation of data: Hou, Zhao, Tianrun Cai, Beaulieu-Jones, Seyok, Dahal, Yuan, Xiong, Bonzel, Fox, Christiani, Liao, Liaw, Tianxi Cai.

Drafting of the manuscript: Hou, Zhao, Tianrun Cai, Beaulieu-Jones, Dahal, Xiong, Bonzel, Jemielita, Liao, Liaw, Tianxi Cai.

Critical revision of the manuscript for important intellectual content: Hou, Tianrun Cai, Beaulieu-Jones, Seyok, Yuan, Fox, Christiani, Liao, Liaw, Tianxi Cai.

Statistical analysis: Hou, Yuan, Jemielita, Tianxi Cai.

Obtained funding: Beaulieu-Jones, Christiani, Liaw, Tianxi Cai.

Administrative, technical, or material support: Hou, Tianrun Cai, Beaulieu-Jones, Xiong, Bonzel, Fox, Christiani, Jemielita, Liao.

Supervision: Tianrun Cai, Liao, Liaw, Tianxi Cai.

Conflict of Interest Disclosures: Dr Hou reported receiving grants from Merck & Co, Inc, during the conduct of the study. Dr Beaulieu-Jones reported receiving grants from Merck SA during the conduct of the study. Dr Fox reported personal fees from and being an employee of Merck Sharp & Dohme Corp, a subsidiary of Merck & Co, Inc, outside the submitted work. Dr Jemielita reported being an employee of Merck & Co, Inc. Dr Liaw reported being an employee of Merck & Co, Inc. No other disclosures were reported.

Funding/Support: The study was supported by Merck & Co, Inc; by grants P30 AR072577 (Dr Liao) and K99NS114850 (Dr Beaulieu-Jones) from the National Institutes of Health (NIH); and grant U01CA209414 from the National Cancer Institute of the NIH (Dr Christiani).

Role of the Funder/Sponsor: Merck & Co, Inc, contributed to the design and conduct of the study; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication but no role in the collection, management, analysis, and interpretation of the data. The NIH played no role in design and conduct of the study; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication; collection, management, analysis, and interpretation of the data.

Additional Information: This study is based on research using information obtained from www.projectdatasphere.org, which is maintained by Project Data Sphere. Neither Project Data Sphere nor the owner(s) of any information from the website have contributed to, approved, or are in any way responsible for the contents of this report. Code for analysis and figure generation is available at https://github.com/hjmarquis/REST_paper1. Summary data that support the findings of this study as well as anonymous data that were used for this study are available on reasonable request to the corresponding author.

^✉

Corresponding author.

PMCID: PMC9227003 PMID: 35737384

Key Points

Question

How might investigators assess and adjust for temporal trends appropriately to correct for confounding bias in studies using clinical evidence?

Findings

This comparative effectiveness study of 943 patients with early-stage colon cancer detected substantial temporal trends by emulating the Clinical Outcomes of Surgical Therapy (COST) Study Group Trial using electronic health records data from 2006 to 2017. Through stratification by calendar time and cotraining of time-varying models, this emulation reached a conclusion agreeing with the COST Study Group Trial.

Meaning

These findings suggest that with proper adjustment of confounding bias from temporal trends, clinical studies conducted with data covering a long time span could supplement randomized clinical trials in the assessment of treatment outcome and its evolution over time.

This comparative effectiveness study investigates temporal trends and corrects for their confounding bias in an emulation of a completed randomized clinical trial using electronic health record data of patients with early-stage colon cancer.

Abstract

Importance

Temporal shifts in clinical knowledge and practice need to be adjusted for in treatment outcome assessment in clinical evidence.

Objective

To use electronic health record (EHR) data to (1) assess the temporal trends in treatment decisions and patient outcomes and (2) emulate a randomized clinical trial (RCT) using EHR data with proper adjustment for temporal trends.

Design, Setting, and Participants

The Clinical Outcomes of Surgical Therapy (COST) Study Group Trial assessing overall survival of patients with stages I to III early-stage colon cancer was chosen as the target trial. The RCT was emulated using EHR data of patients from a single health care system cohort who underwent colectomy for early-stage colon cancer from January 1, 2006, to December 31, 2017, and were followed up to January 1, 2020, from Mass General Brigham. Analyses were conducted from December 2, 2019, to January 24, 2022.

Exposures

Laparoscopy-assisted colectomy (LAC) vs open colectomy (OC).

Main Outcomes and Measures

The primary outcome was 5-year overall survival. To address confounding in the emulation, pretreatment variables were selected and adjusted. The temporal trends were adjusted by stratification of the calendar year when the colectomies were performed with cotraining across strata.

Results

A total of 943 patients met key RCT eligibility criteria in the EHR emulation cohort, including 518 undergoing LAC (median age, 63 [range, 20-95] years; 268 [52%] women; 121 [23%] with stage I, 165 [32%] with stage II, and 232 [45%] with stage III cancer; 32 [6%] with colon adhesion; 278 [54%] with right-sided colon cancer; 18 [3%] with left-sided colon cancer; and 222 [43%] with sigmoid colon cancer) and 425 undergoing OC (median age, 65 [range, 28-99] years; 223 [52%] women; 61 [14%] with stage I, 153 [36%] with stage II, and 211 [50%] with stage III cancer; 39 [9%] with colon adhesion; 202 [47%] with right-sided colon cancer; 39 [9%] with left-sided colon cancer; and 201 [47%] with sigmoid colon cancer). Tests for temporal trends in treatment assignment (χ² = 60.3; P < .001) and overall survival (χ² = 137.2; P < .001) were significant. The adjusted EHR emulation reached the same conclusion as the RCT: LAC is not inferior to OC in overall survival rate with risk difference at 5 years of −0.007 (95% CI, –0.070 to 0.057). The results were consistent for stratified analysis within each temporal period.

Conclusions and Relevance

These findings suggest that confounding bias from temporal trends should be considered when conducting clinical evidence studies with long time spans. Stratification of calendar time and cotraining of models is one solution. With proper adjustment, clinical evidence may supplement RCTs in the assessment of treatment outcome over time.

Introduction

Electronic health record (EHR) data, with detailed longitudinal clinical information, have the potential to generate clinical evidence to assess treatment outcomes. Recently, regulatory agencies have created a framework with guidance on approaches for leveraging clinical evidence to evaluate the effectiveness of medical products.¹ This, together with the wider availability of EHR data, has led to an increasing number of studies comparing EHR-based analyses with randomized clinical trials (RCTs) in recent years, including the use of external controls,^2,3,4 supporting a new indication for approved therapy⁵ and pragmatic trial designs.⁶

In clinical practice, secular trends in management and treatment options can lead to changes in the effectiveness of a given treatment. Although the treatment remains the same, other factors may alter the treatment response or outcome. For example, the overall survival of patients with early-stage cancer after surgery has improved over time as targeted therapies have become available for recurrence and metastasis. Temporal trends are listed as considerations for using EHR external control for RCTs.⁷ Moreover, these temporal trends may cause automated analytical tools trained in 1 year to fail at a later point in time.⁸

Properly understanding and accounting for temporal trends is a crucial step when comparing treatment effectiveness using longitudinal EHR data across calendar years. Shifts in patient population characteristics, treatment patterns, and responses all have significant implications for medical practice and policy making. When both treatment and response patterns shift over time, the calendar year becomes a confounding factor. Reporting of the association between treatment and response should also reflect its time-dependent property. On the other hand, it is important to strike a balance between the temporal adjustment and limited sample size within each calendar time frame. One potential strategy is to adopt a cotraining strategy for models across different periods by allowing some associations identified from data-driven selection to change over time. The objective of this study was to investigate temporal trends and correct for their confounding bias in an emulation of a completed RCT using EHR data. The result is compared with that of the RCT.

Methods

The Mass General Brigham Institutional Review Board approved the use of the EHR data for this study with a waiver of informed consent owing to use of deidentified data. This study followed the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) reporting guideline. eFigure 1 in the Supplement illustrates the study schematics.

The RCT

The multi-institutional Clinical Outcomes of Surgical Therapy (COST) Group Study RCT on laparoscopic-assisted colectomy (LAC) vs open colectomy (OC), conducted from August 1994 to August 2001, was chosen as the target of emulation⁹ and was reported by Nelson et al in 2004.¹⁰ The overall finding of this trial was the noninferiority of LAC compared with OC. The individual-level data, study protocol, and report sheet were made available through Project Data Sphere.¹¹ We extracted data regarding the type of surgical procedure, follow-up duration, vital status, and baseline features, including age, sex, cancer clinical stage, presence of colon adhesion, and tumor location. The tumor location was categorized into the left, right, or sigmoid segment of the colon. Inclusion criteria consisted of a clinical diagnosis of adenocarcinoma of the colon (histologic confirmation was required at surgery), being 18 years or older, and the absence of prohibitive abdominal adhesions. Exclusion criteria included advanced local or metastatic disease, rectal or transverse colon cancer, acute bowel obstruction or perforation due to cancer, and severe medical illness. Inflammatory bowel disease, familial polyposis, pregnancy, or concurrent or previous malignant tumor also precluded enrollment. Patients were required to have a recent radiological test (42 days within registration) and undergo a colectomy within 21 days, so we interpreted the requirement as the implicit eligibility that a colectomy must be performed within 90 days after the colorectal cancer diagnosis. The list of key eligibility criteria, along with the EHR variables corresponding to the RCT eligibility criteria and other features and the extraction rules, are provided in eTable 1 in the Supplement.

EHR Cohort for Emulating the RCT

We created a data mart from the Mass General Brigham Research Patient Data Registry consisting of patients with at least 1 International Classification of Diseases, Ninth Revision, or International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, code for colon cancer by January 1, 2020. We further classified the colon cancer status for patients in the data mart based on the unsupervised multimodal automated phenotyping¹² algorithm validated by 171 labels. Identified patients with colon cancer who underwent a colectomy formed the EHR LAC and OC groups. We identified the first colectomy according to the Current Procedure Terminology code (eTable 2 in the Supplement). The date of the first colectomy code after the colon cancer diagnosis was used as the baseline date for each patient. We included only patients whose colectomy date was 2006 or later, when LAC became widely available. We also excluded patients who underwent colectomy after January 1, 2018, owing to their short follow-up. Patients with multiple surgical codes around the first Current Procedure Terminology code for colectomy (3 days before to 3 days after) were excluded because they likely underwent more complex procedures. Patients whose colectomy dates were 90 days after their colorectal cancer diagnosis were also excluded to remain consistent with the RCT eligibility criteria. We extracted the EHR features that can be mapped to the target RCT eligibility criteria and features (eTable 2 in the Supplement).

Key eligibility criteria of the RCT were matched in our emulation. We excluded patients with (1) diagnosis codes of prior cancers, adhesions, Crohn disease, familial polyposis, and chronic ulcerative colitis; (2) recent diagnosis codes of colon obstruction and perforation; (3) missing stage or stage IV extraction from the natural language–processing interpreter for cancer extraction algorithm¹³; and (4) tumor location outside the right-sided, left-sided, and sigmoid colon or multiple tumor locations. We refer to the final subcohort treated with LAC or OC and satisfying the eligibility criteria as the EHR emulation cohort.

Additional features not considered by the RCT were extracted for confounding adjustments. We extracted obesity^14,15 and additional health care features before the colectomy, including use of health care resources and existing comorbidities as defined by International Classification of Disease codes grouped into clinically relevant groups.¹⁶ To assess temporal trends, we categorized patients into 3 groups according to the year of colectomy: 2006 to 2009, 2010 to 2013, and 2014 to 2017. We refer to these intervals as temporal periods. We split the 12 years into 3 periods to allow for multiple change points while maintaining a reasonable sample size within each temporal period to ensure stable estimation. Details on EHR cohort construction and data curation are provided in eMethods 1 in the Supplement.

Statistical Analysis

Data were analyzed from December 2, 2019, to January 24, 2022. We assessed the temporal trends of the patient clinical profile, the propensity for selecting LAC vs OC, and the patient outcome. Aiming to emulate the RCT, we defined the average treatment effect parameter as the difference between LAC vs OC in 5-year overall survival rate for the EHR-derived target patient populations constructed to match the RCT patient characteristics on the key risk factors, including sex, cancer clinical stage, tumor location, and colon adhesion. Overall survival was the primary end point for the COST Group Study RCT, has been the criterion standard measure for cancer trials,¹⁷ and captures surgery-related mortality,¹⁸ whereas other end points such as progression or cancer-specific mortality are also not readily available in EHRs. To account for confounding, we adjusted for clinically relevant variables, including age, sex, cancer stage, tumor location, colon adhesion, procedure subtypes, and obesity, as well as a broad range of other comorbidities. We adopted a doubly robust causal modeling strategy¹⁹ that combines the regression adjustment approach via outcome regression and the propensity score (PS) weighting approach. To account for temporal changes, we allowed the covariate associations in both the outcome regression and PS models to vary across the temporal periods but adopted a data-driven cotraining strategy to select temporal trends as well as confounding factors.^20,21,22 We assessed the association of temporal trends with outcome regression by testing whether the coefficients of the confounders on the mortality risk differ across temporal periods. Similarly, we tested whether the odds ratios of the confounders on treatment assignment in the PS model differed by temporal periods. To further illustrate the association of temporal trends as a confounding factor, we additionally performed causal modeling using the entire EHR emulation cohort without adjusting for temporal periods as a benchmark analysis. R, version 4.1.1 was used to conduct analyses. Tests were 2-tailed (ie, to test for difference rather than superiority/inferiority).

To form EHR-derived target populations that mimic the RCT population, we considered 4 EHR source populations according to time of colectomy: 2006 to 2017, 2006 to 2009, 2010 to 2013, and 2014 to 2017. For each EHR source population, we created the corresponding EHR-derived target population by weighting patients in the source population to match the key risk factors of the RCT population and calculate the average treatment effect. We constructed SEs and CIs based on the bootstrap method with 10 000 replications. Details of the statistical analyses are provided in eMethods 2 in the Supplement.

Results

Study Cohorts

A total of 943 patients were included in the EHR emulation cohort (452 [48%] men and 491 [52%] women; median age, 63 [range, 20-99] years) (Figure 1 and eMethods 1 in the Supplement). Of these, 518 patients underwent LAC (median age, 63 [range, 20-95] years; 250 [48%] men and 268 [52%] women; 121 [23%] with stage I, 165 [32%] with stage II, and 232 [45%] with stage III cancer; 32 [6%] with colon adhesion; 278 [54%] with right-sided colon cancer; 18 [3%] with left-sided colon cancer; and 222 [43%] with sigmoid colon cancer). A total of 425 patients underwent OC (median age, 65 [range, 28-99] years; 202 [47%] men and 223 [52%] women; 61 [14%] with stage I, 153 [36%] with stage II, and 211 [50%] with stage III cancer; 39 [9%] with colon adhesion; 202 [47%] with right-sided colon cancer; 39 [9%] with left-sided colon cancer; and 201 [47%] with sigmoid colon cancer). The baseline variables of the EHR cohort are presented in Table 1 alongside those of the target RCT cohort. The population sizes of the 4-year temporal periods were 231 patients for 2006 to 2009 (81 undergoing LAC and 150 undergoing OC), 277 for 2010 to 2013 (132 undergoing LAC and 145 undergoing OC), and 435 for 2014 to 2017 (305 undergoing LAC and 130 undergoing OC) (see eFigure 1-4 in the Supplement for a summary of multimodal automated phenotyping, extraction of body mass index, and follow-up durations).

Table 1. Characteristics of the Treatment Groups.

Characteristic	Target RCT cohort			RCT eligible in EHR cohort
	Treatment group^a		OR (95% CI)	Treatment group^a		OR (95% CI)
	LAC (n = 425)	OC (n = 414)	OR (95% CI)	LAC (n = 518)	OC (n = 425)	OR (95% CI)
Age, median (range), y^b	69 (29-94)	70 (28-96)	NA	63 (20-95)	65 (28-99)	NA
Sex
Men	217 (51)	200 (48)	1.12 (0.85-1.46)	250 (48)	202 (47)	1.03 (0.80-1.33)
Women	208 (49)	214 (52)	1.12 (0.85-1.46)	268 (52)	223 (52)	1.03 (0.80-1.33)
Stage
I	176 (41)	145 (35)	1.31 (0.99-1.74)	121 (23)	61 (14)	1.82 (1.30-2.57)
II	136 (32)	147 (35)	0.85 (0.64-1.14)	165 (32)	153 (36)	0.83 (0.63-1.09)
III	113 (26)	122 (29)	0.87 (0.64-1.17)	232 (45)	211 (50)	0.82 (0.64-1.06)
Colon adhesion	94 (22)	57 (14)	1.78 (1.24-2.56)	32 (6)	39 (9)	0.65 (0.40-1.06)
Right colon cancer	231 (54)	228 (55)	0.97 (0.74-1.27)	278 (54)	202 (47)	1.28 (0.99-1.65)
Left colon cancer	32 (7)	28 (7)	1.12 (0.66-1.91)	18 (3)	22 (5)	0.66 (0.35-1.24)
Sigmoid colon cancer	162 (38)	158 (38)	1.00 (0.76-1.32)	222 (43)	201 (47)	0.84 (0.65-1.08)

Open in a new tab

Abbreviations: EHR, electronic health record; LAC, laparoscopy-assisted colectomy; NA, not applicable; OC, open colectomy; OR, odds ratio; RCT, randomized clinical trial.

^{^a}

Except for age, all characteristics are binary/categorical, and the summary statistics are expressed as No. (%) within the group. Percentages have been rounded and may not total 100.

^{^b}

As reported by the target RCT.

Compared with the RCT cohort, the EHR emulation cohort differed most among patients with stage I colorectal cancer. Overall, there was a higher prevalence overall of stage I cancer in the RCT (321 of 839 [38%]) compared with the EHR emulation cohort (182 of 943 [19%]). In the RCT cohort, 425 patients (51%) underwent LAC and 414 (49%) underwent OC, whereas in the EHR emulation cohort, 518 of 943 (55%) underwent LAC and 425 (45%) underwent OC (Table 1). In addition, colon adhesion prevalence was higher in the RCT cohorts (151 of 839 [18%]) compared with the EHR emulation cohorts (71 of 943 [7%]). Meanwhile, the prevalence of sigmoid colon cancer was lower in the RCT cohorts (320 of 839 [38%]) compared with the EHR emulation cohorts (423 of 943 [45%]).

Temporal Trends

The number of patients who fulfilled the eligibility criteria increased during the temporal periods, with 231 in 2006 to 2009, 277 in 2010 to 2013, and 435 in 2014 to 2017 (Figure 1). Eligible patients increasingly underwent LAC, from 81 (35%) in 2006 to 2009 through 132 (48%) in 2010 to 2013 and 305 (70%) in 2014 to 2017 (Figure 1).

Figure 2 summarizes the temporal trends in the distribution of the confounders, the association of the confounders with treatment assignment, and the association with overall survival. Interestingly, the data-driven analysis identified temporal trends in clinically relevant variables (eg, cancer clinical stage) along with variables seemingly not associated with colectomy (eg, codes for sciatica) (eResults in the Supplement). The temporal adjustments in both PS and outcome regression models explained significant variation of LAC assignment and overall survival (χ² test for PS model, 60.3 [P < .001]; χ² test for outcome regression model, 137.2 [P < .001]).

EHR Emulation: LAC vs OC

After controlling for confounding variables as well as temporal shift, we observed that LAC was not associated with a significant change in overall survival, consistent with the RCT findings. The estimated mean survival rates for the RCT target population when treated with LAC or OC using the entire EHR emulation cohort demonstrated similar trajectories and strongly overlapping confidence bands for pooled and time-stratified analyses (Figure 3). The estimated average treatment effect (LAC vs OC) based on the full EHR emulation cohort was −0.007 (95% CI, –0.070 to 0.057), suggesting that 5-year survival was similar after LAC and OC (Table 2). Using time-stratified EHR emulation cohorts as target populations resulted in similar conclusions.

Figure 3. — Lines depict the survival curves for time to death obtained from Kaplan-Meier survival analysis for the RCT and mean estimated overall survival probability based on doubly robust estimation; shaded areas indicate 95% CIs. We report the results based on the density ratio estimation in Table 2.

Table 2. Estimated 5-Year Overall Survival Rates for OC and LAC and Their Difference.

Time	Estimated 5-y survival (95% CI)^a
Time	OC group	LAC group	Average treatment effect^b
EHR emulation cohort
All	0.847 (0.805 to 0.890)	0.841 (0.793 to 0.888)	−0.007 (−0.070 to 0.057)
2006-2009	0.767 (0.694 to 0.837)	0.819 (0.705 to 0.922)	0.052 (−0.076 to 0.166)
2010-2013	0.855 (0.791 to 0.907)	0.834 (0.767 to 0.905)	−0.021 (−0.101 to 0.080)
2014-2017	0.885 (0.803 to 0.948)	0.857 (0.771 to 0.923)	−0.029 (−0.135 to 0.071)
Difference between EHR emulation and RCT cohorts^c
All	0.074 (0.011 to 0.131)	0.061 (–0.001 to 0.122)	−0.013 (−0.077 to 0.093)
2006-2009	–0.006 (–0.090 to 0.079)	0.039 (–0.085 to 0.152)	0.045 (−0.183 to 0.100)
2010-2013	0.083 (0.006 to 0.151)	0.055 (–0.025 to 0.140)	−0.028 (−0.092 to 0.121)
2014-2017	0.112 (0.019 to 0.193)	0.077 (–0.016 to 0.158)	−0.035 (−0.083 to 0.154)

Open in a new tab

Abbreviations: EHR, electronic health record; LAC, laparoscopy-assisted colectomy; OC, open colectomy; RCT, randomized clinical trial.

^{^a}

For each analysis, we applied 3 adjustments: outcome regression, propensity scores, and density ratio.

^{^b}

With the OC group as reference, a positive difference in the 5-year overall survival indicates higher survival probability with OC.

^{^c}

The difference is calculated between 5-year overall survival estimates of OC, LAC, and mean treatment effect for the EHR vs RCT cohorts.

We report the comparison with benchmark analyses in the eResults in the Supplement. The contrast between the benchmark analyses and the temporally adjusted analysis demonstrated the potential confounding bias due to ignoring temporal trends (eFigure 5 in the Supplement).

Although the estimated treatment differences based on EHR cohorts were consistent with those reported in the RCT cohorts, the estimated overall survival rates were higher in the EHR emulation cohorts than in the RCT cohorts (average treatment effect, −0.013 [95% CI, −0.077 to 0.093]) (Table 2 and Figure 3). Compared with the RCT conducted from 1995 to 2001, the estimated 5-year survival rates based on the EHR emulation analyses were 0.074 higher (95% CI, 0.011-0.131) for the OC group and 0.061 higher (95% CI, –0.001 to 0.122) for the LAC group. The estimated overall survival rates also increased during the study period in the EHR emulation cohort. The estimated 5-year survival rates in the time-stratified analyses increased from 0.767 (95% CI, 0.694-0.837) for the OC group and 0.819 (95% CI, 0.705-0.922) for the LAC group in 2006 to 2009 through 0.855 (95% CI, 0.791-0.907) for the OC group and 0.834 (95% CI, 0.767-0.905) for the LAC group in 2010 to 2013 and 0.885 (95% CI, 0.803-0.948) for the OC group and 0.857 (0.771 to 0.923) for the LAC group in 2014 to 2017.

Discussion

In this comparative effectiveness study, we demonstrate the association of temporal trends with study outcomes using EHR data by comparing results of an EHR emulation cohort with or without temporal adjustments with RCT results. Further, we showcased approaches that enable us to account for the temporal trends in clinical evidence through stratification by calendar time and cotraining of the time-dependent models. After accounting for confounding in temporal trends, our EHR emulation replicated the RCT findings of noninferiority in LAC compared with OC for overall survival within 5 years. We identified various temporal trends in the analysis, including the shift in patient population, treatment pattern, and overall survival rate.

Patients enrolled in RCTs may differ significantly from the general patient population in health care systems with regard to the distribution of their risk profiles and disease characteristics. Therefore, the efficacy reported from RCTs may not be always replicable in health care systems.²³ Furthermore, changes in overall disease management may not be observed or foreseen in RCTs but can be assessed using EHR data and provide insights for clinicians when making treatment decisions. Factors such as increasing trends of screening, improvement of surgical skills, or better previous lines of treatment that are associated with the treatment outcome may change over time, resulting in changes in effectiveness.^24,25

The trend of increasing adoption of LAC over time in the EHR emulation cohort was also observed in US nationwide data.^26,27,28 The increasing trend of patients with stages II and III cancer assigned to LAC could be associated with the accumulation of evidence from clinical trials suggesting that LAC is a viable alternative to OC,^29,30,31 leading to an increased adoption rate of LAC and surgeon training opportunities and proficiency.

The observed improvement in patients’ overall survival in the EHR cohort after 2006 vs the RCT cohort before 2001 and within the EHR cohort for early vs later temporal periods may be partially attributed to the development of cancer treatment over the years. For patients with early-stage colon cancer, studies have shown the associations of survival with surgical technical skills,²⁶ adjuvant chemotherapy,³² and mutations.³³ These factors might have suggested the pathway to better survival with better training for surgeons, increased adoption of adjuvant chemotherapy, and expanded options and lines for targeted therapies. Recurrence, a common precursor of mortality, is also reported to decrease.³⁴

Our temporal trend–adjusted analysis is applicable to clinical evidence studies of therapies for other diseases. Existing studies in colorectal cancer,³⁵ diabetes,³⁶ leukemia,³⁷ and other cancers^24,25 using EHR data can span years; however, no adjustment for potential temporal trends was reported. Existence of temporal trends raises doubts regarding the use of an external control group whose time span differs substantially from that of the experimental group. The issue of temporal trends has been raised in previous literature,³⁶ but the solution until now has been simplistic patient exclusion with potential sampling bias.³⁵ In the spirit of our approach of stratification by calendar year, the cotrained time-dependent external control can be considered an alternative approach.

The knowledge of findings from the target trial might be associated with the design of the EHR emulation. Franklin et al³⁸ considered prespecifying the analysis plan to reduce subjective bias, but they only involved structured data. The experience on incorporation of natural language–processing tools and temporal trend adjustment from our study may guide future analysis plans with improved granularity.

Limitations

Our study has limitations. First, treatment by indication bias may exist for the analysis during the 2006 to 2009 temporal period. With the extensive adjustment for confounding, the LAC overall survival is still visibly better than OC overall survival within the first 2 years. We hypothesize that only healthier patients underwent LAC from pioneering surgical teams because new procedures tend to be adopted cautiously. Treatment by indication could be caused by a small number of surgical teams capable of performing LAC during the period. The information regarding the indication mechanism might be contained in detailed radiological characteristics but not extractable for our analysis. Caution should be taken for EHR data collected close to the therapy approval date. Second, missing data on radiological tumor characteristics and stage in EHR may lead to inaccurate matching of the RCT eligibility criteria. We developed a rule-based extraction for tumor location, but validation was infeasible because of the lack of criterion standard information. We were unable to replicate the criterion for tumor size owing to the absence of exact information in EHR medical reports. Patients with missing information on stage were also excluded. Third, physician and surgeon information are not available for the study owing to privacy rules. The proficiency of the surgical team is essential for the quality of the colectomy and may affect the survival through short-term complications and long-term recurrence.^39,40,41 Validation of the analysis on a similar RCT is limited by data availability. These limitations await future study with imaging recognition, special privacy approvals, and advanced methods.

Conclusions

In this comparative effectiveness study, confounding bias due to temporal trends is a major concern for clinical evidence studies spanning several years. Stratification of calendar time and cotraining of models allowed us to account for these temporal trends in models using EHR data. With the proper adjustment, clinical evidence may supplement RCT in assessment of treatment outcomes as well as its evolution over time.

Supplement.

eMethods 1. Details of the Data Curation

eMethods 2. Details of the Statistical Analysis

eResults. Temporal Trends in Identified Confounding Factors and Comparison With Benchmark Analyses

eFigure 1. Study Schematics

eFigure 2. Receptor Operating Characteristics (ROC) for MAP Prediction, Number of Colorectal Cancer Diagnosis Code and Number of Mentions of Colorectal Cancer in Medical Notes

eFigure 3. Concordance Between NLP BMI Extraction and Structured BMI Over Patients With Both Extractions

eFigure 4. Summaries on Follow-up Durations for 943 Patients in EHR Emulation Cohort

eFigure 5. Estimated Overall Survival With 95% Confidence Band From Crude Analysis, Benchmark Analysis, Temporal Effect Adjusted Analysis, and RCT

eTable 1. Key Eligibility Criteria of the Target RCT and Their Correspondence in EHR Data

eTable 2. EHR Variables Forming the Filter for Colorectal Cancer Data Mart

Click here for additional data file.^{(530.4KB, pdf)}

References

1.US Food and Drug Administration . Real-world data: assessing electronic health records and medical claims data to support regulatory decision-making for drug and biological products. Docket Number: FDA-2020-D-2307. September 2021. Accessed May 23, 2022. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/real-world-data-assessing-electronic-health-records-and-medical-claims-data-support-regulatory [DOI] [PMC free article] [PubMed]
2.Carrigan G, Whipple S, Capra WB, et al. Using electronic health records to derive control arms for early phase single-arm lung cancer trials: proof-of-concept in randomized controlled trials. Clin Pharmacol Ther. 2020;107(2):369-377. doi: 10.1002/cpt.1586 [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Davies J, Martinec M, Delmar P, et al. Comparative effectiveness from a single-arm trial and real-world data: alectinib versus ceritinib. J Comp Eff Res. 2018;7(9):855-865. doi: 10.2217/cer-2018-0032 [DOI] [PubMed] [Google Scholar]
4.Ventz S, Lai A, Cloughesy TF, Wen PY, Trippa L, Alexander BM. Design and evaluation of an external control arm using prior clinical trials and real-world data. Clin Cancer Res. 2019;25(16):4993-5001. doi: 10.1158/1078-0432.CCR-19-0820 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Fralick M, Kesselheim AS, Avorn J, Schneeweiss S. Use of health care databases to support supplemental indications of approved medications. JAMA Intern Med. 2018;178(1):55-63. doi: 10.1001/jamainternmed.2017.3919 [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Pletcher MJ, Flaherman V, Najafi N, et al. Randomized controlled trials of electronic health record interventions: design, conduct, and reporting considerations. Ann Intern Med. 2020;172(11)(suppl):S85-S91. doi: 10.7326/M19-0877 [DOI] [PubMed] [Google Scholar]
7.Beaulieu-Jones BK, Finlayson SG, Yuan W, et al. Examining the use of real-world evidence in the regulatory process. Clin Pharmacol Ther. 2020;107(4):843-852. doi: 10.1002/cpt.1658 [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Finlayson SG, Subbaswamy A, Singh K, et al. The clinician and dataset shift in artificial intelligence. N Engl J Med. 2021;385(3):283-286. doi: 10.1056/NEJMc2104626 [DOI] [PMC free article] [PubMed] [Google Scholar]
9.ClinicalTrials.gov. Laparoscopic-Assisted Surgery Compared With Open Surgery in Treating Patients With Colon Cancer. NCT00002575. Accessed April 16, 2020. https://clinicaltrials.gov/ct2/show/NCT00002575
10.Nelson H, Sargent DJ, Wieand HS, et al. ; Clinical Outcomes of Surgical Therapy Study Group . A comparison of laparoscopically assisted and open colectomy for colon cancer. N Engl J Med. 2004;350(20):2050-2059. doi: 10.1056/NEJMoa032651 [DOI] [PubMed] [Google Scholar]
11.Project Data Sphere . Data sharing platform. Accessed April 16, 2020. https://www.projectdatasphere.org/
12.Liao KP, Sun J, Cai TA, et al. High-throughput multimodal automated phenotyping (MAP) with application to PheWAS. J Am Med Inform Assoc. 2019;26(11):1255-1262. doi: 10.1093/jamia/ocz066 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Yuan Q, Cai T, Hong C, et al. Performance of a machine learning algorithm using electronic health record data to identify and estimate survival in a longitudinal cohort of patients with lung cancer. JAMA Netw Open. 2021;4(7):e2114723. doi: 10.1001/jamanetworkopen.2021.14723 [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Senagore AJ, Delaney CP, Madboulay K, Brady KM, Fazio VW. Laparoscopic colectomy in obese and nonobese patients. J Gastrointest Surg. 2003;7(4):558-561. Published correction appears in J Gastrointest Surg. 2003;7(5):712. doi: 10.1016/S1091-255X(02)00124-5 [DOI] [PubMed] [Google Scholar]
15.Cai T, Zhang L, Yang N, et al. EXTraction of EMR numerical data: an efficient and generalizable tool to EXTEND clinical research. BMC Med Inform Decis Mak. 2019;19(1):226. doi: 10.1186/s12911-019-0970-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Denny JC, Bastarache L, Ritchie MD, et al. Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association study data. Nat Biotechnol. 2013;31(12):1102-1110. doi: 10.1038/nbt.2749 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Driscoll JJ, Rixe O. Overall survival: still the gold standard: why overall survival remains the definitive end point in cancer clinical trials. Cancer J. 2009;15(5):401-405. doi: 10.1097/PPO.0b013e3181bdc2e0 [DOI] [PubMed] [Google Scholar]
18.Law WL, Lee YM, Choi HK, Seto CL, Ho JW. Impact of laparoscopic resection for colorectal cancer on operative outcomes and survival. Ann Surg. 2007;245(1):1-7. doi: 10.1097/01.sla.0000218170.41992.23 [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Bang H, Robins JM. Doubly robust estimation in missing data and causal inference models. Biometrics. 2005;61(4):962-973. doi: 10.1111/j.1541-0420.2005.00377.x [DOI] [PubMed] [Google Scholar]
20.Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. J R Stat Soc Series B Stat Methodol. 2006;68(1):49-67. doi: 10.1111/j.1467-9868.2005.00532.x [DOI] [Google Scholar]
21.Zou H. The adaptive lasso and its oracle properties. J Am Stat Assoc. 2006;101(476):1418-1429. doi: 10.1198/016214506000000735 [DOI] [Google Scholar]
22.Wang H, Leng C. A note on adaptive group lasso. Comput Stat Data Anal. 2008;52(12):5277-5286. doi: 10.1016/j.csda.2008.05.006 [DOI] [Google Scholar]
23.Di Maio M, Perrone F, Conte P. Real-world evidence in oncology: opportunities and limitations. Oncologist. 2020;25(5):e746-e752. doi: 10.1634/theoncologist.2019-0647 [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Eisenhauer EA. Real-world evidence in the treatment of ovarian cancer. Ann Oncol. 2017;28(suppl 8):e2218371. doi: 10.1093/annonc/mdx443 [DOI] [PubMed] [Google Scholar]
25.Phillips CM, Parmar A, Guo H, et al. Assessing the efficacy-effectiveness gap for cancer therapies: a comparison of overall survival and toxicity between clinical trial and population-based, real-world data for contemporary parenteral cancer therapeutics. Cancer. 2020;126(8):1717-1726. doi: 10.1002/cncr.32697 [DOI] [PubMed] [Google Scholar]
26.Bardakcioglu O, Khan A, Aldridge C, Chen J. Growth of laparoscopic colectomy in the United States: analysis of regional and socioeconomic factors over time. Ann Surg. 2013;258(2):270-274. doi: 10.1097/SLA.0b013e31828faa66 [DOI] [PubMed] [Google Scholar]
27.Lee MG, Chiu CC, Wang CC, et al. Trends and outcomes of surgical treatment for colorectal cancer between 2004 and 2012—an analysis using National Inpatient Database. Sci Rep. 2017;7(1):2006. doi: 10.1038/s41598-017-02224-y [DOI] [PMC free article] [PubMed] [Google Scholar]
28.de Neree Tot Babberich MPM, van Groningen JT, Dekker E, et al. Laparoscopic conversion in colorectal cancer surgery; is there any improvement over time at a population level? Surg Endosc. 2018;32(7):3234-3246. doi: 10.1007/s00464-018-6042-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Parker JM, Feldmann TF, Cologne KG. Advances in laparoscopic colorectal surgery. Surg Clin North Am. 2017;97(3):547-560. doi: 10.1016/j.suc.2017.01.005 [DOI] [PubMed] [Google Scholar]
30.Kaiser AM. Evolution and future of laparoscopic colorectal surgery. World J Gastroenterol. 2014;20(41):15119-15124. doi: 10.3748/wjg.v20.i41.15119 [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Bonjer HJ, Hop WCJ, Nelson H, et al. ; Transatlantic Laparoscopically Assisted vs Open Colectomy Trials Study Group . Laparoscopically assisted vs open colectomy for colon cancer: a meta-analysis. Arch Surg. 2007;142(3):298-303. doi: 10.1001/archsurg.142.3.298 [DOI] [PubMed] [Google Scholar]
32.Achilli P, Crippa J, Grass F, et al. Survival impact of adjuvant chemotherapy in patients with stage IIA colon cancer: analysis of the National Cancer Database. Int J Cancer. 2021;148(1):161-169. doi: 10.1002/ijc.33203 [DOI] [PubMed] [Google Scholar]
33.Dienstmann R, Mason MJ, Sinicrope FA, et al. Prediction of overall survival in stage II and III colon cancer beyond TNM system: a retrospective, pooled biomarker study. Ann Oncol. 2017;28(5):1023-1031. doi: 10.1093/annonc/mdx052 [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Osterman E, Glimelius B. Recurrence risk after up-to-date colon cancer staging, surgery, and pathology: analysis of the entire Swedish population. Dis Colon Rectum. 2018;61(9):1016-1025. doi: 10.1097/DCR.0000000000001158 [DOI] [PubMed] [Google Scholar]
35.Schröder C, Lawrance M, Li C, et al. Building external control arms from patient-level electronic health record data to replicate the randomized IMblaze370 control arm in metastatic colorectal cancer. JCO Clin Cancer Inform. 2021;5:450-458. doi: 10.1200/CCI.20.00149 [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Seeger JD, Nunes A, Loughlin AM. Using RWE research to extend clinical trials in diabetes: an example with implications for the future. Diabetes Obes Metab. 2020;22(suppl 3):35-44. doi: 10.1111/dom.14021 [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Islam P, Mato AR. Utilizing real-world evidence (RWE) to improve care in chronic lymphocytic leukemia: challenges and opportunities. Curr Hematol Malig Rep. 2020;15(4):254-260. doi: 10.1007/s11899-020-00584-3 [DOI] [PubMed] [Google Scholar]
38.Franklin JM, Patorno E, Desai RJ, et al. Emulating randomized clinical trials with nonrandomized real-world evidence studies: first results from the RCT DUPLICATE Initiative. Circulation. 2021;143(10):1002-1013. doi: 10.1161/CIRCULATIONAHA.120.051718 [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Luglio G, De Palma GD, Tarquini R, et al. Laparoscopic colorectal surgery in learning curve: role of implementation of a standardized technique and recovery protocol: a cohort study. Ann Med Surg (Lond). 2015;4(2):89-94. doi: 10.1016/j.amsu.2015.03.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
40.Damle RN, Macomber CW, Flahive JM, et al. Surgeon volume and elective resection for colon cancer: an analysis of outcomes and use of laparoscopy. J Am Coll Surg. 2014;218(6):1223-1230. doi: 10.1016/j.jamcollsurg.2014.01.057 [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Huo YR, Phan K, Morris DL, Liauw W. Systematic review and a meta-analysis of hospital and surgeon volume/outcome relationships in colorectal cancer surgery. J Gastrointest Oncol. 2017;8(3):534-546. doi: 10.21037/jgo.2017.01.25 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement.

eMethods 1. Details of the Data Curation

eMethods 2. Details of the Statistical Analysis

eResults. Temporal Trends in Identified Confounding Factors and Comparison With Benchmark Analyses

eFigure 1. Study Schematics

eFigure 2. Receptor Operating Characteristics (ROC) for MAP Prediction, Number of Colorectal Cancer Diagnosis Code and Number of Mentions of Colorectal Cancer in Medical Notes

eFigure 3. Concordance Between NLP BMI Extraction and Structured BMI Over Patients With Both Extractions

eFigure 4. Summaries on Follow-up Durations for 943 Patients in EHR Emulation Cohort

eFigure 5. Estimated Overall Survival With 95% Confidence Band From Crude Analysis, Benchmark Analysis, Temporal Effect Adjusted Analysis, and RCT

eTable 1. Key Eligibility Criteria of the Target RCT and Their Correspondence in EHR Data

eTable 2. EHR Variables Forming the Filter for Colorectal Cancer Data Mart

Click here for additional data file.^{(530.4KB, pdf)}

[zoi220535r1] 1.US Food and Drug Administration . Real-world data: assessing electronic health records and medical claims data to support regulatory decision-making for drug and biological products. Docket Number: FDA-2020-D-2307. September 2021. Accessed May 23, 2022. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/real-world-data-assessing-electronic-health-records-and-medical-claims-data-support-regulatory [DOI] [PMC free article] [PubMed]

[zoi220535r2] 2.Carrigan G, Whipple S, Capra WB, et al. Using electronic health records to derive control arms for early phase single-arm lung cancer trials: proof-of-concept in randomized controlled trials. Clin Pharmacol Ther. 2020;107(2):369-377. doi: 10.1002/cpt.1586 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r3] 3.Davies J, Martinec M, Delmar P, et al. Comparative effectiveness from a single-arm trial and real-world data: alectinib versus ceritinib. J Comp Eff Res. 2018;7(9):855-865. doi: 10.2217/cer-2018-0032 [DOI] [PubMed] [Google Scholar]

[zoi220535r4] 4.Ventz S, Lai A, Cloughesy TF, Wen PY, Trippa L, Alexander BM. Design and evaluation of an external control arm using prior clinical trials and real-world data. Clin Cancer Res. 2019;25(16):4993-5001. doi: 10.1158/1078-0432.CCR-19-0820 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r5] 5.Fralick M, Kesselheim AS, Avorn J, Schneeweiss S. Use of health care databases to support supplemental indications of approved medications. JAMA Intern Med. 2018;178(1):55-63. doi: 10.1001/jamainternmed.2017.3919 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r6] 6.Pletcher MJ, Flaherman V, Najafi N, et al. Randomized controlled trials of electronic health record interventions: design, conduct, and reporting considerations. Ann Intern Med. 2020;172(11)(suppl):S85-S91. doi: 10.7326/M19-0877 [DOI] [PubMed] [Google Scholar]

[zoi220535r7] 7.Beaulieu-Jones BK, Finlayson SG, Yuan W, et al. Examining the use of real-world evidence in the regulatory process. Clin Pharmacol Ther. 2020;107(4):843-852. doi: 10.1002/cpt.1658 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r8] 8.Finlayson SG, Subbaswamy A, Singh K, et al. The clinician and dataset shift in artificial intelligence. N Engl J Med. 2021;385(3):283-286. doi: 10.1056/NEJMc2104626 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r9] 9.ClinicalTrials.gov. Laparoscopic-Assisted Surgery Compared With Open Surgery in Treating Patients With Colon Cancer. NCT00002575. Accessed April 16, 2020. https://clinicaltrials.gov/ct2/show/NCT00002575

[zoi220535r10] 10.Nelson H, Sargent DJ, Wieand HS, et al. ; Clinical Outcomes of Surgical Therapy Study Group . A comparison of laparoscopically assisted and open colectomy for colon cancer. N Engl J Med. 2004;350(20):2050-2059. doi: 10.1056/NEJMoa032651 [DOI] [PubMed] [Google Scholar]

[zoi220535r11] 11.Project Data Sphere . Data sharing platform. Accessed April 16, 2020. https://www.projectdatasphere.org/

[zoi220535r12] 12.Liao KP, Sun J, Cai TA, et al. High-throughput multimodal automated phenotyping (MAP) with application to PheWAS. J Am Med Inform Assoc. 2019;26(11):1255-1262. doi: 10.1093/jamia/ocz066 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r13] 13.Yuan Q, Cai T, Hong C, et al. Performance of a machine learning algorithm using electronic health record data to identify and estimate survival in a longitudinal cohort of patients with lung cancer. JAMA Netw Open. 2021;4(7):e2114723. doi: 10.1001/jamanetworkopen.2021.14723 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r14] 14.Senagore AJ, Delaney CP, Madboulay K, Brady KM, Fazio VW. Laparoscopic colectomy in obese and nonobese patients. J Gastrointest Surg. 2003;7(4):558-561. Published correction appears in J Gastrointest Surg. 2003;7(5):712. doi: 10.1016/S1091-255X(02)00124-5 [DOI] [PubMed] [Google Scholar]

[zoi220535r15] 15.Cai T, Zhang L, Yang N, et al. EXTraction of EMR numerical data: an efficient and generalizable tool to EXTEND clinical research. BMC Med Inform Decis Mak. 2019;19(1):226. doi: 10.1186/s12911-019-0970-1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r16] 16.Denny JC, Bastarache L, Ritchie MD, et al. Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association study data. Nat Biotechnol. 2013;31(12):1102-1110. doi: 10.1038/nbt.2749 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r17] 17.Driscoll JJ, Rixe O. Overall survival: still the gold standard: why overall survival remains the definitive end point in cancer clinical trials. Cancer J. 2009;15(5):401-405. doi: 10.1097/PPO.0b013e3181bdc2e0 [DOI] [PubMed] [Google Scholar]

[zoi220535r18] 18.Law WL, Lee YM, Choi HK, Seto CL, Ho JW. Impact of laparoscopic resection for colorectal cancer on operative outcomes and survival. Ann Surg. 2007;245(1):1-7. doi: 10.1097/01.sla.0000218170.41992.23 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r19] 19.Bang H, Robins JM. Doubly robust estimation in missing data and causal inference models. Biometrics. 2005;61(4):962-973. doi: 10.1111/j.1541-0420.2005.00377.x [DOI] [PubMed] [Google Scholar]

[zoi220535r20] 20.Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. J R Stat Soc Series B Stat Methodol. 2006;68(1):49-67. doi: 10.1111/j.1467-9868.2005.00532.x [DOI] [Google Scholar]

[zoi220535r21] 21.Zou H. The adaptive lasso and its oracle properties. J Am Stat Assoc. 2006;101(476):1418-1429. doi: 10.1198/016214506000000735 [DOI] [Google Scholar]

[zoi220535r22] 22.Wang H, Leng C. A note on adaptive group lasso. Comput Stat Data Anal. 2008;52(12):5277-5286. doi: 10.1016/j.csda.2008.05.006 [DOI] [Google Scholar]

[zoi220535r23] 23.Di Maio M, Perrone F, Conte P. Real-world evidence in oncology: opportunities and limitations. Oncologist. 2020;25(5):e746-e752. doi: 10.1634/theoncologist.2019-0647 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r24] 24.Eisenhauer EA. Real-world evidence in the treatment of ovarian cancer. Ann Oncol. 2017;28(suppl 8):e2218371. doi: 10.1093/annonc/mdx443 [DOI] [PubMed] [Google Scholar]

[zoi220535r25] 25.Phillips CM, Parmar A, Guo H, et al. Assessing the efficacy-effectiveness gap for cancer therapies: a comparison of overall survival and toxicity between clinical trial and population-based, real-world data for contemporary parenteral cancer therapeutics. Cancer. 2020;126(8):1717-1726. doi: 10.1002/cncr.32697 [DOI] [PubMed] [Google Scholar]

[zoi220535r26] 26.Bardakcioglu O, Khan A, Aldridge C, Chen J. Growth of laparoscopic colectomy in the United States: analysis of regional and socioeconomic factors over time. Ann Surg. 2013;258(2):270-274. doi: 10.1097/SLA.0b013e31828faa66 [DOI] [PubMed] [Google Scholar]

[zoi220535r27] 27.Lee MG, Chiu CC, Wang CC, et al. Trends and outcomes of surgical treatment for colorectal cancer between 2004 and 2012—an analysis using National Inpatient Database. Sci Rep. 2017;7(1):2006. doi: 10.1038/s41598-017-02224-y [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r28] 28.de Neree Tot Babberich MPM, van Groningen JT, Dekker E, et al. Laparoscopic conversion in colorectal cancer surgery; is there any improvement over time at a population level? Surg Endosc. 2018;32(7):3234-3246. doi: 10.1007/s00464-018-6042-2 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r29] 29.Parker JM, Feldmann TF, Cologne KG. Advances in laparoscopic colorectal surgery. Surg Clin North Am. 2017;97(3):547-560. doi: 10.1016/j.suc.2017.01.005 [DOI] [PubMed] [Google Scholar]

[zoi220535r30] 30.Kaiser AM. Evolution and future of laparoscopic colorectal surgery. World J Gastroenterol. 2014;20(41):15119-15124. doi: 10.3748/wjg.v20.i41.15119 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r31] 31.Bonjer HJ, Hop WCJ, Nelson H, et al. ; Transatlantic Laparoscopically Assisted vs Open Colectomy Trials Study Group . Laparoscopically assisted vs open colectomy for colon cancer: a meta-analysis. Arch Surg. 2007;142(3):298-303. doi: 10.1001/archsurg.142.3.298 [DOI] [PubMed] [Google Scholar]

[zoi220535r32] 32.Achilli P, Crippa J, Grass F, et al. Survival impact of adjuvant chemotherapy in patients with stage IIA colon cancer: analysis of the National Cancer Database. Int J Cancer. 2021;148(1):161-169. doi: 10.1002/ijc.33203 [DOI] [PubMed] [Google Scholar]

[zoi220535r33] 33.Dienstmann R, Mason MJ, Sinicrope FA, et al. Prediction of overall survival in stage II and III colon cancer beyond TNM system: a retrospective, pooled biomarker study. Ann Oncol. 2017;28(5):1023-1031. doi: 10.1093/annonc/mdx052 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r34] 34.Osterman E, Glimelius B. Recurrence risk after up-to-date colon cancer staging, surgery, and pathology: analysis of the entire Swedish population. Dis Colon Rectum. 2018;61(9):1016-1025. doi: 10.1097/DCR.0000000000001158 [DOI] [PubMed] [Google Scholar]

[zoi220535r35] 35.Schröder C, Lawrance M, Li C, et al. Building external control arms from patient-level electronic health record data to replicate the randomized IMblaze370 control arm in metastatic colorectal cancer. JCO Clin Cancer Inform. 2021;5:450-458. doi: 10.1200/CCI.20.00149 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r36] 36.Seeger JD, Nunes A, Loughlin AM. Using RWE research to extend clinical trials in diabetes: an example with implications for the future. Diabetes Obes Metab. 2020;22(suppl 3):35-44. doi: 10.1111/dom.14021 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r37] 37.Islam P, Mato AR. Utilizing real-world evidence (RWE) to improve care in chronic lymphocytic leukemia: challenges and opportunities. Curr Hematol Malig Rep. 2020;15(4):254-260. doi: 10.1007/s11899-020-00584-3 [DOI] [PubMed] [Google Scholar]

[zoi220535r38] 38.Franklin JM, Patorno E, Desai RJ, et al. Emulating randomized clinical trials with nonrandomized real-world evidence studies: first results from the RCT DUPLICATE Initiative. Circulation. 2021;143(10):1002-1013. doi: 10.1161/CIRCULATIONAHA.120.051718 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r39] 39.Luglio G, De Palma GD, Tarquini R, et al. Laparoscopic colorectal surgery in learning curve: role of implementation of a standardized technique and recovery protocol: a cohort study. Ann Med Surg (Lond). 2015;4(2):89-94. doi: 10.1016/j.amsu.2015.03.003 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r40] 40.Damle RN, Macomber CW, Flahive JM, et al. Surgeon volume and elective resection for colon cancer: an analysis of outcomes and use of laparoscopy. J Am Coll Surg. 2014;218(6):1223-1230. doi: 10.1016/j.jamcollsurg.2014.01.057 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi220535r41] 41.Huo YR, Phan K, Morris DL, Liauw W. Systematic review and a meta-analysis of hospital and surgeon volume/outcome relationships in colorectal cancer surgery. J Gastrointest Oncol. 2017;8(3):534-546. doi: 10.21037/jgo.2017.01.25 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Temporal Trends in Clinical Evidence of 5-Year Survival Within Electronic Health Records Among Patients With Early-Stage Colon Cancer Managed With Laparoscopy-Assisted Colectomy vs Open Colectomy

Jue Hou, PhD

Rachel Zhao, BSc

Tianrun Cai, MD, PhD

Brett Beaulieu-Jones, PhD

Thany Seyok, MS

Kumar Dahal, MS

Qianyu Yuan, PhD

Xin Xiong, BSc

Clara-Lea Bonzel, MSc

Claire Fox, MPH

David C Christiani, MD

Thomas Jemielita, PhD

Katherine P Liao, MD, MPH

Kai-Li Liaw, PhD

Tianxi Cai, ScD

Key Points

Question

Findings

Meaning

Abstract

Importance

Objective

Design, Setting, and Participants

Exposures

Main Outcomes and Measures

Results

Conclusions and Relevance

Introduction

Methods

The RCT

EHR Cohort for Emulating the RCT

Statistical Analysis

Results

Study Cohorts

Figure 1. Construction of Electronic Health Record (EHR) Emulation Cohort and Description of Randomized Clinical Trial (RCT) Cohort.

Table 1. Characteristics of the Treatment Groups.

Temporal Trends

Figure 2. Model Coefficients in Propensity Score (PS) and Cox Proportional Hazards Models.

EHR Emulation: LAC vs OC

Figure 3. Estimated Overall Survival With 95% CI Band of the Electronic Health Record (EHR) Emulation vs Randomized Clinical Trial (RCT) Stratified by Time.

Table 2. Estimated 5-Year Overall Survival Rates for OC and LAC and Their Difference.

Discussion

Limitations

Conclusions

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases