Development and Implementation of a Digital Quality Measure of Emergency Cancer Diagnosis

Abstract

PURPOSE

Missed and delayed cancer diagnoses are common, harmful, and often preventable. Automated measures of quality of cancer diagnosis are lacking but could identify gaps and guide interventions. We developed and implemented a digital quality measure (dQM) of cancer emergency presentation (EP) using electronic health record databases of two health systems and characterized the measure's association with missed opportunities for diagnosis (MODs) and mortality.

METHODS

On the basis of literature and expert input, we defined EP as a new cancer diagnosis within 30 days after emergency department or inpatient visit. We identified EPs for lung cancer and colorectal cancer (CRC) in the Department of Veterans Affairs (VA) and Geisinger from 2016 to 2020. We validated measure accuracy and identified preceding MODs through standardized chart review of 100 records per cancer per health system. Using VA's longitudinal encounter and mortality data, we applied logistic regression to assess EP's association with 1-year mortality, adjusting for cancer stage and demographics.

RESULTS

Among 38,565 and 2,914 patients with lung cancer and 14,674 and 1,649 patients with CRCs at VA and Geisinger, respectively, our dQM identified EPs in 20.9% and 9.4% of lung cancers, and 22.4% and 7.5% of CRCs. Chart reviews revealed high positive predictive values for EPs across sites and cancer types (72%-90%), and a substantial percent represented MODs (48.8%-84.9%). EP was associated with significantly higher odds of 1-year mortality for lung cancer and CRC (adjusted odds ratio, 1.78 and 1.83, respectively, 95% CI, 1.63 to 1.86 and 1.61 to 2.07).

CONCLUSION

A dQM for cancer EP was strongly associated with both mortality and MODs. The findings suggest a promising automated approach to measuring quality of cancer diagnosis in US health systems.

INTRODUCTION

Missed and delayed cancer diagnosis is common, affecting over one third of patients with colorectal or lung cancer.^1,2 Delayed cancer diagnosis is often preventable and is a leading cause of harm in US malpractice claims.^3,4 Prolonged diagnostic intervals and treatment delays often arise from missed opportunities to initiate or complete appropriate diagnostic investigation.^1,2 These missed opportunities for diagnosis (MODs) commonly result from diagnostic process breakdowns, for example, missed test results, lost patient referrals, and clinician-patient miscommunication.

CONTEXT

• Key Objective

• How can emergency presentation (EP) of cancer be measured electronically in integrated health systems in the United States, and what is its association with preceding missed opportunities for diagnosis (MODs) and subsequent mortality?

• Knowledge Generated

• Using an approach modeled on the English Routes to Diagnosis program, a digital quality measure of emergency cancer presentation was successfully defined and implemented in Veterans Affairs and Geisinger health systems. EPs of lung cancer and colorectal cancer had prior MODs in 49%-85% of cases by chart review, and 1.8-fold increased odds of 1-year mortality, independent of cancer stage.

• Relevance (S.B. Wheeler)

• Given the importance of intervening to reduce MODs and the higher mortality associated with EP, this measure holds promise for motivating improvement in diagnostic pathways. However, its utility in practice must be explored in other healthcare systems and cancer sites, and attribution must be carefully considered before it can be fully actionable.*

  *Relevance section written by JCO Associate Editor Stephanie B. Wheeler, PhD, MPH.

In 2015, the National Academy of Medicine called for approaches to identify and mitigate MODs.⁵ More recently, the Centers for Medicare & Medicaid Services (CMS) emphasized digital quality measures (dQMs) that evaluate longitudinal electronic health record (EHR) data across multiple care settings to address measurement gaps in quality.⁶ However, no dQMs in the United States assess cancer diagnostic quality from the perspective of MODs. MODs remain difficult to measure and often require labor-intensive chart reviews to detect.⁵ Cancer-related dQMs could help quantify diagnostic quality, identify areas for improvement, and guide interventions to prevent further harm.

In England, markers of diagnostic quality have been implemented through Routes to Diagnosis programs that track and categorize patient pathways to cancer diagnosis.⁷ Such programs define emergency presentation (EP) of cancer as a diagnosis up to 28 days after unplanned inpatient admission. While this definition does not directly measure diagnostic quality, it has good face validity as it predicts higher mortality and poorer patient experience.^8,9 Patient, cancer, and health system factors contribute to EP. While not all EPs are preventable, many related to patient factors, MODs, or system delays are.¹⁰ Typical EP scenarios include (1) missed follow-up of abnormal cancer screen, (2) delayed patient presentation due to psychosocial or logistical barriers, (3) delayed investigation after symptomatic presentation due to waiting times or premature diagnostic closure, or (4) rapidly progressing cancer with minimal prodrome. Scenarios (1) through (3) are potentially preventable.

There is a paucity of literature regarding EPs in the United States, partly due to lack of a standard operational definition suitable to data models.^11-13 Currently, there are knowledge gaps about prevalence and characteristics of EPs in the United States, including their potential association with diagnostic quality and utility as a proxy indicator of MODs. Our objective was to develop, validate, and implement a dQM to quantify EPs of cancer in two US health systems, determine factors contributing to EPs, and assess association of EPs with patient survival.

METHODS

Settings

We developed and evaluated an EP dQM using EHR data from two large, integrated US health systems: US Department of Veterans Affairs (VA) and Geisinger. VA is a national network of over 130 facilities serving over 9 million veterans. Geisinger is an integrated, largely rural system in Pennsylvania serving over 3 million patients. Both have tertiary referral centers, multispecialty clinics, and satellite primary clinics. The study was approved by Baylor College of Medicine's institutional review board (protocol H-46611).

Measure Development

We adapted the Safer Dx Trigger Tools Framework¹⁴ to develop the dQM using a seven-step process: determining data sources, conceptually defining measurement targets, operationally identifying targets in data, constructing electronic algorithms, applying algorithms to data and reviewing identified records, assessing algorithm performance (whether flagged records truly had EP), and iteratively improving performance.

We convened a technical expert panel of physicians, researchers, administrators, and informaticians, and separately, a panel of patients to develop a conceptual definition and approach to operationalizing EP. Using panel input, we conceptually defined EP as a new diagnosis of cancer following an emergency care episode related in some way to the cancer (symptoms or findings). Operationally, we defined episode of emergency care as an emergency department (ED) visit and/or unplanned hospitalization, where any emergency care episodes within 30 days before cancer diagnosis are related (Fig 1). Planned admissions were identified using a previously validated algorithm¹⁵ and not considered EPs.

FIG 1.

Timeline of EP and example scenario with dates. Arrows indicate prespecified intervals used by the digital quality measure and chart reviewers. If a cancer signal is present before EP, reviewers allow 7 days to order follow-up testing and 30 days (for lung cancer) to complete follow-up. Using the dates of the example scenario, follow-up was not ordered at the time of the first cancer signal, which would be judged as a missed opportunity for diagnosis. CRC, colorectal cancer; CT, computed tomography; EP, emergency presentation; FIT, fecal immunochemical test; FOBT, fecal occult blood test; MOD, missed opportunity for diagnosis; PET, positron emission tomography.

Using this definition, we established retrospective cohorts of patients newly diagnosed with primary lung or colorectal cancer (CRC) between January 1, 2016 and December 31, 2020 using cancer registries. To exclude instances without opportunities for earlier care within the system, we excluded patients without primary care encounters within 2 years before diagnosis. This cohort was the dQM denominator. Next, using integrated outpatient/inpatient data, we used electronic algorithms to flag a subset of patients matching our operational EP definition. This subset was the dQM numerator.

Figure 2 shows a flow diagram, using CRC in VA as an example, where 14,674 is the denominator (new CRC diagnoses), and 3,291 is the numerator (EPs). This approach was similar to the UK National Cancer Intelligence Network's (NCIN) Routes to Diagnosis methods that categorize health care encounters preceding cancer diagnosis.⁷

FIG 2.

Flow diagram of digital quality measure criteria and chart review process. For all new cancer diagnoses, the diagram indicates the number meeting criteria for EP, the sampling for chart review, and the number meeting criteria for missed opportunities. The example shown is for colorectal cancer in the VA system. EP, emergency presentation; MOD, missed opportunity for diagnosis; VA, Veterans Affairs.

Data sources included VA's Corporate Data Warehouse¹⁶ and Geisinger's EHR data warehouse. Cancer registries specified tumor stage at diagnosis and were linked deterministically with patient sex, race/ethnicity, and age. Mortality was measured only in VA due to feasibility, using encounters between cancer diagnosis and death, and integrated VA-specific vital status data.

Validation and Implementation

We initially sampled 10-20 flagged records per cancer per system to assess algorithm performance. We trained two physician chart reviewers (U. Mushtaq, U. Mir) with backgrounds in general medicine, and we refined our data collection instrument (Data Supplement, online only), adapted from prior work.^1,2,17-19 After iteratively refining and finalizing algorithms, we randomly sampled 100 EP-flagged records per cancer per system for validation review (presence/absence of EP per operational criteria). True positives were classified into three EP routes: (1) cancer diagnosed in the ED or hospitalization directly linked to the presentation, (2) diagnosis in subsequent encounter when cancer was suspected at ED presentation, and (3) cancer not suspected at ED arrival but concerning findings secondarily discovered and evaluated later. False positives were not assessed further (Fig 2).

For true positive EPs, reviewers assessed for MODs in the 2 years preceding diagnosis. On the basis of prior literature,^1,2,20 reviewers noted a MOD when they found failure to initiate (type I) or complete (type II) guideline-indicated investigation within a predetermined timeframe in the presence of cancer signals (defined as red-flag symptoms and test results that warrant further evaluation for cancer). We prespecified criteria for cancer signals that reviewers could use to evaluate the medical record objectively for cancer-related red flags. We also established specific timeframes on the basis of prior work and published literature to help discern the timeliness of workup on the basis of these signals.^1,2,20-27 We added these elements to the data collection instrument to standardize data collection and review processes to identify MODs. Reviewers discussed all medical records with unclear presence/absence of MODs in larger team meetings. The timeframe to initiate investigation was 7 days from the appearance of a documented cancer signal, and the timeframe to complete investigation was 30 days for lung cancer and 60 days for CRC (Fig 1). There are no current national standards for timely workup and diagnosis of these cancers. The timeframes chosen for this study were previously developed and validated by chart review² and align with existing policies and standards on the timely diagnosis of suspected cancer.²⁸ Reviewers evaluated potential MODs within the clinical context inferred from EHR documentation, accounting for acceptable practice variations. This MOD evaluation included the assessment of whether cancer screening was up-to-date for patients eligible on the basis of guidelines^29,30 at the time of EP.

Finally, reviewers assessed factors contributing to MODs on the basis of five previously described diagnostic process dimensions (patient-related, patient-provider encounter, diagnostic test performance/interpretation, follow-up and tracking, and referrals; see the Data Supplement).¹⁷ Reviewers and analysts met regularly to answer questions and ensure consistency. Figure 2 shows the flow of charts through sampling and manual review using VA CRC as an example.

Statistics

Hypothesis tests for associations in bivariate tabular data were conducted using a two-tailed chi-square test, with Bonferroni multiple comparison correction where applicable. All-cause mortality at 1 year was modeled using logistic regression, parallel to prior work^8,13 that used this method when follow-up was highly complete. EP status, sex, age, cancer stage, rurality, Elixhauser comorbidity index,^31-33 and race/ethnicity were included as predictors. Separate models were fit for lung cancer and CRC. Mortality analysis was restricted to VA due to availability of aggregated data for ascertainment of mortality. Follow-up was measured at 30, 90, 180, and 365 days after cancer diagnosis and defined as any completed visit after the time point. Analyses were conducted in the Python programming language using the SciPy and statsmodels libraries.

RESULTS

There were 38,565 new patients with lung cancer at VA and 2,914 at Geisinger, of which 8,063 (20.9%) and 275 (9.4%), respectively, were flagged by the dQM as EPs. For CRC, there were 14,674 new patients at VA and 1,649 at Geisinger, of which 3,291 (22.4%) and 124 (7.5%), respectively, were flagged as EPs.

EP Characteristics

At VA, the rate of EP lung cancers was greater in Hispanic and Black patients than in White patients (P < .001 for both; Table 1). A trend to more EPs in Hispanic patients was observed in CRC, without statistical significance. VA patients in the middle tertile of age (68-73 years) had significantly fewer EP-flagged lung cancers compared with those in the other tertiles, whereas patients in the upper tertile of age (>73 years) had significantly more EPs for CRC (P < .001 for both). Other comparisons of EP rate among demographic factors showed no significant differences. EP rate increased (P < .001 for all) with increasing cancer stage for both lung cancer and CRC, except for colorectal stage II versus III (Table 1).

TABLE 1.

Rate of Cancer EP by Cancer Type, Demographic Characteristics, and Stage

Characteristic	Lung Cancer (VA)	CRC (VA)	Lung Cancer (Geisinger)	CRC (Geisinger)
Race
Non-Hispanic White	5,716/29,178 (19.6)	2,174/9,925 (21.9)	264/2,816 (9.4)	120/1,538 (7.8)
Non-Hispanic Black	1,616/6,217 (26)	669/2,941 (22.8)	5/40 (12.5)	2/36 (5.6)
Hispanic	273/999 (27.3)	228/871 (26.2)	2/22 (9.1)	0/45 (0)
Asian	28/113 (24.8)	15/104 (14.4)	0/10 (0)	1/11 (9.1)
Pacific	65/275 (23.6)	34/104 (32.7)	0/3 (0)	1/6 (17)
American Indian and Alaska Native	54/282 (19.2)	31/129 (24)	0/5 (0)	0/0
Other/unknown	311/1,501 (20.7)	140/600 (23.3)	0/5 (0)	0/8 (0)
Sex
Male	7,795/37,200 (21)	3,170/14,066 (22.5)	133/1,554 (8.6)	54/856 (6.3)
Female	268/1,365 (19.6)	121/608 (19.9)	142/1,360 (10.4)	70/791 (8.9)
Age
Tertile 1 (<68 years for lung cancer, <66 years for CRC)	3,100/14,135 (21.9)	1,056/5,312 (19.9)	106/1,313 (8.1)	29/712 (4.1)
Tertile 2 (68-73 years for lung cancer, 66-73 for CRC)	2,300/12,059 (19.1)	922/4,951 (18.6)	65/663 (9.8)	30/366 (8.2)
Tertile 3 (>73 years)	2,663/12,371 (21.5)	1,313/4,411 (29.8)	104/938 (11.1)	65/569 (11.4)
Stage group
I	1,348/10,874 (12.4)	440/2,887 (15.2)	51/695 (7.3)	10/243 (4.1)
II	509/3,065 (16.2)	626/2,347 (26.7)	17/239 (7.1)	33/341 (9.7)
III	1,276/6,353 (20.1)	595/2,559 (23.3)	45/562 (8)	33/389 (8.5)
IV	3,337/10,594 (31.5)	712/2,088 (34.1)	147/1,233 (12)	30/402 (7.5)
Unrecorded stage	1,593/7,679 (20.7)	918/4,793 (19.2)	15/185 (8.1)	18/274 (6.6)

NOTE. Values are presented as EP, No./total cancers, No. (%).

Abbreviations: CRC, colorectal cancer; EP, emergency presentation; VA, Veterans Affairs.

The validation review to determine true EP showed positive predictive values (PPVs) of 80% (VA) and 86% (Geisinger) for lung cancer and 72% (VA) and 90% (Geisinger) for CRC (Table 2). Among 400 patients reviewed, most had their diagnosis made in the ED or hospital admission directly linked to it (route A, Table 2). Less commonly, diagnosis was made during an outpatient encounter when cancer was suspected at EP (route B) or when cancer was not suspected at ED presentation, but concerning findings were secondarily discovered and evaluated later (route C). Finally, in 1%-6% of dQM-flagged EPs, a non–cancer-related emergency event coincidentally occurred within 30 days before routine cancer diagnosis (route D, a form of false positive). Notably, only 13.8% of patients with lung cancer and 23.3% of patients with CRCs who were eligible for cancer screening had up-to-date screening at the time of EP.

TABLE 2.

Digital Quality Measure Rates and Validation Results

Diagnosis	Lung Cancer (VA)	Colorectal Cancer (VA)	Lung Cancer (Geisinger)	Colorectal Cancer (Geisinger)
Measure (flagged EPs)	8,063/38,565 (20.9%)	3,291/14,674 (22.4%)	275/2,914 (9.4%)	124/1,649 (7.5%)
True positive EPs	80/100 (95% CI, 71 to 87)	72/100 (95% CI, 62 to 81)	86/100 (95% CI, 78 to 92)	90/100 (95% CI, 82 to 95)
Route A	46	52	55	53
Route B	21	12	25	30
Route C	13	8	6	7
False positive EPs	20/100	28/100	14/100	10/100
Route D (operational false positive)	6	8	2	1
Invalid diagnosis or emergency care (implementation false positive)	14	20	12	9
MODs	39/80 (48.8%; 95% CI, 37.4 to 60.2)	51/72 (70.8%; 95% CI, 58.9 to 81.0)	73/86 (84.9%; 95% CI, 75.5 to 91.7)	70/90 (77.8%; 95% CI, 67.8 to 85.9)
Type I MODs (failure to initiate investigation)	30 (including 19 screening)	38 (including 11 screening)	49 (including 11 screening)	6
Type II MODs (failure to complete investigation)	6	6	8	11
Combined type I + type II MODs	3	7	5	39 (including 14 screening)

NOTE. All values are patients, No. except where indicated.

Abbreviations: EP, emergency presentation; MOD, missed opportunity for diagnosis; VA, Veterans Affairs.

MOD Characteristics

For lung cancer, MODs were present in 48.8% of VA patients with true positive EPs and 84.9% of Geisinger true positives, as determined by lack of documentation of appropriate clinical actions. Most lung cancer MODs were solely type I (failure to initiate workup, including missed opportunities for screening) at both sites (Table 2). The most common missed cancer signal in lung cancer MODs was abnormal imaging (26.3%), followed by cough (worsening persistent cough/bronchitis or new chronic cough >8 weeks, present in 15.8%), and dyspnea (10.5%). MODs were most frequently (52.5%) attributed to lapses or delays in performing diagnostic tests (eg, chest imaging ordered but not performed for suspicious cough), followed by patient-related factors (30.2%, eg, delay in seeking care, or lack of adherence to appointments), and patient-provider encounter factors (30.2%, eg, problems with history and physical exam, problems ordering tests, or problems with data integration/interpretation). Further examples of contributing factors can be found in the Data Supplement.

For CRC, MODs were present in 70.8% of VA true positive EPs and 77.8% of Geisinger true positives. Most CRC MODs were type I alone, or combined type I and type II (Table 2). The most common missed cancer signal in CRC-related MODs was iron deficiency anemia (58.8%), followed by blood in stool (15.7%) and positive fecal occult blood test (5.9%). MODs were most frequently (57.1%) attributed to patient-related factors, followed by patient-provider encounter factors (39.3%), and follow-up and tracking factors (35.7%, eg, missed clinician response to diagnostic findings).

Mortality

In VA, EP cases were associated with a higher all-cause mortality than non-EP cases (P < .005; Table 3). Specifically, lung cancer EPs had 58% 1-year mortality, versus 33% in non-EP. Similarly, CRC EPs had 32% 1-year mortality, versus 13% in non-EP. Furthermore, EP was significantly associated (P < .001) with increased odds of mortality, adjusting for cancer stage and demographic factors, for lung cancer and CRC (odds ratio, 1.78 and 1.83, respectively [95% CIs, 1.63 to 1.86 and 1.61 to 2.07]; Table 4). Follow-up (known alive/dead status) was 96% at 1 year.

TABLE 3.

Observed Mortality and Follow-Up by Hospital System, Cancer Type, and EP Status

Hospital System	Cancer Type, EP Status	30-Day Mortality	90-Day Mortality	180-Day Mortality	365-Day Mortality
VA	Lung, EP	1,005/8,063 (12.5)	2,413/8,063 (29.9)	3,498/8,063 (43.4)	4,687/8,063 (58.1)
	Lung, non-EP	1,054/30,502 (3.5)	3,521/30,502 (11.5)	6,149/30,502 (20.2)	10,111/30,502 (33.1)
	Lung, complete follow-up	37,514/38,565 (97.3)	37,158/38,565 (96.4)	37,036/38,565 (96)	36,972/38,565 (95.9)
	Colorectal, EP	184/3,291 (5.6)	479/3,291 (14.6)	727/3,291 (22.1)	1,044/3,291 (31.7)
	Colorectal, non-EP	146/11,383 (1.3)	487/11,383 (4.3)	869/11,383 (7.6)	1,469/11,383 (12.9)
	Colorectal, complete follow-up	14,436/14,674 (98.4)	14,316/14,674 (97.6)	14,225/14,674 (96.9)	14,062/14,674 (95.8)
Geisinger	Lung, EP	45/275 (16.4)	88/275 (32)	125/275 (45.5)	157/275 (57.1)
	Lung, non-EP	181/2,639 (6.86)	460/2,639 (17.43)	708/2,639 (26.8)	1,049/2,639 (39.7)
	Lung, complete follow-up	—	—	—	2,830/2,914 (97.1)
	Colorectal, EP	5/124 (4)	11/124 (8.9)	25/124 (20.2)	40/124 (32.3)
	Colorectal, non-EP	50/1,523 (3.3)	124/1,523 (8.1)	206/1,523 (13.5)	275/1,523 (18.1)
	Colorectal, complete follow-up	—	—	—	1,563/1,647 (94.8)

NOTE. Values are presented as patients with mortality or follow-up, No./ total patients, No. (%).

Abbreviations: EP, emergency presentation; VA, Veterans Affairs.

TABLE 4.

Logistic Regression of 12-Month Mortality in Veterans Affairs Data

Predictor	Lung Cancer, Adjusted OR (95% CI)	Lung Cancer, P	Colorectal Cancer, Adjusted OR (95% CI)	Colorectal Cancer, P
EP status
Non-EP	Ref		Ref
EP	1.74 (1.63 to 1.86)	<.001	1.83 (1.61 to 2.07)	<.001
Sex
Female	Ref		Ref
Male	0.42 (0.36 to 0.48)	<.001	0.18 (0.14 to 0.22)	<.001
Age, years	0.98 (0.98 to 0.98)	<.001	0.98 (0.98 to 0.99)	<.001
Stage
I	Ref		Ref
II	1.95 (1.76 to 2.16)	<.001	0.92 (0.76 to 1.12)	.398
III	4.20 (3.89 to 4.54)	<.001	1.23 (1.03 to 1.47)	<.001
IV	19.75 (18.34 to 21.27)	<.001	10.34 (8.82 to 12.13)	<.001
Geography (RUCA)
Metropolitan	Ref		Ref
Micropolitan	1.06 (0.98 to 1.16)	.159	0.79 (0.65 to 0.96)	.017
Small town	0.98 (0.87 to 1.09)	.671	0.89 (0.69 to 1.15)	.376
Rural	0.95 (0.83 to 1.08)	.439	1.08 (0.82 to 1.42)	.577
Elixhauser index	1.03 (1.02 to 1.03)	<.001	1.05 (1.04 to 1.05)	<.001
Race/ethnicity
White	Ref		Ref
Non-Hispanic Black	0.72 (0.67 to 0.78)	<.001	0.62 (0.53 to 0.73)	<.001
Hispanic	0.78 (0.65 to 0.93)	.006	0.85 (0.66 to 1.09)	.203
Asian	0.33 (0.18 to 0.63)	.001	0.52 (0.21 to 1.30)	.162
Pacific Islander	0.86 (0.62 to 1.19)	.366	0.90 (0.48 to 1.69)	.736
American Indian, Alaska Native	0.76 (0.55 to 1.06)	.102	0.71 (0.38 to 1.31)	.270
Other/unknown	0.97 (0.84 to 1.13)	.710	0.82 (0.61 to 1.10)	.191

Abbreviations: EP, emergency presentation; OR, odds ratio; Ref, reference; RUCA, rural urban commuting area.

DISCUSSION

We developed, implemented, and validated a dQM to identify EPs of lung cancer and CRC in two large health systems. We found high PPV for identifying EPs for both cancers and for both health systems. Associations of EP with demographic factors and stage (Table 1) were largely consistent with the European literature,^8,34,35 showing a J-shaped association of age with colorectal EP (although none with lung EP). Consistent with these studies, we found EPs are common among new cancer diagnoses and are strongly and independently associated with higher stage and mortality. Furthermore, many patients with EPs experience MODs in the 2 years before diagnosis. This is consistent with longitudinal data from England showing a reduction in EPs with system-level interventions,^36-38 indicating that EPs may be largely driven by preventable MODs and underscoring the need for quality improvement interventions. The PPV, MOD rates, and mortality association all indicate good face validity of this measure and a high concentration of events to guide learning and improvement.

Between the two health systems, there was an apparent lower cancer incidence at Geisinger, presumably related to VA's older, higher-risk population. Geisinger also exhibited lower EP rates (Table 2), which could be attributable to several factors, including differences in service delivery modes or ER visit ascertainment in a less closed system. Among true-positive EPs, MOD rates were similar, except for a lower MOD rate in lung cancer in VA (Table 2). Some variability in rates due to different service delivery modes and practice patterns between systems would not be unexpected.

To our knowledge, this is the first dQM on the basis of EPs and implemented at scale, including in a national US data set. Furthermore, we linked EPs to MODs for the first time using structured record review. Rigorous validation through record review—to confirm true EPs and missed opportunities—adds strength to our work. This measure can be tested in other US health systems, followed by consideration for use by CMS. The dQM appears potentially useful for measuring diagnostic quality at the health system level, especially when ambulatory and/or primary care is better integrated with inpatient care, thus making cancer-related diagnostic quality measurement more feasible. Because many patients use multiple health systems, measure attribution warrants further discussion although it would likely be to the system where the patient receives most ambulatory and primary care. Because current diagnostic quality measures are underdeveloped, and diagnosis is a distributed phenomenon spanning different clinicians and settings of care, CMS's Center for Clinical Standards and Quality could consider creating a new program for enhancing diagnostic quality reporting and accountability and for exploring implementation. Nevertheless, the measure is still useful now for improvement purposes given its strong association with missed opportunities.

We found levels of CRC EP consistent with those reported by England's NCIN (26%), although lung cancer EP rates were lower in our populations than in England (39%).⁷ Our analysis found more EPs in racial/ethnic minorities, although the pattern differed from English Routes to Diagnosis studies.³⁹ Finally, EPs of both cancers were associated with significantly increased stage-adjusted 1-year mortality, also consistent with prior findings.³⁴

Diagnostic testing and patient-related factors were most commonly implicated in lung cancer and CRC MODs, respectively, and may be appealing targets for intervention. However, many patients had more than one contributing factor, suggesting the need for a multifactorial approach. Many were not up to date on cancer screening at the time of EP, sometimes with documentation of patient refusal. Novel strategies are thus needed to promote screening, including using EHR data proactively to identify overdue patients.⁴⁰ Improved screening is not the only viable intervention, however, as only some cancers are screen-detectable. Early detection and measurement of symptomatic cancer diagnosis adds substantial value in assessing cancer-related quality.⁴¹ Future work can investigate cancers where no screening exists or where MODs can occur after presentation. While the dQM partially reflected preventable EPs, future work should investigate whether preventable and nonpreventable EPs can be distinguished using automated means.

An important limitation of our dQM was the inability to query data outside of our health systems, leading to false positives and negatives when patients receive care elsewhere. To minimize this, we selected patients primarily receiving care in each system, limiting to those with in-system primary care encounters within 2 years before cancer diagnosis. Still, the dQM is most applicable to integrated care delivery systems that have longitudinal data on ambulatory visits, ED visits, hospitalizations, and cancer diagnoses. However, as the adoption of health information exchanges increases, more systems could implement this dQM and validate its PPV, making its applicability much broader. Our dQM was further limited in establishing causality between emergency event and cancer diagnosis due to lack of structured referral data. To mitigate this, we used a 30-day linking period between emergency visit and cancer diagnoses and manually reviewed records. A final limitation is that record reviews can carry some subjectivity and hindsight bias. To minimize this, we rigorously trained reviewers, conducted multiple meetings to ensure reliability, used a standardized review instrument, and ensured reviewers would only consider explicitly documented cancer signals to avoid overestimating EPs and MODs.

In conclusion, we developed and implemented a dQM to identify EP in patients with new lung cancer and CRC in two large health systems. We further identified a correlation between EP and missed opportunity for diagnosis, demonstrated a range of factors contributing to missed opportunities, and validated the independent association of EP with mortality. These findings suggest that EP is common, associated with increased mortality, and associated with preventable missed opportunities for earlier diagnosis. Our dQM has demonstrated portability and could detect emergency cancer presentation in other US health systems and investigate care disparities that underlie EPs and lead to diagnostic delays and poor clinical outcomes. Future national implementation of this measure can inform improvement strategies to reduce preventable delays in cancer diagnosis.

AUTHOR CONTRIBUTIONS

Conception and design: Paarth Kapadia, Andrew J. Zimolzak, Divvy K. Upadhyay, Daniel R. Murphy, Gary A. Abel, Georgios Lyratzopoulos, Hardeep Singh

Financial support: Georgios Lyratzopoulos, Hardeep Singh

Administrative support: Alexis Offner, Hardeep Singh

Provision of study materials or patients: Divvy K. Upadhyay, Hardeep Singh

Collection and assembly of data: Paarth Kapadia, Andrew J. Zimolzak, Divvy K. Upadhyay, Saritha Korukonda, Riyaa Murugaesh Rekha, Umair Mushtaq, Usman Mir, Alexis Offner, Hardeep Singh

Data analysis and interpretation: Paarth Kapadia, Andrew J. Zimolzak, Divvy K. Upadhyay, Daniel R. Murphy, Gary A. Abel, Georgios Lyratzopoulos, Luke T.A. Mounce, Hardeep Singh

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Development and Implementation of a Digital Quality Measure of Emergency Cancer Diagnosis

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jco/authors/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).