Predictors of Overtesting in Pulmonary Embolism Diagnosis

Abstract

Background:

The benefits of computed tomography pulmonary angiography (CTPA) for pulmonary embolism (PE) diagnosis must be weighed against its risks, radiation-induced malignancy, and contrast-induced nephropathy. Appropriate use of CTPA can be assessed by monitoring yield, the percentage of tests positive for PE. We identify factors that are associated low CTPA yield, which may predict overtesting.

Methods:

This was a retrospective cohort study of six emergency departments between June 2014 and February 2017. The electronic health record was queried for CTPAs ordered for adult patients in the emergency department. We assessed the following patient factors: age, gender, body mass index, number of comorbidities, race, and ethnicity, provider factors: type (resident, fellow, attending, physician assistant) and environment factors: test time of day, season of visit, and crowdedness of the department.

Results:

A total of 14,782 CTPAs were reviewed, of which 1366 were found to be positive for PE, resulting in an overall CTPA yield of 9.24%. Provider type was not associated with a difference in yield. Testing was less likely to be positive in younger patients, females, those with lower body mass indexes and those identifying as Asian or Hispanic. Testing was also less likely to be positive when ordered during the overnight shift and during the winter and spring seasons.

Conclusion:

Our study identified several patient and environmental factors associated with low CTPA yield suggesting potential targets for overtesting. Targeting education and clinical decision support to assist providers in these circumstances may meaningfully improve yields.

INTRODUCTION

Every year 360,000 Americans are diagnosed with pulmonary embolism (PE) (1,2). This common condition carries a mortality of up to 34% without timely diagnosis and treatment (3). However, diagnosis is challenging as the signs and symptoms of PE are nonspecific, often resulting in low thresholds for imaging (4–6). Computerized tomography pulmonary angiography (CTPA) is sensitive and specific for PE (7); however, it is associated with a median of 10 mSV of radiation, the equivalent of 137 chest x rays (8). The benefits of CTPA for PE diagnosis however must be weighed against its risks, radiation-induced malignancy, and contrast-induced nephropathy (9–13).

CTPA diagnostic yield, or CTPA yield, is the percentage of CTPA tests that are positive for PE. This has been suggested as a metric of appropriateness of use with the British College of Radiologists publishing a target yield of over 15.3% (14). US CTPA yields vary from 6% to 25% across different health care institutions (15,16). Few studies have investigated factors that are associated with differences in CTPA yield. Factors found to predict low CTPA yield include female sex of the patient, for-profit and urban health care settings, and fewer years of provider experience (15,17). Here, we evaluate the association between various patient, provider, and environmental factors and CTPA yield to discover potential predictors of inappropriate testing.

MATERIALS AND METHODS

We performed a multicenter retrospective cohort study by querying Electronic Health Record (EHR) data for CTPA testing performed on adults between June 2014 and January 2017. The study took place at six hospitals across a large health system. The hospitals are supported by the Sunrise Clinical Manager EHR, a subsidiary of Allscripts Healthcare Solutions (Chicago, IL). This study was approved by our health system’s Institutional Review Board.

Computerized CTPA Yield Calculation

As was previously validated, CTPA yield was calculated as the number of ED CTPA orders linked to an inpatient discharge diagnosis of PE divided by the total number CTPAs completed (18). Our validation study found 96.4% accuracy using this method which utilizes EHR reports and not billing data. CTPAs done on the same day as a CTA abdomen/pelvis were eliminated as these were found in our validation study to be done only to evaluate for aortic disease and not PE. PE diagnosis was in accordance with International Classification of Diseases, Clinical Modification codes, versions 9 and 10 (ICD-9-CM and ICD-10-CM). The full range of PE diagnosis codes was used and consisted of the following: 415.0, 415.11, 415.12, 415.13, and 415.19 for ICD-9-CM; and I26.0, I26.01, I26.02, I26.09, I26.9, I26.90, I26.92, and I26.99 for ICD-10-CM.

Patient, Provider, and Environment Factors

Data were collected from the EHR based on visit identification number. Binomial data collected included patient gender. Continuous data collected included patient age, body mass index (BMI), and number of documented comorbidities and were divided into distinct categories. Categorical data collected included race, ethnicity, time of day in 6 hour intervals, time of year in seasonal 3-month intervals, smoking status, ED crowding, and type of provider. The ED was defined as crowed if the number of patients on the day of CTPA order exceeded 257, which was the top quartile of ED daily patients for the study overall.

Data Analysis

Univariate data were assessed with chi-squared or Fisher’s exact test to explore the association between patient, provider, and environment factors and CTPA yield. Logistic regression was performed on the results found initially to be statistically significant in order to assess for the presence of confounding. Statistical analysis performed with SAS version 9.6.

RESULTS

A total of 14,782 CTPAs were reviewed, of which 1366 were found to be positive for PE, resulting in an overall CTPA yield of 9.24%. Distribution of categorical patient data is included in Table 1. Mean age of patients was 57.3 ± 18.8. Mean BMI for patients was 29.5 ± 7.9. Mean number of comorbidities was 1.46 ± 2.27. Missing data included 5 entries for patient gender, 4747 for patient BMI, 36 for provider type, and 4 for test time of day. As more than 10% of BMI data were missing, sensitivity analysis was done by imputing BMI using median (=28.18) to see whether the association between outcome and BMI would be changed, and no change was found.

TABLE 1.

Baseline Characteristics of Patient, Provider, and Environment

Factors	no. (%)	CTPA Yield
Patients
Age
18–30	1489 (10%)	4.9%
31–50	3836 (26%)	7.5%
51–70	5469 (37%)	9.8%
>70	3988 (27%)	11.8%
Sex
Female	9630 (65%)	8.4%
Male	5147 (35%)	10.8%
Body mass index
<18.5	278 (3%)	5%
18.5–24.9	2773 (28%)	10.1%
25–29.9	2996 (30%)	13.4%
≥30	3988 (40%)	13.3%
Number of comorbidities
<5	13,549 (92%)	9.2%
5–10	1128 (8%)	9.4%
>10	105 (1%)	14.3%
Race
Black	3366 (23%)	10.8%
Asian	1049 (7%)	4%
White	7811 (53%)	10%
Other	2556 (17%)	7.1%
Ethnicity
Hispanic	2008 (14%)	6.5%
Non-Hispanic	12,394 (84%)	9.7%
Declined	380 (3%)	9%
Providers
Provider type
Attending	9527 (65%)	9.2%
Resident	3959 (27%)	9.4%
Fellow	51 (0.3%)	9.8%
Nurse practitioner	116 (0.8%)	12%
Physician assistant	1093 (7%)	8.8%
Environment
ED crowding
Crowded	3296 (22%)	9.9%
Not crowded	1,1486 (78%)	9%
Test time of day
8 am to 1:59 pm (morning)	3619 (24%)	9.2%
2 pm to 7:59 pm (afternoon)	5548 (38%)	10.2%
8 pm to 1:59 am (evening)	3975 (27%)	8.5%
2 am to 8 am (overnight)	1636 (11%)	7.8%
Test time of year
January–March (Winter)	3748 (25%)	8.4%
April–June (Spring)	3138 (21%)	8.6%
July–September (Summer)	3824 (26%)	9.9%
October–December (Fall)	4072 (28%)	9.9%

CTPA, computerized tomography pulmonary angiography.

Univariate analysis found CTPA yield to vary by patient age, gender, BMI, race, and ethnicity and test time of day and time of year (Table 2). CTPA yield was lower in patients, who were younger, females, had lower BMIs, and who identified as Asian or Hispanic. We also found CTPA yields to vary based on test time of day and month of the year. We found no difference in CTPA yields based on the patient’s number of comorbidities, how crowded the ED was or the provider type.

TABLE 2.

Factors Associated With Positive CTPA

Factors	Adjusted OR (95% CI)a	p Valueb
Patients
Age
18–30	0.63 (0.48–0.84)	0.001
31–50	0.81 (0.68–0.96)	0.01
51–70	0.81 (0.75–0.97)	0.006
>70	1 [Reference]
Sex
Female	0.85 (0.75–0.97)	0.01
Body mass index
<18.5	0.35 (0.20–0.61)	0.002
18.5–24.9	0.74 (0.63–0.87)	0.002
25–29.9	1.01 (0.87–1.16)	0.93
≥30	1 [Reference]
Race
African American	1.27 (1.10–1.47)	0.002
Asian	0.45 (0.32–0.63)	<0.001
Other	0.85 (0.7–1.05)	0.13
White	1 [Reference]
Ethnicity
Non-Hispanic	1.35 (1.07–1.7)	0.01
Hispanic	1 [Reference]
Environment
Test time of day
8 am to 1:59 pm	0.9 (0.77–1.05)	0.2
2 pm to 7:59 pm	1 [Reference]
8 pm to 1:59 am	0.9 (0.77–1.05)	0.17
2 am to 8 am	0.76 (0.61 –0.94)	0.01
Test time of year
January–March (Winter)	1 [Reference]
April–June (Spring)	1.04 (0.86–1.25)	0.68
July–September (Summer)	1.23 (1.04–1.46)	0.02
October–December (Fall)	1.21 (1.02–1.43)	0.03

OR, odds ratio.

The bold values are those less than <0.05

^aAdjusted model included all characteristics reported in the table.

^bp < 0.05 was the threshold for statistical significance.

All of these factors remained significant in the logistical regression analysis. The odds of having a positive CTPA test were lower for patients age 18–30 (OR 0.63), 31–50 (OR 0.81), and 51–70 (OR 0.81) when these were compared with patients greater than 70 years old. The odds of having a positive test were lower in female patients (OR 0.85) and those with a BMI of <18.5 (OR 0.35) and 18.5–24.9 (OR 0.74) compared with patients having BMIs greater than or equal to 30. Asian patients had lower odds of having a positive test (OR 0.45) while African Americans had higher odds (OR 1.27) when compared with white patients. Non-Hispanics had higher odds of having a positive test (OR 1.35).

CTPA testing done during the overnight shift (2 am to 8 am) was less likely to be positive (OR = 0.76) compared with the afternoon shift (2 pm to 8 pm). When compared with testing done during the January to March months, higher CTPA yields were found for tests done during the summer (July–September; OR = 1.23) and the fall (October–December) seasons (OR = 1.21).

DISCUSSION

Every year about 2.4 million CTPA scans are performed to evaluate for PE in emergency departments (EDs) in the United States (19). Each CTPA carries a 14% risk of contrast-induced nephropathy (20) and a lifetime malignancy risk that can be as high as 2.8% (21). Incidental findings requiring diagnostic follow-up are found in 24% of tests, increasing both costs and harms from repeat imaging (22). A recent study including 2.5 million Medicare emergency visits found utilization to have increased fivefold between 2000 and 2009 with diagnostic yield declining (2). This decrease in diagnostic yield has raised concerns about CTPA over testing.

Our overall CTPA yield of 9.24% is comparable to those reported at other health systems in the United States (4,15,17,23). We identified several clinical and nonclinical predictors of low CTPA yield that have never been evaluated in a comprehensive multivariate analysis. This is the first study to show lower CTPA yields based on test time. These predictors represent targets for quality improvement.

Age, Sex, and Obesity

Increasing age, male sex, and obesity are all associated with an increased risk of venous thromboembolism (24–27). Previous studies have also shown increasing age and male sex to correlate with higher CTPA yields (28,29). To our knowledge, ours is the first study to show an association between lower BMI and yield. Notably, we did not include clinical predictors of PE from common clinical prediction rules as our goal was not to predict PE but physician ordering behavior.

Race and Ethnicity

We found lower CTPA yields in Asian and Hispanic patients. Previous direct comparison studies have found that Asians and Hispanics have lower rates of venous thromboembolism (30–33). This finding represents a health disparity where Asian and Hispanic patients are being significantly overtested for PE, with more unnecessary exposure to radiation. Notably, Asian and Hispanic patients might have been more likely to have limited English proficiency which is associated with lower CTPA yields when interpreters are not utilized (34). We did not have access to English language proficiency in our data set.

Test Time

Our results identified lower yields during the overnight shift, between 2 am and 8 am. This is the first study to show lower CTPA yields based on test time. These results fit into literature demonstrating that provider’s decisions are subject variability based on emotional and psychological factors. For example, providers are more likely to prescribe antibiotics during the later hours of their clinic sessions (35).

Season

With regard to time of year, our data also identified significant CTPA yield variation. Specifically, CTPA yields were higher in the months July through December when compared with January through March. To our knowledge, no significant data exist on the relationship between test time of year and CTPA yield. There is no significant seasonal variability in PE incidence (36). Our results can be explained by higher incidences of influenza and other respiratory infections during the winter and early spring that present in similar ways to PE (37).

Future Directions–Targeted Clinical Decision Support

Clinical prediction rules such as Well’s Score for Pulmonary Embolism and the Geneva Score assist providers in determining pretest probability of PE. Meta-analysis of the effect of Well’s Score for Pulmonary Embolism has shown average CTPA yield to increase from 9% to 12% (38). This corresponds to about a 25% decrease in unnecessary CTPA scans without missing PEs. However, studies suggest these tools are not commonly used with inappropriate CTPA utilization occurring in 45–71% of cases (4,6,39).

The use of clinical decision support to improve testing behaviors in the diagnosis of PE has shown similar improvements in CTPA yields; however, they also have low acceptability among providers (40). Meta-analyses of clinical decision support overall have found provider adoption of these tools to be low (41) with up to 96% being overridden (42). Alert fatigue is a major reason for low acceptability among providers (43). With targeted clinical decision support, we can address areas of low performance while reducing overall amounts of tool triggering. Our study results can be used to inform the targeting of quality improvement efforts, including education clinical decision support.

Limitations

Our method for calculating CTPA yield in the ED depends on an inpatient diagnosis of PE. In the rare, but potentially growing, number of cases in which a patient is diagnosed with PE, but discharged directly from the ED, our methodology would lead us to interpret this situation as a negative CTPA, falsely decreasing CTPA yield. This was a rare occurrence however in our validation study (44). Additionally, in the rare case that a provider orders a CTPA as well as a CTA abdomen/pelvis to evaluate for both PE and aortic disease the study would be eliminated. This was not found to occur in our validation study. With regard to Asian and Hispanic patients, we were not able to ascertain whether a language barrier played a role in lowering CTPA yield.

CONCLUSION

Our study investigated CTPA yield variation with respect to patient, provider, and environmental factors. Among the results, patient age, BMI, gender, race, and ethnicity were predictors of CTPA yield. Additionally, studies done during the overnight shift, and during the winter and spring seasons were less likely to be positive. Further research might assess the impact of using these predictors to target quality improvement efforts to increase overall CTPA yields.