Validation of Quality Indicators Targeting Low-Value Trauma Care

This cohort study investigates the validity of quality indicators on low-value trauma care using Canadian trauma registry data.

Key Points

Question

What is the validity of quality indicators (QIs) on low-value trauma care derived from trauma registry data?

Findings

In this multicenter cohort study including 136 783 admissions, 7 of 12 QIs had moderate to high validity.

Meaning

This study shows the feasibility of assessing low-value trauma care using routinely collected data and provides data on QI properties that can be used to decide which QIs are most appropriate in a given system.

Abstract

Importance

Reducing low-value care has the potential to improve patient experiences and outcomes and free up health care resources. Sixteen quality indicators were recently developed targeting reductions in low-value trauma care based on a synthesis of the best available evidence, expert consensus, and patient preferences.

Objective

To assess the validity of quality indicators on low-value trauma care using trauma registry data.

Design, Setting, and Participants

Data from an inclusive Canadian provincial trauma system were used in this analysis. Included were all admissions for injury to any of the 57 provincial adult trauma centers between April 1, 2013, and March 31, 2020. Metrics for quality indicators were developed iteratively with clinical experts.

Main Outcomes and Measures

Validity was assessed using a priori criteria based on 5 parameters: frequency (incidence and case volume), discrimination (interhospital variation), construct validity (correlation with quality indicators on high-value care), predictive validity (correlation with quality indicators on risk-adjusted outcomes), and forecasting (correlation over time).

Results

The study sample included 136 783 patient admissions (mean [SD] age, 63 [22] years; 68 428 men [50%]). Metrics were developed for 12 of the 16 quality indicators. Six quality indicators showed moderate or high validity on all measurable parameters: initial head, cervical spine, or whole-body computed tomography for low-risk patients; posttransfer repeated computed tomography; neurosurgical consultation for mild complicated traumatic brain injury; and spine service consultation for isolated thoracolumbar process fractures. Red blood cell transfusion in low-risk patients had low frequency but had moderate or high validity on all other parameters. Five quality indicators had low validity on at least 2 parameters: repeated head CT and intensive care unit admission for mild complicated traumatic brain injury, hospital admission for minor blunt abdominal trauma, orthosis for thoracolumbar burst fractures, and surgical exploration in penetrating neck injury without hard signs.

Conclusions and Relevance

This cohort study shows the feasibility of assessing low-value trauma care using routinely collected data. It provided data on quality indicators properties that can be used to decide which quality indicators are most appropriate in a given system. Results suggest that 6 quality indicators have moderate to high validity. Their implementation now needs to be tested.

Introduction

Low-value clinical practices are tests and treatments that are not supported by evidence and may expose patients to physical and psychological harm.^1,2 They are also a major barrier to timely access to appropriate care and threaten the sustainability of modern health care systems.^1,3 Injuries are the second most important contributor to health care costs in North America.^4,5 Given the multitude of possible diagnoses and treatment options and the need to make decisions quickly, the trauma setting is prone to low-value care. However, we currently lack measures to assess this problem.^6,7,8,9,10

Many health care jurisdictions in high-income countries have trauma systems, which include routine evaluation of the quality of care, commonly based on quality indicators (QIs). However, QIs targeting low-value care have not been validated for trauma care. More broadly, according to a systematic review, less than 2% of QIs used to evaluate health care target low-value care.¹¹

We recently developed a trauma-specific list of QIs targeting low-value clinical practices for trauma care using a scoping review of the literature,¹² syntheses of evidence on benefits and harms,^13,14 an evaluation of cost-effectiveness,¹⁵ and an expert consensus study.¹⁶ Two panels of international (76% participation [25 of 33]) and local stakeholders (94% participation [15 of 16]), and 3 patient partners identified 16 clinical practices whose use should be questioned and assessed with QIs. We aimed to evaluate the validity of these QIs using trauma registry data.

Methods

The study protocol, developed by all coauthors, underwent scientific evaluation by a national granting agency peer-review committee, the Canadian Institutes of Health Research, and received approval from the CHU de Québec–Université Laval research ethics committee. This committee granted a waiver for individual patient consent, owing to the use of deidentified patient data. We report results according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines.¹⁷

Study Design and Setting

We conducted a multicenter, retrospective cohort study using data from an integrated Canadian provincial trauma system comprising 57 adult trauma centers (3 level I, 5 level II, 21 level III, and 28 level IV). All centers undergo mandatory verification, which includes evaluation on QIs. Currently, there are 15 QIs measuring compliance to high-value care^18,19 and 4 QIs on risk-adjusted outcomes.^20,21,22,23

Study Population

We included data from all adult (aged ≥16) hospital patients with a primary diagnosis of injury between April 1, 2013, and March 31, 2020, to any of the adult trauma centers in the system. Patients admitted for thermal injuries, foreign bodies, drowning, or late effects of injuries were excluded from the analysis. No data on race or ethnicity were available in the trauma registry.

Data

We obtained data from the provincial trauma registry that trauma centers are required to complete for all trauma admissions. Data are extracted from patient medical records by trained medical coders at each site using a standard data dictionary and are centralized at the Ministry of Health where data validity checks are conducted. Data quality is also ensured by coding forums, regular meetings with coordinators and clinicians, annual training updates, and data reabstraction. The last reabstraction suggested 93% accuracy on variables used for quality evaluation including diagnostic and intervention codes (Amina Belcaid, MSc, Institut National d’Excellence en Santé et Services Sociaux, June 15, 2021).

Development of Metrics for QIs

To develop QI metrics, we used an iterative method. The research team developed initial coding algorithms based on definitions elaborated in the expert consensus study. Advisory committee members then revised patient medical records flagged by the QIs, and the coding algorithms were refined. This process was repeated until no changes were made between steps.

Statistical Analysis

Assessing the Validity of QIs

In line with US Agency for Healthcare Quality recommendations,²⁴ we evaluated the validity of QIs using 5 parameters: (1) frequency, (2) discrimination, (3) construct validity, (4) predictive validity, and (5) forecasting. Frequency was evaluated by calculating incidence proportions and the number of cases per 1000 admissions. Discrimination was defined as the ability of the QIs to discriminate between hospitals and was evaluated by generating incidence estimates for each trauma center using multilevel logistic regression. Intercenter variations were then assessed by the presence of hospital outliers and intraclass correlation coefficients (ICCs). Outliers were hospitals with 95% CIs that did not include the global (provincial) mean. ICCs represent the proportion of total variation in the QI that is explained by variation between centers.²⁵ Construct validity was defined as the correlation of QIs with other measures of quality and measured at the trauma center level. We thus assessed correlation with (1) other QIs targeting low-value care (positive correlations expected) and (2) a composite QI measuring compliance with high-value care, described in detail elsewhere^18,19 (negative correlations expected). Predictive validity was defined as the correlation with QIs on risk-adjusted outcomes measured at the trauma center level: mortality, complications, length of hospital stay, unplanned 30-day readmission, and costs, described in detail elsewhere^{20,21,22,23,26} (positive correlations expected). Forecasting was defined as the association of performance over 2 consecutive periods of time and was assessed by evaluating the correlation of center-level incidence for each QI for 2 periods of time: 2013 to 2016 and 2017 to 2020 (positive correlations expected). To assess correlations, we calculated Pearson correlation coefficients on arcsine-transformed proportions weighted by mean annual patient volume.²⁷

The advisory committee established a priori criteria for validity based on the following considerations. For frequency, we used data on median incidence and case volumes of low-value care from US and Australian hospital discharge data, which were 10% and 10 of 1000 admissions, respectively (Table 1).^{28,29,30,31,32} For discrimination, ICCs were interpreted as conveying low, moderate, and high interhospital variation when they were respectively less than 5%, 5% to 19%, and 20% or more.³¹ For construct and predictive validity and forecasting, we used The BMJ guidelines, whereby correlation coefficients less than 0.2, 0.2 to 0.39, 0.4 to 0.59, 0.6 to 0.79, and 0.8 to 1 represent, respectively, very weak, weak, moderate, strong, and very strong associations.³² We also applied a threshold of significance P < .1 (2-sided) as some QIs only applied to level I and II centers and therefore had a sample size of only 8. We were careful to limit our interpretation of correlations with risk-adjusted outcomes as evidence of predictive validity and not as causal associations. All analyses were conducted using SAS software, version 9.4 (SAS Analytics).

Table 1. Criteria for Interpreting Data on Validity Developed A Priori by the Project Advisory Committee.

Criteria	Validity
	High	Moderate	Low
Frequency, incidence proportion cases/1000 admissionsa	≥10% and ≥10 Cases/1000 admissions	≥10% or ≥10 Cases/1000 admissions	<10% and <10 Cases/1000 admissions
Discrimination, interhospital variationb	ICC ≥20% and hospital outliers	ICC >5% or hospital outliers	All other scenarios
Construct validity, correlation with QI on high-value care correlation with other QI on low-value carec	Negative correlation with high-value QI and positive correlation with at least 2 other low-value QI	Negative correlation with high-value QI or positive correlation with at least 2 other low-value QI	All other scenarios
Predictive validity, correlation with QI on risk-adjusted outcomesc	Positive correlation with at least 2 outcomes and no negative correlations	More positive correlations than negative correlations	All other scenarios
Forecasting, correlation between QI in 2013-2016 and 2017-2020c	r ≥0.8	0.6≤ r <0.8	r <0.6

Abbreviations: ICC, intraclass correlation coefficient; QI, quality indicator.

^a Based on median incidence of low-value practices in US Medicare data²⁸ and median cases of low-value practices / 1000 admissions from US Medicare data²⁹ and Australian hospital discharge data.³⁰

^b ICC less than 5%, 5% to 19.9%, and 20% or greater are considered to represent low, moderate, and high interhospital variation, respectively.³¹

^c Pearson correlation coefficients (r) of 0.2 to 0.39, 0.4 to 0.59, 0.6 to 0.79, and 0.8 or greater are considered to represent weak, moderate, strong, and very strong associations, respectively.³²

Results

Study Population

The trauma registry included 136 783 admissions (mean [SD] age, 63 [22] years) for 2013 to 2020, of which 71 493 patients (52%) were 65 years or older, 68 428 (50%) were men, 68 355 (50%) were women, and 24 782 (18%) had an injury severity score of 12 or greater (eTable 1 in the Supplement). A total of 60 636 patients (44%) were admitted during the 2018 to 2020 period.

QI Metrics

We developed metrics for 12 of the 16 QIs (75%) (eTable 2 in the Supplement) using between 5 and 10 iterations for each metric. Four QIs could not be developed owing to lack of required data in the trauma registry. Examples of changes made over iterations include refining comorbidity codes for preexisting anemia and coronary disease for red blood cell (RBC) transfusion and refining Abbreviated Injury Scale codes for Brain Injury Guidelines criteria for mild complicated traumatic brain injury (TBI).³³

Frequency

Posttransfer repeated computed tomography (CT) and neurosurgical consultation for mild complicated TBI had incidences greater than 10% and case volumes greater than 10 of 1000 admissions (Table 2, Table 3). Spine service consultation for thoracolumbar transverse process fractures and hospital admission in minor abdominal trauma had incidences greater than 10% but low case volumes, whereas head CT, cervical spine CT, and whole-body CT in low-risk patients had low incidences but case volumes greater than 10 of 1000 admissions (all trauma centers: head CT incidence, 8.3%; admissions, 23.5 per 1000; cervical spine CT incidence, 4.2%; admissions, 11.0 per 1000; whole-body CT incidence, 2.2%; admissions 15.1 per 1000). RBC transfusion, repeated head CT, intensive care unit (ICU) admission for mild complicated TBI, and orthosis for thoracolumbar fractures all had low incidences and low case volumes. Surgical exploration for penetrating neck injury had such low case volumes (0.1 of 1000 admissions in all trauma centers) that we could not evaluate its validity any further. When we restricted analyses to level I centers, head CT in low-risk patients and orthosis for A0 to A3 thoracolumbar burst fractures both had incidences greater than 10% and case volumes greater than 10 of 1000 admissions, but no other major changes in frequency were observed.

Table 2. Evaluation of Frequency, Incidence Proportions, and Case Volume.

Quality indicator	Validity: high, moderate, or low	All trauma centers			Level I trauma centers
		No./total No.	Incidence, %	No./1000 admissions	Incidence, %	No./1000 admissions
Head CT in low-risk patients	Moderate	3211/38 528	8.3	23.5	High, 10.7	High, 24.0
Cervical spine CT in low-risk patients	Moderate	1507/36 008	4.2	11.0	7.1	14.8
Whole-body CT in minor or single-system injury	Moderate	2067/92 578	2.2	15.1	6.5	34.8
Posttransfer repeated CT	High	1652/6069	27.2	29.0	31.0	47.2
Red blood cell transfusion in low-risk patients	Low	1000/43 603	2.3	7.3	3.4	9.0
Surgical exploration of penetrating neck injury without hard signsa	Low	19/123	15.5	0.1	14.0	0.4
Neurosurgical consultation for mild complicated TBI	High	2002/8661	23.1	35.2	23.0	37.8
Spine service consultation for isolated thoracolumbar transverse process fractures	Moderate	114/953	12.0	2.0	10.3	2.2
Repeated head CT for mild complicated TBI	Low	732/16 384	4.5	5.4	4.6	8.1
ICU admission for isolated mild complicated TBI	Low	623/11 271	5.5	4.6	4.4	4.3
Hospital admission in isolated blunt abdominal trauma with negative CT	Moderate	754/3322	22.7	5.5	11.7	3.9
Orthosis for A0-A3 thoracolumbar burst fracture	Low	938/9734	9.6	6.9	High, 15.8	High, 10.5

Abbreviations: CT, computed tomography; ICU, intensive care unit; TBI, traumatic brain injury.

^a Case volume was insufficient to evaluate any other aspects of validity.

Table 3. Summary of Results for Each Concept of Validity^a.

Quality indicator	Frequency	Discrimination	Construct validity	Predictive validity	Forecasting
Head CT in low-risk patients	Moderate	Moderate	Moderate	Moderate	Moderate
Cervical spine CT in low-risk patients	Moderate	Moderate	High	Moderate	Moderate
Whole-body CT in minor or single-system injury	Moderate	High	High	Moderate	High
Posttransfer repeated CT	High	Moderate	NAb	NAb	Highb
Red blood cell transfusion in low-risk patients	Low	Moderate	High	Moderate	Moderate
Surgical exploration in penetrating neck injury without hard signs	Low	Lowc	Lowc	Lowc	Lowc
Neurosurgical consultation for mild complicated TBI	High	Moderate	NAb	NAb	NAb
Spine service consultation for isolated thoracolumbar transverse process fractures	Moderate	High	NAb	NAb	Highb
Repeated head CT for mild complicated TBI	Low	Moderate	Low	Moderate	Moderate
ICU admission for isolated mild complicated TBI	Low	High	Low	Low	Moderate
Hospital admission in isolated blunt abdominal trauma with negative CT	Moderate	Moderate	Low	Low	Moderate
Orthosis for A0-A3 thoracolumbar burst fracture	Low	High	Low	Moderate	Moderate

Abbreviations: CT, computed tomography; ICU, intensive care unit; NA, not available; TBI, traumatic brain injury.

^a Table 1 lists the criteria definition.

^b Not informative owing to low sample sizes (n = 8).

^c Could not be calculated owing to low sample sizes.

Discrimination

Hospital outliers were identified for all QIs (eFigures 1-11 in the Supplement). ICCs indicated high interhospital variation for whole-body CT (ICC, 27.46; 95% CI, 23.61-30.94), spine service consultation for thoracolumbar transverse process fractures (ICC, 29.96; 95% CI, 14.03-40.91), ICU admission for mild complicated TBI (ICC, 22.46; 95% CI, 16.71-27.47), and orthosis for thoracolumbar burst fractures (ICC, 47.98; 95% CI, 44.44-51.09) (Table 3, Table 4). Interhospital variation was moderate for initial head CT (ICC, 7.37; 95% CI, 4.49-10.09), cervical spine CT (ICC, 10.48; 95% CI, 6.45-14.17), posttransfer repeated CT (ICC, 11.52; 95% CI, 7.82-14.93), RBC transfusion (ICC, 7.64; 95% CI, 4.54-10.55), repeated head CT for mild complicated TBI (ICC, 19.00; 95% CI, 12.93-24.28), and hospital admission for minor abdominal trauma (ICC, 8.98; 95% CI, 4.76-12.85). Low interhospital variation was observed for neurosurgical consultation (ICC, 3.82; 95% CI, 2.56-5.04), but we did identify 3 hospital outliers among the 5 level I and II centers for this QI (eFigure 6 in the Supplement).

Table 4. Evaluation of Discrimination; Interhospital Variation in Incidence Proportions.

Quality indicator	Validity: high, moderate, or low	Mean (range), %	ICC (95% CI)
Head CT in low-risk patients	Moderate	8.3 (3.0-23.2)	7.37 (4.49-10.09)
Cervical spine CT in low-risk patients	Moderate	4.2 (1.0-14.4)	10.48 (6.45-14.17)
Whole-body CT in minor or single-system injury	High	2.2 (0.2-18.3)	27.46 (23.61-30.94)
Posttransfer repeated CT	Moderate	27.2 (8.4-34.6)	11.52 (7.82-14.93)a
Red blood cell transfusion in low-risk patients	Moderate	2.3 (0.7-5.5)	7.64 (4.54-10.55)
Surgical exploration for penetrating neck injury without hard signs	Low	15.5b	NAb
Neurosurgical consultation for mild complicated TBI	Low	23.1 (13.9-32.7)	3.82 (2.56-5.04)
Spine service consultation for isolated thoracolumbar transverse process fractures	High	12.0 (1.9-36.9)	29.96 (14.03-40.91)
Repeated head CT for mild complicated TBI	Moderate	4.5 (0.6-17.9)	19.00 (12.93-24.28)
ICU admission for isolated mild complicated TBI	High	5.5 (1.3-37.8)	22.46 (16.71-27.47)
Hospital admission in isolated blunt abdominal trauma with negative CT	Moderate	22.7 (8.9-44.7)	8.98 (4.76-12.85)
Orthosis for A0-A3 thoracolumbar burst fracture	High	9.6 (0.2-42.6)	47.98 (44.44-51.09)

Abbreviations: CT, computed tomography; ICC, intraclass correlation coefficient; ICU, intensive care unit; NA, not available; TBI, traumatic brain injury.

^a May not be informative due to low sample sizes (n = 8).

^b Could not be calculated due to insufficient sample size.

Construct Validity

Cervical spine CT, whole-body CT, and RBC transfusion in low-risk patients were positively correlated with each other (correlation coefficients of 0.48 for cervical spine CT and whole-body CT, 0.42 for cervical spine CT and RBC transfusion, and 0.50 for whole-body CT and RBC transfusion) and negatively correlated with the QI on high-value care (correlation coefficients of −0.33, −0.21, and −0.23, respectively) showing evidence of high construct validity, whereas head CT in low-risk patients showed moderate construct validity (correlation coefficient of −0.09 with the QI on high-value care) (Table 1; eTables 3 and 4 in the Supplement; Table 3). Repeated head CT and ICU admission for mild complicated TBI as well as hospital admission in minor abdominal trauma and orthosis for thoracolumbar burst fracture showed low construct validity (correlation coefficients <0.2 with the QI on high-value care and not more than 2 correlation coefficients >0.2 with other QI on low-value care). Correlations with posttransfer repeated CT, as well as neurological and spine consultations, were difficult to interpret owing to low sample sizes.

Predictive Validity

Repeated head CT had positive correlations with readmissions (correlation coefficient, 0.21; 95% CI, −0.05 to 0.45; P = .09) and costs (correlation coefficient, 0.24; 95% CI, −0.03 to 0.47; P = .08), showing evidence of high predictive validity according to our criteria (eTable 4 in the Supplement; Table 3). However, the correlations were weak and not statistically significant. Six QIs showed evidence of moderate predictive validity; RBC transfusion had a positive association with all outcomes except mortality (correlation coefficients 0.25, 0.22, 0.26, and 0.57 with complications, readmission, length of stay, and costs, respectively; P = .001), and QIs on low-value initial imaging (head, cervical spine and whole-body) had negative correlations with mortality but positive correlations with 2 other outcomes. The QI on neurosurgical consultation had a strong positive correlation with mortality (correlation coefficient, 0.70; 95% CI, −0.01 to 0.94; P = .04), and orthosis had a weak positive correlation with complications (correlation coefficient, 0.29; 95% CI, 0.03-0.51; P = .03). Spine service consultation (1 or more correlation coefficients were <−0.2; P < .001), ICU admission for mild complicated TBI (complications, correlation coefficient, −0.27; 95% CI, −0.49 to −0.01; P = .04), and hospital admission in minor abdominal trauma (complications, −0.34; 95% CI, −0.55 to −0.08; P = .01) showed evidence of low predictive validity. Posttransfer repeated CT had no significant correlations with any outcome but this was likely attributable to low sample size.

Forecasting

Three QIs showed good forecasting properties with correlation coefficients between the 2 time periods of 0.8 or more, whereas all other QIs showed moderate forecasting properties (eTable 5 in the Supplement; Table 3). Whole-body CT in minor or single-system injury had a correlation coefficient of 0.99 (95% CI, 0.98-0.99), posttransfer repeated had a correlation coefficient of 0.86 (95% CI, 0.40-0.97), and spine service consultation for isolated thoracolumbar transverse process fractures had a correlation coefficient of 0.93 (95% CI, 0.67-0.99) (eTable 5 in the Supplement).

Missing Data

The Glasgow Coma Scale (GCS) score was missing for 48% of patients (65 656 of 136 783) overall, 16% of patients (4624 of 28 900) with a head injury, and 8% of patients (753 of 9412) with a severe head injury (Abbreviated Injury Scale score ≥4). For QI metrics based on the GCS, we used a registry variable that codes mild, moderate, and severe TBI according to discharge physician notes. We thus assumed a GCS of 13 or more for patients in whom either mild or no TBI was coded.

Discussion

This cohort study provides data on the frequency, discrimination, construct validity, predictive validity, and forecasting properties of QIs targeting low-value trauma care. Initial head, cervical spine, and whole-body CT for low-risk patients, posttransfer repeated CT, neurosurgical consultation for mild complicated TBI, and spine service consultation for thoracolumbar process fractures showed moderate or high validity on all measurable parameters. RBC transfusion in low-risk patients had low frequency but had moderate or high validity on all other parameters. Meanwhile, repeated head CT and ICU admission for mild complicated TBI, hospital admission for minor blunt abdominal trauma, and orthosis for thoracolumbar burst fractures had low validity on at least 2 parameters.

We are unaware of any studies on QIs targeting low-value care for trauma populations but the validity of measures of low-value care has been evaluated in other populations. The frequencies observed in our study were within the range of those observed for measures of low-value care for general hospital admissions calculated using hospital discharge data from the US and Australia.^28,29,34,35 One of these studies, based on US Medicare data, evaluated construct validity²⁹; two-thirds of correlations (10 of 15) among low-value care measures were greater than 0.2 compared with only one-third in our study. However, it is unclear whether their proportions were transformed to respect linearity assumptions. Finally, another study assessed the predictive validity of a composite index of low-value care and observed similar correlations; a weak association with mortality (r = 0.28) and a very weak association with costs (r = 0.18).³⁶

Our results suggest that QIs on low-value initial head, cervical spine CT, whole-body CT, and posttransfer repeated CT have evidence of moderate to high validity on all parameters. Low-value imaging is a well-documented problem and targeted by multiple clinical practice guideline recommendations.^{6,7,8,9,10,37} Consequences include inefficient use of scarce resources, unnecessary irradiation, and incidental findings, which may lead to a cascade of low-value treatment.^2,34,35,38 Although case volumes on these QIs were high and they showed evidence of construct validity and predictive validity, they all had incidences less than 10% and negative associations with trauma center mortality. However, incidence depends on the chosen denominator, which is large because all minor trauma admissions were included. Furthermore, inverse associations with mortality could be explained by information bias whereby trauma centers who perform more imaging have higher injury severity owing to higher injury detection rates, which would then lead to lower risk-adjusted mortality.

Low-value neurosurgical or spine consultation has been reported to be a source of health care overuse elsewhere³⁹ and is targeted in clinical practice guidelines.⁴⁰ These 2 QIs only applied to level I and II centers, therefore, construct and predictive validity were difficult to assess. Nevertheless, the major effect of low-value consultations is the inefficient use of scarce resources, which was not captured in our outcome QI as our cost estimates do not include physician billing. These low-value consultations also often entail low-value interhospital transfers. Of note, only 2 cases per 1000 admissions were flagged for low-value spine service consultation, indicating that it is likely to be a low-yield QI in our system.

RBC transfusion in low-risk patients had low incidence and case volume in our system. This may be attributable to the use of proxies for hemoglobin levels, which are not documented in our trauma registry. However, this QI showed significant interhospital variation even among level I and II centers and was positively correlated with complications, unplanned readmission, length of hospital stay, and costs. We did observe a negative correlation between this QI and trauma center mortality, even though the QI only applies to patients younger than 65 years with no coronary disease or anemia. However, as mentioned earlier, correlations should be interpreted as an indication of predictive validity rather than causal associations. Low-value RBC transfusion is targeted by Choosing Wisely recommendations.⁶

Although it is important to balance frequency with the risks of low-value practices, this study suggests that surgical exploration of penetrating neck injury, repeated head CT for mild complicated TBI, and ICU admission for mild complicated TBI may be too low-yield in our trauma system to be useful QIs, even in level I trauma centers. Penetrating injury is uncommon in Canada but is likely to be more frequent in other settings, such as large urban US centers.⁴¹ Mild complicated TBI is a common diagnosis but repeated CT and ICU admission in this population appear to be rare in our system. This may be attributable to high compliance with Brain Injury Guidelines for these 2 practices.⁴² In addition to low frequency, repeated head CT had low construct validity and ICU admission for mild complicated TBI had low construct and predictive validity. Of note, repeated CT for mild complicated TBI was selected by the panel of international experts in our consensus study but not by local stakeholders, whereas ICU admission for the same diagnosis was selected by local stakeholders but not by international experts.¹⁶

Hospital admission in blunt abdominal trauma with a negative CT had an incidence above 10% and high interhospital variation but low case volume and low construct and predictive validity. This QI was selected by international experts but not by local stakeholders in our consensus study¹⁶ and is one of the few that is not targeted in any clinical practice guidelines we identified. Finally, the QI on orthosis in thoracolumbar burst fractures had low frequency and construct validity. However, this QI showed the highest interhospital variation, was positively correlated with risk-adjusted complications and had high frequency in level I trauma centers. This QI was selected by spine surgeons in our consensus study because of lack of evidence on effectiveness and consequences for patients in terms of discomfort and sleep disturbance, but it was not retained by global panels.

Limitations

The most important limitations of our study were related to the limits of available data. First, as our QIs were based on hospitalized patients, we underestimated frequency for initial diagnostic imaging, particularly for head CT. However, routinely collected emergency department (ED) data lack granularity to accurately identify low-value imaging. Given the strong correlation between QIs, we anticipate that hospitals with high rates of imaging in the ED will be identified using data on hospitalized patients. Second, despite improved granularity over ED or hospital discharge data, lack of clinical information in the trauma registry meant that we could not develop measures for 4 of 16 QIs. However, research suggests that tracking a small number of indicators is likely to help identify systematic waste.⁴³ Furthermore, owing to the lack of data on hemoglobin levels, we used proxies based on anatomic injuries and physiological parameters for the QI on low-value RBC transfusions, which probably led to an underestimation of frequency. As most trauma registries lack serial hemoglobin measures, in future work, we should evaluate whether this QI correlates well with a criterion standard QI, which incorporates hemoglobin data. Similarly, lack of serial physiological data may have led us to inappropriately flag patients with neurological deterioration or patients who became hemodynamically unstable as cases of low-value care. The rapid integration of electronic medical data is likely to address many of these limitations. Third, the validity of QIs relies on accurate injury and intervention coding. Although data coders were consulted throughout the development process, heterogeneous data capture may have affected interhospital comparisons. The implementation of new QIs in quality improvement programs often highlights opportunities to improve data coding, which is under the responsibility of each trauma center. Nevertheless, given the limits of available data and the importance of clinical judgement in patient management, cases flagged by these QIs can only be interpreted as potential cases of low-value care. QIs are intended to identify high-level practice variation between hospitals, and they should be presented as a tool to help identify opportunities for quality improvement rather than as practices to be avoided in all circumstances. This understanding is critical to the fundamental aim of QIs, ie, to inform improvements in patient care rather than to encourage the systematic avoidance of practices which may in some cases be beneficial. Finally, the data generated in our study may not generalize well to trauma systems with different patient populations, trauma system configurations, or health care structures.

Conclusions

This cohort study demonstrated the feasibility of integrating QIs on low-value trauma care into quality assurance programs using trauma registry data. The data we have generated on QI validity can be used to inform the choice of which QIs should be implemented in a given system considering specificities related to patient volumes, case mix, and systems of care. In future steps, we need to evaluate the external validity of these QIs and assess their effectiveness and cost-effectiveness in an implementation trial. The integration of these QIs into trauma quality improvement programs may reduce exposure to unnecessary radiation and adverse events, improve the care experience for patients and their families, reduce the workload of health care professionals, and improve the allocation of health care resources.

Supplement.

eTable 1. Description of the Study Sample

eTable 2. Definitions of Quality Indicators

eFigure 1. Head CT in Low-Risk Patients

eFigure 2. Cervical Spine CT in Low-Risk Patients

eFigure 3. Whole-Body CT in Minor or Single-System Injury

eFigure 4. Posttransfer Repeat CT

eFigure 5. Red Blood Cell Transfusion in Low-Risk Patients

eFigure 6. Neurosurgical Consultation for Mild, Complicated Traumatic Brain Injury

eFigure 7. Spine Service Consultation for Isolated Transverse Process Fractures

eFigure 8. Repeat Head CT for Mild Complicated Traumatic Brain Injury

eFigure 9. Intensive Care Unit Admission for Mild Complicated Traumatic Brain Injury

eFigure 10. Hospital Admission in Isolated Abdominal Trauma With Negative CT

eFigure 11. Orthosis for A0-A3 Thoracolumbar Burst Fractures

eTable 3. Evaluation of Construct Validity; Correlations With QIs on High-Value Care and Other QIs on Low-Value Care; Positive Correlations (green) Are in Line With Our Hypothesis; Negative Correlations (red) Are Contrary to Our Hypothesis

eTable 4. Evaluation of Predictive Validity; Correlations¹ with QIs on Risk-Adjusted Outcomes; Positive Correlations (green) Are in Line With Our Hypothesis; Negative Correlations (red) Are Contrary to Our Hypothesis

eTable 5. Evaluation of Forecasting; Correlation of Quality Indicator Calculated Over 2 Periods of Time (2013-2016 and 2017-2020)