Quality Indicators Targeting Low-Value Clinical Practices in Trauma Care

Key Points

Question

Which clinical practices in acute trauma care should be targeted by quality indicators to reduce low-value care?

Findings

In this consensus study including 46 eligible clinical experts, a set of quality indicators was developed that covered low-value initial and repeated diagnostic imaging, routine blood work, trauma service and intensive care unit admission, neurosurgical and spine consultation, trauma surgery, blood product administration, and medication.

Meaning

The selected indicators represent a trauma-specific list of practices, the use of which should be questioned; trauma quality programs in high-income countries may use these consensus results as a basis to select context-specific quality indicators to measure and reduce low-value clinical practices.

This consensus study investigates which clinical practices may be targeted in acute trauma care to reduce low-value care.

Abstract

Importance

The use of quality indicators has been shown to improve injury care processes and outcomes. However, trauma quality indicators proposed to date exclusively target the underuse of recommended practices. Initiatives such as Choosing Wisely publish lists of practices to be questioned, but few apply to trauma care, and most have not successfully been translated to quality indicators.

Objective

To develop a set of evidence and patient-informed, consensus-based quality indicators targeting reductions in low-value clinical practices in acute, in-hospital trauma care.

Design, Setting, and Participants

This 2-round Research and Development/University of California at Los Angeles (RAND/UCLA) consensus study, conducted from April 20 to June 9, 2021, comprised an online questionnaire and a virtual workshop led by 2 independent moderators. Two panels of international experts from Canada, Australia, the US, and the UK, and local stakeholders from Québec, Canada, represented key clinical expertise involved in trauma care and included 3 patient partners.

Main Outcomes and Measures

Panelists were asked to rate 50 practices on a 7-point Likert scale according to 4 quality indicator criteria: importance, supporting evidence, actionability, and measurability.

Results

Of 49 eligible experts approached, 46 (94%; 18 experts [39%] aged ≥50 years; 37 men [80%]) completed at least 1 round and 36 (73%) completed both rounds. Eleven quality indicators were selected overall, 2 more were selected by the international panel and a further 3 by the local stakeholder panel. Selected indicators targeted low-value clinical practices in the following aspects of trauma care: (1) initial diagnostic imaging (head, cervical spine, ankle, and pelvis), (2) repeated diagnostic imaging (posttransfer computed tomography [CT] and repeated head CT), (3) consultation (neurosurgical and spine), (4) surgery (penetrating neck injury), (5) blood product administration, (6) medication (antibiotic prophylaxis and late seizure prophylaxis), (7) trauma service admission (blunt abdominal trauma), (8) intensive care unit admission (mild complicated traumatic brain injury), and (9) routine blood work (minor orthopedic surgery).

Conclusions and Relevance

In this consensus study, a set of consensus-based quality indicators were developed that were informed by the best available evidence and patient priorities, targeting low-value trauma care. Selected indicators represented a trauma-specific list of practices, the use of which should be questioned. Trauma quality programs in high-income countries may use these study results as a basis to select context-specific quality indicators to measure and reduce low-value care.

Introduction

Low-value clinical practices are defined as a test or treatment that is used in practice but is not supported by evidence and/or exposes patients to unnecessary harm.¹ They expose patients to adverse events, such as unnecessary irradiation, nonsignificant incidental findings, postoperative complications, medication or transfusion adverse effects, delays to effective treatment, and direct and indirect expenses.^2,3,4,5,6 Low-value care also puts strain on tight health care budgets, decreases the availability of scarce resources, and threatens the sustainability of affordable, accessible health care.^2,3,4 In the US in 2018, 34% of Medicare patients received at least one of the low-value clinical practices identified by the educational campaign Choosing Wisely.⁷ Similarly, according to the Canadian Institute for Health Information 2017 report, 1 in 4 Canadians were recommended a test or treatment by a physician that they did not feel was necessary.⁸ In 2013, $270 billion were wasted on excess health care services in the US.⁶

Trauma is the principal cause of life-years lost and one of the most resource-intensive diagnostic groups in North America.⁹ Owing to its time-dependent nature, the multiple clinical specialties involved, and the multitude of diagnostic and treatment options, trauma care represents a fertile breeding ground for low-value care. Several specialty societies publish guidelines on evidence-based trauma care, but the emphasis is on adherence to recommended practices rather than practices that should be avoided.¹⁰ Furthermore, although the Choosing Wisely campaign publishes recommendations on practices to be questioned, few apply to trauma care.^{11,12,13,14,15} Quality indicators have been incorporated into trauma systems in most high-income countries^{16,17,18,19,20,21,22,23} and have been associated with decreases in mortality and hospital days.²⁴ However, these indicators all target the underuse of recommended interventions and fail to capture interventions whose systematic use should be avoided. A review of major national health care performance programs reported that 94% of quality indicators address the underuse of recommended interventions.²⁵ This may be because the risks of underuse are perceived to be much greater than those of overuse.²⁶ Indicators of low-value care (overuse) are needed to ensure health care professionals ask themselves not only “am I doing enough?” but also “am I doing too much?” The objective of this study was to develop an evidence- and patient-informed, consensus-based set of quality indicators targeting low-value clinical practices in acute, in-hospital trauma care.

Methods

This consensus study was conducted from April 20 to June 9, 2021, in accordance with Research and Development/University of California at Los Angeles (RAND/UCLA) recommendations.²⁷ The results are reported according to methodological criteria for reporting of consensus studies.²⁸ The protocol was developed and approved a priori by all authors and a national public granting agency peer review committee, the Canadian Institutes of Health Research. The study was approved by the CHU de Québec-Université Laval research ethics committee who determined that informed consent of health care professionals was not required. Written informed consent was obtained from patient participants. This study followed the American Association for Public Opinion Research (AAPOR) reporting guidelines.

This study was based on 50 potentially low-value clinical practices previously identified in a scoping review and expert consultation survey (36 clinicians; participation rate = 92%), explained in detail elsewhere.^29,30 In a subsequent phase, we synthesized the evidence base for each of these practices using umbrella review (ie, review of reviews) and rapid review methodology in collaboration with Cochrane Canada.^31,32,33 We then estimated the frequency of practices that could be evaluated using trauma registry data from a Canadian provincial trauma system.¹⁰

Panel Formation

The project advisory committee assembled 2 panels of experts, both fulfilling the same tasks. Information regarding panel expert race and ethnicity was not included in this survey because the number of categories required to be inclusive would have precluded analysis based on these variables. First, we assembled an international, multidisciplinary panel by consulting key trauma care organizations and published literature. This panel included key clinical specialties involved in trauma care (emergency medicine, trauma surgery, critical care, neurosurgery, spine and orthopedic surgery, and nursing) as well as trauma medical directors, trauma care coordinators, and trauma accreditation representatives. We selected clinicians based on leadership in their specialty, involvement in injury research, geographical diversity (Canada, the US, Australia, and the UK), and different practice settings. We also assembled a panel of 16 local stakeholders from the provincial trauma system in Québec, Canada. This panel was added in accordance with our integrated knowledge translation approach to facilitate the uptake of selected indicators and to evaluate the association of context with indicator selection. We then identified 3 patient partners who were recruited, trained, and supported in collaboration with the Strategy for Patient-Oriented Research (SPOR) Support for People and Patient-Oriented Research and Trials (SUPPORT) Unit.³⁴ Before the consensus study, we held a preparatory meeting with patient partners to explore their perceptions and priorities on low-value care using a semistructured thematic guide (eMethods 1 in Supplement 1). The meeting was videotaped and transcribed under patient consent.

Questionnaire Development

The web-based³⁵ consensus questionnaire was developed by research team members, then piloted by the advisory committee and refined. The first page included a link to a 5-minute introductory video.³⁶ The questionnaire included a standardized information sheet on each practice with the following information: working definition, example of clinical presentation, synthesis of the best available evidence on effectiveness and harm, and estimates of frequency (eMethods 2 in Supplement 1).¹⁰

Consensus Process

The consensus process comprised 2 rounds, including a videoconference workshop (Figure 1). In round 1 (April 2021), we had 50 clinical practices to rate at an estimated time of 5 minutes per practice. To ensure feasibility, the project advisory committee assigned panelists to practices according to whether they were involved in the administration or decision to administer that practice. Participants received an email with a link to the web questionnaire. They were asked to independently rate each practice according to 4 quality indicator attributes using a 7-point Likert scale (1 = strongly disagree to 7 = strongly agree; eMethods 2 in Supplement 1). Indicator attributes were as follows: the practice addresses an important problem, has a strong evidence base, can be acted upon, and can be measured.

Figure 1. Study Conduct.

^aThe project advisory committee assigned panelists to practices. For example, critical care physicians and neurosurgeons for intracranial pressure monitoring; emergency physicians and orthopedic surgeons for radiographic analysis of a suspected knee injury; and emergency physicians and neurosurgeons for pretransfer head computed tomography.

Round 2 consisted of a half-day videoconference workshop with all participants including the 3 patient partners. Before the workshop, participants received a personalized report with their results and group results from the first round. We held 2 workshops: one with the international panel in English (May 31, 2021) and one with the local stakeholder panel in French (June 9, 2021). Members of the research team first presented background information and results from the first round, and one of the patient partners presented patient perceptions and priorities (eMethods 1 in Supplement 1). Two independent moderators (physicians with expertise in knowledge translation not involved in adult trauma care) then presented practices for which consensus had not been reached in round 1 and facilitated discussions among panelists. Patient partners participated in discussions and were consulted at the end of each discussion for any additional input. All panelists were then asked to rate the practice using a polling system. In each round, participants were given the opportunity to suggest additional practices and to provide comments/revisions for existing ones.

Statistical Analysis

We calculated the median rating for each clinical practice on each of the 4 criteria (of 7 criteria) and globally (of 28 criteria). Clinical practices were considered to be selected by consensus if their median global score was greater than or equal to 24 of 28 with no disagreement.²⁷ We defined disagreement as ratings by at least one-third of participants in each of the 2 extremities of the 7-point scale (1-2 or 6-7). Clinical practices with global median ratings less than or equal to 8 of 24 or with a median rating on the criterion importance of 2 or less were not retained. All cutoff points were established a priori by our advisory committee using RAND/UCLA cutoff points for 9-point Likert scales.²⁷ Analyses were conducted globally and stratified by panel.

Sensitivity analyses, planned a priori, were conducted to evaluate the influence of (1) clinical specialty (restricted to participants involved in the administration or decision to administer the practice), (2) geographic location (restricted to Canadian experts), and (3) the criterion measurable, which should not be a barrier to selecting quality indicators³⁷ (ratings on this criterion excluded). We also conducted analyses separately for the patient partners.

We grouped panelists’ comments in themes as follows: definition modification, support for the indicator, and reservations about the indicator. Definitions were refined throughout the study process using these comments and on consultation with content experts. Statistical analysis was performed using SAS software, version 9.4 (SAS Analytics).

Results

Study Participants

We invited 51 experts to participate, of which 49 were eligible (Figure 2). Of these, 46 experts (94%; 18 experts [39%] aged ≥50 years; 37 men [80%]; 9 women [20%]) completed at least 1 round, and 36 (74%) completed both rounds. Panelists were evenly distributed across age groups (Table 1). Five Canadian provinces were represented, and the international panel included participants from the US, Australia, and the UK. Participants from the international panel practiced in level I or level II trauma centers whereas level III centers were represented on the local stakeholder panel. Participants were evenly distributed across categories of years of practice, and key clinical specialties were represented. Three of the 4 invited patient partners participated in round 2 (Table 2). They had various injury profiles (traumatic brain injury [TBI], orthopedic injury, thoracoabdominal injury) and collective experience of the whole trauma care continuum, including interhospital transfer, surgery, intensive care unit (ICU) admission, and in-patient rehabilitation.

Figure 2. Participation Flowchart.

^aTwo were no longer actively involved in treating trauma patients.

^bIn addition, 3 of 4 invited patient partners participated in round 2.

Table 1. Participant Characteristics^a.

Characteristic	No. (%)
	International (n = 29)	Local (n = 14)
Age, y
30-39	5 (17)	3 (21)
40-49	14 (48)	6 (43)
50-59	7 (24)	2 (14)
≥60	3 (10)	3 (21)
Male sexb	23 (79)	11 (79)
Female sex	6 (21)	3 (21)
Practice location
British Columbia	2 (7)	NA
Alberta	4 (14)	NA
Manitoba	1 (3)	NA
Ontario	3 (10)	NA
Québec	14 (48)	14 (100)
Internationalc	5 (17)	NA
Designation level of practicing trauma centerd
I	22 (76)	6 (43)
II	3 (10)	4 (28)
III	NA	2 (14)
Other	4 (14)	2 (14)
Patient volume, per year
<20	5 (17)	3 (21)
20-50	2 (7)	2 (14)
51-100	3 (10)	1 (7)
101-200	5 (17)	1 (7)
>200	14 (48)	7 (50)
Years of practice
<5	4 (14)	2 (14)
6-10	5 (17)	4 (29)
11-20	13 (45)	5 (36)
>20	7 (24)	3 (21)
Clinical specialty
Emergency physician	4 (14)	5 (36)
Critical care	6 (21)	1 (7)
General/trauma surgeon	9 (31)	1 (7)
Neurosurgeon	3 (10)	NA
Orthopedic surgeon	3 (10)	1 (6)
Nursing	2 (7)	2 (14)
Trauma care coordinator	2 (7)	4 (28)

Abbreviation: NA, not applicable.

^a Demographic information on patient partners is presented in the article.

^b No participants identified as other than male or female.

^c Includes 2 participants from Australia, 2 from the United Kingdom, and 1 from the United States.

^d Level I: high-volume, highly specialized metropolitan centers; level II: specialized urban centers; level III centers: transfer most major trauma to level I/II centers.

Table 2. Summary of Quality Indicators Accepted and Rejected in Each Round.

Indicator	All participants		International experts		Local stakeholders
	Round 1	Round 2	Round 1	Round 2	Round 1	Round 2
Total	50	39	50	39	50	39
Accepted	8	3	NAa	4	NAa	6
Rejected	3	6	NAa	9	NAa	7

Abbreviation: NA, not applicable.

^a Only results from all participants are presented for round 1 because only participants involved in the administration or decision to administer that practice voted on related quality indicators in the first round.

Patient Perceptions and Priorities

Among the practices that should be prioritized for de-adoption, patients named initial and repeated imaging, unnecessary consultations, blood product transfusions, and surgeries (eMethods 1 in Supplement 1). They also identified prescription of opioids without discussing alternatives or weaning strategies as an additional priority. Among strategies that could be used to reduce low-value care while meeting patient and family expectations, they identified shared decision-making, raising public awareness, and educational materials for patients and clinicians.

Selected Indicators

Of 50 clinical practices, 11 (22%) were selected globally, 2 additional practices (4%) were selected by the international panel, and 3 (6%) by the local stakeholder panel (Figure 3 and eTable 1 in Supplement 1). Of the remaining practices, 19 (38%) did not reach consensus, 9 (18%) were rejected globally, 3 (6%) more were rejected by the international panel and another 3 (6%) by the local stakeholder panel. The number of participants voting on each practice for each round is provided in eTable 2 in Supplement 1.

Figure 3. Clinical Practices Selected as Quality Indicators.

ASA, American Society of Anesthesiologists; CT, computed tomography; ICU, intensive care unit; TBI, traumatic brain injury.

Selected indicators targeted 9 areas of trauma care: (1) initial diagnostic imaging (head, cervical spine, ankle, and pelvis), (2) repeated diagnostic imaging (posttransfer computed tomography [CT] and repeated head CT), (3) consultation (neurosurgical and spine), (4) surgery (penetrating neck injury), (5) blood product administration, (6) medication (antibiotic prophylaxis and late seizure prophylaxis), (7) trauma service admission (blunt abdominal trauma), (8) ICU admission (mild complicated TBI), and (9) routine blood work (minor orthopedic surgery). Global scores were similar across panels, except for trauma service admission for blunt abdominal trauma (25.5 and 16 of 28 for international and local stakeholders, respectively) and whole-body CT in minor or single-system trauma (21 and 26.5 for international and local stakeholders, respectively) (Figure 3 and eTable 3 in Supplement 1).

Sensitivity Analyses

When analyses were restricted to clinicians involved in the administration or decision to administer a practice, 3 additional practices were selected (eTable 3 in Supplement 1). When analyses were restricted to participants from Canada, 1 selected practice did not reach consensus, and 2 additional practices were selected. When the criterion measurable was excluded, 2 selected practices did not reach consensus, and 2 additional practices were selected. However, global scores remained similar in all sensitivity analyses for all practices except orthosis in thoracolumbar burst fractures with no neurologic symptoms (25 of 28 for spine surgeons vs 22 of 28 in main analysis).

All indicators that were selected by clinicians were highly rated by patient partners (results not shown, n = 3). Three indicators that were rejected were rated 24 or greater by all patient partners: trauma service admission for mild TBI, wrist radiography for patients negative on a clinical decision rule, and hourly neurologic assessment for stable patients in the ICU.

Panelists’ Comments

Among common supporting comments were the strong evidence base, significant association with costs and resource use, high frequency, and significant practice variation (eTable 4 in Supplement 1). Among the panelists’ concerns were difficulty measuring the practice or capturing clinical indications with routinely collected data, low frequency, lack of expertise in peripheral centers, and legal concerns.

Discussion

In this RAND/UCLA expert consensus study, we identified 11 low-value clinical practices that are considered sufficiently important, evidence-based, actionable, and measurable to be used as quality indicators targeting low-value trauma care. They also align with patient priorities. Five additional practices were selected by either the international or local stakeholder panel. The selected quality indicators cover different aspects of trauma care, including initial and repeated diagnostic imaging, consultation, surgery, blood product administration, medication, trauma service and ICU admission, and routine blood work. They also cover key clinical specialties involved in injury care and various diagnostic groups, including TBI, and thoracoabdominal, orthopedic, and multisystem injury. These indicators provide opportunities to measure and reduce the use of low-value practices in acute trauma care.

Quality indicator development is often based on clinical practice guidelines.³⁸ Although we did not limit ourselves to this approach because guidelines largely focus on increasing the use of recommended practices, in our scoping review, we did scan the guidelines of relevant internationally recognized health care organizations to identify recommendations on trauma care practices that should be avoided.^10,30 These included 5 recommendations of the Choosing Wisely initiative,^{11,12,13,14,15} of which 4 (80%) were selected in our study: initial head, cervical spine and ankle radiography in low-risk patients, and blood product transfusion. Whole-body CT in minor or single-system trauma was selected by the local stakeholder panel but not the international panel. Panelists commented that although there is no evidence that whole-body CT saves lives,³⁹ they can accelerate the process of care.³⁹ Among 13 relevant recommendations identified in other guidelines, 6 (46%) were selected in our study: operative exploration of penetrating neck injury without indications,⁴⁰ late seizure prophylaxis,⁴¹ antibiotic prophylaxis for external ventricular drain placement,⁴² neurosurgical consultation in mild complicated TBI,⁴³ spine consultation for isolated transverse process fractures,⁴⁴ and pelvic radiography in adults with a negative physical examination.⁴⁵ Guideline recommendations that were not selected included operative management of isolated splenic injuries⁴⁶ (selected in sensitivity analyses) or liver injuries⁴⁷ (no consensus) and hospital admission in anterior abdominal stab wounds⁴⁸ (no consensus) in hemodynamically stable patients. In addition, head magnetic resonance imaging (MRI) in TBI,^12,49 high-dose corticosteroids in severe TBI, decompressive craniectomy in diffuse TBI,⁵⁰ and inferior vena cava filters in SCI⁵¹ were all rejected in our study, and therapeutic hypothermia as a stage 2 treatment for TBI did not reach consensus. Comments on these practices indicated they were not selected either because they are very rarely used (operative management of spleen or liver injuries, head MRI, high-dose corticosteroids, therapeutic hypothermia), patient denominators are too low (inferior vena cava filters, anterior abdominal stab wounds), or they are highly driven by patient/family consent (decompressive craniectomy). Only 1 practice selected in our study was not identified in evaluated guidelines: posttransfer repeated CT. However, multiple guidelines recommend avoiding unnecessary irraditation.¹⁵

Previous studies have developed quality indicators targeting low-value care in other diagnostic groups. Notably, a 2016 study systematically reviewed published measures of low-value care⁵² and a 2018 study mapped the recommendations of the Choosing Wisely initiative to quality indicators using hospital discharge data.⁵³ Two indicators reported in these studies were selected in our study: head and cervical spine imaging for low-risk patients. Other measures identified in the review related to trauma care were MRI for head injury and inferior vena cava filters to prevent pulmonary embolism, which were both rejected in our study.

Strengths and Limitations

This study followed RAND/UCLA guidelines²⁷ and the most recent recommendations on expert consensus studies.⁵⁴ The consensus process was informed by up-to-date systematic reviews of the best available evidence and frequency estimates. Panelists covered key clinical and organizational expertise involved in trauma care, represented 5 Canadian provinces, the UK, the US, and Australia and included both internationally renowned injury researchers and local stakeholders. The integration of 3 patient partners was a substantial strength; their perceptions and priorities were used to inform discussions at the virtual workshop, and they voted on all practices. In addition, participation was excellent in both rounds, showing the perceived importance of the topic.

Among study limitations, because round 1 was restricted to participants involved in the administration or decision to administer a given practice, the number of votes was low for some practices, and results could not be stratified by panel. Furthermore, expert consensus studies have been criticized because the process can be driven by strongly spoken individuals and may be influenced by personal bias, anecdotal experiences, or a lack of understanding of study goals.⁵⁵ To mitigate these limitations, we presented an introductory video to ensure that all participants understood the context of the study, we provided extensive, standardized information to inform the decision process, we conducted both an online round and a virtual workshop, and the workshop was moderated by physicians who were independent of the study process.

All selected quality indicators may not have been suitable for implementation. First, lack of accurate data may have been a barrier to developing metrics. However, we have already developed metrics for 9 of 16 selected indicators using trauma registry data.¹⁰ Others may be operationalized by adding fields to trauma registries, linking registry data to other databases or using electronic medical records, where available. Indicators that cannot be measured with available data still have value as recommendations. Second, quality indicators may have been context-specific, as is shown by the variation in results between our 2 panels and sensitivity analyses restricted to Canadian panelists. Furthermore, only 22% of participants on the international panel were from outside Canada. However, we did have representation from Australia, the UK, and the US, and decisions were based on syntheses of international evidence. Third, selected indicators did cover many aspects of trauma care but did not cover, eg, pain management, which was identified as a priority by patient partners. This should be addressed in future work. Finally, quality indicators should not be interpreted as a dictate on practices to be avoided as they are based not only on evidence (sometimes weak) but also on physician and patient personal experiences and beliefs and are likely to be justified in certain contexts. Organizations should strive to reduce the use of such practices, particularly if they are outliers, by encouraging physicians to question their systematic use and engaging in shared decision-making with patients and families.

Conclusions

We have developed a set of consensus-based quality indicators targeting reductions in low-value injury care based on the best available evidence and informed by patient priorities. Selected indicators represent a list of practices that should be questioned. They may be used in trauma quality improvement programs by national, state/provincial, or local organizations with a desire to address the problem of low-value care. The results of this study suggest that highly rated practices may be used as a starting point. Evaluations of frequency and practice variations as well as discussions with stakeholders may be used to select the most relevant quality indicators. Future research should fully validate metrics for selected indicators, explore barriers and facilitators to change in practice, and evaluate the effectiveness and cost-effectiveness of indicators in a pragmatic trial. Reducing low-value injury care has the potential to reduce physical and psychological harm to patients and free-up health care resources. It is also the most promising avenue to ensuring the sustainability of accessible, high-quality trauma care.

Supplement 1.

eMethods 1. Patient Partner Perceptions and Priorities Presented at the Virtual Workshop (Round 2)

eMethods 2. Information Sheets Presented in Round 2 for Clinical Practices Accepted Overall, by the International Panel, by the Local Stakeholders’ Panel, in Sensitivity Analyses or Highly Rated

eTable 1. Results for Each Practice Globally (All Participants) and by Panel (International and Local Stakeholder)

eTable 2. Number of Participants Voting on Each Practice in Each Round

eTable 3. Practices Accepted (✓), Rejected (✗) and Without Consensus (−) in Sensitivity Analyses

eTable 4. Summary of Panelists’ Comments for Selected Practices

Supplement 2.

Nonauthor Collaborators