Abstract
Background:
Under-triage in trauma remains prevalent, in part because of decisions made by physicians at non-trauma centers. We developed two digital behavior change interventions to recalibrate physician heuristics (pattern recognition), and randomized 688 emergency medicine physicians to use the interventions or to a control. In this observational follow-up, we evaluated whether exposure to the interventions changed physician performance in practice.
Methods:
We obtained 2016–2018 Medicare claims for severely injured patients, linked the names of trial participants to National Provider Identifiers (NPIs), and identified claims filed by trial participants for injured patients presenting to non-trauma centers in the year before and after their trial. The primary outcome measure was the triage status of severely injured patients.
Results:
We linked 670 (97%) participants to NPIs, identified claims filed for severely injured patients by 520 (76%) participants, and claims filed at non-trauma centers by 228 (33%). Most participants were white (64%), male (67%), and had more than three years of experience (91%). Patients had a median Injury Severity Score of 16 (IQR 16–17), and primarily sustained neurotrauma. After adjustment, patients treated by physicians randomized to the interventions experienced less under-triage in the year after the trial than before (41% v. 58% [−17%], p=0.015); patients treated by physicians randomized to the control experienced no difference in under-triage (49% v. 56% [−7%], p=0.35). The difference-in-the-difference was non-significant (10%, p=0.18).
Conclusions:
It was feasible to track trial participants’ performance in national claims. Sample size limitations constrained causal inference about the effect of the interventions.
Keywords: trauma triage, guidelines, physician performance, digital behavior change interventions
INTRODUCTION
Half of all injured patients are taken initially to a non-trauma center, where a physician must evaluate their injuries and decide whether they would benefit from transfer to a trauma center (triage).1 Despite 40 years of performance improvement initiatives in trauma, under-triage (failure to transfer those for whom transfer is indicated based on clinical guidelines) remains common.2,3 Patient factors (age, insurance status) and institutional factors (for-profit status, distance to a trauma center) contribute to under-triage.2,4,5 However, physician judgment also plays a key role.
Physicians use pattern recognition “short-cuts” (heuristics) to triage patients. When well calibrated, heuristics produce rapid, cognitively-efficient solutions to complex problems. However, when calibrated poorly, heuristics can produce diagnostic errors, which can result (in this case) in under-triage.6 Professionals develop well-calibrated heuristics through an experience-feedback loop that hones pattern recognition and improves recognition of relevant contextual cues.7 Given the difficulty of replicating that loop outside of extended formal training programs (e.g., residency), initiatives for reducing diagnostic error focus on constraining the use of heuristics.8 Unfortunately, these initiatives have had limited impact.
In prior work, we developed customized, theory-based digital behavior change interventions (DCBIs) to recalibrate physician heuristics, as a novel method of reducing under-triage in trauma. In two clinical trials, exposure to the interventions reduced under-triage on an experimental task.10,11 In this follow-up study, we assessed whether the interventions affected the participating physicians’ real-life practice using Medicare claims. Using a difference-in-difference design, we compared performance in the years before and after the trial for physicians in the treatment and control groups. We anticipated that it would be feasible to use claims to track physicians longitudinally, and hypothesized that greater improvement would occur for the intervention than for the control group.
METHODS
Overview of prior studies and present analysis
In 2016 and 2017, we conducted two discrete trials using DCBIs to improve physicians’ heuristic judgment.9,10 In the first trial, we tested the effect of a DCBI that attempted to recalibrate heuristics through stories (narrative engagement). We randomized participants to spend an hour using the DCBI or completing a text-based educational program modeled on the Advanced Trauma Life Support (ATLS) course (active control).9 In the second trial, we tested the effect of a DCBI that attempted to recalibrate heuristics through structured case comparison (analogical encoding). We randomized participants to spend two hours playing one of the two DCBIs or using the text-based educational program (active control), or to do nothing at all (passive control).10 In both trials, we measured the effect of the interventions on performance with a validated virtual simulation.11 The main results from these studies have been previously reported, and we describe the interventions (both DCBIs and the text-based educational applications) in more detail in the Appendix.
The current study reports follow-up of trial participants, which we performed by linking trial data to claims records from the Centers for Medicare & Medicaid Services (CMS). We used Medicare claims because they: a) include physician and institutional identifiers; c) allow a national sample of patients; c) represent a population (i.e. those over the age of 65) disproportionately affected by under-triage. We identified patients treated by trial participants in the year before and the year after their trial, and compared the triage practices of those exposed to the DCBIs and those in the control (active/passive) groups. The University of Pittsburgh Institutional Review Board approved this study and permitted a waiver of consent given the retrospective nature and minimal risk to participants. We performed all analyses using Stata 15 (Statacorp, TX) with the per-comparison alpha set at 0.017 (using a Bonferroni adjustment for multiple comparisons). We registered the study on ClinicalTrials.gov (NCT04516044).
Participants
We included all physicians recruited for the two previous studies. Using the National Plan & Provider Enumeration System (NPPES), we identified each physician’s unique National Provider Identifier (NPI). For physicians who participated in both trials, we restricted analysis to patients they treated during the first trial. We excluded physicians for whom we could not definitively identify an NPI, who did not file claims for severely injured beneficiaries, and who did not treat beneficiaries at a non-trauma center during the first visit of the first episode of care after a severe injury.
Data sources and variables
We obtained patient-level data from the 2016–2018 Medicare Beneficiary Summary Files, Inpatient and Outpatient Standard Analytic Files, and Carrier Files. We requested claims that involved a moderate to severe traumatic injury (defined as an abbreviated injury score [AIS] ≥3), using validated International Classification of Diseases, version 10, Clinical Modification (ICD-10-CM) diagnostic codes.12 We abstracted patient demographics and vital status from the Medicare Beneficiary Summary File. Information on co-morbid conditions and injury characteristics came from the ICD-10-CM diagnosis codes in the Inpatient and Outpatient claims. We calculated ISS by mapping ICD-10 codes to AIS using a well-validated program (ICDPIC).13 We derived variables on outcomes (e.g. triage status, readmission within 30 days) by linking claims into episodes of care (see below). We classified patient claims as occurring in the pre-trial or the post-trial period (see Appendix).
Physician-level data came from the NPPES and demographic surveys completed as part of trial tasks. We abstracted information on physician characteristics, including sex, board certification, and experience (see Appendix). We categorized participants as those who completed all study tasks, including the post-intervention assessment, and those who did not.
The trauma center designation of hospitals came from the Trauma Information Exchange Program (TIEP). We categorized hospitals as trauma centers (Levels I-V) and non-trauma centers (non-designated hospitals).
Identifying patients treated by trial participants
We identified patients treated by these physicians by searching the Outpatient and Inpatient Analytic Files and the Carrier Files for the NPI numbers of trial participants (see Appendix).
Constructing episodes of care
We used the Inpatient and Outpatient Standard Analytic files to identify visits to acute care, non-federal hospitals. We constructed episodes of care by ordering claims by day and linking visits that occurred within one day of each other into a single episode. For episodes with multiple claims from the same day, we ordered them under the assumption that patients would move from non-trauma centers to trauma centers, and from low-volume hospitals to high-volume hospitals (see Appendix).
Statistical analyses
We excluded episodes of care that began at Level I-IV trauma centers because the American College of Surgeons focuses on reducing under-triage at non-trauma centers, and excluded patients younger than 65 years. We categorized episodes as involving minimal-to-moderate (ISS<15) or severe (ISS>15) injuries, and restricted the analyses to those with severe injuries. We categorized episodes as occurring during the pre-trial or post-trial period.
We defined under-triage as failure to transfer a patient with severe injuries to a higher level of care, either directly from the ED or within one day of admission, as recommended by American College of Surgeons’ guidelines. We therefore categorized those who were admitted as under-triaged. We excluded patients who died on the day of admission, as this could reflect either an error in triage decision-making or an assessment of clinical instability that precluded transfer.
We summarized outcomes for all severely injured patients including: triage status, inpatient mortality, disposition status at the time of discharge from their episode of care, readmission within 30 days, and 30-day mortality.
Effect of the intervention
To determine the difference-in-difference effect of the intervention on participants in the intervention and control arms, we fit a mixed effects logistic regression model with under-triage as the dependent variable and physicians’ exposure as the primary independent variable, using an intention-to-treat approach (ignoring whether participants completed all study tasks). We adjusted for the year in which the patient presented, and clustered at the physician-level. We included the trial period (before or after) as an interaction term. We analyzed only the first episode of care, as we could not assess whether subsequent episodes reflected follow-up care or new injuries. We analyzed only the first visit within the first episode, to avoid confounding introduced by prior physicians’ decisions. In secondary analyses, we tested the effect of the exposure on other outcomes (e.g. 30-day mortality). In sensitivity analyses, we restricted the cohort to physicians who completed all the study tasks, to strengthen our ability to make causal inferences about the role of the intervention in any changes in performance. We also tested an alternative definition of non-trauma centers, reclassifying Level III-V hospitals as non-trauma centers.
Power
We estimated the sample size for the two parent trials based on Cohen’s method of estimating power for behavioral trials, to test the primary hypothesis that exposure to the DCBIs would improve performance on a validated virtual simulation compared to exposure to the control.9,10 Since this study represented a pragmatic, observational follow-up of those trials, we did not estimate a sample size.
Results
Among 688 physicians from two clinical trials, we identified an NPI number for 670 (97%). We restricted data collection for 52 (8%) physicians, who participated in both Trial 1 and 2, to the first trial. We excluded: 1) 98 (14%) physicians who did not file a claim in fee-for-service Medicare for a severely injured patient; 2) 292 (42%) physicians who did not file a claim for an eligible visit (i.e. a first visit of a first episode at a non-trauma center). See Figure 1 for a schematic of the sampling framework.
Figure 1.
Sampling frame for the study
Trial participant characteristics
Overall, the physician cohort was white (64%), male (67%), and had greater than 3 years of experience (91%). Table 1 shows a more detailed characterization of the participants. Physicians filed a median of 2 (IQR 1–3) claims for eligible visits. The median duration between the conclusion of the trial period and the evaluation of patients was 144 days (IQR 83–271).
Table 1.
Characteristics of trial participants
| Characteristics | Participants (n=618) |
|---|---|
| Sex – male (n, %) | 446 (67) |
| Race (n, %)* | |
| White | 361 (64) |
| Black | 20 (4) |
| Asian | 100 (18) |
| Hispanic | 33 (6) |
| Pacific Islander, Native American | 19 (3) |
| Other | 29 (5) |
| Specialty (n, %) | |
| Emergency medicine | 631 (94) |
| Family practice | 18 (3) |
| Internal medicine | 7 (1) |
| Other (e.g. general surgery) | 14 (2) |
| Years in practice based on date of NPI issuance | |
| ≤3 years | 61 (9) |
| 4–9 years | 269 (40) |
| ≥10 years | 340 (51) |
| Practice environment – non-trauma center only (n, %) | 324 (53) |
| Self-reported use of intervention – min (median, IQR)* | 90 (60–120) |
| Number of first visit/episode claims filed for severely injured patients who presented initially to non-trauma centers (median, IQR) | 2 (1–3) |
| Pre-trial duration between evaluation of patients and start of trial – days (median, IQR) | 233 (300–130) |
| Post-trial duration between completion of trial and evaluation of patients – days (median, IQR) | 144 (83–271) |
Data abstracted only from trial records, and therefore represents those physicians whocompleted the demographic survey (n=584)
Patient characteristics
We identified 557 patients over the age of 65 treated by trial participants during an eligible visit. Patients had a mean age of 80 (SD 11), were mostly white (82%) and male (51%). They had a median ISS of 16 (IQR 16–17), and primarily experienced single-system neurotrauma (see Table 2). Table 3 summarizes unadjusted patient outcomes.
Table 2.
Characteristics of patients treated by trial participants, stratified by exposure and period
| Control | Digital Behavior Change Interventions | |||
|---|---|---|---|---|
|
| ||||
| Pre-trial (n=113) | Post-trial (n=156) | Pre-trial (n=154) | Post-trial (n=134) | |
| Age – years (mean [SD]) | 80(11.3) | 81 (9.4) | 79 (13.0) | 79 (10.8) |
| Sex – male (n, %) | 52 (46) | 82 (53) | 66 (43) | 73 (54) |
| Race (n, %) | ||||
| White | 89 (79) | 133 (85) | 131 (85) | 105 (78) |
| Black | 11 (10) | 8 (5) | 7 (5) | 11 (8) |
| Asian | 6 (5) | 5 (3) | 9 (6) | 7 (5) |
| Hispanic | 4 (4) | 5 (3) | 0 (0) | 6 (4) |
| Native American, Pacific Islander | 0 (0) | 1 (0.6) | 0 (0) | 0 (0) |
| Other | 3 (3) | 4 (3) | 7 (5) | 5 (4) |
| Number of comorbidities, median (IQR) | 3 (1–6) | 4 (2–5) | 4 (3–6) | 4 (2–6) |
| Selected comorbidities | ||||
| Congestive heart failure | 25 (22) | 28 (18) | 21 (14) | 17 (13) |
| Cardiac arrhythmias | 41 (36) | 59 (38) | 56 (36) | 39 (29) |
| Renal failure | 25 (22) | 30 (19) | 38 (25) | 25 (19) |
| Liver disease | 2 (2) | 8 (5) | 8 (5) | 3 (2) |
| Lymphoma | 2 (2) | 3 (2) | 3 (2) | 1 (1) |
| Metastatic cancer | 2 (2) | 2 (1) | 4 (3) | 4 (3) |
| Solid tumor without metastasis | 7 (6) | 7 (4) | 4 (3) | 4 (3) |
| Coagulopathy | 20 (18) | 14 (9) | 19 (12) | 15 (11) |
| Weight loss | 11 (10) | 16 (10) | 16 (10) | 16 (12) |
| Fluid/electrolyte disorders | 34 (30) | 58 (37) | 69 (45) | 55 (41) |
| ISS (median, IQR) | 16 (16–20) | 17 (16–17) | 16 (16–17) | 16 (16–17) |
| Injury patterns | ||||
| AIS head≥3 (n, %) | 88 (78) | 122 (78) | 120 (78) | 107 (80) |
| AIS face≥3 (n, %) | 0 (0) | 0 (0) | 0 (0) | 1 (1) |
| AIS chest≥ 3 (n, %) | 3 (3) | 7 (4) | 7 (5) | 3 (2) |
| AIS abdomen≥3 (n, %) | 2 (2) | 1 (1) | 5 (3) | 0 (0) |
| AIS extremities/pelvis ≥3 (n, %) | 5 (4) | 8 (4) | 12 (8) | 3 (2) |
| AIS external≥3 (n, %) | 0 (0) | 0 (0) | 1 (1) | 0 (0) |
Table 3.
Unadjusted outcomes of patients treated by trial participants, stratified by exposure and trial period
| Outcomes | Control | Digital behavior change interventions | ||
|---|---|---|---|---|
|
| ||||
| Pre-trial (n=113) | Post-trial (n=156) | Pre-trial (n=154) | Post-trial (n=134) | |
| Under-triaged | 59 (52) | 77 (49) | 85 (55) | 56 (42) |
| Discharge status, n (%) | ||||
| Home | 50 (44) | 72 (46) | 77 (51) | 65 (49) |
| Continued acute care | 13 (12) | 9 (6) | 10 (7) | 9 (7) |
| Discharge to SNF or to rehabilitation center | 43 (38) | 55 (35) | 52 (34) | 53 (40) |
| Died | 6 (5) | 20 (13) | 12 (8) | 7 (5) |
| Other (e.g. discharged to jail) | 1 (1) | 0 (0) | 1 (1) | 0 (0) |
| Case fatality, n (%) | ||||
| Death on day of admission | 0 (0) | 3 (2) | 2 (1) | 0 (0) |
| Death during hospitalization | 7 (6) | 20 (13) | 12 (8) | 8 (6) |
| Death within 30-days of index hospitalization | 26 (23) | 40 (26) | 26 (17) | 18 (13) |
| 30-day readmission if alive at time of discharge | 10 (9) | 27 (20) | 28 (19) | 24 (19) |
Effect of the interventions
After adjustment for the year of the patient’s presentation and clustering at the physician-level, severely injured patients treated by physicians randomized to the interventions were less likely to be under-triaged in the year after the trial than in the year before the trial (58% v. 41%, p=0.015). Patients treated by physicians randomized to a control arm (active or passive) had no difference in under-triage before and after their trial (56% v. 49%, p=0.35). There was no difference-in-the-difference between the two arms of the trials (10%, p=0.18). Table 4 shows the adjusted probabilities of other outcomes and sensitivity analyses.
Table 4.
Predicted probabilities of outcomes of patients treated by trial participants*
| Outcomes | Game | Control | ||||||
|---|---|---|---|---|---|---|---|---|
|
| ||||||||
| Pre-trial | Post-trial | Pre-trial | Post-trial | Difference-indifference | ||||
|
| ||||||||
| Probability (95% CI) | Probability (95% CI) | p | Probability (95% CI) | Probability (95% CI) | p | Probability | p | |
| Primary outcome | ||||||||
| Under-triage | 0.58 (0.48–0.68) | 0.41 (0.31–0.50) | 0.015 | 0.56 (0.46–0.67) | 0.49 (0.39–0.59) | 0.35 | 0.10 | 0.18 |
| Secondary outcomes | ||||||||
| Death during first episode of care | 0.08 (0.04–0.13) | 0.06 (0.02–0.10) | 0.45 | 0.07 (0.02–0.12) | 0.12 (0.07–0.17) | 0.27 | 0.07 | 0.15 |
| Death within 30 days | 0.19 (0.12–0.25) | 0.12 (0.07–0.18) | 0.17 | 0.27 (0.17–0.37) | 0.23 (0.16–0.30) | 0.58 | 0.03 | 0.48 |
| Readmission within 30 days | 0.19 (0.12–0.26) | 0.17 (0.10–0.24) | 0.73 | 0.09 (0.03–0.15) | 0.17 (0.10–0.23) | 0.13 | 0.10 | 0.10 |
|
| ||||||||
| Sensitivity analyses | ||||||||
| Analysis restricted to physicians who completed study tasks | ||||||||
| Under-triage | 0.61 (0.51–0.72) | 0.40 (0.30–0.51) | <0.001 | 0.58 (0.46–0.71) | 0.49 (0.38–0.60) | 0.26 | 0.12 | 0.17 |
| Level III-V trauma centers reclassified as non-trauma centers | ||||||||
| Under-triage | 0.55 (0.46–0.63) | 0.47 (0.38–0.56) | 0.20 | 0.58 (0.49–0.68) | 0.5 (0.41–0.58) | 0.16 | 0 | 0.83 |
Predicted probabilities of outcomes from a series of mixed effects logistic regression models, clustered at the physician-level. The independent variable was the exposure of the physician. We adjusted for the year of the patient’s presentation and included the trial period as an interaction term. We did not include the trial (i.e. first or second) because of collinearity with the year of presentation.
Discussion
In this longitudinal follow-up of two clinical trials of DBCIs designed to improve trauma triage at non-trauma centers, we used Medicare claims to assess sustained clinical behavior change. The first key finding is that it was feasible to track trial participants’ clinical performance in national claims by linking physician trial participants’ names to publicly-available NPI numbers. Physicians randomized to the intervention under-triaged fewer patients after the trial, while physicians randomized to the control arm made similar triage decisions across the two time periods. However, we lacked the sample size to reject the null hypothesis for the difference-in-difference between the two arms, precluding any causal inferences about the effect of the game on physician triage practices.
Our study exploits the richness of Medicare claims, which include physician and hospital identifiers, allowing the tracking of patients across space and time.14 Others have described the use of claims to measure healthcare utilization after behavioral trials.15 However, they have done so by paying third-party intermediaries to link patients’ social security numbers to claims records, with the objective of minimizing privacy breaches. However, this introduces logistical barriers to the method. In contrast, we relied on publicly-available data to create the linkages, and demonstrated the feasibility of using claims to supplement primary data collection (performance in simulation). Using real-world performance data contributes to the generalizability and validity of our initial findings. It is more convincing than simulation performance because it suggests behavior change despite facilitators and barriers that affect physician performance in practice.16
Our finding that physicians randomized to the DCBIs under-triaged fewer severely injured, older patients after the trial compared with before the trial has several possible explanations. First, the DCBIs, grounded in the well-accepted dual process model of decision making, may have achieved their objective of changing physician behavior.17,18 By treating physicians’ heuristic judgment as an asset that could be improved, instead of as a liability that should be curtailed, we may have improved transference and the durability of the treatment effect.19,8 The observation that (on average) 144 days elapsed between completion of the intervention and the clinical encounter corroborates the durability of the knowledge transfer. Second, the games may simply have increased physicians’ willingness to transfer all patients (i.e., over-triage) instead of improving their recognition of the severely injured. To test this hypothesis, we would have needed to purchase claims filed for patients with minimal injuries, which was beyond the budget of the current study. Third, the difference may reflect secular trends and may be unrelated to the game intervention. We could not reject this hypothesis because the difference-in-differences between the intervention and control groups was not statistically significant. The study was underpowered to test the significance of the observed 10 percentage point difference in performance between the two groups, both because severe injuries at non-trauma centers are relatively rare events and because of our enrollment criteria. A large proportion of the trial participants worked at both trauma and non-trauma centers, which resulted in the exclusion of many of the patients that they treated.
Therefore, our findings regarding the effect of the DCBI on real-world clinical practice must be regarded as exploratory. Nevertheless, taken together with the practice improvement in simulation, there is promise that theoretically-based behavioral interventions may be a method of addressing a refractory clinical problem – under-triage in trauma. Severely injured patients treated at trauma centers have a 10–25% reduction in mortality, increased rates of return to independent living, and less pain at one year.21–24 Half of all injured patients present initially to non-trauma centers, making physician judgment central to efforts to eliminate preventable deaths and morbidity after injury.25 If effective, our interventions could reduce under-triage by as much as 10%, when layered on existing quality improvement interventions. This effect is comparable with the impact of Advanced Cardiac Life Support on return of spontaneous circulation after cardiac arrest (risk difference 10%), widely accepted as an effective method of continuing medical education.26
Our study has several limitations. First, we cannot assess the completeness of our patient sample. As many as 11% of institutional records lack matching professional claims, reflecting both administrative errors and procedures for bundling payments.27 As a result we may not have captured all the patients treated by our trial participants. We cannot predict the effects of these omissions. Second, we recruited physician volunteers at a national meeting of emergency medicine physicians, also with unknown effects on the generalizability of our observations. The pre-trial under-triage rates for these volunteers (54% across all groups) were lower than those reported in observational studies (70–80%).1,3 Third, we used fee-for-service Medicare data to evaluate physician behavior. Our observations may not be generalizable to younger patients. However, patients over the age of 65 make up an increasing proportion of trauma patients (35%), are less likely to be triaged appropriately (odds ratio 0.48), and are more likely to experience adverse consequences after injury.5,28,29 Fourth, we use ICD10-derived ISS to identify our cohort, an imperfect, albeit well-validated, process. However, we have no reason to believe that this misspecification varies systematically. Fifth, our use of Medicare claims to assess performance limited our ability to measure formally the fidelity and sustainability of the intervention.
In summary, we present preliminary evidence that theoretically-based DCBIs may improve triage at non-trauma centers. These observations support experimental observations about the effect of behavioral interventions on diagnostic error and warrant additional investigation.
Supplementary Material
Acknowledgments
The authors thank the trial participants and the game development teams at Schell Games.
Financial support and potential conflict of interest
This work was supported by the National Institutes of Health through grant DP2 LM 012339 (Mohan). The funding agency reviewed the study but played no role in the design, collection, analysis or interpretation.
Grant money for investigator-initiated research
DM reports grant money to the University of Pittsburgh to conduct research conceived and written by Dr. Mohan from the National Library of Medicine (DP2 LM 012339).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Access to data and data analysis
Dr. Mohan had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Data Sharing
De-identified data will be shared on request, conditional on approval by the Institutional IRB.
Meetings at which this paper was presented
None
Conflict of interest
DM, CCC, BF, MRR, DCA, DMY, AEB report no conflict of interest.
REFERENCES
- 1.Mohan D, Wallace DJ, Kerti SJ et al. Association of practitioner interfacility triage performance with outcomes for severely injured patients with fee-for-service Medicare insurance. JAMA Surgery 2019; 154: e193944. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Delgado MK, Yokell MA, Staudenmayer KL et al. Factors associated with the disposition of severely injured patients initially seen at non-trauma center emergency departments. JAMA Surg 2014; 149: 422–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Zhou Q, Rosengart MR, Billiar TR, et al. Factors associated with non-transfer in trauma patients meeting American College of Surgeons’ criteria for transfer at nontertiary centers. JAMA Surg 2017; 152: 369–376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Gomez D, Haas B, de Mestral C, et al. Institutional and provider factors impeding access to trauma center care: an analysis of transfer practices in a regional trauma system. J Trauma 2012; 73: 1288–93. [DOI] [PubMed] [Google Scholar]
- 5.Chang DC, Bass RR, Cornell EE et al. Undertriage of elderly trauma patients to state-designated trauma centers. Arch Surg 2008; 143: 776–781. [DOI] [PubMed] [Google Scholar]
- 6.The National Academies of Sciences, Engineering and Medicine (2016) Improving Diagnosis in Health Care Washington DC: National Academies Press, Washington DC. [Google Scholar]
- 7.Kahneman D and Klein G. Conditions for intuitive expertise: a failure to disagree. Am Psychol 2009; 64: 515–526. [DOI] [PubMed] [Google Scholar]
- 8.Graber ML, Kissam S, Payne VL, et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Qual Saf 2012; 21: 535–557 [DOI] [PubMed] [Google Scholar]
- 9.Mohan D, Farris C, Fischhoff B, et al. Efficacy of educational video game versus traditional educational apps at improving physician decision making in trauma triage: a randomized clinical trial. BMJ 2017. 359:j5416. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Mohan D, Fischhoff B, Angus DC et al. Using serious video games to improve physicians’ heuristics in trauma triage: a randomized clinical trial. PNAS 2018. 115: 9204–9209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Mohan D, Angus DC, Ricketts D, et al. Assessing the validity of using serious game technology to analyze physician decision making. PLoS One 2014; 9: e105445. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Mohan D, Rosengart MR, Farris C, et al. Assessing the feasibility of the American College of Surgeons’ benchmarks for the triage of trauma patients. Arch Surg 2011; 146: 786–792. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Greene NH, Kernic MA, Vavilala MS et al. Validation of ICDPIC software injury severity scores using a large regional trauma registry. Inj Prev 2015; 21: 325–30. [DOI] [PubMed] [Google Scholar]
- 14.Centers for Medicare and Medicaid Services. CCW technical guidance: getting started with CMS Medicare administrative research files (version 2.5) Iowa: Chronic Conditions Warehouse, 2017. [Google Scholar]
- 15.Bhasin S, Gill TM, Reuben DB et al. A randomized trial of a multifactorial strategy to prevent serious fall injuries. NEJM 2020; 383: 129–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Nabhan C, Klink A, and Prasad V. Real-world evidence–what does it really mean? JAMA Oncol 2019; 5: 781–783. [DOI] [PubMed] [Google Scholar]
- 17.Cane J, O’Connor D, and Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implementation Science 2012; 7: 37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Noar SM and Zimmerman RS. Health behavior theory and cumulative knowledge regarding health behaviors: are we moving in the right direction? Health Education Research 2005; 20: 275–290. [DOI] [PubMed] [Google Scholar]
- 19.Klein G Sources of Power Massachusetts: MIT Press, 1985. [Google Scholar]
- 20.Mohan D, Barnato AE, Rosengart MR, et al. Trauma triage in the emergency departments of non-trauma centers: an analysis of individual physician caseload on triage patterns. J Trauma 2013; 74(6): 1541–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.MacKenzie EJ, Rivara FP, Jurkovich GJ, et al. The impact of trauma-center care on functional outcomes following major lower-limb trauma. J Bone Joint Surg Am 2008; 90: 101–9. [DOI] [PubMed] [Google Scholar]
- 22.MacKenzie EJ, Weir S, Rivara FP, et al. The Value of Trauma Center Care. J Trauma 2010; 69: 1–10. [DOI] [PubMed] [Google Scholar]
- 23.MacKenzie EJ, Rivara FP, Jurkovich GJ, et al. A national evaluation of the effect of trauma-center care on mortality. NEJM 2006; 354: 366–378. [DOI] [PubMed] [Google Scholar]
- 24.Celso B, Tepas J, Langland-Orban B et al. A systematic review and meta-analysis comparing outcome of severely injured patients treated in trauma centers following the establishment of trauma systems. J Trauma 2006; 60: 371–78. [DOI] [PubMed] [Google Scholar]
- 25.National Academies of Sciences, Engineering, and Medicine. A national trauma care system: integrating military and civilian trauma systems to achieve zero preventable deaths after injury Washington DC: The National Academies Press, 2016. [PubMed] [Google Scholar]
- 26.Lockey A, Lin Y, and Cheng A. Impact of adult advanced cardiac life support course participation on patient outcomes – a systematic review and meta-analysis. Resuscitation 2018; 129: 48–54. [DOI] [PubMed] [Google Scholar]
- 27.Venkatesh AK, Mei H, Kocher KE et al. Identification of emergency department visits in Medicare administrative claims: approaches and implications. Acad Emerg Med 2017; 24: 422–431. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Calland JF, Ingraham AM, Martin N, et al. Evaluation and management of geriatric trauma: an Eastern Association for the Surgery of Trauma practice management guideline. J Trauma Acute Care Surg 2012; 73: S345–50. [DOI] [PubMed] [Google Scholar]
- 29.Goodmanson NW, Rosengart MR, Barnato AE, et al. Defining geriatric trauma: when does age make a difference? Surgery 2012; 152: 668–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.