EMS Provider Assessment of Vehicle Damage Compared to a Professional Crash Reconstructionist

Abstract

Objective

To determine the accuracy of EMS provider assessments of motor vehicle damage, when compared to measurements made by a professional crash reconstructionist.

Methods

EMS providers caring for adult patients injured during a motor vehicle crash and transported to the regional trauma center in a midsized community were interviewed upon ED arrival. The interview collected provider estimates of crash mechanism of injury. For crashes that met a preset severity threshold, the vehicle’s owner was asked to consent to having a crash reconstructionist assess their vehicle. The assessment included measuring intrusion and external auto deformity. Vehicle damage was used to calculate change in velocity. Paired t-test and correlation were used to compare EMS estimates and investigator derived values.

Results

91 vehicles were enrolled; of these 58 were inspected and 33 were excluded because the vehicle was not accessible. 6 vehicles had multiple patients. Therefore, a total of 68 EMS estimates were compared to the inspection findings. Patients were 46% male, 28% admitted to hospital, and 1% died. Mean EMS estimated deformity was 18” and mean measured was 14”. Mean EMS estimated intrusion was 5” and mean measured was 4”. EMS providers and the reconstructionist had 67% agreement for determination of external auto deformity (kappa 0.26), and 88% agreement for determination of intrusion (kappa 0.27) when the 1999 Field Triage Decision Scheme Criteria were applied. Mean EMS estimated speed prior to the crash was 48 mph±13 and mean reconstructionist estimated change in velocity was 18 mph±12 (correlation -0.45). EMS determined that 19 vehicles had rolled over while the investigator identified 18 (kappa 0.96). In 55 cases EMS and the investigator agreed on seatbelt use, for the remaining 13 cases there was disagreement (5) or the investigator was unable to make a determination (8) (kappa 0.40).

Conclusions

This study found that EMS providers are good at estimating rollover. Vehicle intrusion, deformity, and seatbelt use appear to be more difficult to estimate with only fair agreement with the crash reconstructionist. As expected, the EMS provider estimated speed prior to the crash does not appear to be a reasonable proxy for change in velocity.

Introduction

Emergency Medical Services (EMS) providers are charged with determining if motor vehicle crash (MVC) patients are likely to need the resources of a trauma center. This decision has been shown to directly affect patient outcome.¹ Therefore, it is important that EMS providers identify severe trauma patients to avoid under-triage. Alternatively, over-triage can have negative system effects such as increasing ambulance turn-around times due to longer transport distances, increasing patient volume at the trauma center which could lead to overcrowding, bypassing outlying hospitals which could have negative economic consequences for those hospitals, and the utilization of helicopter transport or rapid lights and siren ground transport could increase the risk of injury due to crashes for the patient and the EMS crew. The American College of Surgeon’s Field Triage Criteria were developed to assist EMS providers in identifying patients who need a trauma center. However, these criteria require providers to estimate vehicle damage and other indicators of the crash mechanism of injury.² It is anecdotally believed that EMS providers cannot accurately estimate many of these factors, including initial speed prior to the crash, intrusion into the passenger compartment, or major auto deformity. No published data was found to support or refute this common assertion and all three variables have been used for field triage.

A trained crash reconstructionist spends approximately three hours examining a vehicle to measure the amount of intrusion and deformity a vehicle has sustained. Determining the vehicle’s change in velocity at the time of impact requires numerous measurements and calculations. Making this determination has been shown to be difficult.^{3, 4} In fact, measuring a vehicle’s change in velocity at the time of impact is thought to be impossible for EMS providers, so instead they are asked to estimate the vehicle’s initial speed prior to the crash. Given the limited time EMS has to assess the physical evidence of a crash scene, the accuracy of their mechanism of injury estimates is unknown and unstudied. It is also unknown if their estimates of initial speed correlate with the vehicle’s actual change in velocity, which has been shown to be a measure of crash severity.^{5, 6}

Motor vehicle crash mechanism of injury continues to be considered an important criteria for EMS to use to identify the most appropriate receiving facility for an injured patient.² The Centers for Disease Control and Prevention have partnered with the American College of Surgeon’s to update the Field Triage Criteria, which in turn has stimulated research in this area. However, it is important to determine if EMS providers’ estimations of the crash mechanism of injury are accurate since research on the predictive value of the mechanism of injury criteria has had conflicting results. ^7–12 The variations in these findings may be because the mechanism of injury is poorly estimated by field providers or because the mechanism of injury is in fact a poor predictor of trauma center need. The objective of this study was to compare EMS estimations of the crash mechanism of injury to measurements and calculations made by a professional crash reconstructionist.

Methods

This was a planned sub-study of a larger prospective investigation that interviewed EMS providers caring for injured adult (age greater than or equal to 18 years) patients arriving to the emergency department by ambulance. The study was conducted at the University of Rochester Medical Center/Strong Memorial Hospital which is the only state-designated regional trauma center in the Rochester, NY area. The emergency department has nearly 100,000 annual visits. Data was collected from March 2007 to March 2009.

Trained patient enrollers interviewed the EMS provider in charge of care for patients who presented with injuries that resulted from an MVC and were transported to the emergency department by ambulance. Patient enrollers were available in the emergency department seven days a week from 8:00am to midnight. The EMS provider was asked to provide their estimates of the crash mechanism of injury including: seatbelt use, rollover, estimated speed of the vehicle immediately prior to the crash, inches of external auto deformation, inches of intrusion into the passenger compartment, principle direction of force in the crash, and where the patient was seated in the vehicle. No education was provided for how to estimate vehicle damage as part of the study. EMS providers should have received training on estimating vehicle damage during their initial and continued medical education. EMS providers were not told the objective of the study, only that we were studying the Trauma Triage Guidelines.

Vehicle owners for crashes that met a preset severity threshold, regardless of vehicle type, were asked to consent to having a crash reconstructionist assess their vehicle for residual damage. The preset severity threshold was: (1) any vehicle occupant that had an immediate trauma team activation upon arrival at the hospital, or (2) EMS reporting significant vehicle damage, which was defined as initial speed greater than 40 mph, intrusion greater than 12 inches, or vehicle deformity greater than 20 inches. Vehicle owners were excluded if the patient had multiple mechanisms of injury (e.g., gunshot wound then motor vehicle accident); an inability of the field provider to estimate crash speed, external auto deformity, or intrusion into the passenger compartment; the vehicle struck an animal (e.g., dog or deer); the patient or vehicle owner spoke a language other than English; the vehicle owner was under 18 years of age; or the vehicle was owned by an agency or company.

If the owner consented to the investigation, a single crash reconstructionist was notified. The crash reconstructionist utilized in the study was an experienced professional mechanical engineer with over 20 years of crash analysis and reconstruction experience. The study team determined the location of the vehicle and the reconstructionist went to the site to measure the damaged vehicle. The reconstructionist was blinded to the estimates provided by the EMS provider(s).

Standard measurement techniques were used to evaluate the residual intrusion and external auto deformity profiles of the damaged vehicles. Occupant compartment intrusion was measured by first identifying the specific vehicle structure that had intruded. The location of the intruded component was then measured with reference to an undamaged fixed point on the vehicle. Comparison measurements of the identified component to the reference point were then taken on an exemplar (undamaged) vehicle. The difference in the measurements yielded the extent/depth of the intrusion.

External auto deformity was measured in a similar manner. The vehicle manufacturer’s specifications were used to establish the location(s) of the subject vehicle’s undamaged plane(s), then the distance from the undamaged plane to the deformed plane (e.g., where the vehicle’s surface was after the crash to where it should have been before the crash) yielded the residual external auto deformity sustained by the vehicle during the crash event. The greatest distance (e.g., the maximum crush value within the profile) was used to compare to the EMS provider’s estimate.

The residual external auto deformity profile, in conjunction with vehicle specific mass-properties, was used as the input to the WinSMASH (version 2008.6.11.50) computer program to calculate the change in velocity (i.e., delta-V).¹³ The WinSMASH program is a Microsoft Windows based enhancement of the two-dimensional CRASH (Calspan Reconstruction of the Accident Speeds on the Highway) Collision Model developed for the National Highway Traffic Safety Administration (NHTSA).

Descriptive statistics were calculated including mean estimates with their standard deviations (±) and box plots of measurement distributions. Paired t-test, correlation, and kappa were used to compare EMS estimates and reconstructionist derived values. EMS providers typically determine if a pre-set threshold has been met for deformity and intrusion in order to consider the patient appropriate for the services of a trauma center. Therefore, we also compared agreement between EMS providers and the reconstructionist when the preset thresholds from the 1999 American College of Surgeon’s Field Triage Decision Scheme were applied. This study was approved by the University of Rochester and the Medical College of Wisconsin Institutional Review Boards.

Results

A convenience sample of 91 vehicle owners consented to the study. The EMS interview for all patients in that vehicle who were transported to the participating study hospital were obtained and used for the analysis. When the reconstructionist attempted to inspect the 91 enrolled vehicles, 33 were not accessible and had to be excluded. These vehicles either could not be found because the location where they were towed was not known by the owner, the personnel at the lot where the vehicle was located would not allow the inspection, the vehicle had been destroyed, or the damage had already been repaired.

The remaining 58 vehicles were inspected. Six of those vehicles had multiple occupants (2 or more people) at the time of the crash and a survey was conducted with the EMS providers who had transported each patient. For those vehicles the data was matched by seat position. If a vehicle had multiple occupants but an EMS provider interview was not conducted for each occupant then only those occupants with an EMS provider interview were included in the study. A total of 68 EMS estimations were analyzed.

The average occupant age was 38 years ± 18 years and 31 (46%) were male. Of the 68 occupants, 19 (28%) were admitted to the hospital from the emergency department and 1 (1%) ultimately died from their injuries.

EMS providers identified 19 rollovers and the reconstructionist confirmed that 18 of those vehicles did in fact have signs of rolling over (Kappa = 0.96). EMS providers and the reconstructionist agreed on seatbelt usage for 55 (81%) of the occupants(Kappa = 0.40). They disagreed on seatbelt usage in 5 (7%) cases, and in 8 (12%) cases the reconstructionist could not determine if seatbelts had been worn.

The mean initial speed prior to the crash estimated by EMS providers was 48 mph ± 13 mph, the reconstructionist estimated change in velocity was 18 mph ± 12 mph. EMS provider estimated initial speed did not correlate with the reconstructionist’s estimated change in velocity (r = −0.45).

EMS providers estimated a mean external auto deformity of 18 inches ± 14 inches compared to the reconstructionist measured mean deformity of 14 inches ± 12 inches (figure 1). The mean difference was 4 inches ± 16 inches (95% CI: −0.03 – 8; p<0.052). EMS providers estimated a mean internal intrusion of 5 inches ± 7 inches, the reconstructionist measured mean intrusion was 4 inches ± 6 inches (figure 2). The mean difference was 1 inch ± 6 inches (95% CI −0.5 –2; p<0.218). Tables 1 and 2 compare agreement between EMS providers and the reconstructionist when the preset thresholds from the 1999 American College of Surgeon’s Field Triage Decision Scheme were applied. EMS providers and the reconstructionist had 67% agreement for determinations of external auto deformity (Kappa = 0.26), and had 88% agreement for determinations of intrusion (Kappa =0.27).

Figure 1.

Box plot of comparison of Crash Investigator and EMS Provider measured Vehicle Deformity

Figure 2.

Box Plot of Comparison of Crash Investigator and EMS Provider measured Vehicle Intrusion

Table 1.

Agreement Between EMS identified external auto deformity of greater than 20 inches to crash reconstructionist measurement of greater than 20 inches

	EMS Estimated
Reconstructionist Measured	Less than or equal to 20 in.	Greater than 20 in.
Less than 20 in.	37 (56%)	18 (27%)
Greater than 20 in.	3 (5%)	8 (12%)

^*Shaded boxes indicate agreement (68%) unshaded boxes indicate disagreement (32%) kappa = 0.26

^**deformity not reported for 2 cases, 66 cases were analyzed

Table 2.

Agreement Between EMS identified intrusion of greater than 12 inches to crash reconstructionist measurements of greater than 12 inches

	EMS Estimated
Reconstructionist Measured	Less than or equal to 12 in.	Greater than 12 in.
Less than 12 in.	57 (85%)	5 (7%)
Greater than 12 in.	3 (5%)	2 (3%)

^*Shaded boxes indicate agreement (88%) unshaded boxes indicate disagreement (12%) kappa =0.27

^**Intrusion not reported for 1 case, 67 cases were analyzed

Discussion

We found that EMS providers were best at determining if the vehicle rolled over. Vehicle intrusion, deformity, and seatbelt use appear to be more difficult for EMS to accurately estimate and only fair agreement with the crash reconstructionist was observed. As we expected, the EMS-estimated initial speed prior to the crash was not a proxy for the change in velocity.

Previous studies have identified discrepancies in medical personnel’s reports of vehicle damage¹⁴ and documentation of vehicle damage.¹⁵ A previous study comparing EMS patient care reports to a formal crash investigation found poor agreement particularly for seatbelt use.¹⁶ Our study found fair agreement on seatbelt use, likely because we interviewed providers about what they observed rather than relying on their documentation. It seems probable that, based on our results, previous findings of EMS providers not accurately reporting seatbelt use is an issue of documentation rather than observation. It is likely that EMS can reliably identify seatbelt use but documentation of that use is not reliable. Unfortunately, in our study we were not able to review EMS provider documentation to determine if it mirrored their reporting during our interview.

The recently released, revised American College of Surgeon’s Field Triage Decision Scheme removed from the criteria many of the measured values we examined in this study.² The revised criteria do not include initial speed, deformity, or rollover as indicators that a victim needs a trauma center. However, it did include intrusion distance and added the use of vehicle telemetry data consistent with a high risk of injury. The exact telemetry parameters that indicate a potential injury are not provided in the Decision Scheme.¹⁷ As more vehicles become equipped with Advanced Automatic Crash Notification devices that can transmit some of the crash mechanism information to dispatchers, it may not be necessary for EMS providers to estimate the amount of damage a vehicle sustained to identify patients who need the resources of a trauma center. However, at this time, use of this data by EMS providers is limited and is only available for newer model vehicles where the owner has subscribed to a paid service to use the device.^{18, 19}

We found that EMS provider’s estimations of vehicle speed were not an accurate proxy for the change in velocity. This was not unexpected since engineers have long stated that the two measures are not associated. The estimation of the change in velocity by conventional reconstruction techniques is known to be challenging due to the forces associated with various types of crashes, particularly side-swipe, roll-over and angled impacts.^{3, 4, 20} Event Data Recorders have their own limitations as they are often unable to record multiple impacts and may be limited in the data they are able to record.²⁰ This raises the question of whether or not the change in velocity as determined either by an Event Data Recorder located in the vehicle or a provider estimation of the change in velocity can be a reliable indicator of injury severity or trauma center need.²¹

Although our work suggests that EMS providers may have fair accuracy when determining intrusion greater than 12 inches, this paper does not address the accuracy of intrusion in predicting trauma center need. Future work looking at EMS providers’ ability to measure vehicle damage will need to consider if the estimates made by field providers are accurate in predicting trauma center need, regardless of whether they are an accurate measure of the damage to the vehicle. It may be that the inaccurate estimates correlate with potential for injury as well or better than the actual amount of vehicle damage. This needs to be determined so that the training implications of these findings can be better understood.

If provider estimates must be accurate to be used to determine the patient’s need for the resources of a trauma center, then based on our findings, it may be necessary to better educate EMS providers on how to make vehicle damage estimates. Our findings show that EMS-estimated intrusion would have led to 5% of patients who the criteria identified as needing a trauma center being under-triaged and 7% being over-triaged. Triage based on only the criteria for deformity would have resulted in the same amount of under-triage (5%) and an even higher level of over-triage, 27%.

It may be useful to develop tools for EMS providers to use to rapidly make vehicle damage estimations. The ability to visualize how the vehicle was shaped prior to the crash and how much the vehicle’s shape changed as a result of the crash is a skill that requires training and practice. It may be possible to develop better ways to train providers to make more accurate estimates than what was seen during this study. However, the usefulness of such training depends on whether the factors being measured actually indicate need for a trauma center.

Limitations

The major limitation of this study was its small sample size. This was the first study that was able to obtain both the vehicle owner consent for a professional crash investigation and an interview of EMS providers to obtain their estimates of the vehicle’s damage. However, because of the cost of the investigations and the difficulty with accessing vehicles we were only able to analyze 68 cases, limiting our statistical power. It is unlikely that the vehicles that were lost to follow-up were systematically different than those that were evaluated by the crash reconstructionist, nor is it likely this was a confounder for our results, since access to the vehicle should not have systematically affected the EMS providers’ estimates.

This study was not able to determine the degree of certainty EMS providers' had with their mechanism of injury estimates. This information would be useful in future studies since uncertain estimates may have been less likely to be accurate. We were not able to collect data on EMS provider demographics so we are not able to differentiate between estimates made by paramedics versus EMT-basics or providers with more experience compared to those with less. Further, because less severe crashes were excluded, this study cannot evaluate the ability of EMS providers to judge crash severity for minor- or moderate-severity crashes. It is also possible that by selecting severe crashes we may have biased the provider estimates by selecting those that were high estimates since by definition the provider had to have identified a severe crash. However, this was likely tempered by also selecting cases where the trauma team was activated regardless of the EMS identified severity of the crash.

The crash reconstructionist was not able to get to the crash scene to evaluate the damaged vehicle immediately after the crash. It is possible that the condition of the vehicle may have been altered during the towing or storing process. However, it is likely that those responsible for the vehicle during this process were careful not to significantly change the condition of vehicle, since crash damaged vehicles have a residual insurance value and care is typically taken during movement of the vehicle not to negatively affect that value. There may have been some minor cosmetic damage from moving the vehicle, but a crash reconstructionist can differentiate between crash damage and post-crash damage because the nature of the physical evidence is different. Further, because change in velocity was estimated based on the WinSMASH program it is possible that error in that estimate was responsible for the lack of correlation with EMS provider initial speed estimates, rather than a true lack of correlation. Many newer model vehicles collect change in velocity using event data recorders. Unfortunately, in our sample we were only able to acquire this data for three vehicles so it could not be used in our analysis.

Conclusion

We found that EMS providers are good at estimating rollover. Vehicle intrusion, deformity, and seatbelt use appear to be more difficult for EMS to estimate, since there was only fair agreement with the crash reconstructionist. As expected, the estimated speed prior to the crash does not appear to be a reasonable proxy for change in velocity.