The new injury severity score underestimates true injury severity in a resource-constrained setting

African relevance

• •This study is one of the first to evaluate the performance of the new Injury Severity Score (NISS) in an African context using an expert panel as the referential standard.
• •NISS significantly underestimated trauma severity among patients evaluated at a tertiary trauma center in South Africa (average 11-point difference) with a greater degree of underestimation for penetrating trauma.
• •South African physician experts assigned higher severity for anatomic injuries that required emergent stabilization, consumed limited resources such as cross-sectional imaging or specialty care, or placed the patient at risk for complications and disability.
• •Future research should evaluate performance of NISS and other injury severity scoring tools in resource-constrained settings using large cohorts across multiple centers with similar resource capability.

Abstract

Background

The new injury severity score (NISS) is widely used within trauma outcomes research. NISS is a composite anatomic severity score derived from the Abbreviated Injury Scale (AIS) protocol. It has been postulated that NISS underestimates trauma severity in resource-constrained settings, which may contribute to erroneous research conclusions. We formally compare NISS to an expert panel's assessment of injury severity in South Africa.

Methods

This was a retrospective chart review of adult trauma patients seen in a tertiary trauma center. Randomly selected medical records were reviewed by an AIS-certified rater who assigned an AIS severity score for each anatomic injury. A panel of five South African trauma experts independently reviewed the same charts and assigned consensus severity scores using a similar scale for comparability. NISS was calculated as the sum of the squares of the three highest assigned severity scores per patient. The difference in average NISS between rater and expert panel was assessed using a multivariable linear mixed effects regression adjusted for patient demographics, injury mechanism and type.

Results

Of 49 patients with 190 anatomic injuries, the majority were male (n = 38), the average age was 36 (range 18–80), with either a penetrating (n = 23) or blunt (n = 26) injury, resulting in 4 deaths. Mean NISS was 16 (SD 15) for the AIS rater compared to 28 (SD 20) for the expert panel. Adjusted for potential confounders, AIS rater NISS was on average 11 points (95 % CI: 7, 15) lower than the expert panel NISS (p < 0.001). Injury type was an effect modifier, with the difference between the AIS rater and expert panel being greater in penetrating versus blunt injury (16 vs. 7; p = 0.04). Crush injury was not well-captured by AIS protocol.

Conclusion

NISS may under-estimate the ‘true’ injury severity in a middle-income country trauma hospital, particularly for patients with penetrating injury.

Introduction

Trauma is a leading cause of morbidity and mortality globally, predominantly among young adults. In 2019, 714 million people sustained injuries that required medical attention and over 4 million died from their trauma, representing 7.6 % of all deaths[1]. Approximately 80–90 % of trauma-related deaths occur in low- and middle-income countries (LMICs), and a large portion of these deaths are the result of road traffic collisions, interpersonal violence, and self-harm[2], [3], [4]. The injured in resource-constrained environments face many obstacles to access urgent care, with delays to treatment leading to hemorrhagic shock and other potentially lethal complications[2]. While LMICs bear the brunt of the global burden of trauma morbidity and mortality, there is no consensus regarding which injury severity rating tools perform best in these contexts. In fact, studies consistently conclude that many scoring systems underpredict mortality in these settings, without measuring or quantifying the degree of underestimation[5].

Standardized trauma scoring tools are important for the study of trauma outcomes, both within and across different health systems. Many of the tools currently in use were developed in high income settings, derived from trauma registry data and/or expert opinion[6]. The Abbreviated Injury Scale (AIS) system, for example, originated in the United States in the mid-1960s for tracking injury from automotive and airplane crashes, but has since expanded into an internationally recognized scoring system for traumatic injuries of all types. Expert opinion has been used heavily in developing iterative versions of AIS, which serves as the backbone for the Injury Severity Score (ISS) and other scoring systems[7].

Historically, the ISS and the updated New Injury Severity Score (NISS) have served as the de-facto “gold standard” for injury severity measurement, including in LMICs[8], [9], [10], [11]. ISS and NISS quantify overall body injury severity in polytrauma patients and enable comparison across populations with divergent injury profiles[12]. NISS was found to be simpler to calculate and more predictive of survival[11]. The benefit of these anatomic scoring systems is that the score remains fixed from the time of injury and can therefore be calculated retrospectively, as opposed to physiologic scoring systems which rely on vital sign parameters that fluctuate over time[13,14]. Criticism of anatomic scoring systems is that they rely on cross-sectional imaging, operative and post-mortem reports which may not be readily or fully available in LMICs[8,9].

Trauma experts have expressed concern that ISS and NISS may underpredict true trauma severity in LMICs where there is poor availability of relevant anatomic injury information and associated documentation[5,8,15]. In a systematic review, Mehmood et al. found ISS to have wide variability for mortality prediction among 11 validation studies conducted in LMICs[8]. However, the observation that ISS and NISS underpredict true injury severity in LMICs has never been formally tested nor quantified. Given the frequent use of the contemporary NISS in LMIC trauma research, and the threat posed by generating incorrect research conclusions, the application of NISS in LMICs requires formal exploration.

Here, we present one of the first primary studies to specifically assess the performance of NISS in a LMIC context. We use expert consensus as the location-specific gold standard for injury severity assignment, mirroring the process by which AIS/NISS were first developed. We have identified South Africa as the ideal location to study NISS performance, in part because it has among the highest injury severity and incidence globally[16,17]. Homicide rates in South Africa are seven times the global average, and injury-related mortality accounts for 12 % of deaths in this country[18,19]. In this report, we provide data to quantify the degree to which NISS underpredicts injury severity in LMICs, and we elucidate contributory reasons.

Methods

New injury severity score and abbreviated injury scale overview

The NISS, a measure of overall injury severity, is a better predictor of post-injury mortality and multi-organ failure compared to the ISS, which is why we chose to study this particular injury severity rating tool[20], [21], [22]. The NISS is calculated as a composite score based on the Abbreviated Injury Scale (AIS)[23]. In the AIS methodology, each anatomic injury is ascribed a pre-specified body region code along with an anatomic body region injury severity score (the “AIS score”), ranging from 1 (minor) to 6 (maximal)[24]. The NISS is calculated as the sum of the squares of the three highest AIS scores, regardless of body region, with a range from 0 to 75[11,25]. Fig. 1 illustrates how the AIS body region code and the AIS score together form the 7-digit AIS code for each anatomic injury.

Fig. 1.

The 7-digit AIS code[7].

The first 6 digits represent the body region code. The last digit, sometimes called the post-dot code, represents the AIS body region injury severity score (AIS score). The AIS score has a 6-point scale[1 ]^to[6]. 6 often represents a non-survivable anatomic injury.

Study aims

Overall, we compare AIS and NISS values scored by a panel of clinical experts versus a certified AIS rater (i.e., an AIS classification expert) using trauma patients’ medical records in South Africa. The primary aim is to assess the degree of difference in the NISS given by the AIS rater versus the expert panel. The secondary aim is to assess the degree of difference in anatomic severity score assigned by the local expert panel compared to the AIS rater for each body region, including identifying which body areas may have the largest differences. Last, we describe the reasons provided by experts for discrepant severity scores between expert panel and AIS rater to better understand the reasons why NISS/AIS may perform differently in lower income settings.

Study design

This was a retrospective chart review of medical records from March 22, 2021 to August 23, 2021 collected during the pilot phase of “Epidemiology and Outcomes of Combat-Relevant Prolonged Trauma Care” study (EpiC), an observational epidemiologic trauma study in the Western Cape Province of South Africa from 2020 to 2024[26]. Ethics approvals have been obtained from Stellenbosch University. The ethics committee waived the need for informed consent, consistent with other trauma studies, given the seriousness and time sensitivity of the clinical condition of trauma patients which makes it unfeasible to obtain consent at the time of hospitalization. Given the observational nature of this study, waiver of consent posed minimal risk to patients. Adult trauma patients (≥18 years) seen primarily or transferred to the tertiary referral center were eligible for inclusion. Similar to the EpiC study, we excluded prisoners, children, pregnant women, patients who were dead on scene prior to EMS arrival, and patients with minor trauma, defined as a triage color of green, which signifies the lowest acuity, on the South African Triage Scale[27].

We randomly selected 60 patient cases from the EpiC database using a balancing algorithm to achieve a representative distribution across the categories of age, sex, number of anatomic injuries, injury mechanism and severity[28]. Paper medical records were obtained for these 60 cases and reviewed for completeness. Cases with missing medical records and cases with no codable anatomic injuries (e.g., patient with post-traumatic knee pain with negative radiographs and no objective anatomic injury by exam) were excluded. Study authors required access to information that could identify individual patients during the initial case selection only (performed October 6 to 15, 2021). Records of included cases were subsequently deidentified and collated before distribution to study participants. No identifiable information was used during the panel discussion or in any subsequent data analysis.

Study participants included five expert panelists and one AIS rater. The AIS rater was a South African physician, employed by the EpiC study, who has completed the standard AIS certificate course and performs AIS coding on a daily basis. The five expert panelists were South African physicians specialized in emergency medicine, trauma surgery, orthopedics and neurosurgery. These experts were selected specifically because of their depth and breadth of experience caring for traumatic injuries of all major anatomic regions. None of the experts were previously familiar with the AIS scoring protocol. Rater and panel were blinded to each other's severity score assignments.

Data collection

In the week prior to the expert panel, the AIS rater reviewed each case independently, assigned 7-digit AIS codes for each anatomic injury per AIS protocol using the AIS Manual 2015 Revision, and compiled a separate simplified list of all identified anatomic injuries using non-AIS clinical terminology (e.g., femur fracture)[23]. Over two days, the experts reviewed these same cases as a panel, taking turns serving as the primary discussant for each case. They reviewed and modified the corresponding list of anatomic injuries (created by the AIS rater) for each case. They subsequently used expert consensus to assign severity scores for each anatomic injury, taking into account all available patient data and their own local expertise on usual care and outcomes associated with particular injuries. The expert severity scores were on a scale of 1 to 6, similar to the AIS scale, for calibration and comparability. Anchor definitions of severity were developed by the expert panelists for each of the 6 levels to help provide internal consistency before the start of the case review. Panelists’ explanations for severity ratings were documented throughout the review process.

Power and sample size

The sample size of 60 individuals was established based on the feasible number of cases for performing the expert panel review and the expectation that 20 % of cases would be excluded due to missing medical records. Given an analytic data set of 48 complete cases, using a paired t-test with 80 % power (alpha = 0.05, two-sided) we can detect a standardized effect size of 0.41, which corresponds to a small effect size[29]. Assuming the standard deviation for NISS is 10, this corresponds to a difference of 4.1 points. We hypothesized that the AIS rater's mean ISS would be 5-points lower than panel NISS, which would be a clinically meaningful difference demonstrating that NISS under-estimates injury severity in a lower income setting. Our study is therefore adequately powered to test our hypothesis.

Statistical analysis

Descriptive statistics were computed for the NISS outcome and for other key variables (hospital, injury type, sex, age group). The primary analysis examined the difference in NISS by AIS rater versus expert panel. To assess the mean difference in AIS rater and panel NISS, we fit a mixed effects linear regression for the NISS outcome with fixed effects for the evaluator type (AIS rater vs. expert panel) and a random effect for the patient case. An exploratory analysis was performed to assess for effect modification by patient characteristics (injury type, injury mechanism, age, gender) by including an interaction between the variable and the evaluator type. Secondary analysis was performed to explore the difference in AIS scores between the AIS rater and expert panel by body region. To assess the mean difference in AIS rater and panel AIS by body region, we fit a similar mixed effects model for the AIS outcome with an interaction between evaluator type and body region (with 9 categories for the different AIS body regions). To assess the extent to which there is disagreement between the rater and panel AIS, we additionally estimated the mean absolute difference by body region using a linear regression model for absolute AIS difference. All hypothesis tests were two-sided with alpha = 0.05. Statistical analyses were performed using R (version 4.0.4).

The third aim involved descriptive analysis. The iterative versions of AIS were developed using expert consensus, and severity was assigned based on several ‘dimensions of severity’ from the AIS manual, such as threat to life, permanent impairment, treatment period, energy dissipation, etcetera[30]. The expert panelists were provided with the AIS ‘dimensions of severity’ list and were asked to assign one or more dimensions for each case. The experts were also asked to provide reasons for any injury reclassifications that were made and to comment on the cases they found challenging to score.

Results

Of 236 eligible cases, there were 234 with available demographic and injury information from which 60 cases were sampled using the balancing algorithm. Chart review of these 60 cases revealed 49 complete cases and 11 cases that were excluded due to missing records, incomplete evaluation, or minor injury (Fig. 2).

Fig. 2.

Diagram depicting case selection process.

Of the 49 included complete cases, the majority were male (n = 38), the average age was 36 (range 18–80), and there were 4 deaths (Table 1). There was a near even split between penetrating (n = 23) and blunt injury (n = 26), and between patients transferred from a regional/district hospital (n = 25) and those seen primarily at the tertiary center (n = 24). The AIS rater assigned 190 AIS scores across all cases. The expert panel reclassified 27 of those anatomic injuries and assigned a total of 189 expert anatomic severity scores across all cases.

Table 1.

Demographics of included complete cases (n = 49).

Characteristic		n/mean (SD)
Sex, n	Male	38
	Female	11
Age, mean (SD)		35.5 (13.8)
Injury Force Type, n	Blunt	26
	Penetrating	23
Dominant Mechanism of Injury, n	Fall or Struck/hit	12
	Firearm	13
	Stabbing or cut	9
	Vehicular Injury	15
Initial Receiving Facility, n	Tygerberg (Tertiary Hospital)	24
	Khayelitsha (District Hospital)	22
	Worcester (Regional Hospital)	2
	Ceres (District Hospital)	1
Identified anatomic injuries per case, mean (SD)	AIS Rater	3.9 (2.7)
	Expert panel	3.9 (2.6)
Mortality, n		4

For the primary outcome of differences in NISS between AIS rater and expert panel, mean rater NISS was 16.3 (SD 14.7) and mean expert NISS was 27.5 (SD 19.7). From the linear mixed effects regression, there was a statistically significant 11.2-point difference in the mean scores (95 % CI: 7.1, 15.3; p < 0.001). The variability of the differences appeared to increase with increasing expert NISS (Fig. 3). Injury type was an effect modifier of the relationship between evaluator type and NISS outcome, with the difference between the rater and panel NISS being greater among those with penetrating injury compared to those with blunt injury (difference = 15.7 points for penetrating vs. 7.3 for blunt; p = 0.04). Patient age, sex, and mechanism of injury did not modify the effect.

Fig. 3.

Scatterplot for rater and expert panel NISS.

Diagonal line indicates perfect agreement between scores.

For the secondary objective of assessing the difference in anatomic severity scores for each body region, the expert panel anatomic severity score was higher for most body regions compared to the AIS rater (Fig. 4). The mean difference in anatomic severity scores between the panel and rater was greatest for the external body region, for which on average the expert panel score was 2.5 points higher than the rater score (95 % CI: 1.4–3.7; p < 0.001), with significant differences also in neck, face, spine, lower extremity, and thorax (Table 2). When examining the mean absolute difference, we find similar results with the addition of statistical significance for the head region, where a higher number of rater scores (n = 5) exceeded the panel scores compared to the other body regions.

Fig. 4.

Plot of difference between anatomic severity score assigned by rater and expert panel by body region.

Size of point indicates the count of individuals; horizontal line indicates perfect agreement between scores; dots above line indicate that panel assigned a higher score compared to rater; dots below line indicate that panel assigned lower score compared to rater.

Table 2.

Difference in anatomic severity score (mean and absolute) by body region.

	Mean difference in anatomic severity score*				Mean absolute difference in anatomic severity score¶
Body region	# Scores available for AIS rater^	#Scores available for panel	Mean difference panel – rater (95 % CI)	p-value	# Scores available for both rater and panel	Mean absolute difference |panel – rater (95 % CI)	p-value
External	10	7	2.52 (1.36, 3.67)	<0.001	7	2.43 (1.83, 3.03)	<0.001
Neck	5	4	1.57 (0.26, 2.89)	0.02	4	1.50 (0.70, 2.30)	<0.001
Face	33	33	1.04 (0.59, 1.50)	<0.001	33	1.03 (0.75, 1.31)	<0.001
Spine	15	15	1.00 (0.33, 1.67)	0.003	15	1.00 (0.59, 1.41)	<0.001
Lower extremity	34	32	0.82 (0.37, 1.28)	<0.001	32	0.84 (0.56, 1.13)	<0.001
Thorax	32	31	0.78 (0.32, 1.25)	0.001	31	0.77 (0.49, 1.06)	<0.001
Upper extremity	22	14	0.50 (−0.15, 1.15)	0.13	14	0.43 (0.002, 0.85)	0.05
Head	51	44	0.15 (−0.25, 0.54)	0.47	39	0.41 (0.15, 0.67)	0.002
Abdomen	9	9	0.13 (−0.76, 1.02)	0.78	9	0.33 (−0.20, 0.86)	0.22

^⁎Results from linear mixed effects model for anatomic severity score outcome.

^¶Results from linear regression for absolute difference in anatomic severity score outcome.

^{^}A total of 211 AIS scores were available, including 190 from original AIS rater codes, and 21 recodes that were done by the AIS rater using the new anatomic injury descriptors identified by the expert panel.

Descriptive findings

The third aim was to describe the expert panel's thought process, specifically exploring reasons for injury reclassification, rationale behind severity score assignments, and challenges encountered when scoring cases.

Injury reclassification

The expert panelists disagreed with several of the anatomic injuries listed by the AIS rater, deleting (9 codes), consolidating (18 to 6 codes), and generating new descriptors (26 codes). There were four major reasons provided by the panel explaining why experts disagreed with the AIS rater's list of injuries. First, clinical expertise helped the experts “read between the lines” when reviewing the medical record. There were many diagnoses that were not formally documented, but that were obvious to the panelists based on the mechanism, exam, and treatments administered. For example, in one case, a patient sustained a gunshot wound that traversed from the chest into the abdomen and experts concluded that there was a penetrating injury through the diaphragm that had not been recorded. Second, experts identified errors within the medical record that led to inaccurate diagnoses. For example, an ankle radiograph was misinterpreted in the medical record. Another common reason for injury reclassification was expert preference for a unifying diagnosis. In severe injuries, they preferred umbrella terms such as “diffuse traumatic brain injury” instead of the granular approach ascribed by AIS methodology. Lastly, the experts introduced several codes (new codes were generated for 6 cases including for 2 out of 4 deaths) to better capture crush injury, a common cause of morbidity in South Africa.

Dimensions of severity

Experts listed several dimensions of severity when determining the anatomic injury severity score (see Table 3). Dimensions of severity fell into three major categories. The first category was the acute risk to the patient and need for emergent stabilization. Experts considered the general mortality risk, along with the risks of airway compromise, and respiratory and circulatory system failure. The second category was anticipated healthcare resource consumption, including the need for observation services, advanced imaging, hospital transfer, specialist intervention, and follow up care. The final category was the long-term impact to the patient, which included risk of delayed complications and disability.

Table 3.

Dimensions of severity identified by expert panel.

Dimension of Severity	Description	Cases
Resource consumption	Need for transfer to higher level of care, subspecialty services, utilization of CT/MRI, follow ups, etc.	21
Other complication risk	Potential for noninfectious/nonbleeding complication	18
Disability	Short- or long-term disability and potential loss of employment/income due to injury	14
Observation	Need for period of emergency department or inpatient monitoring	12
Infectious complication risk	Nature of injury predisposes to the development of infection	11
Acute stabilization of circulatory system	Acute bleeding or shock state that requires rapid treatment for hemorrhage control	10
Operation	Need for operative intervention	9
Energy transfer	Mechanism of injury associated with large force generated	9
Mortality risk	Injury with significant risk of death due to the physiological derangements associated	8
Airway or respiratory compromise	Need for oxygen support, airway monitoring or intubation	5
Bleeding complication risk	Injury associated with risk of bleeding or subacute need for blood products	6

Dimensions that contributed to the severity scores assignments are presented, listed in order of frequency mentioned.

Challenging cases

Experts found it challenging to assign severity scores in a minority of cases (n = 10). They expressed uncertainty due to missing elements of the clinical history/physical exam (n = 2), diagnostic tests (EKG, labs, imaging) (n = 5), operative and forensic pathology reports (n = 2), and patient care timeline (n = 2). Experts also expressed that photographs of skin and soft tissue injuries would have been helpful to better assess the degree of tissue destruction in cases of suspected crush injury and open fracture (n = 4). Clear documentation of the patient care timeline, with timestamps included for time of injury, time of presentation to the hospital, time of administered interventions and vital sign response, was felt important to gauge the severity of certain anatomic injuries. For example, experts explained that in the case of traumatic brain injury, patients may develop cerebral herniation early in the case of a severe intracranial injury due to rapid intracranial hemorrhage, or several hours later in the setting of relatively minor injury, due to the cerebral edema that develops when there is delayed initiation of treatment. Here, the same poor outcome may occur due to injuries of different anatomic severity. Missing data made it difficult to gauge anatomic injury severity in such cases.

Discussion

This study is one of the first to formally evaluate the performance of AIS and NISS in a LMIC context. NISS significantly underestimated injury severity among patients evaluated in a tertiary trauma center in South Africa, with a difference of 11 points between expert NISS and protocol NISS. An expert panel of South African physicians served as the location-specific gold standard, mirroring the process by which AIS was originally developed, lending validity to this methodological approach[7]. The findings from this study quantify and further support concerns from the trauma research community that ISS and NISS do not accurately capture injury severity in LMICs[8,9].

The variability of the differences appeared to increase with increasing expert NISS. This may be due to the exponential nature of NISS, in which the three highest severity AIS scores are squared and then summed. NISS performed particularly poorly for cases of penetrating trauma. NISS underestimated injury severity by an average of 16 points for penetrating injury, compared to 7 points for blunt. Reasons for this discrepancy may be inherent to AIS/NISS, or perhaps context specific. In the Major Trauma Outcomes Study, North American patients with penetrating injury had higher mortality rates in most ISS categories when compared with blunt trauma[31]. A more recent study by Rowell et al. found the same pattern among transfused trauma patients in the United States with severe ISS[32]. It could be that AIS and NISS are sub-optimally calibrated for penetrating injury. Alternatively, the discrepancy may be specific to the South African context. Within our cohort, patients with blunt trauma represented a mix of accidental and intentional injury, whereas >80 % of patients with penetrating trauma were victims of intentional interpersonal violence. Given the high burden of disease in South Africa from interpersonal violence, there may have been greater perceived morbidity and resource utilization for the penetrating subgroup of patients, which likely contributed to higher severity ratings by the experts for this population.

Local experts in this study assigned higher severity scores for injuries that consumed scarce healthcare resources, such as blood products, cross-sectional imaging, and subspecialty care. Experts also assigned higher severity for injuries which posed an increased risk of complications or protracted morbidity. They described challenges associated with wound care, infection prevention, injury rehabilitation, and loss to follow up, along with the financial and social impacts of injury commonly faced by their patients. In the United States, iterations of AIS have anchored on dimensions of severity, examples of which include threat to life, length of stay, need for intensive care, treatment complexity, treatment cost, and disability[33]. Such dimensions vary drastically based on the socioeconomic and healthcare practice environment. It is, therefore, unsurprising that physicians in the South African health system perceived injury severity differently.

Trauma researchers have postulated that anatomic injury severity scoring tools have reduced validity in LMICs, which may have limited diagnostic capabilities and documentation[9]. When calculating ISS/NISS retrospectively using the medical record, information from imaging reports, operative records, and autopsy reports can be critical for injury characterization. When such information is missing, critics argue that it can be difficult to differentiate between severe injury and sub-optimally managed minor injury[34]. These problems did come up during the expert panel process, but in somewhat unexpected ways. The South African health system relies on junior doctors to maintain the written medical record. In our study, we found that these less experienced physicians sometimes misinterpreted clinical data or failed to mention specific diagnoses (e.g., concussion). The experts used their clinical expertise to identify, rename or consolidate injuries that a verbatim review of the medical record failed to capture[5]. This led to many revisions to the list of injuries that were originally developed by the AIS rater. Experts identified only a minority of cases (n = 10) with missing data that directly posed a challenge to severity classification. The unanticipated phenomenon of injury reclassification by experts within our study highlights another limitation of AIS and NISS in LMICs: anatomic scoring tools rely on detailed documentation in the medical record, and the coder is not permitted to make such intuitive leaps in rendering missed or ambiguous diagnosis. Inaccuracies within the written medical record can predispose to errors, not only in the assignment of injury severity, but also in the accurate identification of the injuries themselves. We postulate that there is greater missingness of injury surveillance data in health systems that rely on paper medical records, and that this challenge is not unique to Africa. Zargaran et al. evaluated the effect of implementing electronic injury surveillance forms at Groote Schuur Hospital in Cape Town in 2018; they found significantly higher rates of completion of all studied fields, including complications and missed injuries, when compared to paper records[35]. Further studies are warranted to determine whether accuracy of AIS/NISS differs when using electronic versus paper medical records.

The study also revealed interesting trends by AIS body region. Experts assigned significantly higher anatomic severity scores for all body regions except for head and abdomen. The largest differences were for the neck (mean difference of 1.6) and external body regions (mean difference of 2.5). Penetrating neck wounds received higher expert scores given proximity to critical structures and the likelihood of airway compromise. Injuries to the external body region, which includes skin and soft tissues, received higher expert severity scores due to concerns for crush syndrome. In South Africa and other LMICs, community assault, or mob justice, constitutes a common mechanism of interpersonal violence. Individuals accused of wrongdoing undergo severe beatings by multiple assailants, often with a combination of fists and whips[36,37]. Patients injured in this manner sustain severe soft tissue injury to regions such as the back, buttocks, and extremities, and often present with linear bruises described as ‘tram tracks.’ Many develop severe rhabdomyolysis, acute kidney injury, multi-organ failure, and frequent deaths[37,38]. Findings from this study suggest that the AIS/NISS system does not adequately capture crush injury. The codes for soft tissue injury, such as laceration, abrasion, and contusion, are universally considered low severity by the AIS protocol. These codes appear to lack the required level of specificity, as they do not incorporate total body surface area affected or association with end-organ damage.

Findings from this study emphasize the need to tailor anatomic injury severity scoring tools based on data from regional trauma databases[39]. A future approach could be to introduce weighted coefficients to modify NISS and other similar scoring tools for a particular geographical region. A future study will be required to compare predictive ability of such a weighted score to the standard score, using patient-centered outcomes such as disposition, ICU admission, or mortality. This effort would likely improve the calibration of scoring tools to the injury profiles seen regionally, and further augment local efforts to assess and improve LMIC trauma care systems. However, we recognize that this tailored approach may compromise generalizability, particularly when attempting to compare injury severity across vastly different geographic and socioeconomic areas. Physiology-based scoring systems are a compelling adjunct; they may be more appropriate in environments where delays to care predispose to worse outcomes. Simplified scoring systems such as the Kampala Trauma Score, MGAP (Mechanism/Glasgow Coma Scale/Age/Pressure) and GAP (Glasgow Coma Scale/Age/Pressure) may be more feasible to calculate in resource-constrained settings with similar performance characteristics[40], [41], [42].

This study had several limitations. First, this study took place at a single center in a middle-income country and utilized expert opinion. It is possible that the same discrepancy in severity assignment between AIS rater and expert panel could be found if this study were conducted in a resource-rich setting. Therefore, a large multicenter study using both expert opinion and objective measures of injury severity (e.g., ICU admission, mortality rate), conducted in both resource-constrained locations and resource-rich locations, is needed to draw definitive conclusions about whether AIS/NISS performance varies by geography and/or resource availability. It is, however, important to note that AIS was originally derived based on expert opinion, so the presented methodology has a historic basis. Second, we excluded cases of minor trauma, and we suspect there would have likely been greater agreement between panel and rater on anatomic severity scores for minor injury. Third, we only had one AIS rater review the cases due to time and resource constraints; however, use of multiple raters is unlikely to have changed the rater score assignments given the protocolized nature of the work. Another limitation was that the expert panel was composed of senior physicians working at an academic hospital that receives a large volume of trauma. This may limit generalizability of study findings to hospitals with smaller volumes. Finally, we were only able to report on descriptive findings when exploring the reasons behind discrepant severity ratings. A more formal qualitative study may have unveiled richer themes.

Conclusion

This retrospective single-center study found that NISS significantly underestimates injury severity among a cohort of trauma patients in South Africa when compared to local physician expertise, particularly for penetrating trauma. Physicians frequently considered need for acute resuscitation, healthcare resource utilization, long-term complications, and disability into their severity score assignments. These findings require further corroboration with large cohorts across both well-resourced and under-resourced settings to draw definitive conclusions. Importantly, if definitive evidence of NISS underscoring is found, additional research will be required to determine whether such underscoring impacts prediction of patient-centered outcomes such as disposition, ICU admission and mortality. Ultimately, this research raises the question of whether a corrective measure must be developed to modify AIS and NISS for application in resource-constrained environments.

Disclaimer

Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the United States Department of Defense, Stellenbosch University, Western Cape Government Department of Health and Wellness, and the University of Colorado.

Authors’ contribution

Authors contributed as follow to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting the work or revising it critically for important intellectual content: SB contributed 40 %; NM and KS 20 %; HL and ES contributed 10 % each. All authors approved the version to be published and agreed to be accountable for all aspects of the work.

Dissemination of results

Results from this study were shared with departmental leadership at this single center hospital through an informal presentation and circulated as a memo.

Declaration of Competing Interest

The authors declared no conflicts of interest.