The American College of Surgeons (ACS) Needs-Based Assessment of Trauma Systems (NBATS): Estimates for the State of California

Abstract

Background

In 2015, the American College of Surgeons Committee on Trauma convened a consensus conference to develop the Needs Based Assessment of Trauma Systems (NBATS) tool to assist in determining the number of trauma centers (TCs) required for a region. We tested the performance of NBATS with respect to the optimal number of TCs needed by region in California.

Methods

TC data was obtained from the California Emergency Services Authority (CEMSIS). Numbers of admitted trauma patients (ISS >15) were obtained using statewide non-public admissions data from the California Office of Statewide Health Planning and Development (OSHPD), CEMSIS, and data from Local Emergency Medical Service Agency (LEMSA) Directors who agreed to participate in a telephone survey. Population estimates per county for 2014 were obtained from the US Census. NBATS criteria used included population, transport time, community support, and number of discharges for severely injured patients (ISS >15) at non-TCs and TCs. Estimates for the number of trauma centers per region were created for each of the 3 data sources and compared to the number of existing centers.

Results

A total of 62 state-designated TCs were identified for California- 13 (21%) Level I, 36 (58%) Level II, 13 (11%) Level III. NBATS estimates for the total number of TCs in CA were 27–47% lower compared to the number of trauma centers in existence, but this varied based on urban/rural status. NBATS estimates were lower than current state in 70% of urban areas, but were higher almost 90% of rural areas. All data sources (OSHPD, CEMSIS, local data) produced similar results.

Conclusions

Estimates from the NBATS tool are different from what is currently in existence in California, and differences exist based on whether the region is rural or urban. Findings from the current study can help inform future iterations of the NBATS tool.

Background

Trauma systems aim to best match resources to population need. Challenges in the process of identifying the optimal number, distribution, and configuration of trauma centers have been described in the literature for over a decade.⁽¹⁾ Though the value of adding trauma center resources to areas that lack coverage may seem obvious, research has shown that adding a second trauma center in a stable region can double the cost of personnel and decrease the volume of injuries necessary for training and education.⁽²⁾ Therefore, careful consideration is warranted when considering whether a new trauma center should be established in a regional system.

Towards this end, the American College of Surgeons Committee on Trauma convened a consensus conference in 2015 to develop the Needs Based Assessment of Trauma Systems (NBATS) tool to provide estimates for the number of trauma centers (TCs) required for a region.⁽³⁾ This tool has not yet been evaluated in a practical setting. We sought to evaluate the performance of the NBATS tool in the state of California. We selected California as it can be viewed as a microcosm representative of the United States. California has a varied topography that includes mountain ranges (i.e. Sierra Nevada, Tehachapi Mountains), valleys (agriculture), and desert. It is the third largest state in size (158,706 square miles) and is the most populous (39,144,818 people in 2015), encompassing both large metropolitan areas and sparsely populated rural areas.⁽⁴⁾

Based on review of the NBATS tool in combination with our knowledge of the California trauma system, we hypothesized that estimates generated by NBATS would differ from what currently exists in the state. NBATS The tool does not specify the type of data required for determining estimates (i.e. administrative, registry, etc.), so we also compared data obtained from administrative sources, the statewide trauma registry, and from local emergency medical service agencies.

Methods

The Needs Based Assessment of Trauma Systems (NBATS) tool was obtained from the American College Surgeons website (https://www.facs.org/quality-programs/trauma/tscp/nbats). The tool was constructed to determine the number of trauma centers needed for a specific region, termed Trauma Service Area (TSA). It consists of 6 components including: (1) population size; (2) median transport times; (3) evidence of lead agency, system stakeholder, or community support; (4) the number of severely injured patients, defined by an injury severity score (ISS) of >15, discharged from acute care facilities not designated as Level I, II, or III trauma centers; (5) the number of Level I, II, and III trauma centers that already exist; and (6) the predicted number of patients with an ISS>15 vs. the actual number seen at Level I and II trauma centers.

California’s emergency medical services are administered through the authority of 33 Local Emergency Medical Service Agencies (LEMSAs). Some of these represent single counties, and others administer emergency services for more than one county. For the purposes of the current study, we considered the LEMSA to be the TSA. We focused the analysis on adult patients and adult trauma centers.

We used data from three different sources for the calendar year of 2014 to generate estimates from the NBATS tool: (1) statewide administrative discharge data from the California Office of Statewide Health Planning and Development (OSHPD);⁽⁵⁾ (2) statewide trauma registry data from the California Emergency Services Authority Information Systems (CEMSIS); (3) LEMSA representatives who agreed to participate in a telephone survey. Not all data sources could address all of the questions, so a set of model assumptions were made and sensitivity analyses were performed to account for the missing data (Table 1). Population estimates for 2014 were obtained from the US Census.⁽⁶⁾ For the purpose of this study, state trauma designation was used to identify Level I, III, and III trauma centers.

Table 1. NBATS Criteria and Assumptions used for NBATS Model for Missing Data.

Components and Assumptions used for the NBATS model. The six categories represent the components of the NBATS tool. In the case of missing data, points were assigned based on the table. Base case refers to the basic assumption, while the lower and upper bounds indicate how the value was ranged in sensitivity analyses.

		Assumptions
	Range of possible points	Base Case	Lower Range	Higher Range
1. Population size	2 to 10	N/A	N/A	N/A
2. Median Transport Times	0 to 4	0	0	4
3. Community Support	0 to 5	0	0	5
4. Number of patients with ISS>15 at non trauma centers (non Level I/II or III centers)	0 to 4	0	0	4
5. Number of Level I, II, or III trauma centers	0 to negative number based on multiplier	N/A	N/A	N/A
6. Estimated number ISS>15 patients based on number of trauma centers vs. actual number seen at Level I/II centers	−2 to 2	0	−2	2

ISS Injury Severity Score, N/A Not Applicable (no missing data)

For patient-level data, patients were included in the analysis if they were 18 years or older and had a discharge from an acute care hospital in California in 2014. For OSHPD data, injuries were identified if the there was a primary ICD-9-CM diagnosis code consistent with injury.⁽⁷⁾ ICD-9 diagnoses indicating injury include codes 800.00 to 959.0, excluding 905 to 909 (late effects of injury), 910 to 924 (blisters, contusions, abrasion, and insect bites), and 930 to 949 (foreign bodies). Injury Severity Scores (ISSs) were derived using the International Classification of Diseases Programs for Injury Categorization (ICDPIC) program.⁽⁸⁾ Patients with “severe injuries” were defined as having an ISS score >15. Urban and rural classifications were provided by the California Emergency Services Authority for each LEMSA. LEMSAs with a population greater than 1,000,000 were defined as urban; those with a population between 2,000,000 and 1,000,000 as suburban; and those with less than 2,000,000 as rural.

LEMSA directors were contacted to request his or her participation in a 30-minute telephone survey. Respondents were then scheduled for a telephone conference and the survey was conducted by two investigators (KLS, TUL). Quantitative and qualitative data was recorded by both separately, and verified by both investigators to confirm accurate documentation from the call.

Data manipulation and processing were done with SAS, 9.4 (SAS Institute, Cary NC) and STATA, 2014 (College Station, TX: Stator LP)

Results

Nineteen (58%) of the LEMSAS initially agreed to participate in our telephone survey, 15 of which followed through on the survey (45%). Three (9%) declined participation, and eleven (33%) did not respond after multiple attempts. Census data for 2014 showed the population of California to be approximately 38 million (Table 2). Of the 33 LEMSAs, the majority were urban or suburban (70%), while 30% were rural. There were 62 state-designated Level I, II, and III trauma centers throughout the state in 2014, of which 13 (21%) were Level I, 36 (58%) were Level II, and 13 (21%) were Level III.

Table 2. Characteristics of the California Trauma System.

Displays basic information on characteristics of the state and the trauma system.

		Number
Population		38,163,912
Total Square Miles		154,953
LEMSA Regions		33
Proportion LEMSAS by Urban/Rural	Urban	24%
	Suburban	45%
	Rural	30%
State Designated Adult Trauma Centers	Level I	13
	Level II	36
	Level III	13

We first evaluated the ability of each of the 3 data sources to address the individual NBATS components (Table 3). Data were available for all 3 models for population size, number of trauma centers, and number of patients with an ISS>15 treated at trauma centers. Median transport times for trauma patients and whether or not there was community support for a trauma center were only available from local agencies, while the number of patients with an ISS>15 at non-Level I, III, and III centers were mostly only obtainable from OSHPD data sources. The data source least likely to be able to complete the NBATS tool was the CEMSIS statewide trauma registry, as it lacked data on median transport times, community support, and the number of patients with an ISS>15 at non-trauma centers. LEMSAs were able to address most questions >87% of the time. The most notable challenge for LEMSAs was for having a knowledge of the number of patients with an ISS>15 who were seen at non-trauma centers, although 20% did have some information on non-trauma center admissions.

Table 3. Availability of Data by Sources.

Displays how often values were missing for specific NBATS components by data source.

	Percent of Data Available
	OSHPD	CEMSIS	LEMSA (only for participating regions)
Population size*	100%	100%	100%
Median Transport Times	0%	0%	87%
Community Support	0%	0%	100%
Number of patients with ISS>15 at non trauma centers (non Level I/II or III centers)	100%	6%	20%
Number of Level I, II, or III trauma centers*	100%	100%	100%
Estimated number ISS>15 patients based on number of trauma centers vs. actual number seen at Level I/II centers	100%	100%	100%

SHPD Office for Statewide Planning and Development; CEMSIS California Emergency Medical Services Information System; LEMSA Local Emergency Medical Service Agency

^*Source for data was census and state, so do not apply

Estimates for the number of Level I and II trauma centers needed for the state were generated by the NBATS tool for each of the 3 data sources (Table 4). Overall, estimates generated by NBATS were lower than the current number of trauma centers by 27–47%, depending on the data source. Estimates ranged widely in sensitivity analyses; with the CEMSIS model ranging the most, from 33 to 116. The greatest difference between the NBATS estimate and the current state was for the model that used LEMSA data (47% difference).

Table 4. NBATS Estimates vs. Current State.

Compares number of existing centers against NBATS estimates and LEMSA preference.

Data Source	Number Existing Level I/II Centers	LEMSA Preference Number Level I/II Centers	NBATS Estimates for Number of Level I/II Trauma Centers			Difference between Current State and Base Case		Difference between Current State and LEMSA Preference
			Base Case	Lower Est.	Upper Est.	Difference	% Current State	Difference	% Current State
OSHPD	49	N/A	36	36	92	13	27%
CEMSIS	49	N/A	33	33	116	16	33%
LEMSA*	32	33	17	17	27	15	47%	−1	6%

^*Values only for regions that participated in survey

NBATS Needs Based Assessment of Trauma Systems; OSHPD Office for Statewide Planning and Development; CEMSIS California Emergency Medical Services Information System; LEMSA Local Emergency Medical Service Agency; N/A Not Applicable; Est. Estimate

We also compared the number of trauma centers in 2014 with what LEMSA directors felt was needed for their regions. Almost all LEMSA administrators surveyed felt the current numbers of trauma centers in their regions were sufficient. Only one of the surveyed LEMSAs felt they could both use and support one additional trauma center.

While the overall number of trauma centers estimated to be needed by the state by NBATS was low compared to the actual number of trauma centers, estimates varied by LEMSA (Table 5). For 30–33% of LEMSAs, estimates were lower than what current exists, and in 40–49%, estimates were higher. NBATS estimates were the same as the number of existing trauma centers in 21–27% of LEMSAs. These differences were largely associated with the LEMSA’s urban/rural status (Figure 1). For rural LEMSAs, 88% of the NBATS estimates were higher than the current state, whereas for urban LEMSAs, 70% of the estimates were lower than the current state.

Table 5.

Over- and Underestimates for NBATS versus the Current State.

	% LEMSAs for which NBATS Estimate is Same as Current State	% LEMSAs for which NBATS Estimate is Higher than Current State	% LEMSAs for which NBATS Estimate is Lower than Current State
OSHPD	21.2%	48.5%	30.3%
CEMSIS	21.2%	45.5%	33.3%
LEMSA*	26.7%	40.0%	33.3%

NBATS Needs Based Assessment of Trauma Systems; OSHPD Office for Statewide Planning and Development; CEMSIS California Emergency Medical Services Information System; LEMSA Local Emergency Medical Service Agency

^*Values only for regions that participated in survey

FIGURE 1.

PERCENT OF LEMSAS FOR WHICH NBATS ESTIMATES ARE DIFFERENT THAN CURRENT STATE, BY URBAN/RURAL STATUS

Discussion

Overall, we found estimates generated by the NBATS tool to correlate with the number of existing trauma centers for only 20–30% of regions. NBATS estimates tended to be higher than what currently exists for rural areas, and lower for urban areas. Estimates also differed when compared to the opinions of LEMSA administrators. The overall number of trauma centers predicted by the tool was approximately ½ of what LEMSA administrators felt was required. While there is no gold standard to which estimates might be compared, it could be argued that local administrator opinion is the most informed. That said, the absence of a true gold standard makes calibration of such an instrument challenging. It may be that future iterations of a tool may benefit from the establishment of measurable pre-defined goals, such as trauma center access, under-triage rates, or trauma system capacity measures.

While we were only able to survey approximately ½ of the LEMSAs, it may reasonable to assume these findings can be extrapolated to all LEMSAs. For one, the urban/rural mix was relatively even across survey respondents (24% urban, 45% suburban, and 30% rural). We also found that the patterns described in the results section consistently held true with each LEMSA interview. While we cannot determine if the observed patterns would be found to be the same for non-participating LEMSAs, the consistency of the observations suggest it is more likely than not that the results are representative of LEMSAs across California.

In exploring why NBATS estimates were different than current state, we discovered differential performance of the model in urban versus rural settings. One of the model components deducts points for the presence of existing trauma centers. In the case of population-dense regions that already have a greater number of trauma centers, this deduction appears to results in lower estimates for the number of needed centers. Another issue related to population size and density is that California has widely varying populations for its urban areas. The point scoring systems “caps out” at a population size of 2.4 million, limiting regions that have larger populations than this. It may be that urban and rural regions would require different metrics, or that geography and population need to be considered together.

NBATS performance in rural areas highlights another issue. In some rural LEMSAs, there are only a few seriously injured patients over the course of a year. The NBATS tool has no threshold for the minimum number of injured patients to require a trauma center. When only a few patients may benefit from trauma center care, the question of value is raised. Many rural LEMSAs felt that the costs of supporting a Level I or Level II center were too high relative to the number of patients who would benefit from services. One particular challenge cited by rural LEMSA directors included maintaining surgical call panels, particularly when many rural regions struggle to get basic surgical services. Many directors felt that their population was more efficiently served through other mechanisms. These mechanisms included such things as well-coordinated pre-hospital emergency medical services (EMS) efforts, and agreements with nearby LEMSAs or states to use their trauma centers. When these mechanisms were not available, LEMSAs focused on enhancing the capabilities at their existing hospitals. While the tool was designed to address need, not capability, of a region to have a trauma center, the tension between these two factors has a significant impact on those involved in the process of establishing a new trauma center. Addressing these issues as part of the tool may facilitate efforts to establish a trauma centers in areas of need.

One of the NBATS survey elements caused estimates to vary in sensitivity analysis—whether there was support for a new trauma center by the community, stakeholders, or the lead agency. This was a question that could not be easily answered by most data sources. When discussing with LEMSA administrators, it was clear that this was a difficult question to answer beyond the perceived needs by the agency. Several LEMSAs commented that one could always get support for a trauma center from some members of the community. It is likely that not all community support is the same, nor is it likely all stakeholders have the same knowledge of what goes in to establishing a trauma center. The “community support” question may benefit from further refinements.

Another issue we sought to address is whether estimates are more dependent on the data source or if the tool produces precise estimates regardless of data source. For example, only administrative data can reliably address the number of severely injured patients at non-trauma centers, while only local regions know transport times. It appears that the tool is largely precise regardless of data source as all models produced values that were similar in direction and magnitude. This would suggest that the difference between NBATS estimates and current state are driven more by the nature of the tool than by the type of data selected for the analysis. While the data source did not strongly affect results of the tool, consideration should be given to the types of data available to those who hope to apply it.

There are several limitations to this study. The first is that we were not able to answer all questions in the NBATS tool given the data sources available. Presumably we could have created a “blended” model, but we felt this would detract from others who might test the tool but not have the same types of data. Another limitation is that not all LEMSAs participated in the phone survey. It may be that the LEMSAs who did not respond in fact were having issues with trauma center access and may have provided different responses. While this is possible, we qualitatively heard the same messages from LEMSA respondents, suggesting that pain points and NBATS perceptions were stable throughout the state. Despite these limitations, we feel the application of the tool across the widely different regions in California does provide relatively consistent results, particularly when considering urban and rural regions.

Conclusions

The tool performs similarly regardless of whether administrative, trauma registry, or local data are used. NBATS estimates are different than the number of trauma centers currently in existence, and differ from what local administrators feel is adequate for their areas. Results from the current study suggest that while the NBATS tool represents an initial step in developing an objective assessment of need for trauma center resources, it requires development and further study to perform adequately across the spectrum of potential regions to be served.