Retention, Learning by Doing, and Performance in Emergency Medical Services

Abstract

Objectives

To examine the strength of the volume–outcome relationship among paramedics, a group of providers that has not been previously studied in this context. By identifying the effects of individual learning on performance, we also assess the value of paramedics' retention. The prehospital emergency medical services (EMS) setting allows us to interpret any volume–outcome relationship as learning by doing, uncontaminated by reputation-based referrals because ambulance units are dispatched based on proximity.

Data Sources

Incident-level EMS data spanning 1991 to 2005 from the Mississippi Emergency Medical Services Information System collected by the Mississippi Department of Health.

Research Design

Using linear and quantile methods with and without provider fixed effects, we estimate the relationship between experience accumulation and performance using the universe of trauma incidents involving injured patients (including motor vehicle crashes, falls, stabbings, and shootings).

Principal Findings

We find that greater individual volume is robustly related to improved performance. In addition, we find that the benefit of learning operates through both recent and past experiences, accrues differentially across tenure groups, and operates on both mean performance and the upper quantiles of the performance distribution.

Conclusions

Persistent past and current volume effects suggest that policy and managerial implications in EMS should be directed at retention efforts to take advantage of individual learning by paramedics.

While there is considerable evidence in the case of hospitals and surgeons that high volume is associated with better patient outcomes across a variety of medical conditions (Luft, Hunt, and Maerki 1987; Tu, Austin, and Chan 2001; Birkmeyer et al. 2002, 2003; Halm, Lee, and Chassin 2002; Elixhauser, Steiner, and Fraser 2003; Gandjour, Bannenberg, and Lauterbach 2003; Huckman and Pisano 2006; Marcin et al. 2007), to date no study has examined this relationship in a prehospital setting.

Emergency medical services (EMS) networks respond to, stabilize, and transport trauma patients involved in situations, such as automobile accidents, injuries from falls, and violence. EMS is a crucial part of the health care system and the public's emergency medical safety net. However, more than three decades after Congress passed the Emergency Medical Services Systems Act of 1973, trauma victims are treated by EMS with little or no evidence that the care they receive is optimal (Delbridge et al. 1998).

There is little research addressing the effectiveness of EMS practice (Pointer 2001) and there are even fewer studies examining the effect of experience accumulation, as proxied either by certification levels (Pollock, Brown, and Dunn 1997) or tenure (Soo et al. 1999) on performance. Our study is the first to use detailed data on experience accumulation in addition to certification levels and tenure to document the performance gains from higher volume.

Learning by doing is one channel through which skills are acquired and therefore provides a potential rationale for the observed positive correlation between volume and outcome. Under this mechanism, more acquired experience improves performance. Nevertheless, the causality may run in the opposite direction, through a selective referrals channel, whereby better performers command greater demand for their services (Luft, Hunt, and Maerki 1987).When using retrospective data to study volume–outcome relations for surgeons, the selective referrals problem is difficult to overcome credibly. Unlike settings which involve the choice of provider (e.g., elective surgery), the unpredictable nature of EMS does not lend itself to selective referrals. Ambulance units are dispatched based on proximity and not on reputation, and trauma victims are not offered the choice of emergency care providers on scene.

In this paper, we focus on trauma-related ambulance runs to study the effect of volume and experience accumulation on performance. The evidence regarding the appropriateness of prehospital interventions on trauma victims is mixed. However, the importance of getting the patient to definitive care as soon as possible, allowing only for the performance of essential procedures, is widely accepted in EMS and prehospital time markers are frequently used process measures. Shorter out-of-hospital time intervals represent a potentially important factor in survival, especially in the case of trauma incidents (Sampalis et al. 1993; Feero et al. 1995; Nichol et al. 1996; IOM 2000; Blackwell and Kaufman 2002; Carr et al. 2006). For trauma patients, the first hour of care (also referred to as the “golden hour”) is usually considered critical with some evidence that when the time from the incident to hospital treatment is within this critical first hour, the patient's likelihood of survival is increased (Rawlinson and Crews 2003). In trauma care, the goal is to get seriously injured patients into the operating room of a trauma center with an experienced team of appropriately specialized trauma surgeons as fast as possible. Therefore, it is not surprising that dispatching emphasizes ambulance proximity and EMS contracts often include response standards.

We hypothesize that paramedic learning will lead to shorter out-of-hospital times. In particular, as paramedics become more proficient in diagnosing patients' acuity, identifying the appropriate procedures, mastering protocols and techniques, gaining familiarity with local roads and traffic conditions, and identifying quicker routes to scene and to the hospital, the shorter is the time spent out of hospital, and, in particular, the time spent at the scene.

Using the universe of trauma-related ambulance runs in Mississippi between 1991 and 2005, we find that greater volume is robustly related to improved paramedic performance. In specifications that include both individual- and firm-level measures of recent volume, we find small productivity spillovers. In addition, we find that the benefits of recent volume accrue differentially across tenure groups, favoring paramedics with above median tenure.

Data limitations have led most researchers to focus on the effect of recent experience (e.g., volume in the past quarter or year) on performance.1 This is appropriate only if past experience rapidly loses its relevance. Alternatively, if past experience matters, high turnover rates may affect overall prehospital performance as paramedic human capital accumulation is subject to more frequent interruptions. The richness of our data allows us to follow 1,728 paramedics (85 percent of all paramedics in our data) for whom we observe their full histories in the profession. We construct measures of their tenure and cumulative experience as well as of recently acquired experience. Hence, by evaluating the contribution of cumulative experience, we assess the value of paramedic retention in performance units.

The benefit of greater experience operates both through recent and past experiences. This has important implications for retention: retaining the typical paramedic in our sample for an additional quarter is associated with a reduction in out-of-hospital time of approximately 4 minutes. Moreover, 3 years after replacement of an experienced paramedic by a new one, the loss in out-of-hospital time from turnover is between 1 minute and 90 seconds depending on functional form.

Finally, we find evidence that individual experience operates not only on mean performance but also on the upper quantiles of the paramedic performance distribution, which can be interpreted as greater experience leading to better standardization with the severity of trauma.

BACKGROUND

The EMS Workforce

The three national standard levels of training for Emergency Medical Technicians (EMT) are EMT-Basic (EMT-B), EMT-Intermediate (EMT-I), and EMT-Paramedic (EMT-P).2 In addition, Mississippi requires operators of ambulance vehicles to be EMS-Driver certified (EMS-D), by participating in a training program in operation of emergency vehicles. Licenses for EMTs must be renewed, typically every 2–3 years.

A combination of factors renders the hiring and retention of EMTs problematic, including concerns regarding personal safety, stressful working conditions, irregular hours, excessive training and requirements, limited mobility, and low wages (Institute of Medicine 2000).

Paramedics face a difficult, often hazardous, work environment. They are exposed to potentially infectious bodily fluids, for instance through contact with contaminated needlesticks, and to the hepatitis B virus (Delbridge et al. 1998). Moreover, they are frequently exposed to the threat of violence, incur injuries associated with lifting or falling, and face oncoming traffic at the scene of motor vehicle crashes (Franks, Kocher, and Chapman 2004). Occupational fatality rates for EMTs are comparable to those of police and fire personnel (Maguire et al. 2002).3

In 2001, the Journal of Emergency Medical Services conducted a salary survey consisting of 371 EMS organizations. The survey found salaries to be about 10 percent lower in the Southeastern Census region, which includes Mississippi, compared with the national average. Tenure for paramedics was 7.3 years in private ambulance companies compared with 10.8 years in fire-based EMS agencies. Finally the average yearly turnover rate for private ambulance companies was 18 percent, as compared with 6 percent for fire-based agencies.

EMS Performance

Some studies have found evidence that prehospital advanced life support (ALS) interventions reduce the severity of injuries, lower mortality rates in the first 24 hours, improve outcomes from multiple trauma, and improve survival among patients with blunt trauma (Isenberg and Bissell 2005). Nichol et al. (1996) perform a meta-analysis and find shorter out-of-hospital time to be related to lower mortality. However, overall, the medical literature provides mixed evidence regarding the effectiveness of out-of-hospital management of trauma victims (Carr et al. 2006; Delbridge and March 2007).

Rural trauma patients, on average, face considerably lengthier transport to definitive care (Isenberg and Bissell 2005). Studies focusing on rural areas have found increased survival as a result of on-scene interventions, suggesting that delaying the onset of definitive care may be justified despite the prolonged prehospital intervals (Branas et al. 2005).

To date, only one study has examined the effect of ambulance crew's tenure on outcomes of out-of-hospital cardiac arrests (Soo et al. 1999). The authors found improved survival when patients were attended by paramedics with >1 year of tenure or technicians with >4 years of tenure. The study focused on Nottingham, England, and found no effect of tenure on the time interval spent at the scene. However, it did not look at trauma victims, omitted incident-level characteristics, ignored team composition, and studied tenure (months since completion of training) as opposed to volume (number of runs).

The medical literature provides little guidance as to the right approach for managing out-of-hospital trauma victims. Yet conditional on the characteristics of patients, paramedics, injury, medical interventions performed, and of the scene, there is no dispute that shorter out-of-hospital intervals are preferred by patients and paramedics to longer ones. In this paper, we study the effects of changes in individual paramedic volume on total out-of-hospital time, which is considered a key marker of EMS performance (Carr et al. 2006). In addition, we repeat our analysis for the time spent at the scene of trauma incidents, conjecturing that better diagnostic and therapeutic expertise may reduce on-scene time and hence contribute to the reduction in total out-of-hospital time.

DATA

Our data were obtained from the Office of Emergency Planning and Response at the Mississippi Department of Health. Since 1991, this office has systematically collected incident-level EMS data through the Mississippi Emergency Medical Services Information System (MEMSIS). The raw data are recorded at the individual patient level by local EMS providers.

We study incidents between 1991 and 2005 for which the initial call was related to trauma, defined as motor vehicle crashes, pedestrian injuries, stabbings, assaults, gunshots, or falls. To focus on EMS runs where time to definitive care is most likely to be important, we exclude cases of death on arrival and limit the sample to calls involving at least one patient injury and ending in transfer to hospitals by ground transportation.4 All time stamps in the MEMSIS data are rounded to the nearest minute.

Given the highly skewed nature of reported interval times and the possibility of extreme values due to miscoding, we exclude calls for which either the reported time from dispatch to arrival on the scene (i.e., response time) or the reported time from leaving the scene to arrival to a hospital (i.e., transport time) exceeds 60 minutes. This criterion excluded <1 percent of trauma observations that satisfied the other criteria and does not affect our results.

Detailed data on procedures performed by paramedics are available only for the 2001 to 2005 period. While we restrict our volume–outcome analysis to this latter period, we use data for all years (1991–2005) to construct the history of paramedics' experiences, encompassing approximately 613,000 trauma runs. Fifteen percent of paramedics who appear in the 2001–2005 sample are left-censored (i.e., appear in the 1991 data and for whom the entry date is unknown) and the rest enter after 1991. The long time series and the relatively large number of uncensored paramedics enable us to account more precisely for the cumulative number of runs of each individual paramedic. The final sample includes approximately 175,000 observations.

Prehospital emergency medical care is provided by a network of ambulance services organized by county or city. We identify 86 contracting municipalities in Mississippi, where 56 different EMS providers operate. EMS dispatchers can be accessed either by 911 or enhanced-911 service throughout the region. Transport services are provided by fire department agencies, private ambulance services, or hospital-based agencies. In our final sample, 98 percent of all trauma transports are provided by ALS-trained providers. Approximately 10 percent of trauma patients in our sample were directly transported, via ground units, to a Level I trauma facility.5

ALS-certified paramedics can engage in advanced airway management, cardiac monitoring, drug therapy, and other advanced techniques that are prohibited by EMTs with lower certification levels. Team composition may therefore affect prehospital intervals through the quantity and complexity of procedures performed on trauma patients. We control for the experience and certification levels of both team members as well as for the procedures performed.

On-scene (and total out-of-hospital) time is also expected to increase with the number of victims at the scene. Approximately 75 percent of trauma incidents involved a single patient and 98 percent involved at most three individuals. To proxy for underlying patient severity, we control for the procedure indicators, the type of trauma, and patient demographic characteristics. In addition, we control for all possible combinations of the injured body part (i.e., arm, leg, chest, hip, back, neck, head, face, abdomen, and eye) and the type of injury (i.e., pain, burn, laceration, soft tissue, blunt, fracture, dislocation, penetrating trauma, and amputation). These injury combinations are likely to be correlated with the patient severity, which, in turn, is likely to be an important determinant of out-of-hospital time.

EMS in Mississippi is provided by several different types of agencies. In our data, about 19 percent of agencies are community based (mostly integrated with local fire departments), 27 percent are hospital based, and the remaining 54 percent are large private ambulance companies. EMS agencies integrated into and operated by hospitals may have different approaches to the role of paramedics due to closer medical supervision. Similarly, paramedics working for a large private multistate company may have access to better training, equipment, and operate under more stringent protocols compared with a small local fire-based agency.

While we are interested in the effect of paramedic volume on total out-of-hospital and on-scene times, there are many other factors that may affect these markers of performance. These confounders, most of which are summarized in Table 1, include the type of trauma, the incident location, patient characteristics, 42 indicators of procedures performed (listed in Table 2), the number of victims, year, month, day of week, hour of day, the certification levels of the EMTs in each unit, the type of firm that employs them, and the municipality they operate in.6 When studying on-scene time we also include the time it took the team to arrive at the scene.

Table 1.

Summary Statistics

Variable	Mean		Standard Deviation
Total out-of-hospital time	36.05	Minutes	16.62
On-scene time	14.67	Minutes	8.496
First-response time	8.07	Minutes	6.073
Transport time	13.36	Minutes	9.723
Paramedic number of runs in last 3 months	17.69	Trauma runs	12.240
Paramedic total number of runs (noncensored)	408.75	Trauma runs	299.150
Firm number of runs in last 3 months	456.68	Trauma runs	528.720
Paramedic tenure (noncensored)	6.48	Years	3.189
Number of procedures and EMT certifications
Number of procedures in incident	2.00	Procedures	2.185
Cerification level: EMT-Basic	2.56%		0.158
Cerification level: EMT-Intermediate	0.57%		0.075
Cerification level: EMT-Paramedics	96.87%		0.174
Patient demographics and people in incident
Patient age	42.12	Years	25.10
Patient race: African American	40.80%		0.491
Patient race:White	55.86%		0.497
Patient gender: female	55.10%		0.497
Number of victims in incident	1.33	Victims	0.741
Type and location of trauma
Type of trauma: fall	31.59%		0.465
Type of trauma: motor vehicle crash	53.02%		0.499
Type of trauma: motorcycle accident	1.14%		0.106
Type of trauma: pedestrian accident	1.69%		0.129
Type of trauma: cut/stabbing	2.32%		0.151
Type of trauma: assault	8.74%		0.282
Type of trauma: gunshot	1.50%		0.122
Location of trauma: city street	20.70%		0.405
Location of trauma: county road	9.26%		0.290
Location of trauma: state/federal highway	23.68%		0.425
Location of trauma: residence	30.53%		0.461
Location of trauma: other	15.83%		0.365
Year and month
Year 2001	19.61%		0.397
Year 2002	21.55%		0.411
Year 2003	19.66%		0.397
Year 2004	19.75%		0.398
Year 2005	19.43%		0.396
January	7.54%		0.264
February	7.78%		0.268
March	8.76%		0.283
April	8.79%		0.283
May	9.15%		0.288
June	8.54%		0.279
July	8.78%		0.283
August	7.90%		0.270
September	7.99%		0.271
October	8.20%		0.274
November	8.33%		0.276
December	8.24%		0.275
Provider
Public EMS provider (e.g., fire department based)	18.64%		0.389
Hospital-based EMS company	27.23%		0.445
Private EMS provider (corporate)	54.13%		0.498

Table 2.

Returns to Paramedic Experience

	On-Scene Time			Total Out-of-Hospital Time			Dispatch Time
		Firm-by-Contract	Paramedic		Firm-by-Contract	Paramedic	Paramedic
	Cross-Section	Area Fixed Effects	Fixed Effects	Cross-Section	Area Fixed Effects	Fixed Effects	Fixed Effects
Return to quarterly volume	−0.0300	−0.0142	−0.0137	−0.0604	−0.0385	−0.0439	0.0020
(91 days)	[0.0079]***	[0.00506]***	[0.00337]***	[0.01546]***	[0.01194]***	[0.00747]***	[0.00478]
[174,164 Obs.; 2,010 Paramedics]
Paramedic volume	−0.0409	−0.0153	−0.0142	−0.0814	−0.0379	−0.0350	0.0055
	[0.01066]***	[0.00026]***	[0.00337]***	[0.02159]***	[0.01241]***	[0.00721]***	[0.00487]
Firm volume	0.0008	0.0007	0.0001	0.0016	−0.0004	−0.0017	−0.0005
	[0.00022]***	[0.00003]***	[0.00019]	[0.00054]***	[0.00072]	[0.00048]***	[0.00019]***
[174,164 Obs.; 2,010 Paramedics]
First tenure quartile	−0.0078	0.0012	−0.0045	−0.0413	−0.0264	−0.0282	0.0077
(Tenure <2 years)	[0.00549]	[0.00408]	[0.00465]	[0.01571]***	[0.01248]**	[0.0087]***	[0.00797]
Second tenure quartile	−0.0490	−0.0289	−0.0219	−0.0812	−0.0528	−0.0439	0.0011
(Tenure 2–4 years)	[0.00713]***	[0.0054]***	[0.00498]***	[0.01328]***	[0.01019]***	[0.00895]***	[0.00707]
Third tenure quartile	−0.0658	−0.0364	−0.0195	−0.0962	−0.0555	−0.0465	0.0000
(Tenure 4–6.2 years)	[0.0085]***	[0.00589]***	[0.00481]***	[0.01614]***	[0.01008]***	[0.011]***	[0.00793]
Fourth tenure quartile	−0.0715	−0.0405	−0.0208	−0.0916	−0.0525	−0.0684	−0.0056
(Tenure above 6.2 years)	[0.01014]***	[0.00768]***	[0.00645]***	[0.02009]***	[0.0137]***	[0.01569]***	[0.01019]
[144,495 Obs.; 1,726 Paramedics]
Log cumulative volume	−0.7307	−0.4626	−0.4583	−0.8536	−0.4163	−1.0077	0.1272
(Excluding left-censored paramedics)	[0.07052]***	[0.05706]***	[0.13347]***	[0.15366]***	[0.08581]***	[0.26716]***	[0.16789]
[144,495 Obs.; 1,726 Paramedics]
Log cumulative volume	−0.6862	−0.4134	−0.5034	−0.7627	−0.3195	−0.9388	0.1547
(Including left-censored paramedics)	[0.06381]***	[0.05143]***	[0.1317]***	[0.13967]***	[0.07948]***	[0.26608]***	[0.17197]
[174,164 Obs.; 2,010 Paramedics]

Notes: Standard errors are reported in brackets below the estimated coefficients, and are clustered at the paramedic level. *, **, and *** indicate significance at the 10%, 5%, and 1% levels, respectively. All models include controls for paramedic and driver certification levels, patient demographics, type of trauma, number of patients in incident, time and location of incident, injury characteristics, procedures performed, and class of firm responding to the incident. Patient demographics include indicators for race/ethnicity, for gender, and for 12 age categories. The types of trauma are falls, gunshot wounds, cuts or stabbings, assaults, motor vehicle crashes, and motorcycle and pedestrian accidents. The time of incidents includes indicators year, month, day of week, and eight hour-of-day categories. Locations of incidents include residences, city streets, county roads, and state or federal highways. Injury characteristics include 70 interactions of injured body part and injury type. Procedures performed by paramedics, each assigned an indicator in our models, are Oxygen Therapy, Airway, Cardiac Monitor, Cardioversion, CPR, Crisis Intervention, Extrication, Mast Applied, Mast Installed, OB Delivery, Spinal Immobilization, Splint Extremity, Suction, Ventilation, Wound Management, Assist Patient Nitroglycerin, Administer Activated Charcoal, Administer Glucose, AED, Obtain Venous Blood Sample, Defib, Drug Administration, Intubation–Oral, Intubation–Nasal, Intaosseous Infusion, Pacing, Pulse Oximetry, End-Tidal CO2 detection, Peak Expiratory Flow Testing, Tracheobronchial Sunctioning, Insertion of NG Tube, Insertion of OG Tube, Gastric Decompression, Transport Ventilation, Needle Cricothyrotomy, Blood Glucose Check, Attainment of 12-lead EKG, Vascular Access-Peripheral, Vascular Access-Jugular, Vascular Access-Central Port/VAD, Insertion of IO Line, and Needle Chest Decompression. The three broadly defined firm classes are [1] fire department based or small privately/locally owned providers, [2] hospital based, and [3] large private providers. On-scene time regressions also control for first-response time.

EMPIRICAL FRAMEWORK

We estimate several models linking paramedic volume with out-of-hospital and on-scene times. The input of interest is the experience of paramedics, as measured by different functions of their accumulated volume as of the date of the incident.

Consider a trauma scene at date t in which injured patient k requires some prehospital care by paramedic i. Prehospital time may be written as

(1)

where T_ikt is either the time spent on scene or the total out-of-hospital time for paramedic i, patient k, and date t. VOL_it is a measure of paramedic i's volume as of date t. X_kt captures the characteristics of the patient, such as her age, sex, and race as well as all interactions of injury type and injured body part. It also captures characteristics of the incident, such as type and location of trauma. W_it includes other paramedic characteristics, such as their certification levels, and the type of firm they work for. ϕ_t is a vector of indicators for hour of day, day of the week, month, and year. μ_i are individual EMT fixed effects and ɛ_ikt is a random disturbance. In some of our specifications, our measure of volume, VOL_it, represents the aggregate experience level of paramedic i in period t, and in other specifications, the experience accrued solely in the last quarter. VOL_it accumulates experience over running 3-month windows, recording paramedic volume at a given date as the number of trauma runs accumulated during the prior 91 days. This measure is more precise than fixed calendar quarters, used extensively in the literature due to data limitations, as it responds instantaneously to any changes in the recent experience profile. Experience accumulation with moving windows can be viewed as smoothing the calendar quarter step function and alleviating the imprecision that increases the further incidents are from the beginning of the quarter.

EMT fixed effects are introduced to mitigate concerns that the volume–outcome relationship we observe is driven by composition effects. For example, low quality/ability paramedics might be lower volume, whereas more able paramedics may accumulate more experience by working more intensely and/or staying in the profession longer. Hence, EMT fixed effects guarantee that the experience parameter in [1] is identified from improvements in performance within paramedic.

While EMT fixed effects provide the most flexible specification for studying individual learning-by-doing, we also estimate firm-by-municipality fixed effect models. This captures systematic time-invariant differences across municipality–firm pairs in the structure of EMS. The interaction between contracting municipality characteristics, such as urban–rural continuum or trauma infrastructure, and firm characteristics, such as EMS protocols, training and equipment, may be correlated with both paramedic volume and on-scene time.

While fixed effects capture time-invariant factors related to individual performance and to volume, such as ability, it may be important to consider bias resulting from evolutionary forces. In particular, a volume–outcome relation may simply result from attrition of poor performers. To address this concern, we perform commonly used variable addition tests (Verbeek and Nijman 1992; Wooldridge 2002) and find no evidence of attrition bias.7

Conditional on observables and on the fixed effects, ɛ_ikt is likely to be unrelated to any EMT characteristics, be they quality, ability, or, importantly, experience, since the selective referral channel is unlikely to drive the relationship between volume and out-of-hospital time intervals.

RESULTS

In this section, we present estimates of the experience premium in EMS performance, as measured by decreased out-of-hospital and on-scene durations, by estimating equation (1).8

Our models are estimated by OLS cross-sectionally, with firm-by-municipality fixed effects, and with paramedic fixed effects. Standard errors are clustered at the paramedic level to allow for correlation in performance across incidents within paramedic.9

In the first row of Table 2, we document a relationship between recent (quarterly) EMT volume and performance that implies that a one standard deviation increase in the number of trauma runs accumulated by an individual paramedic over a given quarter is associated with a reduction of approximately 35 (10) seconds in total out-of-hospital (on-scene) time. Coefficient estimates are statistically significant at the 1 percent level.

There is a strong tie between retention and learning measured at the firm level, as high labor turnover rates may result in a loss of accumulated experience and hence lower productivity. The short timeframe and, for the most part, lack of individual identifiers led previous studies (e.g., Thiemann et al. 1999; Peterson et al. 2004; Marcin et al. 2007) to focus on learning at the firm level, which confounds individual learning and spillover and turnover effects, with no ability to separate between these three vastly different mechanisms. Our data allows us to follow paramedics over a decade and a half and hence identify the contribution of these three channels to overall performance.

According to Rosen (1972), “learning by experience suggests that the production of knowledge is at least partially acquired through the productive process itself.” An additional EMS run adds to both the individual paramedic and the firm's stock of experience. Firm experience represents the extent to which individual experience spills over to other members of the firm. This calls for a specification in which recent firm volume is added to the one capturing recent paramedic volume.

The results, reported in the second and third rows of Table 2, reveal magnitudes for returns to paramedic volume similar to those in the first row. However, the effect of firm volume is of small magnitude as well as varying signs and degrees of significance for both on-scene and total out-of-hospital times. This is likely due to the limited scope for spillover in tasks performed in teams that rarely exceed two individuals.

To allow for heterogeneity in learning, we stratify the analysis by paramedic tenure under the hypothesis that paramedics of different tenures might exhibit different gains to recent volume. The third panel of Table 2 reports estimates of models in which paramedic volume is interacted with four indicators of tenure quartiles. The quartiles correspond to <2, 2–4, 4–6.2, and over 6.2 years as a paramedic, respectively.

We find that the benefits of recent volume accrue differentially across tenure groups, with small gains in the first two quartiles of the tenure distribution. Paramedics with 6.2 years of tenure and above enjoy about 85 percent higher returns to volume than in the pooled regression for on-scene time. The analogous figure for total out-of-hospital time is 55 percent. This result is somewhat puzzling given the existing evidence that learning is stronger at the beginning of individuals' careers. Our results are consistent with the notion that recent volume affects performance more with greater exposure to trauma scenes and/or with the idea that more senior paramedics have greater influence over the various aspects of EMS delivery as well as the management of patients at the scene (e.g., less reliance on medical direction and better management of other paramedics). Alternatively, composition within tenure quartiles may account for observing learning only among long-tenured paramedics. While the upper tenure quartiles include only paramedics with relatively long job duration, paramedics with low tenure represent a mixture of individuals who are either in the beginning of a longer career in EMS or who do not stay in the profession very long. Therefore, our results are consistent with an explanation whereby short job duration reflects insufficient learning due to lack of either ability or effort.10

While existing volume–outcome studies are almost exclusively focused on providers of tertiary services (e.g., cardiac surgeons) where career durations are extensive, the data used typically span only 2–5 years, accommodating only recent volume effects. Our data covers 15 years of professionals whose average career duration is relatively short and therefore allows us to examine an alternative view of the volume–outcome relationship, in which the benefit accrues to cumulative rather than recent volume. The lower panel of Table 2 reports parameter estimates from models in which the variable of interest is defined as the log of cumulative volume. The log transformation is used to account for the skewed nature of the cumulative volume distribution across paramedics, and it is commonly employed in the volume–outcome literature.11

Since we do not observe the full history of runs for paramedics who appear at the very beginning of the sample, we perform the analysis separately for all 2,010 paramedics (lower panel) and for 1,726 paramedics who where certified (for the first time) after 1991 (upper panel).12 The coefficient estimates for all paramedics are similar in magnitude to the ones for uncensored paramedics, suggesting that experience accumulated before 1991 has little or no relevance by 2001.13

These results allow us to compute the value of retention in performance units. More specifically, we compute the increase in time spent on scene that results from replacing the average paramedic in our data with a new one as follows:

(2)

where is the coefficient estimate for log cumulative volume, is the average cumulative volume across paramedics, and v₁ is the average quarterly volume across paramedics. Equation (2) computes the difference in average out-of-hospital (on-scene) times within a typical quarter between the average paramedic and a new one. The reductions in total out-of-hospital and on-scene times associated with the retention of the typical paramedic in our sample for an additional quarter are approximately 4 and 2 minutes, respectively.

Figure 1 plots the average monthly difference in total out-of-hospital and on-scene times between an experienced paramedic and a new one. The first period represents the hypothetical month of replacing the average paramedic (in volume terms) in our sample with a new one. We then use equation (2), where v₁ is defined at the monthly level, for the subsequent 36 months. The figure describes the loss in total out-of-hospital and on-scene times from the replacement as we follow the new paramedic and compare her average performance to that of the experienced paramedic (had she not exited), therefore allowing for learning by doing by both paramedics. While the cost of turnover is higher in the first months following replacement, differences in performance are (slowly) exhausted over time. Here again, these results do not appear to be driven by attrition.

Figure 1.

Cost of Replacing an Average Paramedic with a New One, by Month (Measured in Additional Minutes of Prehospital Time)

In the right most column of Table 2, we implement a falsification exercise by estimating all models described above, in which we replace the dependent variables, total out-of-hospital and on-scene times, with an alternative marker of system performance, dispatch time. Dispatch time is defined as the length of time between a 9-1-1 call and the moment paramedics are notified and dispatched to the scene. This measure provides the basis for a credible falsification test as, unlike time spent on scene or out of hospital, paramedics have no influence on it. If learning is the mechanism through which greater volume yields shorter on-scene and prehospital times, then we should not find a relationship between paramedic experience and a margin over which they have no influence, such as dispatch time. The results confirm this intuition, lending credibility to our results.

In Table 3, we explore the possibility that the performance benefits of experience operate in dimensions other than just mean prehospital times. Specifically, we posit that paramedic volume might shrink the conditional distributions of total out-of-hospital and on-scene duration. To this end, we employ quantile regression methods, which serve to describe how independent variables affect the entire distribution of the dependent variable, as opposed to just its mean. In this analysis, quantile regressions were estimated cross-sectionally and with firm-by-municipality fixed effects at seven different percentiles (0.05, 0.15, 0.25, 0.5, 0.75, 0.85, and 0.95).14 The results indicate that greater recent paramedic volume compresses the upper tail of the conditional on-scene time distribution.15 For instance, according to the firm-by-municipality fixed effects specification, a one standard deviation increase in the number of trauma runs accumulated by an individual paramedic over a given quarter reduces the 95th percentile of total out-of-hospital (on-scene) time by 38 (20) seconds, which is slightly more than double the median and mean effects.

Table 3.

Effect of Paramedic Experience on Quantiles of Performance

	Quantiles
	0.05	0.15	0.25	0.5	0.75	0.85	0.95
Dependent variable: On-scene time
Cross-section	0.0008	−0.0144	−0.0201	−0.0279	−0.0325	−0.0379	−0.0405
	[0.00221]	[0.00137]***	[0.00135]***	[0.00176]***	[0.0029]***	[0.00428]***	[0.00955]***
Firm-by-contract	0.0036	−0.0064	−0.0094	−0.0136	−0.0189	−0.0201	−0.0295
Area fixed effects	[0.00268]	[0.00155]***	[0.00151]***	[0.0018]***	[0.00272]***	[0.00408]***	[0.00863]***
Number of observations	174,323	174,323	174,323	174,323	174,323	174,323	174,323
Dependent variable: Total out-of-hospital time
Cross-section	0.0147	0.0083	0.0010	−0.0359	−0.0998	−0.1243	−0.1405
	[0.00352]***	[0.00319]***	[0.00302]	[0.00392]***	[0.00631]***	[0.0098]***	[0.02141]***
Firm-by-contract	−0.0105	−0.0196	−0.0213	−0.0338	−0.0486	−0.0473	−0.0576
Area fixed effects	[0.00336]***	[0.00311]***	[0.0028]***	[0.00378]***	[0.00563]***	[0.00825]***	[0.01725]***
Number of observations	174,323	174,323	174,323	174,323	174,323	174,323	174,323

Notes: Standard errors are reported in brackets below the estimated coefficients, but are not clustered at the paramedic level, and are therefore likely to underestimate the true variability of the estimated coefficients. Inferences based on them should therefore be made with caution. *, **, and *** indicate significance at 10%, 5%, and 1% levels, respectively. See Table 2 notes for list of controls.

Interventions on scene, injury profile, trauma characteristics, and patient demographics are used, in part, to proxy for patient severity. While these may reflect severity only to a limited extent, concerns regarding omitted variables are not likely to be important given the current EMS system design. It is difficult to argue for a correlation between severity and experience due to the fact that dispatching for ALS incidents is determined by proximity to the scene and not by EMT reputation. However, even if EMS did match paramedic experience with indications of acuity, it is unlikely that more time-consuming, higher severity incidents would result in the least experienced crews being dispatched, which is the only mechanism that would account for our results. If there is matching between highly experienced EMTs and patient severity, our results underestimate the true reduction in prehospital time due to an increase in experience.16

Finally, while our regressions control for indicators of prehospital interventions, one might worry about selection on the complexity of procedures performed by paramedics. For example, if inexperienced paramedics choose simpler procedures, which require relatively fewer minutes, we might infer that less experience results in shorter prehospital times conditional on procedures performed. This would lead us to underestimate the magnitude of the experience premium.

DISCUSSION

Our results have policy implications for local governments, charged with EMS delivery, and for public and private insurers, who design reimbursement schedules for emergency transports.

In the context of hospitals, policy relevance and implications vary depending on the underlying mechanism behind the volume–outcome relation (Luft, Hunt, and Maerki 1987). When it is driven by selective referrals, no regulatory remedies are advised, as procedures are already concentrated in facilities that provide greater value for patients. Conversely, if causation runs from volume to outcome, the benefit of centralizing care in a small number of facilities may call for regulating entry as well as existing capacity using antitrust analysis of hospital mergers (Birkmeyer et al. 2002; Gaynor, Seider, and Vogt 2005) or policies such as regionalization (Luft, Hunt, and Maerki 1987).

We find that greater volume is robustly related to reduced total out-of-hospital time as well as time at the trauma scene. This benefit operates through both recent and past volume and accrues differentially across tenure groups. In light of our findings, and taking into account the important institutional details that govern the field of EMS, we examine three policies designed to increase volume: consolidation of EMS delivery organizations, redesigning the allocation of workload to insure that current volume is allocated across fewer paramedics, and improving retention of paramedics to prevent past volume from being lost to turnover.

Firm-level studies, which dominate the literature, focus on firm-level interventions by suggesting that large-scale effects raise the benefit from consolidation. While subtle, this assertion relies on the importance of facility-level volume beyond the individual provider's volume. In our data, we find little or no such incremental firm-scale effects. While the importance of distinguishing between learning at the individual level and at the firm level was raised (Luft, Hunt, and Maerki 1987) and tested (Huckman and Pisano 2006) in the case of surgeons, no study to date has examined the returns to volume in a prehospital setting.17 In EMS, absent any knowledge spillovers across paramedics, the individual volume effect we find cannot autonomously constitute grounds for supporting larger firms and therefore yields no direct antitrust implications for mergers. Nevertheless, the massive consolidation of EMS delivery organizations, mainly in the early to mid 1990s, can be justified on different grounds, such as eliminating duplication and lowering average costs through economies of scale (David and Chiang 2009).

As all benefits of greater volume accrue to individual paramedics with no evidence of its transferability across paramedics, policies designed to expose paramedics to greater volume should be considered. Individual volume can be increased in one of two ways, static or dynamic, with each way requiring its own distinct mechanisms. The first requires redesigning the allocation of workload across individuals such that within a given time frame, a given paramedic is handed more runs. The second calls for policies to reduce labor turnover rates among paramedics.

Allocation of workload across paramedics can raise volume by insuring the concentration of runs at the hands of fewer paramedics. But unlike surgery, where individual capacity can be efficiently reached through scheduling, the random nature and location of trauma events requires potentially long periods of inactivity as well as geographical spread of units. Reducing the number of paramedics, regardless of their volume–outcome quality benefits, would translate into longer wait times between initial emergency calls and arrival at the trauma scene. In addition, imposing longer shifts is likely to lead to fatigue, burnout, and dissatisfaction. Using this mechanism is more likely to weaken performance than to improve it through increased individual volume, especially since the effect of recent volume on performance is small. On the other hand, our finding that past volume is important in explaining current outcomes suggests concentrating volume in the hands of paramedics who have already accumulated experience (Gaynor, Seider, and Vogt 2005). Absent mechanisms to induce demand, the salient route for increasing lifetime volume is by increasing the career duration for paramedics. This calls for improving their retention.

In addition, we find that volume is related to improvements in both mean performance and in the upper quantiles of the performance distribution, which can be interpreted as greater experience leading to better standardization with the severity of trauma. This suggests that the reduction in average prehospital times is one of many potential benefits of retention, which may include mentorship, improved quality of care, greater stability of EMS teams, etc. These benefits should be weighted against all the costs associated with retention.

While more research is needed, our results suggest that policy and managerial implications regarding volume–outcome relations in EMS should be directed toward retention to take advantage of individual human capital accumulation.

Supporting Information

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.