Measuring teamwork and conflict among Emergency Medical Technician personnel

Abstract

Objective

We sought to develop a reliable and valid tool for measuring teamwork among Emergency Medical Technician (EMT) partnerships.

Methods

We adapted existing scales and developed new items to measure components of teamwork. After recruiting a convenience sample of 39 agencies, we tested a 122-item draft survey tool. We performed a series of Exploratory Factor Analyses (EFA) and Confirmatory Factor Analysis (CFA) to test reliability and construct validity, describing variation in domain and global scores using descriptive statistics.

Results

We received 687 completed surveys. The EFA analyses identified a 9-factor solution. We labeled these factors [1] Team Orientation, [2] Team Structure & Leadership, [3] Partner Communication, Team Support, & Monitoring, [4] Partner Trust and Shared Mental Models, [5] Partner Adaptability & Back-Up Behavior, [6] Process Conflict, [7] Strong Task Conflict, [8] Mild Task Conflict, and [9] Interpersonal Conflict. We tested a short form (30-item SF) and long form (45-item LF) version. The CFA analyses determined that both the SF and LF versions possess positive psychometric properties of reliability and construct validity. The EMT-TEAMWORK-SF has positive internal consistency properties with a mean Cronbach’s alpha coefficient ≥0.70 across all 9-factors (mean=0.84; min=0.78, max=0.94). The mean Cronbach’s alpha coefficient for the EMT-TEAMWORK-LF version was 0.87 (min=0.79, max=0.94). There was wide variation in weighted scores across all 9 factors and the global score for the SF and LF versions. Mean scores were lowest for the Team Orientation factor (48.1, SD 21.5 SF; 49.3 SD 19.8 LF) and highest (more positive) for the Interpersonal Conflict factor (87.7 SD 18.1 for both SF and LF).

Conclusions

We developed a reliable and valid survey to evaluate teamwork between EMT partners.

INTRODUCTION

Teamwork is important to safe healthcare delivery. Key components of teamwork include mutual trust, shared mental models, and closed loop communication.¹ Poor teamwork behaviors are associated with negative patient outcomes.^2-4 Teams of EMS providers are most often configured with two EMT-paramedics or one EMT-paramedic and one EMT-Basic,⁵ though some EMS agencies configure teams with EMT-Intermediates, prehospital nurses, or teams of two EMT-Basics. Research characterizing teamwork in EMS is limited and teamwork measurement tools are lacking.

Teamwork in Emergency Medical Services (EMS) is critical to effective and safe care delivery. EMT partners must work together to establish scene safety, load and move patients, deliver stabilizing care on scene and during transport, and transfer patient care to receiving facilities.^6-9 An EMT must anticipate the actions and expectations of his/her partner with minimal disruption. Stress, fatigue, and frequent turnover in partnerships are common characteristics of EMS work that may threaten teamwork between EMT partners.^10-14

A tool that can measure teamwork between EMT partners is absent; such a tool may help evaluate training and improve safety by diagnosing poor or positive teamwork and optimize crew pairings. We sought to develop a reliable and valid tool for measuring teamwork among EMTs at the dyadic (2 person) level.

METHODS

Study Design and Sample

We used a cross-sectional study design to develop and test a teamwork instrument on EMTs and paramedics recruited from a convenience sample of EMS agencies. The University of Pittsburgh Institutional Review Board approved this study.

Instrument Development

We followed prescribed steps for survey development by adopting the Salas et al’s “Big Five” framework of teamwork measurement.^1,15 This framework operationalizes teamwork into five constructs (Team Leadership, Team Orientation, Mutual Performance Monitoring, Back-Up Behavior, Adaptability) and three coordinating mechanisms (Closed Loop Communication, Shared Mental Models, and Mutual Trust).¹ We added an additional construct, the measurement of conflict among teammates. Our focus group work identified conflict as important to EMT teamwork and performance.

We reviewed the literature to identify established reliable and valid survey scales of teamwork (e.g., the TeamSTEPPS survey and Collective Orientation Scale).^16,17 Key search terms included: teamwork, teams, safety, partners, partnerships, and team performance. We also obtained scales under development.^18,19 We adapted the wording of all existing scales to reference EMS partnerships at the dyadic level. We then assigned each item in every scale to a construct to designate the component of teamwork or conflict to which it was intended to measure (Figure 1).

Figure 1.

Step-by-step illustration of survey development

We recorded possible item responses on a 7-point Likert scale (Strongly Disagree to Strongly Agree). Forty-five items included the additional response option “Does Not Apply.” We included this response option to: 1) examine the utility of this option for EMS crews as a standard neutral response rather than the standard “Neither Agree nor Disagree”, and 2) to provide EMTs a response option for partnerships where interactions (i.e., patient care episodes) may be limited. In addition, we included 7 questions on EMT demographic characteristics.

Recruitment of Agencies

We recruited a convenience sample of EMS administrators affiliated with the National EMS Management Association (NEMSMA). After distributing a letter on the association’s email Listserv describing the study, we recruited administrators with an interest in study participation and contacted each to enroll his/her agency. Eligible agencies provided state licensed prehospital emergency care services and configured crews in dyads. Thirty-nine administrators agreed to participate.

Study Protocol

We administered our EMT-TEAMWORK survey via a secure Internet-based survey system. EMS agency administrators used our survey system to prerecord the following information for each employee: employee email address, employment status (Full-time, Part-time, Volunteer), percent of clinical work (0-100%), years of service at agency (e.g., 5 years) and level of certification (Paramedic, EMT-Basic, Other). The administrators circulated a study flyer and communicated the importance of the study to employees during group meetings and other agency functions. The administrators used our survey system to distribute the EMT-TEAMWORK survey. Individual EMTs received an email from the University of Pittsburgh that included a brief description of our study, a secure link to the survey, and link for opting-out.

We collected data from January to December 2010. We asked administrators to promote participation by letting their employees know they strongly supported the study. Administrators periodically logged into their survey system account to view their agency’s overall response rate. We gave administrators the option of sending the survey links with the opt-out option multiple times. We designed our survey system to keep the respondent / non-respondent status of email addresses confidential. This strategy encouraged administrators to focus on the agency as a whole to promote participation, as they could not follow-up with individual non-respondents. We gave a two-part incentive for the agency with the highest overall response rate. The first part was a $25 dollar gift card to the agency. The second part was a combination of a $5 dollar gift card for all employees in the agency and 30 high-end stethoscopes that the agency administrator could use to restock agency issued first-in bags or distribute to individual employees.

Analysis

At the end of the data collection phase of the study, we received a dataset with coded responses linked to randomly generated agency and survey ID numbers. We calculated percentages, means, and standard deviations to describe the agency and respondent characteristics. Using parametric tests (Pearson correlations and t-tests) and non-parametric tests (chi-square tests), we explored differences in respondents versus non-respondents. We weighted responses from 0 to 100 to improve interpretation of scores by increasing the spread of data (Strongly Disagree=0, Disagree=16.6, Slightly Disagree=33.3, Neither Agree nor Disagree=49.9, Slightly Agree=66.6, Agree=83.3, Strongly Agree=100). We reverse coded negatively worded items.

We performed a series of Exploratory Factor Analyses (EFA) to reduce the total number of items in our draft survey (n=122) and identify distinct/measurable factors of teamwork and conflict.²⁰ We sought to develop two versions of a final survey tool. One version (the Short Form SF) included the minimally recommended number of three items per construct and satisfied the requirements to be low-burden on respondents and most applicable to day-to-day human resource management.^15,20 The second version (Long Form LF) included as many items as the CFA analysis would allow and be applicable to investigations of teamwork and have applications in the education and simulation arena.

Our first EFA specified a 10-factor model corresponding to the Salas Big Five framework, TeamSTEPPS survey, and Collective Orientation Scale.^1,16,17 Separate EFAs assessed specific 4-factor conflict structure. We used the principal factors approach and oblique rotation method. The principal factors method is a standard approach to extracting a meaningful factor structure and latent variables, and the oblique rotation assumes latent factors are correlated.²⁰ We performed multiple EFAs until we obtained a simple factor structure.²⁰

We retained items with a ≥0.40 factor loading to develop a short form and long form draft of the EMT-TEAMWORK survey.^20,21 We excluded items with high loadings on multiple factors.^20,21 We used Confirmatory Factor Analysis (CFA) to test if the items selected for the EMT-TEAMWORK-SF and EMT-TEAMWORK-LF surveys exhibited positive model fit characteristics.^20,21 We evaluated six standard measures of model fit (construct validity) and three standard measures of internal consistency (reliability): the Goodness of Fit Index (GFI), Bentler’s Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), Bentler & Bonett’s Non-normed Index (NNFI), Bentler & Bonnett’s Normed Fit Index (NFI), Standardized Root Mean Square Residual (SRMR), Cronbach’s alpha coefficients, and item-factor Pearson correlation coefficients, and Pearson factor-to-factor correlation coefficients. An SRMR less than 0.08, RMSEA less than 0.06, and GFI, CFI, NNFI, and NFI ≥0.9 are considered acceptable indices of instrument validity and model fit.^20,22-24 Cronbach’s alpha coefficient scores ≥0.70, item-scale Pearson correlations ≥0.40, and factor-to-factor Pearson correlation coefficients that are less than factor Cronbach’s alpha coefficients are evidence that each factor measures unique variance in the data.^25-28

We calculated factor summary scores by dividing the sum of item scores by total items retained for the factor. A global teamwork score arose by dividing the sum of factor summary scores by the total number of retained factors. We reverse scored conflict items so that a “Strongly Disagree” was connected to a 100 value – more positive/less conflict score – and a “Strongly Agree” with a 0 – a more negative/more conflict score. For purposes of calculating Pearson correlations, we inverted the scoring to match the valence of all non-conflict items (i.e., Strongly Disagree = 0). Using inverted scores, a negative Pearson correlation coefficient between teamwork constructs and conflict constructs would provide evidence that our survey distinguishes between positive and non-positive teamwork.²⁵ Conversely, a positive correlation would suggest that our specified model is an inaccurate (invalid) fit for the data.

We performed EFA analyses on a subset of completed surveys, excluding surveys with a “Does Not Apply” response to any item (Dataset A). We created two additional datasets (Dataset B and Dataset M) to: 1) evaluate need (utility) of a “Does Not Apply” response option over a standard neutral response (i.e., Neither Agree nor Disagree), and 2) evaluate impact of imputing mean versus neutral response options. In dataset B we treated surveys with a “Does Not Apply” response as a “Neither Agree nor Disagree” response. We excluded surveys from this imputation if the survey included a “Does Not Apply” response for all items assigned to a factor. This approach was used to identify and exclude potentially erroneous data. Mean item imputation is a widely used survey research technique for recovering items with missing data.²⁹ We used this technique, not to recover missing data, but to create a third dataset (Dataset M) to test the approach of replacing “Does Not Apply” responses with the mean item response from surveys included in dataset A. Discovery of low variation in teamwork scores across datasets A, B, and M would suggest that it is acceptable to impute “Neither Agree nor Disagree” neutral responses where respondents indicated “Does Not Apply.” SAS version 9.2 (Cary, North Carolina) was the designated statistical program.

RESULTS

We received completed surveys from 687 EMS personnel. The overall response rate was 30%, which ranged from 10% to 80% across 39 EMS agencies (mean agency level response rate was 33%). Across the total number of surveys (n= 687), nearly three quarters of respondents were male (72%; Table 1). Among respondents, the most common shift worked was 12-hour (42%) followed by 24-hour shifts (37%). Compared to non-respondents, respondents had greater mean years of agency service (7.2 vs. 5.9, p=0.0001), included a greater proportion of EMT-Paramedics (61% vs. 51%, p=0.0014), and greater proportion of full-time employees (72% vs. 63%, p<0.0001). The mean percentage of clinical work (non-administrative work) among respondents was slightly lower than among non-respondents (66% vs. 70%, p=0.02).

Table 1.

Demographic characteristics of study sample

Agency demographics	% (freq)	Respondent demographics	Respondents % (freq) mean (SD)	Non- Respondents % (freq) mean (SD)
Total Agencies	N=39	Total	N=687	N=1,627
Census Region		Sex
South	7.7% (3)	Female	27.7% (190)	---
Northeast	35.9% (14)	Male	72.3% (497)	---
Midwest	48.7% (19)	Certification *
West	7.7% (3)	Paramedic	61.4% (422)	50.9% (828)
Staffing Mix		EMT-Basic	28.7% (197)	36.0% (586)
All paid	69.2% (27)	Other	9.9% (68)	13.1% (213)
Mix paid and volunteer	23.1% (10)	Employment status*
All volunteer	5.1% (2)	Full-Time	71.7% (493)	63.3% (1,030)
Agency Classification		Part-Time	21.1% (145)	30.2% (491)
Hospital-Based	18.0% (7)	Volunteer	7.2% (49)	6.6% (106)
Fire-Based	7.7% (3)
Third Service	41.0% (16)	Most common shift worked
Rescue Squad	5.1% (2)	24 hour	36.5% (251)	---
Other	28.2% (11)	12 hour	42.1% (289)	---
Agency Ownership		8 hour	17.1% (117)	---
Private For-Profit	7.7% (3)	<8 hour	4.3% (30)	---
Private Non-Profit	69.2% (27)
Government Funded	17.9% (7)	Mean years service at agency*	7.2 (SD 7.1)	5.9 (SD 6.6)
Member Supported	2.6% (1)
Other	2.6% (1)	Mean % time doing clinical work	65.8 (SD 37.1)	69.9 (SD 39.0)

^*Table Notes: Indicates a p-value <0.05 for comparisons of proportions between respondents and non-respondents.

Respondents and non-respondents are affiliated with 39 diverse EMS agencies, half of which are located in the Midwest U.S. Census region (Table 1). Sixteen agencies self-classified as a third-service delivery model and 69% operated as a private non-profit.

Datasets A, B, and M were developed for purposes of our EFA and CFA analyses. Dataset A included 318 surveys after excluding any item with a “Does Not Apply” response. Datasets B and M included 548 surveys after using item recovery techniques described in our analysis section. The EFA analyses performed on all datasets independently determined that 14 of the 103 tested teamwork and conflict items failed to reach the ≥0.40 factor loading benchmark or loaded on to multiple factors. The remaining 89 items loaded onto a 9-factor structure that we labeled [1] Team Orientation (TO), [2] Team Structure & Leadership (TSL), [3] Partner Communication, Team Support, & Monitoring (PCTSM), [4] Partner Trust and Shared Mental Models (PTSMM), [5] Partner Adaptability & Back-Up Behavior (PABUB), [6] Process Conflict, [7] Strong Task Conflict, [8] Mild Task Conflict, and [9] Interpersonal Conflict.

CFA analysis linked 30 high loading items to the 9-factor structure of teamwork and conflict specified in the EFA findings. Values for CFA construct validity measures were positive and include: SRMR=0.016, RMSEA=0.056, GFI=0.93, CFI=0.97, NNFI=0.94, and NFI=0.95. Negative correlations between the five teamwork factors and four conflict factors are further support for construct validity (mean=−0.30, min=0.00, max=−0.69; Table 2). We labeled this 30-item survey model the EMT-TEAMWORK-Short Form (EMT-TEAMWORK-SF).

Table 2.

Construct correlation matrix

	Team Orientation (TC)	Team Structure & Leadership (TSL)	Partner Communication, Team Support, & Monitoring (PCTSM)	Partner Trust and Shared Mental Models (PTSMM)	Partner Adaptability & Back-Up Behavior (PABUB)	Process Conflict (PC)	Strong Task Conflict (STC)	Mild Task Conflict (MTC)	Interpersonal Conflict (IC)
TSL
ShortForm	0.05
LongForm	0.11*
PCTSM
ShortForm	0.23†	0.45†
LongForm	0.25†	0.54†
PTSMM
ShortForm	0.17†	0.41†	0.50†
LongForm	0.21†	0.42†	0.55†
PABUB
ShortForm	0.11*	0.32†	0.66†	0.30†
LongForm	0.18†	0.45†	0.73†	0.45†
PC
ShortForm	−0.18†	−0.35†	−0.41†	−0.69†	−0.33†
LongForm	−0.21†	−0.36†	−0.42†	−0.68†	−0.41†
STC
ShortForm	−0.17†	−0.31†	−0.42†	−0.68†	−0.27†	0.71†
LongForm	−0.20†	−0.32†	−0.41†	−0.69†	−0.35†	0.71†
MTC
ShortForm	−0.10*	−0.07	0.00	−0.21†	0.07	0.26†	0.37†
LongForm	−0.11*	−0.05	0.01	−0.21†	0.03	0.26†	0.37†
IC
ShortForm	−0.14*	−0.36†	−0.41†	−0.61†	−0.34†	0.69†	0.63†	0.20†
LongForm	−0.16†	−0.34†	−0.42†	−0.63†	−0.42†	0.69†	0.63†	0.20†
Overall Teamwork Score
ShortForm	0.41†	0.54†	0.68†	0.79†	0.54†	−0.81†	−0.81†	−0.43†	−0.75†
LongForm	0.43†	0.56†	0.69†	0.80†	0.63†	−0.81†	−0.81†	−0.43†	−0.76†

^*Table Notes:. = p-value <0.05.

^†= p-value <0.0001.

Correlations based on analysis of Dataset M (n=548).

CFA analyses of the survey that included the additional items linked 45 total items to the 9-factor structure identified in EFA analyses. Tests of model fit and construct validity of this 45 item survey support the 9-factor solution: SRMR=0.025, RMSEA=0.058, GFI=0.85, CFI=0.94, NNFI=0.90 and NFI=0.89. The mean correlation between the five teamwork constructs and four conflict constructs was −0.32 (min=0.03, max=−0.69; Table 2). We labeled this 45-item version of the survey the EMT-TEAMWORK-Long Form (EMT-TEAMWORK-LF).

The EMT-TEAMWORK-SF has positive internal consistency properties. The mean Cronbach’s alpha coefficients for the 9-scales was above the ≥0.70 benchmark (mean=0.84; min=0.78, max=0.94). The mean item-scale Pearson correlation was greater than the ≥0.40 benchmark for high reliability (mean=0.84; min=0.58, max=0.97). The mean factor-to-factor Pearson correlation among the five teamwork constructs was 0.32 (min=0.05, max=0.66), indicating that these factors measure unique variance in the data (Table 2).

The EMT-TEAMWORK-LF also exhibited positive internal consistency and reliability properties. The mean Cronbach’s alpha coefficient was 0.87 (min=0.79, max=0.94). The mean item-scale Pearson correlation was 0.80 (min=0.45, max=0.97). The mean factor-to-factor Pearson correlation among the five teamwork constructs was 0.39 (min=0.11, max=0.73).

There was wide variation in weighted scores across all 9 factors and the global teamwork score of the EMT-TEAMWORK-SF and EMT-TEAMWORK-LF (Table 3). One in every 15 respondents had a global teamwork score ≤50 while 42% scored ≤75.

Table 3.

EMT-TEAMWORK survey Short Form and Long Form summary scores

Construct	Mean	SD	Median	Min	Max
Team Orientation (TO)
Short Form (n=3 items)	48.1†	21.5†	50.0	0	100
Long Form (n=6 items)	49.3†	19.8†	50.0	0	100
Team Structure & Leadership (TSL)
Short Form (n=3 items)	85.8†	14.8†	83.3α	0	100
Long Form (n=6 items)	81.9†	14.3†	83.3†	0	100
Partner Communication, Team Support, & Monitoring (PCTSM)
Short Form (n=6 items)	78.0†	16.2†	80.6	8.3	100
Long Form (n=10 items)	79.2†	15.5†	81.6α	5.0	100
Partner Trust and Shared Mental Models (PTSMM)
Short Form (n=3 items)	85.0α	19.1†	88.9	5.6α	100
Long Form (n=6 items)	82.9α	16.9†	86.1α	22.2α	100
Partner Adaptability & Back-Up Behavior (PABUB)
Short Form (n=3 items)	74.7α	18.0†	77.8α	11.1	100
Long Form (n=5 items)	76.6†	16.0†	80.0α	23.3	100
Process Conflict (PC) (3-items)***
Short Form (n=3 items)	83.9α	18.5†	83.3	0	100
Long Form (n=3 items)	83.9α	18.5†	83.3	0	100
Strong Task Conflict (STC) (3-items)***
Short Form (n=3 items)	80.3α	21.1†	83.3	0∞	100
Long Form (n=3 items)	80.3α	21.1†	83.3	0∞	100
Mild Task Conflict (MTC) (3-items)***
Short Form (n=3 items)	54.1α	24.7†	50.0	0	100
Long Form (n=3 items)	54.1α	24.7†	50.0	0	100
Interpersonal Conflict (IC) (3-items)***
Short Form (n=3 items)	87.7†	18.1†	94.4α	0∞	100
Long Form (n=3 items)	87.7†	18.1†	94.4α	0∞	100
Total / Overall Teamwork Score (27-items)***
Short Form (n=30 items)	75.4†	12.0†	77.8†	20.7	99.4
Long Form (n=45 items)	75.2†	11.8†	77.6†	23.7†	98.1†

Table Notes: Scores based on analysis of Dataset M (n=548 surveys).

^***=Conflict scores are scored in reverse “Strongly Agree = 0 to Strongly Disagree = 100” when calculating the mean, std, median, minimum, and maximum for this table. We used these scores when calculating the Total / Overall Teamwork Score. This allowed the overall score to range from a minimum 0 to maximum 100.

^†=signifies that the standard deviation of scores across Datasets A, B, and M was greater than 0 and less than 1 percentage points,

^α= less than 5 percentage points,

^∞= less than 10 percentage points.

The use of mean item imputation, imputation of “Neither Agree nor Disagree” response, or exclusion of observations with a “Does Not Apply” response had no meaningful impact on factor or global teamwork score calculations (Table 3). The mean global teamwork score across all three datasets for the EMT-TEAMWORK-SF was 75.2 (min=74.3, max=75.8). The mean global score for the EMT-TEAMWORK-LF was 75.0 (min=74.2, max=75.6).

DISCUSSION

Recent data show that many EMTs work with 19 different partners annually – and some work with more than 50 partners in one year.¹⁴ Frequent turnover in partnerships can disrupt team cohesiveness, influence behavior, and threaten safety.^30-35 Improving safety in response to these potential threats to teamwork requires careful diagnosis of deficits in teamwork between partners. We developed the EMT-TEAMWORK survey to be EMT and dyad specific and to identify deficits in teamwork between EMTs. In this study sample, tests confirm both the short form and long form versions of the EMT-TEAMWORK survey have positive psychometric properties of reliability and construct validity. These findings are evidence the EMT-TEAMWORK survey measures multiple components of teamwork that in other settings have been found to impact safety, quality, and performance.^3,4

A previous study by Kalisch and colleagues used the Salas “Big Five” framework to develop a similar teamwork measurement tool specific to in-hospital nurses.¹⁸ Their approach was similar to ours; they began with 74 candidate items and used EFA techniques to identify a parsimonious factor structure. They then used CFA to develop a reliable and valid survey tool. Their EFA and CFA findings led them to identify five factors and 45 items for use in teamwork tool. The methods and findings of the Kalisch et al study are similar to our own and provide evidence in support of developing survey tools calibrated to the unique interactions of teammates in different occupations.

We are at the earliest stages of teamwork assessment in EMS. We designed the scoring of each domain (construct) and global score to range from 0 to 100 with higher scores indicating more positive perceptions. Table 3 provides an initial illustration of score variation for nine domains of teamwork and across respondents. We recognize that longitudinal study designs are needed to identify change in EMT-TEAMWORK survey scores in response to changes in partnerships over time and to test hypothesized associations between partner familiarity, teamwork, and safety outcomes. These studies may help determine cut points in domain scores where threats to patient and provider safety are elevated.

We have the following recommendations for those that wish to employ the EMT-TEAMWORK survey before the completion of additional research. First, we recommend that EMS managers administer the survey to EMTs for each unique partnership and document scores over time at the individual and team level. Second, memory of occupational experiences decays at a rapid pace.^36,37 We recommend managers administer the survey no more than two months post shift/partnership to obtain accurate responses to EMT-TEAMWORK items.³⁶ Third, the EMT-TEAMWORK LF and SF were developed for two different purposes. The EMT-TEAMWORK-LF includes additional items that make it better suited for research purposes. For example, investigators may use the EMT-TEAMWORK-LF to evaluate the impact of experimental team training programs in the formative or continuing education settings. On the other hand, because of its brevity (roughly 5 minutes), the SF form seems more appropriate for day-to-day human resource purposes. Fourth, confidentiality of responses may be an issue that affects participation and validity of responses from EMTs. We recommend managers offer assurances to EMTs that their scores will be held confidential and will not be shared with their partners. See Appendix A for copies of both the SF and LF EMT-TEAMWORK surveys.

LIMITATIONS

At the individual respondent level, our study sample is analogous to samples of EMS workers in other studies in age, sex, employment status, and level of certification (Table 4). However, our results cannot be generalized to determine the level of teamwork in EMS agencies across the United States. However, generalizability of these scores was not an aim of this study. Our primary aim was to develop a valid and reliable survey tool for evaluating teamwork between EMS provider partnerships at the dyadic level – not agency level. The dyadic configuration of EMS partnerships is the most common structure of EMS workers and is not limited to one type of delivery model. Tables 1 and 4 highlight the diversity of our study sample at the agency level, which we believe strengthens application of the EMT-TEAMWORK survey to most EMS delivery models. Modifications in item wording would be necessary for EMS agencies that configure teams of three or four.

Table 4.

Characteristics of study sample compared to other research studies of EMS workers

Characteristic	Study Sample		LEADS Sample38	High Response EMS Sample39	NHTSA Workforce Report40 **	Resuscitation Outcomes Consortium Agencies41	Longitudinal Study of EMS Turnover42	National Survey of EMS Safety Culture12
	Respondents (n=687)	Non- Respondents (n=1,627)
Individual Characteristics
Sex
Male	72.3%	---	72.9%	71.8%	71-77%	---	---	73.2%
Female	27.7%	---	27.1%	28.2%	23-29%	---	---	26.8%
Certification
EMT-Basic	28.7%	36.0%	58.1%	50.7%	#72%	58.2%	---	19.4%
Paramedic	61.4%	50.9%	34.6%	49.3%	#22%	34.2%	---	62.1%
Mean Age in Years	36.9	---	---	---	35	---	---	---
Employment Status
Full-Time	71.7%	63.3%	---	---	$89%	---	---	77.6%
Part-Time	21.1%	30.2%	---	---	$11%	---	---	20.6%
Volunteer	7.2%	6.6%	---	---		---	---	1.8%
Agency Affiliation (EMT unit of measurement)	Fire includes Rescue squads				+Includes
Fire-Based	11.8%	7.9%	34.1%	---	county/3rd	---	---	---
County/3rd Service	60.0%	50.6%	12.1%	---	+30%^^	---	---	---
Hospital	16.2%	20.1%	9.9%	---	20%^^	---	---	---
Other	12.0%	21.5%	43.8%	---	50%^^	---	---	---
Mean Years of Service at Agency	7.2 (SD 7.1)	5.9 (SD 6.6)	---	---	---	---	---	---
Mean Percentage of Work devoted to Clinical Field Work (not administrative)	65.8 (SD 37.1)	69.9 (SD 39.0)	---	---	---	---	---	---
Agency Characteristics (Agency Unit of Measurement)
Agency Affiliation
Fire-Based	(+ Rescue Squad) 10.5%		---	---	---	62.5%	10.0%	11.5%
County/3rd Service	39.5%		---	---	---	25.7%	22.5%	19.7%
Hospital	18.4%		---	---	---	N/A	27.5%	29.5%
Other (i.e. private)	29.0%		---	---	---	11.8%	40.0%	39.3%

^**The NHTSA Workforce report includes statistics based on data from the 2003 and 2005 Current Population Survey (CPS), the 2007 National Registry of EMTs (NREMT) registration database, the 2004-05 Edition of the Bureau of Labor Statistics Occupational Outlook Handbook.

^#=indicates source was the 2007 NREMT database statistics cited within the NHTSA Workforce report and excludes statistics from EMT-Intermediates.

^$=is used to indicate data from this source does not stratify EMTs by volunteer status and automatically labels an EMT as Full-Time based on the EMT working greater than or equal to 35 hours per week.

^{^^}Categories reported in the NHTSA Workforce report are not completely analogous to the stratums defined in this study. We collapsed several categories in the NHTSA Workforce report deemed similar to stratums in this study (e.g. 50% Other includes “private ambulance services and ‘other’ in the Workforce Report; and 30% County/3^rd service includes all types of local government types).

^@=The mean of medians reported in Table 2 of the Davis et al, 2007 publication.

For the ROC agency affiliation, the categories are Fire-Based, Non-fire government (county/3^rd service), and Private (other).

We identified differences in respondents and non-respondents that may have an impact on scores calculated for the 9 factors and global score of the EMT-TEAMWORK survey. Part-time and volunteer EMTs were more common among non-respondents. In some paid agencies, part-timers may be used to fill empty or unfilled shifts or called upon on an as needed basis. Their perceptions of one or more components of teamwork may be lower and contribute to a drop in the mean weighted scores for one or more component scores of the EMT-TEAMWORK survey. This concern extends to agencies with all-volunteer personnel who represent a small proportion of respondents. Again, these differences in respondents and non-respondents do not impact the development of the tool or findings from psychometric tests. These differences may impact scores across the 9 factors and global score measured by the EMT-TEAMWORK survey.

CONCLUSIONS

We developed a reliable and valid survey that can evaluate nine components of teamwork between EMT partners. The EMT-TEAMWORK survey scores can be used to diagnose deficits in teamwork between newly formed to well-established partnerships.