Psychometric properties of the Chinese version of the TeamSTEPPS teamwork perceptions questionnaire to measure teamwork perceptions of Chinese residents: a cross-sectional study

Abstract

Objectives

The purpose of this research was to evaluate the psychometric properties of the TeamSTEPPS Teamwork Perception Questionnaire (T-TPQ) among the Chinese residents.

Design

Cross-sectional study.

Setting

A clinical hospital of the China Medical University in Liaoning Province, China.

Participants

A total of 664 residents were enrolled in this research. The valid response rate was 83.0% (664 of 800 residents).

Main outcome measures

Internal consistency and test–retest reliability were used to assess the reliability of the questionnaire. The construct validity of the Chinese T-TPQ was evaluated by confirmatory factor analysis. Furthermore, the concurrent, convergent and discriminant validity were analysed.

Results

Cronbach’s α coefficient of the T-TPQ in Chinese language was 0.923. Except for the communication dimension (0.649), the Cronbach’s α coefficient of all dimensions were satisfactory. The T-TPQ and its five dimensions reported a good test–retest reliability (0.740–0.881, p<0.01). Moreover, the results of the confirmatory factor analysis demonstrated that the construct validity of the Chinese T-TPQ was satisfactory. All dimensions significantly correlated with the Hospital Survey on Patient Safety Culture (HSOPSC) teamwork within units dimension and the Safety Attitudes Questionnaire (SAQ) teamwork climate dimension (p<0.01), and the questionnaire showed satisfactory convergent and discriminant validity.

Conclusions

The T-TPQ in Chinese language demonstrated good psychometric characteristics and was a reliable and valid questionnaire to measure the Chinese health professionals’ perception of teamwork. Thus, the Chinese version of the T-TPQ could be applied in teamwork training programmes and medical education research.

Strengths and limitations of this study.

• To our knowledge, this is the first study to translate and validate the TeamSTEPPS Teamwork Perception Questionnaire (T-TPQ) in China.

• In our study, we used international standards to examine the psychometric properties of the Chinese version of the T-TPQ.

• This study provides a benchmarked instrument, which may act as a basis for future studies on teamwork perception in Chinese medical education.

• This study was conducted only in one university hospital, therefore study population may not be representative of all the Chinese health professionals.

Introduction

Teamwork is important for improving healthcare quality and increasing patient safety. Effective teamwork in healthcare not only augments patient’s satisfaction but also decreases burnout among the health professionals.^1–4 The Agency for Healthcare Research and Quality (AHRQ), the Joint Commission and other institutions ranked teamwork as a dominant factor for enhancing the healthcare quality.^{5 6} Furthermore, teamwork has been classified as an important competency to help optimise the healthcare services and poor teamwork could increase the medical errors and reduce patient safety.^7–11 As there is a great need in improving patient safety and healthcare quality, team training has been widely recognised to enhance teamwork.^12–15 However, health professionals today are not competent in teamwork, and team training has not attracted the attention of medical institutions.^16–18 The Chinese Hospital Association has reported that adverse patient events consumes extensive medical resources every year in Chinese healthcare institutions.^{19 20} Therefore, cultivating teamwork competency in healthcare professionals has become a crucial and urgent factor for improving the patient safety in China.

Evaluation of competency of health professionals’ in teamwork has proven to be important in both, team training and medical education^{21 22} and an inaccurate evaluation may lead to unreliable conclusions.²³ Teamwork evaluation is recommended as a key process in residency training.²⁴ For the past few years, much research has been conducted on evaluating the perceptions of health professionals to better understand their teamwork competency. Therefore, a good measuring tool is especially important to evaluate teamwork perceptions of the health professionals.^25–27 The TeamSTEPPS Teamwork Perception Questionnaire (T-TPQ) is one of the most frequently applied tools used for such evaluation. It is a self-report questionnaire, which evaluates perceptions of a healthcare professional on group-level teamwork situation in a medical team.²⁸

The T-TPQ was developed by American Institutes for Research developed in year 2010.²⁸ When considering the importance of teamwork in healthcare and medical education, the AHRQ developed the TeamSTEPPS. TeamSTEPPS is a teamwork training course and framework which is useful for improving quality of healthcare and reducing medical errors.^{29 30} The T-TPQ was based on the five important teamwork factors of the TeamSTEPPS, including team structure dimension, leadership dimension, situation monitoring dimension, mutual support dimension and communication dimension. The questionnaire was cross-culturally validated in different countries and languages, including the USA,³¹ Norway,³² Korea,³³ Brazil³⁴ and Scotland,³⁵ among others. Furthermore, the T-TPQ has shown to be reliable and valid tool among the physicians, nurses, medical students, residents and pharmacists.^31–35 All versions of the T-TPQ contains the same content, with minor modifications to reflect the clinical practices.

The T-TPQ in Chinese language was translated by our research team.³⁶ In adapting to the Chinese version, we followed the process of translation and adaptation as suggested by WHO guidelines for validation of the scale.³⁷ In this, the main steps were: forward translation, specialist review, back-translation, pretesting, cognitive interviewing and formation of the questionnaire. So far, no research had used the questionnaire in healthcare professionals of China, therefore, the psychometric properties of the Chinese T-TPQ have not yet been assessed. The purpose of this research was to evaluate the psychometric properties of the T-TPQ among the Chinese residents. The results of our research may be useful to fully understand the teamwork perception of residents and other healthcare professionals in China. Further, the findings of this research may be helpful to other countries in developing the T-TPQ for their medical institutions and healthcare professionals.

Methods

Participants and procedures

For this study, 800 residents from three grades were recruited, and the study was conducted between June 2018 and October 2018 at a clinical hospital of the China Medical University, Shenyang, China. The training programme of Chinese residents typically lasts for about 3 years, during which, young residents acquire the knowledge and skills of their specialties or subspecialties, and develop attitudes, behaviours, habits and values that are helpful for their subsequent professional life. Most studies and assessments are based on the clinical context.

The paper version of the questionnaires was handed out to the residents on-site, and each participant completed a self-administered questionnaire. Every questionnaire was coded by number (eg, 1, 2, 3) after participants submitting their questionnaire. Experienced researchers then checked the questionnaires and verified if there were any invalid/incomplete questionnaires. A questionnaire was deemed as invalid if more than 20% of the data were missing. Of the 800 residents, 664 completed the questionnaire, the valid response rate was 83.0%. The study size was based on the item per participant ratio of 1:10 principle.³⁸ A previous study reported that the sample size to evaluate test–retest reliability coefficient was 52.³⁹ In our study, a total of 72 respondents were randomly selected to answer the questionnaire, and again after 2 weeks, 60 among them completed the questionnaire.

Measures

The questionnaire comprised of four parts including basic information (gender, age, marital status, grade and monthly income), the Chinese version of the T-TPQ, the Hospital Survey on Patient Safety Culture (HSOPSC)⁴⁰ and the Safety Attitudes Questionnaire (SAQ).⁴¹ The T-TPQ assessed the respondents’ perception of group-level teamwork competency in a department, and it consisted of 35 items in five dimensions namely—teamwork structure, leadership, situation monitoring, mutual support and communication. Each dimension contained seven items, the response to which were given on a 5-point Likert scale (1=disagree strongly to 5=agree strongly). The T-TPQ and its dimension scores were computed to an average score.²⁸

The HSOPSC and the SAQ are generic scales for patient’s safety measurement that are reliable and valid to evaluate a hospital’s teamwork and patient safety.^{42 43} The HSOPSC consists of 42 items in 12 dimensions, and the SAQ consists of 36 items in six dimensions. A dimension of the HSOPSC (teamwork within units dimension) and a dimension of the SAQ (teamwork climate dimension) were used to test the concurrent validity in this study. The two dimensions were scored on a 5-point Likert scale (ranging from 1=disagree strongly to 5=agree strongly). The reliability of these two dimensions were found to be satisfactory (Cronbach’s α coefficients were 0.891 and 0.909, respectively).

Statistical analysis

Twenty-four missing data distributed in 18 respondents. To satisfy the requirements of the study, each item’ missing data were replaced by the median value of all item scores in the relevant dimension.³² Descriptive statistics (mean, SD, skewness (Sk), kurtosis (Ku), floor and ceiling effects) were performed on all items and dimensions in the Chinese version of the T-TPQ. The absolute values of Sk and Ku higher than 3 and 10, respectively, showed a significant deviance from a normal subjects distribution.^{38 44 45} If the percentage of items with the lowest or the highest score was more than 20%, floor or ceiling effects were considered as significant. In our study, the Cronbach’s α coefficient was computed to evaluate the internal consistency of the T-TPQ. The internal consistency was deemed to be acceptable when the Cronbach’s α coefficient value was higher than 0.7.^46–48 The test–retest reliability was evaluated by the intraclass correlation coefficient (ICC) with the two-way random model. If the ICC was higher than 0.7, the test–retest reliability was considered satisfactory.⁴⁹

Regarding construct validity, the original five-factor model of the T-TPQ was tested using a confirmatory factor analysis (CFA). CFA is a robust method of statistical analysis to test a predetermined factor structure or a hypothetical theory, and it can describe how well each item evaluates the measure’s dimensionality. In many studies, it has been suggested that the CFA is very important for scales that have been culturally adapted.^{50 51} The goodness of fit was assessed through the following indicators: the χ² goodness of fit (χ²), the root mean square error of approximation (RMSEA), the Comparative Fit Index (CFI) and the Adjusted Goodness-of-Fit Index (AGFI); among which the RMSEA was considered as the best index.^{32 51} If the RMSEA was below 0.08, and the CFI was higher than 0.90, we deemed that it was a good fit. An AGFI value higher than 0.85 was deemed a satisfactory model fit.⁵⁰ The correlations between each dimension of the T-TPQ were evaluated by computing the Pearson’s correlation coefficient.

To test the concurrent validity, the Pearson’s correlation analysis of T-TPQ with the HSOPSC teamwork within the units dimension and the SAQ teamwork climate dimension was conducted. Regarding convergent and discriminant validity, we only used the T-TPQ for this analysis. A Pearson’s correlation coefficient of >0.4 for an item with its respective dimension indicated satisfactory convergent validity.^{52 53} Items showing lower correlations with other dimensions than those with their respective dimensions showed satisfactory discriminant validity.⁵⁴ In this research, we used SPSS V.20.0, AMOS V.21.0 software of the Windows. A p value<0.05 was defined to be statistically significant.

Patient and public involvement

Patients or the public were not involved in the design, conduct or the analysis of our research.

Results

Characteristics of respondents

The valid response rate of the overall research was 83.0% (664/800) and that of retest was 83.3% (60/72). The average age of the respondents was 25.83 years (SD=1.61). Most respondents were woman and more than half of the participants were urban residents. The sociodemographic characteristics of responders have been summarised in table 1.

Table 1.

Basic characteristics of respondents (n=664)

Characteristics	Number	%
Gender
Male	228	34.3
Female	436	65.7
Age
≤25 years old	312	47.0
>25 years old	352	53.0
Residence
Urban	429	64.6
Rural	235	35.4
Grade
1	205	30.9
2	266	40.0
3	193	29.1
Marital status
Married	34	5.1
Unmarried	630	94.9
Monthly income (¥)
≤1000	156	23.5
1001–2000	494	74.4
2001–3000	14	2.1
Region
Eastern China	425	64.0
Central China	154	23.2
Western China	85	12.8

The mean of the T-TPQ among the Chinese residents was 4.10±0.37. Regarding dimensions, the team structure dimension had the highest score (4.24±0.44), while the mutual support dimension had the lowest score (3.95±0.45). The score of the T-TPQ was as shown in online supplemental material 1. All items and dimensions displayed acceptable Sk (1.02 to −0.07) and Ku (−0.38 to 3.53) coefficients. No significant floor effects were observed in all items and dimensions. None of the dimensions showed significant ceiling effects. However, most items displayed significant ceiling effects, except for items 15, 16, 26, 27, 28 and 33.

Reliability

The internal consistency of the overall T-TPQ was excellent. All dimensions had satisfactory internal consistency, except the communication dimension, which was slightly below the acceptable internal consistency coefficient (0.700). The split-half reliability coefficient for the total T-TPQ was satisfactory (0.843). Additionally, the test–retest reliability of the Chinese T-TPQ was satisfactory, and the ICC of all dimensions was good. These results are shown in table 2.

Table 2.

Reliability of the T-TPQ in Chinese language

Dimensions	Cronbach’s α coefficient (n=664)	ICC (95% CI) (n=60)
Team structure	0.801	0.877 (0.749 to 0.948)**
Leadership	0.831	0.749 (0.507 to 0.917)**
Situation monitoring	0.820	0.740 (0.530 to 0.908)**
Mutual support	0.720	0.849 (0.702 to 0.932)**
Communication	0.649	0.745 (0.449 to 0.910)**
T-TPQ	0.923	0.881 (0.783 to 0.945)**

**P<0.01.

T-TPQ, TeamSTEPPS Teamwork Perception Questionnaire.

Construct validity

In this study, the CFA was performed to test the five-factor model, which displayed an acceptable fit with the data (χ²=1815.176, df=550, p<0.001; CFI=0.837; RMSEA=0.059 (90% CI: 0.056 to 0.062); and AGFI=0.829). Except for the eight items (items 1, 2, 26, 27, 28, 32, 33 and 35), all other items had an acceptable factor load with its respective dimension (factor load >0.5), and the path coefficients between each dimension were acceptable, as displayed in figure 1.

Figure 1.

Overview of the structure of the Chinese version of the T-TPQ based on CFA. The results of CFA demonstrated that the construct validity of the Chinese version of T-TPQ was satisfactory. CFA, confirmatory factor analysis; T-TPQ, TeamSTEPPS Teamwork Perception Questionnaire.

Correlations among the dimensions of the questionnaire

The Chinese version of the T-TPQ showed a significant correlation between each dimension of the questionnaire. The correlations among the dimensions of the questionnaire were determined by the Pearson’s correlation coefficient and are shown in table 3.

Table 3.

Correlations among the dimensions of the T-TPQ in Chinese language (n=664)

Dimensions	Team structure	Leadership	Situation monitoring	Mutual support	Communication
Team structure	–	0.667**	0.625**	0.517**	0.565**
Leadership		–	0.641**	0.495**	0.474**
Situation monitoring			–	0.619**	0.548**
Mutual support				–	0.532**
Communication					–

**P<0.01.

T-TPQ, TeamSTEPPS Teamwork Perception Questionnaire.

Concurrent validity

The correlation coefficients of the T-TPQ with the HSOPSC teamwork within units dimension and the SAQ teamwork climate dimension were as shown in table 4. The total questionnaire and its five dimensions significantly correlated with the two subscales. All the correlation coefficients were higher than 0.40, except the association between ‘mutual support’, ‘communication’ dimension and the HSOPSC teamwork within units dimension, and the association between the ‘mutual support’ dimension and the SAQ teamwork climate dimension. These findings showed acceptable concurrent validity of the adapted Chinese version of the T-TPQ.

Table 4.

Concurrent validity of the T-TPQ in Chinese language (n=664)

Dimensions	Pearson’s correlation coefficient
	HSOPSC teamwork within units dimension	SAQ teamwork climate dimension
Team structure	0.465**	0.446**
Leadership	0.511**	0.506**
Situation monitoring	0.497**	0.501**
Mutual support	0.384**	0.398**
Communication	0.360**	0.419**
T-TPQ	0.551**	0.563**

**P<0.01.

HSOPSC, Hospital Survey on Patient Safety Culture; SAQ, Hospital Survey on Patient Safety Culture; T-TPQ, TeamSTEPPS Teamwork Perception Questionnaire.

Convergent and discriminant validity

In this study, convergent and discriminant validity of the T-TPQ was analysed. The Pearson’s correlation coefficients for each item with its respective dimension were satisfactory (>0.4) and showed a good convergent validity. For the discriminant validity, all items displayed a higher correlation with their respective dimensions than with other dimensions of the T-TPQ, which were satisfactory (table 5).

Table 5.

Summary of the convergent and discriminant validity (n=664)

Dimensions	Pearson’s correlation coefficient range		Convergent validity		Discriminant validity
	r1	r2	Success/total	%	Success/total	%
Team structure	0.541–0.743**	0.236–0.586**	7/7	100	7/7	100
Leadership	0.632–0.741**	0.276–0.566**	7/7	100	7/7	100
Situation monitoring	0.608–0.749**	0.331–0.486**	7/7	100	7/7	100
Mutual support	0.571–0.655**	0.228–0.478**	7/7	100	7/7	100
Communication	0.537–0.624**	0.100–0.495*	7/7	100	7/7	100

r₁, the correlation coefficients for each item with its respective dimension; r₂, the correlation coefficients for each item with other dimensions of the T-TPQ.

**P<0.01; *P<0.05.

T-TPQ, TeamSTEPPS Teamwork Perception Questionnaire.

Discussion

Standardised and effective evaluation of the teamwork is critical to improve the perceived functioning of a medical team.^{55 56} The results of our research showed that the Cronbach’s α coefficient of the Chinese T-TPQ was 0.923. Except the communication dimension, the Cronbach’s α coefficient of all dimensions were satisfactory. The T-TPQ reported a good test–retest reliability. Moreover, the construct validity of the questionnaire was satisfactory. The Chinese version of the T-TPQ showed acceptable concurrent validity, satisfactory convergent and discriminant validity.

The response rate of the current study was 83.0%, which was similar to that of the Korean study,³³ and was better than the Norwegian study.³² All items and dimensions of the Chinese version of the T-TPQ displayed acceptable Ku and Sk coefficients, and the floor effects of all the items and dimensions were below the accepted threshold of 20%. However, in our study, several items showed significant ceiling effects. The ceiling effect of our research was understandable, as most residents felt more comfortable when their teamwork perception was good and approved by others.⁴⁴ In our study, we found that the Chinese T-TPQ had a satisfactory internal consistency, that was similar to other cross-cultural studies performed in different countries and regions, for instance, Norway, Korea and the USA.^31–33 Our study showed that the Cronbach’s α coefficient of the questionnaire was relatively high, and that the internal consistency of most dimensions was satisfactory. A good internal consistency of the questionnaire suggests that most items and dimensions measured the same concept, namely the perceptions of residents about the teamwork at their typical workplace. The satisfactory Cronbach’s α coefficient values on the dimensions illustrate the high internal consistency of the total questionnaire.⁵⁷ In line with research findings among the healthcare personnel of Norway,³² our study showed that the Chinese T-TPQ had good a test–retest reliability. In some previous studies, it was suggested that test–retest reliability can be used to evaluate the temporal fluctuations.⁵⁸ Many measurement experts suggest that, in healthcare studies, compared with internal consistency, the test–retest reliability is considered to be of more significance.⁵⁸

The CFA showed that the original five-dimension structures of the T-TPQ provides a generally satisfactory fit for our research data, and the result was in lines with the previous validation study of T-TPQ.^{31 32} Our results revealed that, based on the goodness-of-fit indices, the construct validity of the Chinese T-TPQ was acceptable. We found that the RMSEA index was 0.059, indicating a good fit. The model derived in this study was a better fit compared with the Norwegian study by Ballangrud et al³² (RMSEA index=0.069) and the study by Keebler et al³¹ (RMSEA index=0.068). The CFI (0.837) and AGFI (0.829) were slightly below the cut-off values for a satisfactory evidence of model fit. However, the literature suggests that the most effective and informative criteria is RMSEA in covariance structure modelling.^{32 51} The Norwegian study by Ballangrud et al³² reported a CFI of 0.833, which was similar to that in our study, while, the study by Keebler et al³¹ displayed a better CFI (0.925). The study samples may have had an impact on the findings of these researches, and a larger sample size may have led to a better fit within this data.⁵⁹ The sample size of American study of participants from the US Army medical facilities was 1700.³¹ A total of 247 healthcare personnel in different hospitals responded to the Norwegian study by Ballangrud et al.³² The sample size in our research included 664 residents. The factor load of each item with its respective dimension were acceptable, except the eight items (items 1, 2, 26, 27, 28, 32, 33 and 35), and the path coefficients among the dimensions were also acceptable. The study of T-TPQ performed in American healthcare settings showed better factor load than that in our study.³¹ Keebler et al suggested that some items within T-TPQ dimensions, such as items 26 and 27 under the mutual support dimension, containing highly similar content may lead to their corrected errors, which should be modified to get the better model fit indices.³¹ Overall, our results suggest that the model of the Chinese version of the T-TPQ was appropriate for the future studies in China.

Significant correlations were observed among the dimensions of the Chinese version of the T-TPQ. Significant correlations between each dimension were also revealed by the research in the America and Norway.^{31 32} Our results showed that the correlation coefficient between team structure and leadership was the highest, indicating that if a medical team had a better team structure, the leadership of the team could be improved. The Norwegian study by Ballangrud et al³² revealed that the highest correlation coefficient was between the team structure and communication. The American study³¹ displayed that the situation monitoring strongly correlated with the mutual support, thereby showing that the situation monitoring skill of health professionals could be enhanced by improving the mutual support.

Concurrent validity was shown to have significant correlations with the HSOPSC teamwork within units dimension and the SAQ teamwork climate dimension. Our study reported that the T-TPQ and its five dimensions significantly correlated with the HSOPSC teamwork within units dimension (r=0.360–0.551, p<0.01), and the SAQ teamwork climate dimension (r=0.398–0.563, p<0.01). However, the correlation with the HSOPSC teamwork within units dimension was bit lower than the findings of the validation study by the American Institutes for Research (r=0.60–0.81, p<0.01).²⁸ More specifically, our study demonstrated that the convergent and discriminant validity of the Chinese T-TPQ was satisfactory. These results are important and noteworthy, because if one item reported a better correlation with one of the other dimensions, than with its initially assigned dimension, then it could be argued that this item should be modified or reassigned to the other dimension.⁶⁰

The strength of our study is that we provided a Chinese language version of the T-TPQ, which may act as a basis for the future studies on teamwork perception and climate in healthcare and medical education setting of China. However, there were some limitations in our study. First, the respondents of the research were from only a single medical institution in China. Moreover, the findings of our study were limited by the representativeness and scale of the study population. Furthermore, the nature of the cross-sectional study hinders with an appropriate assessment of this questionnaire’s sensitivity to change. Third, the factor loadings of some items were lower than the critical value, especially some items in the dimensions of ‘mutual support’ and ‘communication’. These items may have cultural adaptability problem, which may be further studied. The Chinese version of the T-TPQ seems to be promising benchmark that is instrumental for future studies focusing on teamwork in healthcare settings in China. Further studies could enhance representativeness by expanding the respondent’s diversity and sample size. Determination of teamwork perception is of clinical relevance. We thereby suggested the researchers of medical education to consider using this questionnaire for teamwork studies not only among the residents but also other healthcare professionals of China. The T-TPQ could also be used to identify interprofessional teamwork in healthcare setting, and it may suggest researchers to use this questionnaire for the relevant studies of China in the future. For future studies, the T-TPQ could be used as a measuring tool to verify the impact of training programmes related to teamwork on healthcare professionals.

Conclusions

In this study, we evaluated the psychometric properties of the T-TPQ in Chinese language. Our findings provided evidence that the T-TPQ in Chinese language is a reliable and valid questionnaire for measuring teamwork perception of the Chinese residents, and in cross-cultural comparative studies on the teamwork perception of health professionals. It can therefore be applied in teamwork training programmes and medical education research.