Measuring engagement in nurses: the psychometric properties of the Persian version of Utrecht Work Engagement Scale

Abstract

Background: Considering the overall tendency in psychology, researchers in the field of work and organizational psychology have become progressively interested in employees’ effective and optimistic experiments at work such as work engagement. This study was conducted to investigate 2 main purposes: assessing the psychometric properties of the Utrecht Work Engagement Scale, and finding any association between work engagement and burnout in nurses.

Methods: The present methodological study was conducted in 2015 and included 248 females and 34 males with 6 months to 30 years of job experience. After the translation process, face and content validity were calculated by qualitative and quantitative methods. Moreover, content validation ratio, scale-level content validity index and item-level content validity index were measured for this scale. Construct validity was determined by factor analysis. Moreover, internal consistency and stability reliability were assessed. Factor analysis, test-retest, Cronbach’s alpha, and association analysis were used as statistical methods.

Results: Face and content validity were acceptable. Exploratory factor analysis suggested a new 3- factor model. In this new model, some items from the construct model of the original version were dislocated with the same 17 items. The new model was confirmed by divergent Copenhagen Burnout Inventory as the Persian version of UWES. Internal consistency reliability for the total scale and the subscales was 0.76 to 0.89. Results from Pearson correlation test indicated a high degree of test-retest reliability (r = 0. 89). ICC was also 0.91. Engagement was negatively related to burnout and overtime per month, whereas it was positively related with age and job experiment.

Conclusion: The Persian 3– factor model of Utrecht Work Engagement Scale is a valid and reliable instrument to measure work engagement in Iranian nurses as well as in other medical professionals.

↑ What is “already known” in this topic:

While it has been shown work engagement differs among various professions, it is revealed that it also has a negative association with burnout level.

→ What this article adds:

Psychometric properties of Persian version of the Utrecht Work Engagement Scale was satisfactory. Thus, as a valid tool, it can be used for measuring work engagement of Iranian Nurses.

Introduction

Considering the overall tendency in psychology (1), researchers in the field of work and organizational psychology have become progressively interested in employees’ effective and optimistic experiences at work. For instance, besides burnout, researchers have focused on its complementary concepts such as work engagement (2). Work engagement was primarily theorized by Kahn (1990). He defined it as the “attaching of organization memberships to the work roles: in engagement, people employ and express themselves physically, cognitively, emotionally and mentally during role performances.” Kahn suggested that engagement is presumed to develop affirmative consequences both at the individual and at the organizational level (3).

According to Schaufeli and Bakker, work engagement is described by vigor, dedication, and absorption. Vigor is defined as high levels of power and psychological tolerance while working. An intelligence of importance, eagerness, inspiration, pride, and challenge is defined as dedication. As a final point, absorption is considered by being fully focused on and happily absorbed in one's duty, so time passes rapidly and the individual has problems with leaving their own duty (4). The potential consequences of work engagement that are related to optimistic outlooks to the work and the organization include work satisfaction, organizational commitment, and low interest in turnover (5). Work engagement differs in various occupations: a high level was observed in teachers, managers, artisans, nurses, salespeople, and farmers, whereas a low level was found among workers in assembly line, retailers, typists, household workers, and police officers (6).

Regarding the influence of health system quality on the health of human beings, it has been proved that development of this system is of paramount importance in every society. This would need healthy and motivated health care workers. Among these specialists, the nurses constitute a large body of skillful health care providers. Therefore, hospitals are faced with achieving constantly higher quality patient outcomes, while having growing economic limits. Studies of organizations with high-performance propose that success is achieved when engaged staff proactively solves problems of their own performance. In addition, increases in nurses’ scores of engagement have been determined to have a better effect on patient outcomes than growths in nurse staffing (7). Based on the results of some studies, work engagement level varies among different wards; for example, in intensive care unit, the engagement of nurse specialists was higher (8). In addition, work engagement had a negative association with burnout level (9).

To put it differently, it seems vital to make some changes into the work characteristics to decrease burnout by increasing work engagement in nurses, and more importantly, to evaluate the effectiveness of these changes. To do so, reliable validated scales are needed (10,11). According to the results of many researches, the most generally used scale of work engagement in the original studies is the Utrecht Work Engagement Scale (UWES) (12). The strength and applicability of the concept of work engagement have been confirmed in various cultures (4,9). The UWES has 17 items and uses a 4-item Likert-type scale (1 means never and 4 means always) and includes 3 subscales: Vigor (6 items), dedication (5 items), and absorption (6 items) (13). This instrument was mostly accepted in international studies with different translated versions such as Italian, Norwegian, Japanese, and Spanish (4,9).

However, in spite of the importance of the concept in health care specialists including the nurses, the absence of rigorous validation studies of the UWES in the Persian context is obvious.

As a result, the current study aimed at examining the psychometric properties of the Persian version of the Utrecht Work Engagement Scale (P-UWES). The aims were as follow: (1) translation and cultural adaptation; (2) assessing face and content validity; 3) evaluating the factorial validity, in which after cultural adaptation Confirmatory Factor Analysis (CFA) changes into Exploratory Factor Analysis (EFA) to explore a favorite structural model; 4) examining the reliability of the scale using Cronbach’s alpha and intraclass correlation coefficient (ICC); 5) examining the summary of work engagement across demographic subgroups ( job experiment, overtime per month and age); and 6) exploring the divergent validity through the correlation between work engagement and validating variables such as burnout.

Methods

Design and Participants

This was a methodological research. Participants were employees in a general hospital affiliated to Dezful University of Medical sciences. Recommendations about an acceptable sample size for factor analysis are different, and some authors suggest 5 cases per item and others 10 (14). In this study, 300 questionnaires were distributed among the nurses working in different wards. Those nurses who were qualified based on the study criteria were given the questionnaire. The inclusion criteria were tenure in their current position ranging from 2 months to 30 years (15). Data collection was performed during June and July 2015. Study objectives and the procedure, and questionnaire items were explained to the nurses by the main researcher who was a master’s student. Then, demographics (age, sex, educational level, marital status, and occupational characteristics), UWES, and CBI questionnaires were given to the participants. The current study was conducted in several main steps that had been summarized as aims of the study in above sentences.

After receiving permission from the original designer of the UWES, the items of the scales were translated. This phase was carried out in accordance with WHO guidelines (16). The steps to implement this method are as follow:

Forward translation: The English version of the questionnaire was translated into Persian by 2 translators proficient in both languages (English and Persian); expert panel: Bilingual experts were employed to compare the 2 translated versions and prepare the final version after applying a few changes; Back-translation: The final translation of UWES was translated back into English by 2 other translators, both of whom were proficient in both languages; pre-testing and cognitive interviewing: Ten nurses were randomly selected from the target population to test the tentative final version; final version: After applying some small revisions, the final Persian version of the questionnaire was designed; documentation: All the cultural adaptation processes are apparent through the proper documents, and then the translated questionnaire underwent the validation process.

To evaluate the face validity of the UWES, the main researcher asked the 15 nursing staff to complete the items of the scale in the presence of the researcher, and their commentaries and suggestions were revised by the expert team and considered in the final version. To assess content validity, the qualitative well as the quantitative methodology were assessed. Thus, 10 experts (2 experts in clinical psychology, 2 psychiatric nurses, 2 nursing management trainers, and 4 nursing assistant professors expert in development of relevant instruments) were invited to revise the translated questionnaire. All the revisions were collected after 2 weeks. The comments were carefully evaluated, and some of the items were reworded. Moreover, we measured content validation ratio (CVR), scale-level content validity index (SCVI), and item-level content validity index (IVCVI) for this scale.

Then, considering that after cultural adaptation CFA changes into EFA to explore a favorite structural model (14), after translation and cultural adaptation process and some predictable problems, we conducted EFA, instead of CFA, via principal components analysis followed by a varimax rotation to assess the factor constructs of all the 17 items for the UWES-Persian version subscales. Then, the researchers conducted EFA among the variables several times to find the acceptable model to fit, and finally a new 3-factor model was selected by item- to factor loading > 0.4, applying varimax rotation, eigenvalues > 1 (17). Acceptance of the 3 – factor model was based on this fact, implying that confirmatory factor analyses shows that the hypothesized 3-factor structure of the UWES is superior to the one-factor model and fits well to the data of various samples (6,9).

For assessing reliability, the internal consistency of the P-UWES was assessed via the Cronbach’s alpha coefficient. Values of Cronbach’s α ≥ 0.70 were explained as satisfactory internal consistency (18). Stability reliability of test- retest and ICC was determined via completing 30 questionnaires of the Persian version of P-UWES by nurses in 2-week intervals in the presence of the same main researcher. ICC values of 0.40 or above were considered satisfactory (r≥ 0.81-1.0 as excellent, 0.61- 0.80 very good, 0.41-0.60 good, 0.21- 0.40 fair, and 0-0.20 poor) (19).

Study measurements

UWES: The English version of the scale was achieved from the designer’s website (http://www.shaufeli.com/ Test_forms.htm). The scale has acceptable psychometric properties and contains 3 engagement subscales: vigor (6 items), dedication (5 items), and absorption (6 items). The sum of all items can be considered as a total score for engagement. The value of the items are on a 7 point scale from 0 (never) to 6 (always). The average scores of each subscale items were calculated by totaling the item scores and then dividing them by the number of items in each subscale. Therefore, the potential range for each subscale and the total score can be 0-6. Higher scores mean higher levels of engagement. The UWES scores are classified as very low, low, average, high, and very high. This classification has been done according to the engagement scores acquired from an international sample, which is explained in the UWES Manual (6). Considering the cut-off point for work engagement based on the guidelines of the designers, the scores below 3.07 were considered as low level of work engagement. Measures of eigenvalues confirmed acceptable values for 3 factors (13,20).

P-CBI: In the current study, burnout was evaluated by the Persian version of Copenhagen Burnout Inventory (P-CBI). Four- factor P-CBI has 19 items and uses a 5-item Likert-type scale (0 means never or a very low degree, and 100 means always or a very high degree) and includes 4 subscales of personal burnout (7 items), work-characteristic-related burnout (3 items), work-distaste-related burnout (3 items) and client related burnout (6 items). Coefficient of Cronbach’s alpha for its subscales before factor analysis was 0.87 to 0.89 and it was 0.84 to 0.89 after it. The Intra-class coefficient correlation for test-retest reliability of the subscales of personal burnout, work-characteristics-related burnout, work-distaste, as well as client-related burnout was 0.95, 0.84, 0.83, and 0.90 (p < 0.001), respectively (21). Scoring of P-CBI was considered as follows: Always: 100; often: 75; sometimes: 50; seldom: 25; and never/almost never: 0. The total score of the scale was not reported and score of each subscale was calculated and reported separately (22).

Ethical Consideration

Ethical authorization and agreement were obtained from the research and ethics committees of Dezful University of Medical Sciences. Written informed consents were also acquired from all the participants who took part in the study.

Statistical Analysis

Exploratory factor analysis (EFA) was performed using LIEREL8.5 to evaluate the scale’s structure. Exploratory principal component factor analysis (EFA) was analyzed by employing the varimax rotation method. Other statistical analyses were conducted using IBM SPSS 20. To assess the reliability of the P-UWES, indicators of internal consistency, Cronbach’s alpha coefficients were examined. Independent t tests were used to compare the level of the P-UWES across gender and age group. The construct validity of the P-UWES was evaluated through partial correlations between the P-UWES and the validation variables. It was anticipated that the P-UWES be negatively correlated with burnout. Data normality was obtained by Kolmogorov-Smirnov test, and the results revealed normality of the sample.

Results

In the present study, 300 questionnaires were distributed; out of which, 282 questionnaires were returned (response rate = 94%). Those questionnaires that had not been returned were equally from all wards, so they did not disturb the balance of sampling the study. Women accounted for 87.9% of the population, and the mean (SD) age, over time hours, and job experiments were 33.06 (8.2), 62.2 (33.3) and 8.7 (7.5), respectively. More details of the demographic and work characteristics of the participants are summarized in Table 1.

Table 1. Demographic and Work Characteristics of the Nurses (n = 282) .

Variable	Category	N	%
Sex	Female	248	87.9
	Male	34	12.1
Educational level	BS	274	97.2
	MSc	8	2.8
Marital status	Single	100	35.5
	Married	182	64.5
Type of employment	formal employment	54	19.1
	employment contract	84	29.8
Type of shift	just morning	46	16.3
	rotational	220	78
Type of over time	optional	102	36.2
	non-optional	180	63.8

The findings of the quantitative content validity revealed that all of the items had impact scores ≥ 1.5. According to the expert panel opinions, qualitative content validity was accepted with some minor modifications; and CVR of this scale was >1.5, ICVI was >0.87, and SCVI was about 0.91.

In construct validity, as mentioned in the third step of the methods section, after translation and cultural adaptation, we conducted EFA to achieve the well-construct model of UWES-Persian version. Therefore, Bartlett’s test and KMO were examined in the primarily data to reach sample adequacy, Bartlett’s test of sphericity (p< 0.001 χ2 = 2686.5 and KMO = 0.895).

According to factor loading and Eigenvalue’s cut-point in the methods section, no item was deleted, but some items were dislocated among the 3 factors. The new 3-factor model [modified UWES] was extracted as the best model, accounting for 61.2% of the total variance. The new 3-factor model contains 17 items with the Likert scale (0-6 scores) (Table 2).

Table 2. Summary of results of the exploratory factor analysis and Cronbach’s alpha values of the Iranian nursing staffs (n = 282) .

Factors and Items	Factor loading	Cronbach’s alpha
Factor1: Vigor		0.89
Feel bursting with energy	0.723
Work full of meaning	0.696
Feel strong at job	0.610
Enthusiastic about job	0.868
Inspires with job	0.720
Like going to work in the morning	0.667
Immersing in work	0.735
Factor 2: Dedication
Proud of the work	0.554	0.80
Job is challenging	0.462
Resilient at job	0.818
Persevere at work	0.654
Factor 3: Absorption
Time flies when working	0.475	0.76
Forgetting around when working	0.673
Feel happy when working intensely	0.527
Can work for long periods	0.566
Carried away when working	0.766
Difficult to detach from job	0.719
Total Engagement Scale		0.84

The results of the reliability test revealed that all of the Cronbach’s alpha coefficients (0.89 for vigor, 0.80 for dedication, 0.76 for absorption, and .84 for total scale) were higher than 0.70 (Table 2). In addition, Pearson correlation test of the scale confirmed the correlation between the responses in the 2 weeks’ interval (r = 0.89); and the ICC was achieved 0.91, and it was good to excellent for the subscales (0.64-0.85). The quantitative analysis was concluded after approving the Persian version of UWES as a valid and reliable scale. Table 3 shows the correlations between the P-UWES and the validating variables, burnout, job experiment, and over time per month. Positive relations were found between the P-UWES and job experiment at the weak level (r= .063 to .18, p<.05). The P-UWES was negatively related to over time per month (r = −.18 to −.23, p<.01) at the weak level. In addition, negative correlations of stronger level were found between the P-UWES and P-CBI subscales (r = −.29 to −.47, p<.01).

Table 3. Correlations of the new three-factor UWES subscales with P-CBI subscales, participants’ age, job experiment and working over time per month (Pearson correlation coefficient) (N = 282) .

P-CBI and Other Related Factors	P-UWES
	Subscales
	Vigor	Dedication	Absorption
Personal burnout	- 0.456**	- 0. 387**	- 0. 283**
Work-characteristic-related burnout	- 0.467**	- 0.409 **	- 0. 278**
Work-distaste-related burnout	- 0.444**	- 0. 271**	- 0. 254**
Client-related burnout	- 0.490**	- 0. 487 **	- 0. 288**
Participant age	0.112*	0.176**	0.135*
Job experiment	0.063	0.128*	0.092
Over Time per Month	- 0.232**	- 0.178**	- 0.218**

** Correlation was significant at 0.01 levels (1-tailed)

* Correlation was significant at 0.05 levels (1-tailed)

Table 4 demonstrates the descriptive statistics of the P-UWES through gender and age subgroups. However, male and female nurses did not vary significantly in the total score of engagement (t= -1.31, p= 0.2), vigor (t = -0.2, p = 0.23), dedication (t = -1.48, p= 0.15), and absorption (t = -1.57, p= 0.25). A significant difference was found across age groups in the total score of engagement (t= -3.08, p= 0.002), vigor (t = -3.43, p = 0.017), dedication (t= -3.48, p = 0.001), and absorption (t = -2.3, p= 0.022), respectively.

Table 4. Descriptive statistics of the UWES-C across demographic subgroups .

mean	By Gender		By Age Groups
	Male (N=34)	Female (N=248)	Age<30 (N=148)	Age>30 (N=134)
Engagement	3.76(1.44)	4.1(0.94)	3.88(1.1)	4.25(0.82)
Vigor	4(1.31)	4.24(1.06)	4.06(1.2)	4.37(0.94)
Dedication	4.04(1.71)	4.49(1.01)	4.22(1.25)	4.67(0.91)
Absorption	3.23(1.54)	3.55(1.22)	3.35(1.43)	3.69(1.03)

Discussion

The aim of questionnaire adaptations and translations was to produce a questionnaire that is equal to the original one. In the cross-cultural assessment, equivalence is as significant as reliability and construct validity. Beaton, Bombardier, Guillemin, and Bosi Ferraz (2000) focused on 4 situations in which a cross-cultural adaptation is necessitated. The fourth situation includes adaptations for use in a different country by people who speak a different language. In this situation, in addition to cultural adaptation, translation is required (19). Based on these facts, translation and cultural adaptation were conducted in the first step of the current study and were matched and confirmed by the original English version, so that it could be similar to the original scale both conceptually and linguistically. In general, the findings of the study showed that P-UWES is a valid and reliable scale for measuring work engagement.

Face validity was approved by target groups’ (nurses) satisfaction with the excellence of the scale items. Both quantity and quality of the content validity were confirmed by expert panel designs with slight modifications, which were in line with Schaufeli and Bakker (4).

Construct validity was evaluated by EFA. Because methodological problems may be predictable through translation of questionnaires for cross-cultural studies (23), researchers consented to modify the model (14). CFA was replaced by EFA to find an acceptable construct model (23). Therefore, at first, the Kaiser – Mayor-Olkin (KMO) measure confirmed sample size adequacy for exploratory factor analysis, as Anderson explained the KMO value more than 0.6 is acceptable for factor analysis (23). Finally, by conducting several exploratory factor analyses, a new model was determined. This new model was constructed based on some criteria, one of which was having a factor loading >0.4. Garson maintained that factor loadings of <0.4 are weak and factor loading >0.6 are very strong (24). This new model of UWES has 3 factors containing 17 items that were confirmed by both the statistical evidence (61.2% of total variance) and theoretical framework. Pett et al. suggested that a new construct should explain 60% of the total variance (25). Although primary finding of this study was supported by Bilgel et al’s study, it did not use EFA to develop a new model (9). However, in many studies CFA revealed that the 3-factor models of UWES were preferred to the one – factor model; moreover, they had more satisfactory fit indices in many countries such as Netherlands, Spain, and Portugal (4).

After EFA, divergent validity was conducted between P-UEWS and P-CBI. In this research, findings reported that work engagement and burnout had a significant negative correlation. The findings are in line with those of Sebaufeli & Bakkar on an original designer of UWES (4) as well as Beligil et al study in Turkey that has cultural similarities with Iran (9). Also, the findings of the current study are in line with an American study in which critical care nurses’ data showed a significant negative correlation between burnout and engagement in all subscales as well as the total scores of UWES (r= - 0.49; p<0.05) (8). An exciting finding was that work engagement was more strongly correlated with client –related burnout and personal burnout than the nature of work and work distaste –related burnout. However, this finding may indicate an adjacent relationship between client and personal aspects of burnout and work engagement; and it may be ascribed to the pattern of intrinsic thought for the professional nurses. This hypothesis was almost confirmed by Shaufeli and Bakker (2003) although they used dissimilar work engagement and burnout tools (Maslach Burnout Inventory-Student Survey and UWES-Student) (6), supporting the construct validity of the P-UWES.

As accepted, P- UWES revealed more satisfactory internal consistency reliability than the original model, which could be due to the similarity between data and model and conducting psychometric steps properly (18). In other words, model modification provides the ground for elevating internal consistency among the questionnaire items (20). A well-developed instrument shows Cronbach’s α ≥ 0.7, which is an acceptable level of internal consistency (23). The Cronbach’s alpha of each subscale is evenly matched to the obtained values in the other studies especially that of the original scale (6,9). Besides, scale stability was assessed via intraclass coefficient (test- retest), which is supported by other studies (9).

Findings to examine the summary of work engagement across demographic subgroups (job experience, overtime per month, and age) suggest that nurses who have more job experiment were probably experiencing higher levels of work engagement (positive but weak relations between work engagement and job experiment). The point that work engagement was negatively but weakly correlated with over time per month appears to imply that having more over time could reduce the level of work engagement. Generally, the results are consistent with prior studies (6,9). Although the female nurses seemed to exhibit higher levels of engagement than the male, the gender comparison was not statistically significant. The contrast through age subgroups explored a reliable characteristic that older nurses showed significantly higher levels of engagement. The positive correlation between age subgroups and engagement was in line with results from prior research (9).

The total mean score of work engagement was moderate, which is not only in line with Mason et al. study but it is also higher than some other studies (8,6). Comparison of the norm scores of work engagement was found from 12 161 samples from 9 various countries, which are available in the UWES Manual (6). The nurses in the current study showed higher levels of work engagement. Among the subscales, the highest mean at work engagement was assigned to dedication, and the lowest mean to absorption. Schufeli and Bakkar suggested that vigor and dedication are the core concepts of work engagement (4). However, it seems that absorption may be a consequence of work engagement (26). Likewise, other studies conducted about Iranian nurses have revealed that the leadership style of the managers is effective on work engagement (27). In addition, significant relationships were obtained between work engagement component and organizational commitment. Overall, several factors including personality, organizational, and cultural traits can influence work engagement and its components (28). There were several limitations in the current study. First, the study sample was based on a specific occupational field and site, and the participants were mainly female. Whereas many studies have discovered the factors of occupational work engagement and burnout in nurses (9), the current study did not contain any measures on the job demands of the nurses. However, work characteristics such as high job demand, low job control, and low social support as well as organizational factors such as lack of communication, collaboration, resources, and bad organizational culture have been defined as work engagement and burnout factors, which should be considered seriously (29). Additionally, the self-report survey nature of this study may result in the problem of common process difference.

Conclusion

The result of the present study revealed that the new 3-factor model of UWES is a valid and reliable instrument for measuring work engagement. Face validity was approved by a group of nurses’ opinion about the quality of the instrument items. Likewise, content validity was approved by a comprehensive team of expert panel’s comments and conducting few changes in some items measuring some quantitative indices. After these two steps, construct validity was assessed by EFA and divergent validity. In this study, response rate was 94%, while the scale response rate of more than 50% is satisfactory.

The P-UWES is a valid and reliable instrument to measure work engagement in Iranian nurses as well as in other medical professionals. P-UWES could help health centers’ managers and decision- makers to determine the work engagement, related factors, and programs for enforcement. In addition, P-UWES has technical benefits such as having short questions and easy response, easy scoring, as well as easy interpretation. The P-UWES can help find and foresee problems and factors that are related to work engagement. It is hoped that the confirmation and validation of the P-UWES will not only support a superior understanding of work engagement concept among Iranian nurses and other health care workers but also motivate regional and international research cooperation on work wellbeing and engagement. Considering the limitations of the current study, the authors suggest that future research explore the relationship between the P-UWES and many job demands, and work and personality characteristics of the employees to explain the construct. Moreover, it is recommended that future studies combine objective procedures to evaluate the level of work engagement.

Acknowledgement

We would like to express our thanks to the research deputy of Dezful University of Medical Sciences for supporting our study. Moreover, the authors wish to thank Dr. Schaufeli, the designer of the public instrument.

Conflict of Interests: The authors declare no conflict of interest in this study.

Cite this article as: Torabinia M, Mahmoudi S, Dolatshahi M, Abyaz MR. Measuring engagement in nurses: the psychometric properties of the persian version of Utrecht Work Engagement Scale. Med J Islam Repub Iran. 2017 (28 Feb);31:15. https://doi.org/10.18869/mjiri.31.15