Abstract
Background
The ProQoL (30 items) is a widely used instrument of work-related quality of life for health care workers. Recently, a shorter 9-item version of the ProQoL was developed and validated among palliative care workers. The ProQoL-9 consists of three subscales: compassion satisfaction (CS), burnout (BO), and compassion fatigue (CF). Care aides (personal support workers, nursing assistants) are an understudied population in terms of their professional quality of life. It is critical to use validated instruments to measure their experiences. The purpose of this study was to examine the internal consistency and factorial structure of the ProQoL-9 among care aides working in LTC.
Methods
We used surveys collected by the Translating Research in Elder Care (TREC), a pan-Canadian program that collects longitudinal surveys from the healthcare workforce in the LTC. We used TREC surveys containing information on demographics, characteristics of LTC homes (e.g., ownership model), and the ProQoL-9. Our sample included all care aides who completed TREC surveys in the province of Alberta, Canada, from 2020–2021. We examined internal consistency via alpha and omega coefficients. To examine the factorial structure, we conducted confirmatory factor analysis (CFA) testing one factor, two factors (CF and BO together & CS), and three factor models (CF, BO, and CS).
Results
N = 760 care aides completed the surveys in Alberta. The majority were female (90.79%) and worked in general LTC units (55.29%). The Cronbach’s alpha results showed an overall α = 0.56 for the whole scale and adequate reliability of the subscales (α = 0.73 for CS, α = 0.68 for CF, and α = 0.75 for BO). The omega reliability results for all the subscales were ≥ 0.70, reflecting good internal consistency (BO = 0.77, CS = 0.73, and CF = 0.70). The three-factor model had the best goodness of fit values, reflecting an adequate goodness of fit (X2 = 165.82, DF = 24, X2/DF = 6.9, P < .0001, RMSEA = 0.08, CFI = 0.92, TLI = 0.89).
Conclusions
The ProQoL-9 is a valid and reliable instrument among care aides in LTC. The factorial structure shows that this shorter version of the ProQoL is rigorously designed and can be utilized by health service researchers in LTC.
Keywords: ProQ0L-9, Care aides, Factorial structure
Background
Job related burnout has gained the attention of researchers in recent years, particularly since the COVID-19 pandemic [1]. Care aides, sometimes referred to as nursing assistants (personal support workers), are unregulated health care workers that make up the majority of the workforce in Long-Term Care (LTC). Care aides are prone to many stressors and are at risk of job burnout [2]. The Professional Quality of Life (ProQoL) instrument is often used by researchers to measure burnout and other measures of the quality of life for individuals employed in helping professions such as nurses and care aides. The original ProQoL was first developed by Stamm (30 items) and argued to have two subscales: Compassion Fatigue (CF) and Compassion Satisfaction (CS). The CF sub scale was argued to further be composed of sub scales Burnout (BO) and secondary traumatic stress. Higher score on CS reflects a better sense of satisfaction from working as a care aide. Higher CF and BO scores reflect higher risk for fatigue due to care giving and higher risk of BO, respectively [3].
While the ProQoL (30 items) has been widely used in various studies, it has been under scrutiny since it was developed. Researchers argued that items are not well constructed. In particular, the factor structure of the ProQoL has been under question [4]. A recent meta-analysis conducted on the factorial structure of the ProQoL using the results of 27 international studies encouraged researchers to use a parsimonious version of the ProQoL [5]. The ProQoL-9 was developed to address the need for a shorter tool that is theoretically and psychometrically sound. The shorter version of the ProQoL (ProQoL-9) consists of 9 items which has three subscales—CS, CF, and BO—and became available in 2020. The ProQoL-9 was initially validated among Spanish palliative care providers. The goal of the ProQoL-9 was to provide a shorter version while retaining items that adequately represent the ProQoL subscales [6]. To date, no study (other than the developers of the ProQoL-9) has examined the factorial structure of the ProQoL-9. A few studies have examined the reliability of the ProQOL-9, which was found to be adequate [6, 7]. One such study assessed the impact of COVID-19 on healthcare workers in the ICU. The reliability analysis revealed good alpha values for CS (α = 0.84), CF (α = 0.84), and BO (α = 0.68) [7].
To build psychometric evidence for the ProQoL-9, variation that exists within different health systems and healthcare workers should be captured [8]. However, no psychometric evidence of the ProQoL-9 has been reported for care aides working in LTC settings. While the ProQoL-9 developers hypothesize a 3-factor solution, because there has been only one study using the ProQoL-9 reporting psychometric properties, we proposed to examine the reliability and find the best model fit for the ProQoL-9 among care aides working in LTC for 3 models. The research aims were to examine:
1)A one-factor model in which all the items are loaded into a single factor representing the ProQoL-9.
2)A two-factor model consisting of a joint CF and BO factor as well as an independent CS factor.
3)Three-factor model consisting of CF, CS, and BO.
Methods
We used data from the Translating Research in Elder Care (TREC) program in this retrospective study. TREC is a pan-Canadian health service research program that has collected longitudinal data from regulated and unregulated healthcare workers in LTC since 2008. The goal of TREC is to improve the quality of care for residents and healthcare professionals working in LTC [9]. To date, TREC has completed seven waves of data collection. For this study, we used Wave-6 data collected during the COVID-19 pandemic (2021–2022) from care aides. Our participants included care aides in the province of Alberta, Canada. Eligibility criteria was to have worked in the same unit in the LTC home for ≥ 3 months with at least 6 shifts monthly [9]. We had conducted cognitive interviewing prior to administering the ProQoL-9 surveys to ensure that care aides understand the complex wording of items. We invited five care aides with ESL for the interviews. Results showed that care aides understood the items well. Only after concluding that care aides understood the items sufficiently, we use the ProQoL-9 for our study. This study was approved by the Ethics Board at the University of Alberta Pro00037937.
Statistical analysis
We used descriptive statistics, including means, standard deviations, and ranges for continuous variables and frequencies and percentages for categorical variables.
Internal consistency
We assessed internal consistency using Cronbach's alpha and omega. We considered Cronbach's alpha [10] and omega values ≥ 0.70 as acceptable [11]. We computed both Cronbach's alpha and omega reliability testing because α is concerned with correlation between items, whereas omega [11] better reflects the true population estimate [12]. We calculated the item‒total correlation to determine the degree to which each survey item contributes to the consistency of the ProQoL-9 [13]. We also computed inter-item correlation to assess how each ProQoL item score is related to all other items in the ProQoL-9. This was done to examine whether the items are capturing the same content [14].
Factorial structure
To examine the factorial structure, we conducted confirmatory factor analysis (CFA) using three a-priori factor structures outlined in the aims. CFA allows the examination of internal structural validity [15]. To determine goodness of fit, we used a variety of indices including chi-square, Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), and Tucker–Lewis Index (TLI). A CFI and TLI cutoff value closer to 0.95 will lead to lower levels of type II error [16]. Therefore, to assess the model fit, we followed these criteria: CFI > 0.90 and TLI > 0.90 to reflect adequate fit and CFI > 0.95, TLI > 0.95 to reflect good fit [17]; RMSEA closer to 0.053 to demonstrate good fit [18]. We used SAS Enterprise Guide version 8.4 for all the analyses.
Results
Care aides’ characteristics
The total sample was 760 care aides from the province of Alberta, Canada. The majority of the care aides were female (90.79%), were ≥ 40 years old (74.61%), had English as their second language (75.23%), and worked in the general LTC unit (55.29%). The care aides had high levels of compassion satisfaction (12.98 ± 2.02), medium levels of burnout (6.93 ± 2.73) and low levels of compassion fatigue (5.51 ± 2.25). The demographic characteristics of the care aides and the LTC home characteristics are presented in Table 1.
Table 1.
Characteristics of care aides in our sample (N = 760)
| Variable | ||||
|---|---|---|---|---|
| Care aides’ characteristics | N | % | ||
| Age | ||||
| < 30 | 51 | 6.71 | ||
| 30–39 | 142 | 18.68 | ||
| 40–49 | 253 | 33.29 | ||
| 50–59 | 203 | 26.71 | ||
| > = 60 | 111 | 14.61 | ||
| Gender | ||||
| Male | 68 | 8.95 | ||
| Female | 690 | 90.79 | ||
| Other/missing | 2 | 0.26 | ||
| English as the first language | ||||
| Yes | 188 | 24.77 | ||
| No | 571 | 75.23 | ||
| M | SD | Min | Max | |
| Years of experience | 11.98 | 8.75 | 0 | 43 |
| ProQoL subscales (possible range 1–15) | ||||
| Burnout | 6.93 | 2.73 | 1 | 15 |
| Compassion Fatigue | 5.51 | 2.25 | 3 | 14 |
| Compassion satisfaction | 12.98 | 2.02 | 4 | 15 |
| N | % | |||
| LTC home characteristics | ||||
| Zone | ||||
| Edmonton | 460 | 60.53 | ||
| Calgary | 300 | 39.47 | ||
| Size | ||||
| Large > 120 beds | 556 | 73.16 | ||
| Medium (80–120 beds) | 126 | 16.58 | ||
| Small (< 80 beds) | 78 | 10.26 | ||
| Ownership model | ||||
| Private for profit | 222 | 29.21 | ||
| Public not for profit | 144 | 18.95 | ||
| Voluntary not for profit | 394 | 51.84 | ||
| Unit type | ||||
| General LTC | 418 | 55.29 | ||
| Non-secure dementia | 56 | 7.41 | ||
| Secure dementia | 157 | 20.77 | ||
| Secure mental health/psychiatric | 19 | 2.51 | ||
| Other | 106 | 14.02 | ||
Internal consistency
The Cronbach’s alpha for the whole scale of the ProQoL-9 was 0.56. For the subscales, Cronbach’s alpha was 0.68 for CF, 0.73 for CS, and 0.75 for BO. The omega coefficient value was 0.77 for BO, 0.73 for CS, and 0.70 for CF. Item characteristics are summarized in Table 2.
Table 2.
Item characters and reliability of ProQoL-9
| Variable | Item (range 1–5) | α1 | CITCR2 | MICC3 |
|---|---|---|---|---|
| CF1 | I think I have been affected by the traumatic stress of my residents (CF) | 0.38 | 0.50 | 2.03 |
| BO1 | I feel trapped by my job as a care aide (BO) | 0.44 | 0.33 | 1.69 |
| CS1 | I like my work as a care aide (CS) | 0.56 | −0.03 | 4.43 |
| CF2 | I feel depressed because of the traumatic experience of my residents (CF) | 0.41 | 0.41 | 1.85 |
| CS2 | My work makes me feel satisfied (CS) | 0.59 | −0.14 | 4.17 |
| BO2 | I feel worn out because of my work as a care aide (BO) | 0.42 | 0.40 | 2.52 |
| BO3 | I feel overwhelmed because my workload seems endless (BO) | 0.41 | 0.41 | 2.72 |
| CF3 | As a result of my work, I have frightening intrusive thoughts (CF) | 0.44 | 0.34 | 1.63 |
| CS3 | I am happy that I chose to do this work (CS) | 0.57 | −0.07 | 4.37 |
1 α Cronbach’s α coefficient with deleted variables, 2CITCR Corrected item-total correlation, 3MIIC Mean Inter-Item Correlations
Confirmatory factor analysis
We tested one, two, and three-factor models using CFA. The one-factor model represented all 9-items loaded together showing poor goodness of fit indices (RMSEA = 0.16, CFI = 0.74, TLI = 0.64). The two-factor model also did not show adequate fit (RMSEA = 0.10, CFI = 0.89, TLI = 0.85). While the model fit improved in the two-factor model, it was still not within the acceptable cutoff. The three-factor model had the best goodness of fit indices among the three models and overall showed adequate fit indices (RMSEA = 0.08, CFI = 0.92, TLI = 0.89) based on the criteria we had initially set out (CFI > 0.90 and TLI > 0.90). The model fit indices are presented in Table 3. We also present diagrams for the three tested models in Fig. 1.
Table 3.
Model fit indices from the confirmatory factor analysis of three models
| Model | X2 | DF | X2/DF | P | CFI | TLI | RMSEA | 90%CL |
|---|---|---|---|---|---|---|---|---|
| One factor | 537.38 | 26 | 20.66 | < .0001 | 0.74 | 0.64 | 0.16 | 0.15–0.17 |
| Two-factor | 229.25 | 26 | 8.81 | < .0001 | 0.89 | 0.85 | 0.10 | 0.09–0.11 |
| Three-factor | 165.82 | 24 | 6.9 | < .0001 | 0.92 | 0.89 | 0.08 | 0.07–0.10 |
Fig. 1.
Graphical summary of one-factor, two-factor and three-factor models. A One factor model. B Two factor model. C Three factor models
Discussion
This is the first study, of which we are aware, to investigate the factorial structure of the ProQoL-9 in care aides (or any group) working in LTC settings. The majority of the participants were female with ESL and worked in large size LTC home. Since in countries such as Canada, the majority of care aides working in LTC are immigrants and have English as their second language [19], the results generated from this study may be applicable to other regions and countries with similar composition of care aide workforce working in LTC settings.
The Cronbach’s alpha for the ProQoL-9 as a whole scale did not meet the threshold that we had set for adequate internal consistency (≥ 0.70). However, all the ProQoL-9 subscales showed good internal consistency, as evidenced by the alpha and omega coefficients. While Cronbach’s alpha is inadequate for estimating reliability, we used it in this study because health care researchers often use and expect this reliability index in psychometric studies. Cronbach’s alpha should be interpreted with caution, as the omega coefficient provides a better estimate of reliability [20].
Our results confirm that the three-factor ProQol-9 consisting of subscales of CF, CS, and BO had the best factorial structure compared to the one and two-factor models. The only psychometric study on the ProQoL-9 factorial structure was conducted with palliative care health care workers. The results of this study revealed an adequate factor structure (3-factor model) among the Spanish speaking palliative care professionals in Brazil, Argentina, and Spain (CFI = 0.92), with good internal consistency (all of the subscales had α > 0.70). The other factorial structure studies have been conducted on the longer versions of the ProQoL. These studies support a three-factor model factorial structure consisting of CS, BO, and secondary traumatic stress [21, 22].
Validation studies on the factorial structure of the ProQoL are valuable. The ProQoL, published by Stamm, has information on only Cronbach’s alpha, reflecting its reliability, without providing any information on validity, especially the factorial structure of the ProQoL [1]. Because care aides are not well-studied and there is a growing interest in understanding the quality of life among health care professionals, this study provides an opportunity for researchers to use the ProQoL-9 with emerging evidence of factorial validity as well as reliability. We recommend that future researchers validate the ProQoL-9 in other settings in which care aides typically work, such as acute and hospice care. Managers and directors of care in LTC homes may benefit from this validation study and be interested to use the ProQoL-9 as a management tool to gain insight into staff work-related quality of life.
Strenghts and Limitations
A strength of our study is our large sample size, a novel participant group and setting, and both reliability and validity assessment. The care aides in our sample also had diverse backgrounds (e.g., age and English as the first language). We investigated reliability indices via the omega coefficient, which adds to the strength of our psychometric analysis. Our study has some limitations. The majority of our participants were women. Therefore, the results may not be applicable for health care occupations with other gender identities. We conducted this analysis in Alberta LTC homes, where contextual factors influencing professional quality of life may differ from those in other provinces or jurisdictions.
In this manuscript, we did not investigate the influence of demographics on response differences and factorial structure of ProQoL-9, thus it is possible that demographic characteristics such as age and length of service may influence the results.
Conclusion
The internal consistency assessment showed acceptable levels for the ProQoL-9 subscales. The three-factor model of ProQoL-9 with subscales of CF, CS, and BO demonstrated adequate fit. This validation study is an important step towards using the ProQoL-9 in LTC research.
Authors’ contributions
All authors contributed to the concept and design of the study. SS conducted the analysis based on AW recommendations and expertise. All authors reviewed analysis and contributed to the draft of the manuscript.
Funding
This work was funded by the Alberta Ministry of Health (AHW011810 Estabrooks) and the Canadian Institutes of Health Research (RES0053356 Estabrooks).
Data availability
The data used for this article are housed in the Health Research Data Repository (HRDR) in the Faculty of Nursing at the University of Alberta, in accordance with (a) the health privacy legislation of participating TREC jurisdictions and (b) ethics approvals of universities and institutions participating in TREC. The data were provided under specific data sharing agreements only for approved use by TREC within the HRDR. Where necessary, access to the HRDR in order to review the original source data may be granted to those who meet pre-specified criteria for confidential access, available from data unit in TREC.
Declarations
Ethics approval and consent to participate
We conducted our study in alignment with the Declaration of Helsinki. We obtained ethics approval from the Ethics Board at the University of Alberta (#Pro00037937). We explained the study purpose, risks and benefits to the participants using the informed consent form approved by ethics board at the University of Alberta. We clarified that participation is voluntary prior to obtaining consent. We obtained informed consent from all participants.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Hotchkiss JT, Wong CMY. Truth on ProQOL and Burnout Assessment—Development and Intercultural Validation of the Brief ProQOL-12 from the Professional Quality of Life, Version 5. Am J Hosp Palliat Medicine®;0(0):10499091241260284 10.1177/10499091241260284. [DOI] [PubMed]
- 2.Duan Y, Song Y, Thorne TE, Iaconi A, Norton PG, Estabrooks CA. The Complexity of Burnout Experiences among Care Aides: A Person-Oriented Approach to Burnout Patterns. Healthcare (Basel) 2023;11(8). 10.3390/healthcare11081145. Published Online First: 20230417. [DOI] [PMC free article] [PubMed]
- 3.Stamm B. The Concise ProQOL Manual: The concise manual for the Professional Quality of Life Scale , 2 nd Edition, 2010.
- 4.Geoffrion S, Lamothe J, Morizot J, Giguère C. Construct Validity of the Professional Quality of Life (ProQoL) Scale in a Sample of Child Protection Workers. J Trauma Stress. 2019;32(4):566–76. 10.1002/jts.22410. Published Online First:20190702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hotchkiss JT, Wong MYC. Factorial Structure of the ProQOL—Systematic Meta-analysis and Integration of 27 International Factor Analysis Studies. Trends in Psychology 2022.10.1007/s43076-022-00184-5
- 6.Galiana L, Oliver A, Arena F, et al. Development and validation of the Short Professional Quality of Life Scale based on versions IV and V of the Professional Quality of Life Scale. Health Qual Life Outcomes. 2020;18(1):364. 10.1186/s12955-020-01618-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Moreno-Mulet C, Sansó N, Carrero-Planells A, et al. The Impact of the COVID-19 Pandemic on ICU Healthcare Professionals: A Mixed Methods Study. Int J Environ Res Public Health 2021;18(17). 10.3390/ijerph18179243. Published Online First: 20210901. [DOI] [PMC free article] [PubMed]
- 8.Singh J, Karanika-Murray M, Baguley T, Hudson J. A psychometric evaluation of Professional Quality of Life Scale Version 5 (ProQOL 5) in a UK-based sample of allied mental health professionals. Curr Psychol. 2024. 10.1007/s12144-024-05966-x. [Google Scholar]
- 9.Estabrooks CA, Hutchinson AM, Squires JE, et al. Translating research in elder care: an introduction to a study protocol series. Implement Sci. 2009;4:51. 10.1186/1748-5908-4-51. Published Online First:20090810. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53–5. 10.5116/ijme.4dfb.8dfd. PublishedOnlineFirst:20110627. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Stensen K, Lydersen S. Internal consistency: from alpha to omega? Tidsskr Nor Laegeforen 2022;142(12). 10.4045/tidsskr.22.0112. Published Online First: 20220823. [DOI] [PubMed]
- 12.Dunn TJ, Baguley T, Brunsden V. From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. Br J Psychol. 2014;105(3):399–412. 10.1111/bjop.12046. [DOI] [PubMed] [Google Scholar]
- 13.Wang M, Batt K, Kessler C, et al. Internal consistency and item-total correlation of patient-reported outcome instruments and hemophilia joint health score v2.1 in US adult people with hemophilia: results from the Pain, Functional Impairment, and Quality of life (P-FiQ) study. Patient Prefer Adherence 2017;11:1831–39. 10.2147/ppa.S141391 Published Online First: 20171025. [DOI] [PMC free article] [PubMed]
- 14.Piedmont RL. Inter-item Correlations. In: Michalos AC, editor. Encyclopedia of Quality of Life and Well-Being Research. Dordrecht: Springer, Netherlands; 2014. p. 3303–4. [Google Scholar]
- 15.Rogers P. Best practices for your confirmatory factor analysis: A JASP and lavaan tutorial. Behav Res Methods. 2024. 10.3758/s13428-024-02375-7. [DOI] [PubMed] [Google Scholar]
- 16.Hu Lt, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary J. 1999;6(1):1–55. 10.1080/10705519909540118.
- 17.Walker TJ, Rodriguez SA, Vernon SW, Savas LS, Frost EL, Fernandez ME. Validity and reliability of measures to assess constructs from the inner setting domain of the consolidated framework for implementation research in a pediatric clinic network implementing HPV programs. BMC Health Serv Res. 2019;19(1):205. 10.1186/s12913-019-4021-5. Published Online First:20190329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Goretzko D, Siemund K, Sterner P. Evaluating Model Fit of Measurement Models in Confirmatory Factor Analysis. Educ Psychol Measur. 2023;84(1):123–44. 10.1177/00131644231163813. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Iaconi A, Hoben M, Berta PhDW, et al. The Association of Nursing Homes’ Organizational Context With Care Aide Empowerment: A Cross-Sectional Study. Gerontologist. 2024;64(6):gnae033. 10.1093/geront/gnae033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Crutzen R, Peters G-JY. Scale quality: alpha is an inadequate estimate and factor-analytic evidence is needed first of all. Health Psychol Rev. 2017;11(3):242–47 10.1080/17437199.2015.1124240. [DOI] [PubMed]
- 21.Misouridou E, Mangoulia P, Pavlou V, et al. Reliability and Validity of the Greek Version of the Professional Quality of Life Scale (ProQOL-V). Mater Sociomed. 2021;33(3):179–83. 10.5455/msm.2021.33.179-183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Keesler JM, Fukui S. Factor structure of the professional quality of life scale among direct support professionals: factorial validity and scale reliability. J Intellect Disabil Res. 2020;64(9):681–9. 10.1111/jir.12766. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data used for this article are housed in the Health Research Data Repository (HRDR) in the Faculty of Nursing at the University of Alberta, in accordance with (a) the health privacy legislation of participating TREC jurisdictions and (b) ethics approvals of universities and institutions participating in TREC. The data were provided under specific data sharing agreements only for approved use by TREC within the HRDR. Where necessary, access to the HRDR in order to review the original source data may be granted to those who meet pre-specified criteria for confidential access, available from data unit in TREC.