Cross-Cultural Adaption and Validation of SERVPERF Tool for Measuring Healthcare Quality in an Oncology Public Hospital, Vietnam

Abstract

SERVPERF (Service Performance) tool needs to be adapted to the context and the usage purpose. Our study aimed to validate the context adapted SERVPERF tool in an Oncology public hospital in Vietnam. A study was conducted in 2020 with 227 in-patients as respondents the modified SERVPERF tool. Data collected were analyzed for tool assessment (reliability and validity). The new order in 5 factors in the modified tool were: (1) Responsiveness; (2) Empathy; (3) Reliability; (4) Tangible and (5) Assurance. The modified tool has a high Cronbach’s alpha of .94. The tool validity was confirmed by Confirmatory Factor Analysis (CFA) where the healthcare service quality item corresponds as χ²/df = 3.79, Comparative Fit Index (CFI) = 0.85, Tucker Lewis Index (TLI) = 0.83, and Root Mean Squared Error of Approximation (RMSEA) = 0.08 are good fit indices. The modified tool SERVPERF with high reliability and validity could be applied for measuring the clients’ perceptions about healthcare service quality in other Oncology public hospitals in Vietnam.

• What do we already know about this topic?

• This topic is about a cross-cultural adaptation of the healthcare service quality model, SERVPERF.

• How does your research contribute to the field?

• Our research contributes to have a new modified tool for specific hospitals, especially the new order of dimensions as well as the new order of items in each dimension.

• What are your research’s implications toward theory, practice, or policy?

• This study aimed at cross-culturally adapting the original SERVPERF tool in an Oncology public hospital in Vietnam.

Introduction

Service quality may be defined or conceptualized as customers’ overall feeling about the superiority or inferiority of the service provider’s services.¹ Currently, there are 2 main tools commonly used SERVQUAL (Service Quality) and SERVPERF (Service Performance). SERVQUAL covers the expectations and perceptions of customers’ evaluation, meanwhile SERVPERF merely explores the customers’ perceptions. Both SERVQUAL and SERVPERF are equally valid predictors of overall service quality. Depending on the purpose of study, type of services, and level of involvement, the appropriate tool could be selected. SERVQUAL is considered helpful for diagnostic purpose, and SERVPERF is recommended for the sound theoretical model.²

Cronin and Taylor (1992) proposed SERVPERF model to measure the quality of healthcare services through patients’ perceptions. It includes 22 items in 5 dimensions: tangibles, reliability, responsiveness, empathy, and assurance.³ This tool merely considers the customers’ perceptions. In some previous studies, authors had modified the tool adapting to regional responders’ perceptions.^4-6

In healthcare services, the ultimate goal is to help patients improve their disease conditions along with good quality services, patient safety as well as the patients’ positive experience while staying at hospitals.⁷ The demand for good services is increasing in countries with fast economic growth like Vietnam. The people’s income level has been improved, thus they are willing to pay for high-cost services including healthcare services.⁸ Since 2019, Vietnam Government has invested more budget for infrastructure, equipment at public health settings. In addition, some international projects and programs have supported Vietnam in capacity building, such as high quality medical human resources for better diagnosis, care, and treatment at all levels of hospitals.⁹ To assure the healthcare quality, Vietnam Ministry of Health issued the set of criteria for quality assurance. The implementation of quality assurance requires the commitment of all health staff, especially creating a culture of quality assurance.¹⁰

With the higher demand of healthcare quality from patients in the developing countries like Vietnam, our research aimed to compose the content of SERVPERF tool adapted to an Oncology hospital context and assess its reliability and validity for exploring patients’ perception on healthcare quality.

Methods

SERVPERF Tool Context Adaption

Scale development and testing process developed by Hinkin was adopted. Stage 1: item generation, stage 2: scale development, and stage 3: scale evaluation.¹¹ The original SERVPERF⁸ was translated into Vietnamese. The tools were revised with comments from experts and then tested with 30 patients and culture adjusted accordingly. Afterward, the instrument was back-translated into English and compared with the original version to ensure the accuracy and quality of the translation. This step is considered as “Cross-cultural adaptation” which refers to the process of internal change in item contents so as to be able to function in an unfamiliar culture.¹²

Study Design

A cross-sectional design was used for validating the contextualized SERVPERF tool. The 400 in-patients, whose discharge procedures were arranged, were chosen conveniently, even though the proposed sample size was only 220 with the criterion of 5 to 10 participants per item (the questionnaire with 22 items).¹³

*Independent variables

The main variables were 22 questions in the SERVPERF scale; they were adapted to the hospital context, and a 5-Likert scale, from “Strongly disagree” to “Strongly agree,” was used for all of the questions.

*Dependent variables

The study outcome was service quality, a hidden variable generated from 22 variables during Structural Equation Modelling (SEM) analysis.

Data Analysis With SPSS 20 and AMOS 24

Reliability was evaluated by internal consistency (Cronbach’s alpha). If the alpha value is .7 or higher, this suggests that the test is reliable and the scale could be used properly.^14,15

Principal component analysis and varimax as the rotation method was used to identify dimension with factor loadings. The cut-off for meaningful factor loadings is defined as greater than 0.30 at minimum or at least 0.45 as “fair” as suggested by.Pett et al¹³ and Floyd and Widaman¹⁶ The SEM was used as an analysis in the construction of the SERVPERF model.

Confirmatory Factor Analysis (CFA) was conducted to certify the elementary factors using Comparative Fit Index (CFI), Tucker Lewis Index (TLI), and Root Mean Squared Error of Approximation (RMSEA).¹⁷ These indices’ criteria must be met for a satisfactory fit model: (i) χ²/df ratio should be between 1 and 5; (ii) CFI and TLI must approach 1;^4,16,17 (iii) RMSEA should be up to 0.09 with 90% confidence interval values below 0.1.¹⁷

Standardized regression weight would be used if corresponding values are under 0.9 to show all variables can represent a significant indicator and predictor for latent variables.¹⁸

Ethical Approval

This study was approved by University of Public Health, Hanoi, Vietnam (Decision 124/2020/YTCC-HD3 on 30 March 2020). Respondents signed informed consent at the beginning of the self-administered questionnaire and were informed that all data collected would be anonymous.

Results

Tool Adaption (Content, Order, and Reliability)

Data collected for assessing the modified tool should be validated before running the analysis. One of the tests was Bartlett’s Test of Sphericity and Kaiser-Meyer-Olkin Test (KMO). Bartlett’s Test of Sphericity showed the significant value of .000, indicating P < .05. Thus, it means that the correlation among items is sufficient to run the factor analysis. In KMO, the value is 0.90 (greater 0.5). This means that these items are relevant to the factor analysis performed and showed no serious multicollinearity data (see Table 1).

Table 1.

The Reliability Assessment of the Modified SERVPERF Tool.

Factors (orginal order)	Cronbach’s α	Eigen value	The minimum value of factor loadings
Tangibles	.85	1.2	0.58
Reliability	.86	1.5	0.48
Responsiveness	.85	9.6	0.56
Assurance	.76	0.9	0.68
Empathy	.84	1.7	0.62

(i) Overall Cronbach’s α = .94; (ii) KMO = 0.92; (iii) Barlett’s test: P < .001.

The sample size 227 was reasonable for factor analysis with KMO 0.92 and all 22 items were correlated with each other with Barlett’s test (P < .001). The modified tool had the high reliability with the internal consistency of 22 items with the high overall Cronbach’s alpha value of .94 and each factor value in range from .76 to .86 in details as Assurance (.76/lowest), Empathy (.84), Responsiveness (.85), Tangible (.85), and Reliability (.86/highest).

Eigenvalues (the variance explained by each factor) was from 0.9 and above. The lowest factor loadings (ie, the correlation coefficient between variables and factors) across all factors exceeded 0.4. We proposed to re-order the factors in the modified tools from the highest Eigen value to the lowest as follows: (1) Responsiveness; (2) Empathy; (3) Reliability; (4) Tangibles, and (5) Assurance. This new order is different to the original one due to cultural aspects or perception or expectation of patients at Oncology hospital (see Table 2).

Table 2.

Dimension Code and Item Contents.

Dimension and older order	Item contents adapted to Vietnamese culture	New order
res3	Health staff never refuse legitimate patient requests	1
res2	Health staff are ready to support patients	2
res4	Patients receive services timely	3
ass1	Healthcare staff create a belief in patients (new res.)	4
emp3	Health workers understand legitimate patient requests	1
emp1	Hospital gives individual attention to each patient	2
emp2	Patients receive the care of health staff	3
emp5	Health check-up time is convenient	4
emp4	Patient’s benefits are the hospital’s concern	5
rel5	The patient’s medical records are well stored	1
rel4	Hospital provides services as the time in the notice	2
rel1	Patients are well served as in the hospital’s commitments	3
res1	The healthcare service delivery is fully informed to patients (new rel.)	4
rel2	Hospital supports patients well	5
rel3	Hospital responds to the patient’s legitimate needs	6
tan3	Hospital has good signs for guidance	1
tan4	Health staff’ are well dressed and appear neat	2
tan2	The health check-up area is arranged conveniently	3
tan1	Hospital has up-to-date equipment	4
ass2	Health staff are highly qualified	1
ass4	Health staff are attentive and polite	2

Tool Validity Assessment

Factor loading values in all items should be higher than 0.4 and the item order within each dimension was re-ordered regarding to its loading value. We performed regression weight through the Critical Ratio (C.R) for indicating all variables representative as significant indicators and predictor for latent variables (see Table 3).

Table 3.

Items Loadings and Regression Weight in the New Order.

Dimension and item code	Item loadings	Estimate	SE	CR	P value
res3	0.8	1.08	0.07	14.77	*
res2	0.78	1.00			*
res4	0.63	1.23	0.08	14.55	*
res 5 (ass1)	0.56	0.69	0.06	11.18	*
emp3	0.77	1.09	0.09	12.17	*
emp1	0.73	1.00
emp2	0.71	1.05	0.08	12.20	*
emp5	0.7	0.58	0.06	8.49	*
emp4	0.62	0.71	0.07	9.28	*
rel5	0.74	0.77	091	8.49	*
rel4	0.69	1.16	0.09	12.16	*
rel1	0.68	1.00
rel 6 (res1)	0.55	0.88	0.08	10.81	*
rel2	0.5	1.15	0.09	12.52	*
rel3	0.48	1.38	0.10	12.83	*
tan3	0.76	1.75	0.16	10.59	*
tan4	0.72	1.38	0.13	10.12	*
tan2	0.68	1.59	0.15	10.11	*
tan1	0.58	1.00
ass2	0.81	1.00
ass4	0.75	1.04	0.10	10.04	*
ass3	0.68	0.97	0.09	9.90	*

^*P value < .001.

We found the new grouping items among the 3 dimensions (Responsiveness, Reliability, and Assurance). The item ass1 moved to Responsiveness as a new item (res5) and res1 moved to Reliability as a new item (rel6) with significant P-value.

Tool modelling

The data collected from 227 patients were good enough for assessing modified SERVPERF tool with 22 items and 5 factors basing on model fit indicators (Figure 1). The RMSEA score of 0.08 indicates a close fair fit. The goodness of fit indices CFI, TLI, χ²/df, and P-value was acceptable (see Table 4).

Figure 1.

Measurement model for the dimension of modified SERVPERF.

Table 4.

Model Fit Indicators.

Statistics	Recommended thresholda	Results
Sample size (SS)	≥220	227
Number of variables/items	12 < items < 30	22
RMSEA	≤0.08	0.08
CFI	≥0.08	0.85
TLI	≥0.08	0.83
χ2/df	1-5	3.79
P value	<.05	<.01

^aApplied for the model with a SS of (n) ≥ 220 and the number of items between 12 and 30.

Structural Equation Modelling (SEM) is the final analysis for th construction of SERVPERF items. CFA was used with survey data of in-patients’ perception of health service quality to verify the adapted factors that have been produced by the EFA and to validate these constructs. The modified SERVPERF constructs showed the χ²/df = 3.79 < 5 and other specifications such as RMSEA value (0.08), CFI value (0.85), and TLI value (0.83) were good fit indices. Thus, all variables can represent a significant indicator and predictor for latent variables.

Discussion

The order of dimensions and detailed questions can affect the correctness of the answers. Some previous studies found that the idea that earlier items in a questionnaire can affect later responses,¹⁹ the size of question order effects may further depend on the topic of the questionnaire,²⁰ the respondents’ characteristics,²¹ and the interviewer behavior.²² We modified major changes in dimensions order as well as detailed items based on model statistical estimation results. The original dimensions are logically ordered from visual to perceptive things or levels and types of expectations from patients, such as Tangible, Reliability, Responsiveness, Assurance, and Empathy. Based on Eigenvalue, we proposed the very new order in 5 factors: (1) Responsiveness; (2) Empathy; (3) Reliability; (4) Tangible; and (5) Assurance. And we also found a new way of item grouping (the assurance item was re-grouped in responsiveness; meanwhile, the responsiveness item was re-grouped in reliability), which was different from the original scale. The main explanation for our new findings is the Vietnamese cancer patients’ needs. As their sickness are often severe, they demand high responsiveness in providing medical care for them rather than the tangible (place and staff uniforms), and the next things are the empathy of the health staff for them while having cancer diseases. For reliability, it is in the third order since the only Oncology hospital could treat cancer diseases. It seems that the patients have no other choice. The 2 last factors are tangible and assurance. The cancer patients do understand the overcrowded cancer patients and the high risk of mortality, thus they have not required much about facilities and treatment outcome assurance.

As in many previous studies on SERVPERF, the dimensions and items order were kept as the original one, our findings are totally new in rearranging the popular SERVPERF tool with fully statistical tests. Our modified tool has high reliability and validity with a high Cronbach’s alpha of .94 and met all statistical requirements as well as the threshold values from CFA analysis with SEM modeling such as χ²/df (3.79), CFI (0.85), TLI (0.83), and RMSEA (0.08). These parameters are various between studies depending on purposes, study populations etc. so we only self-validated them with standard threshold values.

Being evidenced by the CFA and supported by the value of Standardized Regression Weights through the CR, the modified SERVPERF model is well-fitted. The use of CFA and SEM model validating the questionnaire to measure the quality of care has been found in worldwide studies,^6,23.

The findings of this study can be used by policymakers, especially the ministry of health in the development of a validated instrument for measuring the quality of healthcare service in hospitals. The ministry of health issued a set of quality criteria for Vietnam hospitals where a healthcare service quality tool was also developed and used for general and specialized public hospitals.¹⁰ However, no academic evidence was found about its validity and reliability.

Some limitations of this study should be noted. The study was only conducted in an Oncology hospital. Thus, the findings were generalized to other Oncology health facilities, not to other general ones. The differences in languages and cultures may lead to different perceptions causing the different order of factors as well as items within each factor and the new item grouping. This change should be studied more in other Oncology hospitals for confirmation.

Conclusion

The contextualized SERVPERF for Vietnamese Oncology patients was well-validated with high validity and reliability. This tool could be used to measure patients’ perception of healthcare quality in other Oncology hospitals in Vietnam.