Abstract
Background and aims
Patient satisfaction is one of the important ways to measure the results of treatment and the quality of medical services. The purpose of the present study was to verify the reliability and validity of the indicator system for evaluating satisfaction of outpatients and inpatients in Chinese hospitals.
Methods
The study was based on the satisfaction questionnaire program of the national doctor– patient experience research center with data from 99,802 outpatients and 229,215 inpatients, collected in China between 2016 and 2017. We adopted exploratory factor analysis for validity analysis and the method of split-half reliability and Cronbach’s α coefficient for reliability analysis.
Results
In the validity analysis of the indicator system for outpatients, the factor loading was between 0.438 and 0.919, and the reliability was excellent. In the validity analysis of the indicator system for nonsurgical inpatients, the factor loading of the secondary indicators was between 0.417 and 0.75, and the reliability was excellent. In the validity analysis of the indicator system for surgical inpatients satisfaction, the factor loading of the secondary indicators range was 0.391–0.751, and the reliability was excellent.
Conclusion
The indicator systems for evaluating satisfaction of outpatients, surgical inpatients and nonsurgical inpatients all have excellent reliability and good validity. They can be widely used for an outpatient and inpatient satisfaction questionnaire in Chinese hospitals.
Keywords: patient satisfaction questionnaire, reliability, validity, evaluation
Introduction
The patient satisfaction is the patients’ subjective evaluation of the accepted medical service under the influence of various factors after comparing expectations with feelings.1 It is an important evaluation tool for the quality of hospital services. Research on customer satisfaction evaluation systems is of great significance both in theory and practice to improve further the management theory of medical service quality, to guide medical institutions to improve service quality and eventually to win the trust of the patients.2
In other countries outside China, research on the theory and methods of patient satisfaction evaluation started early.3,4 Among them, one of the most classic evaluation scales was the patient satisfaction questionnaire (PSQ) developed by Ware et al, which was the most widely used questionnaire in the 1990s.5 In addition to the evaluation of medical service quality, it can also provide a reference for the overall layout of hospitals, capital utilization and personnel deployment. The Hospital Consumer Assessment of Healthcare Providers and Systems questionnaire developed by the Centers for Medicare and Medicaid Services and Agency for Healthcare Research and Quality is the second most influential.6 Compared with other questionnaires, it includes employee factors and can be used for comparison and analysis of patient satisfaction in different hospitals. It was officially used in the US in 2008.
As patient satisfaction was widely introduced into hospitals in developed countries and achieved significant results, Chinese hospitals have begun to recognize the essence of “patient orientated health care” in recent years. Xia et al (2010)11 followed the basic steps of the measuring tools proposed by Streiner et al7 and designed six questionnaires, such as “the questionnaire on the satisfaction of discharged patients” and “the questionnaire on the satisfaction of outpatients”.11 Chen (2011)8 divided the patient satisfaction evaluation system into three levels, including 6 primary indicators, 13 secondary indicators and 32 three-level indicators, and built a simple outpatient satisfaction indicator system using Analytical Hierarchy Process. Based on the past inpatient satisfaction model, Tan L10 constructed an inpatient satisfaction indicator system of a grade III level A hospital with 1 primary index, 5 secondary indicators and 29 three-level indicators in Jiangkou city.10 Kong and Chen (2007)9 selected 30 experts in different institutions in Beijing, Ji’nan and Shenzhen. Through the investigation of community health services, a reliability test of expert consultations was conducted.
However, due to the late start of patient satisfaction research in China, both theory and practice necessarily are at the exploratory stage, and many problems need to be solved urgently. On the one hand, most of the domestic questionnaires are distributed “on the spot” and are carried out by hospitals themselves. Most hospitals do not entrust third party evaluation agencies to investigate them. It is difficult to construct a reliable mathematical model based on the deviation of measurement data caused by the above reasons.12 On the other hand, there is no scientific and effective PSQ that can be widely used in China to date, and the available PSQs lack scientific credibility and reliability. Furthermore, existing questionnaire research does not address applicability or take practical considerations into account.13 Most of the existing PSQ are not universal and representative, which makes the evaluation results difficult to compare objectively. In addition, the majority of the domestic PSQs used involved medical quality, attitude of medical staff and the medical environment, without mentioning important content such as the patient’s expectations, value perception and loyalty to the medical institution. The evaluation results are therefore difficult to use to provide an effective scientific basis for choosing a medical institution. Therefore, a standardized and reliable indicator system for a PSQ remains to be developed.
Therefore, in the present study, we developed a patient satisfaction indicator system that can be widely used in outpatient and inpatient satisfaction questionnaires (IPSQs) in Chinese hospitals following strict scientific criteria. The study aims to confirm the reliability and validity of the indicator system for evaluating the satisfaction of outpatients and inpatients.
Methods
Compilation of the indicator system
This study is a research project of the national doctor–patient experience research center. In the early stages, the study passed analysis of the medical practice process and many rounds of domestic and foreign expert demonstrations. The indicator systems for evaluating satisfaction of outpatients, surgical inpatients (including all types of surgery) and nonsurgical inpatients were initially constructed. The indicator system of outpatient satisfaction is a two-level indicator system, with 5 primary indicators and 31 secondary indicators (Table S1). The indicator system of inpatient satisfaction is a two-level indicator system, with 6 primary indicators and 43 secondary indicators (Table S2). Among them, the satisfaction indicator system of nonsurgical inpatients had 5 items under the primary index, including hospitalization service, treatment, auxiliary examination, service attitude, environment and logistics service, and 38 items under the secondary index. The satisfaction indicator system of surgical inpatients had 6 items under the primary index, including hospitalization service, treatment, auxiliary examination, service attitude, environment and logistic services, surgical anesthesia, and 45 items under the secondary index. We used a balanced Likert 5-point scale to record responses and assigned a score of 1, 2, 3, 4 and 5 for strongly disagree, disagree, neutral, agree and strongly agree, respectively. The higher the score, the higher the satisfaction of the item. Each item score adds up to the total score of the indicator system. With the original indicator system, the full score of the satisfaction questionnaire of outpatients is 155 points, for nonsurgical inpatients, 190 points and for surgical inpatients, 225 points.
Subjects
All the hospitals investigated in our study voluntarily participated in the PSQ of the national doctor and patient experience research center. The 8%–30% of outpatients in a single day were randomly selected from the list of patients who received 3 consecutive days of medical treatment provided by the hospital participating in the project. Between 30% and 60% of hospitalized patients, who fulfilled inclusion criteria in a single day, were selected from the total number of inpatients in the participating hospitals, and the inpatient components of each department and ward. Hospitalized patients who had been admitted in the hospital for >24 hours and therefore had a certain experience of hospitalization were included in this study.
The interviewees were patients or their family members. All patients who were aged ≥18 years old and conscious completed the questions by themselves or with the help of investigators on the mobile computer provided. Only when patients were <18 years old and unable to complete the questions by themselves or with the help of an investigator (medical conditions included severe mental disorder, coma, dementia, etc.), family members answered the questions on behalf of the patient.
On-site investigation process
The indicator system used in our study aimed to understand patient satisfaction on various aspects of the hospital service, thus permitting the evaluation of the overall service quality of the hospital. A third-party company was hired and the investigators conducted paperless questionnaires on subjects using a unified mobile terminal system. During the investigation, the investigators explained the purpose of the study through detailed written and verbal understandable instructions and asked the patients to truly reflect their own thoughts and opinions based on their own experience and feelings. The researchers guaranteed that the information provided by each patient was strictly confidential and that they would never provide any information obtained to others without the consent of the patient. Also, the hospital and department was kept strictly confidential. They did not provide any person with any information on a specific hospital or department, which unintentionally may affect a patient’s medical treatment and perception of the hospital. The investigators used a unified guide to describe the way the mobile terminal questionnaire was filled in, the research content and the purpose of the study, and also informed the patient that there was no right or wrong answer, just follow faithfully their innermost thoughts. Each patient completed the questionnaire independently; patients who had difficulty doing this task, due to physical illness, were interviewed by a researcher or relatives, who completed the questionnaire for them. The investigator waited patiently and requested the patient to answer the questions in an objective, neutral manner. The completed questionnaire was uploaded from the computer as soon as it was completed. The average time required to complete the questionnaire was circa 10–15 minutes.
In total, 99,802 outpatient and 229,215 up-to-standard inpatient data were collected. Among the inpatient questionnaire data, 58,220 came from surgical patients and 170,995 from the nonsurgical patients.
Statistical analyses
This study confirmed the final structure of the indicator system and tested the structural validity of the indicator system by exploratory factor analysis. The appropriateness of factor analysis was tested using the Kaiser–Meyer–Olkin adequacy (KMO) index and Bartlett’s test of sphericity. If the KMO of the indicator system was greater than 0.5 and the Bartlett’s test of sphericity significant (P<0.05), then the indicator system was deemed suitable for factor analysis. After determining the number of factors in each indicator system according to its structure, the cumulative variance contribution rate was obtained by factor analysis. Principal component analysis was used to extract the factor, and then a rotation transformation was carried out to find the best analysis effect. If the factor of the indicator system could explain >50% variation and each item had enough loading on the corresponding factors (≥0.4), the indicator system was considered to have good structural validity. After omitting loadings <0.3 and forming a new indicator system based on expert opinions and the actual situation,14 we used the Spearman–Brown reliability and Cronbach’s α coefficient to test the split-half reliability and internal consistency reliability of the indicator system, respectively.
In general, a split-half reliability coefficient of >0.75 was excellent and <0.4 poor. α>0.8 indicated excellent internal consistency, 0.6<α<0.8 an acceptable level and α<0.6 was indicative of poor internal consistency.15 All statistical analyses were carried out using SAS 9.4 software.
Results
Characteristics of the study subject
This study collected data from 98,192 outpatients in 197 hospitals from 22 provinces and 229,215 hospitalized patients distributed in 413 hospitals in 28 provinces in China (Table S3).
In this study, 19,953 (20.3%) outpatient data were collected in 2016 and 78,243 (79.7%) outpatient data in 2017. Of the outpatients, 40,072 (40.8%) were male, with an age range mainly between 18 and 59 years old (81,960: 83.5%). In addition, outpatients who completed the questionnaire were mainly from grade III hospitals (82.3%), and many of them had local medical insurance (53.8%) (Table 1).
Table 1.
Characteristics of outpatients and inpatients
| Demographic variables | Outpatients (n=98,196) | Nonsurgical inpatients (n=170,995) | Surgical inpatients (n=58,220) | |||
|---|---|---|---|---|---|---|
|
| ||||||
| Number | Constituent ratio (%) | Number | Constituent ratio (%) | Number | Constituent ratio (%) | |
|
| ||||||
| Gender | ||||||
| Male | 40,072 | 40.8 | 84,161 | 49.2 | 29,627 | 50.9 |
| Female | 58,124 | 59.2 | 86,834 | 50.8 | 28,593 | 49.1 |
| Age (years) | ||||||
| <18 | 6,347 | 6.5 | 11,401 | 6.7 | 2,614 | 4.5 |
| 18–59 | 81,960 | 83.5 | 100,042 | 58.5 | 40,676 | 69.9 |
| >60 | 9,889 | 10.1 | 59,552 | 34.8 | 14,930 | 25.6 |
| Source of cost | ||||||
| Local medical insurance | 52,877 | 53.8 | ||||
| Remote medical insurance | 6,453 | 6.6 | ||||
| Self-financed | 38,866 | 39.6 | ||||
| Year | ||||||
| 2016 | 19,953 | 20.3 | 93,958 | 54.9 | 25,251 | 43.4 |
| 2017 | 78,243 | 79.7 | 77,037 | 45.1 | 32,969 | 56.6 |
| Hospital level | ||||||
| <Grade III | 17,361 | 17.7 | 62,162 | 36.4 | 10,988 | 18.9 |
| Grade III | 80,835 | 82.3 | 108,833 | 63.6 | 47,232 | 81.1 |
Among the outpatient questionnaires collected, 170,995 came from nonsurgical patients and 58,220 from surgical patients. There were 84,161 males (49.2%) in the nonsurgical inpatient group investigated. Patients between 18 and 59 years of age accounted for 58.5% of the total inpatients under investigation, and only 6.7% of the inpatients were under the age of 18. In addition, the nonsurgical inpatients were mainly from grade III hospitals (63.6%). There are 29,627 men (50.9%) in the surgical inpatients. Patients aged between 18 and 59 years of age accounted for 69.9% of the total surgical patients. The surgical inpatients were mainly from grade III hospitals (81.1%) (Table 1).
Results of structural validity
Validity analysis of the indicator system for evaluating outpatient satisfaction
Exploratory factor analysis was used to investigate the structural validity of the indicator system of the outpatient satisfaction questionnaire. The KMO of the indicator system was 0.938 and the Bartlett’s test value of sphericity 2,553,071.12 (P<0.001), which indicated that conducting factor analysis on the data was appropriate.
The secondary index of the indicator system of outpatient satisfaction questionnaire was also analyzed. The total variance contribution rate of the five primary indicators of pre-diagnosis, treatment, auxiliary examination, drug withdrawal, the environment and logistic services was 68.0% (Table 2). The Scree plot showed that the first five factors accounted for most of the variability. From the fifth factor, the characteristic value was small and exhibited a flat downward trend, with each factor accounting for only a small component of the variability (Figure 1).
Table 2.
Eigenvalue and contribution rates of the five factors in the indicator system of the outpatient satisfaction questionnaire
| Factors | Eigenvalues | Contribution rate (%) | Cumulative contribution rate (%) |
|---|---|---|---|
|
| |||
| 1 | 12,873.47 | 0.416 | 0.416 |
| 2 | 3,532.823 | 0.114 | 0.530 |
| 3 | 1,845.447 | 0.060 | 0.589 |
| 4 | 1,567.751 | 0.051 | 0.640 |
| 5 | 1,231.178 | 0.040 | 0.680 |
Figure 1.
Scree plot of the outpatient satisfaction indicator system.
In order to facilitate analysis, each index was encoded in the order of b1–b31. In the rotated factor matrix by varimax rotation, the index of the maximum absolute value of factor loading in each factor was grouped (Table 3). The indicators under the “pre-diagnosis” dimension were b1–b6. They were dominated by the fourth factor, and the factor loading was between 0.531 and 0.707. The indicators under the “treatment” dimension were b7–b12. They were dominated by the second factor, and the factor loading was between 0.488 and 0.803. The indicators under the “auxiliary examination” dimension were b13–b20. They were dominated by the first factor, and the factor loading was between 0.636 and 0.919. The indicators under the “drug withdrawal” dimension were b21–b23. They were dominated by the fifth factor, and the factor loading was between 0.684 and 0.788. The indicators under the “environment and logistics service” dimension were b24–b31. They were dominated by the third factor, and the factor loading was between 0.438 and 0.749.
Table 3.
Principal component analysis of the indicator system for evaluation of outpatient satisfaction
| Index | Principal components
|
||||
|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | |
|
| |||||
| b1 | 0.138 | 0.203 | 0.166 | 0.576 | 0.081 |
| b2 | 0.132 | 0.213 | 0.163 | 0.597 | 0.081 |
| b3 | 0.118 | 0.163 | 0.169 | 0.677 | 0.065 |
| b4 | 0.105 | 0.198 | 0.162 | 0.707 | 0.080 |
| b5 | 0.108 | 0.242 | 0.229 | 0.648 | 0.059 |
| b6 | 0.136 | 0.449 | 0.203 | 0.531 | 0.077 |
| b7 | 0.170 | 0.488 | 0.221 | 0.383 | 0.098 |
| b8 | 0.180 | 0.670 | 0.204 | 0.345 | 0.109 |
| b9 | 0.150 | 0.775 | 0.158 | 0.307 | 0.118 |
| b10 | 0.176 | 0.803 | 0.178 | 0.275 | 0.123 |
| b11 | 0.225 | 0.722 | 0.227 | 0.211 | 0.206 |
| b12 | 0.243 | 0.667 | 0.256 | 0.231 | 0.200 |
| b13 | 0.635 | 0.223 | 0.164 | 0.153 | 0.082 |
| b14 | 0.718 | 0.165 | 0.159 | 0.128 | 0.094 |
| b15 | 0.802 | 0.150 | 0.143 | 0.117 | 0.099 |
| b16 | 0.819 | 0.135 | 0.144 | 0.107 | 0.116 |
| b17 | 0.919 | 0.088 | 0.138 | 0.106 | 0.117 |
| b18 | 0.910 | 0.080 | 0.149 | 0.099 | 0.134 |
| b19 | 0.698 | 0.149 | 0.168 | 0.133 | 0.226 |
| b20 | 0.651 | 0.168 | 0.186 | 0.138 | 0.270 |
| b21 | 0.359 | 0.242 | 0.235 | 0.154 | 0.684 |
| b22 | 0.350 | 0.215 | 0.283 | 0.138 | 0.773 |
| b23 | 0.312 | 0.230 | 0.284 | 0.143 | 0.788 |
| b24 | 0.129 | 0.184 | 0.471 | 0.328 | 0.219 |
| b25 | 0.137 | 0.170 | 0.507 | 0.323 | 0.150 |
| b26 | 0.100 | 0.157 | 0.584 | 0.305 | 0.113 |
| b27 | 0.177 | 0.183 | 0.749 | 0.176 | 0.114 |
| b28 | 0.190 | 0.200 | 0.740 | 0.165 | 0.114 |
| b29 | 0.331 | 0.137 | 0.540 | 0.126 | 0.123 |
| b30 | 0.421 | 0.115 | 0.438 | 0.103 | 0.139 |
| b31 | 0.249 | 0.306 | 0.458 | 0.208 | 0.218 |
The factor of the indicator system can explain >50% of the variation, and each item had enough loading on the corresponding factors (≥0.4), so the indicator system had good structural validity.
Validity analysis of the indicator system for evaluating nonsurgical inpatient satisfaction
Exploratory factor analysis was used to investigate the structural validity of the secondary indicators of the nonsurgical inpatients indicator system. The KMO of the indicator system was 0.984 and the Bartlett’s test value of sphericity 5,441,976.14 (P<0.001), which indicates that conducting factor analysis on the data was appropriate. The total variance contribution rate of the five indicators of hospitalization service, treatment, auxiliary examination, service attitude, environment and logistics service was 68.5% (Table 4). The Scree plot showed the first five factors accounted for most of the variability. From the fifth factor, the characteristic value was small and exhibited a flat downward trend, and each factor accounted for only a small component of the variability (Figure 2).
Table 4.
Eigenvalue and contribution rates of the indicator system for evaluation of nonsurgical inpatient satisfaction
| Factors | Eigenvalue | Contribution rate (%) | Cumulative contribution rate (%) |
|---|---|---|---|
|
| |||
| 1 | 20,464.54 | 0.539 | 0.539 |
| 2 | 1,941.828 | 0.051 | 0.590 |
| 3 | 1,471.356 | 0.039 | 0.629 |
| 4 | 1,176.957 | 0.031 | 0.660 |
| 5 | 953.6677 | 0.025 | 0.685 |
Figure 2.
Scree plot of the nonsurgical inpatient satisfaction indicator system.
In order to facilitate analysis, each index was encoded in the order of b1–b38. In the rotated factor matrix by varimax rotation, the index of the maximum absolute value of factor loading in each factor was grouped (Table 5). The index of the maximum absolute value of factor loading in each factor was also grouped. The indicators under the “hospitalization service” dimension were b1–b7. They were dominated by the second factor, and the factor loading was between 0.420 and 0.730. The indicators under the “treatment” dimension were b8–b20. They were dominated by the first factor, and the factor loading was between 0.503 and 0.597. The indicators under the “auxiliary examination” dimension were b21–b26. They were dominated by the third factor, and the factor loading was between 0.649 and 0.775. The indicators under the “service attitude” dimension were b27–b29. They were dominated by the fifth factor, and the factor loading was between 0.417 and 0.579. The indicators under the “environment and logistic services” dimension were b30–b38. They were dominated by the fourth factor, and the factor loading was between 0.485 and 0.713.
Table 5.
Principal component analysis of the indicator system for evaluation of nonsurgical inpatient satisfaction
| Index | Principal components | ||||
|---|---|---|---|---|---|
|
| |||||
| 3 | 1 | 2 | 3 | 4 | 5 |
|
| |||||
| b1 | 0.200 | 0.545 | 0.191 | 0.183 | 0.139 |
| b2 | 0.273 | 0.653 | 0.182 | 0.218 | 0.167 |
| b3 | 0.252 | 0.736 | 0.206 | 0.230 | 0.113 |
| b4 | 0.261 | 0.732 | 0.215 | 0.232 | 0.108 |
| b5 | 0.303 | 0.684 | 0.208 | 0.219 | 0.125 |
| b6 | 0.229 | 0.510 | 0.168 | 0.184 | 0.093 |
| b7 | 0.243 | 0.420 | 0.176 | 0.220 | 0.039 |
| b8 | 0.503 | 0.424 | 0.188 | 0.219 | 0.145 |
| b9 | 0.544 | 0.472 | 0.237 | 0.262 | 0.052 |
| b10 | 0.572 | 0.445 | 0.265 | 0.278 | 0.027 |
| b11 | 0.546 | 0.317 | 0.320 | 0.265 | −0.021 |
| b12 | 0.588 | 0.349 | 0.311 | 0.287 | 0.039 |
| b13 | 0.597 | 0.384 | 0.291 | 0.289 | 0.109 |
| b14 | 0.562 | 0.374 | 0.275 | 0.277 | 0.163 |
| b15 | 0.545 | 0.332 | 0.310 | 0.287 | 0.138 |
| b16 | 0.547 | 0.334 | 0.276 | 0.292 | 0.201 |
| b17 | 0.560 | 0.303 | 0.237 | 0.280 | 0.313 |
| b18 | 0.597 | 0.285 | 0.261 | 0.307 | 0.311 |
| b19 | 0.549 | 0.264 | 0.289 | 0.336 | 0.248 |
| b20 | 0.514 | 0.284 | 0.299 | 0.310 | 0.345 |
| b21 | 0.367 | 0.221 | 0.649 | 0.265 | 0.093 |
| b22 | 0.314 | 0.219 | 0.726 | 0.264 | 0.100 |
| b23 | 0.282 | 0.238 | 0.735 | 0.262 | 0.141 |
| b24 | 0.223 | 0.231 | 0.775 | 0.273 | 0.109 |
| b25 | 0.215 | 0.238 | 0.757 | 0.300 | 0.130 |
| b26 | 0.208 | 0.253 | 0.735 | 0.304 | 0.180 |
| b27 | 0.282 | 0.379 | 0.428 | 0.308 | 0.434 |
| b28 | 0.323 | 0.335 | 0.330 | 0.309 | 0.579 |
| b29 | 0.333 | 0.339 | 0.354 | 0.383 | 0.417 |
| b30 | 0.306 | 0.257 | 0.332 | 0.485 | 0.147 |
| b31 | 0.266 | 0.254 | 0.308 | 0.487 | 0.089 |
| b32 | 0.309 | 0.264 | 0.281 | 0.588 | 0.216 |
| b33 | 0.209 | 0.217 | 0.197 | 0.709 | 0.148 |
| b34 | 0.214 | 0.242 | 0.211 | 0.713 | 0.159 |
| b35 | 0.197 | 0.200 | 0.277 | 0.666 | −0.015 |
| b36 | 0.254 | 0.222 | 0.284 | 0.673 | 0.057 |
| b37 | 0.349 | 0.278 | 0.271 | 0.556 | 0.214 |
| b38 | 0.362 | 0.320 | 0.266 | 0.513 | 0.250 |
The five factors of the indicator system can explain >50% of the variation, and each item had enough loading on the corresponding factors (≥0.4), which indicated that the indicator system had good structural validity.
Validity analysis of the indicator system for evaluating surgical inpatient satisfaction
Exploratory factor analysis was used to analyze the structural validity of the original indicator system of surgical inpatients. The KMO of the indicator system was 0.987 and the Bartlett’s test value of sphericity 2,235,436.36 (P<0.001), indicating sample adequacy.
Exploratory factor analysis was further used to examine the structural validity of the secondary indicators of the surgical inpatients indicator system. The total variance contribution rate of the six primary indicators of hospitalization service, treatment, auxiliary examination, service attitude, environment and logistics service, surgical anesthesia was 71.0% (Table 6). The Scree plot revealed that the first six factors accounted for most of the variability. From the sixth factor, the characteristic value was small and exhibited a flat downward trend, and each factor accounted for only a small part of the variability (Figure 3).
Table 6.
Eigenvalue and contribution rates of the indicator system for the evaluation of surgical inpatient satisfaction
| Factors | Eigenvalue | Contribution rate (%) | Cumulative contribution rate (%) |
|---|---|---|---|
|
| |||
| 1 | 24,117.75 | 0.561 | 0.561 |
| 2 | 1,856.685 | 0.043 | 0.605 |
| 3 | 1,493.016 | 0.035 | 0.639 |
| 4 | 1,113.706 | 0.026 | 0.665 |
| 5 | 1,025.655 | 0.024 | 0.689 |
| 6 | 904.2898 | 0.021 | 0.710 |
Figure 3.
Scree plot of the surgical inpatient satisfaction indicator system.
In order to facilitate analysis, each index was encoded in the order of b1–b43, and six principal components were taken for principal component analysis. In the rotated factor matrix by varimax rotation, the index of the maximum absolute value of factor loading in each factor was grouped (Table 7). The indicators under the “hospitalization service” dimension were b1–b7. They were dominated by the first factor, and the factor loading was between 0.412 and 0.720. The indicators under the “treatment” dimension were b8–b20. They were dominated by the third factor, and the factor loading was between 0.467–0.578. The indicators under the “auxiliary examination” dimension were b21–b26. They were dominated by the second factor, and the factor loading was between 0.653 and 0.745. The indicators under the “service attitude” dimension were b27–b29. They were dominated by the sixth factor, and the factor loading was between 0.391 and 0.559. The indicators under the “environment and logistics service” dimension were b30–b38. b30–b37 were dominated by the fifth factor, and the factor loading was between 0.422 and 0.688. Factor b38 was dominated by the fourth factor, and the factor loading was 0.465. Its factor loading under the fifth factor was 0.392. The indicators under the “surgical anesthesia” dimension were b39–b43, and the factor loading was between 0.520 and 0.752.
Table 7.
Principal component analysis of the questionnaire for evaluation of surgical inpatient satisfaction
| Index | Principal components
|
|||||
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | |
|
| ||||||
| b1 | 0.527 | 0.231 | 0.177 | 0.182 | 0.188 | 0.115 |
| b2 | 0.649 | 0.209 | 0.241 | 0.212 | 0.208 | 0.152 |
| b3 | 0.720 | 0.222 | 0.207 | 0.202 | 0.215 | 0.107 |
| b4 | 0.716 | 0.221 | 0.253 | 0.227 | 0.191 | 0.125 |
| b5 | 0.656 | 0.207 | 0.298 | 0.234 | 0.188 | 0.122 |
| b6 | 0.471 | 0.171 | 0.262 | 0.144 | 0.162 | 0.066 |
| b7 | 0.412 | 0.180 | 0.260 | 0.129 | 0.226 | 0.020 |
| b8 | 0.456 | 0.203 | 0.467 | 0.187 | 0.203 | 0.116 |
| b9 | 0.503 | 0.244 | 0.512 | 0.238 | 0.212 | 0.029 |
| b10 | 0.457 | 0.265 | 0.520 | 0.273 | 0.232 | 0.009 |
| b11 | 0.408 | 0.280 | 0.548 | 0.292 | 0.227 | −0.006 |
| b12 | 0.391 | 0.295 | 0.541 | 0.285 | 0.248 | 0.045 |
| b13 | 0.418 | 0.269 | 0.556 | 0.253 | 0.247 | 0.106 |
| b14 | 0.392 | 0.258 | 0.526 | 0.282 | 0.208 | 0.128 |
| b15 | 0.360 | 0.287 | 0.506 | 0.276 | 0.234 | 0.098 |
| b16 | 0.327 | 0.270 | 0.522 | 0.293 | 0.263 | 0.141 |
| b17 | 0.303 | 0.265 | 0.559 | 0.217 | 0.247 | 0.246 |
| b18 | 0.304 | 0.275 | 0.578 | 0.247 | 0.290 | 0.246 |
| b19 | 0.283 | 0.306 | 0.515 | 0.230 | 0.327 | 0.213 |
| b20 | 0.303 | 0.303 | 0.504 | 0.241 | 0.288 | 0.288 |
| b21 | 0.246 | 0.653 | 0.322 | 0.210 | 0.244 | 0.063 |
| b22 | 0.245 | 0.713 | 0.296 | 0.225 | 0.236 | 0.066 |
| b23 | 0.255 | 0.703 | 0.278 | 0.236 | 0.223 | 0.128 |
| b24 | 0.244 | 0.737 | 0.199 | 0.202 | 0.249 | 0.097 |
| b25 | 0.241 | 0.745 | 0.185 | 0.211 | 0.274 | 0.108 |
| b26 | 0.254 | 0.708 | 0.203 | 0.234 | 0.269 | 0.170 |
| b27 | 0.365 | 0.398 | 0.298 | 0.308 | 0.234 | 0.425 |
| b28 | 0.311 | 0.319 | 0.348 | 0.263 | 0.263 | 0.559 |
| b29 | 0.347 | 0.337 | 0.331 | 0.303 | 0.328 | 0.391 |
| b30 | 0.270 | 0.323 | 0.273 | 0.298 | 0.422 | 0.138 |
| b31 | 0.261 | 0.317 | 0.218 | 0.280 | 0.461 | 0.083 |
| b32 | 0.250 | 0.315 | 0.292 | 0.340 | 0.503 | 0.169 |
| b33 | 0.232 | 0.231 | 0.219 | 0.191 | 0.688 | 0.119 |
| b34 | 0.255 | 0.222 | 0.226 | 0.234 | 0.689 | 0.135 |
| b35 | 0.194 | 0.289 | 0.178 | 0.251 | 0.618 | 0.023 |
| b36 | 0.245 | 0.295 | 0.248 | 0.301 | 0.599 | 0.057 |
| b37 | 0.257 | 0.270 | 0.319 | 0.429 | 0.441 | 0.170 |
| b38 | 0.291 | 0.260 | 0.325 | 0.465 | 0.392 | 0.221 |
| b39 | 0.305 | 0.252 | 0.282 | 0.520 | 0.319 | 0.169 |
| b40 | 0.287 | 0.245 | 0.285 | 0.643 | 0.269 | 0.143 |
| b41 | 0.277 | 0.242 | 0.245 | 0.679 | 0.277 | 0.090 |
| b42 | 0.243 | 0.255 | 0.233 | 0.752 | 0.263 | 0.077 |
| b43 | 0.239 | 0.258 | 0.238 | 0.720 | 0.266 | 0.080 |
The factors of the indicator system can explain more than 50% of the variation, and each item had enough loading on the corresponding factors (≥0.4), which indicated that the indicator system had good structural validity.
Results of reliability analysis
We adopted the statistical methods of Spearman–Brown split-half reliability and Cronbach’s α coefficient for reliability analysis. The outpatient and inpatient satisfaction indicator system was confirmed by structural validity analysis, and their respective primary indicators were analyzed by the statistical method of split-half reliability and consistency of internal reliability. The results showed that the overall Cronbach’s α coefficient of the outpatient satisfaction indicator system was 0.956, and Cronbach’s α coefficient of each primary index was between 0.860 and 0.946, and the internal consistency reliability was excellent. The overall split-half reliability was 0.956, and split-half reliability of each primary index between 0.906 and 0.976. The split-half reliability was excellent, which was consistent with the evaluation results of Cronbach’s α coefficient (Table 8).
Table 8.
Reliability evaluation of the indicator system for the outpatient and inpatient satisfaction
| Type of patient | Primary index | Cronbach’s alpha coefficient | Split-half reliability |
|---|---|---|---|
|
| |||
| Outpatient | |||
| Pre-diagnosis | 0.859 | 0.905 | |
| Treatment | 0.920 | 0.916 | |
| Auxiliary examination | 0.946 | 0.976 | |
| Drug withdrawal | 0.931 | 0.916 | |
| Environment and logistical services | 0.875 | 0.923 | |
| Indicator system | 0.952 | 0.982 | |
| Inpatient | |||
| Nonsurgical inpatient | Hospitalization service | 0.888 | 0.893 |
| Treatment | 0.952 | 0.938 | |
| Auxiliary examination | 0.945 | 0.945 | |
| Service attitude | 0.882 | 0.885 | |
| Environment and logistics services | 0.927 | 0.921 | |
| Indicator system | 0.977 | 0.971 | |
| Surgical inpatient | Hospitalization service | 0.893 | 0.908 |
| Treatment | 0.958 | 0.951 | |
| Auxiliary examination | 0.947 | 0.952 | |
| Service attitude | 0.891 | 0.896 | |
| Environment and logistics services | 0.932 | 0.927 | |
| Surgical anesthesia | 0.931 | 0.938 | |
| Indicator system | 0.980 | 0.978 | |
The overall Cronbach’s α coefficient of the satisfaction indicator system of inpatients who did not undergo surgery was 0.975, Cronbach’s α coefficient for each primary index was between 0.887 and 0.947 and the internal consistency reliability was excellent. The overall split-half reliability was 0.967, and split-half reliability of each primary index ranged between 0.882 and 0.944. The split-half reliability was excellent, which was consistent with the evaluation results of Cronbach’s α coefficient.
The overall Cronbach’s α coefficient of the satisfaction indicator system for surgical inpatients was 0.976, and Cronbach’s α coefficient for each primary index was between 0.882 and 0.951, and the internal consistency reliability was excellent. The overall split-half reliability was 0.976, and split-half reliability of each primary index ranged between 0.885 and 0.946. The split-half reliability was excellent, which was consistent with the evaluation results of Cronbach’s α coefficient.
Discussion
Our research is based on the satisfaction questionnaire project of the national doctor–patient experience research center with questionnaire data obtained from outpatients and inpatients between 2016 and 2017. After determining the indicator system by exploratory factor analysis, the outpatient and inpatient satisfaction indicator systems were analyzed by the statistical method of split-half reliability and consistency of internal reliability. The results showed that all of the satisfaction indicator systems of outpatients, nonsurgical inpatients and surgical inpatients have good structural validity and excellent reliability.
In previous studies, Chen and Zheming had initially developed an IPSQ after reviewing the literature and consulting hospital management experts.16 The first eight factors were admission process, cost, doctor’s service, food supply, assistant department service, nursing, treatment results, medical environment and facilities. However, as the internal consistency was low (α coefficient of admission process and cost was 0.22 and 0.36, respectively), the researchers considered revising the structure of the questionnaire. Wang et al17 extracted five factors through exploratory factor analysis, namely, nursing service, discharge guidance, related services, doctor diagnosis and treatment and logistic support. The overall Cronbach’s α of the questionnaire was 0.868, and the Cronbach’s α of the internal consistency of each primary index was between 0.354 and 0.834.17 There are some deficiencies in the reliability of the indicator systems described above. In recent years, Yangzi has constructed a Chinese patient satisfaction questionnaire containing seven dimensions (doctor–patient relationship, doctor–patient communication, medical service, auxiliary service, environmental sanitation, medical ethics, procedure signs).18 The Cronbach’s α of internal consistency of the questionnaire was 0.975, the Cronbach’s α of seven dimensions was between 0.815 and 0.938, the overall split-half reliability 0.93, and the split-half reliability of each dimension lay between 0.816 and 0.913. Although the structure of the indicator system constructed in the above study was similar to that constructed in the present research, the reliability of the indicator system was better than that of the above study (vide supra).
The exploratory factor analysis of the surgical patients’ indicator system established in our study showed that the factor loading of the secondary index “overall service flow” was 0.395 (<0.4). There was a misclassification in the principal component analysis of the secondary index “medical ethics”. It is not classified under the primary index “environment and logistic services” which it subordinates to. Instead, it is classified under the primary index “surgical anesthesia”. First, our study did not adjust or modify the indicator system directly according to the results of exploratory factor analysis. The main reason is that the indicator system built in our study was based on the Delphi method. Second, the reliability of the indicator system constructed was excellent, indicating that the current indicator system is more than satisfactory. Third, the factor loading of the secondary index “overall service flow” in the surgical patient satisfaction indicator system was <0.4 (>0.3), but it was >0.4 in the nonsurgical patient satisfaction indicator system. Furthermore, it will facilitate the further comparative analysis of the index score of each factor in the surgical and nonsurgical IPSQ. As a result, our study retained the index in the indicator system for evaluating satisfaction of surgical inpatients. On the other hand, the second-level index “medical ethics” has been misclassified in the analysis of disaster factors, but the factor loading in its primary index “environment and logistic services” was 0.392–0.3. This will facilitate the further comparative analysis of surgical and nonsurgical inpatient satisfaction and consider that medical ethics is an important factor influencing patient satisfaction in China.19–21 Finally, based on comprehensive expert opinions and a comprehensive survey of the relevant literature, this study retained the secondary index “medical ethics” in the surgical inpatients satisfaction indicator system and subordinated it in the primary index “environment and logistic services”.
However, there are some limitations in this study. First, we only developed an indicator system for evaluating satisfaction of outpatients, surgical inpatients and nonsurgical inpatients in general. Further evaluation for specific diseases still remains to be constructed. We would like to develop an indicator system including different dimensions of departments or diseases in the future research. Besides, in order to improve the health care services of the hospitals, further investigation about impact factors of patients’ satisfaction among different institutions, regions, and characteristic of people need to be further explored.
Conclusion
The indicator systems constructed through exploratory factor analysis for evaluating satisfaction of outpatients, surgical inpatients and nonsurgical inpatients all had excellent reliability. They can be widely used in an outpatient and IPSQ in Chinese hospitals.
Ethics approval and consent to participate
This study was conducted in accordance with the Declaration of Helsinki. Written informed consent has been provided by the patients, or a parent or legal guardian if under the age of 18 years. Data in our study were obtained from electronic hospitalization summary reports (HSRs) of the National Healthcare Data Center of China. The data were masked by the National Healthcare Data Center (data source) and there is no patient name, identification number, contact information and other privacy information in our database. Therefore, any patient privacy was well protected. In addition, the data we used in our study are freely available. Thus, an ethic approval is not necessarily for our study.
Data sharing statement
The datasets analyzed during the current study will be available from the corresponding author on reasonable request.
Footnotes
Disclosure
The authors report no conflicts of interest in this work.
References
- 1.Urden LD. Patient satisfaction measurement: current issues and implications. Lippincotts Case Manag. 2002;7(5):194–200. doi: 10.1097/00129234-200209000-00006. [DOI] [PubMed] [Google Scholar]
- 2.Liu S. doctoral thesis. Jilin: Jilin University; 2013. Construction and Empirical Analysis of Patient Satisfaction Index Model in Large Comprehensive Hospital. [Google Scholar]
- 3.Baker R, Streatfield J. What type of general do patients refer? Exploration of practice characteristics influencing patient satisfaction. Br J Gen Pract. 1995;45(401):654–659. [PMC free article] [PubMed] [Google Scholar]
- 4.Carey RG, Posavac EJ. Using patient information to identify areas for service improvement. Health Care Manage Rev. 1982;7(2):43–48. [PubMed] [Google Scholar]
- 5.Ware JE, Jr, Snyder MK, Wright WR, Davies AR. Defining and measuring patient satisfaction with medical care. Eval Program Plann. 1983;6(3–4):247–263. doi: 10.1016/0149-7189(83)90005-8. [DOI] [PubMed] [Google Scholar]
- 6.Wilkins D, Hallam L, Dogget MA. Measures of Need and Outcome for Primary Health Care. Vol. 36. Oxford; Oxford University Press; 1992. [Google Scholar]
- 7.Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. Oxford; Oxford University Press; 1995. pp. 5–7. [Google Scholar]
- 8.Chen X. The Application of Analytic Hierarchy Process in the Analysis of Out-patient Satisfaction. Shanxi: Shanxi Finance and Economics University; 2011. [Google Scholar]
- 9.Kong Y, Chen J. The reliability analysis of the expert consultation method in the community health service satisfaction index system. J Prev Med Chin People’s Liberation Army. 2007;25(4):259–261. Chinese. [Google Scholar]
- 10.Tan L. Construction and Application of the Index System for Evaluating Patient Satisfaction in Grade III Level A Hospitals. Guangdong: Wuyi University; 2016. [Google Scholar]
- 11.Xia P, Huang H, Lv Y, et al. The establishment and improvement of customer satisfaction measurement and management system of Guangdong Provincial Hospital of Chinese Medicine. Chin Hosp. 2010;14(10):43–47. Chinese. [Google Scholar]
- 12.Yuan X, Zhang X, Dechuan J. The status and thinking of hospital patient satisfaction survey. Med Soc. 2010;23(8):50–51. Chinese. [Google Scholar]
- 13.Yang H, Liu F, Zhang T, Shane T. Problems with patient satisfaction surveys and suggestions. Chin J Hosp Admin. 2005;21(7):437–441. Chinese. [Google Scholar]
- 14.Kim JO, Charles MW. Introduction to Factor Analysis. Shanghai: Truth and Wisdom Press; 2016. [Google Scholar]
- 15.DeVellis RF. Scale Development: Theory and Applications. Chongqing: Chong Qing University Press; 2010. [Google Scholar]
- 16.Chen P, Zheming H. Investigation and assessment of satisfaction level of health service clients. Chin Hosp Manag. 1999;7:19–22. Chinese. [Google Scholar]
- 17.Wang Y, Yu Z, Jin X, Shao MS. The development and reliability test of the satisfaction questionnaire for hospitalized patients. J Nurs Train. 2013;28(6):533–536. Chinese. [Google Scholar]
- 18.Yang-zi CH. The Development and Validation of Chinese Patient Satisfaction Questionnaire. Anhui: Wan Nan Medical College; 2016. [Google Scholar]
- 19.Chen X. Theoretical and empirical study on patient conative loyalty model. J Third Mil Med Univ. 2009 Chinese. [Google Scholar]
- 20.Sun Y, Chen L, Chen N. The effect of grading-up and recombinating flow-sheet on out-patient’s waiting time. Modern Clin Nur. 2006;5(4):63–64. Chinese. [Google Scholar]
- 21.Zhou W, Zhang Y, Fan S, et al. Analysis of the effect of patient satisfaction on loyalty in outpatient department in Beijing municipal hospital. Hosp Manag Forum. 2015;32(11):18–20. [Google Scholar]