Comparison of the Assessment of Patient Satisfaction Using Active and Passive Feedback

Abstract

Introduction

The outpatient department of any hospital is the first direct point of contact to the patients with the hospital. To understand the difficulties faced by the patients and to understand their perceptions, it is important to assess patient satisfaction. This study was designed to compare the difference in patient satisfaction responses and outcomes using two methods: active feedback collection (AFC) and passive feedback collection (PFC).

Methods

The study was conducted for a period of 2 months using a validated, structured questionnaire in four languages. To differentiate the questionnaires, those for PFC were marked P and those for AFC as A. The questionnaire consisted of 21 questions. PFC was obtained when patients voluntarily filled out the feedback forms placed at different locations, and AFC was obtained by systematically approaching randomly selected patients.

Results

Of the 809 patients who participated in the study, 131 were passive and 678 active. The study revealed that the satisfaction level was higher in the AFC group. It was observed that 82% of those in the PFC group and 35% of those in the AFC group had given specific written comments. The negative comments were higher in the PFC group than in the AFC group.

Conclusions

The AFC method gives a good overview of the patients' journeys through the system and it can be used for systemic feedback collection. The PFC method provides an avenue to get more written suggestions and adverse comments that could help in planning remedial measures. The study showed that both methods collect complementary information for the managers to facilitate improvement of services.

INTRODUCTION

Over the past few decades, patient satisfaction surveys have gained increasing attention as a useful source of information for identifying gaps, and developing effective action plans for quality improvement measures in healthcare organizations.[1] Patient satisfaction surveys are the best tools for gathering feedback, when we want to know the views of a large number of people using the service.[2] These studies help in understanding the patterns and trends, show if a problem is occurring more or less frequently over time, and the types of people who seem most likely to experience it.

Selecting the appropriate methods to measure patient satisfaction is a critical challenge for healthcare managers because there is limited knowledge of the different methods and their relative merits. In their study on the measurement of satisfaction with healthcare, Crow and Gage[3] recommended that researchers should seek to collect high-quality information about consumers' views, and should pay particular attention to how different ways of conducting surveys affect response rates and patients' evaluations. While doing sampling methods, they should ensure that disadvantaged groups are included, and while promoting high response rates, ensure protection against low response bias. There are two approaches for evaluating patient satisfaction—qualitative and quantitative. Qualitative approaches provide in-depth information, in contrast to the reductionism implied by quantitative approaches.[3] In his study on patient satisfaction in primary healthcare, Pascoe[4] defined satisfaction as an individual's experience compared with his or her expectations. Recent studies show that current research is less interested in correlations between patients' expectation and satisfaction, and more focused on improving the quality of care and service delivered to patients.[4]

Many authors tend to have different perceptions of patient satisfaction. Jenkinson et al[5] and Ahmed et al[6] pointed out that patient satisfaction mostly appears to represent attitudes toward care or aspects of care. While Mohan et al[7] referred to patient satisfaction as patients' emotions, feelings, and their perception of delivered healthcare services.

Patient satisfaction measurement tools should be reliable and valid in order to precisely measure and understand the main goal of collecting patients' feedback.[8] The quantitative approach helps to measure patient satisfaction more accurately. Administering standardized questionnaires using different methods has been the most commonly used assessment tool for conducting patient satisfaction studies.[9,10]

Kulkarni et al[11] has recommended the placement of suggestion boxes in all outpatient departments, so that patients can freely submit their complaints and suggestions for improvement of services provided in the hospital. The assessment of the suggestions should be done periodically.

There are different methods of collecting patient feedback. Each method has its own advantages and disadvantages. Identifying the right method of feedback collection can go a long way in understanding the patients. This study was conducted using two different methods of feedback collection termed active feedback collection (AFC) and passive feedback collection (PFC). The objective of the study was to understand the difference between these methods, in terms of the information collected and the quality of patient feedback.

METHODS

This study was approved by the institutional review board (IRB) at Christian Medical College Hospital, Vellore, India and was performed in accordance with the Declaration of Helsinki. The requirement for informed consent was waived by the IRB.

This study was carried out for a period of 2 months at clinics in the outpatient department of Christian Medical College Hospital, Vellore, India with the approval of the IRB. A structured questionnaire was prepared with various categories covering the registration process, guidance given by the reception staff, care given by the doctors, care given by the nurses, experiences at pharmacy counters, and so on. In all, there were 21 questions, each measured on 5-point Likert scale, with 1 as very poor and 5 as excellent. Space was provided for patients to write their feedback on matters not covered by the questionnaire.

Face validity of the questionnaire was done and subsequently pilot tested with 50 respondents. Internal consistency was checked using the Cronbach alpha measure. The questionnaire was highly reliable with a Cronbach alpha value of 0.932. Questionnaire was prepared in four languages because the hospital caters to patients from these linguistic groups. For both methods, the same questionnaire was used. To differentiate between the questionnaires, P was stamped on the questionnaire for the PFC method, and A for the AFC method.

In AFC, the questionnaire was administered by trained research staff to randomly selected patients. There are 94 clinics, serving more than 7000 patients a day. Each day three clinics were covered and over the period of 2 months all the clinics were covered twice. From each clinic 7% of the total number of patients who registered were chosen by using a stratified random sampling technique; every seventh patient in the appointment schedule was selected. Per statistical analysis, 10% of the population (∼700 participants) would be the representative sample. With an expected drop rate of more than 60%, it would result in surveying 1500 samples to achieve the desired numbers. We planned to include every seventh sample in each clinic two times, which resulted in 1680 samples. Among them, approximately 40% responded per the expectation, resulting in 678 responses, which is nearly 10% of the total population. They were requested to drop the completed forms into the boxes kept for this purpose. A total of 1680 forms were administered in 2 months and 678 responses were received for a 40% response rate.

The PFC process involved placing the questionnaires, marked P in all outpatient clinics. Passive feedback is provided when patients voluntarily fill out the questionnaires and deposit them in the boxes provided at different locations. Blank questionnaires were replenished regularly, and the completed questionnaires deposited in the boxes were collected daily. Both AFC and PFC were carried out during the same two months. The data collected were analyzed to compare the satisfaction level of both the groups. The written comments were collated and were identified as positive or negative.

Data were summarized using mean and standard deviation for continuous variables and the categoric data were expressed as frequency along with percentages. The mean score of each item in the questionnaire was compared using a z-test for means. The proportion of comments and negative comments among the passive and active responses were compared using a z-test. All the proportion estimates and their differences were presented with 95% CI. The analysis was done using STATA IC 16.0 (StataCorp, College Station, TX).

RESULTS

A sample size of 809 patients participated in the study, of which 131 belonged to the PFC group and 678 to the AFC group. Table 1 captures the response of the subjects. It appears that the satisfaction level was higher in the active group on every single item and the difference was statistically significant. The mean difference presented is calculated as the difference in mean score between the two groups.

Table 1.

Comparison of passive and active group satisfaction levels by item

Question	Total, mean (SD)	Passive, mean (SD)	Active, mean (SD)	Mean Difference	p -Value
Q1	2.76 (1.42)	2.31 (1.25)	2.86 (1.44)	−0.55	< 0.001
Q2	3.63 (0.93)	3.04 (1.2)	3.75 (0.82)	−0.71	< 0.001
Q3	3.64 (0.95)	2.95 (1.13)	3.78 (0.84)	−0.83	< 0.001
Q4	3.01 (1.1)	2.55 (1.12)	3.09 (1.07)	−0.54	< 0.001
Q5	4.03 (0.86)	3.45 (1.22)	4.13 (0.73)	−0.68	< 0.001
Q6	4.07 (0.88)	3.38 (1.21)	4.19 (0.74)	−0.81	< 0.001
Q8	3.7 (0.93)	2.79 (1.11)	3.89 (0.77)	−1.1	< 0.001
Q7	3.75 (0.92)	2.87 (1.14)	3.92 (0.76)	−1.05	< 0.001
Q9	3.91 (0.88)	3.31 (1.18)	4.02 (0.76)	−0.71	< 0.001
Q10	3.79 (1.11)	3.4 (1.25)	3.87 (1.07)	−0.47	< 0.001
Q11	3.84 (0.85)	3.22 (1.03)	3.96 (0.76)	−0.74	< 0.001
Q12	3.73 (0.85)	3.12 (1.02)	3.85 (0.75)	−0.73	< 0.001
Q13	3.97 (0.85)	3.35 (1.09)	4.09 (0.73)	−0.74	< 0.001
Q14	3.88 (0.87)	3.21 (1.14)	4.01 (0.74)	−0.8	< 0.001
Q15	3.83 (1.03)	3.14 (1.28)	3.96 (0.91)	−0.82	< 0.001
Q16	3.68 (1.08)	2.91 (1.36)	3.83 (0.95)	−0.92	< 0.001
Q17	3.7 (0.93)	3.1 (1.07)	3.82 (0.86)	−0.72	< 0.001
Q18	3.56 (1.06)	2.78 (1.18)	3.71 (0.96)	−0.93	< 0.001
Q19	3.58 (0.94)	2.77 (1.07)	3.74 (0.82)	−0.97	< 0.001
Q20	4 (1.01)	3.18 (1.35)	4.16 (0.84)	−0.98	< 0.001
Q21	4.23 (1)	3.44 (1.36)	4.39 (0.83)	−0.95	< 0.001

Overall, 83.2% (95% CI: 76.70, 89.20) of the PFC and 35.3% (95% CI: 31.70, 39.00) of the AFC group had given written comments (Table 2). The difference was statistically significant (p < 0.001). The AFC group had more written comments than the PFC group.

Table 2.

Written comments in active and passive feedback methods

Comments	Passive, n (%)	Active, n (%)	Total, n (%)
Written	109 (83.2)	239 (35.3)	347 (42.9)
Not Written	22 (16.8)	439 (64.7)	461 (57.1)
Total	131 (100.0)	678 (100.0)	808 (100.0)

The negative comments in the PFC group were 93.58% (95% CI: 88.98, 98.18) and AFC group were 79.08% (95% CI: 73.92, 84.24) (Table 3), and the difference was statistically significant (p < 0.001). Negative comments were more common in the PFC group than in the AFC.

Table 3.

Positive and negative comments in both feedback methods

Comments	Passive, n (%)	Active, n (%)	Total, n (%)
Positive comments	3 (2.8)	39 (16.3)	42 (12.1)
Negative comments	102 (93.6)	189 (79.1)	291 (83.6)
Both comments	4 (3.7)	11 (4.6)	15 (4.3)
Total	109 (100.1)	239 (100.0)	348 (100.0)

DISCUSSION

This study highlights the difference in the results of the assessment of patient satisfaction between the AFC and PFC groups. Choosing an appropriate method to capture patient feedback for quality improvement is a challenge for healthcare managers. Assessing patient satisfaction helps in understanding the difficulties faced by the patients and the adequacy of the service provided.

The comparative analysis of satisfaction level between the two groups shows that the AFC group reported a higher satisfaction level compared with the PFC group. The following factors could have played a role in AFC group: (1) most of these patients would not have considered providing feedback and when the questionnaires were handed out they may have had limited time to think through their responses; (2) it is likely that they could have felt that their responses were not confidential because they were approached by a researcher; and (3) there may have been a fear that negative feedback could result in adverse responses from the healthcare providers. Active feedback was taken after they had completed their consultations with the doctors and, at this point, the positive impact of meeting with the physicians and the relief of having completed the treatment process could be another explanation for this. This is similar to the findings of the study done by Kinnersley et al[12] in which the satisfaction level of patients completing the questionnaires immediately after the consultations was higher than those completing the questionnaires later at home.

In the passive group there were more negative comments and the satisfaction level was also low. The explanations for this could be that it was the patients who made the decision to give the feedback. They were therefore likely to give feedback when they had strong feelings, either positive or negative. It was more likely that they might be inclined to give feedback if their experience had been negative because they might look at completing the questionnaire as a mode of grievance redressal. It is also likely that the responses from passive feedback were more thought through.

The response rate for active feedback group was 40%. Response rate ensures reliability and credibility of the results and permits the research to be generalized to the larger population.[13] In the study done by Godden et al[14] on the impact of response rate on Hospital Consumer Assessment of Healthcare Providers and System (HCAHPS), they found that with increased response rates, hospitals received valuable feedback from a higher proportion of patients, creating a more representative sample of the total hospital patient population. While we could get feedback from an adequate number of patients in AFC, it was not possible to ensure feedback from a representative group in the PFC. Of the total sample of 809 responses, only 131 were collected through the PFC method. It still holds great value because it helps to capture feedback of patients who are either aggrieved or are desirous of complimenting the services. Furthermore, it saves them the effort of seeking out the appropriate person who may not also be available all the time, to report their grievances. Therefore, placing feedback forms in crucial locations is essential to reduce frustration of such patients. Also, placing feedback forms in multiple strategic locations gives the impression to the patients that the organization is keen to hear from them.

Although the number of respondents were fewer the in PFC group compared with the AFC group, the PFC group had more written comments compared with the AFC group. This is particularly important because the written comments give a broader perspective of the overall experience and whether it is positive or negative. Recent research shows that numerical ratings do not sufficiently capture the range of consumer experiences and that comments contain additional information that complements survey responses.[15]

Per the analysis, of the total comments received it was found that the negative comments were more in the PFC group than the AFC group. This suggests that the patients chose to give the feedback more when they were aggrieved rather than when they were satisfied. In the study done by Huppertz and Smith[16] on the value of patients' handwritten comments on HCAHPS surveys, they found that patients who wrote negative comments gave the hospitals significantly lower satisfaction and quantitative HCAHPS ratings. In addition, quantitative HCAHPS ratings appear to underestimate the feelings of people who write negative comments, validating practices at hospitals that use surveys containing negative anecdotes in quality improvement initiatives. It is found that these comments contain additional information that complements the quantitative responses. Considering only the quantative responses for decision making underestimates the feelings of people who write negative comments. This suggests that negative comments should be taken as suggestions that will help managers to improve the facilities per patients' expectation.

Conclusion

AFC gives a good overview of the patients' journeys through the system and it can be used for systemic feedback collection. PFC provides more written suggestions and adverse comments that could help in planning remedial measures. This study shows that both methods provide complementary information for the managers to facilitate improvement of services. Thus, it may be prudent to use both these methods in healthcare settings.

Funding Statement

Source of support: None.