Assessing residents’ knowledge of patient satisfaction: a cross-sectional study at a large academic medical centre

Abstract

Objectives

Patient satisfaction impacts healthcare quality and outcomes. Residents play an important role in patient satisfaction at academic institutions. This study aims to assess residents’ patient satisfaction knowledge and determine which learning experiences contributed to their knowledge acquisition.

Settings

This study was conducted at a health science university in a large, urban, tertiary-care academic medical centre in the USA.

Participants

All residents from internal medicine (n=185) and paediatrics (n=156) were asked to participate.

Design

Residents completed a survey from April 2013 to December 2013 that assessed (1) knowledge of factors that impact patient satisfaction and (2) learning experiences that may have contributed to their understanding of the drivers of patient satisfaction (eg, experiential (personal or clinical) or didactics). Trainees identified the importance of factors in determining patient satisfaction on a five-point Likert scale; answers were compiled into a knowledge score. The score was correlated with prior personal/clinical experience and didactics.

Results

Of the 341 residents, 247 (72%) completed the survey. No difference was found in knowledge among training levels or residency programme. More than 50% incorrectly thought physician board certification, patient’s education, patient’s income and physician’s age impacted satisfaction. Personal experience, through hospitalisation of a relative or friend, was correlated with higher knowledge (67% vs 71%, p=0.03). Ninety-nine per cent (n=238) stated peer observation, and all stated faculty feedback impacted their patient satisfaction knowledge. Seventy-seven per cent (n=185) had attended didactics on satisfaction, but attendance did not correlate with higher scores.

Conclusions

Our study showed trainees have a few gaps in their patient satisfaction knowledge, and attending past educational sessions on patient satisfaction did not correlate with higher knowledge scores. Our data suggest that academic centres should leverage residents’ personal experiences, their observations of peers and faculty feedback to enhance patient satisfaction knowledge.

Strengths and limitations of this study.

• Residents serve a dual role as providers and learners in academic settings and can greatly influence hospital quality metrics, specifically patient satisfaction. However, few studies have assessed residents’ knowledge of what drives overall patient satisfaction or determined which types of learning experiences correlate with residents’ acquisition of knowledge.

• Recognising gaps in residents’ knowledge of patient satisfaction allows hospital administrators and academic institutions to develop targeted, practical and sustainable interventions to augment trainee knowledge and improve patient care, experience and reimbursement.

• The study was performed at a large health sciences university with multiple educational clinical sites in a single academic medical centre.

• Residents’ knowledge scores were not correlated with clinical performance, including provider satisfaction scores or patient evaluation of residents, due to the anonymity of the survey.

Background

Patient satisfaction is an important component of patient-centred care; it is linked to healthcare quality and associated with improved compliance and adherence.^1–5 Patient satisfaction and patient experience have received increasing emphasis in healthcare institutions in the USA since the Affordable Care Act (a comprehensive healthcare reform act in the USA enacted in 2010), as a hospital’s reimbursement is impacted by the value of care it provides rather than traditional fee for service. The ‘value’ is calculated by the hospital’s value-based total performance score, which includes several domains, one of which is patient satisfaction.^{6 7} Although value-based care purchasing applies to only the Center for Medicare and Medicaid Services (CMS), other private payers have added satisfaction scores to their pay-for-performance measures.⁶ For CMS, 25% of value-based purchasing will be based on the results of Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS), an instrument to assess patient satisfaction.^8–10 With such emphasis on payment and the link to patient satisfaction and improved patient outcomes, it is incumbent on all caregivers, particularly physicians in training, to understand what contributes to patients being satisfied with their care.

Many factors, such as responsiveness, communication and interpersonal manner of caregivers, are positive determinants, or drivers, of patient satisfaction; however, whether trainees are aware of these factors remains unclear.² Functioning as both learners and providers, residents are important to the framework, quality and outcomes of the healthcare delivered in an academic setting, specifically patient satisfaction.^{11 12} Residents may have been taught some components of patient satisfaction in medical school. However, teaching and learning are not synonymous; and, therefore, the information may not have been retained.^13–16 Previous studies have assessed interventions targeting residents to improve patient satisfaction, such as generalised education and incentives, but the literature regarding residents’ current knowledge of patient satisfaction is sparse.¹⁰ To develop practical, cost-effective, sustainable interventions that benefit the trainee as well as the institution, understanding the gaps in residents’ knowledge regarding drivers or positive determinants of patient satisfaction is a critical and necessary first step to changing their practice. The primary study aim was to assess residents’ knowledge of factors strongly correlated with patient satisfaction, termed drivers. A secondary aim was to determine which types of learning experiences (didactic, personal or clinical) most strongly correlate with residents’ knowledge.

Methods

Setting and participants

This study was conducted at a large, urban health sciences university in a tertiary-care academic medical centre in Houston, Texas, USA. All residents from internal medicine (n=185) and paediatrics (n=156), which comprised approximately 40% of the resident staff in our training institutions, were asked to participate. Residents do not train in one primary university-affiliated training hospital; rather, they rotate through five affiliated institutions. They receive diverse exposure to outpatient and inpatient care across private, federal (Veterans Affairs) and county hospitals.

Participant exposure to patient satisfaction metrics

Patient satisfaction data at the affiliated institutions are collected by a CMS-approved third-party vendor, a private organisation whose questionnaires are used by >7000 facilities in the USA to survey patients regarding their experiences in receiving healthcare at the institution.^{10 17} Residents’ exposure to these data varies between rotation and sites, and is difficult to quantify due to their rotation schedules. Residents rotate through different inpatient or outpatient sites every four weeks, while the patient experience data are usually reviewed monthly or quarterly.

Survey instrument

A 31-item survey was developed through review of patient satisfaction literature, validated surveys and our prior work.^{2 18–24} The questionnaire focused on three concepts: (1) knowledge of factors that influence patient satisfaction, (2) personal and clinical experiences that contribute to a resident’s satisfaction knowledge and (3) prior educational sessions (didactics) received related to patient satisfaction. Crow et al reviewed 139 international articles and 127 data sets and concluded that determinants of patient satisfaction can be broken down into two groups: characteristics of the healthcare delivery system and patient experience.² Patient experience can be a surrogate marker of patient satisfaction; therefore, current validated surveys that assess patient satisfaction via patient experience were also reviewed for themes and important determinants of patient satisfaction. The validated surveys were the HCAHPS survey, Clinician and Group CAHPS for outpatient use and Patient Satisfaction Questionnaire from Rand Health.^{19 20} The questionnaire was developed through an iterative process that incorporated a psychometrician, health services researchers and residency programme faculty. Pilot testing using a think-aloud process was conducted with a group of internal medicine residents who were not part of the study. Internal assessment and feedback from these individuals improved the clarity of the items and the general format.

Knowledge of factors impacting patient satisfaction

Questions 1–20 assessed knowledge of the factors related to patient satisfaction using a five-point Likert-type scale (1=not at all important to 5=extremely important) (see online supplementary appendix 1 survey Instrument). For this study, patient satisfaction knowledge will refer to knowledge of the drivers of patient satisfaction.

Based on our literature review, we identified five domains of patient satisfaction that were assessed in the knowledge portion of the survey (table 1).^{2 18–24} To minimise bias and limit survey length, the final survey had 11 variables consistently related to patient satisfaction and 9 non-drivers that are commonly presumed to affect satisfaction but have been shown not to be associated.^{2 19–24} Inclusion of commonly mistaken non-drivers in the survey was done because the authors thought it was important to also know where the current misconceptions regarding patient satisfaction were to be found.

Table 1.

Domains of patient satisfaction and linked survey question

Driver domains	Abbreviated survey question	Percentage correct (95% CI)
Accessibility, convenience and responsiveness	Responsiveness of ancillary staff	94.1 (90.592 to 96.627)%)
Communication	Physician explanations	96.7 (94.853 to 98.976)%)
	Nurse willingness to listen	96.2 (93.203 to 98.148)%)
	Nurse explanations	80.3 (74.931 to 85.007)%)
	Physician listening	97.1 (94.293 to 98.710)%)
	Discussion about treatment by nurses	67.4 (61.224 to 73.086)%)
Interpersonal manner of caregiver	Courtesy and respect from nurses	92.9 (89.078 to 95.661)%)
	Courtesy and respect from physicians	96.2 (93.203 to 98.148)%)
Personal factors of patient	Poor health status of patient	55.6 (49.300 to 61.861)%)
	Age of patient	15.1 (10.942 to 20.026)%)
Technical quality and care from doctors	Thoroughness and competence of physician	87.4 (82.783 to 91.210)%)

An answer was correct if the resident strongly identified whether the item was related to patient satisfaction (answer of 4 or 5 for true variables/drivers and an answer of 1 or 2 for the non-drivers/false variables). Other responses were deemed incorrect (answer of 1, 2 or 3 for true variables/drivers and an answer of 3, 4 or 5 for the non-drivers/false variables). Selection of ‘3’ for either category was regarded as a neutral response. Scoring was dichotomised to either correct or incorrect and was reported as a proportion of 100% (eg, 10 correct answers=50%).

Experiential learning: personal and clinical experiences

Questions 21–27 explored personal and clinical experiences that may impact knowledge of patient satisfaction. The personal experiences focused on personal or relative’s/friend’s hospitalisation. Clinical experiences included clinical observations of faculty and peers with patients as well as feedback from evaluations of faculty, nurses or patients. All these experiences were assessed independently. This portion was assessed using a four-point Likert scale (1=no impact to 4=large impact; 0=not applicable). The respondents were dichotomised into two groups: those who had the specific experience (response of 1–4) and those who did not (response of not applicable). Each experience item was analysed as a continuous variable (degree of impact) and a categorical variable (dichotomised into those who had the specific experience and those who did not). For this analysis, the degree of impact and the dichotomised responses were correlated with the knowledge score.

Educational sessions (structured didactics) on patient satisfaction

Questions 28–30 explored how often residents had received prior didactics on patient satisfaction. Respondents were asked to approximate how many times they had received an education session on patient satisfaction in medical school or residency. The last question of this section assessed how these didactics were given (eg, lecture, workshops or hospital orientation). Responses to this section were correlated with the knowledge score.

The final question asked residents to respond to the statement, ‘I am confident that my patients are satisfied with the care I provide’. Responses were based on a five-point Likert scale (1=strongly disagree to 5=agree). Demographic data also were collected on participants.

Survey distribution

A member of the research team administered an anonymous survey at resident teaching conferences from April 2013 to December 2013. Each completed survey was assigned a unique study number. Respondents were monitored to prevent duplicate submissions. Participation in the survey was voluntary, and no penalty was given for not participating; incentives were not given.

Statistical analysis

Descriptive statistics were computed on demographic data. Frequencies and mean scores (including trainee year and programme type) were compared using analysis of variance (ANOVA). Point-biserial correlation was used to assess the correlation between the knowledge score and each type of learning experience (didactic or experiential (personal or clinical experiences with hospitalisation)). Frequencies and means of the knowledge score stratified by type of learning experiences were also compared using ANOVA. Spearman’s r was used to assess the correlation between knowledge score and the degree of impact of the experiences. Data were analysed using IBM SPSS Statistics V.23. The study was approved by the Institutional Review Board and the Research and Development Committee.

Results

In total, 112 of 156 paediatric residents (72%) and 135 of 185 medicine residents (73%) completed the survey. We analysed 239 surveys, excluding 8 for incomplete data. Internal medicine-paediatrics residents were grouped with the medicine residents. See table 2 for demographics.

Table 2.

Demographics of participants

	N (%)
Postgraduate year
1	118 (49.4)
2	62 (25.9)
3 and 4	59 (24.7)
Residency programme
Internal medicine	118 (49.4)
Paediatrics	96 (40.2)
Medicine/paediatrics	14 (5.9)
Other*	11 (4.6)
Age†
≤29	201 (84.1)
>29	37 (15.5)
Gender†
Male	95 (39.7)
Female	143 (59.8)
Ethnicity‡
Hispanic	17 (7)
Black	11 (4.6)
White	118 (49.4)
Asian	88 (37)
Other	5 (2)
Additional training†
None	206 (86.6)
Master of Public Health	8 (3.4)
Master of Business Administration	2 (0.8)
Other	22 (9.2)

*This category includes anaesthesia, family medicine, surgery and emergency medicine.

†One respondent did not answer.

‡Two respondents did not answer this question and two respondents selected multiple categories.

Knowledge of factors impacting patient satisfaction

Knowledge scores are shown in table 3. The mean score was 70%. More upper-level residents incorrectly rated physician rank in medical school as a contributor to satisfaction (p=0.02); otherwise, there were no differences in scores among trainee levels. Additionally, no difference was noted in knowledge scores when comparing paediatrics, internal medicine or med-peds (mean score ranged from 68% to 70%). In general, recognition of the correct drivers of patient satisfaction was high (80%), except for recognising that patients’ health status, age and rating of the nurses’ discussion about treatment are important drivers (table 3).² More than half of all residents incorrectly reported the following were drivers of patient satisfaction: physician age, patient income, physician board certification and patient level of education.

Table 3.

Results of patient satisfaction knowledge survey

Driver or non-driver	Abbreviated survey question	Percentage correct (95% CI)
Driver	Poor health status of patient	55.6 (49.300 to 61.861)%)
	Physician explanations	96.7 (94.853 to 98.976)%)
	Responsiveness of ancillary staff	94.1 (90.592 to 96.627)%)
	Thoroughness and competence of physician	87.4 (82.783 to 91.210)%)
	Nurse willingness to listen	96.2 (93.203 to 98.148)%)
	Nurse explanations	80.3 (74.931 to 85.007)%)
	Courtesy and respect from nurses	92.9 (89.078 to 95.661)%)
	Courtesy and respect from physicians	96.2 (93.203 to 98.148)%)
	Physician listening	97.1 (94.293 to 98.710)%)
	Age of patient	15.1 (10.942 to 20.026)%)
	Discussion about treatment by nurses	67.4 (61.224 to 73.086)%)
Non-driver	Physician United States Medical Licensing Exam (USMLE) score	92.9 (89.078 to 95.661)%)
	Income of patient	48.5 (42.236 to 54.871)%)
	Physician age	29.3 (23.780 to 35.296)%)
	Medical school attended	74.1 (68.220 to 79.317)%)
	Board certification status of physician	44.8 (38.547 to 51.119)%)
	Education level of patient	20.9 (16.117 to 26.425)%)
	Physician rank in medical school	90.4 (86.129 to 93.651)%)
	Gender of patient	64.4 (58.208 to 70.315)%)
	Gender of physician	52.3 (45.960 to 58.587)%)

Experiential learning: personal and clinical experiences

Of the 239 respondents, 87 (36%) had been hospitalised and 187 (78%) had experienced hospitalisation with a family member or friend. Hospitalisation of a family member or friend was significantly correlated with higher knowledge of patient satisfaction drivers (p=0.03). A significant relationship was not seen when assessing a trainee’s own hospitalisation.

The majority stated that observing peers (n=238), observing supervisors (n=239) and receiving feedback from faculty evaluations (n=233) influenced their understanding of what affects patient satisfaction (99%, 100% and 97%, respectively); these experiences did not correlate significantly with a higher knowledge score when dichotomised to those who had the experience compared with those who did not. When examining the level of impact (1–4) of these observations, higher-impact ratings for faculty evaluations was the only factor that correlated with significantly higher knowledge scores, but the correlation was weak (p<0.05, correlation coefficient 13; Spearman’s r correlation). Feedback from nurses and patients via evaluations did not significantly correlate with higher knowledge scores. Of note, only 55% (n=132) and 56% (n=134) of trainees reported feedback from nurses and patients, respectively, impacted their patient satisfaction knowledge.

Structured didactics on patient satisfaction

Of the 239 respondents, 185 (77%) had attended an educational session on patient satisfaction. No significant difference was found in the knowledge score of those who had attended a session compared with those who had not. Almost half (47.5%) did not attend an educational session during residency, whereas only 67% had attended a session during medical school (n=161). Nonetheless, 88% (n=211) of respondents agreed or strongly agreed that their patients are satisfied with their care. There was no significant difference in knowledge score of residents who rated a higher level of confidence.

Discussion

Residents or physicians in training are important contributors to patients’ experiences in academic institutions and are key determinants of patient satisfaction.4 10–12 18 We conducted a study to gauge residents’ knowledge of drivers of patient satisfaction. We found the mean knowledge score for all resident levels to be 70%. Interestingly, the score did not increase with training level. Residents had difficulty recognising that personal and demographic features of patients can affect their satisfaction. Specifically, they understood that interpersonal and communication skills are important (except for discussion about treatment by nurses), but surprisingly, they thought patients cared about board scores and board certification status.

Identifying the most common and universally accepted drivers of patient satisfaction can provide a foundation for curricula to address trainee knowledge gaps in these areas. Knowing what residents currently know is an important first step to changing their practices.^13–16 Based on these results, a suggested starting point may be curricula geared towards augmenting nurse inclusion in treatment plans given this was an area of deficiency. Regarding patient features, patients with more comorbidities may have lower satisfaction scores; however, how acute illness affects satisfaction remains undetermined.² These patients have specific preferences based on their health status. Learning how to affectively decipher these preferences before making treatment plans is a potential focus area for future curricula.

We also investigated what factors contribute to residents’ patient satisfaction knowledge to guide future educational initiatives. Most residents have not been hospitalised, which is not surprising given the age of trainees in this study (84% were younger than 30 years). Hospitalisation of a family member or friend, however, resulted in a significantly higher knowledge of patient satisfaction compared with those who did not have this experience. Even though most residents had experienced the patient’s perspective on hospitalisation via a relative, friend or personal hospitalisation, these experiences may not be a practical approach to guide educational initiatives for trainees who did not have these experiences. A significant relationship was not seen when assessing a trainee’s own hospitalisation, likely due to the low number of individuals who had previously been hospitalised. On the other hand, peer and faculty observations are major ways residents are taught about patient satisfaction. Faculty role modelling is known to influence resident education, specifically shaping trainees’ values, attitudes and ethics; the modelling provided by faculty behaviour is known as the hidden curriculum.²⁵ Peer observation and feedback are useful for medical learners as well, especially in the development of professionalism, teamwork and interprofessional skills.^{26 27} These topics may provide a potential area of focus to augment learning on patient satisfaction.

We also looked at structured didactics that aim to teach residents about patient satisfaction. Although 185 of 239 respondents (77%) had attended an educational session on patient satisfaction, most of these sessions were delivered in medical school, which may explain the lack of increase in knowledge with trainee years. Teaching and learning are not interchangeable, and there is a complex interplay of many factors, such as attention, cognitive load and practice, that result in knowledge being retained in long-term memory. Knowledge that is not used, rehearsed or revisited is often forgotten. These factors may explain the lack of improvement in knowledge following a previous lecture on satisfaction.^13–15 The details of these didactic sessions were not addressed by our survey, thereby precluding the ability to make any comment on the content. A recent single-site survey found that bundled interventions that included didactics, real-time feedback on patient satisfaction scores and monthly recognition of trainees with high scores resulted in improvement in patient satisfaction scores.¹⁸ Moreover, didactics in patient satisfaction alone may not be sufficient to augment patient satisfaction knowledge and change physician behaviour.

Limitations

The study was performed at a single academic centre, which may limit generalisability. However, the participants in this study rotate through multiple, highly diverse affiliated institutions, and the results are likely externally valid. The survey tool, although developed through an iterative process and pilot tested, has not previously been validated. The survey was however grounded in the literature and derived from validated survey measures of patient satisfaction developed for performance metrics purposes. Several questions asked the residents to recall previous learning experiences, and the results may be limited by recall bias. Another limitation is the lack of correlation with the resident’s clinical performance. The anonymous nature of the survey prevented us from correlating knowledge scores, experiences and confidence of a resident with actual provider satisfaction scores or evaluations from patients, which would have enabled us to correlate knowledge with behaviour. However, as the first step in understanding the trainees’ experiences, the benefit of anonymity and honesty of reporting outweighed the ability to correlate knowledge with actual clinical behaviour.

Conclusion

With the move towards patient-centred care, patient satisfaction and experience are core outcomes for hospitals today.⁶ Residents are important providers of medical care in academic institutions; their daily interactions with patients impact satisfaction, hospital quality metrics and reimbursement.4 10–12 18 We demonstrated some gaps in knowledge concerning drivers of patient satisfaction that did not vary by training year. Residents’ awareness of patient satisfaction was impacted by experiential learning (clinical and personal), specifically hospitalisation of a close contact, peer and faculty observation, and faculty evaluations. Hospital administrators and educators should recognise didactics alone may not be sufficient to augment trainees’ knowledge of factors related to patient satisfaction. More consideration should be given to the effect of peer and faculty role modelling when developing future interventions to improve satisfaction for patients cared for by resident providers.