The Effect of Medical Student Teaching on Patient Satisfaction in a Managed Care Setting

Abstract

OBJECTIVE

To measure the effect on patient satisfaction of medical student participation in care and the presence of medical student teaching.

DESIGN

Prospective cohort study.

SETTING

Eight outpatient internal medicine departments of a university-affiliated HMO in Massachusetts.

PATIENTS

Two hundred seven patients seen on teaching days (81 patients who saw a medical student-preceptor dyad and 126 patients who saw the preceptor alone), and 360 patients who saw the preceptor on nonteaching days. Five hundred (88%) of 567 eligible patients responded.

MEASUREMENTS AND MAIN RESULTS

Thirteen closed-response items on a written questionnaire, measuring satisfaction with specific dimensions of care and with care as a whole. Visit satisfaction was similar among patients on teaching and nonteaching days. Ninety-one percent of patients seeing a medical student, 93% of patients seeing the preceptor alone on teaching days, and 93% of patients on nonteaching days were satisfied or very satisfied with their visit; less than 2% of patients in each group were dissatisfied with their visit. Satisfaction on all measured dimensions of care was similar for patients seeing a medical student, patients seeing the preceptor alone on teaching days, and patients seeing the preceptor on nonteaching days.

CONCLUSIONS

Medical student participation and the presence of medical student teaching had little effect on patient satisfaction. Concerns about patient satisfaction should not prevent managed care organizations from participating in primary care education.

Medical education is shifting from hospitals to outpatient settings.^1–3 While some managed care organizations have partnered with medical schools and even established “teaching HMOs,” most have been reluctant to participate in medical education, especially the teaching of medical students in the ambulatory setting.^4,5 Concerns about diminished patient satisfaction have contributed to the unwillingness of managed care organizations to embrace medical student education.^4,6,7 Whether or not medical student teaching actually affects patient satisfaction is unclear from existing studies. Several studies have shown that care by residents or medical students does not compromise patient satisfaction.^8–16 However, to our knowledge, no study has examined the effect of medical student teaching on patient satisfaction in a managed care setting.

We therefore studied the effects of medical student teaching on the satisfaction of patients in the outpatient internal medicine departments of a managed care organization. We hypothesized that patients' perceptions of their care might be affected not only by seeing medical students but also by seeing their physician during a clinical session in which he or she is teaching. That is, the presence of medical students and the act of medical student teaching might affect patient's perceptions of care, even if the patients are not directly involved in teaching.

METHODS

Setting

The study was carried out within Harvard Pilgrim Health Care (HPHC), a not-for-profit managed care organization in New England with 14 health centers that provide primary care for over 300,000 members. At the time of the study, these health centers, now known as Harvard Vanguard Medical Associates, operated under a staff-model HMO financial structure. HPHC is a major teaching affiliate of Harvard Medical School.¹⁷

Students, Preceptors, and the Clerkship

Eight internal medicine departments of HPHC served as teaching sites for 15 medical student-preceptor dyads participating in a year-long (January–December) primary care clerkship. All students (N = 15) and preceptors (N = 14; one preceptor had a different student on 2 separate days) agreed to participate. Students were in the 9th and 10th month of their third year of medical school (the first clinical clerkship year) and, prior to the study, had spent 3 to 4 months at the site with the preceptor for once- weekly clinical sessions. Students in this clerkship were assigned to teaching sites without regard to students' special interest in primary care or managed care. Preceptors in this study were general internists in full-time clinical practice. The recommended format for clinical teaching in this clerkship was for the preceptor to gain the patient's permission for the student to participate in the visit. The student was to see and evaluate the patient alone, then to present the assessment and plan to the preceptor. The student and preceptor would then discuss the case, usually outside the examination room, then see the patient together to complete the visit.

Study Design

Data were collected during 68 clinical practice sessions. After their office visit, 650 consecutive patients were approached and asked to complete the survey form. Eighty-three patients (13%) who refused to participate had demographic and clinical characteristics similar to participants. Of the 567 patients who agreed to participate, 81 had seen a medical student, 126 had seen the preceptor alone on a teaching day, and 360 had seen the preceptor on a nonteaching day.

While we do not know in detail how preceptors selected patients to participate in medical student teaching, several preceptors noted that the process was more or less random and a matter of which patients were available when the student was ready to see them. To explore the possibility that students saw a biased sample of patients, we assessed whether preceptors intentionally excluded some patients from medical student encounters. Before or after 7 practice sessions (4 teaching and 3 nonteaching), we asked 3 preceptors to assess whether a convenience sample of 41 patients not seen by the medical students would have been “appropriate and acceptable” for medical student teaching. These preceptors reported that 37 (90%) would have been acceptable for medical student teaching.

The study design was approved by the human studies committees of Harvard Medical School and Harvard Pilgrim Health Care.

Survey Instrument

The patient satisfaction survey questionnaire was created from the HPHC Clinician Office Quality of Care Monitor, an instrument modified for use within HPHC from a previously validated survey of outpatient satisfaction.¹⁸ We pilot tested the modified instrument among 40 patients in internal medicine departments of HPHC not selected for this study to ensure readability and to minimize respondent burden. To help counteract the expected ceiling effect of high satisfaction ratings, the questionnaire was framed to encourage critical responses. The introductory paragraph assured patients that their responses would be confidential and would not be shared with their clinicians. In addition, the survey explained that the goal was to identify areas where care could be improved. The final survey included 13 closed-response questions and took, on average, 5 minutes to complete. Twelve survey items, using 3-point (e.g., too long, about right, too short) or 5-point (e.g., poor, fair, good, very good, excellent) rating scales, measured patients' perceptions of the process of care (e.g., respect shown for their privacy, opportunity to ask questions, time spent in the examination room, explanation of what will happen next, thoroughness and appropriateness). One item measured overall satisfaction (Overall, how satisfied were you with today's visit?).

Survey Administration

Research assistants, identified to patients as independent of the clinical staff, handed surveys to patients as they completed their office visit. Patients were encouraged to complete the survey form before leaving the waiting area but were also provided a return envelope if they preferred to take the form home. Approximately 75% of respondents returned the survey at the time of their visit, and 25% returned the forms by mail. Nonrespondents were contacted by mail 2 and 4 weeks after their visit to remind them to complete and return the form. Response rates were similar among all 3 groups of patients: 94%, 89%, and 87% among patients who saw a medical student, the preceptor on a teaching day, and the preceptor on a nonteaching day, respectively (χ² = 3.33, P = .19). At the request of 2 patients, both seen on nonteaching days, the research assistant read the survey questions to the patient and recorded the answers. Although the survey form allowed participants to have a family member complete the form on their behalf, no patients indicated that their surveys were completed by proxy.

Other Measurements

Patient identifiers were linked to the automated medical record system of HPHC, providing demographic and clinical characteristics.^19,20 Clinical characteristics included physician-coded diagnoses and level of visit complexity (scale 1–5, 5 being most complex). Patients' addresses identified their census tract, from which average per capita income and average racial composition of the census tract were assigned to each patient. We examined the relationship between these characteristics and patient satisfaction to determine whether they confounded or modified the effect of medical student teaching on satisfaction.

Statistical Analysis

Our analyses were according to 3 groups of patients: those who saw a medical student, those who saw the preceptor alone on a teaching day, and those who saw the preceptor on a nonteaching day. The primary analysis involved those patients on a teaching day, comparing responses of patients who saw a medical student to those who saw the preceptor alone. This analysis examined the effect of having a medical student involved in care, adjusting for the office environment (i.e., teaching vs nonteaching day). The secondary analysis examined only those patients who did not see a medical student, comparing those seen on a teaching day to those seen on a nonteaching day. This latter analysis provided insight into the effect of the presence of students during a clinical session, adjusting for the direct effect of student care by excluding patients who saw a medical student.

We used SAS (Version 6.12, Cary, NC) to perform Fisher's exact test and Student's t test or 1-way ANOVA for categorical and continuous variables, respectively. We performed logistic regression to determine whether age, gender, race, per capita income, visit complexity, or presence of a medical student independently predicted overall visit satisfaction. The odds ratio (OR) and 95% confidence interval (CI) were calculated.

Because patients were not randomly assigned to a particular physician, the patients seeing each physician may have shared characteristics or have responded to survey items in a manner different from patients assigned to another physician. To determine whether clustering of patients in each physician's practice could have explained variation between study groups, we performed analyses with the physician as the unit of analysis. We found no differences at the level of the physician (i.e., no cluster effect) and present results with the individual patient as the unit of analysis.

The study was designed to have 80% power (with 2-tailed α = 0.05) to detect a 10-percentage-point difference between patients seeing the student and patients seeing the preceptor alone in the proportion rating the visit overall as satisfied or very satisfied. All P values are 2-tailed.

RESULTS

Age, race, per capita income, and visit complexity were similar among patients seen on nonteaching days, those seen by a student-preceptor dyad, and those seen by the preceptor alone on teaching days. Overall, the average age was 48 years; 69% of patients were white, and the average per capita income was $20,365. The proportion of female patients seen on teaching days was higher than on nonteaching days (49% vs 36%, P < .001) but was similar among patients seen on teaching days by either the preceptor alone or the student with the preceptor. Mean visit complexity was similar among patients seen on nonteaching days (2.2), those seen by a preceptor alone (2.3), and those seen by a student with the preceptor (2.2; P = .61). Among the 81 patient encounters with a medical student-preceptor dyad, the three most common principal diagnoses were initial or periodic health assessment (n = 20), infectious diseases (n = 15), and musculoskeletal complaints (n = 11).

Satisfaction was high among all 3 patient groups. Ninety-one percent of patients seeing a medical student, 93% of patients seeing a preceptor on teaching days, and 93% of patients on nonteaching days were satisfied or very satisfied overall with their visit. The distribution of responses for overall visit satisfaction for patients seen on teaching and nonteaching days is shown in Figure 1. Less than 2% of patients in each group were dissatisfied or very dissatisfied with their visit. Similar proportions of male (93%) and female (94%) patients were satisfied or very satisfied with their visit (P = .85). White and nonwhite patients were similarly likely to be satisfied or very satisfied with their visit (95% vs 94%, P = .78).

FIGURE 1.

Distribution of responses to the question, “Overall, how satisfied were you with today's visit?”

For patients seen on teaching days, satisfaction on all measured dimensions of care was similar for patients seeing a medical student and patients seeing the preceptor alone (Table 1). Patients seeing the preceptor alone on teaching days and patients seeing the preceptor on nonteaching days reported similar levels of satisfaction with individual dimensions of care. In multivariate analysis, controlling for age, gender, race, per capita income, and visit complexity, we found that patients seen on teaching days were as likely as patients seen on nonteaching days to be very satisfied with their visit (OR, 0.75; 95% CI, 0.49 to 1.14). We found that gender, race, income, and visit complexity did not independently predict overall visit satisfaction.

Table 1.

Percent of Patients Satisfied with Specific Dimensions of the Visit on Teaching and Nonteaching Days

	Teaching Day
Dimension of Care	Student andPreceptor(N = 76)	PreceptorAlone(N = 112)	Nonteaching Day(N = 312)	P Value*	P Value†
Explanation of care	73.7	75.9	82.0	.73	.21
Thoroughness of care	74.7	78.6	81.7	.60	.49
Advice about your condition	73.7	82.4	83.7	.20	.77
Respect for privacy	91.9	88.2	89.6	.47	.72
Clinician's personal manner	86.8	87.5	88.8	.99	.73
Comfort asking questions	98.7	94.7	95.7	.25	.61
Explanation of information	97.3	94.6	95.8	.48	.60
Responsiveness to questions	94.6	92.0	94.8	.57	.35
Appropriateness of care	94.6	90.3	93.2	.41	.31
Instruction about future care	91.9	81.4	86.9	.06	.16
Time spent in examination room	86.3	86.5	91.5	.99	.18
Likelihood of future recommendation‡	78.7	79.3	82.9	.43	.56
Overall duration of visit	84.9	86.4	89.9	.83	.38

^*Comparison of “student and preceptor” vs “preceptor alone” on teaching days.

^†Comparison of “preceptor alone” on teaching days vs patients seen on nonteaching days.

^‡Percent of patients rating as very good or excellent “the likelihood that you would recommend the doctor you saw to your family or friends.”

DISCUSSION

We found that medical student teaching in the outpatient setting of a managed care organization did not affect patients' satisfaction with care. Patients seen on teaching and nonteaching days reported similar levels of satisfaction with their care overall and with components of satisfaction included in the survey. Despite conventional wisdom that medical student teaching may harm patient satisfaction, these results provide strong evidence to the contrary.

One can imagine that medical student teaching might have an adverse effect on specific aspects of the outpatient care experience but that a “halo effect” might lead patients to respond favorably when asked about overall satisfaction. The survey instrument was designed to minimize the potential for such a bias by not asking patients about their views of the medical student or the provider but by asking instead about various aspects of their care. Our observation that levels of satisfaction with individual components of care were similar among patients who did and did not see medical students argues against the presence of a halo effect.

A growing body of evidence suggests that patients do not object to medical student participation in their care. Previous studies in various settings, including inpatient maternity,⁹ geriatric,¹⁰ and surgical¹¹ services, as well as ambulatory surgery centers,¹² community-based family practice offices,¹³ and urgent care clinics,¹⁴ suggest that patients' satisfaction might not be harmed by medical student teaching. The findings of the present study strengthen the assertion that medical student teaching in general does not harm patient satisfaction and provide evidence that medical student teaching within managed care organizations does not adversely affect member satisfaction.

Strengths of this study include the use of two concurrent control groups, enabling us to measure two distinct effects: the effect of medical students and the effect of teaching on patient satisfaction. This study design enabled us to demonstrate not only that patients seeing medical students are as satisfied as patients not seeing medical students but also that patients seen on teaching days—whether or not they participate in the actual teaching process—are as satisfied as patients seen on nonteaching days.

The main limitation of this study is that patients were not randomly allocated to teaching and nonteaching days, nor were patients on teaching days randomly assigned to be seen by students or not. In such settings, preceptors use various selection criteria to choose which patients they will invite to participate in medical student education; preceptors are likely to choose patients they believe will offer the best learning opportunities for their students, and such selection may result in a bias toward patients who are likely to be more satisfied with students participating in their care. We used several measures in our design and analyses to explore the possibility of the presence of such a selection bias. We asked preceptors about a sample of patients on teaching and nonteaching days to identify which patients would be acceptable and appropriate for student teaching. Preceptors reported that 90% of patients in this small sample would have been “eligible” for student teaching. Furthermore, we compared demographic and clinical characteristics among patients seen on teaching and nonteaching days and found the patients similar except for gender.

An additional limitation of this study is that it represents the experience of only one managed care organization with a long history of collaboration with Harvard Medical School. Students at Harvard may not reflect typical American medical students; nevertheless, the 15 students in this study were participating in a required clerkship and were not known to be systematically different from their class as a whole. These students were not a select group aiming to enter careers in primary care or managed care. The 14 physicians in this study volunteered to participate as preceptors in the longitudinal primary care clerkship and may be systematically different from other unselected HMO-based physicians. Nevertheless, these physicians are full-time clinicians, not full-time academic faculty, and they are likely to be similar to other HMO-based physicians who volunteer to precept medical students. Conducting this study in community-based internal medicine departments in metropolitan and suburban Boston may suggest generalizability of the results to many settings.

Another important limitation of this study, as with many studies using satisfaction surveys to measure patients' care experiences, is that survey instruments such as the one we employed may not be sensitive enough to detect meaningful differences in patients' opinions.^21,22 Despite the fact that we based our items on those in a well-established survey instrument and reordered the response categories in an effort to prevent a “ceiling effect,” responses to most survey items clustered around the highest levels of satisfaction. This study used both process of care measures as well as perceptions of overall satisfaction; however, even this variety of items may not adequately differentiate subtle differences in patients' experiences with office visits. The context of the questionnaire covered interpersonal treatment and information more than physical competence or treatment decisions. The absence of independent predictors of overall satisfaction in this study may be a result of this ceiling effect.

In this setting, satisfaction levels were high whether medical students were present or absent. In settings with lower “baseline” satisfaction, medical student involvement may be perceived differently. Studies in other settings are needed to answer this question.

Leaders of managed care organizations will weigh multiple factors when considering adopting medical student teaching programs. The results of this study show that concerns about patient satisfaction should not cause managed care organizations to avoid participating in primary care education. Other factors, such as dollar costs, space for teaching, and demands on physician preceptors, merit further study.