Defining the Resident Continuity Clinic Panel Along Patient Outcomes: a Health Equity Opportunity

Abstract

Background

Continuity clinics are a critical component of outpatient internal medicine training. Little is known about the population of patients cared for by residents and how these physicians perform.

Objectives

To compare resident and faculty performance on standard population health measures. To identify potential associations with differences in performance, specifically medical complexity, psychosocial vulnerability, and rates of patient loss.

Setting and Participants

Large academic primary care clinic caring for 40,000 patients. One hundred ten internal medicine residents provide primary care for 9,000 of these patients; the remainder are cared for by faculty.

Study Design

Descriptive analysis using review of the medical record and hospital administrative data.

Main Measures

We compared resident and faculty performance on standard population health measures, including cancer screening rates, chronic disease care, acute and chronic medical complexity, psychosocial vulnerability, and rates of patient loss. We evaluated the success of resident transition by measuring rates of kept continuity visits 18 months after graduation.

Key Results

Performance on all clinical outcomes was significantly better for faculty compared to residents. Despite similar levels of medical complexity compared to faculty patients, resident patients had significantly higher levels of psychosocial vulnerability across all measured domains, including health literacy, economic vulnerability, psychiatric illness burden, high-risk behaviors, and patient engagement. Resident patients experienced higher rates of patient loss than faculty patients (38.5 vs. 18.8%) with only 46.5% of resident patients with a kept continuity appointment in the practice 18 months after graduation.

Conclusions

In this large academic practice, resident performance on standard population health measures was significantly lower than faculty. This may be explained in part by the burden of psychosocial vulnerability of their patients and systems that do not effectively transition patients after graduation. These findings present an opportunity to improve structural equity for these vulnerable patients and developing physicians.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11606-020-06420-x.

INTRODUCTION

The American College of Graduate Medical Education (ACGME) mandates that at least 33% of internal medicine training occur in the outpatient environment.¹ This training occurs primarily in resident continuity clinics, where residents manage the primary care of a limited panel of patients under faculty supervision. The ACGME outlines the importance of practice-based learning and improvement as a core competency, whereby residents must demonstrate facility in assessing their own care of patients in order to provide them with experience in self-evaluation and life-long learning.¹ Similarly, primary care has incorporated the principles of population health management, with individual and group performance measurement becoming a primary feature of continuous quality improvement and reflective practice. The American Academy of Medical Colleges outlined a series of key features of high functioning primary care training environments. Among these is a focus on population health management, facilitated by efforts to retain patients within the practice and data-driven quality improvement.

It is unclear, however, how well the principles and practice of population health management have been integrated into residency training. In addition, little is known about how residents perform on routine population health measures compared to their faculty colleagues, representing a potential opportunity to improve the educational experience for residents and the clinical care of their patients. Furthermore, there is evidence that residents care for some of the most socioeconomically disadvantaged patients in their health systems,^3–7 representing a vulnerable physician-patient dyad for which there are opportunities to improve structural equity.^8–14 Finally, while some studies have demonstrated the efficacy of structural interventions on educational outcomes,^15–24 few have assessed impacts on clinical outcomes.

We aim to compare resident and faculty performance on standard population health measures in a large academic primary care clinic and to identify potential associations with differences in performance, specifically medical complexity, psychosocial vulnerability, and rates of patient loss. We hypothesize that higher levels of psychosocial vulnerability in the resident population and high rates of patient loss after resident graduation are associated with disparities in outcomes between faculty and resident physicians.

METHODS

Setting and Participants

Data for this study was collected in a large academic hospital-based primary care clinic where 60 faculty preceptors and 110 residents provide primary care on multidisciplinary teams to 40,000 patients in a large metropolitan area. Residents serve as the primary care physician for 9,000 of these patients, with faculty preceptors providing oversight. Each preceptor supervises three residents, one per graduate year. Each resident cares for 75–100 patients during their tenure and then transitions those patients to an incoming intern provider upon graduation. Fidelity between the preceptor and resident patients is prioritized with few exceptions.

Measures and Data Collection Tools

Clinical Performance

To measure resident performance, we used standard clinical performance measures as defined by Healthcare Effectiveness Data and Information Set (HEDIS), including the following: colorectal, breast, and cervical cancer screening rates; diabetes control (percent of patients with diabetes with glycated hemoglobin (HbA1C) less than 9%); and hypertension control (percent of patients with hypertension with a blood pressure less than 140/90). We queried our population health database to ascertain these rates for each provider. The database was developed and operationalized by the former Director of Population Health (NB). It combines insurance claims data, electronic health record (EMR) structured data, and empanelment data in order to measure clinical performance and drive improvement. These unblinded data have been traditionally reported monthly to faculty and provide actionable patient-specific reports at the point of care.

Medical Complexity, Social Vulnerability

We measured medical and social complexity on a random sample of 316 patients. To ensure balanced representation of physicians within each cohort, our sampling strategy used the patient as the unit of observation, resulting in an even distribution of individual physicians. We measured acute medical complexity by counting the number of hospitalizations and emergency department visits within our institution in the last 10 years. We measured chronic medical complexity by counting the number of chronic medical conditions being managed for each patient and a variety of markers of chronic disease including body mass index, total cholesterol, systolic and diastolic blood pressure, and A1C.

We developed an a priori list of variables that we hypothesized would represent markers of social vulnerability under the following domains: low health literacy, economic vulnerability, psychiatric illness burden, high-risk behaviors, and markers of disengagement with the healthcare system. We developed a data extraction tool to measure the presence of these variables, which was piloted on 25 charts to ensure it was properly operationalized. We collected a random sample of 316 medical charts (158 each from faculty and resident panels). A member of the research team (MA) reviewed each chart in our sample and used the data extraction tool to measure each of the variables.

Patient Loss and Successful Transition After Graduation

As a main outcome measure, we assessed patient loss in the practice using a visit-based data repository collected by the practice as part of usual operations for all patient encounters. The repository reports on all patient visits with the date of service linked to the primary care physician, which has a standard way of defining resident vs. faculty status based on the provider identification number. We defined the outcome of patient loss as any time a given patient’s visit date interval was ≥ 3 years. We queried the database from July 11, 2014, to June 30, 2019, representing 402,415 visits to measure the rates of patient loss in the resident and faculty panels.

To evaluate successful patient transition, we established a set of process measures related to kept visit types 18 months after resident graduation. Members of the research team (LF, MG, SK) performed a chart review on a 50% sample of resident panels from the class of 2016 (between June 23, 2016, and December 31, 2017). A patient was considered successfully transitioned after graduation if they attended a continuity visit within the practice, including with an alternative provider to the one intended by the transition process. This included the following visit types: established care with the assigned resident, established care with another resident, or established care with a faculty member. We considered the patient as not successfully transitioned if the patient only attended an urgent care visit or specialist visit, or if there were no visits after 18 months.

Please refer to the Appendix for additional details on our data sources, sampling strategy, and variable definitions.

Analytic Approach

Clinical Performance, Patient Loss, Kept Visit Rates

We compared faculty and resident performance on standard population health measures, patient loss rates, and kept visit rates using a chi-squared test of significance.

Medical Complexity, Social Vulnerability

We compared the resident and faculty practice using chi-squared tests for all binary variables and used t test and Wilcoxon ranked-sum tests for parametric and non-parametric continuous variables respectively.

All statistical analyses were performed using SAS version 9.3 (Cary, NC).

Ethical Issues

The Institutional Review Board at Beth Israel Deaconess Medical Center approved the protocol as exempt from further review as an educational research project.

RESULTS

Performance on all clinical outcomes was significantly better for faculty compared to residents, including colorectal cancer screening (81.2 vs. 64.4%, p < 0.001), breast cancer screening (82.3 vs. 66.7%, p < 0.001), cervical cancer screening (77.9 vs. 64.9%, p < 0.001), diabetes control (85.8 vs. 75.2%, p < 0.001), and hypertension control (67.0 vs. 58.0%, p < 0.001) (Fig. 1).

Figure 1.

Clinical outcomes¹ by provider type. Cancer screening metrics for colorectal, breast, and cervical cancer and measures of diabetes and blood pressure control based on provider type—faculty vs. resident. Cancer screening measurements represent percentage of patients screened within the age-appropriate time interval. Diabetes control measurements represent percentage of patients with a %A1C less than 9%. Hypertension control measurements represent percentage of patients with a blood pressure less than 140/90. ¹Cancer screening rates were determined by the presence of a billed screening test appropriate for the age of the patient. Diabetes control was determined by a %A1C of less than 9%. Hypertension control was determined by a blood pressure less than 140/90. ²p values derived using a chi-squared test. ³National averages for clinical outcomes are as follows: Colorectal cancer screening 63%,²⁵ breast cancer screening 73%,²⁵ cervical cancer screening 81%,²⁵ diabetes control 70%,²⁶ hypertension control 24% (270).

Markers of clinical complexity were similar between resident and faculty cohorts. Specifically, markers of chronic illness burden were similar between resident and faculty cohorts, including average total number of chronic medical conditions (3.4 vs. 3.5, p = 0.3), average BMI (28.9 vs. 28.6, p = 0.7), average systolic blood pressure (127.1 vs. 126.3, p = 0.7), average diastolic blood pressure (75.9 vs. 74.9, p = 0.4), average %A1C (7.4 vs. 7.0, p = 0.7), and average pain scores (1.9 for both cohorts). Markers of acute illness burden were also similar between resident and faculty cohorts. Resident patients had a similar average number of hospitalizations (1.8 vs. 1.4, p = 0.8) and emergency department visits (3.4 vs. 2.3, p = 0.1) in the last 10 years (Table 1).

Table 1.

Patient Characteristics Along Medical and Social Factors by Provider Type

	Resident cohort N = 158	Faculty cohort N = 158	p1
Markers of chronic illness burden2, mean (sd)
Number of chronic medical conditions	3.4 (1.5)	3.5 (1.4)	0.3
Body mass index	28.9 (6.9)	28.6 (6.6)	0.7
Total cholesterol	177 (38.9)	186.5 (37.5)	0.04
Systolic blood pressure	127.1 (16.6)	126.3 (14.9)	0.7
Diastolic blood pressure	75.9 (11.1)	74.9 (10.5)	0.4
%Glycated hemoglobin (%A1C)	7.4 (1.6)	7.0 (1.2)	0.7
Pain score during last visit3	1.9 (3.1)	1.9 (3.1)	0.8
Markers of acute illness burden, mean (sd)
Number of hospitalizations in the last 10y	1.8 (4)	1.4 (3)	0.8
Number of emergency department visits in the last 10 years	3.4 (7.2)	2.3 (4.6)	0.1
Markers of limited health literacy (%)
Limited English proficiency	12.7	4.4	0.007
Less than a high school degree	58.9	31.8	0.002
Markers of economic vulnerability (%)
Public health insurance	14.6	7	< 0.001
Vulnerable neighborhood4	20.9	15.2	0.2
Requires community resources5	32.9	19	0.005
Racial/ethnic minority status	51.9	35.4	0.003
Markers of psychiatric illness burden (%)
Major psychiatric comorbidity6	17.1	4.4	< 0.001
Required urgent psychiatric evaluation during visit	9.5	4.4	0.08
Psychiatric hospitalization this year	7	4.4	0.3
High-risk behaviors (%)
Current smoker	13.3	5.7	0.02
Alcohol use disorder	17.7	8.9	0.02
Substance use disorder	21.3	5.8	< 0.001
Markers of engagement with healthcare system (%)
College student status	7.6	1.9	0.02
< 30 years of age	17.1	5.1	< 0.001
Continuity with primary care physician in the last year7	84	68.3	< 0.001
Un-kept visit/same day cancellation rate in the last year	14.9	9.4	0.007

¹p values were derived using a chi-squared test for rates, t tests for parametric continuous variables, and Wilcoxon ranked-sum tests for continuous non-parametric variables

²Clinical values determined from most recent measurement

³Using a standard 1–10 scale when vital signs are measured

⁴Neighborhoods in Boston that have been historically marked for disinvestment, including Dorchester, Roxbury, and Mattapan

⁵Determined by a referral to a staff community resource specialist at our clinic

⁶Based on prescriptions for mood stabilizers and anti-psychotics

⁷Percentage of total clinic visits each patient had with their primary care physician during the last year

Markers of social vulnerability were significantly higher in the resident cohort compared to the faculty cohort. Resident patients had significantly more markers of low health literacy compared to the faculty cohort, with higher rates of limited English proficiency (12.7 vs. 4.4%, p = 0.007), and lower education levels, with 58.9% having less than a high school education vs. 31.8%, p = 0.002. Resident patients had higher levels of economic vulnerability compared to faculty patients with 32.9% requiring practice assistance obtaining basic needs vs. 19%, p = 0.005. Resident patients had higher levels of psychiatric illness burden with 17.1% having a major psychiatric illness vs. 4.4%, p < 0.001. Resident patients had higher rates of high-risk behaviors, including tobacco use (13.3 vs. 5.7%, p = 0.02), alcohol use disorder (17.7 vs. 8.9%, p = 0.02), and substance use disorders (21.3 vs. 5.8%, p < 0.001). Lastly, resident patients also tended to be less engaged participants in the healthcare system, with a higher number of patients less than 30 years old (17.1 vs. 5.1%, p = 0.02), higher rates of students (7.6 vs. 1.9%, p < 0.001), and higher un-kept visit rates (14.9 vs. 9.4%, p = 0.007) (Table 1).

Rates of patient loss, defined as experiencing a gap in visits to the practice of greater than or equal to 3 years, were significantly higher in the resident practice compared to the faculty practice (38.5 vs. 18.8%, p < 0.001) (Fig. 2). To evaluate patient loss further, we reviewed the charts of 1,439 resident patients (50% sample of the 2016 graduating senior resident patients). Successful transition, defined as a kept continuity visit with a provider in the practice 18 months after graduation, occurred in 665 (46.5%) of transitioned patients. Of those successfully transitioned, 402 (27.9%) had established care with the assigned resident, 161 (11.2%) established care with another resident, and 106 (7.4%) established care with a faculty member. Conversely, 769 (53.4%) patients were considered not successfully transitioned in the practice. Seventy-five (5.2%) were seen in the practice for urgent care only, 96 (6.6%) were seen in the hospital network for specialist visits only, and 598 (41.6%) had no visits within our EMR during the designated 18-month period (Table 2).

Figure 2.

Rates of patient loss¹ by provider type. Rates of patient loss based on provider type—faculty vs. resident. Patient loss is defined as a patient experiencing greater than or equal to a gap in visits to the practice of 3 years. ¹Defined as a patient experiencing greater than or equal to a gap in visits to the practice of 3 years. ²p values derived using a chi-squared test to compare rates of patient loss between July 11, 2014, and June 30, 2019, representing 402,415 visits.

Table 2.

Disposition¹ of Resident Panel Patients 18 months After Resident Transition

Disposition	n = 1,439
Successfully transitioned1   Established care with assigned resident   Established care with another resident   Established care with a faculty member	669 (46.5%) 402 (27.9%) 161 (11.2%) 106 (7.4%)
Not transitioned   Urgent care visit only2   Specialist visit only3   No visit within system4	769 (53.4%) 75 (5.2%) 96 (6.6%) 598 (41.6%)

¹We developed an a priori list of process measures for patient retention after their primary care physician graduated. We defined “successfully transitioned” as having established care with any provider in the practice even if it was not with the assigned resident provider within 18 months of resident graduation. “Not transitioned” was defined as not having re-established primary care in the practice or only interfacing with urgent/specialty care providers within the 18-month timeframe

²Acute problem-based visit with a provider in the practice, not with the intent of preventive or continuity care

³Specialist visits captured from visit notes within our system

DISCUSSION

In this large academic primary care clinic with greater than 100 resident primary care physicians caring for 9,000 patients, we found significant disparities in performance measures between faculty and resident providers. This pattern among our trainees and other local training programs⁷ signals a need for better integration of population health management, in the form of clinical resources, operational support, and actionable data, into resident training and the environments in which they practice medicine.

Our initial findings revealed that performance on five major clinical measures differed based on provider type (faculty vs. resident providers) (Fig. 1). The marked increased in multiple measures of social vulnerability in the resident practice compared to that of the faculty and low retention of patients after graduation demonstrate that this problem is complex. It should be noted that our practice performs significantly above national averages ²⁷ for all cited measures, with residents performing at or only slightly below these averages, suggesting that years of experience is likely not a significant driver. We also found that gaps in chronic disease management measures between faculty and residents were smaller than gaps in cancer screening measures. It is possible that the reduced length of the relationship between a resident and their patients (less than or equal to 3 years) poses challenges in providing care that occurs beyond the interval of 1 year, such as cancer screening, which occurs every 2–10 years, where issues that are managed within the year, like diabetes and hypertension control, are easier to accomplish with shorter patient-physician relationships. For example, there may be three to four resident hand-offs during the interval of colon cancer screening (10 years for normal screening colonoscopy). These findings suggest that resident outcomes may unmask vulnerabilities in the systems in which they practice, as these physicians rely heavily on practice resources to optimize population health. Notably, cancer screening relies more heavily on smooth interdepartmental systems outside of the context of the visit where chronic disease management is more influenced by the physician themselves.

We found that resident patients were significantly more psychosocially vulnerable than faculty patients across all domains, despite having similar levels of medical complexity. Patients in the resident practice were more likely to demonstrate markers of low health literacy and economic vulnerability. Both of these factors likely present logistical challenges for otherwise engaged patients to meet the demands of participating in care—from being able to afford the time off from work for additional visits, to being able to interpret complicated instructions associated with screening procedures. Resident patients were also more likely to have a serious psychiatric diagnosis, which may affect their ability to engage in preventive medical care as their psychiatric illness is often a more immediate priority. Lastly, residents were more likely to care for a more transient population of younger patients and students, which contributes challenges to obtaining documented cancer screenings amidst many disjointed EMRs. Resident patients were also more likely to not keep appointments for primary care, indicating a significant challenge to implement healthcare screenings. It is plausible that this burden of psychosocial complexity in the resident practice presents a unique challenge in implementing population health and may partially explain the differences we captured. It is also notable that residents are providing care to a significantly more vulnerable population than faculty. This may be due in part to residents’ open clinical access, unlike their faculty counterparts who may have closed panels. In this vein, residents are serving an essential role for the public health as points of entry for these patients into the healthcare system. Practice leaders should meet these clinicians and their patients with exceptional systems to provide the support they require.

We found significantly higher rates of patient loss in the resident practice compared to the faculty practice. To evaluate this further, we examined rates of kept visit types in a 50% sample of resident patients, where fewer than half (46.5%) attended a visit with a continuity provider within the practice at 18 months after resident graduation (Table 2). This is a signal that our patient hand-off processes at the time of resident transitions are a potential systems-based explanation for the observed difference in performance on population health measures. Our study adds to the literature by identifying this problem with patient transitions after graduation, providing residency programs nationally with a potential intervention strategy should they observe similar issues in their environments. In response to these findings, we have implemented a robust population health team meeting structure within the residency program whereby residents have protected time to meet with outreach staff and educational leadership every 6 weeks to review their panels and address care gaps. The residents also use the time to prepare for graduation at the end of their senior year to optimize the hand-off process, with the primary aim of reducing patient loss in the resident panel. This intervention is currently a part of an ongoing study that we plan to disseminate once complete to provide residency programs with an approach to improving population health should their systems have similar latent inequities.

Our findings that resident primary care providers underperform on routine population health measures compared to faculty are consistent with those of other local academic institutions⁷ and may represent a latent issue in residency programs nationally. These findings should be further evaluated in a multi-center study for generalizability. If these patterns persist across the country, graduate medical educators and leadership within academic medical centers (AMCs) will need to think collectively about how to support this essential component of resident education and provide equitable care and support to these patients and providers. Designing educational and clinical systems that integrate population health seamlessly into resident education—and the settings where they practice—will be a critical next step.

Our study has several limitations. Because of the single-center design, our findings may not be generalizable to AMCs beyond our region. Another limitation is that there is not a standardized method for measuring empanelment, aside from manual chart review, indicating a need for a standard definition.

In this large AMC with over 100 resident primary care physicians, we found that the patients of resident physicians had lower performance on routine population health measures as compared to patients of faculty. Resident patients were significantly more psychosocially complex, with more economic vulnerability, lower health literacy, higher burden of major psychiatric illness, and higher markers of disengagement with the healthcare system, despite having similar degrees of acute and chronic illness burden. These factors provide a potential patient-based reason for the difference in population health measures. High rates of patient loss after resident graduation present a potential systems-based explanation for these findings, indicating that physician transition may be a reason patients do not stay engaged in care and that practices must employ robust transition processes when physicians leave the practice. To address this problem, clinical and educational leaders at AMCs will need to work together to integrate the principles and practice of population health into graduate medical education, ultimately providing equity to these providers and their patients.

Funding

This work was funded by the American Medical Associations 2019 Accelerating Change in Medical Education Grants Program

Compliance with Ethical Standards

Conflict of Interest

The authors have no conflicts of interest to disclose