Resident Involvement in Curricular and Clinical Practice Change and Satisfaction With Training According to Length of Training in Family Medicine

Abstract

Background and Objectives:

Most research in residency training has focused on quality improvement within a single program. We explored resident involvement in curricular and clinical practice change, the learning environment, and resident satisfaction in 3-year family medicine residencies compared to matched 4-year residencies.

Methods:

We used two surveys to capture data. One was for program directors, which assessed the level of resident involvement in curricular and practice transformation. The second was a resident survey, which asked residents to rate their involvement in curricular change and practice transformation, the learning environment, and satisfaction with training. Both were administered annually between 2013 and 2019. Response rates ranged from 84.6% to 100%.

Results:

Findings revealed no overall difference in resident involvement in curricular change, but the program director survey findings indicated that a higher proportion of residents in 4-year programs were using a broader diversity of approaches to working on quality improvement (QI) projects compared to those in 3-year programs. We also found statistical differences in the number of QI projects completed per year, with 34.1% completing three or more in 4-year programs compared to 13.3% in 3-year programs (P<.001). We found a positive correlation between resident involvement, learning environment, and satisfaction with training for both 3-year (range 0.489–0.666; P=.001) and 4-year residents (range 0.441–0.529; P=.001).

Conclusions:

Four-year residents were involved in a greater number of quality improvement projects and had a more diverse profile of involvement than those in 3-year residency programs. Involvement in practice and curricular change and the learning environment were associated with greater levels of resident satisfaction with training in both 3-year and 4-year programs.

INTRODUCTION

Clinical education in medicine is increasingly dynamic. Emerging conditions such as the COVID-19 pandemic, awareness about physician wellness, novel treatments, innovation management, and health systems science all contribute to continuous change. Including residents in the change process is increasingly common for clinical transformations as well as for changes to residency training. 1, 2, 3, 4 Evidence shows that involving residents in the change process better prepares them for change management, improves their satisfaction with their training environment, and contributes to higher levels of wellness. 5, 6, 7, 8 In addition, the development of a learning-oriented culture and favorable work environment should be a high priority for residency programs and their organizations 9 and should include building learner identity, teaching learning skills, and creating opportunities for collaborative learning. 10 Existing literature focuses more on resident involvement in quality improvement and patient safety activities than it does on other aspects of residency training. 11, 12, 13, 14 This is likely due to the Accreditation Council for Graduate Medical Education (ACGME) requirements to address competencies in systems-based practice, 15 which include residents’ participation in identifying system errors and implementing potential systems solutions. In addition, existing literature largely focuses on resident involvement in change management at a single training site. 12, 13

Our own prior research has involved several primary care residencies on a variety of curricular and clinical change topics, including but not limited to resident satisfaction, the impact of novel training tracks, and interprofessional care and education. We found that resident satisfaction was highest when residency programs implemented integrated case management and was lowest when these programs expanded clinic hours and implemented new electronic health records. 16 We also found that residency graduates who underwent flexible maternal health training, which involved a higher volume of maternity experiences, were more likely to deliver babies (86.7% vs 14.6%), perform C-sections as the primary surgeon (80.0% vs 4.7%), and care for hospitalized children (86.7% vs 34.4%) compared to those who did not undertake this training. 17 Lastly, interprofessional team learning has increased significantly across primary care medical residencies, and meeting regularly, sharing best practices, and building trusting relationships made the greatest impact on residency transformation efforts. 18

Further research is needed to explore the relationship between resident involvement in curricular and clinical practice change, the residency learning environment, and resident satisfaction. Variations in the length of residency training may allow for greater resident engagement in impacting change. In this study, we explored how resident involvement in curricular and clinical practice change at residency training sites influences perceptions about the learning environment and how this involvement may differ among family medicine residencies undergoing 3-year (3YR) training programs compared to matched residencies undergoing 4-year (4YR) training programs.

METHODS

Length of Training Pilot (LoTP)

The LoTP, which ran from 2013 to 2023, was a mixed-methods, prospective, case-control pilot study designed to assess several associations between the length of residency training in family medicine and learner outcomes, such as scope of practice, preparedness for independent practice, and clinical knowledge. 19 Several published papers related to this study can provide additional background. 20, 21, 22, 23, 24 Briefly, a total of 17 residency programs, which were all in good standing with the ACGME and which agreed to participate in required evaluation activities, were selected to participate (seven 3YR civilian programs, six 4YR civilian programs, and four Navy programs). We excluded Navy programs in these analyses because their training setting and content differed from civilian programs. Curricular structures varied in the programs offering 4 years of training. All evaluation activities were overseen by researchers in the Department of Family Medicine at Oregon Health & Science University (OHSU). All LoTP programs obtained local institutional review board approval, and OHSUs institutional review board granted an educational exemption to obtain data from the study sites (IRB # 9770).

Data Collection

We used two surveys to capture data included in our analyses. One was a program directors’ survey administered annually between 2013 and 2019, which included three variables designed to assess the level of resident involvement in curricular and practice transformation, including involvement in clinical team meetings. These survey questions used a 4-point response category of 1=not at all involved, 2=somewhat involved, 3=moderately involved, and 4=extremely involved, which we collapsed into two categories (not at all/somewhat involved and moderately/extremely involved). Five additional questions assessed resident involvement in designing and implementing quality improvement (QI) projects, including whether residents were involved at all (Yes/No), how they were involved (independently, with faculty, with resident teams, or with multidisciplinary teams), the number of QI projects they undertook according to program year, and estimates of the percent of QI projects that have resulted in improved care processes and patient outcomes. The response rate for this survey was 100% in all project years.

The second survey was an annual residents’ survey, also administered between 2013 and 2019, which asked residents to rate their involvement in curricular change and practice transformation using a 5-point scale (1=substantially weak, 2=somewhat weak, 3=neutral, 4=somewhat strong, and 5=substantially strong). The survey also asked residents four questions about their learning environment in terms of adequate supervision, adequate feedback, effectiveness of faculty and staff in creating an environment of scholarship and inquiry, and satisfaction with the research or scholarly opportunities their program provided, using a 4-point scale (1=not at all, 2=somewhat, 3=moderately, 4=extremely). Lastly, the survey asked one global satisfaction question regarding their residency training, using a 5-point scale (1=very unsatisfied, 2=somewhat unsatisfied, 3=neutral, 4=somewhat satisfied, 5=very satisfied). The response rate for this survey ranged from 84.6% to 100% with an overall average response rate of 96.8%.

Data Analyses

We used descriptive statistics (means, standard deviations, frequencies, and proportions) to assess outliers, evaluate the shape of the data to determine the appropriate statistical tests, and characterize study findings. Because our preliminary analyses found no differences between 3 and 4 years of training over time, we collapsed all years of training for each length of training category. In addition, we found no differences according to program year, so we collapsed the data into 3 years and 4 years of training overall. We used c² to assess for differences according to length of training for categorical data, and we used independent samples t tests for continuous data. Summary scores were created for resident involvement in either curricular or practice transformation (total possible range, 1–10), for learning environment (total possible range, 1–16), and resident satisfaction (total possible range, 1–10) by summing the scores for variables attributable to these domains. We used Pearson’s correlation coefficient to perform correlations. All tests were two-tailed. Alpha was set at 0.05 to determine statistical differences for resident demographic data, and it was set at 0.01 for program director and resident survey data to account for multiple comparisons.

RESULTS

A total of 2,576 resident surveys were included in our analyses: 1,090 were from 3YR programs, and 1,486 were from 4YR programs. These surveys were completed by 370 residents in 3YR programs and 432 residents in 4YR programs. Participants, as determined during the PGY1 year, were predominantly female (>56.9%), White (>71.9%), non-Hispanic (>86,5%), single (>52.3%), and with no children (>83.3%; Table 1). The majority were also graduates from medical schools based in the United States (>85%). The only statistical difference between 3YR residents and 4YR residents was that those in 3YR programs were more likely to be female compared to those in 4YR programs (65.7% vs 56.9%; P=0.01). Participating programs ranged in size from 6 to 22 residents per year and were predominantly community-based, medical school affiliated programs (57.1% in 3YR programs and 66.7% in 4YR programs; Table 1).

Table 1. Characteristics of PGY1Residents and Residency Training Programs Included in Analyses.

Resident characteristics	Length of training		P value
	3 years (n=370), n (%)	4 years (n=432), n (%)
Mean age (SD) in years Range	29.4 (3.9) 25–50	29.0 (3.3) 23–50	.72
Gender Male Female Nonbinary	— 127 (34.3) 243 (65.7) (<1)	— 186 (43.1) 246 (56.9) (<1)	.01
Race American Indian/Alaska Native Asian/Pacific Islander Black White Mixed race/other	— (<1) 71 (19.2) 18 (4.9) 266 (71.9) 20 (5.4)	— (<1) 71 (16.4) 13 (3.0) 324 (75.0) 25 (5.8)	.91
Ethnicity Hispanic Non-Hispanic Preferred not to answer/missing	— 43 (11.6) 320 (86.5) 7 (1.9)	— 33 (7.6) 397 (91.9) (<1)	.37
Marital status Single Married/partnered Separated Divorced Widowed	— 198 (53.5) 166 (44.9) (<1) (<1) (<1)	— 226 (52.3) 198 (45.8) (<1) (<1) (<1)	.94
Parental status Had children	— 46 (12.4)	— 72 (16.7)	.26
US m edical s chool graduate Yes No Missing	— 315 (85.1) 52 (14.1) (<1)	— 375 (86.8) 54 (12.5) (<1)	.81
Program characteristics	(n=7) n (%)	(n=6) n (%)
Size (range in number of residents per year)	6–11	6–22
University-based	2 (28.6)	2 (33.3)
Community-based, affiliated with medical school	4 (57.1)	4 (66.7)
Community-based, unaffiliated with medical school	1 (14.3)	0
Required 4 years of training	–	4
Optional 4 years of training	–	2

Abbreviations: PGY, postgraduate year; SD, standard deviation

We received 45 surveys received from 3YR residency program directors and 40 from 4YR residency program directors. No statistically significant differences were noted according to length of training for resident involvement in curricular change, practice transformation, clinical team meetings, or residency directors’ estimates of QI projects that resulted in improved care processes or patient outcomes (Table 2). We did find statistical differences in program directors’ assessment of how residents work when designing and implementing QI projects. Program directors from 4YR programs indicated that 75.6% of residents worked independently, 95.1% worked with faculty, and 75.6% worked in resident teams or multidisciplinary teams compared to 53.3%, 73.3%, 55.6%, and 57.8%, respectively, for 3YR residency programs (P<.001). We also found statistical differences in the number of QI projects completed per year, with 34.1% completing three or more in 4YR programs compared to 13.3% in 3YR programs (P<.001).

Table 2. Program Directors’ Report of Resident Involvement in Curricular Change, Practice Transformation, and Quality Improvement Activities According to 3YearsVersus 4 Yearsof Training.

Assessment variables	Length of training		P value
	3 years (n=45)* n (%)	4 years (n=41)* n (%)
Level of resident involvement with curricular changes in your program None/somewhat Moderately/extremely	— 1 (2.2) 44 (97.7)	— 5 (12.5) 35 (87.5)	.037**
Level of resident involvement with practice transformation None/somewhat Moderately/extremely	— 27 (60.0) 18 (40.0)	— 16 (40.0) 24 (60.0)	.223**
Current level of resident involvement with clinical team meetings None/somewhat Moderately/extremely	— 25 (55.6) 20 (44.4)	— 21 (52.5) 18 (46.2)	.925**
Resident involvement designing and implementing QI projects (If they did any) Independently Integrated with faculty In resident teams In multidisciplinary teams	— 24 (53.3) 33 (73.3) 25 (55.6) 26 (57.8)	— 31 (75.6) 39 (95.1) 31 (75.6) 31 (75.6)	— <.001 <.001 <.001 <.001
Number of QI projects completed per year (If they did any) 1 2 >3	— 23 (51.1) 7 (15.6) 6 (13.3)	— 19 (46.3) 3 (7.3) 14 (34.1)	<.001
	Mean (SD)	Mean (SD)
Estimate of the percent (%) of resident QI projects overall that have resulted in improved care processes	52.2% (26.9)	57.9% (29.6)	.350***
Estimate of the percent (%) of resident QI projects overall that have resulted in improved patient outcomes	37.6% (27.0)	32.2% (24.2)	.337***

*Missing <5%

**χ² test

***Independent samples t test

Abbreviations: QI, quality improvement; SD, standard deviation

Residents’ self-reported involvement with curricular and clinical practice change did not statistically differ between those training in 3YR programs and 4YR programs (Table 3), though residents in 4YR training programs more highly rated the adequacy of supervision, effectiveness in creating an environment of scholarship and inquiry, and overall satisfaction with training compared to 3YR residents (Table 3). Similarly, when we assessed summary scores for resident involvement, learning environment, and resident satisfaction with training, we found no statistical differences in the summary resident involvement score, while the learning environment summary score (10.20 vs 9.94; P<.001) and resident satisfaction with training summary scores (7.72 vs 7.33; P<.001) were statistically higher for residents in 4YR programs compared to those in 3YR programs (Table 4 ).

Table 3. Residents’ Report of Their Involvement in Curricular Change, Practice Transformation, and Satisfaction With Training According to 3 YearsVersus 4 Years of Training.

Assessment variables	Length of training		P value*
	3 years (n=1,089) Mean (SD)	4 years (n=1,486) Mean (SD)
Resident involvement
Resident involvement with curricular changea Range	4.22 (0.84) 1–5	4.19 (0.89) 1–5	.40
Resident involvement with clinical practice changea Range	3.98 (0.92) 1–5	3.98 (0.91) 1–5	.98
Learning env ironment
Adequacy of supervision received from faculty and staffb Range	3.51 (0.61) 1–4	3.58 (0.55) 1–4	.003
Adequacy of feedback from faculty and staffb Range	3.17 (0.72) 1–4	3.23 (0.68) 1–4	.023
Effectiveness of faculty and staff in creating an environment of scholarship and inquiryb Range	3.27 (0.76) 1–4	3.44 (0.64) 1–4	<.001
Satisfaction with research or scholarly opportunitiesb Range	3.03 (0.83) 1–4	3.31 (0.72) 1–4	<.001
Overall satisfaction
Overall satisfaction with residency trainingc Range	4.40 (0.77) 1–5	4.54 (0.66) 1–5	<.001

*P for independent samples t test of 3 versus 4 years of training all years combined (n=2,582)

^aScale: 1=substantially weak, 2=somewhat weak, 3=neutral, 4=somewhat strong, 5=substantially strong

^bScale: 1=not at all, 2=somewhat, 3=moderately, 4=extremely

^cScale: 1=very unsatisfied, 2=somewhat unsatisfied, 3=neutral, 4=somewhat satisfied, 5=very satisfied

Table 4. Self-Reported Summary Scores of Residents’ Involvement, Learning Environment, and Satisfaction with Training According to 3 Years vs 4 Years of Training .

Assessment variables	3 years (n=1,089) Mean (SD)	4 years (n=1,486) Mean (SD)	P value*
Resident involvement** summary score Range	8.19 (1.63) 2–10	8.17 (1.64) 2–10	.774
Learning environment summary score Range	9.94 (1.72) 3–12	10.2 (1.47) 4–12	<.001
Resident satisfaction with training summary score Range	7.33 (1.39) 1–9	7.72 (1.15) 3–9	<.001

*P 3 versus 4 years of training all years combined (n=2,582)

**Involvement in curricular and clinical practice change combined

The summary scores for the correlation among resident involvement, learning environment, and satisfaction with training ranged between 0.489 and 0.666 among residents receiving 3 years of training, representing a statistically positively correlation (P=.001). The correlations for these variables among residents receiving 4 years of training were similarly high (range 0.441–0.529; P=.001; Table 5).

Table 5. Correlations Between Resident Involvement in Curricular and Practice Change, Learning Environment, and Satisfaction With Training (All Years Combined).

	Resident involvement	Learning environment	Satisfaction with training
3-year programs (n=1,089)
Resident involvement**	1.000*	0.532*	0.489*
Learning environment		1.000	0.666*
Satisfaction with training			1.000
4-year programs (n=1,485)
Resident involvement	1.000*	0.441*	0.498*
Learning environment		1.000	0.529*
Satisfaction with training			1.000

*P=.001 Pearson’s correlation coefficient (two-tailed test)

**Involvement in curricular and clinical practice change combined

DISCUSSION

This is the first study to explore the impact of 3 versus 4 years of training on the interplay between resident involvement in curricular and clinical practice change, the learning environment, and satisfaction with residency training. We found that residents training in 4YR programs were more than twice as likely to undertake three or more QI projects per year compared to those training in 3YR programs. Additionally, compared to residents in 3YR programs, residents in 4YR programs used a broader diversity of approaches to working on quality improvement projects. This difference could be secondary to increased schedule flexibility, allowing for more time to work on a variety of projects. Additionally, the longer curricular time frame may have afforded them the opportunity to work on long-term projects.

This finding is important because the American Board of Family Medicine now requires clinically active physicians to meaningfully participate in and complete one performance improvement every 3 years to successfully meet requirements of the family medicine certification process. 25 The stated purpose of this requirement is to demonstrate that physicians are able to reflect on their practice, identify opportunities to improve care delivery, and implement changes to address the performance gap. 26 Our findings appear to show that residents graduating from 4YR programs are better prepared to do QI projects independently, while working with faculty, and when working in resident or multidisciplinary teams; the latter could also benefit other team members where they practice after training. We did not ask program directors what types of QI projects residents were working on, which prevented us from characterizing the intensity or longitudinal nature of those projects. We did ask program directors to estimate the percentage of QI projects that resulted in improved care processes and patient outcomes, and we found no difference in reported impact from 3YR versus 4YR programs. This finding suggests that the QI projects of both program durations may have been similar in construct.

Resident survey findings indicated that residents in 3YR and 4YR training programs were similarly involved in curricular change and practice transformation and that this involvement was overall rated as a beneficial experience. The entire range of the scale was used, indicating some variability in residents’ experiences. Though we found statistically significant differences between 3YR and 4YR residents’ assessments of scholarly opportunities, effectiveness in creating an environment of scholarship and inquiry, and overall satisfaction with training, with 4YR residents’ scores higher than 3YR residents scores, whether the differences found were educationally meaningful is unclear. The study groups were large, with more than 1,000 respondents in each group. We pooled the data in this way because the survey was administered annually and residents’ assessments of the program and the program itself can change over time. The pooling did result in considerable statistical power to find potentially small differences.

When we analyzed the summary scores for resident involvement in change, the learning environment, and resident satisfaction, we found no differences between the two groups for resident involvement in change, though we did find statistical differences between the two groups for the learning environment and resident satisfaction with training, with 4YR program residents rating these higher. However, these differences may not be educationally meaningful. The absolute difference for the learning environment was 0.26, and for overall satisfaction with training, it was 0.39. Prior work by Norman et al 26 suggested that meaningful differences for health-related quality of life is half a standard deviation between the two groups. If we apply this to our findings, the difference for the learning environment would need to achieve a 0.80 difference, and resident satisfaction with training would need to achieve a difference of 0.635, neither of which we found.

Our assessment of the correlations between resident involvement, the learning environment, and satisfaction with training were positively statistically significant among residents in both 3YR and 4YR training programs. This finding suggests that the relationships among both resident involvement and the learning environment are highly correlated with each other and with satisfaction with training, while the duration of training appears to be less influential.

The relationships between quality improvement projects, practice transformation, and job satisfaction are supported by other studies. The Healthy Work Place trial found that involvement in changing the work environment, such as quality improvement projects, has been shown to increase job satisfaction and decrease burnout among clinicians. 27 While transitioning to a patient-centered medical home, the Veteran Health Administration found that burnout improved with increased engagement in evidence-based quality improvement projects. 28 Quality improvement work and the skills needed to implement projects successfully may be protective against burnout and improve work conditions for clinicians.

We found a correlation between resident involvement in the combination of both practice change and curricular change with satisfaction with training. We cannot conclude the relative weighting of these two measures of change on the satisfaction outcome. We did not ask about the number or composition of curricular changes led by residents. Prior research on resident satisfaction with training has not examined the impact of length of training, so we cannot compare our findings with other published literature. That would be an interesting area for further study.

Among the strengths of this research were the considerable response rates we achieved for both the resident and program director surveys, our inclusion of residency programs with geographic representation, and our inclusion of programs that varied in terms of being community-based and university based. Because attaining high response rates is so important for educational research, we worked diligently with programs to develop a culture and relationships where contributing to evaluation is valued, which is described in detail in a recent publication. 29 Weaknesses included that this was a pilot study, which allowed us to explore relationships rather than fully test them, given that our study design was a prospective case-control design rather than a randomized design. Because of this, we cannot assume that causation exists between the comparisons of 3YR versus 4YR training programs.

CONCLUSIONS

In conclusion, 4YR training programs appear to support a higher volume of quality improvement projects with diverse approaches to undertaking such projects, which may foster a broader skill set. Residents’ satisfaction with training is correlated with their involvement in curricular and practice transformation and the learning environment, especially where scholarly work is involved. These correlations do not appear to be influenced by length of training. Future research should further explore the correlation of curricular and clinical practice change and resident satisfaction with the learning environment and overall satisfaction with training.