Clinical Assessment of Residents: A Survey of Clinician Educators Regarding Resident Assessment Burden and Modifiable Factors

Abstract

Background

A fundamental role of the clinician educator is to provide thoughtful assessments for resident development. A gap in the literature exists about whether the completion of assessments contributes to clinician educator burden.

Objective

We sought to understand the degree to which completing resident assessments contributes to clinician educator burden, the drivers behind such perception, and whether modifiable factors exist.

Methods

In October 2020, we conducted a cross-sectional study of adult hospital medicine clinician educators to explore burden associated with resident assessment. The authors developed a 10-item electronic survey (Likert type and sliding scale responses), asking about demographics, context, frequency and degree of burden, burdensome aspects of assessments, estimated time for assessments, and percentage of assessments turned in late or never. We conducted subgroup analyses for differences in responses based on sex and number of years practicing, and regression analyses for predictors of burden degree.

Results

Fifty of 81 (62%) surveyed faculty responded. Two percent (1 of 50) reported no burden, while 42% (21 of 50) reported infrequent (“never,” “rarely,” “sometimes”) and 56% (28 of 50) reported frequent (“often,” “always”) burden. Of those experiencing burden, 67% (33 of 49) reported slight or moderate, and 33% (16 of 49) reported significant or extreme burden. Potentially modifiable causes included assessment request boluses, lag time between resident service and assessment requests, and technology involved. Female clinician educators estimated submitting a higher percentage of late assessments than males (65% vs 41%, P=.02). Number of years practicing was inversely associated with assessment time (β=-0.28, P=.01).

Conclusions

Our findings suggest that resident assessments are a source of burden among adult hospital medicine clinician educators and that several potentially modifiable factors may underlie this burden.

Introduction

Clinical teaching faculty have a distinct set of training responsibilities that overlap with patient care, including the timely completion of resident assessments.¹ While assessments are an essential means of providing feedback to residents, completing them may add to the administrative and cognitive burden that these clinician educators experience.

Burden is often a vaguely defined concept and varies according to person and context. Work-related burden, for example, is conceptually related to the job strain model described by Karasek et al as the combination of high job demand and low control.² In medicine, burden has been described as anything that hinders patient care, either directly or indirectly.³ Practice burden is increasingly recognized as an issue for clinicians and clinician educators.^4-7

The need for clinician educators to complete considerable volumes of resident assessments may be an underrecognized source of burden. Studies of resident perceptions of burden associated with their assessments of faculty and peers and their evaluations of their program curricula have been reported,^8-11 but relatively little has been described about clinician educators' collective experience with assessment burden in the United States and whether modifiable factors exist.

We sought to understand whether completion of resident assessments is perceived as a burden by adult hospital medicine (AHM) clinician educators, and if differences in this perception exist according to clinician educator characteristics. We also sought to identify predictors for the frequency and degree of perceived burden and potential root causes of assessment burden with opportunity for mitigation.

Methods

In October 2020, we conducted a cross-sectional survey of clinician educators at the University of California, San Francisco (UCSF) in the Division of Hospital Medicine. UCSF is a large, urban, academic medical center with approximately 180 internal medicine residents. Eighty-one AHM physicians, referred to here as clinician educators, instruct on the inpatient teaching services. UCSF uses MedHub for resident assessments.¹²

Absent prior relevant survey instruments, we developed a 10-item survey to explore assessment burden (provided as online supplementary data). The survey was reviewed by 6 UCSF clinician educators with expertise in resident education and clinician burnout but was not otherwise tested. Questions included experience within the last 5 years with inpatient resident assessment (yes/no), department (internal medicine [AHM], pediatric hospital medicine), number of years performing inpatient resident assessment, frequency and degree of burden (Likert scale), one multiselect question with an optional free text response eliciting burdensome aspects, and 3 sliding scale questions (0%-100%) on the estimated percentage of work time spent completing assessments and the percentage of assessments turned in late or never.

The REDCap platform (Vanderbilt University) was used for survey creation, participant recruitment, and data collection.^13,14 Recipients were given 2 weeks to respond to surveys, with a total of 3 reminders sent to nonrespondents: at 1 week, 10 days, and 13 days. No incentives were offered.

Responses in the optional free text field that were unambiguously identical to an existing multiselect answer choice were folded into that answer choice. The remaining free text responses were grouped as “Other.” We analyzed differences by sex using the Mann-Whitney U test for continuous and Likert response types, and the Fisher's exact test for dichotomous responses. In analyzing response differences by number of years practicing, we used simple linear regression for continuous and Likert response types, and logistic regression for multiselect questions with dichotomous response types. To ascertain predictors of 2 aspects of burden (frequency of perceived burden and degree of perceived burden), we performed multiple regression analysis with sex, number of years practicing, respondents' estimate of the percent of time spent assessing residents, and department as independent variables. P values <.05 were considered significant for all tests. STATA version 17.0 (StataCorp LLC) was used for analyses.¹⁵

For nonresponse bias analysis, we compared the distributions of sex and years practicing between respondents and nonrespondents using the Fisher's exact test and 2-sided independent samples t test, respectively.

This study was reviewed and approved by the UCSF Institutional Review Board.

Results

Of UCSF's 81 AHM clinician educators, 50 (62%) responded, and all completed the survey in its entirety. Thirty-four percent of respondents (17 of 50) were male (Table 1). Respondents had a mean (SD) of 8.76 (6.47) cumulative years (range=1-25, median=6.5) as clinician educators. Ten percent (n=5) practiced both pediatric and adult hospital medicine. Respondents estimated spending a mean (SD) of 6.52 (4.88%) of their work time on resident assessments (range=1%-20%, median=5.0%).

Table 1.

Demographics of Survey Respondents

Variable	Overall	Male	Female
	N, Mean (SD)
Years practicing as clinician educator	50, 8.76 (6.47)	17, 7.79 (7.20)	33, 9.26 (6.11)
Department	N (%)
Internal medicine (AHM clinician educators only)	45 (90)	17 (34)	28 (56)
Both (adult+pediatric hospital medicine)	5 (10)	0 (0)	5 (10)
Total	50 (100)	17 (34)	33 (66)

Abbreviations: AHM, adult hospital medicine; SD, standard deviation.

With respect to timeliness and completion rates, respondents reported submitting an average (SD) of 56.8 (36.5%) of assessments late, and never submitting 15.3 (21.2%) of assessments. When participants were asked how often they felt that completing assessments was burdensome, 56% (28 of 50) responded “often” or “always” (Table 2 and online supplementary data). Of those respondents who had reported ever experiencing assessment burden, 49% (24 of 49) reported that the degree of burden was “moderate,” 27% (13 of 49) reported it to be “significant,” and 6% (3 of 49) reported it as “extreme” (Table 2 and online supplementary data). The 49 respondents endorsing some degree of burden were asked to compare the level of burden due to resident assessments relative to the burden of other aspects of their work; 31% (15 of 49) stated that assessments produced “somewhat more” or “much more” burden.

Table 2.

Results of 2-Sample Mann-Whitney U (Continuous and Likert Responses) and Fisher's Exact Tests (Dichotomous Non-Mutually Exclusive Multiselect Responses), Analyzing for Differences Between Male and Female Clinician Educators

Variable	Overall			Male			Female
	N	Mean	SD	N	Mean	SD	N	Mean	SD	P value
Time practicing as a clinician educator (years)	50	8.76	6.47	17	7.79	7.20	33	9.26	6.11	.20
Estimated work time completing assessments (%)	50	6.52	4.88	17	6.47	5.10	33	6.55	4.85	.77
Estimate of assessments submitted late (%)	50	56.8	36.5	17	40.7	35.5	33	65.1	34.6	.024
Estimate of assessments never completed (%)	50	15.3	21.2	17	13.9	23.2	33	16.0	20.4	.62
How often do you feel that completing assessments is burdensome?	N	%		N	%		N	%		P value
1-Never	1	2		0	0		1	3		.22
2-Rarely	6	12		5	29		1	3
3-Sometimes	15	30		4	24		11	33
4-Often	18	36		5	29		13	39
5-Always	10	20		3	18		7	21
Total	50	100		17	100		33	100
How much burden do you feel associated with completing assessments?a	N	%		N	%		N	%		P value
1-Slight burden	9	18		6	35		3	9		.37
2-Moderate burden	24	49		5	29		19	59
3-Significant burden	13	27		5	29		8	25
4-Extreme burden	3	6		1	6		2	6
Total	49	100		17	100		32	100
Compared to other aspects of your work that cause you to feel burden, how much does completing resident assessments contribute to the burden you experience?a	N	%		N	%		N	%		P value
1-Much less	6	12		3	18		3	9		.36
2-Somewhat less	18	37		7	41		11	34
3-About the same	10	20		2	12		8	25
4-Somewhat more	13	27		5	29		8	25
5-Much more	2	4		0	0		2	6
Total	49	100		17	100		32	100
What aspects of resident assessments do you find burdensome? (Choose all that apply)b	N	%		N	%		N	%		P value
The time required to complete them	38	78		13	77		25	78		>.99
The bolus of assessment requests coming all at once	33	67		9	53		24	75		.21
Other more pressing demands on my time	27	55		7	41		20	63		.24
Feeling that I have insufficient information about a resident to supply an “adequate” assessment	25	51		10	59		15	47		.55
The unpleasant feeling of providing a negative assessment	25	51		6	35		19	59		.23
Burdensome technology used to fill out and submit assessments	23	47		7	41		16	50		.77
The lag time between the resident's service and the request for the assessment makes it hard to remember details	23	47		9	53		14	44		.56
Assessment requests for residents I don't remember	11	22		3	18		8	25		.73
Other	9	18		1	6		8	25		.14
Assessment requests for residents with whom I did not spend time	7	14		2	12		5	16		>.99

^aOne respondent who answered “Never” to “How often do you feel that completing assessments is burdensome?” was not presented with these follow-up questions.

^bFisher's exact tests (dichotomous non-mutually exclusive multiselect responses).

Those respondents who perceived any degree of assessment burden were asked to multiselect from categories and/or write in free text responses associated with feeling burdened. Over half of respondents selected the following aspects of resident assessment contributing to experiencing burden: the time required to complete assessments (78%, 38 of 49), the bolus of assessment requests coming all at once (67%, 33 of 49), having more pressing demands (55%, 27 of 49), having insufficient information to supply an “adequate” assessment (51%, 25 of 49), and the unpleasantness associated with providing a negative assessment (51%, 25 of 49; online supplementary data).

Female clinician educators reported submitting a higher percentage of their assessments late than males (65% vs 41%, P=.024; Table 2). No other differences in survey responses between male and female respondents were found. With respect to practice experience, there was an inverse relationship between years practicing and estimated work time spent on assessments (β=-.28, P=.01; online supplementary data). No other responses were associated with years practicing.

Multiple regression analyses did not find any associations between frequency or degree of assessment burden with any of the 4 independent variables considered (sex, number of years practicing, estimated percent of work time spent completing resident assessments, and department; online supplementary data).

A limited nonresponse bias analysis showed that there was no difference in number of years practicing (P=.51) between respondents and nonrespondents, but that there was a difference in sex between the groups (66% female in respondents versus 44% female in nonrespondents, P=.03).

Discussion

In this 2020 survey of AHM clinician educators at one institution, over half of respondents “often” or “always” perceived burden associated with resident assessment. Over 80% of those perceiving burden endorsed “moderate” to “extreme” degrees of burden, and nearly a third endorsed assessments as more burdensome than other aspects of their work. Assessment burden factors reported by half or more of respondents included: time required, boluses of assessment requests, having more pressing demands, insufficient information to supply an “adequate” assessment, and unpleasantness associated with providing a negative assessment. Being in practice longer was associated with less time spent on assessments.

Some of these factors may be modifiable, such as those related to assessment timing and technology. For example, some groups have developed user interfaces to streamline the assessment reporting process.^16-19 Furthermore, as evidence demonstrates that the electronic health record and other digital health systems play roles in clinician burnout,^20-22 further investment in health information technology design, infrastructure, and training for resident assessment tools may improve perceived burden.^23,24

Our findings are also consistent with studies analyzing effects of the Competence by Design program,²⁵ recently implemented in Canadian graduate medical education, which requires more frequent faculty assessments. Several studies found increased time required for clinician educators to complete entrustable professional activity resident assessment after transition to Competence by Design.²⁶-²⁸ In addition, one study found similar root causes to assessment burden, including the time required to complete assessments, having other competing demands on the clinician educator's time, and a lack of user-friendly and convenient information technology.^>28

This study is limited by a small sample size; with no prior power calculations, differences between groups may have been missed. The respondents are from a single department in one urban academic institution, which limits generalizability to other settings. The survey was not pretested; thus, respondents may not have interpreted questions as intended. The survey was distributed during the COVID-19 pandemic, which may have had unknown effects on responses as well. Finally, the R-squared values of our regressions are low, which suggests that the independent variables selected may not account for a large portion of the variation in assessment burden degree or frequency.

Future research might assess the generalizability of clinician educator perceptions of resident assessment burden and potentially modifiable factors in other settings and specialties. Interventions targeting these factors would be worthwhile next steps, given the importance of resident assessments.

Conclusions

This survey study of internal medicine resident assessments by AHM clinician educators at one institution found that over half of respondents often or always perceived burden associated with these assessments, and many reported that the degree of burden was moderate or extreme. Boluses of assessments, burdensome technology, and lag time were the most common readily modifiable root causes of burden that might be addressed with process and technology improvement.

Objectives

We sought to understand the degree to which completing resident assessments contributes to clinician educator burden, the drivers behind such perception, and whether modifiable factors exist.

Findings

Our findings suggest that resident assessments are a source of burden among adult hospital medicine clinician educators and that several potentially modifiable factors (assessment request boluses, lag time between resident service and assessment requests, technology involved) may underlie this burden.

Limitations

This study is limited by a small sample size from a single department in one urban academic institution.

Bottom Line

Process and technology improvements should be undertaken to address modifiable aspects of resident assessment to reduce burden among adult hospital medicine clinician educators, while further research should be conducted to understand assessment burden in other specialties and settings.

Funding Statement

Funding: This study was funded by The Haile T. Debas Academy of Medical Educators Innovation Funding for Education grant.