Resident Research and Scholarly Activity in Internal Medicine Residency Training Programs

Abstract

OBJECTIVES

1) To describe how internal medicine residency programs fulfill the Accreditation Council for Graduate Medical Education (ACGME) scholarly activity training requirement including the current context of resident scholarly work, and 2) to compare findings between university and nonuniversity programs.

DESIGN

Cross-sectional mailed survey.

SETTING

ACGME-accredited internal medicine residency programs.

PARTICIPANTS

Internal medicine residency program directors.

MEASUREMENTS

Data were collected on 1) interpretation of the scholarly activity requirement, 2) support for resident scholarship, 3) scholarly activities of residents, 4) attitudes toward resident research, and 5) program characteristics. University and nonuniversity programs were compared.

MAIN RESULTS

The response rate was 78%. Most residents completed a topic review with presentation (median, 100%) to fulfill the requirement. Residents at nonuniversity programs were more likely to complete case reports (median, 40% vs 25%; P =.04) and present at local or regional meetings (median, 25% vs 20%; P =.01), and were just as likely to conduct hypothesis-driven research (median, 20% vs 20%; P =.75) and present nationally (median, 10% vs 5%; P =.10) as residents at university programs. Nonuniversity programs were more likely to report lack of faculty mentors (61% vs 31%; P <.001) and resident interest (55% vs 40%; P =.01) as major barriers to resident scholarship. Programs support resident scholarship through research curricula (47%), funding (46%), and protected time (32%).

CONCLUSIONS

Internal medicine residents complete a variety of projects to fulfill the scholarly activity requirement. Nonuniversity programs are doing as much as university programs in meeting the requirement and supporting resident scholarship despite reporting significant barriers.

Graduate medical education involves the period of residency training following medical school and prepares physicians for the independent practice of medicine. The Accreditation Council for Graduate Medical Education (ACGME) and the Residency Review Committee for Internal Medicine (RRC-IM) develop and enforce standards for the training of internal medicine residents. These requirements are intended to ensure a high standard of graduate medical education while maintaining a proper balance between patient care and learning. The ability of residency programs to comply with the entire range of residency requirements, in particular as new limits are placed on resident duty hours, have refocused attention on the educational mission of graduate medical training.^1–5

Both the ACGME and the RRC-IM support research as an important component of residency training. In 1994, the RRC-IM established a new requirement that “prior to the completion of training, each resident must demonstrate acceptable scholarly activity,” defined as “original research, comprehensive case reports, or review of clinical and research topics.”⁶ Resident scholarship may have potential benefits for trainees.^7–10 However, significant resources and time may be needed to comply with the scholarly activity requirement. Interpretation and implementation of this requirement may not be uniform across all residency programs. Furthermore, because research is a key mission of medical schools, residency programs that are based at a primary teaching hospital of a medical school (university-based programs) may have available to them greater expertise and resources that support research than do nonuniversity-based programs. Despite these concerns, the state of resident research at internal medicine residency training programs has not been systematically studied since initiation of the RRC scholarly activity requirement in 1994.¹¹

The purposes of this study were 1) to describe how internal medicine residency programs fulfill the scholarly activity requirement including the current context of resident scholarship, and 2) to compare findings between university-based and nonuniversity-based programs.

METHOD

Survey Administration

Internal medicine residency program directors were identified using the ACGME database of accredited U.S. residency training programs.⁶ In March 2002, surveys were mailed to all 391 categorical internal medicine residency program directors. Follow-up mailings, email, or facsimile reminders were sent to encourage full participation. The study was approved by the Institutional Review Board of the Johns Hopkins Bayview Medical Center.

Survey Content

The survey instrument was developed through review of the relevant literature, interviews with 6 internal medicine residency program directors in Maryland, and revised after piloting the survey on 5 experts in medical education and survey design as well as 3 internal medicine residency program directors. The 31-item questionnaire was organized into the following 5 topic areas: 1) interpretation of the RRC requirement for scholarly activity, 2) support for resident scholarship, 3) scholarly activities of residents, 4) attitudes toward resident research and scholarly activity, and 5) residency program characteristics. Questions were formatted as multiple choice, yes-or-no, short answer, and 3-or 5-point Likert scales.

The RRC requirement for scholarly activity includes “original research, comprehensive case reports, or review of clinical and research topics” in its description of scholarly activity. To make a distinction between original research and other scholarly activities, we used the term “hypothesis-driven research” to refer to projects that involved hypothesis generation, data collection, and analysis. In questions asking about residents' research and scholarly activities, support for resident scholarship, and program directors' attitudes, we used the phrase “resident research and/or scholarly activity.”

The ACGME designates residency program training requirements as must, should, or desirable. Must requirements are mandatory. Failure to comply with mandatory requirements places a program in jeopardy of losing its accreditation status. Programs may also be cited for not meeting requirements which are designated as should or desirable, although these requirements are not mandatory. The scholarly activity requirement is a must requirement.⁶ These designations, which are well known to program directors, were reiterated in the survey instrument.

To assess the level of institutional and program support for resident scholarship, program directors were asked about specific factors that have been associated with successful research activities, such as the presence of a research curriculum, protected time, funding, and faculty mentors.^12–15 Information collected on residency program characteristics included university affiliation, number of residents, and percentage of residents pursuing fellowship training. Where possible, we verified the accuracy of responses to questions about program characteristics (number of residents and university affiliation) using the ACGME, Association of American Medical Colleges (AAMC), and American Medical Association (AMA) websites.^6,16,17

Data Analysis

Descriptive statistics were used to summarize the responses to all questions. Responses to 5-point Likert scales were dichotomized and analyzed as proportions. t test, χ², and Wilcoxon ranksum tests were used to compare responses between university-based programs (programs based at a primary teaching hospital of an AAMC-accredited medical school) and nonuniversity-based residency programs (programs with or without affiliation with an AAMC-accredited medical school but not based at the primary teaching hospital of a medical school). We chose to compare university-based with nonuniversity-based programs because we hypothesized that programs based at the primary teaching hospital of an AAMC-accredited medical school might have greater resources to support resident research and therefore demonstrate increased productivity in terms of resident scholarly activity compared to nonuniversity-based programs. Recognizing that not all university-based programs are the same and that hospital or medical school resources may not necessarily equate with support for resident scholarly activity, our survey specifically asked programs about the resources available to support resident scholarly activity.

To assess for heterogeneity between nonuniversity-based programs, we conducted a preliminary analysis that made three-way comparisons on all reported variables between university-based programs, university-affiliated programs, and programs with minor or no university affiliation (data not shown). These data demonstrate no significant differences between university-affiliated programs and programs with minor or no medical school affiliation. Data were analyzed using Stata 8.0 (Stata Corporation, College Station, TX).

RESULTS

Surveys were returned by 305 of the 391 program directors for a response rate of 78%. Sixty-six percent of responses came from nonuniversity-based residency programs. The median number of residents at responding programs was 42. University-based programs were significantly larger than nonuniversity-based programs (median number of residents, 81 vs 31; P <.001) and more likely to send their residents on to fellowship training (mean, 59% vs 44%; P <.001).

There were no significant differences between respondents and nonrespondents in terms of university affiliation (66%; P .05) and number of residents (median, 44; P .05).

Interpretation of and Compliance with the RRC Scholarly Activity Requirement

Program directors were asked which types of scholarly activity projects fulfill the RRC requirement at their program. Most program directors agreed that topic reviews with presentation (67%), case reports (78%), and hypothesis-driven research (88%) fulfill the requirement. A minority stated that presentation at morning report (16%) or an article review at journal club (24%) would suffice. Nonuniversity-based programs were less likely than university-based programs to report that topic reviews with presentation (62% vs 79%; P =.002) or case reports (74% vs 87%; P =.007) were acceptable activities to fulfill the RRC requirement.

Ten percent of responding residency programs reported having been cited by the ACGME for lack of demonstration of scholarly activity by residents since the introduction of the requirement in 1994. Nonuniversity-based programs were more likely to have been cited (14% vs 2%; P =.002) than university-based programs.

Support for Resident Research and Scholarly Activity

Table 1 lists residency program and institutional factors which support resident scholarship at university-and nonuniversity-based programs. About half of responding programs (46%) designated funds specifically for resident scholarly activity. The mean amount of funding was $6,126 per year (standard deviation [SD], $6,985; range $0 to $120,000). There were no statistically significant differences between university-and nonuniversity-based residency programs in the amount (P =.18) or sources (P =.37) of funding. Programs reported using funding to support travel for presentation of residents' work at local and national meetings (47%), technical support (31%), computers and computer software (31%), administrative support (28%), start-up costs for resident projects (25%), and research assistants (9%).

Table 1.

Presence of Residency Program and Institutional Factors Supporting Resident Research and Scholarly Activity^*

Research-related Characteristics	All Programs†N (%)	University n (%)	Nonuniversity n (%)	P Value
Funding	135 (46)	42 (43)	93 (48)	.43
Research curriculum	141 (47)	39 (39)	102 (51)	.05
Research director	126 (42)	35 (35)	91 (46)	.07
Protected time	97 (32)	30 (29)	67 (34)	.47
Research committee	82 (28)	23 (23)	59 (30)	.19
Mandatory research‡	92 (31)	23 (23)	69 (35)	.03
Faculty serving as mentors, median percent	20	19	25	<.001

^*P values for the comparisons of proportions and medians for university- versus nonuniversity-based programs.

^†Because of nonresponse to questions, the total N may be less than 305.

^‡Residency program has an established requirement that all residents conduct hypothesis-driven research.

Nonuniversity-based programs were more likely to have a curriculum for resident research and scholarly activity (51% vs 39%; P =.05; Table 1). At residency programs with research curricula, the mean length of time that the curricula had been in place was 5 years (SD, 9 years). These curricula provide, on average, a total of 16 hours (SD, 32 hours) of instruction. Forty-two percent of programs have a designated research director and 28% have a research committee to oversee resident scholarship. Most programs allow residents to use elective time for research (85%). One third of programs (32%) offer protected time (apart from elective rotations) for residents to work on research. Nonuniversity-based residency programs were more likely to have a mandatory requirement for residents to conduct hypothesis-driven research (35% vs 23%; P =.03) than were university-based programs. Nonuniversity-based residency programs reported a greater percentage of faculty acting as research mentors (median 25% vs 19%; P <.001; Table 1).

Resident Research Activities

Program directors were asked to estimate the percentage of residents conducting various types of scholarly activities during the years 1998 through 2001 (Fig. 1). The majority of residents had completed a topic review with presentation (median, 100%). Residents from nonuniversity-based programs were more likely to complete case reports (median, 40% vs 25%; P =.04) and present at local or regional meetings (median, 25% vs 20%; P =.01) and just as likely to conduct hypothesis-driven research (median, 20% vs 20%; P =.75) and present at national meetings (median, 10% vs 5%; P =.96) as residents at university-based programs. Residents at university-based programs were more likely to publish in peer-reviewed journals (median, 10% vs 5%; P =.02).

FIGURE 1.

Median response and interquartile range, percent of internal medicine residents involved in various scholarly activities during the years 1998 through 2001. ^*P <.05, Wilcoxon ranksum test for comparison of university versus nonuniversity programs.

Attitudes About Resident Research

When asked for their views about how they thought the scholarly activity requirement should be designated, 76% of program directors responded that it should be a must or should requirement. Most program directors rated the importance of learning research-related topics during residency highly (Table 2). Despite rating these topics highly, program directors at only 18% to 63% of programs believed that these topics were thoroughly taught to their residents. These findings did not differ by university status (all P >.05).

Table 2.

Frequency with Which Program Directors Agree or Strongly Agree on the Importance of Teaching Research-related Topics During Residency Training and How Well They Believe Those Topics Are Taught at Their Residency Programs^*

Research-related Topic	Importance† (%)	Thoroughly Taught‡ (%)
Literature searching	98	63
Critical appraisal skills	98	62
Research ethics	83	21
Biostatistics	79	23
Research design	78	18

^*There were no statistically significant differences between university-and nonuniversity-based programs (allP .05).

^†Percentage of program directors who agree or strongly agree on the importance of teaching these research-related topics using a 5-point Likert scale, 1=strongly agree, 2=agree, 3=neutral, 4=disagree, 5=strongly disagree.

^‡Three-point Likert scale, 1=not taught, 2=taught somewhat, 3=thoroughly taught.

Program directors described barriers to resident research and scholarly activity at their programs (Table 3). Nonuniversity-based programs were more likely to agree or strongly agree that a lack of faculty time (73% vs 58%; P =.01), faculty mentors (61% vs 31%; P <.001), resident interest (55% vs 40%; P =.01), and technical support (46% vs 33%, P =.04) were major barriers to resident scholarship.

Table 3.

Perceived Barriers to Resident Research as Reported by Internal Medicine Residency Program Directors^*

Barriers	All Programs†N (%)	University n (%)	Nonuniversity n (%)	P Value‡
Lack of faculty time	197 (67)	57 (58)	140 (73)	.01
Lack of funding	180 (62)	60 (61)	120 (62)	.79
Lack of faculty mentors	148 (51)	31 (31)	117 (61)	<.001
Lack of resident interest	146 (50)	39 (40)	107 (55)	.01
Lack of technical support	121 (41)	33 (33)	88 (46)	.04

^*Five-point Likert scale, 1 =strongly agree, 2 =agree, 3 =neutral, 4 =disagree, 5 =strongly disagree; response represents agree or strongly agree.

^†Because of nonresponse to questions, the total N may be less than 305.

^‡P values for the comparisons of proportions for university-versus nonuniversity-based programs.

DISCUSSION

Program directors seem to agree on which types of scholarly activities fulfill the RRC requirement. They report in-depth topic reviews with presentation and case reports as the most common scholarly activities completed by residents. In addition, many residents conduct hypothesis-driven research and disseminate their work through presentations and publications. These are laudable tasks to accomplish during an already demanding clinical training period.

There are many arguments as to why resident research is worthwhile. Research experience can impart skills that are invaluable to the practice of medicine such as literature searching and critical appraisal.^{7–10,18–24} Early research training has been associated with continued scholarly work and may help to inform residents' career choices.^18,25–29 However, research activity may take time away from clinical learning, the primary goal of residency training, and many practical barriers to successful efforts exist.^11,20,30,31 Further studies are needed to better characterize the educational value of resident scholarship in terms of measurable educational and patient care outcomes.

This survey is the first effort to systematically review the state of resident research at internal medicine training programs since the introduction of the scholarly activity requirement in 1994. Compared to previous work, it appears that residents have become more involved in scholarly activities. In 1993, program directors reported that in the previous year, 18% of residents had conducted hypothesis-driven research and 33% had completed topic reviews.¹¹ The current averages of 31% (95% confidence interval [CI], 27 to 35) performing hypothesis-driven research and 72% (95% CI, 67 to 77) preparing topic reviews described in this study suggest a substantial improvement. It is possible that the actual research activities of residents prior to 1993 may have been underreported because the need for program directors to provide evidence of resident scholarly activity was not yet obligatory. Nonetheless, it is our belief that most program directors should have been aware of their residents' activities, in which case the findings of this study likely represent a true increase in resident scholarship.

We had hypothesized that university-based residency programs would fare better than nonuniversity-based programs with respect to support for resident research and in terms of the scholarly productivity of their residents. The survey findings did not support this theory. For example, nonuniversity-based programs were more likely to have a mandatory requirement for residents to conduct hypothesis-driven research, provide a structured research curriculum, and have as many or more residents involved in various scholarly projects than did university-based programs. Despite this apparent success, nonuniversity-based programs describe greater barriers to resident research and report having been cited more often by the RRC since initiation of the requirement in 1994 for inadequately fulfilling the scholarly activity requirement. One explanation for this finding might be that because nonuniversity-based programs have less research-related resources and expertise, they must provide more support, such as through research curricula, for their residents to succesfully complete scholarly projects. If this is the case, the current level of support and research activity at nonuniversity-based programs may represent succesful efforts to overcome barriers to resident scholarship. Alternatively, nonuniversity-based programs may have been cited more often despite efforts to support scholarly activity because the quality of that support may not compare favorably with university-based programs. For example, in other studies on resident research, mentoring has been shown to be important for success.^12,15 Perhaps the research and mentoring skills of faculty at nonuniversity-based programs may not be the same as those of faculty at university-based programs.

We could not determine from our survey exactly how much effort each program must expend to fulfill the RRC requirement or whether resources including funds or time are diverted from other components of residency training. Developing and sustaining a productive resident research program requires significant creativity and resources.^{10,11,32–35} Given the many objectives of residency education, the time and resources allotted for any one specific requirement must be carefully considered in the context of the overall residency curriculum.

Several limitations of this study should be considered. First, although the response rate to this survey was high, we cannot be sure that our findings are truly representative of the research activities and environments at all internal medicine residency programs. Second, we relied on the report of program directors to gauge the number of residents involved in various activities and to estimate the level of support for resident scholarship. Although failure to demonstrate resident scholarly activity may result in citation by the RRC, we have no reason to believe that program directors would have given inaccurate responses to this confidential survey. Furthermore, programs must provide evidence of resident scholarly activity during accreditation review site visits. All programs in this study had at least one site visit within the last 7 years (data not shown) and therefore program directors should have been familiar with information on resident scholarship compiled in preparation for accreditation review. Third, our findings are based on data collected using an instrument that has not been validated. However, we made considerable efforts to design a valid instrument and verified responses where possible. Finally, due to the cross-sectional design of our study, our conclusions are limited to observations of the current state of resident research. We are unable to link causally our observations about resident research and scholarly activity with the implementation of the RRC requirement; in particular, our study was not designed to identify the causes of citation for lack of scholarly activity among residency programs.

The current state of resident research in internal medicine training programs appears to be robust. Residency programs support resident scholarship with funding, curricula, protected time, and research mentors. Residents complete a variety of scholarly projects and many present or publish their work. Nonuniversity-based programs are doing as much as university-based programs in fulfilling the RRC requirement in spite of greater barriers. Nevertheless, the value and cost of resident scholarship in terms of educational and patient care outcomes remain unknown. An ongoing challenge to graduate medical education, particularly in light of new limitations placed on resident work hours, will be to evaluate critically all components of residency training to prove their merit.