Educational Impact of a Structured Radiation Oncology Clerkship Curriculum: An Inter-Institutional Comparison

Abstract

Purpose

Many medical school clerkships have structured curricula, however most radiation oncology clerkships do not. The Radiation Oncology Education Collaborative Group (ROECSG) implemented a curriculum for fourth-year radiation oncology clerkships at 14 institutions. We hypothesized that students completing clerkships with the curriculum would report greater subjective knowledge and comfort to function as a radiation oncology resident compared with students completing clerkships without the curriculum.

Methods and Materials

The ROECSG curriculum included three 1-hour lectures and a 1-hour hands-on radiation treatment planning workshop. Applicants to a single radiation oncology residency program in the 2014–2015 academic year were sent an anonymous, validated clerkship experience survey. Students indicated if clerkships were completed at a curriculum site. Likert-type data (1=not at all, 5=extremely) are reported as median(interquartile range).

Results

Respondents described 276 clerkship experiences of which 64(23.2%) were completed at a curriculum site. Students whose first clerkship was at a curriculum site perceived greater post-clerkship confidence in knowledge of radiation biology (3 (3–4) vs 2(2–3), p<0.01), treatment set-up/positioning (3(2–3) vs. 2(2–3), p<0.05), treatment planning (3(2–3) vs. 2(2–3), p<0.01), and ability to integrate evidence-based medicine into treatment (4(2–4) vs. 3(2–4), p<0.05). Students who completed any clerkship with the curriculum had greater post-clerkship confidence to function as a radiation oncology resident (3(3–4) vs. 3(2–3), p<0.05).

Conclusions

These results support the curriculum’s ability to increase student knowledge in radiation oncology, especially in the students’ first clerkship. Further, these findings suggest that expanded implementation of such curricula may ensure a rewarding educational experience during radiation oncology clerkships.

Summary Sentence

In this study, MS4s who completed a clerkship at a site with a structured didactic curriculum reported a more structured educational experience, increased knowledge of radiation oncology topics, and a significant increase in preparedness to function as a radiation oncology resident.

Introduction

Medical student clinical rotations such as internal medicine [1], surgery [2], emergency medicine [3], dermatology [4], urology [5], and palliative medicine [6] have structured didactic curricula to complement the clinical experience, but data suggest radiation oncology does not [7]. Despite medical students completing a median of three clerkships at multiple institutions before applying to residency, a national survey of students applying to radiation oncology in 2012 and 2013 revealed that most clerkships lacked a structured didactic curriculum [7,8].

In a subspecialty such as radiation oncology, the yearly sample size of trainees at any single institution is too small to acquire meaningful objective results on the impact of the curriculum [9]. To overcome the small number of medical students at single institution, the multi-institutional collaborative group research model was adapted. The collaborative group model has demonstrated effectiveness in other settings such as treatment of rare diseases by pooling patients from multiple institutions treated in a given timeframe [10]. Initially piloted at two institutions in 2012 [11], the curriculum was expanded to 11 institutions in 2013 [12] and to 14 institutions in 2014, thus forming the Radiation Oncology Education Collaborative Study Group (ROECSG; https://roecsg.uchicago.edu/). Students anonymously evaluated the curriculum upon completion of each clerkship. Although the evaluations were favorable, there was no comparison of student clerkship experiences and curriculum outcomes to a control group.

In a parallel project, a survey on radiation oncology externship experiences was sent to all fourth-year medical students (MS4s) who applied to a radiation oncology residency at one institution annually since 2013 [7]. For each clerkship completed, students reported on curriculum experiences and self-assessed their post-clerkship knowledge and confidence in various facets of radiation oncology. The survey results demonstrated a high degree of variability in clerkship educational experiences. However, students who completed clerkships with formal didactic components reported greater preparedness to function as a radiation oncology resident [8].

We hypothesized that MS4s who participated in a clerkship at curriculum sites would report a more functional educational experience and increased confidence in various clinical topics in radiation oncology in comparison to those who did not participate in a clerkship at a curriculum site. To test this hypothesis, the 2015 iteration of the clerkship survey contained an identifying question as to whether or not the clerkship was completed at an institution participating in the curriculum. Utilizing survey results, clerkship experiences that included the curriculum are compared to those clerkship experiences without it. Students’ perception of the educational experience and associated changes in confidence and knowledge are analyzed.

Methods

Curriculum

The development of the curriculum has been described previously [11]. Kern et al.’s six step approach to medical education curriculum development was used as a conceptual framework to develop the radiation oncology clerkship curriculum to meet the needs outlined in a targeted needs assessment [9]. The curriculum includes three 1-hour lectures given by residents or faculty members [13]. The lecture topics include 1. an overview of radiation oncology history, treatments, and clinic flow; 2. fundamentals of radiation biology and physics; and 3. radiation treatment simulation, planning, and emergencies. All students are required to attend one session per week, although scheduling is left to the discretion of each institution. Lectures are open format with students encouraged to ask questions. One institution uses online lectures. To teach students the fundamentals of radiation treatment planning, residents or faculty lead a 1-hour hands-on radiation treatment planning workshop [14]. The students use a treatment planning workstation while following a step-by-step guide to achieving an optimal plan. The workshop is available for download through MedEdPORTAL at https://www.mededportal.org/publication/9297. Each institution is encouraged to adapt the lectures according to specific institutional practices, but the core curricular elements (lectures and planning session) are preserved across sites. Students anonymously evaluate the curriculum upon its completion at each institution using an online Research Electronic Data Capture (REDCap) [15].

Clerkship Experience Survey

An anonymous, internet-based clerkship experience survey was developed, validated, and used previously with data collected using REDCap [7,8]. These electronic data capture tools are hosted at the University of Chicago [15]. REDCap is a secure, web-based application designed to support data capture for research studies, and provides the following: 1. an intuitive interface for validated data entry; 2. audit trails for tracking data manipulation and export procedures; 3. automated export procedures for seamless data downloads to common statistical packages; and 4. procedures for importing data from external sources.

The first section of the survey contained questions regarding respondent demographics in addition to experiences and confidence prior to beginning radiation oncology clerkships. The second section contained a set of questions for each clerkship completed by the student, allowing for information on up to four clerkship experiences. These questions characterized each department's demographics, curricular components, and the respondent's confidence in clinical competency at the end of the clerkship. To maintain anonymity, respondents did not report the name of any clerkship institution, although for the 2015 survey, students identified if they completed each clerkship at an institution with the curriculum. Question response format included a multiple-item Likert-type scale (5=“Extremely,” 4=“Quite,” 3=“Moderately,” 2=“Somewhat,” and 1=“Not at all”), Yes/No, and free-response questions. Likert-type results are reported as median(interquartile range). Significance was defined as p<0.05.

To establish validity, multiple radiation oncologists and medical students reviewed the survey prior to distribution and internal reliability was high. The survey has been modified and distributed annually since 2012 and the results have been reported [7,8]. Cronbach alpha values for “Prior confidence in clinical competency” and “Post-confidence in clinical competency” were 0.86 and 0.97, respectively.

Survey invitations were emailed to all applicants to a radiation oncology residency program after the 2015 National Residency Matching Program (NRMP) rank-list deadline on February 27, 2015. Three reminder emails were sent to students who had not completed the survey. The survey was closed on March 16, 2015, 1 week before NRMP Match Day, to avoid influence of match results on survey responses. A $5 coffee card was offered via email to respondents who completed the survey. Statistical analysis was performed using Stata v13.0 (StataCorp LP, College Station, TX) with descriptive statistics and the Wilcoxon rank-sum test. The University of Chicago Institutional Review Board and the NRMP approved this study.

Results

Survey response rate, demographics, and program characteristics

A survey response rate of 57.7% (109/189 complete responses) was obtained (Table 1). Overall, 109 respondents reported on 276 unique clerkship experiences. Of these clerkship experiences, 64 (23.2%) were completed at a curriculum institution, with 54 (49.5%) respondents having completed at least one clerkship at a curriculum institution. This subset of students is subsequently referred to as “curriculum students,” while students who did not complete any clerkships at a curriculum institution are referred to as “non-curriculum students.” Among the curriculum students, 25 (39.1%), 22 (34.3%), and 17 (26.5%) completed their first, second, and/or third clerkship at a curriculum institution, respectively. Forty-four respondents (81.5%) completed one clerkship and 10 (8.5%) completed two at curriculum institutions. When asked how they selected their first clerkship site, 78.9% of all students reported that it was their home institution. Demographic characteristics were similar between curriculum and non-curriculum students (Table 1). Reasons for choosing clerkship sites did not differ significantly between the two groups (data not shown). More non-curriculum students reported having received a lecture on radiation oncology during their pre-clinical years (Table 1). There were no other significant differences in prior experience related to radiation oncology or pre-clerkship confidence with specialty decision. The institutions that curriculum students selected were more likely to possess a larger number of residents. None, 42.2%, 31.3%, and 26.6% of curriculum institutions had 0–4, 5–11, 12–15, and 16 or greater residents, respectively. In comparison, 5.7%, 59%, 15.1%, and 12.7% of noncurriculum institutions had 0–4, 5–11, 12–15, and 16 or greater residents, respectively (p<0.01).

Table 1.

Demographics and survey respondent characteristics.

	All students n = 109	Curriculum students n = 54	Non-curriculum students n = 55	p value
What is your medical school education track? n (%)
MD	90 (80.6)	42 (77.8)	48 (87.3)	0.22
DO	1 (0.9)	0	1 (1.8)
MD/PhD	16 (14.7)	10 (18.5)	6 (10.9)
Other	2 (1.8)	2 (3.7)	0
Where was your medical school located? n (%)
United States	105 (96.3)	54 (100)	51 (92.7)	0.04
Outside the United States	4 (3.7)	0	4 (7.3)
What is your gender? n (%)
Female	38 (34.9)	18 (33.3)	20 (36.4)	0.74
Male	71 (65.1)	36 (66.7)	35 (63.6)
Does your medical school hospital or a hospital directly affiliated with your medical school have a radiation oncology residency program? n (%)
Yes	83 (76.2)	43 (79.6)	40 (72.7)	0.40
No	26 (23.8)	11 (20.4)	15 (27.3)
Prior to your first radiation oncology rotation, did you do any of the following? n (%)
Worked in a radiation oncology department conducting research	51 (46.8)	30 (55.6)	21 (38.2)	0.07
Spent time in a radiation oncology department shadowing physicians	66 (60.6)	34 (63)	32 (58.2)	0.61
Had a lecture on radiation oncology during pre-clinical years	25 (22.9)	7 (13)	18 (32.7)	0.01
Other radiation oncology experience	13 (11.9)	5 (9.3)	8 (14.6)	0.40
Prior to your first rotation, you had no radiation oncology clinical or research experience	24 (22)	11 (20.4)	13 (23.6)	0.68
Including your home institution, how many radiation oncology departments did you rotate through? n (%)
1	15 (13.8)	5 (9.3)	10 (18.2)	0.11
2	24 (22)	8 (14.8)	16 (29.1)
3	55 (50.5)	34 (63.0)	21 (38.2)
4	13 (11.9)	6 (11.1)	7 (12.7)
5	2 (1.8)	1 (1.9)	1 (1.8)
Prior to beginning your first radiation oncology rotation, were you DEFINITELY going to apply for a radiation oncology residency position? n (%)
Yes	53 (48.6)	24 (44.4)	29 (52.7)	0.39
No	56 (51.4)	30 (55.6)	26 (47.3)

Clerkship experiences

Curriculum clerkships were more likely to have a lecture specifically for medical students (76.6% vs. 19.8%, p<0.01) and a hands-on didactic session (62.5% vs. 16.5%, p<0.01). Curriculum clerkships were also more likely to have a greater number of lectures for medical students while non-curriculum sites were more likely to have no lectures specifically for medical students (75.9% vs. 17.2%, p<0.01; Table 2). Non-curriculum clerkships were more likely to have none of the listed formal curriculum components (51.9% vs. 10.9%, p<0.01). Medical student lectures were more likely to be given by a resident at curriculum sites (64.1% vs. 11.3%, p<0.01).

Table 2.

Clerkship characteristics for the clerkships on which students reported. Data were not collected on the students’ fifth clerkship.

	All student experiences n = 276	Curriculum site clerkships n = 64	Non-curriculum site clerkships n = 212	p value
Formal educational curricular elements. n (%)
Prepared case discussion (note: this excludes informal discussions in clinic)	91 (33)	22 (34.4)	69 (23.5)	0.79
Hands-on didactic session (i.e. contouring at a planning station on a pre-set plan; this excludes contouring for a patient during the actual planning process)	75 (27.2)	40 (62.5)	35 (16.5)	<0.01
Other (please specify)	4 (1.5)	1 (1.6)	3 (1.4)	0.93
None of the above	117 (42.4)	7 (10.9)	110 (51.9)	<0.01
Lectures specifically designed for medical students. n (%)
0	172 (62.3)	11 (17.2)	161 (75.9)	<0.01
1	17 (6.2)	5 (7.8)	12 (5.7)
2	17 (6.2)	3 (4.7)	14 (6.6)
3	33 (12)	18 (28.1)	15 (7.1)
4	26 (9.4)	20 (31.3)	6 (2.8)
5 or more	10 (3.6)	6 (9.4)	4 (1.9)
I do not recall	1 (0.4)	1 (1.6)	0

Clinical experiences were also reported. Curriculum students were more likely to have been given the opportunity to contour at a planning station for a clinical case (79.7% vs. 65.6%, p<0.05). No differences emerged in the number of students reporting other clinical experiences including performing an oncologic history and physical, writing or dictating a consult, participating in a brachytherapy case, participating in a stereotactic radiosurgery case, participating in a stereotactic body radiation therapy case, or reviewing port films.

Confidence after clerkships

Students who completed their first clerkship at a curriculum site had greater confidence after the first clerkship compared to non-curriculum site students in their ability to integrate evidence-based medicine into radiation treatment decisions (4 (3–4) vs 3 (2–4), p<0.05), their knowledge of radiation biology (3 (3–4) vs. 2 (2–3), p<0.01), their knowledge of radiation treatment set-up and positioning (3 (2–3) vs. 2 (2–3), p<0.05), and their knowledge of radiation treatment planning (3 (2–3) vs. 2 (2–3), p<0.01).

Regardless of whether or not the curriculum clerkship was their first, second, or third, curriculum students were more likely to report greater confidence with their knowledge of radiation biology at the conclusion of the clerkship (Table 3). There was a trend towards curriculum students having better knowledge of radiation treatment planning. No significant difference emerged in the students’ comfort with taking an oncologic history and physical, their knowledge of radiation physics, their knowledge of radiation treatment set-up and positioning, their ability to evaluate a dose-volume histogram, and their comfort with the decision to pursue the specialty. At the end of their clerkships, curriculum students felt more prepared to function as a first year radiation oncology resident compared to non-curriculum students (Table 3).

Table 3.

Student self-rated confidence in radiation oncology knowledge and skills after completion of clerkships. Results are reported as median Likert-type score(interquartile range) with 5=“Extremely,” 4=“Quite,” 3=“Moderately,” 2=“Somewhat,” and 1=“Not at all.”

Confidence category	All students n = 109	Curriculum students n = 54	Non-Curriculum students n = 55	p value (Wilcoxon Rank Sum)
Ability to take a full and complete oncologic history and physical	4(3–5)	4(4–5)	4(3–5)	0.38
Ability to integrate evidence-based medicine into radiation treatment decisions	4(3–4)	4(3–4)	3(3–4)	0.07
Knowledge of radiation biology	3(2–3)	3(2–4)	3(2–3)	<0.01
Knowledge of radiation physics	2(2–3)	3(2–3)	2(2–3)	0.18
Knowledge of radiation treatment set-up and positioning	3(2–3)	3(2–3)	3(2–3)	0.14
Knowledge of radiation treatment planning	3(2–3)	3(2–3.5)	3(2–3)	0.07
Ability to evaluate a dose-volume histogram	3(2–4)	3(2–4)	3(2–4)	0.17
Decision to pursue radiation oncology as a specialty	5(4–5)	5(4–5)	5(4–5)	0.83
Preparation to function in the clinic as a PGY- 2 radiation oncology resident	3(2–3)	3(3–4)	3(2–3)	0.03

Discussion

In this study, MS4s who completed a clerkship at a site with a structured didactic curriculum reported a more structured educational experience, increased knowledge of radiation oncology topics, and a significant increase in preparedness to function as a radiation oncology resident. More curriculum students reported receiving formal lectures for medical students and hands-on contouring sessions. This result confirms that the curriculum was implemented at the curriculum sites for the vast majority of students and that the students remembered the curricular elements. Curriculum students reported greater post-clerkship confidence with several radiation oncology topics after their first clerkship and in one topic regardless if the clerkship was the students’ first. After completion of all clerkships, the students did not differ in self-rated confidence in most surveyed areas despite curriculum students having significantly greater comfort with more topics after the first clerkship. This diminishing benefit of the planned curriculum may indicate that its benefits are greatest for inexperienced students rather than students who have completed multiple clerkships. Therefore, radiation oncology departments should implement a curriculum for medical students at their home program, where most students engaged in their first clerkship, to better prepare them for subsequent clerkships.

Several important non-significant results were seen that establish the similarity of the institutions and clerkship experiences that the curriculum does not affect. First, no differences emerged in the non-curricular elements of the clerkships, such as when the clerkships started and their length. Also, curriculum and non-curriculum students did not report differing clinical experiences, except contouring at a planning station. One difference between curriculum and non-curriculum sites was that residents were more likely to give lectures at curriculum sites, which may be attributable to the finding that these sites had a greater number of residents.

Others have implemented a didactic experience for radiation oncology targeted at all medical students at a single institution, rather than only those completing a four week radiation oncology clerkship [16]. The Oncology Education Initiative includes a single lecture that increased posttest scores assessing radiation oncology knowledge. Conceivably, a structured curriculum that includes multiple didactics and hands-on sessions would provide further improvements in knowledge of radiation oncology and preparedness to function as a resident while also making the experience more rewarding for students [17].

This study has several limitations. These findings were derived from the reports of 109 students who applied to a single radiation oncology program; however, the results can be extrapolated to all students completing rotations in radiation oncology and perhaps those who do not apply to radiation oncology residencies. Students who felt strongly about their rotations or those who had extremely positive or negative experiences may have responded preferentially. This was countered by offering a small incentive for survey completion. Also, students relied on their memory to complete this survey. It is concerning that 23.4% of curriculum students did not report receiving any structured lectures and 37% did not report participating in a hands-on didactic session; however, the ability of students to recall lectures and planning workshops approximately 6 months after the actual clerkship experience reflects accurate memory and an influential curriculum. Additionally, not all curriculum sites may have been conducting the curriculum for all rotating students. Nevertheless, significant increases in confidence and knowledge were seen. A final limitation is the lack of objective measurement of knowledge, which would strengthen the validity of the self-reported data.

Refinement of the curriculum is being actively pursued based on the data reported. Future directions include using objective tests to evaluate knowledge transfer and retention. Also, curriculum students could be surveyed longitudinally to reveal if they obtain radiation oncology residencies at a higher rate, if they perform better during the first year of radiation oncology residency, and if they ultimately feel better prepared at the start of residency.