Creation and Evaluation of a Cancer Survivorship Curriculum for Pediatric Resident Physicians

Abstract

Purpose:

There is a paucity of formal clinician education concerning cancer survivorship care, which produces care barriers and poorer outcomes for survivors of childhood cancer. To address this, we implemented a curriculum in childhood cancer survivorship care for pediatric residents at the University of California Los Angeles (UCLA). We examined the efficacy of this curriculum following program completion.

Methods:

A case-based curriculum was created and integrated within existing educational structures using Kern’s model. We utilized the retrospective pre-posttest method to evaluate participating residents’ knowledge, clinical skills, and attitudes towards cancer survivorship topics before and after receiving the curriculum. Pre-posttest items were compared using paired t-tests and one-sided binomial tests. We analyzed free-response question items for major themes using constant comparative methods.

Results:

Thirty-four residents completed the curriculum and its evaluation. Each assessment item significantly increased from pre- to post-curriculum; p<0.05. Greater than 40% of residents improved in all but one assessment item post-curriculum; p<0.05. Residents reported the curriculum enhanced their pediatric knowledge base (M=3.24; SD=0.65) and would recommend it to other residency programs; M=3.24; SD=0.69. Major themes included residents’ request for additional oncofertility information, training in counseling survivors, and cancer survivorship training opportunities.

Conclusions:

A cancer survivorship curriculum can successfully increase trainees’ knowledge, clinical skills, and comfort in discussing topics relevant to survivorship care.

Implications for Cancer Survivors:

With increasing numbers of childhood cancer survivors living into adulthood, residents will likely treat this population regardless of intended career path. This curriculum represents one method to deliver formal cancer survivorship training.

Introduction

As of 2011, there were an estimated 388,501 childhood cancer survivors living in the United States [1]. This number has continued to increase due to the rising incidence of childhood cancer diagnoses and continued advances in lifesaving treatments [1–2]. While higher survival rates are encouraging, treatments used in childhood cancer result in long-term and late-onset side effects due to their impact on healthy, growing tissues. Childhood cancer survivors are at an increased risk of overall and specific disease morbidity and mortality compared to their age-matched peers [1, 3]. Though over 70% of survivors report having general physical examinations, a minority receive survivor-focused care or care at a cancer center. Community physicians provide most of their health care services [4]. Therefore, primary care clinicians (including resident trainees) are integral to the delivery of preventive and acute health care for survivors of childhood cancer [5]. However, many of these providers are not comfortable taking care of this high-risk population. They are unaware of their specific health care needs, such as surveillance requirements [6–10]. A seminal report on cancer survivorship care from the Institute of Medicine (IOM) in 2006 pointed to a lack of formal education on the part of providers as a barrier to optimal care of cancer survivors. The IOM recommended educational interventions at the undergraduate, graduate, and continuing medical education levels to address these gaps in knowledge [11].

Currently, there is a paucity of graduate medical education interventions designed to improve knowledge on topics related to cancer survivorship [12–14]. A 2011 study found that of 56 North American programs that train residents and fellows in Pediatric Hematology/Oncology, only 13.9% of programs required residents to attend traditional lectures on cancer survivorship care, and approximately half of the programs offered clinical experiences pertaining to cancer survivors [14]. Prior to this study, the Pediatric Residency Training Program of the University of California Los Angeles (UCLA) did not include formal instruction in cancer survivorship topics within its larger curriculum, despite the inclusion of several learning objectives pertaining to cancer survivorship care in the list of competencies expected of program graduates [15].

The purpose of our study was to implement and evaluate a cancer survivorship curriculum aimed at a large pediatric residency program at UCLA that addressed competencies in caring for childhood cancer survivors in various clinical environments, including inpatient and outpatient practices in general pediatrics and pediatric subspecialties. We hypothesized residents’ assessment of their knowledge of, clinical skills required to care for, and comfort discussing aspects of care with childhood cancer survivors would be low prior to initiation of the curriculum. We also hypothesized residents’ attitudes on the importance of receiving training in cancer survivorship issues would be low prior to initiation of the curriculum. We postulated greater than 40% of residents would improve in their knowledge, clinical skills, comfort in discussing care topics, and attitudes towards the need for cancer survivorship care following curriculum completion.

Methods

Curriculum Design:

We utilized Kern’s six-step model to develop this curriculum [16]. Prior to creating the curriculum, a needs assessment survey was sent electronically to UCLA pediatric resident physicians of all training levels during the 2016–17 academic year (N=84) to elicit feedback on the most desired educational format in which to deliver content on childhood cancer survivorship issues. The survey was anonymous, non-mandatory, and had a response rate of 37% (n=31). Key findings from the needs assessment included residents favored (1) working in small groups, (2) a case-based format, and (3) curriculum delivery in an outpatient setting.

Based on the findings from this needs assessment survey, we developed a case-based curriculum with the guidance of medical education experts at the David Geffen School of Medicine at UCLA. Supported by principles of adult learning, case-based methods benefit from being grounded in experience and providing an active learning experience. There is strong evidence that this method is efficient and favored by both learners and faculty [17–19]. A panel consisting of UCLA pediatric residents, general pediatricians, and pediatric oncologists determined the curriculum’s content. Residents and general pediatrician panelists were selected from within UCLA Pediatric Residency Training Program leadership based on their interests in primary care medicine, medical education, and educational program development. Pediatric oncologists were selected for their expertise in cancer survivorship. The curriculum included the IOM recommendations for survivorship training for health care providers, including knowledge of prevention of secondary and recurrent cancers, assessment of medical and psychosocial late effects, and ability to intervene for consequences of cancer and its treatments [11, 20]. It also included content that focused on existing UCLA Pediatric Residency Training Program learning objectives in cancer survivorship [15], long-term follow-up (LTFU) guidelines of the Children’s Oncology Group (COG) [21], psychosocial aspects of caring for cancer survivors and their families [22–23], and additional recommendations from UCLA faculty in pediatric oncology with clinical and research expertise in survivorship care. Learning objectives for the curriculum were formulated according to Bloom’s taxonomy [24].

Guided by the recommendation of residents in the needs assessment, we integrated the curriculum into the existing outpatient continuity clinic curriculum at UCLA, which already employed small group, case-based learning. In accordance with standards required by the Accreditation Council for Graduate Medical Education (ACGME), each pediatric resident has a “longitudinal general pediatric outpatient experience” in the form of a weekly continuity clinic [25]. The UCLA Pediatric Residency Training Program has six different continuity clinic sites, which care for patients in several different settings that include: (1) a quaternary children’s hospital ambulatory clinic, (2) a clinic based in a county “safety net” facility (meaning, a medical center that has a legal obligation to provide health care for individuals regardless of their insurance status or their ability to pay) (3) a federally qualified health center serving primarily Medicaid patients, (4) a community clinic serving private insurance patients, and (5 and 6) two “free clinic” federally qualified health centers. An average of two to eight residents will have their clinic scheduled per day at each site. Prior to the start of each clinic session, one resident facilitates a case-based discussion on a pediatric care topic, which is moderated by a UCLA Department of Pediatrics faculty member. The resident facilitator receives the topic script, printouts of “primary references” for the topic, and website links for “additional resources” in advance to prepare their talking points. Topics rotate weekly. There is a different resident facilitator for each clinic talk; therefore, while each topic is presented once daily during that week, the presentation may vary slightly according to the residents’ teaching style. Topics rotate on a one to two year basis, meaning residents experience the same topic two to three times during residency. The decision to place the curriculum within this existing structure helped it reach the maximum number of residents (since a longitudinal outpatient experience is a standard component of residency training), allowed us to focus our learning objectives towards educating the primary care pediatrician on cancer survivorship topics, and will ensure the curriculum’s sustainability within the program.

The framework of the finalized cancer survivorship curriculum, including learning objectives, primary references, resident moderator outline, and additional resources, can be found in Appendix 1.

Participants:

This curriculum was established at the UCLA Pediatric Residency Training Program. Based in west Los Angeles, California, it is a medium-to-large-sized program consisting of roughly 30 categorical, four combined internal medicine-pediatrics, and two child neurology residents per year. Residents work with socioeconomically, ethnically, and culturally diverse populations across Los Angeles, caring for patients with a full spectrum of clinical pathologies and providing care at a quaternary children’s hospital, two combination academic-community hospitals, and a “safety net” hospital, in addition to the previously described continuity clinics. Through experiences at these sites, as well as opportunities in both the local and international community, residents receive a breadth of experience for training in general pediatrics.

Inclusion criteria were as follows: participants must be UCLA pediatric resident physicians of any training level during the 2016–17 academic year (N=84). Residents were excluded if they 1) did not attend continuity clinic the week the curriculum was offered, 2) did not participate in the curriculum, or 3) did not complete and return the curriculum assessment following program conclusion. Informed consent was obtained from all individual participants included in the study. All study data was kept anonymous. The UCLA Institutional Review Board approved this study.

Curriculum Evaluation:

We used the retrospective pre-posttest method to evaluate our curriculum’s efficacy. In this format, participants answer questions about their attitudes, knowledge, and/or skill level after an intervention concludes, both as it currently stands (posttest) and prior to the beginning of the intervention (retrospective pretest) [26]. The retrospective pre-posttest method has several merits that make it ideal for use in our study. First, it requires only one point of data collection, making it quick and easy for busy residents to participate. This singular data point also creates an internal standard that more accurately measures each participant’s learning [26–29]. Finally, it has been shown to be effective tool in measuring program development, especially in the evaluation of changes in learner comfort and self-efficacy. Learners are able to look retrospectively at what they knew before the learning experience, and often realize in hindsight that their level of performance was lower than they thought at the time [30–32].

Each resident received a curriculum survey containing a retrospective pre-posttest self-assessment and demographic questions immediately following the completion of each session at all continuity clinic sites. The survey was mandatory for inclusion in the study and anonymous. In the retrospective pre-posttest portion, residents were asked to indicate their level of agreement with statements concerning their knowledge and clinical skills as well as attitudes towards topics pertaining to childhood cancer survivorship both before and after their curricular session. An even-numbered Likert scale was used to produce a forced choice measurement [33]. There were free-response questions at the end of the survey that asked residents for additional qualitative feedback regarding the content of the curriculum and suggestions for future improvement.

Quantitative data analysis:

The numeric responses from the retrospective pre-posttests were examined and compared using paired t-tests. One-sided binomial tests were used to quantify the percentage of residents that showed improvement on each assessment item from retrospective pre- to posttest. Each survey question and its associated responses were coded as either zero (indicating no change in the response from retrospective pre- to posttest) or one (indicating a positive change in response from retrospective pre- to posttest). Binomial tests were completed in which p=0.4, signifying 40% or more of residents were predicted to have improvement in their score for the specified item from retrospective pre- to posttest. Data compilation and analyses were conducted using SAS® software (Version 9.4 of the SAS System for Windows. Copyright © 2005 SAS Institute Inc.).

Qualitative data analysis:

Resident feedback regarding curriculum content and areas for improvement were examined through analysis of free-response question from the curriculum survey. Responses were compiled at the end of the study period for analysis as a whole. Resident responses were reviewed and manually coded by hand by the first author (LFS). The codes were then organized into major themes by the first author (LFS). Codes and themes were created using constant comparative methods [34–36]. The first author reviewed the codes and themes with the other authors (CHB, RLK, MSS, JNC) to assess trustworthiness and resolve any disagreements. Quotes from the responses were extracted to exemplify each theme.

Results

The curriculum launched in February 2017, and 41 residents participated over a one-week period. Thirty-four residents completed the survey for a total response rate of 82.9%. Those who did not complete the surveys either 1) missed the curriculum due to tardiness to clinic (met exclusion criteria #2) or 2) did not return their surveys after participating in the curriculum (exclusion criteria #3). Participant breakdown by post-graduate year (PGY) was as follows: 14 participants were PGY-1s (41%), 13 were PGY-2s (38%), and seven were PGY-3s (21%).

Quantitative results:

Data from the curriculum surveys are shown in Figures 1 and 2. For residents’ ratings of their knowledge and clinical skills related to caring for childhood cancer survivors, each item showed a significant increase in perceived level of competence from prior to post-curriculum (p<0.05). Residents rated themselves highest both before and after program completion in their survivor medical history taking skills. In terms of counseling survivors and their families on topics pertinent to their care, each item showed a significant increase in level of comfort from prior to post-curriculum (p<0.05). Residents rated themselves highest in their ability to counsel on long-term side effects of immunosuppressants and the lowest for discussing fertility with survivors and their families both before and after program completion.

Figure 1:

Resident responses to survey items concerning knowledge and clinical skills related to survivorship care

Figure 2:

Resident responses to survey items concerning counseling survivors

Residents reported it was important to know the details of managing and caring for childhood cancer survivors both before (M=3.15; SD=0.78) and after (M=3.68; SD=0.47) receiving the curriculum. This finding improved by a significant degree following program completion; t(33)=5.02, p<0.05. Residents reported it was important to know short- and long-term morbidities affecting childhood cancer survivors both before (M=3.18; SD=0.87) and after (M=3.74; SD=0.45) receiving the curriculum, and this finding also improved by a significant degree following program completion; t(33)=4.94, p<0.05. Residents reported this curriculum enhanced their knowledge in general pediatrics (M=3.24; SD=0.65) and would recommend it to other pediatric residency training programs; M=3.24; SD=0.69.

In order to further determine the degree of resident improvement for different areas of cancer survivorship knowledge and types of clinical skills, one-sided binomial tests were utilized. Table 1 shows the results of these one-sided binomial tests. At least 40% of residents improved on each assessment item (p<0.05), except for their ability to take a survivor’s medical history; p=0.06.

Table 1:

Results of one-sided binomial tests

	Percentage of Residents Showing Any Level of Improvement, % (95% CI)	One-sided Pr >/= 0.4
Rate yourself on each of the following BEFORE this lecture (retrospectively) and CURRENTLY:
Skill in taking a medical history of pediatric survivors	52.94 (36.16, 69.72)	0.06
Knowledge of short- and long-term morbidities affecting survivors of childhood cancer	79.41 (65.8, 93)	<0.05
Knowledge of short- and long-term morbidities that apply to children after stem cell transplant	58.82 (42.28, 75.37)	<0.05
Familiarity with incidence of drug use and risky sexual behaviors in pediatric survivors	73.53 (58.7, 88.36)	<0.05
Skill in recognizing/treating associated pain disorders in pediatric survivors	61.76 (45.43, 78.1)	<0.05
Skill in recognizing/treating associated neurocognitive deficits in pediatric survivors	61.76 (45.43, 78.1)	<0.05
Understanding of race, ethnicity, and gender disparities in treatment of pediatric survivors	58.82 (42.28, 75.37)	<0.05
Rate your comfort in counseling patients and parents on these topics pertinent to pediatric survivors BEFORE this lecture (retrospectively) and CURRENTLY:
Fertility	55.88 (39.19, 72.57)	<0.05
Immunizations	61.76 (45.43, 78.1)	<0.05
Long-term side effects of chemotherapeutic agents	67.65 (51.92, 83.37)	<0.05
Long-term side effects of immunosuppressants (including corticosteroids)	58.82 (42.28, 75.37)	<0.05
Long-term side effects of radiation	64.71 (48.64, 80.77)	<0.05
Effects on siblings and other family members	67.65 (51.92, 83.37)	<0.05

Qualitative results:

The curriculum assessment included free-response questions at the end of the survey. Of the 34 residents who completed the survey, half left detailed, quotable responses (n=17). There was no suggestion of difference in impact or experience of the curriculum between resident PGYs. Analysis of the residents’ responses identified the following key themes with associated exemplary quotations:

More fertility information:

Though most residents asked for more information on specific survivorship knowledge areas, a major theme found was that they wanted more information on cancer treatments’ effects on fertility.

“More details on specifics of fertility impact.”

Additional training in counseling:

In terms of clinical skills, a major theme that emerged was residents’ request for additional training in counseling survivors and their families in the form of counseling “tips and tricks” or specific encounter examples.

“Counseling is usually more in depth. Would focus on tactics to counsel families on some of the sensitive topics.”

Further cancer survivorship education opportunities:

The final major theme to emerge was residents were interested in additional formal training in cancer survivorship issues. They felt that due to the complexity of cancer survivorship care, there was not enough time with just one session to cover all issues.

“Wish there was more time to cover all the topics. Can we also make this a grand rounds?”

Discussion

To our knowledge, this is the first study that examines an educational program designed for pediatric residents to improve delivery of childhood cancer survivorship care. Previous literature has described the creation and evaluation of cancer survivorship curricula for medical students [13], Radiation Oncology residents [12], and Hematology/Oncology fellows [12,13], and as well as the number and characteristics of programs in the United States that offer cancer survivorship training to residents [14]. Our curriculum is similar to other studies in that we incorporated cancer survivorship care guidelines from the IOM [12–13], but our curriculum was unique in that we also incorporated the LTFU guidelines of the COG [21] as well as training in the psychosocial aspects of caring for cancer survivors and their families [22–23]. This created a curriculum that specifically addressed the clinical and psychosocial care of childhood cancer survivors and their families.

Overall, the curriculum evaluation showed a significant improvement in residents’ knowledge, clinical skills, and comfort in discussing topics relevant to survivorship care. These skill areas were low at baseline and increased by a significant degree following program completion. Interestingly, residents stated that receiving training in cancer survivorship issues was important to them before participating in the curriculum. We were able to further increase their interest by a significant degree following curriculum completion. Residents also requested more formal opportunities for educational training in cancer survivorship issues, with an emphasis on oncofertility training, in the free-response questions of the curriculum assessment survey. Residents’ recognition of the importance of survivorship training is encouraging given that with increasing numbers of childhood cancer patients surviving into adulthood, it is feasible any pediatric residency trainee or graduate could encounter a cancer survivor in their current or future practice. This is regardless of intended career path, considering the increased morbidity and mortality cancer survivors face that stretch across multiple subspecialties and practice settings [3–4].

An important finding was that while residents’ post-curricular mean score was the lowest for comfort in discussing fertility with survivors and their families, it showed a statistically significant increase from previous comfort level, and greater than 40% of residents showed improvement post-curriculum. Though oncofertility training for cancer survivors was a featured component of the curriculum, residents requested more information on this topic in the free-response questions. This reflects a national problem: a majority of practicing clinicians are uncomfortable and do not feel they possess the knowledge and skills to discuss fertility issues with cancer survivors and their families [37–40].

Additional unique findings from this study done within a large quaternary university health care system included high resident ratings for their medical history taking skills and ability to counsel on long-term side effects of immunosuppressants both pre- and post-curriculum. Medical history taking skills was also the only item to not achieve at least 40% resident improvement. As a large quaternary care referral center, the UCLA Mattel Children’s Hospital treats pediatric patients with very high levels of acuity and complexity, including many patients with liver, kidney, small bowel, multi-visceral, and hematopoietic stem cell transplant. Therefore, these ratings may reflect higher than average strengths in patient-physician communication, knowledge, and clinical skills that our residents gained from caring for these highly complex pediatric patients at UCLA prior to curriculum reception.

We recognize that our study has some limitations. Our project was implemented with residents from a single program; however, the UCLA Pediatric Residency Training Program has a long track record of recruiting diverse trainees. We used one time point to collect data on program efficacy, though the retrospective pre-posttest tool has been successfully used in the past to evaluate educational programming [30–32]. Additional data collection at future time points would be needed to assess longitudinal data retention. Though one of our goals was to reach as many residents as possible, around half of residents in the program during 2016–17 did not receive the curriculum. Their exclusion may have altered program outcomes. As for our qualitative data, half of the resident participants left program feedback. These results were expected to be suggestive, not definitive, of resident attitudes toward the curriculum and require further follow up. Finally, our assessment of participant’s knowledge, clinical skills, and attitudes for cancer survivorship reached the second level (or “knows how” level) of Miller’s Pyramid [41]. As cognition may not translate to behavior, future research could evaluate whether our curriculum correlates with changes in action, through clinic observations or simulations.

For future directions for this work, we plan to adjust the curriculum’s content and structure to include more targeted information on cancer-related fertility counseling in order to increase post-curricular comfort levels in this topic. As this curriculum is now integrated into the UCLA Pediatric Residency Training Program outpatient curriculum, it is an expectation that each resident will have received this curriculum at least twice prior to graduation. We plan to collect additional data with each presentation of the curriculum to determine the reproducibility of our findings longitudinally. We also plan to follow up on the themes that emerged from our qualitative analysis by constructing a survey to determine the overall prevalence of resident viewpoints towards the curriculum. Interestingly, our qualitative analysis showed no significant differences in how each resident PGY group experienced the curriculum. This represents a notable focus for future work in that with further data collection over multiple years, this finding may change. Given the significant findings of increased resident knowledge and clinical skills in the delivery of cancer survivorship care through this curriculum coupled with resident recommendation to deliver this program to other residents, future research will include the adaptation of our curriculum to other pediatric residency programs. The curriculum’s case-based discussion nature could ease the transition of this program to others, since it is a nationally favored format in graduate medical education [17–19]. This curriculum could also be adapted to educate other providers engaged in the care of childhood cancer survivors, such as medical students, nurse practitioners, and attending-level faculty in pediatrics, internal medicine, and family medicine, who may feel inadequately prepared to take care of this special population [8–9].

In summary, this study offers an effective educational intervention to improve pediatric residents’ knowledge, clinical skills, and attitudes towards cancer survivorship care. Residents rated the curriculum highly and were interested in furthering their education and receiving additional training to effectively care for this high-risk population. Future research will include evaluation of educational programs in United States residency programs with the goals of closing the provider knowledge gap and, ultimately, improving outcomes for survivors of childhood cancer.