Abstract
Background:
Studies have shown that there is a deficiency in focused oncology teaching during medical school in Canada. This study aimed to develop oncology education goals and objectives for medical students through consensus of oncology educators from across Canada.
Methods:
In 2014 we created a comprehensive list of oncology education objectives using existing resources. Experts in oncology education and undergraduate medical education from all 17 Canadian medical schools were invited to participate in a 3-round modified Delphi process. In round 1, the participants scored the objectives on a 9-point Likert scale according to the degree to which they agreed an objective should be taught to medical students. Objectives with a mean score of 7.0 or greater were retained, those with a mean score of 1.0-3.9 were excluded, and those with a mean score of 4.0-6.9 were discussed at a round 2 Web meeting. In round 3, the participants voted on inclusion and exclusion of the round 2 objectives.
Results:
Thirty-four (92%) of the 37 invited oncology educators, representing 14 medical schools, participated in the study. They included oncologists, family physicians, members of undergraduate medical education curriculum committees and a psychologist. Of the 214 objectives reviewed in round 1, 146 received a mean score of 7.0 or greater, and 68 were scored 4.0-6.9; no objective received a mean score below 4.0. Nine new objectives were suggested. The main themes of participants' comments were to minimize the number of objectives and to aim objectives at the knowledge level required for family physicians. In round 2, the participants were able to combine 28 of the objectives with other existing objectives. In round 3, 7 of the 49 objectives received consensus of at least 75% for inclusion. The final Canadian Oncology Goals and Objectives for Medical Students contained 10 goals and 153 objectives.
Interpretation:
Through a systematic process, we created a comprehensive, consensus-based set of oncology goals and objectives to facilitate the design of undergraduate medical education curricula and improve oncology education for medical students.
It is estimated that cancer will develop in 40% of Canadians over their lifetime, and cancer is now the leading cause of death in Canada.1 Most physicians, regardless of their specialty, will be involved in the care of patients with cancer. Medical students who go on to pursue careers in family medicine or internal medicine and many of its subspecialties will be involved in screening, diagnosis and follow-up of patients with cancer.2 Despite these responsibilities, studies have shown that there is a deficiency in focused oncology teaching during medical school in Canada and the United Kingdom and many other European countries.3-7 Our recent national survey showed that most educators and learners believe oncology education in Canadian family medicine and internal medicine training programs at both the undergraduate and postgraduate levels is inadequate.4 When comparing the teaching of disease related to 10 different medical subspecialties, the educators and learners agreed that oncology was the most poorly taught to medical students. A total of 95% of members of undergraduate medical education curriculum committees and 91% of medical students thought that a standard set of oncology objectives for medical students would be useful.4
The Australian Cancer Society's Ideal Oncology Curriculum for Medical Schools8 was developed after a study had shown that graduating medical students at Australian medical schools reported substantial variability in experience in oncology and lack of important oncology knowledge.9 This document was subsequently used to develop new oncology curricula for medical schools in Australia.10 In Canada, there is no equivalent consensus regarding how oncology should be taught to medical students. Various medical schools, such as those at the University of British Columbia and the University of Alberta, have their own set of oncology objectives for their students, but many other schools do not have a comprehensive set of oncology objectives or a dedicated oncology curriculum.4 The purpose of this study was to develop a standard set of oncology education goals and objectives for medical students based on consensus by oncology educators from across Canada.
Methods
Drafting oncology goals and objectives
The study was conducted from April 2014 to April 2015. We reviewed oncology-related objectives from the Medical Council of Canada,11 Australian Ideal Oncology Curriculum for Medical Schools8 and existing objectives from various Canadian medical schools to generate an inclusive preliminary draft of potential oncology goals and objectives. The Canadian medical schools that had oncology objectives and were willing to share this information were the University of British Columbia, the University of Alberta, Western University, the University of Ottawa and Dalhousie University. Similar objectives were combined and rewritten into a single objective. We created a comprehensive draft and categorized the goals and objectives into the following headings: basic science of oncology, public health, diagnosis, treatment, prognosis, knowledge of common cancers, psychosocial issues, ethics and professionalism, communication and essential oncology experiences for medical students.
Study population
We contacted at least 1 medical oncologist from each of Canada's 17 medical schools by email and asked for names of colleagues who coordinated undergraduate oncology teaching at their institution. Oncology educators (oncologists [medical, radiation, surgical, gynecologic and pediatric], family physicians, assistant or associate deans of undergraduate medical education, members of undergraduate curriculum committees, directors of training programs in medical and radiation oncology and a psychologist) were identified, and we invited them by email to participate in the study. Delphi panels generally include fewer than 50 participants owing to feasibility issues.12 Based on similar studies, we anticipated that 25 to 40 participants would be optimal for this study.13,14
Modified Delphi process
The Delphi technique is a tool to obtain reliable consensus among a group of experts. It consists of a series of sequential questionnaires with controlled feedback in between.15 We created a modified Delphi process for this study using resources describing the Delphi process and previous similar consensus studies as a guide.13-15
We emailed a survey containing the list of draft goals and objectives to the oncology educators who had agreed to participate in the study. They were asked to assess each of the potential goals and objectives and score them on a 9-point Likert scale from 1 (strongly disagree) to 9 (strongly agree) according to the degree to which they believed each objective or goal should be taught to medical students. Participants were also given the opportunity to suggest new objectives for inclusion in the subsequent rounds of the Delphi process. Items receiving a mean score of 7.0 or higher were automatically included in the final goals and objectives, and those with a mean score of 1.0 to 3.9 were automatically excluded. Objectives with a mean score between 4.0 and 6.9 were marked for discussion in the second Delphi round. Oncology educators who failed to respond to the initial invitation within 2 weeks were sent a follow-up email reminder.
For the second Delphi round, we created an online poll and asked participants to state their availability for a 2-hour Web meeting. A meeting time when the most participants were available was chosen. During the meeting, objectives from round 1 with a mean score of 4.0 to 6.9 and any newly suggested objectives proposed during the previous round were discussed. Reasons for inclusion or exclusion of these objectives from the final list were discussed in detail and documented. We distributed a summary of the meeting by email to all study participants and requested additional comments, particularly from those who were unable to attend the Web meeting.
In the third Delphi round, we emailed participants a survey that included a summary of the discussion points from the Web meeting and asked them whether each of the objectives discussed in round 2 should be included or excluded from the final goals and objectives list. Objectives that 75% or more of round 3 participants voted for inclusion were retained among the final goals and objectives. There is variation in Delphi studies in the definition of consensus agreement.15 Our level of 75% or greater is consistent with that used in previous studies.13,14 Three newly suggested objectives were also included: appropriate diagnostic and treatment referral algorithms for patients with various common cancers, knowing the role of a palliative care physician/team and family physician in the care of patients with cancer, and understanding epidemiologic features, risk factors, prevention, screening, presentation, diagnosis, staging, basics of treatment, prognosis and follow-up/survivorship care for the most common cancers (prostate, lung, breast, colorectal, skin).
The final goals and objectives were assembled, refined and categorized into the predefined headings. We emailed the participants a draft of the final document, requesting suggestions for changes to wording; no additions or deletions of objectives were allowed. The goals and objectives were subsequently finalized and made available online.
We obtained ethics approval from the University of Calgary Conjoint Health Research Ethics Board.
Results
Of the 37 oncology educators invited to participate in the study, 34 (92%) agreed. Their demographic characteristics are shown in Table 1. The participants represented 14 (82%) of Canada's 17 medical schools (Table 1). Despite repeated attempts, we were unable to identify any faculty members at the Université de Montréal, the Université de Sherbrooke or Memorial University who were willing to participate in the study.
Table 1: Demographic characteristics of the participants.
| Characteristic | No. (%) of participants (n = 34) |
|---|---|
| Sex | |
| Male | 18 (53) |
| Female | 16 (47) |
| Professional role* | |
| Medical oncologist | 23 (68) |
| Radiation oncologist | 4 (12) |
| Director of medical or radiation oncology program | 4 (12) |
| Family physician | 3 (9) |
| Member of undergraduate medical education curriculum committee | 3 (9) |
| Pediatric oncologist | 1 (3) |
| Surgical oncologist | 1 (3) |
| Gynecologic oncologist | 1 (3) |
| Psychologist | 1 (3) |
| Location | |
| Western Canada | 13 (38) |
| University of Calgary | 6 (18) |
| University of British Columbia | 4 (12) |
| University of Alberta | 3 (9) |
| Central Canada | 2 (6) |
| University of Manitoba | 1 (3) |
| University of Saskatchewan | 1 (3) |
| Ontario | 16 (47) |
| University of Toronto | 9 (26) |
| McMaster University | 2 (6) |
| Northern Ontario School of Medicine | 2 (6) |
| Queen's University | 1 (3) |
| University of Ottawa | 1 (3) |
| Western University | 1 (3) |
| Quebec | 2 (6) |
| McGill University | 1 (3) |
| Université Laval | 1 (3) |
| Maritimes | 1 (3) |
| Dalhousie University | 1 (3) |
*Some participants had more than 1 role.
The initial comprehensive list of goals and objectives contained 224 items (10 goals and 214 objectives). A summary of the development process for the objectives is shown in Figure 1.
Figure 1.
Development process for oncology education objectives for medical students. *Similar objectives from the sources were combined and rewritten into a single objective.
Thirty-four participants took part in round 1. In this round, all 10 goals received a mean score of 7.0 or greater and were included among the final goals and objectives. A total of 146 objectives received a mean score of 7.0 or greater and were also included among the final goals and objectives. The highest-scoring objectives are shown in Table 2. Sixty-eight objectives received a mean score of 4.0-6.9. No objective received a mean score below 4.0.
Table 2: Highest-scoring objectives in the first Delphi round.
| Objective | Mean score ± SD* |
|---|---|
| Demonstrate the ability to perform a focused medical history when cancer is suspected (i.e., symptoms based on primary cancer location and symptoms related to spread to common metastatic sites, risk factors, family history) | 8.7 ± 0.6 |
| Demonstrate an understanding of presentations of cancer that represent emergencies (e.g., superior vena cava obstruction, cardiac tamponade, spinal cord compression, pulmonary embolism, symptomatic brain metastases, cancer-related bleeding) | 8.7 ± 0.6 |
| Demonstrate the ability to perform a focused physical examination for a patient with suspected cancer with emphasis on the primary cancer and possibly sites of metastases | 8.6 ± 0.7 |
| Describe nonspecific physical symptoms and signs associated with common cancers (e.g., unexplained weight loss, pain, lymphadenopathy, palpable mass, bleeding, thrombosis, change in bowel habit, biliary tract obstruction) | 8.6 ± 0.7 |
| Demonstrate an understanding of the role of a primary care physician in the treatment and follow-up of cancer patients | 8.5 ± 0.7 |
| List cancers which are screened for in the periodic health exam and the specific investigations that are used (cervical, breast, colon, prostate) | 8.4 ± 1.0 |
| Describe the importance of tissue sampling for diagnosis of malignancy and for identification of molecular predictive factors | 8.4 ± 0.9 |
*On a 9-point Likert scale from 1 (strongly disagree) to 9 (strongly agree) according to the degree to which participants believed each objective should be taught to medical students.
Nine new objectives were suggested for discussion. They dealt with understanding case-control and cohort studies, how radiation is used to treat cancer, how novel targeted agents are used to treat cancer and how they differ from traditional cytotoxic chemotherapy, and understanding that some metastatic cancers are curable.
Expert feedback in round 1 had 3 common themes. First, oncology education objectives for medical students should be approached from the perspective of what a family physician would need to know about oncology. It was suggested that more emphasis be placed on risk factors, symptoms, screening and diagnosis, and less emphasis be placed on treatment, complications, chronic issues and diagnosing recurrent cancer. Second, many participants were concerned that there were too many objectives. They suggested that some similar objectives could be combined and that others could be learned in other parts of the medical school curriculum. Third, a suggestion was made to consider having 1 standard format for the "Knowledge of common cancers" section, with 1 template for common cancers and another, less detailed template for less common cancers.
Twelve (35%) of the 34 participants were able to take part in the round 2 Web meeting. During the course of the discussion, the participants were able to combine 28 of the objectives with other objectives.
All 34 participants took part in round 3. The 49 objectives evaluated in this round are given in Appendix 1 (available at www.cmajopen.ca/content/4/3/E359/suppl/DC1). Seven objectives received at least 75% (range 77%-100%) consensus for inclusion and were retained among the final goals and objectives (Table 3). The remaining 42 objectives received less than 75% (range 0%-67%) consensus and were excluded from the final goals and objectives.
Table 3: Objectives that received at least 75% consensus for inclusion in the third Delphi round and were retained among the final goals and objectives.
| Objective | Agreement for inclusion, % |
|---|---|
| Public health | |
| Demonstrate an understanding of case-control and cohort studies with respect to how they are used to study the impact of risk factors on the development of cancer | 77 |
| Diagnosis | |
| Identify appropriate diagnostic and treatment referral algorithms for patients with various common cancers | 90 |
| Treatment | |
| Demonstrate an understanding of the general principles of how radiation is used to treat cancer and different types of radiation (e.g., external beam, brachytherapy, stereotactic radiation) | 84 |
| Demonstrate an understanding of novel targeted agents which are now used to treat cancer and how they differ from traditional cytotoxic chemotherapy | 90 |
| Demonstrate an understanding of the role of the palliative care physician/team and family physician in the care of cancer patients | 100 |
| Prognosis | |
| Demonstrate an understanding that some metastatic cancers are curable | 97 |
| Knowledge of common cancers | |
| Demonstrate an understanding of the epidemiology, risk factors, prevention, screening, presentation, diagnosis, staging, basics of treatment, prognosis and follow-up/survivorship care for the most common cancers including: prostate cancer, lung cancer, breast cancer, colorectal cancer and nonmelanoma skin cancer | 87 |
A total of 10 goals and 153 objectives were assembled, refined and categorized into the predefined headings. No significant changes were made to the wording of the goals and objectives on subsequent review by the participants. The final Canadian Oncology Goals and Objectives for Medical Students are listed in Appendix 2 (available at www.cmajopen.ca/content/4/3/E359/suppl/DC1).
Interpretation
We created a comprehensive set of oncology education goals and objectives for medical students through a national consensus process with oncology educators. This resource can be used to improve oncology education for future physicians across Canada.
Our previous study indicated that most educators and learners believed that a standard set of oncology objectives would be useful for learners and that a textbook or Web book focusing on oncology education for generalist physicians would be useful.4
A previous survey study of administrators at Canadian schools of medicine, nursing and pharmacy and postgraduate resident training programs showed that the current level of oncology education in these programs was inadequate.3 Training programs for pharmacists, nurses and other health care professionals may find our national objectives useful as a starting point for the development of oncology objectives for their students. In fact, we are aware of an initiative among Canadian oncology pharmacists who have recently created a task force to improve oncology education at pharmacy schools across Canada (Melanie Danilak, Cross Cancer Institute, Edmonton, and Jennifer Jupp, Alberta Children's Hospital and Foothills Medical Centre, Calgary: personal communication, 2015). In addition, our undergraduate objectives may be used as a starting point to develop more advanced objectives suitable for postgraduate medical trainees in family medicine and internal medicine programs.
Australia is the only other country with a standard set of oncology goals and objectives for medical students. To our knowledge, the United States and United Kingdom have not developed national oncology goals and objectives for their medical students.
Limitations
One possible limitation of our work is the fact that curricula at medical schools across Canada are variable, and our document does not address how these objectives should be taught. Medical schools will need to determine where and how best to integrate these objectives into their respective programs. Another limitation is that these goals and objectives for medical students represent the product of a modified Delphi process carried out by participants who were identified by their peers as key undergraduate oncology educators. The work is generally representative of a national collaboration of oncology educators, but despite our best efforts to obtain representation from every medical school in Canada, we were unable to recruit participants from 3 medical schools. However, the 14 medical schools represented by the participants cover every region in Canada and graduate 80% of all Canadian doctors.16 Another potential limitation is that most of the experts surveyed were medical oncologists. However, our participants did include family physicians and other oncology specialists, such as in radiation, surgical, gynecologic and pediatric oncology. Finally, only 12 experts participated in round 2 of the modified Delphi process, but this was mitigated by the fact that 34 experts participated in rounds 1 and 3.
Conclusion
Through a systematic process, we created a comprehensive, consensus-based set of oncology education goals and objectives that may be used in the design of undergraduate medical education curricula and also by educators and medical students. It is hoped that these goals and objectives will facilitate improvements in oncology education for medical students and the care they provide for their patients with cancer.
Supplemental information
For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/4/3/E359/suppl/DC1
Supplementary Material
References
- 1.Toronto: Canadian Cancer Society. Cancer statistics at a glance. 2015. [accessed 2015 Oct. 19]. Available www.cancer.ca/en/cancer-information/cancer-101/cancer-statistics-at-a-glance/?region=ab.
- 2.Del Giudice ME, Grunfeld E, Harvey BJ, et al. Primary care physicians' views of routine follow-up care of cancer survivors. J Clin Oncol. 2009;27:3338–45. doi: 10.1200/JCO.2008.20.4883. [DOI] [PubMed] [Google Scholar]
- 3.Cheung WY, Fishman PN, Verma S. Oncology education in Canadian undergraduate and postgraduate training programs. J Cancer Educ. 2009;24:284–90. doi: 10.1080/08858190902973143. [DOI] [PubMed] [Google Scholar]
- 4.Tam VC, Berry S, Hsu T, et al. Oncology education in Canadian undergraduate and postgraduate medical programs: a survey of educators and learners. Curr Oncol. 2014;21:e75–88. doi: 10.3747/co.21.1667. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Payne S, Burke D, Mansi J, et al. Discordance between cancer prevalence and training: a need for an increase in oncology education. Clin Med (Lond) 2013;13:50–6. doi: 10.7861/clinmedicine.13-1-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Cave J, Woolf K, Dacre J, et al. Medical student teaching in the UK: How well are newly qualified doctors prepared for their role in caring for patients with cancer in hospital? Br J Cancer. 2007;97:472–8. doi: 10.1038/sj.bjc.6603888. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Robèrt KH, Einhorn J, Kornhuber B, et al. European undergraduate education in oncology: a report of the EORTC Education Branch. Acta Oncol. 1988;27:423–5. doi: 10.3109/02841868809093568. [DOI] [PubMed] [Google Scholar]
- 8.Sydney (Australia): Cancer Council Australia. Ideal oncology curriculum for medical schools: knowledge, skills and attitudes of medical students at graduation. 2007. [accessed 2015 Oct. 19]. Available www.cancer.org.au/content/pdf/HealthProfessionals/OncologyEducation/IdealOncologyCurricDEC07-updatedcover.pdf.
- 9.Tattersall MHN, Langlands AO, Simpson JS, et al. Undergraduate education about cancer: a survey in Australian medical schools. Eur J Cancer Clin Oncol. 1988;24:467–71. doi: 10.1016/s0277-5379(98)90018-9. [DOI] [PubMed] [Google Scholar]
- 10.Koczwara B, Barton MB. The ideal oncology curriculum for medical students. J Clin Oncol. 2006;24:5334. doi: 10.1200/JCO.2006.07.5465. [DOI] [PubMed] [Google Scholar]
- 11.Ottawa: Medical Council of Canada. Objectives for the qualifying examination. 2015. [accessed 2015 Oct. 19]. Available http://apps.mcc.ca/Objectives_Online/objectives.pl?loc=home&lang=english.
- 12.Witkin B, Altschuld J. Thousand Oaks (CA): Sage Publications. Planning and conducting needs assessments: a practical guide. 193-202. 1995. [Google Scholar]
- 13.Aronson BD, Janke KK, Traynor AP. Investigating student pharmacist perceptions of professional engagement using a modified Delphi process. Am J Pharm Educ. 2012;76:125. doi: 10.5688/ajpe767125. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Giuliani ME, Gillan C, Milne RA, et al. Determining an imaging literacy curriculum for radiation oncologists: an international Delphi study. Int J Radiat Oncol Biol Phys. 2014;88:961–6. doi: 10.1016/j.ijrobp.2013.12.009. [DOI] [PubMed] [Google Scholar]
- 15.Powell C. The Delphi technique: myths and realities. J Adv Nurs. 2003;41:376–82. doi: 10.1046/j.1365-2648.2003.02537.x. [DOI] [PubMed] [Google Scholar]
- 16.Toronto: Ivy Global. Medical schools in Canada. 2014. [accessed 2015 Oct. 19]. Available www.ivyglobal.ca/mcat/med_schools_canada.asp.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.