Call for improvement in medical school training in genetics: results of a national survey

Abstract

PURPOSE:

To assess, from the student perspective, medical school training in genetics and genomics.

METHODS:

In 2019, the Undergraduate Training in Genomics (UTRIG) Working Group developed genetics-related survey and knowledge questions for the RISE-FIRST, an exam administered to postgraduate year 1 (PGY1) pathology residents in the United States during their first months of training. Survey questions focused on perceived knowledge in genetics and the structure and quality of training with responses compared with those in control areas.

RESULTS:

There were 401 PGY1 pathology residents who took the 2019 RISE-FIRST (65% of those in the United States). There was significantly lower perceived understanding of genetics compared with nongenetics topics. Respondents also reported less time spent learning genetics and lower quality training compared with control areas. Only 53% indicated an interaction during medical school with a medical geneticist. Residents also did not perform as well on the UTRIG-developed knowledge questions than those in other areas of pathology.

CONCLUSION:

The RISE-FIRST is a useful tool in assessing the current state of medical school training in genetics. This needs assessment may serve as a call to action to improve medical school genetics education and promote greater understanding of the role of genetics professionals in patient care.

INTRODUCTION

Genetic testing has entered almost every medical specialty. As such, physicians in all areas of medicine need to understand principles of genetics and genomics to best care for their patients.¹ There is evidence, however, of poor physician knowledge in this area. A number of studies have highlighted the utility of genetic counselor screening of molecular testing orders in ensuring more efficient genetic testing, as physicians may unintentionally order inappropriate assays.^2,3 In a recent systematic review, limitations to genetics knowledge and skill were found to be an important barrier to integrating genetics into physicians’ usual practice.⁴ In the most recent of an ongoing case series, published in 2019, of adverse events related to genetic testing, “improving genetics and genomics education of non-genetics health care professionals” was listed as one of several solutions to help ensure correct genetic test ordering, interpretation, and use.⁵

To help address this physician knowledge gap, in 2016, the Undergraduate Training in Genomics (UTRIG) Working Group was formed through the Undergraduate Medical Educators Section (UMEDS) of the Association of Pathology Chairs with the primary goal of improving genomics literacy among medical students.⁶ UTRIG, an interprofessional working group, leverages expertise in education, genetics, and molecular pathology to create pragmatic educational resources designed to improve genomics education. In addition to medical school course directors, the UTRIG working group includes representatives from major national and international pathology and genetics organizations.

UTRIG is based on the Training Residents in Genomics (TRIG) Working Group, which was created for pathology resident education. TRIG has created a number of educational resources and, since 2012, has developed survey questions for the American Society for Clinical Pathology (ASCP) Resident In-Service Exam (RISE).⁷ Administered in every pathology residency program in the United States, the RISE provides a unique large-scale opportunity to assess pathology residents’ knowledge of molecular pathology and genomics.

In 2014, ASCP developed the RISE-FIRST for postgraduate year 1 (PGY1) pathology residents. RISE-FIRST, taken in the first few months of residency, provides residency program directors data on resident baseline knowledge, enabling them to optimize subsequent training. Approximately 65% of the all PGY1 pathology residents in the United States take the RISE-FIRST.

Given that RISE-FIRST is administered to PGY1 residents who have just finished medical school, UTRIG recognized its potential to assess the current state of undergraduate medical education in genetics and genomics. In this article, we report the responses for the survey questions developed by the UTRIG Working Group for the 2019 RISE-FIRST. We also report the results of pilot genomics knowledge questions developed by the UTRIG Working Group with comparison of resident performance to other medical school topics. While other reports have examined current training of medical students from the course directors’ perspective this report represents a national assessment by recent medical school graduates. The results demonstrate that there is a significant need for improvement in genetics education within medical school curricula as well as increased contact between medical students and genetics professionals.

MATERIALS AND METHODS

RISE-FIRST

The ASCP RISE-FIRST has been offered to pathology residency programs since 2014 and is administered over approximately a 4-week period in June and July to PGY1 residents who have recently graduated medical school. Program directors use the exam to assess resident baseline knowledge and identify areas of strength and/or improvement to best plan the residency-training curriculum. Individual programs, both in the United States and Canada, register to have their trainees take the RISE-FIRST.

The RISE-FIRST includes an untimed demographic section, also containing UTRIG-developed survey questions, followed by a timed 2-hour and 45-minute examination. The exam has approximately 150 multiple-choice questions in a “single-best-answer” format. A total percent score is provided as well as two additional scores: one in anatomic pathology (AP) and another in clinical pathology (CP). The knowledge questions are solicited from practicing pathologists and then vetted by a RISE-FIRST committee consisting of experts in all tested areas. These questions are also tagged by specialty area (e.g., hematopathology, molecular pathology/genomics).

RISE-FIRST question design process

Since 2015, the UTRIG Working Group has developed survey questions for the RISE-FIRST. Using email, in-person meetings, and monthly conference calls, these questions have been honed over time and the results from the 2019 survey, included in the demographics portion of the RISE-FIRST, are presented in this paper. The questions addressed perceived ability related to various topic areas in molecular genetics and genomic medicine. Control questions focused on core, more established, nongenetic pathology and other medical knowledge topics during the preclinical (pharmacology and pathology) and clinical years (cardiology). Other survey questions focused on the degree of contact the participants had with genetics professionals during medical school as well as the time and quality of teaching modalities used for their genetics education. The full survey is provided as Supplemental material.

For the 2019 RISE-FIRST exam, UTRIG also provided knowledge-based genetics/genomics questions. Members drafted questions that were then vetted by the entire UTRIG Working Group prior to submitting to the RISE-FIRST Committee. Six UTRIG questions were chosen as experimental questions on the exam. Experimental questions are not identified as such to test takers and they are not a part of the trainees’ final scores.

Rasch and statistical analysis

For survey questions, the response percentages are reported. The percent correct is reported for knowledge questions.

For the first survey question regarding self-perceived knowledge, a two-tailed t-test was used to compare ratings for genetic versus nongenetic topics. Rasch analysis was also employed to assess the quality of this question and rating scale.⁸ Rasch analysis determines the “fit” of actual responses with those expected using a model based on item “difficulty” and individual “ability.” For this question, the overall construct was comprehensiveness of medical education as indicated by overall knowledge of the different topics queried. That is, all listed topics should have been covered in medical school and the more comprehensive a responden’s medical school education, the more topics that they would have rated highly concerning knowledge. For a well-fitting survey question, the responses would be consistent with the model. For example, items with overall low ratings (high “difficulty”) in regard to knowledge would still be more likely to be rated highly by those that had the most comprehensive education (high “ability”). The “fit score,” based on χ² analysis, is a measure of statistical fit of actual answers with those expected by the model with an optimal score of one. Person reliability, similar to Cronbach’s alpha, can also be calculated. The quality of the rating scale can be assessed by the use distribution of the different rating scale choices.

RESULTS

RISE-FIRST demographics

There were 401 PGY1 pathology residents in United States who took the 2019 RISE-FIRST representing approximately 65% of PGY1 pathology residents in the United States and 64% of US pathology residency programs. All areas of the United States were represented with a relative distribution of 14% Mid Atlantic, 19% Midwest/North Central, 27% Northeast, 13% West, 18% South Central, 8% Southeast.

Perceived knowledge of genetics and nongenetics areas

Respondents (PGY1 pathology residents recently graduated from medical school) were asked to rate their understanding of 13 topic areas on a five-point scale (poor, fair, good, very good, excellent; Fig. 1). Rasch analysis demonstrated a high person reliability of 0.89 and an optimal average fit score of 1. The analysis also showed that the rating scale was appropriately used among respondents across ability levels.

Fig. 1. Perceived understanding of medical school topics.

Postgraduate year 1 (PGY1) residents were asked, “How would you rate your understanding of [topics on x-axis]”. Answer choices were as follows: “Not covered in medical school training,” “I am not familiar with this at all” (poor); “I have very limited knowledge” (fair); “I have some knowledge” (good); “I have a solid knowledge base” (very good) “I have proficient knowledge” (excellent). The y-axis indicates the percent answering good, very good, or excellent. Hgb, hemoglobin; Hist inflame, histology of inflammation; PCR, polymerase chain reaction; Histo neo, histology of neoplasia; Ethics, ethical issues related to genetic testing; Testing, the role of genetic testing in the diagnosis and treatment of genetic disease; Whole/exome, whole genome/exome sequencing; Seq, sequencing; Online tools, use of online tools to determine clinical significance of genetic variants.

In general, nongenetic topics were rated more highly than genetics topics with 50% of participants rating their knowledge of nongenetics topics as good or better compared with 32% for genetics-related topics (p = 0.0049). For the control topics of glucose or hemoglobin level test interpretation and histologic changes associated with inflammation or neoplasia, 60%, 54%, 46%, and 43% of respondents, respectively, rated their knowledge as good or better. Polymerase chain reaction (PCR) was the genetics topic for which the most (45%) residents rated their understanding as good or higher. The only other genetic content areas showing a similar result (≥40%) were the topics of “the role of genetic testing in the diagnosis and treatment of disease” and “ethical issues related to genetic testing.” The genetic content areas with the lowest proportion of residents (≤25%) indicating good or better understanding were “whole genome/exome sequencing,” “single-gene sequencing methods,” and “use of online tools to determine clinical significance of genetic variants.”

PGY1 residents had little exposure to online genomics tools (Fig. 2). The majority of trainees had not heard of all the provided websites except clinicaltrials.gov and PubMed. While 95% of trainees responded that they had used PubMed (the control), only two other websites, OMIM (23%) and clinicaltrials.gov (35%), had been used by greater than 10% of the participants.

Fig. 2. Familiarity with online genetics tools.

Postgraduate year 1 (PGY1) residents were asked, “Have you heard of or used any of the following websites?” (specific websites listed along x-axis). Answer choices were as follows: “I have never heard of this website,” “I have heard of but have never used this website,” “I have used this website.” The y-axis represents the percent responding that they had used the website.

Interaction with genetics professionals

Participants were asked, “During which years of medical school did you interact clinically or through educational sessions?” with different genetics professionals (Fig. 3a). As expected, for the control questions, almost all residents had interacted with a pathologist and cardiologist during their training. In contrast, only 53% indicated any interaction with a medical geneticist and only 32% with a genetic counselor. Similar percentages indicated that medical geneticists and genetic counselors were not involved in teaching related to genetics and genomic medicine. If there was an interaction, most reported it was “minimal” or there was an occasional lecture or learning session. In contrast, 87% of participants answered that a pathologist played a role in genetics education with 44% indicating “major faculty” (many lectures or learning sessions) or course director.

Fig. 3. Interactions with health-care professionals.

(a) Postgraduate year 1 (PGY1) residents were asked, “During which years of medical school did you interact clinically or through educational sessions with [genetic counselors, medical geneticists, pathologists/pathology faculty, cardiologists]”. Answer choices included the following: “did not interact,” “preclinical,” and “clinical” (y-axis, percentage of residents indicating each answer choice). Total numbers for each health-care professional may add up to more than 100% as residents were allowed to select more than one answer (e.g., if they had worked with one of the professionals both in the preclinical and clinical years). Cards cardiologists, GC genetic counselors, Geneticists medical geneticists, Path pathologists/pathology faculty. (b) Medical school training. PGY1 residents were asked, “During which years of medical school did you receive training in [molecular genetics and/or genomic medicine, pharmacology, cardiology]”. Answer choices included the following: “did not have training,” “preclinical,” and “clinical” (y-axis, percentage of residents indicating the answer choices). Total numbers for each health-care professional may add up to more than 100% as residents were allowed to select more than one answer. Cards, cardiology; Mol gen/genomics, molecular genetics and/or genomic medicine; Pharm, pharmacology.

Genetics education during medical school

Over 30% of PGY1 participants responded they did not have training in genetics and genomic medicine during medical school (Fig. 3b). When they did, it was primarily during the preclinical years. In contrast, for the control topics, almost all respondents had training in pharmacology, more commonly during the preclinical years, and in cardiology, occurring equally during the clinical and preclinical years.

Concerning format for teaching genetics, 32% indicated exposure to a standalone genetics course and 69% indicated an exposure to an integrated course during the preclinical years. While 25% reported availability of elective genetics courses during the clinical years, only 6% reported a required genetics course or rotation. With regard to teaching modalities in which they learned genetics content in these courses and/or rotations, almost all residents reported experiences with lecture format. Approximately 20–30% also learned genetics content through either team-based learning, problem-based learning, or small group sessions. Only 12% and 3% interacted with actual patients or simulated/standardized patients, respectively.

Over 80% of participants reported that their time spent learning cardiology, pathology, and pharmacology, in terms of contact hours, was either “more” or “much more” than genetics. Over 50% also indicated the quality of training was better in these subject areas when compared with genetics (Fig. 4).

Fig. 4. Comparison of time and quality of training in molecular genetics and genomic medicine.

Left: PGY1 residents were asked, “In terms of contact hours, how did the time spent in molecular genetics and genomic medicine compare to the following other topics” (topic choices were pharmacology, pathology, and cardiology). Answer choices included the following: “much less than molecular genetics and genomic medicine,” “less than molecular genetics and genomic medicine,” “about the same as molecular genetics and genomic medicine,” “more than molecular genetics and genomic medicine,” “much more than molecular genetics and genomic medicine.” Zero respondents reported that time spent in pharmacology, pathology, and cardiology was less than in molecular genetics and genomic medicine. Right: participants were asked, “In terms of quality of instruction, how do you feel molecular genetics and genomic medicine compare with the following other topics” (same three topics for comparison). Answer choices included the following: “much worse than molecular genetics and genomic medicine,” “worse than molecular genetics and genomic medicine,” “about the same as molecular genetics and genomic medicine,” “better than molecular genetics and genomic medicine,” “much better than molecular genetics and genomic medicine” (y-axis, percentage of residents responding with indicated answer choice). Zero respondents reported that quality of training in pharmacology, pathology, and cardiology was less than in molecular genetics and genomic medicine.

Knowledge questions

The results of the six piloted knowledge questions on the RISE-FIRST are shown in Table 1. The results are compared with the percent correct for each of the scored molecular pathology questions as well as the overall percent correct for all AP and all CP questions.

Table 1.

Percent correct on RISE-FIRST questions.

Undergraduate Training in Genomics (UTRIG)-developed pilot genetics questions	% Correct
Utility of BRCA testing	63%
Utility of whole exome sequencing	61%
Next-generation sequencing coverage	29%
Germline vs. somatic testing	31%
Genetic Information Nondiscrimination Act (GINA)	7%
Use of online genomics tools	26%
Average	36%
Graded molecular pathology/genomics (MP/G) questions
Principles of polymerase chain reaction (PCR)	81%
Epigenetic changes	80%
Association of Lynch syndrome with microsatellite instability	67%
Karyotype abbreviation inversion; definition	90%
Retinoblastomas; variants in the RB1 gene	84%
Variants that introduce a stop codon	81%
Karyotype interpretation; amniotic fluid	81%
Average	81%
Anatomic pathology questions (90 questions)
Average	69%
Clinical pathology questions (71 questions; includes graded MP/G questions)
Average	63%

The overall percent correct for AP and CP was approximately 65%. PGY1 residents performed better on the scored molecular pathology/genomics questions with an average of 81%. These questions targeted more “classic” molecular pathology and cytogenetics. The average percent correct for the UTRIG genetics questions was 36%. Most respondents answered questions related to depth of coverage for next-generation sequencing, the difference between somatic versus germline testing, use of online tools, and the Genetic Information Nondiscrimination Act (GINA) incorrectly.

DISCUSSION

This report demonstrates medical school curricula are not integrating genetics and genomic medicine and lack interactions with genetics professionals. These findings suggest a basis for the physician knowledge gap in genetics and genomics. Although knowledge of genetic testing is important across medical specialties, the time allocated to and the perceived quality of genetics education was less than that of other medical school content areas. Optimal patient and interprofessional care comes from physicians understanding the roles of genetics professionals and how to best use their consultative services. These objectives are best achieved through exposure to genetics professionals in medical school and graduate training. However, only 32% of PGY1 residents had interacted with a genetic counselor during medical school. In addition, previous publications note that only one third of those entering clinical genetics residency did an elective prior to entering field.^9,10 This low degree of exposure not only contributes to practical knowledge gaps in genomics, but also to limited familiarity with the different genetics-related career pathways, which is critical given the current shortage of genetics professionals.¹¹

As shown in previous studies, training in genetics is still heavily weighted toward the preclinical years.¹² While providing such a framework early in training is important, there needs to be an understanding of the clinical impact of genetics. Integration of genetics training into the clinical years is particularly important as preclinical time is decreasing at some medical schools.

It is interesting that over 30% of the participating PGY1 residents responded that they did not have “training” in molecular genetics and/or genomic medicine. As phrased in the survey, the definitions of training as well as molecular and genomic medicine were left up to the respondent. Clearly, all medical students are receiving some education related to genetics. Due to space limitations, we were not able to probe further into the respondents’ definitions and perspectives in this regard. However, the results show that a significant number of the PGY1 residents believed their educational experience in genetics differs in regard to quality and curricular time from what was provided for the control specialties. The perceived shortcomings in the genetics curriculum are also consistent with the findings related to knowledge of different topic areas and interactions with genetics professionals.

The results are a call of action to medical educators of all fields to work with the genetics community in improve genetics training in medical school. Several resources are available. For example, the G2C2 website (Genetics/Genomics Competency Center; https://genomicseducation.net/) lists educational resources grouped by topics. The Association of Professors in Human and Medical Genetics (APHMG) has published medical school core competencies and their Medical School Genetics Course Directors Special Interest Group (CD SIG) is working on several projects including the update of the Medical School Genetic curriculum guidelines, and aligning their question bank with the National Board of Medical Examiners (NBME) style and mapping to curriculum competencies.^13,14 The UTRIG Working Group has developed genetics/genomics modules designed with the intent of being easy to integrate into existing medical school curricula. The sessions are based in active learning principles and lend themselves to interdisciplinary and interprofessional facilitation. This pragmatic curriculum and tools for implementation are available to download free of charge at http://www.pathologylearning.org/trig/resources. This resource provides the opportunity for greater interaction of geneticists and genetic counselors with medical students. Additional support may come from genetic professional organizations funding medical school genetics interest groups or simply suggesting members reach out to course directors for presentation time to explain the important roles genetics professionals play. Non-genetics residency programs should also be aware of this deficit in medical school education and should consider approaches to fill these gaps during graduate training.

The results from the knowledge question component of the exam support the findings from the survey portion. The PGY1 residents had relatively high self-reported knowledge of PCR, which correlated with a correct response rate of 81% for the PCR-related knowledge question. PGY1 residents also scored well on questions related to cytogenetics and basic molecular genetics such as understanding variants that lead to a stop codon. PGY1 residents performed more poorly on all of the UTRIG-developed questions, which focused more on modern genomics concepts such as next-generation sequencing depth of coverage and GINA. Participants also had difficulty with differentiating somatic versus germline variants. While these UTRIG knowledge-based questions do support a need for improved genetics training in medical school, there were only a small number of pilot questions, thus precluding statistical analysis and necessitating caution in interpretation. Recently, results from the 2019 pathology RISE have recently been published.¹⁵ PGY4 residents performed worse on questions in molecular pathology and genomic medicine when compared with a control of hematopathology, suggesting a need for improved training not only in medical school but in residency. The reported use of online tools also supports a need for greater genetics education in medical school. As would be expected, almost all students had used PubMed as it is a general resource for searching the literature. While it might be expected that medical students would not have been exposed to all tools on the list, less than 25% had used COSMIC, OMIM, and/or ClinVar. Of note, in the aforementioned paper on the 2018 RISE results, PGY4 pathology residents, reported greater, albeit still inadequate, exposure to online tools, with 30% or more having used these tools.

There are several other limitations to our study. First, we were able to survey only a small fraction of graduating medical students and they were all individuals who chose pathology as a specialty. However, our results are perhaps more sobering as pathology trainees may be more likely to pursue genetics training in medical school than other graduates as it is a core component of pathology residency programs. It is important to also note that approximately 50% of PGY1 pathology residents were international (non-US) medical school graduates. As such, these findings reflect education both inside and outside the United States and provide evidence of a need for improved undergraduate training on an international scale.

To our knowledge, this study is also the largest to have assessed the current state of medical student training in genetics directly from recent medical school graduates. The Medical School Graduation Questionnaire (GQ), which annually provides aggregate data from all US graduates of Liaison Committee on Medical Education (LCME) accredited medical schools, gathers information on preparation in many areas of medicine. This represents a much wider scope of student feedback; however, the assessment for “genetics” is quite limited as it is based on a single self-assessment question—“How well did your study of the following sciences basic to medicine prepare you for clinical clerkships and electives?”—without a parallel knowledge-based assessment. Genomics and/or molecular diagnostics are not among the topics assessed by the GQ. In addition, as questions are only asked regarding core clerkships, no information is obtained on genetics education during the clinical years of training. Due to space limitations, we were also not able to include a larger number of control specialty questions. Cardiology and pharmacology were selected as specialties with expected significant education in the clinical and preclinical years, respectively.

The RISE-FIRST is a useful tool for querying recent medical school graduates about their training in genomics. We hope to continue to utilize this assessment tool to assess and track changes in PGY1 pathology resident knowledge and perspectives related to their genetics and genomics medical school educational experience. Our results serve as a needs assessment for a call to action to improve medical school genetics education and promote a better understanding of the role of genetics professionals in patient care.

THE UNDERGRADUATE TRAINING IN GENOMICS (UTRIG) WORKING GROUP

Richard L. Haspel⁵, Rebecca Wilcox⁶, Patricia V. Adem⁷, Hana Anderson⁸, James B. Atkinson⁹, Leah W. Burke¹⁰, Loren Joseph⁵, Robin D. LeGallo¹¹, Madelyn Lew¹², Christina M. Lockwood¹³, Rizwan Naeem¹⁴, Hasan Rizvi¹⁵, Julian Sanz Ortega¹⁶, Kate Shane-Carson¹⁷, Mark E. Sobel¹⁸, Eric Suarez¹⁹, Laura J. Tafe²⁰ and Jason Wang²¹

⁵Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, MA, USA. ⁶University of Vermont Medical Center and University of Vermont Larner College of Medicine, Burlington, VT, USA. ⁷Memorial Sloan Kettering Cancer Center, New York, NY, USA. ⁸University of California, Davis, School of Medicine, Sacramento, CA, USA. ⁹Vanderbilt University Medical Center, Nashville, TN, USA. ¹⁰University of Vermont Larner College of Medicine, Burlington, VT, USA. ¹¹University of Virginia School of Medicine, Charlottesville, VA, USA. ¹²Michigan Medicine, University of Michigan, Ann Arbor, MI, USA. ¹³University of Washington, Department of Laboratory Medicine, Seattle, WA, USA. ¹⁴Montefiore-Einstein College of Medicine, Bronx, NY, USA. ¹⁵Barts Health NHS Trust, London, UK. ¹⁶Clinica Universitaria de Navarra, Anatomía Patológica, Madrid, Spain. ¹⁷The Ohio State University College of Medicine, Columbus, OH, USA. ¹⁸American Society for Investigative Pathology, Rockville, MD, USA. ¹⁹Uniformed Services University, Pathology, Bethesda, MD, USA. ²⁰Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA. ²¹Cook Children’s Medical Center, Fort Worth, TX, USA.