Abstract
This position paper discusses on-going academic remediation challenges within the field of medical education. More specifically, we identify three common contemporary problems and propose four recommendations to strengthen remediation efforts. Selecting or determining what type of remediation is needed for a particular student is akin to analyzing a Gordian knot with individual, institutional and systemic contributors. More emphasis, including multi-institutional projects and research funding is needed. Recommendations regarding language use and marketing of such programs are given.
Keywords: Medical education, Academic remediation problems
Background
Academic remediation is a near universal problem faced by medical schools and residency training programs. Studies suggest that the need for remediation is significant given 10% of medical students encounter an academic failure at some point during their training [1]. In response to this need, a high percentage of medical schools now offer some form of academic support for their students. Offering such services is laudable and necessary, but as many who have sat down across from a struggling student can attest, analyzing the contributing factors and deciphering a solution is akin to having a Gordian knot perched on your desk.
Curriculum reform efforts in the US are shifting to learner-based strategies, competency-based programs and respect for multiple modes of learning [2–5]. Similar curriculum reform efforts in Europe and the United Kingdom (UK) have been discussed and debated in recent years [6] including incorporation of the World Health Organization and World Federation of Medical Education (WHO/WFME) guidelines [7]. In addition, recruitment processes have widened their search for capable matriculants from diverse backgrounds [8–10] which requires changes in views of learning. All of these provide a unique opportunity to evaluate and revise aspects of the system.
Academic remediation within medical school programs is a topic of significant interest and has produced notable publications (e.g., Guerrasio [11] and Kalet and Chou [12]). However, remediation as a systematic process has not been overhauled from the traditional view that academic struggles are solely an individual problem which is handed off to remediation personnel. In this regard, we agree with Cleland and associates [13] and with Kalet, Ellaway and colleagues [14, 15] that remediation should be an explicit part of the structure of medical education with shared responsibilities rather than as an afterthought or “outsider” activity.
Nearly all medical schools invest in remediation resources, but most err in not having cross-talk between remediation specialists and instructors or curriculum managers at strategic points throughout students’ developmental process of becoming a physician. Instead remediation is relegated away from the core action of medical education; consigned to manage the ‘penalty box’ where struggling learners are sent after committing academic ‘fouls’. Without avenues of two-way communication between the remediation program and the larger educational program and without revising our view of remediation as ‘fixing’ struggling learners, remediation will remain, in our opinion, less effective and more costly to individual learners and to the school (see also [16]).
Cleland and associates article [13] as well as others have recently written about the disconnect between individual remediation needs and the larger educational system and have raised important associated questions. While Cleland and colleagues do offer some specific suggestions, overall they highlighted more problems and points for discussion rather than providing specific solutions. In this paper, we build upon their work and identify three common problems associated with the disconnect between the larger medical education process and remediation and subsequently propose four recommendations to minimize those problems.
Problem 1. Struggling learners—Can the system find them early?
There are four potential sources for identifying struggling learners: pre-matriculation and demographic data, self-identification, student advancement or academic review committees, and the use of multiple-sourced, detailed, across-time performance data (portfolios).
Many risk contributors have been studied and are known before the first day of medical school. Some of those known risk factors are briefly reviewed below. Ferguson, James and Madeley [17] in their meta-analysis of studies involving students attending UK medical schools found that (1) women slightly out-perform men, and (2) identification as an ethnic minority tends to be a slight disadvantage. More recently, Woolf, Potts & McManus [18] found in their meta-analysis of UK physician training, wide-spread differences between white and non-white candidates. In addition, they found undergraduate grade point average (GPA) predicted only 23% of the variance in medical school grades, and learning style (e.g., Kolb’s convergers or the tripartite model’s deep and strategic) had weak correlations with academic performance (0.18 to 0.26). Donnon and associates’ [19] meta-analysis of Medical College Admissions Test (MCAT) score’s’ predictive validity of medical school performance found small to medium effect sizes (essentially zero up to 0.4) for various measures of later performance. Dyrbye et al. [20] found that students’ “serious thought” of dropping out of medical school was associated with demographic or event factors, e.g., being older in medical school (> 30 years old), having a child, being in the 3rd year of medical school, and self-identifying as an American Indian or Pacific Islander. As anticipated, they also found greater risk of academic problems was associated with significant psychosocial stressors such as major personal illness, major illness in a significant other, and divorce. No researchers have proposed that the above noted data points should be the sole identifiers of at-risk students. While these are known relationships, to the best of our knowledge, no researchers have combined known risk factors with concerning early performance data in an effort to identify struggling learners at the beginning of their academic training.
A second source for identifying struggling learners is self-referral. Only a small percentage of students self-refer to remediation resources. Combining 10 years of data, Guerrasio et al. [21] found approximately 7% of students were self-referred to a remediation program for medical students and professionals. Based on our experiences, many of these students tend to be high achievers whose drivers for self-referral include chronic anxiety about their performance. Thus seeking assistance or reassurance is not a new behavior for them. In our view, there is little change needed at the system level for this subset of struggling learners.
A third source for identification of an at-risk student is through a more formal process led either by faculty (Ellaway et al. [14]) or an academic or competency review committee. Typically, committees review a student’s situation following a critical academic or performance failure and then make recommendations to the school’s leadership regarding that particular student’s status within the school. While there is little question that the students who are reviewed by such committees are ‘at-risk’ learners, this action commonly occurs very late in the process. That is, remediation programs and experts sit by the “sidelines” until there has been a significant failure or a series of failures by an individual student. These institutional-level processes typically proceed slowly, are perceived as punitive by students and are designed to capture those students with the most serious academic or professional behavior problems, i.e., those who may be facing the possibility of disenrollment.
Stegers-Jager and colleagues [22] presented a different model from a Dutch medical school implemented recently which did include early identifications occurring at 4, 7 and 12 months into the preclerkship studies. Warnings were given at the 4- and 7-month time points, and academic probation was applied at 12-months. They did find more students participating in supportive services under this model (than the previous process). They found those students participating in one-on-one remediation services were significantly more likely to satisfactorily complete their preclerkship studies.
As for longer-term functioning of students reviewed by an academic review committee, Durning et al. [23] analyzed the performance of Post Graduate Year-1 (PGY1) residents comparing those who had been referred for academic review to those who had not. They found that being presented to the committee was associated with a higher risk of below average performance during PGY-1. However, they also found that the overall percentage of those students who interacted with the committee and had below average performance ratings during internship was low. The authors noted the need for more research including: (a) the timing and intensity of remediation efforts and whether either is associated with higher risk for later performance problems and (b) looking at the strength of the relationship with later performance comparing those students with primarily academic issues from those with primarily professional problems.
Similar to Kalet and colleagues [15, 24], we believe that as a student is identified later into the curriculum, the higher the likelihood of that student presenting with exhaustion, discouragement, and a lag in academic performance, which could have a potentially negative impact on the student’s ability to modify or attempt changes in their study process.
A fourth potential source for identification is the use of multiple-sourced, across-time performance of students analyzed with the goal of early identification of struggling learners. The notion of a learning or progress portfolio for medical students is certainly not new (e.g., [25–28].
Portfolios can be used as to means to comprehensively collect available performance data including academics. Recently, Kalet and Pusic [29] advocated use of a holistic and competency-based portfolio aggregating various scores and ratings from multiple sources. This process is currently in development at their program (New York University School of Medicine). Unfortunately, few details were provided regarding the management and design of this process. However, it is clear that the profile metrics are available to the student and their mentor for periodic review. One example table in their chapter listed 21 specific content “buckets” for various performance metrics such as histology, pathology, pharmacology, etc. as well as displaying for each student their performance as compared to their cohort for each content area in a box-and-whisker format. This is an example of an individualized and very detailed tracking system for students’ academic and professional performance and is in concert with our intended meaning for portfolio.
In Kalet and Pusic’s school, it appears such data remains with the student and their mentor and is not used by the larger system in identification of at-risk learners. Presumably, a student who consistently shows academic difficulties would be referred for additional assistance, but when or how the system interacts with or uses individuals’ data in the portfolio is not clear.
Problem #2. Remediation—Which flavor works for which student?
Remediation, as a label, has been applied to a diverse set of processes including various skill-building presentations, tutoring, conceptual models, and one-on-one activities. With a few notable exceptions (see below), little is available which dissects the procedure and, most critically, details how to individualize remediation to meet the needs of a specific struggling learner. As noted by Winston [30] and others, simply teaching to pass the next exam does not work in the longer-term for struggling learners.
Over two thirds of medical school remediation programs make available services such as time management strategies, test taking techniques, note taking strategies, reviews of class exams, review of course content, modeling use of resources, and modeling problem-solving [31]. As noted by Cleland and colleagues a most noticeable downside to the above practices can be summed up as a systemic view that quick and easy fixes have the ability to address the problem of struggling learners [13]. Alternatively these generic services can ignore some of the individual contributors of a particular struggling learner. Some medical schools address the needs of at-risk students by providing pre-matriculation programs focused on enhancing study-skills and/or tutoring to promote early exposure to and learning of the material. Miller [32] described one such program which resulted in students modifying their anticipated study plan, but unfortunately no data was presented related to actual academic outcomes.
Remediation approaches applied in medical education research have been categorized into four models: analytic, developmental, synthetic and competency-based [33]. Unfortunately, such models have been found to explain little of the outcome variance whether referring to clinical skills development, basic science knowledge or professionalism [16, 34–37].
Winston [30] discussed an in-depth, year long, multi-pronged, intensive program for students repeating a portion of the curriculum involving regular faculty-facilitated, small group work with integration of study skills and presented data indicating their success. Sayer et al. [38] described a small, reportedly successful program in a UK medical school specifically targeting academic issues with lengthy one-on-one tutoring, a collaboratively designed remediation plan and a careful individualized assessment process.
An alternative approach is a listing of potential types of learner deficits (“diagnoses”) such as difficulties in fund of knowledge, clinical reasoning, history-taking, physical examination or professionalism [39]. Guerrasio [11] adds the following additional remediation ‘diagnoses’: time management and organizational skills, interpersonal skills, communication, practice-based learning, system-based learning and mental well-being.
Beyond descriptive summaries and looking at scattered outcome metrics, little research has focused on how to build an effective and individualized process to provide successful remediation. Ascertaining what works and doesn’t work in remediation is difficult. Guerrasio [11] has perhaps one of the most detailed and practical discussions based on their remediation process at University of Colorado School of Medicine and she diligently references many sources supportive of particular processes. Kalet and Chou [12] also document many practical aspects and viewpoints related to conducting remediation with individual students. As these authors and others (e.g., [13, 40]) note most research on medical education academic remediation is focused on a retake of a failed examination and provides little basis for determining of what types of support are needed for which students or what type of remediation is most helpful. Kalet, et al. [12] rightly note “as yet [there is] little evidence supporting how and why remediation in medical education works” (pg 24).
A second complication in individualizing effective remediation is the varied background of staff who are tasked to create or manage remediation programs. A survey of 134 medical schools found that less than half of the staff who were responsible for remediation in medical schools had graduate degrees in education, learning or in enhancing individual behavioral change [31]. Over one third of these staff members reportedly had no particular training nor prior experience in providing academic support. Similar to above, this data suggests remediation at the graduate level is a young field with the added disadvantage of not having a solid basis in theory or discipline.
A third complication, as identified by Guerrasio [11], Sayer et al. [38] and Mcloughlin [41] focuses on the numerous contributors to academic difficulties. For a particular student, the number of contributors to academic struggles is typically multiple per struggling learner and many are related to the student’s background experiences and/or to current vulnerabilities within their social and family settings. Thus, teaching specific skills, e.g., time management or the notion of spaced learning, is often an inadequate response for at least a portion of struggling students. Cleland and associates [13] also highlight the need to recognize pressures from institutional culture as well as individual student backgrounds.
Durning and associates [33] take a different approach. Rather than attempting to match remediation strategies to a specific student or their learning difficulties, they took an empirically and theory-based application of a self-directed learning process which is generic to topic and context and adapted it for struggling medical learners. This model, Self-Regulated Learning—Microanalytic Assessment and Training (SRL-MAT), is a problem-solving approach which functions as a three-phase cyclical loop and is applicable across learning situations. Specifically the three cycles are: (a) Forethought including strategic planning and motivational factors, (b) Performance involving metacognitive monitoring, self-testing and self-control and (c) Self-Reflection about performance such as causal attribution, self-evaluation and satisfaction. These skills are intended for the application of immediate to longer-term work products. Notably, this model is based on a solid foundation in educational research with data showing successful learning outcomes from multiple learning contexts and has been successfully utilized in improving the diagnostic reasoning of medical students [42].
Fourth, rarely is there any formal assessment of the struggling learner, beyond an unstructured interview, to objectively measure their academic and cognitive strengths and weakness. While many remediation programs include an interview in their process, what is meant by ‘an interview’, the degree of structure and coverage for that interview and how that information is used is a glaring hole.
A welcome exception is Yellin’s [43] work which discusses the issues associated with formal cognitive assessment, determining strengths and weaknesses and if testing supports it, applying diagnosis of learning disabilities for medical students. He advocates for the use of broad-based cognitive testing to ascertain struggling medical students’ relative strengths and weaknesses within six neurodevelopmental constructs (attention, language, memory, temporal-sequential ordering, spatial ordering and higher-order cognition) to individualize the remediation plan. His process outlines the use of objective testing to create a cognitive profile used to help students better understand their cognitive and learning relative strengths and weaknesses. Based on the individual profile, the remediation team develops a tailored learning plan or a “management by profile” plan. When testing is not utilized, some academic weaknesses may not be recognized or addressed such as subtle reading comprehension problems, a need for better writing skills or below average clinical reasoning skills [44]. A downside to the above process is the high cost and time needed for formal assessment as well as a specialist (e.g., neuropsychologist) who is familiar with the necessary testing tools and, critically important, is willing to frame an evaluation within the context of graduate medical education (rather than simply using general population or age-based norms).
At the individual student level, as Mcloughlin [41] and Yellin [43] noted many students use the same strategies which produced success for them in their secondary and post-secondary educational endeavors. When faced with performance difficulties in medical school, these students often redouble their use of those same well-used strategies. Finding a new strategy is time-intensive and few medical students take the risk of potentially wasted time in trying new strategies. Medical students, almost by definition, are highly successful learners across decades of learning and adaptation to many environments. For these students any serious academic problem, not to mention consistent academic failures, is an unexpected and foreign hardship which they have no experience managing or learning from in productive ways. For the student in the throes of early professional identity development as a physician, serious academic problems can challenge their resiliency, motivation and career goals.
Fifth, research which incorporates demographic and cultural differences as covariates beyond gender and age are lacking. There are many who advocate the need for diversity and minority representation in the medical professional [45–49]. Difficult issues such as communication within cross-cultural supervision and mentoring of medical trainees has been addressed [50]. Little if any attention is focused on any empirical remediation investigations which also included broader diversity issues in the medical literature.
Sixth, there is scant recognition in the remediation literature of the difficulties all learners face when attempting to change their well-learned and previously servable strategy. The most commonly used skills deficit model in remediation asks students to change strategies and habits which have been personally effective for years and successfully used for thousands of hours. Changing deeply ingrained behaviors is difficult, stressful, and early attempts can be easily abandoned before experiencing success with an alternative strategy.
A final consideration is acknowledgement that a massive percentage of students are masterfully successful in adapting to the challenges of graduate medical education. Unfortunately, almost nothing is known about how high performing versus ‘below average’ medical students adapt. Information regarding how ‘super’ learners in medical school change, adapt and accomplish what they do is lacking in the educational literature. Discovering their successful adaptive strategies may add additional tools to the remediation process.
Problem 3. Stigma—It continues to interfere
At the individual student level, most medical students are historically unacquainted with academic difficulties [13] and are reluctant to come forward and ask for additional assistance [41, 51, 52]. In addition, some students have a bias in their perceptions of what causes these academic problems. For example, Cleland and associates [40] found that 4th year medical students in a UK medical school with below average performance tended to blame external factors, believe that their most recent failure was an isolated event and believe that faculty did not offer the appropriate assistance (when in fact multiple messages to aid these same students had been sent).
At the institutional level, few would find overt bias against students needing additional assistance. However, there seems to be a subtle belief which may affect how leaders and universities perceive remediation programs. Specifically, a view that students with early academic difficulties will continue to struggle throughout their professional development, leading to a future of mediocre doctoring skills. For example, Cleland et al. [40] stated “weak medical students go on to become weak doctors” and cited two references [53, 54]. Both of the cited sources highlight professional behavior problems and/or communication problems between team members. Neither source is focused on the relationship between early academic difficulties and later “poor” doctoring. For example, the first source, Papadakis and associates [53] concluded that low MCAT scores and low grades during the first two years of medical had “only ¼ the risk” of subsequent disciplinary action (as compared to the risk faced by student with early professional behavior problems). Based on the numbers Papadakis and colleagues presented, a student with an academic failure of a medical course during the 1st or 2nd year of medical school, had only a 7% risk of subsequent disciplinary action by state board. In contrast, those with documentation of unprofessional behavior during medical school, had more than three times that risk (26%) of subsequent disciplinary action by state board. In a similar study Papadakis [55] did not find academic metrics (e.g., undergraduate GPA, lower quartile MCAT performance, not passing one or more medical school courses) were a significant predictor using logistic regression modeling of later professional behavior problems. A statistically significant difference was noted between the undergraduate GPA of the group with later disciplinary actions and other graduates (3.3 vs. 3.4), but this is not a meaningful difference at the level of individual students nor was this variable a significant predictor in the logistic regression results.
Similarly, a study of over 3000 Canadian physicians found no statistically significant relationship between a score indicative of knowledge competency (score from a traditional written exam with 450 multiple choice questions (MCQs)) and future patient complaints against the physicians [56]. Guerassio [21] in a study of over 150 struggling learners spanning from undergraduate to graduate medical education, found only professionalism issues were a significant predictor of later probation, and that medical knowledge was not a predictor. In Santen and associate’s [57] study of students identified by student academic promotions committee either through academic issues or behavioral problems, approximately 4% of those students later had state medical board actions taken against them for unprofessional behavior. All of the students in their sample who had professional behavior difficulties also had academic issues. Scrutiny of Santen’s published summary of their data suggests it is possible students who only had academic issues and were not also identified as having behavioral problems, were not later disciplined by state boards for unprofessional behavior (however, this is uncertain based on how they presented their data). LaRochelle et al. [58] found evidence that pre-clerkship performance in clinical reasoning (small group exercises, multiple choice examination focused on clinical reasoning and Objective Structured Clinical Examination (OSCE) performance) did have small, but significant predictive value of resident ratings of medical expertise by faculty.
In addition, there are studies that suggest early academic performance or even board examination scores are unrelated to other important non-knowledge-based aspects of physician care. For example, Hojat et al. [59] found no relationship between scores on United States Medical Licensing Examination (USMLE) Step 1 examination and empathy scores. No studies were found directly linking early low academic performance (pre-med school or pre-clerkship) as a primary or substantial contributor to later poor quality of doctor-patient relationships, diagnostic errors, treatment errors, or patient satisfaction. This view is perhaps the flip side of a long tradition of high academic performance being the principal gatekeeper for entry into certain specialties; academic performance is seen as a proxy for doctoring capacity or aptitude (see Prober et al. [60] for additional discussion of the pros and cons of using Step exam scores as primary selection variable of residents).
An additional factor to be identified when considering the subtle institutional bias against remediation is that much of academic remediation literature borrows heavily from the universally familiar medical model, that is, find the diagnosis and once the diagnosis is known, that diagnosis ‘determines’ treatment. While this view obviously has heuristic usefulness, the extent that this viewpoint should be extended to remediation of individual students is unclear. Given this model is steeped in the assumptions of disease, pathology processes and (body) system failures, it is easy to imagine how, even if unintentionally, promotion of this view can carry negative assumptions. For example, even if a student is identified as a struggling learner, this does not necessarily mean that the student has a fundamental flaw or set of deficits that would negatively impact that student’s functioning as a physician. Much of medical education support services, e.g., teaching a specific and presumed missing skill, are based on the above unstated assumption that the pace and rigor of medical school exposes a pre-existing skills deficit. That is, the student performed lower on an examination than expected, therefore the student has an academic or study skills deficit and/or a motivational problem.
Potential solutions
Next we turn to potential solutions to deal with these challenges that face the medical education community. Here, we list four recommendations.
Recommendation #1. Use systematically captured data, across-time, multiple-sourced information with one of the goals being early identification of struggling learners
The concept of and use of portfolios in medical education is certainly not new. However, many uses of portfolios cited in the medical education literature appears to be a collection of student performance datum without organization into competency domains and without ways to represent growth over time within those competency domains. For example, Chertoff and associates [61] surveyed 71 Liaison Committee on Medical Education (LCME) accredited medical school. Approximately half reported use of portfolios, but less than 10% of total respondents noted they felt their portfolio process included a visual display of competency growth. Other recent publications noting uses of portfolios are within particular modules or rotations, e.g., Sanchez Gomez [62] or within particular aspects of training such as simulation centers [63]. While helpful, these do not provide indications of overall growth or, in the case of remediation, lack of individual progress.
We advocate for the use of a portfolio as an across-the-curriculum, integrated longitudinal assessment tool to guide and help the remediation process such as discussed by Van Tartwijk et al. [64]. A comprehensive portfolio includes an organizational schema such that various assessments are gathered from multiple sources and placed into one or more general competency domain or “bucket” as labelled by Kalet’s group [29]. It allows learner and teachers to support students’ learning and closely assess students’ progression [65].
Specifically, we recommend a comprehensive portfolio of all performance data (knowledge-based, group performance, OSCE-type performance, written reports of patient encounters, self-assessment of professional growth, ratings or commentary of videotapes of leaning interactions, etc.) be collected. Further that this collected data be made available to the student as well as to selected key individuals on the academic team (e.g., the remediation team and those charged with tracking performance and with early identification of struggling learners). Importantly, we recommend that at least one reviewer of student portfolio data should be tasked with taking a longitudinal view of the learner’s progress particularly within competency ‘buckets’ as discussed above. That is, the longitudinal reviewer is not as concerned with rating or grading a student performance related to current courses, but is to monitor, over time, for marginal performance within a competency area. Thus, remediation is not just studying for the next exam, but also analyzing developmental patterns. Furthermore the concept of portfolio-based monitoring and learning may be particularly important in the remediation process [25, 27]. The use of a portfolio that contains a large amount of longitudinal information about learning interactions and learners’ performance across settings and topical domains. Thus, such information may be particularly useful in developing and implementing remediation plans for some students which otherwise would have gone unrecognized. In addition, once such a longitudinal portfolio process is put into place, the school could eventually analyze previous years of data to potential identify school-specific performance markers within early curriculum experiences which, when paired with known pre-matriculation risk factors, could help identify students at risk for later serious academic problems.
The use of performance portfolio raises the concern of ‘forward-feeding’ performance problems to subsequent faculty or clerkship directors. This has been debated in the literature [16, 24, 66–68]. Clearly there are privacy concerns and other issues to consider. Much of above dialogue centered upon who should have access to such information including clerkship directors and/or faculty and discusses the potential for negative (and positive) biases. Our opinion is that there should exist a separation of the processes which determines official student status (typically a slow-moving, large group-based process triggered by significant failure or string of failures) and the remediation team (desirably a faster-moving, dynamic process). Ideally this latter team is a small group, is closer to ‘real time’ in its tracking of student performance and is skilled in its identification of struggling students. Its function falls within student advocate services and it should engage early and often with the struggling student. We strongly recommend those involved in reviewing and using that data for early identification of struggling learners be either separate from or not have a formal vote into procedures which determine student status within the program.
Recommendation 2. Encourage and fund research investigating the complex processes of medical education at the level of individual change including a variety of contributors
Directly assessing remediation is difficult. However, finding active change agents is critical. In medical research, we sensitively delve into individuals’ complex personal lives to probe aspects of cancer, diabetes, degenerative diseases, etc. and compare treatment alternatives. However, we are slower to apply similar rigor, energy and funding to medical remediation. Critics may argue academic difficulties during medical education is not a disease nor a pathology which affects the population at large or may consider it a non-problem as only affects a small percentage of students and thus doesn’t need additional funding or focus. In our view, this perspective is out-of-touch with the current directions and values of medication education.
Acquiring large numbers of struggling learners to study is difficult at the institutional level, but is less so if collaborative multi-institutional studies are implemented. Implementation of work might involve consortiums across a small number of medical schools, shared development of performance portfolios, identification of similar performance metrics, as well as integrating and analyzing de-identified performance data across institutions.
In addition, many remediation programs appear to function as independent islands and offer services that are based on the experience and skillsets of individual remediation specialists. More coordination and sharing of ideas, resources, and processes could help more remediation forward as well as generate data to answer hypothesis-driven questions concerning optimum remediation methods.
Recommendation 3. Use of De-stigmatizing processes and language
The traditional ‘diagnosis➔treat’ view of remediation is problematic. No heuristic is perfect. But perhaps a better model for conceptualizing struggling learners within medical education is that of athletic conditioning for a team of gymnasts or decathlon athletes. Each athlete has years of experience, a history of many high-level performances, but each athlete, each coach and the institution understand qualification is at an individual level and must occur in a variety of skillsets or performances. Every athlete experiences strong events and weak performances. For these high-functioning athletes, conditioning (or remediation) is not about discovering a pre-existing flaw. The foundation of conditioning is a careful and detailed analysis of recent performances, identification of relative strengths and weaknesses and then designing targeted exercises to enhance performance, dedicated time to build up weaker skills as well as developing and practicing strategies for key performances.
We recommend reducing the use of terms such as remediation, diagnosis, deficits, etc. A more useful articulation may be using terms such as ‘not meeting a qualification standard’. While cumbersome, such phrasing suggests a student did not achieve their maximum potential for a particular performance event rather than the implication that the student, as a holistic organism, is dysfunctional, incapable or lacks an essential skillset which bars them from a success within the medical school context.
We are working with highly qualified and gifted students who are being stretched to their near maximum capacity for years at a time. Thus, we recommend terms such as academic conditioning, performance coaching, learning strengths and weaknesses (or cognitive and learning profile), and adaptive strategies. Perhaps even terms such as “ath-learner” or “aca-lete” should be considered. While this last suggestion is somewhat tongue-in-cheek, the underlying notion detailed above is not. We are not simply recommending a change in the label. The recommendation is to move the culture away from terminology which inherently supports an assumption of pathology and deficit and instead use language which recognizes: (a) there are complex factors impacting performance and (b) these students are above average learners operating in a very high-demand environment.
Recommendation 4. Encourage self-identification and normalize help-seeking
A one-time lecture simply reiterating above messages is unlikely to provoke change in student’s help seeking behavior (see [69] for example). Friedman and associates [70] found, in their efforts to reduce de-stigmatizing attitudes toward mental health usage, that a lecture-based format was perceived as patronizing by medical students. However, they did find that students were more receptive to testimonials from individuals who were using mental health-related services. In recent work, Cleland and colleagues [13] discussed the importance of the institutional culture affecting help-seeking behavior and they offered a few specific suggestions, e.g., asking the student their views of contributors to academic difficulties, working in partnership with struggling learners, and the staff role-modeling self-care.
A useful option to normalize help seeking over time may be for medical schools to annually ask for volunteers among graduating students and faculty to record brief personal audio or video ‘testimonials’ of healthy help-seeking options they utilized during medical school (each volunteer choosing to specifically identify themselves or not). A brief suggested outline or reminders (e.g., list of common hurdles) might assist volunteers to address known barriers, e.g., confidentiality, worries about effect on career as a student, being reluctant to self-identify as having additional needs, etc. [71, 72]. It may be appropriate to instruct the volunteers to not individually name less-than-helpful local individuals or providers. Obviously, the volunteers would not be expected to disclosure personal details of their difficulties, but are asked to provide only a generic reference to how stressors impacted them personally while in medical school and then focus on helpful options they experienced which helped them not only manage, but succeed. These personalized help-seeking declarations could then be made electronically available to successive cohorts who could access them as needed through their schooling via networks internal to the medical school. Making it known that certain individuals within leadership and faculty of the medical school are willing to provide such testimonials may encourage graduating students to do the same.
In addition, brief messages from the current school leadership supporting appropriate help-seeking by students being available within the same venue may be helpful as well. Also, as part of school leadership messages, referencing studies noting the percentage of medical students who do seek assistance can be a normalizing factor. For example, Dyrbye and associates [71] found approximately 40% of medical students annually seek out formal or informal supports.
Third, we recommend applying the well-established medical model of consultation to academic issues and struggling learners. Again, leadership and supervisor messaging is critical. That is, rather than telling students with academic problems they are in need of an evaluation to detect and diagnosis weakness and deficits, we recommend referring to need for additional resources as an academic consultation. As physicians, providers are expected to recognize when specialty care or evaluation is needed particularly when standard treatments are not effective. Why should not a similar attitude be fostered from the beginning of their medical training? The message should be, “If you have adequately applied the recommended ‘treatments’ to your studies and those are not effective, it is time to obtain a consultation!”.
There are, of course, limitations to our discussion and proposals. All of these issues are challenging and no solution will fit all situations or schools. Along with research investigating active ingredients of successful remediation, research into the links between remediated students and longer-term outcomes is needed.
Conclusions
This position paper discusses some on-going problems within the field of medical education, identifying four contemporary problems and proposing four recommendations to strengthen remediation efforts. Earlier identification and assistance to learners “not meeting qualification standards” has been discussed by many, but is rarely implemented in a systematic manner. We recommend the use of a professional performance portfolio organized by competency domains and include techniques to show learner progress over time as a way of addressing this long-standing problem. Curriculum revision offers opportunities to incorporate programmatic approaches to remediation. Selecting or determining what type of remediation is needed for a particular student is a Gordian knot. More emphasis, including multi-institutional projects and research funding is needed. Remediation as an effort continues to be a seen as a necessary evil within the larger mission of medical education as opposed to a program that strengthens and conditions individual learners, and in turn the institution and ultimately medical practice. Recommendations regarding language use and marketing of such programs are given.
Acknowledgments
Disclaimer
The views expressed herein are those of the authors and not necessarily those of the Department of Defense or other federal agencies.
Abbreviations
- GPA
Grade Point Average
- LCME
Liaison Committee on Medical Education
- MCAT
Medical College Admissions Test
- MCQ
Multiple Choice Questions
- OSCE
Objective Structured Clinical Examination
- PGY1
Post Graduate Year 1
- SRL-MAT
Self-Regulated Learning—Microanalytic Assessment and Training
- UK
United Kingdom
- USMLE
United States Medical Licensing Examination
- WFME
World Federation for Medical Education
- WHO
World Health Organization
Authors’ contributions
LB initial draft and corresponding author. SD, JL, MY, DS-G, BR, DT all contributed editorial comments and review. All authors read and approved the final manuscript.
Ethics approval and consent to participate
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Layne D. Bennion, Email: layne.bennion@usuhs.edu
Steven J. Durning, Email: steven.durning@usuhs.edu
Jeffrey LaRochelle, Email: Jeffrey.LaRochelle@ucf.edu.
Michelle Yoon, Email: myoon515@gmail.com.
Deanna Schreiber-Gregory, Email: deanna.schreiber-gregory.ctr@usuhs.edu.
Brian V. Reamy, Email: brian.reamy@usuhs.edu
Dario Torre, Email: dario.torre@usuhs.edu.
References
- 1.Holland C. Critical review: medical students’ motivation after failure. Adv health Scie Educ. Theory Pract. 2016;21:695–710. doi: 10.1007/s10459-015-9643-8. [DOI] [PubMed] [Google Scholar]
- 2.Irby DM, Cooke M, O'Brien BC. Calls for reform of medical education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Acad Med. 2010;85:220–227. doi: 10.1097/ACM.0b013e3181c88449. [DOI] [PubMed] [Google Scholar]
- 3.Cooke M, Irby DM, O'Brien BC. Educating physicians: a call for reform of medical school and residency. New York: Wiley; 2010. [Google Scholar]
- 4.Ogur B, Hirsh D, Krupat E, et al. The Harvard Medical School-Cambridge integrated clerkship: an innovative model of clinical education. Acad Med. 2007;82:397–404. doi: 10.1097/ACM.0b013e31803338f0. [DOI] [PubMed] [Google Scholar]
- 5.Jones R, Higgs R, De Angelis C, et al. Changing face of medical curricula. Lancet. 2001;357:699–703. doi: 10.1016/S0140-6736(00)04134-9. [DOI] [PubMed] [Google Scholar]
- 6.Likic R, Dusek T, Analysis HD. Prospects for curricular reform of medical schools in Southeast Europe. Med Educ. 2005;39:833. doi: 10.1111/j.1365-2929.2005.02228.x. [DOI] [PubMed] [Google Scholar]
- 7.Karle H. Global standards and accreditation in medical education: a view from the WFME. Acad Med. 2006;81:S43. doi: 10.1097/01.ACM.0000243383.71047.c4. [DOI] [PubMed] [Google Scholar]
- 8.Rumala BB, Cason FD., Jr Recruitment of underrepresented minority students to medical school: minority medical student organizations, an untapped resource. JNMA. 2007;99:1000–1009. [PMC free article] [PubMed] [Google Scholar]
- 9.Cohen JJ, Gabriel BA, Terrell C. The case for diversity in the health care workforce. Health Aff. 2002;21:90–102. doi: 10.1377/hlthaff.21.5.90. [DOI] [PubMed] [Google Scholar]
- 10.Agrawal JR, Vlaicu S, Carrasquillo O. Progress and pitfalls in underrepresented minority recruitment: perspectives from the medical schools. JNMA. 2005;97:1226–1231. [PMC free article] [PubMed] [Google Scholar]
- 11.Guerrasio J. Remediation of the struggling medical learner. Irwin PA: Association for Hospital. Med Educ. 2013;184.
- 12.Kalet A, Chou CL, editors. Remediation in medical education. New York: Springer; 2014. p. 367.
- 13.Cleland J, Cilliers F, van Schalkwyk S. The learning environment in remediation: a review. Clin Teach. 2018;15:13–18. doi: 10.1111/tct.12739. [DOI] [PubMed] [Google Scholar]
- 14.Ellaway RH, Chou CL, Kalet AL. Situating remediation: accommodating success and failure in medical education systems. Acad Med. 2018;93(3):391–8. [DOI] [PubMed]
- 15.Kalet A, Chou CL, Ellaway RH. To fail is human: remediating remediation in medical education. Perspect Med Educ. 2017;6(6):418–424. doi: 10.1007/s40037-017-0385-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Frellsen SL, Baker EA, Papp KK, et al. Medical school policies regarding struggling medical students during the internal medicine clerkships: results of a national survey. Acad Med. 2008;83:876–881. doi: 10.1097/ACM.0b013e318181da98. [DOI] [PubMed] [Google Scholar]
- 17.Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ. 2002;324:952–957. doi: 10.1136/bmj.324.7343.952. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Woolf K, Potts HWW, McManus IC. Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis. BMJ. 2011;342:d901. doi: 10.1136/bmj.d901. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. 2007;82:100–106. doi: 10.1097/01.ACM.0000249878.25186.b7. [DOI] [PubMed] [Google Scholar]
- 20.Dyrbye LN, Thomas MR, Power DV, et al. Burnout and serious thoughts of dropping out of medical school: a multi-institutional study. Acad Med. 2010;85:94–102. doi: 10.1097/ACM.0b013e3181c46aad. [DOI] [PubMed] [Google Scholar]
- 21.Guerrasio J, Garrity MJ, Aagaard EM. Learner deficits and academic outcomes of medical students, residents, fellows, and attending physicians referred to a remediation program, 2006–2012. Acad Med. 2014;89:352–358. doi: 10.1097/ACM.0000000000000122. [DOI] [PubMed] [Google Scholar]
- 22.Stegers-Jager KM, Schotanus J, Splinter TAW, Themmen APN. Academic dismissal policy for medical students. Med Educ. 2011;45:987. doi: 10.1111/j.1365-2923.2011.04004.x. [DOI] [PubMed] [Google Scholar]
- 23.Durning SJ, Cohen DL, Cruess D, et al. Does student promotions committee appearance predict below-average performance during internship? A seven-year study. Teach Learn Med. 2008;20:267–272. doi: 10.1080/10401330802199609. [DOI] [PubMed] [Google Scholar]
- 24.Chou CL, Kalet A, Hauer KE. A research agenda for remediation in medical education. In: Kalet A, Chou CL, editors. Remediation in medical education: a mid-course correction. New York: Springer Hill; 2014. pp. 339–348. [Google Scholar]
- 25.Snadden D, Thomas ML, Griffin EM, et al. Portfolio-based learning and general practice vocational training. Med Educ. 1996;30:148–152. doi: 10.1111/j.1365-2923.1996.tb00733.x. [DOI] [PubMed] [Google Scholar]
- 26.Driessen E, van Tartwijk J, Vermunt J, et al. Use of portfolios in early undergraduate medical training. Med Teach. 2003;25:18–23. doi: 10.1080/0142159021000061378. [DOI] [PubMed] [Google Scholar]
- 27.Thomas DS, Snadden M. The use of portfolio learning in medical education. Med Teach. 1998;20:192–199. doi: 10.1080/01421599880904. [DOI] [PubMed] [Google Scholar]
- 28.Challis MAMEE. Medical education guide no. 11 (revised): portfolio-based learning and assessment in medical education. Med Teach. 1999;21:370–386. doi: 10.1080/01421599979310. [DOI] [Google Scholar]
- 29.Kalet A, Pusic M. Defining and assessing competence. In: Kalet A, Chou CL, editors. Remediation in medical education. New York: Springer; 2014. pp. 3–16. [Google Scholar]
- 30.Winston K. Core concepts in remediation: lessons learned from a 6-year case study. Med Sci Educ. 2015;25(3):307–315. doi: 10.1007/s40670-015-0149-z. [DOI] [Google Scholar]
- 31.Saks N, Rashid H, Lebeau RA. National Survey of US medical schools: current status of academic support. Seattle, WA: [PowerPoint presentation]. Presented at: AAMC Conference (Association of American Medical Colleges); 2016. [Google Scholar]
- 32.Miller CJ. Implementation of a study skills program for entering at-risk medical students. Adv Physiol Educ. 2014;38:229–234. doi: 10.1152/advan.00022.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Durning SJ, Cleary TJ, Sandars J, et al. Perspective: viewing “strugglers” through a different lens: how a self-regulated learning perspective can help medical educators with assessment and remediation. Acad Med. 2011;86:488–495. doi: 10.1097/ACM.0b013e31820dc384. [DOI] [PubMed] [Google Scholar]
- 34.Markert RJ. The relationship of academic measures in medical school to performance after graduation. Acad Med. 1993;68:S31–S34. doi: 10.1097/00001888-199302000-00027. [DOI] [PubMed] [Google Scholar]
- 35.Papadakis MA, Arnold GK, Blank LL, et al. Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards residency training performance and subsequent disciplinary action. Ann Intern Med. 2008;148:869–876. doi: 10.7326/0003-4819-148-11-200806030-00009. [DOI] [PubMed] [Google Scholar]
- 36.Greenburg DL, Durning SJ, Cohen DL, et al. Identifying medical students likely to exhibit poor professionalism and knowledge during internship. J Gen Intern Med. 2007;22:1711–1717. doi: 10.1007/s11606-007-0405-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Hodgson CS, Teherani A, Gough HG, et al. The relationship between measures of unprofessional behavior during medical school and indices on the California psychological inventory. Acad Med. 2007;82:S4–S7. doi: 10.1097/ACM.0b013e318140047f. [DOI] [PubMed] [Google Scholar]
- 38.Sayer M, Chaput De Saintonge M, Evans D, et al. Support for students with academic difficulties. Med Educ. 2002;36:643–650. doi: 10.1046/j.1365-2923.2002.01259.x. [DOI] [PubMed] [Google Scholar]
- 39.Hauer KE, Teherani A, Kerr KM, et al. Student performance problems in medical school clinical skills assessments. Acad Med. 2007;82(Suppl):S69–S72. doi: 10.1097/ACM.0b013e31814003e8. [DOI] [PubMed] [Google Scholar]
- 40.Cleland J, Leggett H, Sandars J, et al. The remediation challenge: theoretical and methodological insights from a systematic review. Med Educ. 2013;47:242–251. doi: 10.1111/medu.12052. [DOI] [PubMed] [Google Scholar]
- 41.Mcloughlin CS. Characteristics of students failing medical education: an essay of reflections. Med Educ Online. 2009;14:1–6. [Google Scholar]
- 42.Cleary TJ, Dong T, Artino AR. Examining shifts in medical students’ microanalytic motivation beliefs and regulatory processes during a diagnostic reasoning task. Adv health Scie Educ. Theory Pract. 2015;20:611–626. doi: 10.1007/s10459-014-9549-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Yellin PB. Learning Differences and medical education. In: Kalet A, Chou CL, editors. Remediation in medical education. New York: Springer; 2014. pp. 157–172. [Google Scholar]
- 44.Wilkerson L. Identification of skills for the problem-based tutor: student and faculty perspectives. Instr Sci. 1994;22:303–315. doi: 10.1007/BF00891783. [DOI] [Google Scholar]
- 45.Saha S, Guiton G, Wimmers PF, et al. Student body racial and ethnic composition and diversity-related outcomes in US medical schools. JAMA. 2008;300:1135–1145. doi: 10.1001/jama.300.10.1135. [DOI] [PubMed] [Google Scholar]
- 46.Elam CL, Johnson MM, Wiggs JS, et al. Diversity in medical school: perceptions of first-year students at four southeastern US medical schools. Acad Med. 2001;76:60–65. doi: 10.1097/00001888-200101000-00017. [DOI] [PubMed] [Google Scholar]
- 47.Terrell C, Beaudreau J. 3000 by 2000 and beyond: next steps for promoting diversity in the health professions. J Dent Educ. 2003;67:1048–1052. [PubMed] [Google Scholar]
- 48.Smedley BD, Stith AY, Colburn L, et al. In: the right thing to do, the smart thing to do: enhancing diversity in the health professions–summary of the symposium on diversity in health professions in honor of Herbert W. Nickens, MD. 2001. The right thing to do, the smart thing to do. Enhancing diversity in the health professions; pp. 1–35. [PubMed] [Google Scholar]
- 49.Smedley BD, Stith AY, Nelson AR. Patient-provider communication: the effect of race and ethnicity on process and outcomes of healthcare. In: Smedley BD, Stith AY, Nelson AR, editors. Institute of Medicine (US) committee on understanding and eliminating racial and ethnic disparities in health care. Unequal treatment: confronting racial and ethnic disparities in health care. Washington (DC): National Academies Press (US); 2003. pp. 1–28. [PubMed] [Google Scholar]
- 50.Brondolo E, Jean-Pierre KL. “You Said, I Heard”: speaking the subtext in interracial conversations. In: Kalet A, Chou CL, editors. Remediation in medical education. New York: Springer; 2014. pp. 131–154. [Google Scholar]
- 51.Cleland J, Arnold R, Chesser A. Failing finals is often a surprise for the student but not the teacher: identifying difficulties and supporting students with academic difficulties. Med Teach. 2005;27:504–508. doi: 10.1080/01421590500156269. [DOI] [PubMed] [Google Scholar]
- 52.Students MS. Tutors and relationships: the ingredients of a successful student support scheme. Med Educ. 2000;34:635–641. doi: 10.1046/j.1365-2923.2000.00541.x. [DOI] [PubMed] [Google Scholar]
- 53.Papadakis MA, Teherani A, Banach MA, et al. Disciplinary action by medical boards and prior behavior in medical school. NEJM. 2005;353:2673–2682. doi: 10.1056/NEJMsa052596. [DOI] [PubMed] [Google Scholar]
- 54.Challis AF, Gifford BM. An accident waiting to happen? A case for medical education. Med Teach. 1999;21:582–585. doi: 10.1080/01421599979310. [DOI] [PubMed] [Google Scholar]
- 55.Papadakis MA, Hodgson CS, Teherani A, et al. Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med. 2004;79:244–249. doi: 10.1097/00001888-200403000-00011. [DOI] [PubMed] [Google Scholar]
- 56.Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA. 2007;298:993–1001. doi: 10.1001/jama.298.9.993. [DOI] [PubMed] [Google Scholar]
- 57.Santen SA, Petrusa E, Gruppen LD. The relationship between promotions committees’ identification of problem medical students and subsequent state medical board actions. Adv health Scie Educ. Theory Pract. 2015;20:421–430. doi: 10.1007/s10459-014-9536-2. [DOI] [PubMed] [Google Scholar]
- 58.Larochelle J, Dong T, Durning S. Pre-clerkship assessment of clinical skills and clinical reasoning: the longitudinal impact on learner performance. Med Educ. 2015;180:43–46. doi: 10.7205/MILMED-D-14-00566. [DOI] [PubMed] [Google Scholar]
- 59.Hojat M, Mangione S, Nasca TJ, et al. An empirical study of decline in empathy in medical school. Med Educ. 2004;38:934–941. doi: 10.1111/j.1365-2929.2004.01911.x. [DOI] [PubMed] [Google Scholar]
- 60.Prober CG, Kolars JC. First LR, et al. a plea to reassess the role of United States medical licensing examination step 1 scores in residency selection. Acad Med. 2016;91:12–15. doi: 10.1097/ACM.0000000000000855. [DOI] [PubMed] [Google Scholar]
- 61.Chertoff J, Wright A, Novak M, Fantone J, Fleming A, Ahmed T, Green MM, Kalet A, Linsenmeyer M, Jacobs J, Dokter C. Status of portfolios in undergraduate medical education in the LCME accredited US medical school. Medical Teach. 2016;38:886–896. doi: 10.3109/0142159X.2015.1114595. [DOI] [PubMed] [Google Scholar]
- 62.Gómez SS, Ostos EM, Solano JM, Salado TF. An electronic portfolio for quantitative assessment of surgical skills in undergraduate medical education. BMC Med Educ. 2013;13:65. doi: 10.1186/1472-6920-13-65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Maddox RW, Schmid RJ. New frontiers in medical education simulation technology at Campbell University School of Osteopathic Medicine. N C Med J. 2014;75:59–61. [PubMed] [Google Scholar]
- 64.Van Tartwijk J, Driessen EW. Portfolios for assessment and learning: AMEE guide no. 45. Med Teach. 2009;31:790–801. doi: 10.1080/01421590903139201. [DOI] [PubMed] [Google Scholar]
- 65.Driessen E. Do portfolios have a future? Adv health Scie Educ. Theory Pract. 2017;22:221–228. doi: 10.1007/s10459-016-9679-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Cleary L. “Forward feeding” about Students’ progress: the case for longitudinal, progressive, and shared assessment of medical students. Acad Med. 2008;83:800. doi: 10.1097/ACM.0b013e318181cfbc. [DOI] [PubMed] [Google Scholar]
- 67.Cox SM. “Forward feeding” about Students’ progress: information on struggling medical students should not be shared among clerkship directors or with Students’ current teachers. Acad Med. 2008;83:801. doi: 10.1097/ACM.0b013e318181cfe6. [DOI] [PubMed] [Google Scholar]
- 68.Pangaro L. “Forward feeding” about Students’ progress: more information will enable better policy. Acad Med. 2008;83:802–803. doi: 10.1097/ACM.0b013e318181d025. [DOI] [PubMed] [Google Scholar]
- 69.Papish A, Kassam A, Modgill G, Vaz G, Zanussi L, Patten S. Reducing the stigma of mental illness in undergraduate medical education: a randomized controlled trial. BMC Med Educ. 2013;13:141. doi: 10.1186/1472-6920-13-141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Friedrich B, Evans-Lacko S, London J, Rhydderch D, Henderson C, Thornicroft G. Anti-stigma training for medical students: the education not discrimination project. Br J Psychiatry. 2013;202:s89–s94. doi: 10.1192/bjp.bp.112.114017. [DOI] [PubMed] [Google Scholar]
- 71.Dyrbye LN, Eacker A, Durning SJ, Brazeau C, Moutier C, Massie FS, Satele D, Sloan JA, Shanafelt TD. The impact of stigma and personal experiences on the help-seeking behaviors of medical students with burnout. Acad Med. 2015;90:961–969. doi: 10.1097/ACM.0000000000000655. [DOI] [PubMed] [Google Scholar]
- 72.Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. 2010;10:113. doi: 10.1186/1471-244X-10-113. [DOI] [PMC free article] [PubMed] [Google Scholar]