Early practical experience and the social responsiveness of clinical education: systematic review

Abstract

Objectives To find how early experience in clinical and community settings (“early experience”) affects medical education, and identify strengths and limitations of the available evidence.

Design A systematic review rating, by consensus, the strength and importance of outcomes reported in the decade 1992-2001.

Data sources Bibliographical databases and journals were searched for publications on the topic, reviewed under the auspices of the recently formed Best Evidence Medical Education (BEME) collaboration.

Selection of studies All empirical studies (verifiable, observational data) were included, whatever their design, method, or language of publication.

Results Early experience was most commonly provided in community settings, aiming to recruit primary care practitioners for underserved populations. It increased the popularity of primary care residencies, albeit among self selected students. It fostered self awareness and empathic attitudes towards ill people, boosted students' confidence, motivated them, gave them satisfaction, and helped them develop a professional identity. By helping develop interpersonal skills, it made entering clerkships a less stressful experience. Early experience helped students learn about professional roles and responsibilities, healthcare systems, and health needs of a population. It made biomedical, behavioural, and social sciences more relevant and easier to learn. It motivated and rewarded teachers and patients and enriched curriculums. In some countries, junior students provided preventive health care directly to underserved populations.

Conclusion Early experience helps medical students learn, helps them develop appropriate attitudes towards their studies and future practice, and orientates medical curriculums towards society's needs. Experimental evidence of its benefit is unlikely to be forthcoming and yet more medical schools are likely to provide it. Effort could usefully be concentrated on evaluating the methods and outcomes of early experience provided within non-experimental research designs, and using that evaluation to improve the quality of curriculums.

Introduction

The norm for the past century has been for medical students to learn theory for two to three years before seeing it applied in practice. Encouraged by professional bodies such as the UK General Medical Council, many medical schools are “vertically integrating” various types of practical experience into the early years,^1-3 yet exponents of vertical integration have not clearly argued the case for it, let alone presented evidence in its favour. We developed a theory, grounded in consensus, that early practical experience (“early experience”) could orient medical curriculums towards the social context of practice, and strengthen students' affective and cognitive learning.⁴ We have now extended this research by systematically reviewing publications on the topic⁵ under the auspices of the recently formed Best Evidence Medical Education (BEME) collaboration.⁶ This paper summarises the review and interprets its main findings in relation to contemporary trends in medical education.

Methods

Methods are reported in full elsewhere.⁵ We have not completed the QUOROM statement since this is not a systematic review of clinical trials. The topic review group (all of whose members are authors) was recruited to be international, experienced in innovative clinical education, representative of community and hospital perspectives, familiar with horizontal and vertical integration, and conversant with evidence based practice. A medical student was a lead member.

Review question, definitions, and inclusion criteria

The review question was: “How can experience in clinical and community settings contribute to early medical education?” Early was defined as “what would traditionally have been regarded as the preclinical phase; usually the first two years” and experience as “authentic (real as opposed to simulated) human contact in a social or clinical context that enhances learning of health, illness and/or disease, and the role of the health professional.” All empirical studies (verifiable, observational data) were included, whatever their design, method, or language of publication. Although the search was directed towards medical education, evidence from other health professions was not excluded if it could be applied to medicine.

Box 1: Sources of references

Bibliographical databases screened electronically

British Education Index (BEI)

Education Resources Information Center (ERIC)

Medline

CINAHL

Embase

PsychINFO

TimeLIT

EBM reviews

SIGLE

Cochrane databases

Journals hand screened

Academic Medicine

Advances in Health Sciences Education

Journal of Educational Psychology

Medical Education

Medical Teacher

Teaching and Learning in Medicine

Search strategy and reference handling

The review covered the decade 1992-2001. After a scoping search, the research group ran a main search across seven major bibliographical databases (box 1), searched three other databases by keywords, hand-searched six journals, and screened the reference lists of all informative articles. We used EndNote bibliographical software (Thomson, Philadelphia, USA) to handle the citations, of which there were 6981 after elimination of duplicates.

Article selection, coding of outcomes, and synthesis of results

All articles presenting empirical evidence on the effect of experience on early education in the health professions, according to the definitions above, went to two members of a coding pair, who independently identified all outcomes and coded them for their strength (on the 1-5 scale described in box 2) and educational importance (using Kirkpatrick's hierarchy for educational outcomes,⁷ which is also included in box 2). The researchers resolved disagreements by consensus and entered the outcomes and their metadata into a database. The review yielded 277 educational outcomes from 73 studies. This report is based on the 116 of those outcomes from 38 studies, which the review group judged admissible as evidence because their strength was rated 3 or higher and their Kirkpatrick level was 2a or higher. The group divided the studies into positive outcomes (evidence of benefit), negative outcomes (no benefit or harm) and adverse outcomes, and coded them as coming from a descriptive or comparative study. The theory previously developed by consensus⁴ provided an interpretive structure. One author then re-read the original papers and wrote this narrative; the other five authors did a critique of it.

Results

Nature, strength, and importance of the evidence

Sixty nine per cent of studies were from North America, 23% from Europe, and 8% from other parts of the world. Ninety three per cent were in medicine and the remainder in pharmacy (6%) or nursing (1%). Seventy two per cent of interventions were in primary care, family medicine, or community settings; and 28% in hospital, hospice, or medical school. Sixty nine per cent were clinical placements, ranging from a single half day session to half day clinical visits through both preclinical years; 8% were skills training events; 7% were placements in “lay” community settings; 7% were attachments to a single patient or family; and 9% were combinations. The number of learners ranged from six to 1081. Most evidence was from descriptive studies (72% descriptive v 28% comparative) with evidence of positive publication bias (91% of outcomes positive), more so in descriptive studies (99% of outcomes positive) than comparative studies (70% of outcomes positive).

Effect on career choice

Recruitment of doctors to underserved, rural communities is a major public health issue in the United States, whose publications dominated the evidence base. We found strong evidence from comparative studies with long follow-up that students who had early experience were more likely than controls to choose residencies in primary care,^8-12 and show positive attitudes towards rural practice.^13-16 However, the students in those studies often chose to have early experience and may have had more positive attitudes from the outset. Moreover, their early experience was in rural communities, usually as part of a recruitment drive.¹⁷

Effect on students' learning

Early experience influenced students in several ways.

Attitudes

Attitudes are notoriously hard to quantify, and it is hard to compare without quantifying, so only 16% of attitudinal outcomes were from comparative studies. However, descriptive methods are well suited to exploring students' complex emotional reactions and the factors that trigger them. Early experience motivated students in numerous ways.^16-21 It reminded them of their vocation to be a doctor. It showed the practical relevance of the theory they were studying and made it easier to learn. It motivated them by giving welcome respite from the discipline of preclinical studies and exposing them to patients they could empathise with and doctors they could strive to emulate. It also made students more confident to meet and interview patients.¹³,¹⁵,^19-25 Students who chose to have early experience were more satisfied with their studies²⁶ because it gave them insight into the social and psychological problems of “real people.”²³ It helped build self awareness, including the ability to recognise and respond to feelings of uncertainty and inadequacy.¹⁸ Early experience helped students develop empathic responses to ill patients.¹⁸,²⁷ Some of the evidence concerned students' socialisation to their role as a clinical learner and future health professional. Two comparative studies found no association between early experience and the outcomes examined,²²,²⁶ although six descriptive studies found positive effects,¹⁴,¹⁵,¹⁷,¹⁹,²⁷,²⁸ including becoming more mature, acclimatising to professional settings, and identifying with doctor role models. Looking back on their education, graduates felt that early experience had reduced the stress they experienced when they first met patients during clerkships.²⁷

Understanding of subject matter

These findings were almost all positive and from descriptive studies, reporting qualitative changes in intellectual perspectives and patterns of thought. Early experience made students more confident in their knowledge, taught them things “that could not be learned from books,”¹⁴,²⁰,²⁴,²⁵,²⁷ brought diseases to life, and made medical science more comprehensible.¹³,¹⁷,²⁰,²⁹ It provided a framework to explain clinical practice, showed students how professionals viewed their interactions with patients, and helped develop “clinical ways of thinking.”¹⁷ It also taught students how people live, how their living conditions affect their health, and how important it is that health services are readily accessible to them.¹⁴,³⁰,³¹ It helped them appreciate the impact of illness¹⁸,²⁸,³⁰ and strengthened their knowledge of healthcare delivery systems,¹⁴,¹⁶,³² health professional roles and responsibilities, and the importance of good interprofessional communication and multidisciplinary working.¹⁸,¹⁹,³⁰,³¹ It helped students learn biomedical sciences,¹⁹ behavioural and social sciences, and the ethical dimension of patient care.¹⁸,²³

Clinical skills

Again, much of the research explored the quality rather than quantity of educational benefit, though nearly half the outcomes came from comparative studies. Early experience improved students' ability to relate to patients and communicate empathy.²⁰,²² It helped them understand the doctor-patient relationship and the importance of listening to patients, carers, and other professionals.¹⁹,³⁰ Students valued exploring social and psychological determinants of health and disease through contact with real patients²³ and were able to deliver preventive healthcare.³³ They appreciated learning to take diagnostic histories¹³,²²,²³,²⁸,^34-36 and performing simple physical examinations.¹³,²⁴,^35-37 Equipped with those skills, they felt better able to approach patients.²⁷

Study skills

Qualitative analysis of learning logs showed early experience to uncover differences in students' ability to learn reflectively.³⁸

Performance in examinations

It was concluded from several comparative studies that early experience improved performance in examinations, but the data were often sketchy, the study methods weak, the effect sizes small, and the benefits inconsistent.¹⁵,²⁰,^39-42 Experience in hospital had comparable effects to experience in community.⁴³

Effects on teachers, organisations, populations, and patients

Descriptive studies reported that students were not the only beneficiaries of early experience, which could motivate and reward teachers⁴⁴ and patients in hospital,⁴⁵ increase the breadth of placements that organisations provided,¹³ and bring health care to underserved populations.³⁷

Discussion

As part of a complex curriculum intervention,⁴⁶ early experience helped recruit residents to rural primary care in the US. Many countries need urgently to recruit health professionals to deliver primary care to underserved populations. The nature of the research, however, makes it unsafe to conclude that the benefit is restricted to rural practice, or even primary care. The students who opted for rural primary care had opted to have early experience, which was often provided within rural primary care. Early experience in an urban setting or secondary or tertiary career might have led to very different career choices. It is reasonable to conclude, however, that early experience has a strong formative influence that can be used to foster a socially responsive career orientation. In line with our previous consensus survey, biomedical science had only a small place in the rationale for early experience.⁴ One of us has argued elsewhere that “the new medical education” is not forsaking biomedical science but putting it in a broader, social context.⁴⁷ We view the move to offer early experience as a sign of that paradigm shift because it adds professional socialisation, the development of appropriate attitudes, interpersonal skills and study skills, and familiarity with the healthcare system, to the benefits of a grounding in basic science. In making the curriculum more socially responsive, it aligns medical education with contemporary concepts of professionalism, which emphasise doctors' account-ability to the society they serve.⁴⁸,⁴⁹

Box 2: Best Evidence Medical Education (BEME) coding scheme for strength and importance of evidence as applied to this review

Strength

It would be possible to have a strong study with low impact, and vice versa. Strength equates with critical appraisal and is a statement of your confidence that the results of the study are credible. Having considered the study design, the way the study was performed, and the data analysis, we rated the outcome:

1. No clear conclusions can be drawn; not strong

2. Results ambiguous; there seems to be a trend

3. Conclusions can probably be based on the results

4. Results are clear and very likely to be true

5. Results are unequivocal

Importance

Level 1: Participation—covers learners' views on the learning experience, its organisation, presentation, content, teaching methods, and aspects of the instructional organisation, materials, and quality of instruction

Level 2a: Modification of attitudes or perceptions—outcomes here relate to changes in the reciprocal attitudes or perceptions between participant groups towards intervention or simulation

Level 2b: Modification of knowledge and skills—for knowledge, this relates to the acquisition of concepts, procedures, and principles; for skills this relates to the acquisition of thinking and problem solving, psychomotor and social skills

Level 3: Behavioural change—documents the transfer of learning to the workplace or willingness of learners to apply new knowledge and skills

Level 4a: Change in organisational practice—wider changes in the organisation or delivery of care, attributable to an educational programme

Level 4b: Benefits to patient or clients—any improvement in the health and wellbeing of patients and clients as a direct result of an educational programme

Note: The term importance is used to describe the Kirkpatrick hierarchy of educational evidence is the authors', not the BEME collaboration's.

What is already known on this subject

Integrating various types of practical experience into the early years of clinical education is becoming increasingly common

This practice is strongly advocated by the UK General Medical Council, but theoretical arguments and empirical support for it are fragmentary

What this study adds

A systematic review of research evidence published in 1992-2001 provides an inventory of educational outcomes that can be enhanced by early experience

Evidence shows early experience has a strong formative influence that can be used to foster a socially responsive career orientation.

The review gives pointers for future research effort

Strengths and limitations of the study

This review is one of the early products of the BEME collaboration, which seeks to bring medical education into the “evidence era.”⁶ Educational interventions are often complex and educational outcomes can be hard to measure, so education systematic review has to accommodate different research methodologies. Nowhere is Einstein's epithet that “not everything that can be counted counts, and not everything that counts can be counted” truer than of educational outcomes. The evidence reviewed here includes comparative and descriptive studies, and qualitative and quantitative methods, weak and strong. Some of the studies that yielded the most robust causal inferences were qualitative and non-comparative. The review group has striven to present this very varied evidence base in a way that will help curriculum planners draw their own conclusions. The strength of this review is in the rigour of its methods, which evaluated complex evidence by consensus between a heterogeneous, multinational group of reviewers. They entered the review with open minds and communicated solely through the internet, keeping decision making relatively objective. The limitations are the lack of rigour in many of the studies reviewed and the inherent subjectivity of exploratory research. A predominance of underpowered and poorly designed studies, coupled with positive publication bias, would tend to exaggerate the effect of early experience. However, qualitative analysis of the benefits of a complex intervention should only bias the opinions of uncritical readers, while informing critical ones. Any readers wishing to review the evidence for themselves can trace each conclusion back to its primary source from a summary table in the detailed report.⁵

These findings do not prove, but are consistent with, the theory reported in our previous publication that early experience could strengthen and deepen cognitively, broaden affectively, contextualise, and integrate medical education.⁴ They also support the view that avoiding an abrupt transition into the clinical environment would give students an easier passage through medical education.⁴