Table 5.
Studies reporting impact of FM clerkships on students according to Kirkpatrick outcome levels
| Kirkpatrick level 1 | |||||
|---|---|---|---|---|---|
| Authors (year) | Country | Clerkship features | Study methods | Key findings | Strength Grade |
| Bahn et al (2003)43 | USA | Y 3; 4 weeks; O; mixed setting | Patient encounter logs for clinical exposure and student involvement; (RR 87/105; 2591 encounters FM and 2527 IM); students report 30 patients for FM and 30 for IM; 7–8 patients/week not on same day | Similar exposures for 5/10 diagnoses in FM and IM; encounters students ‘observed only’ lower in FM (15%) vs IM (19%) p<0.001; students conducted PE more often in FM (77%) vs IM (73%) p<0.001 | 4 |
| Carney et al (2000)44 | USA | Y 3; 8 weeks; O; mixed setting | Patient encounter logs for cases encountered and level of observation/feedback by tutor (RR 63/63 students; 4083 encounters=3221 patients); students reported 1 full day/week | Exposure: acute care (39%), health maintenance visit (27%), chronic diseases (21%) and their acute exacerbations (13%); 63% performed Hx taking and 48% PE unobserved; in 49% of encounters students received no feedback | 3 |
| Carney et al (2002)45 | USA | Y 3; 8 weeks; O; mixed setting | Patient encounter logs for cases encountered and level of observation/feedback by tutor (RR 15 759 cards/?; 59% FM, 22% Ped, 12% IM); validity and reliability of the forms reported (κ coeff 0.68) | Students in FM had more continuity visits (18% of visits vs 11% each for IM and Ped); behaviour change counselling, clinical procedures as well as a better mixture of chronic and acute visits; in FM student did more Hx taking (61%) and PEs (47%) by themselves (unobserved); they received more feedback and teaching on diagnosis and management during IM clerkship | 3 |
| Cullen et al (2004)58 | Ireland | Y 5/6; ? weeks; O; mixed setting | Patient encounter logs for cases and involvement of two cohorts of students (RR 186/227; 3710 consultations); students reported 20 consecutive patients on day 5 of clerkship | In 53% of visits student observed the FP; in 12% took Hx; in 32% did PE; in 12% did a procedure/investigation; 78% of visits were with adults and 18% of them were elderly (≥66 years old) | 3 |
| Schamroth et al (1990)48 | UK | Y 4; 3 weeks; O; urban | Patient encounter forms and activity logs (RR 48/84) | 85% of time is spend observing passively; 69% of cases discussed with tutor; average 19 patients/day and median one home visit/day; highly rated (3/4) usefulness and stimulation effect of the FP tuition | 3 |
| Chenot et al (2009)52 | Germany | Y 5; 2 weeks; O | Pre–post clerkship questionnaires (mandatory and web based); 2 cohorts (RR 695/695) | Satisfaction with clerkship 8.1/10; contributions: recognition of frequent health problems (85%), communication (65%) and PE skills (61%); majority had home visits (95%); did supervised PE (94%) and Hx taking (89%) | 3 |
| Cooper (1992)61 | Australia | Y 4/5; 2 weeks; mixed setting | Post-clerkship questionnaires; retrospective analysis of 2 cohorts (RR 386/398) | Satisfaction: 68.6% excellent/very good; contributions: variety of problems encountered (39.2%), experience in managing common problems (33.5%); performing practical procedures (24.8%); qualities of FM teaching: willing to answer question (46%), set aside time to discuss (32%), enthusiasm, welcoming and friendly (52.1%) | 3 |
| Foldevi (1995)34 | Sweden | Y 4 and 5; 5 weeks; O; mixed setting | Post-clerkship questionnaires (RR 85/115); factor analysis of questionnaires items reporting good construct validity | Satisfaction: overall rating 79±23/100; quality of tutoring 79±18/100; feedback: 45±14/100; student responsibility: 47±11/100 | 4 |
| Iqbal (2010)59 | Pakistan | Y 3; 2 weeks; O | Pre–post clerkship questionnaires (RR 46/46) | Most important things learned: confidence to deal with common health problems, empathy and communications skills | 3 |
| Kalantan et al (2003)36 | Saudi Arabia | Y 5 and 6; 6 weeks; O; urban | Pre–post clerkship questionnaires (RR 177/177) | Best things: friendly welcoming attitude of FP and staff (92%), gaining experience in managing common clinical problems (87.6%) and insight in FP life (86.4%); 59.3% expected more from the FM clerkship in regard to practical procedures, involvement in consultation and time for discussion; quality of FM teaching: willing to answer questions (82%), set aside time to discuss (56.5%), encouraged me to ask questions (61%), friendly and welcoming (93.2%) | 4 |
| Kavukcu et al (2012)53 | Germany | Y 6; ? weeks; O | Post-clerkship DREEM questionnaires to FM and Sports medicine clerkships in a primary care centre (RR 55/55 each) | DREEM score for FM 139.45/200 and sports medicine 140.05/200 p<0.05; overall score 140/200 of the out-of-hospital educational environment | 3 |
| Rabinowitz (1992)38 | USA | Y3; 6 weeks; O; mixed setting | Post-clerkship questionnaires; 3 cohorts (RR 850/?) | FM highest rated clerkship among all required clerkships (no numbers reported) | 2 |
| Morrison and Murray (1996)49 | UK | Y final; 4 weeks; O | Pre–post clerkship questionnaire (RR 131/206) | 4% (pre) to 47% (post) had FM as their 3 most enjoyed subjects | 4 |
| Svab and Petek-Ster (2008)40 | Slovenia | Y final; 8 weeks; O | Pre–post clerkship questionnaires; 2 cohorts 10 years apart (RR 127/172 pre and 123/140 post) | Satisfaction: 8.73±0.93/10 at first cohort and 9.04±0.93/10 at second cohort; p=0.035 | 4 |
| Vinson and Paden (1994)46 | USA | Y3/4; 4 weeks; O; mixed setting | Post-clerkship questionnaires (RR 43/46) | Quality of feedback: good/excellent in 14/15 practices that did not consider clerkship as recruiting tool; in the rest (31) practices quality of feedback: fair/poor | 3 |
| Sprenger et al (2010)30 | Austria | Y 6; 5 weeks; O | Post-clerkship questionnaires (RR 146/146) | 87% ‘strongly agree’ and 13% ‘agree’: clerkship was overall positive experience; 79% ‘ideal supervision’, 82%: tutor’s expertise excellent | 4 |
| McKee et al (1998)28 | USA | Y 3; 6 weeks; O; urban | Daily activity logs and quality scores; students and preceptors at community health centres; (RR 14/16; 232 sessions) | Quality of learning: 63/100; not correlated to clinical productivity of preceptors; students saw independently 2.52±1.71 out of 4.45±3.34 pts/session and received feedback 2.44±2.76 times/session; students quality rate higher (63) than preceptors (54) p=0.003, but only 62/232 sessions were matched student-preceptor | 3 |
| Lloyd and Rosenthal (1992)50 | UK | Y 4; 4 weeks; O; urban | Pre–post clerkship questionnaires (RR 70/95) | Scores of achievement areas (post) < scores of expectations (pre); psychological and social aspects of disease, communication skills, clinical decision-making skills and management plans had higher achievement scores (although < expectations); PE skills, taking blood and performing a PAP smear had lower scores; 57% report gaining insight in the FP's work and life and knowledge content of FM as main contributions | 4 |
| Senf and Campos-Outcalt (1995)47 | USA | Y3; 6 weeks; O; mixed setting | Pre–post questionnaires; 10 cohorts (RR 997/1095); post-clerkship evaluation | 54.1%: FM clerkship ‘somewhat’ or ‘a lot’ better than previous clerkships | 4 |
| Sprenger et al (2008)54 | Austria | Y 6; 5 weeks; O | Post-clerkship questionnaire (RR 30?/?) | Very positively rated by students (visual scale shown, but rating not clear) | 2 |
| Svab (1998)57 | Slovenia | Y 6; 7 weeks; O | Post-clerkship questionnaires (RR 135/175) | 73%: favourable score on cooperation with tutor; highest score for learning on record keeping, referrals and prescribing | 3 |
| Peleg et al (2005)55 | Israel | Y 5; 6 weeks; O | Post-clerkship questionnaires; 2 cohorts (RR 186/186 and 176/186) | Mean evaluation and satisfaction score: 3.4/4; ranked high among other clerkships (no numbers reported) | 3 |
| Mash and de Villiers (1999)60 | South Africa | Y final; 2 weeks; O | Post-clerkship questionnaire and focus group (RR 108/121) | 7.8/10 ‘useful and relevant’; 59% of the material covered: new/not duplicate of previous teaching; focus group themes: patient-centeredness and continuity of care, management of common and undifferentiated problems, holistic assessments, communication skills, primary care team | 3 |
| Dahan et al (2001)56 | Israel | Y 6; 5 weeks | Post-clerkship questionnaire and focus group; 2 cohorts (RR 49/80 and 52/80); 2 years before and after organisation and content change of clerkship | Satisfaction score improved from 85 to 97/100 | 3 |
| Mattsson et al (1991)51 | UK | Y final; 2 weeks; O | Post-clerkship interviews (RR 20/20); 10 with higher and 10 lower grades and tutors’ comments | Appreciated contributions: focus on communications skills, whole person care and continuity of care | 3 |
| Snaddena and Yaphe (1996)39 | UK | Y 4; 4 weeks; O | Post-clerkship questionnaire, interviews and focus group (RR 75/75) | Overall experience: 4.78/5; wide range of clinical experiences, home visits, preventive medicine, referrals, learning communication skills, seeing the patient as a person not as a disease, insight in organisation of FM centre and staff; friendly atmosphere; good level of tutoring (students want more seeing of patients alone then observing) | 3 |
| Kirkpatrick level 2A | |||||
| Dixon et al (2000)33 | Hong Kong | Y final; 2 weeks; O | 15 post-clerkship focus groups (RR 110/110) | Previous negative stereotypes of FPs (easy and boring job and making lots of money) changed into understanding that FM is not boring and has its own diagnostic challenges | 3 |
| Iqbal (2010)59 | Pakistan | Y 3; 2 weeks; O | Pre–post clerkship questionnaires (RR 46/46) | Increase in those interested in future FM career (7% pre to 37% post); those ‘not sure’ reduced (69% pre to 43% post); those already decided for ‘no' (24% pre to 20% post) | 3 |
| Kruschinski et al (2011)63 | Germany | Y 5; 3 weeks; O | Pre–post clerkship questionnaires (RR pre 287/423 and post 165/287) | Post-clerkship more positive attitudes toward FM as a discipline; no significant change in future career plans; gender more influential on future career choices than attitudes | 4 |
| Lloyd and Rosenthal (1992)50 | UK | Y 4; 4 weeks; O; urban | Pre–post clerkship questionnaires (RR 70/95) | ∼65%: clerkship changed their attitudes toward FM: 48% in favour, 14% against and 40% neutral; 37%: clerkship had influence on career intentions: 63% in favour; 25% against and 12% neutral | 4 |
| Maiorova et al (2008)64 | The Netherlands | Y 5/6; 12 weeks; O | Pre–post clerkship questionnaires in three clerkship: FM (RR 168/206), internal medicine (RR 247/347), surgery (RR 178/378) | Increased perceived likelihood of choosing a specialty after the clerkship: FM (29%), IM (30%) and surgery (31%); majority had no change (63%, 49%, 59% respectively) | 3 |
| Morrison and Murray (1996)49 | UK | Y final;4 weeks; O | Pre–post clerkship questionnaires; postal questionnaire 1 year after graduation (16–26 months after clerkship) (RR 131/206) | % of students likely to choose FM career: 38.8% pre to 53.5% post clerkship; those unlikely: 18.6–13.2%; 1 year after graduation only 34.9% likely and 24.8% unlikely to choose FM | 4 |
| Musham and Chessman (1994)65 | USA | Y 3; 4 weeks; O | Post-clerkship focus groups (RR 122/122) | Negative pre-clerkship stereotype ‘FM=low status and intellectually unchallenging’ changed to ‘FM intellectually challenging and not inferior to other specialties’; increased interest in FM career for those who had not decided yet | 3 |
| Paulman and Davidson-Stroh (1993)32 | USA | Y 4; 8 weeks; O; rural | Pre–post clerkship questionnaires on specialty preferences and data on final specialty selection; 4 cohorts of students (RR 598/598) | No change of career interests: 78.1% (other specialties) and 16.4% (FM); 3.8%:positive shift of interest toward FM; 1.7%: negative shift | 4 |
| Sadikoglu et al (2006)66 | Turkey | Y final; 4 weeks; O | Pre–post clerkship questionnaires on specialty choices (RR 90/93) | Statistically significant increase in ranking of FM as a career choice: 4.19±0.10 pre to 3.88±0.10 post; pre–post change in attitude toward FM as a career: not significant | 3 |
| Senf and Campos-Outcalt (1995)47 | USA | Y 3; 6 weeks; O; mixed setting | Pre–post clerkship questionnaire on attitudes and specialty preferences, and data on final specialty selection; 10 cohorts of students (RR 997/1095) | Unchanged specialty preferences: 66% (other specialties) and 18% (FM); 4%: negative change of preferences; 12%: positive; 8% net increase of interest in FM | 4 |
| Svab and Petek-Ster (2008)40 | Slovenia | Y final; 8 weeks; O | Pre–post clerkship questionnaires of two cohorts between 10 academic years (RR 127/172 pre and 129/140 post) | Statistically significant positive changes in scores of attitudinal statements on role and importance of FM; no stat significant increase in preferences for FM careers pre–post clerkship and between 10 years | 4 |
| Tai-Pong (1997)67 | Hong Kong | Y 4/5; 2 weeks; O | Post-clerkship questionnaires and 1 year after graduation (18–26 months after clerkship) (RR 88/138) | At 18–26 months: 54% ‘clerkship had positively changed their attitudes towards FM’; 27% ‘it had positively changed their decision to pursue a FM career’; 10% had negative change | 3 |
| Kirkpatrick level 2B | |||||
| Beasley et al (1992)71 | USA | Y 3; 2–3 months; E | National board of medical examiners (NBME) part 2 examination scores of 95 students who took FM clerkship and two control groups (similar NBME 1 scores) who did not take clerkship | No statistically significant difference in scores of medicine and surgery parts of examination; those with FM clerkship significantly higher scores only in public health items | 4 |
| Gjerde et al (1997)72 | USA | Y 3; 2 weeks; O; mixed setting | Students’ self-report on involvement during the clerkship (checklist of skills, diagnoses and procedures); 3 cohorts (RR 486/486) | Actively performing well-baby examination (72%), managing upper respiratory infections (85%), acute otitis media (81%), sinusitis (70%) and sore throat (70%), performing breast (64%), pelvic and PAP smear (59%), prostate (58%) examinations and laceration suturing (52%) | 3 |
| Gjerde et al (1998)35 | USA | Y 3; 2–3 weeks; O; mixed setting | Pre–post clerkship students’ self-report on involvement during clerkship (checklist of skills, diagnoses and procedures) (RR 87/87) | >50% actively performed/managed only after the FM clerkship: preventive skills (5/10 skills), acute sprain/strain, low back pain, sinusitis, strep throat, acute bronchitis and osteoarthritis (6/31 diagnoses), removal of foreign body from eye, incision and drainage of external haemorrhoids’ thrombosis and infant circumcision (3/39 procedures) | 4 |
| Jacques (1997)73 | USA | Y 3; 4 weeks; O | Written examinations (MCQs) scores between two schools with different clerkship schedules (RR school A 232/232; school B 188/188) | Increase of scores after clerkship: school A 63.4% pre—82.6% post=19%; school B 2 65.5% pre-80.5% post=15%; no significant difference between schools with different system of clerkships’ scheduling | 3 |
| Maple et al (1998)37 | USA | Y 3; 4 weeks; O; mixed setting | Pre–post clerkship self-assessment of students (RR 349/521) | Gain in knowledge and skills for 25/26 core medical conditions if FM clerkship before and 16/26 if after IM, ob-gyn and psychiatric clerkships | 4 |
| O’Hara et al (2000)74 | USA | Y 3; 4 weeks; O; mixed setting | Patient encounter logs with students’ perceived competence/confidence in dealing with 10 most frequent ENT diagnoses (RR 445/445?) | Higher than average levels of students’ perceived competence/confidence in dealing with the 10 most frequent ENT diagnoses (no numbers reported) | 3 |
| O’Hara et al (2001)75 | USA | Y 3; 4 weeks; O; mixed setting | Patient encounter logs with students’ perceived competence/confidence in dealing with 10 most frequent psychiatric diagnoses (RR 445/445?) | Higher than average levels of students’ perceived competence/confidence in dealing with the 10 most frequent psychiatric diagnosis (‘competent’: 52.1% vs 53.3% for total diagnoses encountered in clerkship; ‘confident/skilled’: 18.2% vs 19.1%; p<0.001) | 3 |
| O’Hara et al (2002)76 | USA | Y 3; 4 weeks; O; mixed setting | Patient encounter logs with students’ perceived competence/confidence in dealing with 10 most frequent ob-gyn diagnoses (RR 445/445?) | Lower than average levels of students’ perceived competence/confidence in dealing with 10 most frequent ob-gyn diagnoses (‘competent’: 49.6% vs 53.3% for total diagnoses encountered in clerkship; ‘confident/skilled’: 18.8% vs 19.2%; p<0.001) | 3 |
| Saywell et al (2002)77 | USA | Y 3; 4 weeks; O; mixed setting | Patient encounter logs with students’ perceived competence/confidence in dealing with 10 most frequent muscular-skeletal diagnoses (RR 445/445?) | Lower than average levels of students’ perceived competence/confidence in dealing with 10 most frequent muscular-skeletal diagnoses (‘competent’: 49.5% vs 53.3% for total diagnoses encountered in clerkship; ‘confident/skilled’: 15.8% vs 19.1%; p<0.001) | 3 |
| Schwiebert and Davis (1995)78 | USA | Y 3; 4 weeks; O; mixed setting | Pre–post clerkship self-assessment of students’ confidence for a list of skills; 4 cohorts (RR 358/358) | Mean change in students’ confidence significant (p<0.001); highest change for risk-oriented Hx taking (1.80); applying sensitivity/specificity (1.57); performing cerumen removal (1.44); geriatric evaluation and assessment (1.43); performing a focused Hx taking and PE (1.41); obtaining basic family information (1.40) | 4 |
| Sprenger et al (2008)54 | Austria | Y 6; 5 weeks; O | Post-clerkship self assessment of students (RR 30/30?) | Figure reporting level of competence of 30 students for a list of practical skills that they need to do themselves, but results not very clear | 2 |
| Svab (1998)57 | Slovenia | Y final; 7 weeks; O | Post-clerkship self assessment of students and tutors’ assessment; 2 cohorts (RR 135/175) | Students highest rate for knowledge on referral process (4.47/5), record keeping (4.47/5) and prescribing (4.45/5); tutors highest rate for students’ performance in communication (4.82/5) and cooperation with the team (4.84/5) | 3 |
| Townsend et al (2001)41 | UAE | Y 6; 10 weeks; O | Pre–post clerkship OSCE scores (RR 28/28?) | Improvement of scores: mean score 57.3/100 pre to 82.8/100 post-clerkship; consistent throughout the year and highest for stations on prescription writing, dealing with ethical problems and problem solving | 4 |
| Kirkpatrick level 3 | |||||
| Campos-Outcalt and Senf (1999)68 | USA | Y 3; varied duration; O; mixed setting | National data on FM specialty selection of graduates from schools with and without FM clerkship (RR 108/121 schools) | Mean change of % graduates entering FM specialty training in schools with FM clerkship (3 year pre and post start of clerkship) and schools without was 2.29, 95% CI 1.01 to 3.58, p=0.01 | 4 |
| Kassebaum and Haynes (1992)69 | USA | Y 3; 4 weeks; O; mixed setting | National data on graduation questionnaire and specialty selections and graduates entering FM specialty training for schools with and without FM clerkship (RR 57 with and 64 without/126) | % graduates planning FM specialty training (15.6) and certification (15.5) and starting FM specialty training (14.7%) for schools with required FM clerkship vs schools without (6.9%, 7.0%, 7.2% respectively) | 3 |
| Levy et al (2001)42 | USA | Y 3; 3 weeks; O | Data from matriculation and graduation questionnaire and final specialty selection for five cohorts of students (RR 913/969) | Rating the FM clerkship's value as ‘high’/‘very high’ increased odds to enter FM specialty training even after adjusting for socio-demographics and personal preferences (OR 2.9, 95% CI 1.1 to 7.3, p=0.024) | 5 |
| Paulman and Davidson-Stroh (1993)32 | USA | Y 4; 8 weeks; O; rural | Pre–post clerkship questionnaires on specialty preferences and data on final specialty selection; 4 cohorts of students (RR 598/598) | Only 33 (5.5%) changed specialty preference post-clerkship: 23 (3.8%) positive change toward FM and 10 (1.7%) negative change (p<0.01); 15 (65%) of those with positive change entered FM specialty training | 4 |
| Senf and Campos-Outcalt (1995)47 | USA | Y 3; 6 weeks; O; mixed setting | Pre–post clerkship questionnaire on attitudes and specialty preferences and data on final specialty selection; 10 cohorts (RR 997/1095) | Only 1/4 of those who had a positive change toward FM specialty at end of clerkship entered FM specialty training | 4 |
| Stine et al (1992)70 | USA | Y 3/4; mixed setting; O/E; varied duration | Questionnaire for medical schools and national data on specialty selection on percentage of graduates entering FM specialty training in schools with and without FM clerkship (RR 104/126 schools) | 74% of schools in highest quartile of % graduates entering FM specialty training (≥17%) had a required FM clerkship vs 25% of schools in lowest quartile (≤7.7%) p=0.0013; association not stat. significant for elective clerkship | 3 |
?, No data available in the paper; Coeff, coefficient; DREEM, Dundee Ready Educational Environment Measure; E, elective; ENT, ear-nose-throat; FM, family medicine/general practice; FP, family/general practitioner; Hx taking, history taking; IM, internal medicine; MCQ, multiple choice questions; O, obligatory; ob-gyn, obstetrics-gynaecology; OSCE, objective structured clinical examination; PE, physical examination; Ped, paediatrics; RR, response rate; Y, year.