. 2020 Jul 9;20:215. doi: 10.1186/s12909-020-02074-9

Table 1.

Summary of characteristics of eligible studies

First Author and year	Number & Struggling learner Characteristic & Country	MERSQI Score	Description of Assessment for identifying struggling learner	Intervention	Outcomes (Kirkpatrick level of evaluation)
Bodenberg 2015 ⁽⁴⁴⁾	8 Pharmacy Student US	8	• Midpoint evaluation and concern cards to alert the Director of Experiential Education of potential behavioral or learning issues. • The faculty developed a section in the student evaluation form in which the preceptor can suggest longitudinal monitoring or remediation needed for the student. • 9% had a communications skills deficiency.	The Director of Experiential Education creates a performance improvement plan. This individualized plan takes into account input from the student, preceptor, and the Director of Experiential Education. A list of communication tools includes: • Direct observations of the student during case presentations, • Counseling, • Drug information, • Topic presentations, and • An oral “End of Block” exam.	The on-time graduation using the developed remediation plan was seven of eight students (88%), and the overall graduation success rate was eight of eight students (100%). *(Level 2)*
Chang 2008 ⁽⁴⁰⁾	23 Medical Student US	5.5	The clinical performance examination (CPX) is an eight-station, high-stakes examination in which SPs assess early Year 4 medical students’ competence in clinical and communication skills. Students scoring ≥2 standard deviations (SD) below the class average in one or both skill domains (history taking and physical examination [H & P] or patient–doctor interaction [PDI]) are required to remediate.	Developed a four-step CPX remedial training program that incorporates diagnosis of learner problems, individual reflection, faculty feedback and supervised practice. Step 1 consists of an individual review in which: • Students watch their own videotaped examinations, • Reflect on their performances, and • Develop personal learning goals. Step 2, the faculty remediation director: • Watches the videotape independently and • Generates a ‘learning prescription’, Step 3 involves a one-to-one video review guided by the learning prescription and the student’s self-identified goals with a designated faculty member. In Step 4, students attend one or both evening workshops in H & P or PDI, depending on their area of need. Workshops consist of: an introductory 30-min didactic session addressing basic concepts; case-based skills exercises in which subsets of 2–3 students rotate between examination rooms with a faculty facilitator and an SP, and a whole-group concluding session to review learning goals.	Students who require remedial training in clinical and communication skills appreciate individualized feedback and skills sessions with preceptors and SPs. According to personal communication, all students in the program succeeded. *(Level 1)*
Deveugele 2005⁽³²⁾	Medical Student Belgium	7.5	• Following a longitudinal communication curriculum, students assessed every year using an OSCE with SP with specific objectives. • Two examiners rate the student and come to a final mark after discussion.	Three different remediation strategies depending on year level: • During the 2nd year of the Bachelor degree, remedial teaching consists of one up to three extra training sessions in a small group. • During the 1st year of the Master education, every student who failed is invited to exercise during 1 h with a simulated patient. The student can formulate his/her own learning objectives. The whole session is videotaped and the taped is reviewed by one of the trainers in order to give feedback to the student. The student can rehearse this three times. • During the 2nd year of the Master degree, the student exercises with a simulated patient in the presence of one of the trainers and gets immediate feedback.	No outcome stated
Dowell 2006⁽³³⁾	28 Medical Student UK	8	• OSCE to assess communication skills “Consultation skills” as a screening tool. • Three 4-min OSCE to assess aspects of communication skills. • Scoring was done using the SEGUE framework by trained tutors.	• Attend a week of additional training. • Interactive teaching contained many opportunities to develop, videotape, review and practice both basic and more complex consultation skills. • At the end, a separate four-station OSCE was used to assess skills using another three four-minute consultation skills stations plus an additional ten-minute assessment that allowed students to complete a basic consultation. • This was videotaped to enhance feedback and reliability.	Students succeeded and progressed to next year. *(Level 2)*
Goulet 2005⁽⁴¹⁾	220 Family Physician and 85 Specialist Consultant Canada	9	Physicians with clinical performance problems identified through: • Professional Inspection Committee (PIC), • Complaints forwarded to the inquiry division, or • Processes initiated by physicians who would like to re-orient their careers or come back to practice after a period of inactivity of over four years. Every assessment process is conducted using a: • Standard grid listing the criteria of quality of care, • Chart keeping, and • Office practices. Sometimes a more in-depth evaluation is conducted: • An evaluation of specialists’ clinical performance by a peer. • The structured oral interview (SOI). Six objective structured clinical examination (OSCE) stations were added to the SOI. (includes knowledge, Physical exam, Doctor-patient relationship)	To improve Physician practice, the Professional Inspection Committee (PIC) may recommend that: 1. The physician participates in specific CME activities. 2. The CMQ admin committee imposes a remedial retraining. Interventions include: • Clinical training programs, • Tutorials, • Focused readings, and • Various courses or workshops developed and organized by the CMQ in association with the medical schools in Quebec or the Quebec College of Family Physicians.	• 70% of the retraining activities led to attainment of the training objectives; • 15% led to partly attained objectives, • 13% failed to lead to attainment of objectives, and • 2% involved missing data or withdrawal. The 70% resumed their practices. *(Level 2)*
Guevara 2011⁽⁴⁵⁾	16 IM, Diagn Rad, Ob/Gyn, ER, Gen Surg, Peds Residents US	8	• Monthly evaluation from Faculty, peers and students. • 360-degree evaluation, which includes input from nurses and case managers. • Evaluations reviewed by Internal Residency Review Committee (IRRC). • The IRRC charges the PD or Chief Resident to counsel the resident and monitor their progress. If no improvement formal IEP may be required.	• Individualized Education Plans (IEP) that includes a listing of competencies, intervention designed to address, reassessment with objective metrics and milestones for completion of IEP. • Resident input is solicited to refine the IEP and to identify a faculty mentor. • Interventions include, remedial tutorials, frequent meetings with mentor, speech therapy, counseling	12/16 successfully graduated from the IEP program. *(Level 2)*
Lin 2001⁽³⁴⁾	1 Medical Student US	8	• Clinical preceptors assessment and end-of-year clinical practice exam. • OSCE/SP encounters in communication Skills.	A year-long intensive remedial curriculum in communication skills. Includes: • Pairing with a clinical preceptor for intensive skills training, including a weekly preceptor clinic, • Structured readings, • SP exercises, • Communications workshops, and • End-of-year standardized clinical evaluations.	Improvement in communication skills. Additionally the student unexpectedly wrote a 12-page guide to interview skills for his preclinical colleagues. *(Level 2)*
Malau-Aduli 2013⁽⁴⁶⁾	18 Medical Students (4th and 5th year) Australia	11	Students were identified for remediation due to the following: • Failure in examinations or repeating the year. • Workplace-based assessments.	Remediation program developed based on socio-cognitive Self-Efficacy beliefs to improve academic and clinical performance. • A multi-dimensional ten-week support programbased on individual assessment offered primarily as a group learning experience. • Individual counseling to provide psychological support. The program consisted of the following elements: • Presentation Skills workshops • OSCE Practice (4-station) with Clinical Teachers • OSCE Practice with other Students • Bedside Teaching	Performance on all measures improved after the remediation program with statistically significant improvements on management plan (MP), diagnostic skills (DS), communication skills (CS) and number of stations passed (NSP). All 18 participants in the remediation program were successful in their end of 4th year summative OSCE assessments. In their 5th year (eight of them) passed all their examinations without any support or intervention. *(Level 2)*
Myung 2013⁽⁴²⁾	23 Medical Students Korea	10	Clinical performance examination (CPX) (8 stations with SP encounters using a checklist).	A six-week remediation program (3 weeks Internal Medicine classes and 3 weeks Family Medicine classes). Includes: • 1:1 tutoring sessions • Re-examination • Feedback from SP Designed with 3 parts: Diagnosis, learning activities, and re-examination.	• Students’ scores on the CPX exam improved. *(Level 2)*
Rowland 2012⁽⁴⁷⁾	225 Surgery Residents US	9.5	• Identified by Directors: (Mock Oral Exams, Case Presentations, Journal Clubs, Mortality & Morbidity conferences, outpatient clinics, and hospital and operating room settings). • Failed Surgery Certifying Exam. • Rowland Communication Skills Inventory used to identify candidates with severe communication problems that might need further assistance with their communication skills before entering the course.	5-Day Oral Examination Course: • Didactics, • Mini oral examinations in suites, • Individual assessments, • Small-group exercises, • Formal mock oral examinations, • Individual debriefing sessions with a general surgeon and a behavioral scientist that summarized individual improvement, communication competency, strengths and weaknesses on the formal mock oral examination, and a remediation plan for future improvement. Resident received a personal digital video of their formal mock oral for review and self-critique and individual remediation plan. In 2007, course shortened to 3 Days.	Primary outcome measure is the successful completion of the Certifying Exam. 218 residents followed their remediation plan and successfully Passed the Surgery CE on first attempt *(Level 2)*
Ryan 2010⁽³⁶⁾	64 Nursing 46 Medical Students UK	7	A screening program in communication and consultation skills (CCS) using: • Trained Standardized Patient Educators (SPEs) • A previously validated global rating scale for CCS. Almost three quarters of medical students (33/46; 72%) and 81% of nursing students (56/64) passed the CCS assessment in both communication and attitudes categories.	One-on-one CCS training.	*(Level 2)*
Saxena 2009⁽²⁴⁾	Medical Students US	11	Comprehensive Assessment Test (Cross-disciplinary exam with SPs). • Reviewing exam scores (96%) • Reviewing video of failing student exam (57%) • Meeting with failing stud (49%)	• Precepted video review • Preceptorship • Independent Study (Web-Based module, reading) • Stud independently reviews exam recording • Practice with SP • Skills workshops, seminars or group discussions	• Study measured confidence and not outcomes of remediation. *(Level 2)*
Sperry 2010⁽³⁷⁾	3 Medical Students (4th yr) US	11	Clinical performance examination (CPX) using SP. • Evaluation includes Medical History, physical exam, communication and relationship issues, diagnosis and management.	Individualized Doctor-Patient Communications and Psychosocial Interviewing remediation curriculum addresses communication skills deficits: • 2-week Didactic and experiential components including role-play, videos, personal reflections, performing interviews, history taking and Physical exam with patients presenting to clinic. • Observed Live and provided feedback by family physicians. • Written test assess knowledge of communication strategies before and after remediation. • Patients completed a satisfaction survey.	• No difference in written test scores. • Patient satisfaction indicated positive qualities with no difference before and after. • Preceptors’ evaluation of students indicated an improvement in CS. • All passed CPX and one passed a re-sit of USMLE-CS *(Level 2)*
Torbeck 2009⁽²⁰⁾	Surgery Residents US	6	• Assessment tools used were not addressed.	Program director devise individual remediation plans and monitor progress. Most programs use primarily 3 methods for remediating residents: • To increase direct observation of the resident by the attending in the clinic/operating room/wards, • To have the resident undergo psychological counseling, and • To have the resident attend organized professionalism or communication workshops/seminars. Among the other methods reported: • 360° evaluations, • Specific counseling with the PD, • Reviewing How to Win Friends & Influence People on a weekly basis with the program director, • Have residents present frequently, • Sending a resident to an English tutor, • Recommending a speech/communication coach, and • Counseling for problems related to hostile relationships/interactions.	*(Level 1)*
Wiskin 2013⁽³⁸⁾	1 Medical Student UK	8	• Clinical OSCE	• One-on-One coaching • Individual and group teaching, • Individual support and remedial teaching, • Workshop program, • OSCE course.	No outcome stated
Zbieranowski 2013⁽³⁹⁾	100 Medical Resident Canada	8	• Identified by Failure to meet criteria of CanMEDS roles. • Board of Examiners for Postgraduate Programs (BOE-PG) objectively review cases of postgraduate students in academic difficulty and determine appropriate course of action, which could include: Remediation, probation, or dismissal. • 49% had weakness as Communicator CanMed Role.	CanMEDS Roles are used as the organizational framework for the individual formal remediation plans developed by the residency program director.	78% Completed Residency Education. *(Level 2)*

First Author and year

Number & Struggling learner Characteristic
& Country

MERSQI Score

Description of Assessment for identifying struggling learner

Intervention

Outcomes (Kirkpatrick level of evaluation)

Bodenberg 2015 ⁽⁴⁴⁾

Pharmacy Student

• Midpoint evaluation and concern cards to alert the Director of Experiential Education of potential behavioral or learning issues.

• The faculty developed a section in the student evaluation form in which the preceptor can suggest longitudinal monitoring or remediation needed for the student.

• 9% had a communications skills deficiency.

The Director of Experiential Education creates a performance improvement plan.

This individualized plan takes into account input from the student, preceptor, and the Director of Experiential Education.

A list of communication tools includes:

• Direct observations of the student during case presentations,

• Counseling,

• Drug information,

• Topic presentations, and

• An oral “End of Block” exam.

The on-time graduation using the developed remediation plan was seven of eight students (88%), and the overall graduation success rate was eight of eight students (100%).

(Level 2)

Chang 2008 ⁽⁴⁰⁾

Medical Student

5.5

The clinical performance examination (CPX) is an eight-station, high-stakes examination in which SPs assess early Year 4 medical students’ competence in clinical and communication skills.

Students scoring ≥2 standard deviations (SD) below the class average in one or both skill domains (history taking and physical examination [H & P] or patient–doctor interaction [PDI]) are required to remediate.

Developed a four-step CPX remedial training program that incorporates diagnosis of learner problems, individual reflection, faculty feedback and supervised practice.

Step 1 consists of an individual review in which:

• Students watch their own videotaped examinations,

• Reflect on their performances, and

• Develop personal learning goals.

Step 2, the faculty remediation director:

• Watches the videotape independently and

• Generates a ‘learning prescription’,

Step 3 involves a one-to-one video review guided by the learning prescription and the student’s self-identified goals with a designated faculty member.

In Step 4, students attend one or both evening workshops in H & P or PDI, depending on their area of need.

Workshops consist of: an introductory 30-min didactic session addressing basic concepts; case-based skills exercises in which subsets of 2–3 students rotate between examination rooms with a faculty facilitator and an SP, and a whole-group concluding session to review learning goals.

Students who require remedial training in clinical and communication skills appreciate individualized feedback and skills sessions with preceptors and SPs.

According to personal communication, all students in the program succeeded.

(Level 1)

Deveugele 2005⁽³²⁾

Medical Student

Belgium

7.5

• Following a longitudinal communication curriculum, students assessed every year using an OSCE with SP with specific objectives.

• Two examiners rate the student and come to a final mark after discussion.

Three different remediation strategies depending on year level:

• During the 2nd year of the Bachelor degree, remedial teaching consists of one up to three extra training sessions in a small group.

• During the 1st year of the Master education, every student who failed is invited to exercise during 1 h with a simulated patient. The student can formulate his/her own learning objectives. The whole session is videotaped and the taped is reviewed by one of the trainers in order to give feedback to the student. The student can rehearse this three times.

• During the 2nd year of the Master degree, the student exercises with a simulated patient in the presence of one of the trainers and gets immediate feedback.

No outcome stated

Dowell 2006⁽³³⁾

Medical Student

• OSCE to assess communication skills “Consultation skills” as a screening tool.

• Three 4-min OSCE to assess aspects of communication skills.

• Scoring was done using the SEGUE framework by trained tutors.

• Attend a week of additional training.

• Interactive teaching contained many opportunities to develop, videotape, review and practice both basic and more complex consultation skills.

• At the end, a separate four-station OSCE was used to assess skills using another three four-minute consultation skills stations plus an additional ten-minute assessment that allowed students to complete a basic consultation.

• This was videotaped to enhance feedback and reliability.

Students succeeded and progressed to next year.

(Level 2)

Goulet 2005⁽⁴¹⁾

220

Family Physician and

Specialist Consultant

Canada

Physicians with clinical performance problems identified through:

• Professional Inspection Committee (PIC),

• Complaints forwarded to the inquiry division, or

• Processes initiated by physicians who would like to re-orient their careers or come back to practice after a period of inactivity of over four years.

Every assessment process is conducted using a:

• Standard grid listing the criteria of quality of care,

• Chart keeping, and

• Office practices.

Sometimes a more in-depth evaluation is conducted:

• An evaluation of specialists’ clinical performance by a peer.

• The structured oral interview (SOI). Six objective structured clinical examination (OSCE) stations were added to the SOI. (includes knowledge, Physical exam, Doctor-patient relationship)

To improve Physician practice, the Professional Inspection Committee (PIC) may recommend that:

1. The physician participates in specific CME activities.

2. The CMQ admin committee imposes a remedial retraining.

Interventions include:

• Clinical training programs,

• Tutorials,

• Focused readings, and

• Various courses or workshops developed and organized by the CMQ in association with the medical schools in Quebec or the Quebec College of Family Physicians.

• 70% of the retraining activities led to attainment of the training objectives;

• 15% led to partly attained objectives,

• 13% failed to lead to attainment of objectives, and

• 2% involved missing data or withdrawal.

The 70% resumed their practices.

(Level 2)

Guevara 2011⁽⁴⁵⁾

IM,

Diagn Rad, Ob/Gyn, ER,

Gen Surg, Peds

Residents

• Monthly evaluation from Faculty, peers and students.

• 360-degree evaluation, which includes input from nurses and case managers.

• Evaluations reviewed by Internal Residency Review Committee (IRRC).

• The IRRC charges the PD or Chief Resident to counsel the resident and monitor their progress. If no improvement formal IEP may be required.

• Individualized Education Plans (IEP) that includes a listing of competencies, intervention designed to address, reassessment with objective metrics and milestones for completion of IEP.

• Resident input is solicited to refine the IEP and to identify a faculty mentor.

• Interventions include, remedial tutorials, frequent meetings with mentor, speech therapy, counseling

12/16 successfully graduated from the IEP program.

(Level 2)

Lin 2001⁽³⁴⁾

Medical Student

• Clinical preceptors assessment and end-of-year clinical practice exam.

• OSCE/SP encounters in communication Skills.

A year-long intensive remedial curriculum in communication skills. Includes:

• Pairing with a clinical preceptor for intensive skills training, including a weekly preceptor clinic,

• Structured readings,

• SP exercises,

• Communications workshops, and

• End-of-year standardized clinical evaluations.

Improvement in communication skills. Additionally the student unexpectedly wrote a 12-page guide to interview skills for his preclinical colleagues.

(Level 2)

Malau-Aduli 2013⁽⁴⁶⁾

Medical Students (4th and 5th year)

Australia

Students were identified for remediation due to the following:

• Failure in examinations or repeating the year.

• Workplace-based assessments.

Remediation program developed based on socio-cognitive Self-Efficacy beliefs to improve academic and clinical performance.

• A multi-dimensional ten-week support programbased on individual assessment offered primarily as a group learning experience.

• Individual counseling to provide psychological support.

The program consisted of the following elements:

• Presentation Skills workshops

• OSCE Practice (4-station) with Clinical Teachers

• OSCE Practice with other Students

• Bedside Teaching

Performance on all measures improved after the remediation program with statistically significant improvements on management plan (MP), diagnostic skills (DS), communication skills (CS) and number of stations passed (NSP).

All 18 participants in the remediation program were successful in their end of 4th year summative OSCE assessments. In their 5th year (eight of them) passed all their examinations without any support or intervention.

(Level 2)

Myung 2013⁽⁴²⁾

Medical Students

Korea

Clinical performance examination (CPX) (8 stations with SP encounters using a checklist).

A six-week remediation program (3 weeks Internal Medicine classes and 3 weeks Family Medicine classes). Includes:

• 1:1 tutoring sessions

• Re-examination

• Feedback from SP

Designed with 3 parts: Diagnosis, learning activities, and re-examination.

• Students’ scores on the CPX exam improved.

(Level 2)

Rowland 2012⁽⁴⁷⁾

225

Surgery Residents

9.5

• Identified by Directors: (Mock Oral Exams, Case Presentations, Journal Clubs, Mortality & Morbidity conferences, outpatient clinics, and hospital and operating room settings).

• Failed Surgery Certifying Exam.

• Rowland Communication Skills Inventory used to identify candidates with severe communication problems that might need further assistance with their communication skills before entering the course.

5-Day Oral Examination Course:

• Didactics,

• Mini oral examinations in suites,

• Individual assessments,

• Small-group exercises,

• Formal mock oral examinations,

• Individual debriefing sessions with a general surgeon and a behavioral scientist that summarized individual improvement, communication competency, strengths and weaknesses on the formal mock oral examination, and a remediation plan for future improvement. Resident received a personal digital video of their formal mock oral for review and self-critique and individual remediation plan.

In 2007, course shortened to 3 Days.

Primary outcome measure is the successful completion of the Certifying Exam.

218 residents followed their remediation plan and successfully Passed the Surgery CE on first attempt

(Level 2)

Ryan 2010⁽³⁶⁾

Nursing

Medical Students

A screening program in communication and consultation skills (CCS) using:

• Trained Standardized Patient Educators (SPEs)

• A previously validated global rating scale for CCS.

Almost three quarters of medical students (33/46; 72%) and 81% of nursing students (56/64) passed the CCS assessment in both communication and attitudes categories.

One-on-one CCS training.

(Level 2)

Saxena 2009⁽²⁴⁾

Medical Students

Comprehensive Assessment Test (Cross-disciplinary exam with SPs).

• Reviewing exam scores (96%)

• Reviewing video of failing student exam (57%)

• Meeting with failing stud (49%)

• Precepted video review

• Preceptorship

• Independent Study (Web-Based module, reading)

• Stud independently reviews exam recording

• Practice with SP

• Skills workshops, seminars or group discussions

• Study measured confidence and not outcomes of remediation.

(Level 2)

Sperry 2010⁽³⁷⁾

Medical Students

(4th yr)

Clinical performance examination (CPX) using SP.

• Evaluation includes Medical History, physical exam, communication and relationship issues, diagnosis and management.

Individualized Doctor-Patient Communications and Psychosocial Interviewing remediation curriculum addresses communication skills deficits:

• 2-week Didactic and experiential components including role-play, videos, personal reflections, performing interviews, history taking and Physical exam with patients presenting to clinic.

• Observed Live and provided feedback by family physicians.

• Written test assess knowledge of communication strategies before and after remediation.

• Patients completed a satisfaction survey.

• No difference in written test scores.

• Patient satisfaction indicated positive qualities with no difference before and after.

• Preceptors’ evaluation of students indicated an improvement in CS.

• All passed CPX and one passed a re-sit of USMLE-CS

(Level 2)

Torbeck 2009⁽²⁰⁾

Surgery Residents

• Assessment tools used were not addressed.

Program director devise individual remediation plans and monitor progress. Most programs use primarily 3 methods for remediating residents:

• To increase direct observation of the resident by the attending in the clinic/operating room/wards,

• To have the resident undergo psychological counseling, and

• To have the resident attend organized professionalism or communication workshops/seminars.

Among the other methods reported:

• 360° evaluations,

• Specific counseling with the PD,

• Reviewing How to Win Friends & Influence People on a weekly basis with the program director,

• Have residents present frequently,

• Sending a resident to an English tutor,

• Recommending a speech/communication coach, and

• Counseling for problems related to hostile relationships/interactions.

(Level 1)

Wiskin 2013⁽³⁸⁾

Medical Student

• Clinical OSCE

• One-on-One coaching

• Individual and group teaching,

• Individual support and remedial teaching,

• Workshop program,

• OSCE course.

No outcome stated

Zbieranowski 2013⁽³⁹⁾

100

Medical Resident

Canada

• Identified by Failure to meet criteria of CanMEDS roles.

• Board of Examiners for Postgraduate Programs (BOE-PG) objectively review cases of postgraduate students in academic difficulty and determine appropriate course of action, which could include: Remediation, probation, or dismissal.

• 49% had weakness as Communicator CanMed Role.

CanMEDS Roles are used as the organizational framework for the individual formal remediation plans developed by the residency program director.

78% Completed Residency Education.

(Level 2)