Effect of Experiential Communication Skills Education on Graduate Medical Education Trainees’ Communication Behaviors: A Systematic Review

Abstract

Purpose

A better understanding of how communication skills education impacts trainees’ communication skills is important for continual improvement in graduate medical education (GME). Guided by the Kirkpatrick Model, this review focused on studies that measured communication skills in either simulated or clinical settings. The aim of this systematic review was to examine the effect of experiential communication skills education on GME trainees’ communication behaviors.

Method

Five databases were searched for studies published between 2001 and 2021 using terms representing the concepts of medical trainees, communication, training, and skills and/or behaviors. Included studies had an intervention design, focused only on GME trainees as learners, used experiential methods, and had an outcome measure of communication skills behavior that was assessed by a simulated or standardized patient (SP), patient, family member, or outside observer. Studies were examined for differences in outcomes based on study design; simulated versus clinical evaluation setting; outside observer versus SP, patient, or family member evaluator; and length of training.

Results

Seventy-seven studies were ultimately included. Overall, 54 (70%) studies reported some positive findings (i.e., change in behavior). There were 44 (57%) single-group pre-post studies, 13 (17%) nonrandomized control studies, and 20 (26%) randomized control studies. Positive findings were frequent in single-group designs (80%) and were likely in nonrandomized (62%) and randomized (55%) control trials. Positive findings were likely in studies evaluating communication behavior in simulated (67%) and clinical (78%) settings as well as in studies with outside observer (63%) and SP, patient, and family member (64%) evaluators.

Conclusions

This review demonstrates strong support that experiential communication skills education can impact GME trainees’ communication behaviors. Marked heterogeneity in communication trainings and evaluation measures, even among subgroups, did not allow for meta-analysis or comparative efficacy evaluation of different studies. Future studies would benefit from homogeneity in curricular and evaluation measures.

In 2001, the U.S.-based Accreditation Council for Graduate Medical Education (ACGME) introduced interpersonal and communication skills as a core competency of medical education training that should be taught and assessed as a part of its Outcome Project.¹ Accreditation organizations from countries outside of the United States have similarly recognized the importance of including communication skills in graduate medical education (GME) and included it in their competency frameworks.^2,3

These accreditation requirements are accompanied by increased recognition of the importance of communication skills to quality clinical care. The Institute of Healthcare Improvement, an international organization dedicated to improving health and health care worldwide, emphasizes communication as a means to improving patient experience and patient safety.⁴ The U.S. Department of Health and Human Services public health goals for the country include objectives for improved clinical communication in a variety of disease contexts.⁵ Furthermore, substantial research literature has shown the importance of good clinical communication to patient and family experience⁶ and important outcomes, such as patient adherence,⁷ malpractice claims,⁸ patient safety,⁹ and patient health outcomes.^10,11

This emphasis on communication skills has inspired many new educational initiatives for GME trainees, including research studies to examine the outcomes of communication skills education. Targeting GME trainees for communication skills is particularly important as these trainees regularly care for patients and have a strong knowledge base but are still developing communication practices and habits that they will use with patients and families for decades to come. Although the efficacy and outcomes of communication skills education have received significant attention in the literature focused on undergraduate medical education (UME) and continuing medical education (CME),^12–16 the effectiveness of this education at the GME level remains uncertain. There is a lack of synthesis of the evidence attained from more than 20 years of research on this topic, particularly about the central question: Does GME communication skills education result in participants’ communication behavior change? A better understanding of how communication skills education impacts trainees’ communication skills is important for continual improvement in GME.

A useful conceptual model for understanding the impact of communication skills education is the Kirkpatrick Model.¹⁷ Initially developed for program evaluation, the model was later applied to medical education.¹⁸ The model posits that higher levels of evaluation are needed to detect more complex behavior change. Konopasek et al’s application of the model to communication skills education¹⁹ conceptualizes the levels as follows. Level 1 is the evaluation of reaction, which includes satisfaction with the learning session. Level 2 is the evaluation of learning, which focuses on both self-efficacy and changes in attitudes (level 2A) and evaluation of knowledge and skill levels in a simulated setting (level 2B). Level 3 is the evaluation of behavior, which focuses on participants’ demonstration of applying what they learned in a patient care setting. Finally, level 4 is the evaluation of results, which assesses changes in patient outcomes related to the learning session.

The ACGME’s emphasis on competency-based GME guides us to focus primarily on trainees’ behaviors, that is, levels 2B and 3. We refer to these as levels 2 and 3 below. These are the evaluation levels at which skills are demonstrated in simulated and clinical settings and can thus be evaluated as part of each training program’s milestones. Thus, the aim of this systematic review was to examine the effect of experiential communication skills education on GME trainees’ communication behaviors. Further, we evaluated the included studies by 4 study characteristics that we expected may be salient factors in terms of whether behavior change occurred: (1) study design, (2) evaluation setting, (3) evaluator type, and (4) length of training.

Method

We followed the guidelines in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement²⁰ in writing this systematic review. We registered our systematic review protocol with PROSPERO (https://www.crd.york.ac.uk/). The registration number is CRD42017056076.

Eligibility criteria

Eligible studies were published between January 1, 2001, and September 30, 2021. We chose 2001 as the earliest limit as that is when the ACGME introduced the Outcome Project, which formalized interpersonal and communication skills as a competency in the United States.^1,21 Inclusion criteria were: (1) an intervention study design (pre-post or controlled trial), (2) focused only on GME trainees as the learners (i.e., residents, fellows, or the equivalent), (3) training focused on clinical communication, (4) use of experiential methods (i.e., practice of skills with feedback), and (5) had an outcome measure of communication skills behavior that was assessed by a simulated or standardized patient (SP), patient, family member, or outside observer (faculty or trained coder). Only peer-reviewed papers and those written in English were included.

Information sources

Five bibliographic databases of high-quality literature in the areas of health sciences, education, and communication were searched for studies that fit our inclusion criteria. These included MEDLINE (via PubMed), ERIC, Web of Science, Communication & Mass Media Complete (via EBSCO), and Academic Search Premier (via EBSCO). Our first search was performed on March 19, 2018, and our last search was performed on September 30, 2021.

Search terms

The 5 databases were searched using terms representing the concepts of medical trainees, communication, training, and skills and/or behaviors. The full search strategy for MEDLINE is included in Supplemental Digital Appendix 1 (at [LWW INSERT LINK]).

Study selection

We used the Covidence systematic review management system (Covidence, Melbourne, Victoria, Australia) to manage the study selection process. First, titles and abstracts were screened and were either excluded or moved on to full-text assessment. Each full-text article was then screened for inclusion or exclusion in the review. All abstracts were independently screened by the first author (C.L.B.) and a second author (randomly selected by Covidence). All full-text articles were screened by 2 authors (randomly selected by Covidence). Most authors participated in the study selection process (T.S.V., E.B.P., K.C.B., P.D.-P., R.M., R.W., M.E.R.).. The final decision on any disagreements was made by either the first (C.L.B.) or senior author (M.E.R.).

Data collection process

Teams of 2 authors (the first author [C.L.B.] plus one other author [see below]) each independently extracted data from each article and reconciled differences. Most authors participated in the data extraction process (T.S.V., E.B.P., K.C.B., R.M., N.S.O.). We extracted data related to the study design, type and number of medical trainees, type of communication education, assessment mechanism, and outcomes measures.

Quality assessment

As quality assessment is an important component of systematic review methodology, 2 authors (C.L.B., T.S.V.) independently assessed each included study using a modified version of the Medical Education Research Study Quality Instrument (MERSQI).²² The MERSQI has been used since 2007 as a measure of quality in medical education research and has been shown to be valid and reliable.²² It includes 10 items clustered in 6 domains with a potential high score of 18, with higher scores indicating better quality. However, due to the inclusion criteria of our review, potential MERSQI scores for our included studies could range from 7 to 17. All differences were reconciled through discussion.

Outcome measures

For each included study, we determined whether the findings within a skill set or domain were positive (statistically significant skill increase), negative (statistically significant skill decrease), or mixed (some skill increase and some skill decrease that were undifferentiated in the study). We used the terms increase and decrease here broadly to capture both frequency and quality of skills. Studies that had no significant findings were also noted. If a study reported on multiple skills within a skill set or domain and 75% or more of those showed an increase, these were coded as positive findings. For consistency, where studies had multiple, longitudinal post-intervention assessments, only the first one was coded. For studies with control groups, only between-group differences are reported.

Synthesis of results

Studies were examined for differences in outcomes based on study design; simulated versus clinical evaluation setting; outside observer versus SP, patient, or family member evaluator; and length of training. To this end, we first separated the studies into 3 categories based on study design: (1) single-group studies that used a pre-post evaluation method, (2) nonrandomized control group studies, and (3) randomized control group studies. Studies with a control group could either use pre-post or post-only evaluation. Within those 3 categories, we then divided the studies into those that evaluated behavior in a simulated setting (Kirkpatrick’s level 2) and those that evaluated behavior in a clinical setting (Kirkpatrick’s level 3). Studies were further divided by whether the evaluation was performed by an outside observer or a SP, patient, or family member. Finally, we examined differences by whether the training was less than 7 hours or was 7 hours or more in length.

We used the vote counting process to summarize the findings.²³ Vote counting weights each study the same and is used when it is not feasible to perform a meta-analysis due to heterogeneity of measurement (e.g., a study with a sample size of 30 is weighted the same as a study with a sample size of 75).

Results

The systematic search identified 8,603 studies for screening after duplicates were removed. Of these, we assessed 349 full-text articles for eligibility, resulting in 77 studies considered suitable for inclusion in the final review (Figure 1).^24–100 Overall, 54 (70%) studies reported some positive findings (i.e., change in behavior). Table 1 summarizes the characteristics of the included studies. There were 44 (57%) single-group, pre-post studies^24–67; 13 (17%) nonrandomized control studies^68–80; and 20 (26%) randomized control studies.^81–100 The majority of studies were from the United States (52; 68%). The most frequent topic was general communication skills (27; 35%) as opposed to specific communication topics (e.g., giving bad news, end of life discussions). Pediatrics (16; 21%) and internal medicine (11; 14%) were the most frequent trainee specialties.

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram for a 2021 systematic review examining the effect of experiential communication skills education on graduate medical education (GME) trainees’ communication behaviors.²⁰

Table 1.

Description of 77 Included Studies in a 2021 Systematic Review Examining the Effect of Experiential Communication Skills Education on Graduate Medical Education Trainees’ Communication Behaviors

Study characteristic	No. (%)
Study design
Single-group, pre-post	44 (57)
Nonrandomized control group	13 (17)
Randomized control group	20 (26)
Evaluation setting a
Simulated	51 (66)
Clinical	27 (35)
Evaluator type b
Outside observer (faculty or trained coder)	49 (64)
Simulated or standardized patient, patient, or family member	42 (55)
Study location
North America	55 (71)
United States	52 (68)
Canada	3 (4)
Europe	13 (17)
Belgium	3 (4)
The Netherlands	3 (4)
Switzerland	2 (3)
Other c	5 (6)
Asia	7 (9)
Iran	2 (3)
Other d	5 (6)
Australia	2 (3)
Course content e
General communication skills	27 (30)
Giving bad news	13 (14)
End of life	8 (9)
Multiple topics (more than 3)	4 (4)
Empathy	4 (4)
Smoking cessation	4 (4)
Death notification	3 (3)
Palliative care	3 (3)
Alcohol intervention	2 (2)
Health literacy	2 (2)
Motivational interviewing	2 (2)
Obesity	2 (2)
Patient-centered communication	2 (2)
Diagnostic or unnecessary testing	3 (3)
Otherf	11 (12)
Specialty
Pediatrics	16 (21)
Internal medicine	11 (14)
Mixed	10 (13)
Oncology	9 (12)
Surgery	9 (12)
General practice	5 (6)
Internal medicine and pediatrics	3 (4)
Psychiatry	4 (5)
Anesthesia	2 (3)
Emergency medicine	2 (3)
Family medicine	2 (3)
Otherg	4 (5)

^aOne study³¹ measured in both settings, so the total in this portion of the table is higher than 77 and the percentages equal more than 100%.

^bMany studies used more than one type of evaluator, so the denominator in this portion of the table is 90 and percentages do not equal 100% due to rounding.

^cOther European study locations are England, Germany, Italy, Portugal, and Spain; for each, n = 1 (1%).

^dOther Asian study locations are China, Israel, Lebanon, Syria, and Turkey; for each, n = 1 (1%).

^eSome courses included more than one type of content, so the total in this portion of the table is higher than 77 and the percentages equal more than 100%.

^fOther course content topics are anger de-escalation, code status discussions, cultural proficiency, family meeting, limited English proficiency, patient-centered interviewing, physical activity, psychological risk screening, rapport, relational communication, and specialty referral; for each, n = 1 (1%).

^gOther specialties are nephrology, neurology, obstetrics and gynecology, and pulmonary or critical care; for each, n = 1 (1%).

Single-group, pre-post studies

Table 2 summarizes the 44 single-group, pre-post studies included in the review,^24–67 with 29 studies using a simulated evaluation and 16 using a clinical evaluation (1 study³¹ reported on evaluations in both settings). Of the single-group, pre-post studies, 23 used outside observers to evaluate behavior change in a simulated setting,^{24–26,28,30,31,33,36–40,42,47,51,54,60,62–67} with 20 reporting significant positive findings of skills change in at least one skill set or domain. Of the 11 studies using SP evaluations to evaluate behavior change in a simulated setting,^{26,27,33,34,42,44,47–49,52,54} 8 reported significant positive findings in at least one skill set or domain. Ten studies used outside observers to evaluate communication in a clinical setting,^{29,31,43,45,46,53,56,57,59,61} with 5 reporting significant positive findings in at least one skill set or domain. Ten studies used patients or family members to evaluate communication in a clinical setting,^{31,32,35,41,45,46,50,55,56,58} with 5 studies reporting significant positive findings in at least one skill set or domain.

Table 2.

Single-Group, Pre-Post Test Studies (n = 44) Included in a 2021 Systematic Review Examining the Effect of Experiential Communication Skills Education on Graduate Medical Education Trainees’ Communication Behaviors

First author, yearref	Country	Participantsa	Contentb	Length (in minutes [m], hours [h], or days [d])	Outside observer (faculty or trained coder)		Simulated or standardized patient (simulated settings) or patient or family member (clinical settings)		MERSQI scoree
					Findingsc	Skills measuredd	Findingsc	Skills measuredd
Simulated setting (n = 29)
Amsalem, 202124	Israel	Psychiatry; n = 28	BN	5 h	+	BN	---	---	14
Back, 200725	US	Oncology; n = 115	BN, Pall	4 d	+	BN, Pall, Empathy	---	---	14
Bai, 201926	China	Surgery; n = 210	General	27 h	+	General	+	Sat Ratings	12
Barbosa, 201927	Portugal	Mixed; n = 64	Rel Com	10 h	---	---	+, MF	Cues, BN	12.5
Blanco, 201328	US	Mixed; n = 41	General	.5 d and 4 h	NS	General	---	---	11.5
Burton, 201630	US	Int Med and Peds; n = 86	Obes, MI	3 h	+	Global MI Score	---	---	12.5
Bylund, 201831,f	US	Oncology; n = 262	Multiple	2 d	+, MF	Overall, Individual	---	---	12.5
Cannone, 201933	US	Oncology; n = 22	Multiple	16 h	+	BN	+	Overall	13
Chandawarkar, 201134	US	Surgery; n = 44	General	90 m	---	--	+, NS	Individual, General	15
Clayton, 201336	Australia	Mixed; n = 22	EOL	3 h	+	Global Behavior Score, EOL	---	---	13
Ditton-Phare, 201637	Australia	Psychiatry; n = 30	General	6 h	+, −	Agenda, Questioning	---	---	13
Douglas, 201838	US	Neurology; n = 18	BN	5 h	+	Overall	---	---	12
Esfahani, 201439,g	Iran	Psychiatry; n = 7	General	2 d	+	Empathy, General	---	---	12
Fune, 202140	US	Peds; n = 10	LEP	3 h	+	Overall	---	---	12
Ghoneim, 201942	US	Peds; n = 15	BN	4.5 h	NS	BN	NS	General	13
Harahsheh, 201644	US	Peds; n = 48	Referral	90 m	---	---	+	Comm Ratings	11
Hobgood, 200547	US	Emerg Med; n = 20	Dth Not	2 h	+	Competency	NS	Comm Ratings	12
Hochberg, 201049	US	Surgery; n = 15	General	6 h	---	---	+	Individual	11
Hochberg, 201248	US	Surgery; n = 9	General	6 h	---	---	NS	Professional	11
Kapadia, 202051	US	Surgery; n = 17	General	2 h	+	Overall	---	---	14
Karam, 201752	Lebanon	Anesthesia; n = 16	Dth Not	4 h	---	---	+	Overall, Comm Ratings	12
Meltzer, 201754,h	US	Int Med; n = 33	EOL	6 h	+	Comm Ratings	+	Comm Ratings	11
Reed, 201560	US	Peds; n = 29	Dth Not	2 h	+	BN	---	---	13.5
Rimondini, 201062	Italy	Psychiatry; n = 10	General	16 h	+	Overall	---	---	12
Roter, 200463	US	Peds; n = 28	General	4 h	+	General, Individual	---	---	12
Schell, 201864	US	Nephrology; n = 33	BN	3 d	+	Overall	---	---	14
Tobler, 201465	Canada	Peds; n = 33	BN	5 h	+	Individual	---	---	12
Williams, 201166	US	Int Med; n = 24	EOL	3 h	+	Overall, Individual	---	---	11.5
Yakhforoshha, 201967	Iran	Oncology; n = 19	BN	1 d	MF	BN	---	---	12.5
Clinical setting (n = 16)
Brown, 201029	US	Oncology; n = 45	Multiple	2 d	+, MF	Overall, Individual	---	---	13.5
Bylund, 201831,f	US	Oncology; n = 262	Multiple	2 d	NS	Overall, Individual	NS	Comm Ratings	12.5
Canales, 201932	US	Anesthesia; n = 10	General	20 h	---	---	+	Empathy, Professional	11.5
Cinar, 201235	Turkey	Emerg Med; n = 20	General	9 h	---	---	+	Comm Ratings	10.5
Gelfman, 201441	US	Mixed; n = 9	Multiple	2 d	---	---	No Sig Testing	---	12
Green, 201443	US	Int Med; n = 31	H Lit	6 h	+, NS	Plain Language, Other	---	---	11.5
Hart, 200645	US	Int Med and Peds; n = 28	General	60–90 m	+, NS	Overall, Individual	NS	Ratings	14.5
Hershberger, 200846	US	Family Med; NR	Cultural	---	NS	General	NS	Comm Ratings	7.5
Hymowitz, 200150,g	US	Peds; n = 27	Sm Cess	4 h	---	---	+	Tobacco	10.5
Kramer, 200453	Netherlands	Gen Practice; n = 25	General	2 h per weeki	+, MF	Overall, Individual	---	---	14
Newcomb, 201855	US	Surgery; n = 27	Multiple	40 h	---	---	+	Ratings	13.5
Noordam, 201956	Netherlands	Mixed; n = 9	PCC, Empathy	3 d	NS	General	+	Empathy	13.5
Oh, 200157	US	Int Med; n = 14	General	---	MF	General	---	---	11
Oladeru, 201758	US	Mixed; n = 44	General	2 h	---	---	NS	Via the CGCAHPS/HCAHPS	12.5
Peterson, 201659	US	Peds; n = 43	General	---	MF	General	---	---	10
Reinders, 201061,g	US	Gen Practice; n = 30	General	---	+	Overall	---	---	13.5

^aParticipants abbreviations: Int Med, internal medicine; Peds, pediatrics; Emerg Med, emergency medicine; Family Med, family medicine; NR, not reported; Gen Practice, general practice.

^bContent abbreviations: BN, giving bad news; Pall, palliative care; General, general communication skills; Rel Com, relational communication; Obes, obesity; MI, motivational interviewing; Multiple, multiple topics (more than 3); EOL, end of life; LEP, limited English proficiency; Referral, specialty referral; Dth Not, death notification; H Lit, health literacy; Cultural, cultural proficiency; Sm Cess, smoking cessation; PCC, patient-centered communication.

^cFindings abbreviations: +, positive findings (skills improved); MF, mixed findings; NS, no significant findings; −, negative findings (skills worsened); No Sig Testing, no significance testing was conducted. If a study reported on multiple skill outcomes within a skill set or domain and 75% or more of those showed an increase, these were coded as positive findings.

^dSkills measured abbreviations: BN, giving bad news; Pall, palliative care; General, general communication skills; Sat Ratings, satisfaction ratings; Cues, acknowledged patient cues and concerns; MI, motivational interviewing; Overall, total score on assessment tool; Individual, individual skills; EOL, end of life; Agenda, agenda setting skills; Questioning, asking open or clarifying questions; Comm Ratings, overall mean communication ratings; Competency, skill acquisition/competence; Professional, professionalism skills; Plain Language, no medical jargon; Other, teach back and encouraging questions; Ratings, global ratings on a specific skill; Tobacco, communicating about tobacco cessation; CGCAHPS/HCAHPS, Clinician and Group Consumer Assessment of Healthcare Providers and Systems/Hospital Consumer Assessment of Healthcare Providers and Systems.

^eA modified version of the MERSQI (Medical Education Research Study Quality Instrument) was used, so scores are on a scale of 7–17, where higher scores indicate better quality.

^fThis study reported on evaluations in both settings, so it is included in each portion of the table.

^gThese were control group studies, but for the purposes of this review, only a single-group portion was applicable.

^hThe simulated or standardized patient and the faculty observer worked together to come up with one score in this study.

ⁱThe authors of the study reported that the program lasted 3 years.

Nonrandomized control group studies

Table 3 summarizes the 13 studies included in the review that used a nonrandomized control group,^68–80 with 10 studies using a simulated evaluation and 3 using a clinical evaluation. In simulated settings, 4 studies used outside observers,^68,69,74,80 with only 1 study showing positive findings in at least one skill set or domain. Six studies used SPs as evaluators in a simulated setting,^{71,73,75,76,78,79} with 4 demonstrating positive findings in at least one skill set or domain. In clinical settings, both the 2 studies using outside observers^70,77 and the 1 study using patients or family members had positive findings in at least one skill set or domain.⁷²

Table 3.

Nonrandomized Control Studies (n = 13) Included in a 2021 Systematic Review Examining the Effect of Experiential Communication Skills Education on Graduate Medical Education Trainees’ Communication Behaviors

First author, yearref	Country	Participantsa,b	Contentc	Length (in minutes [m], hours [h], or days [d])	Outside observer (faculty or trained coder)		Simulated or standardized patient (simulated settings) or patient or family member (clinical settings)		MERSQI scoref
					Findingsd	Skills measurede	Findingsd	Skills measurede
Simulated setting (n = 10)
Alexander, 200668	US	Int Med; n = 37/19	Multiple	16 h	+, NS	BN, Pt Pref	---	---	13.5
Bradley, 201069	US	Surgery; n = 7/6	Pall, EOL	6 h	NS	Pall	---	---	13.5
Hochberg, 201671	US	Surgery; n = 15/16	General	6 h	---	---	+	Overall	12
Lee, 200473	US	Peds; n = 12/14	Sm Cess	2 h	---	---	+, NS	Provision, Quality	13
MacLeod, 200874	US	Mixed; n = 15/23	Alc Int	8 h	MF	Individual	---	---	13
McCallister, 201575	US	Pulmonary or CC; n = 11/5	Fam Mtg	3 h	---	---	+	Overall	11.5
Mukerji, 201776	Canada	Int Med and Peds; n = 57/26	Testing	2.5 h	---	---	NS	Overall	14
Pagels, 201578	US	Family Med; n = 18/7	H Lit	90 m	---	---	NS	Overall	10.5
Silva, 200879	US	Peds; n = 5/6	BN	4 h	---	---	+	BN	9.5
Slort, 201480	Netherlands	Gen Practice; n = 54/51	Pall	6 h	NS	Pall	---	---	13.5
Clinical setting (n = 3)
Dow, 200770	US	Int Med; n = 14/6	Empathy	6 h	+	Empathy	---	---	14
Jay, 201072	US	Int Med; n = 12/11	Obes	5 h	---	---	+, NS	Quality, Provision	14
Norfolk, 200977	England	Gen Practice; n = 37/10	Rapp, Empathy	1.5 d	+	Rapp, Empathy	---	---	14

^aSample sizes are presented as intervention/control.

^bParticipants abbreviations: Int Med, internal medicine; Peds, pediatrics; CC, critical care; Family Med, family medicine; Gen Practice, general practice.

^cContent abbreviations: Multiple, multiple topics (more than 3); Pall, palliative care; EOL, end of life; General, general communication skills; Sm Cess, smoking cessation; Alc Int, alcohol intervention; Fam Mtg, family meeting; Testing, diagnostic or unnecessary testing; H Lit, health literacy; BN, giving bad news; Obes, obesity; Rapp, rapport.

^dFindings abbreviations: +, positive findings (skills improved); NS, no significant findings; MF, mixed findings. If a study reported on multiple skill outcomes within a skill set or domain and 75% or more of those showed an increase, these were coded as positive findings.

^eSkills measured abbreviations: BN, giving bad news; Pt Pref, assessing patient preferences; Pall, palliative care; Overall, total score on assessment tool; Provision, whether counseling was provided; Quality, quality of counseling provided; Individual, individual skills; Rapp, rapport.

^fA modified version of the MERSQI (Medical Education Research Study Quality Instrument) was used, so scores are on a scale of 7–17, where higher scores indicate better quality.

Randomized control group studies

Table 4 summarizes the 20 randomized control studies,^81–100 with 12 studies using a simulated evaluation and 8 using a clinical evaluation. In simulated settings, 8 studies used outside observers,^{84,87,91,95–97,99,100} with 4 having positive findings in at least one skill set or domain. Six studies used SP evaluators in a simulated setting,^{84,85,87,89,93,98} with 1 having positive findings in at least one skill set or domain. In clinical settings, 2 studies used outside observers;^81,92 neither had positive findings. Eight studies used patients or family members in a clinical setting,^{81–83,86,88,90,92,94} with 7 having positive findings in at least one skill set or domain.

Table 4.

Randomized Control Studies (n = 20) Included in a 2021 Systematic Review Examining the Effect of Experiential Communication Skills Education on Graduate Medical Education Trainees’ Communication Behaviors

First author, yearref	Country	Participantsa,b	Contentc	Length (in hours [h] or days [d])	Outside observer (faculty or trained coder)		Simulated or standardized patient (simulated settings) or patient or family member (clinical settings)		MERSQI scoref
					Findingsd	Skills measurede	Findingsd	Skills measurede
Simulated setting (n = 12)
Downar, 201784	Canada	Int Med; n = 39/34	EOL	---	NSg	Empathy	NSg	Empathy	15
Duran, 202185	US	Mixed; n = 15/15	Testing	1.5 h	---	---	NS	Empathy	13.5
Fenton, 201687,h	US	Mixed; n = 30/31	Testing	1 h	NS	PCC	+	Sat Ratings	14
Hilgenberg, 201989	US	Peds; n = 43/41	Anger	1.5 h	---	---	NS	Anger Ratings	14.5
Liénard, 201091	Belgium	Oncology; n = 50/48	BN, General	30 h	+, MF	Pre-delivery, Individual	---	---	13.5
Lozano, 201093	US	Peds; n = 8/8	MI	9 h	---	---	NS	MI	14.5
Merckaert, 201395	Belgium	Oncology; n = 48/47	BN, General	30 h	+, MF	Pre-delivery, Individual	---	---	13.5
Moral, 200396	Spain	Family Med; n = 94/71	General	28 h	NS	General	---	---	15.5
Nikendei, 201197	Germany	Peds; n = 14/14	PCI	---	MF	Individual	---	---	13.5
Steinemann, 200598	US	Surgery; n = 6/10	Sm Cess	4 h	---	---	NS	Use of 5As	13
Szmuilowicz, 201099	US	Int Med; n = 23/26	EOL	1 d	+, MF	Overall, Individual	---	---	14.5
Szmuilowicz, 2012100	US	Int Med; n = 19/19	DNR	4 h	+, MF	Overall, Individual	---	---	14.5
Clinical setting (n = 8)
Bashour, 201381	Syria	OBGYN; n = 137/0i	General	20 h	NS	General	NS	Sat Ratings	14
Chossis, 200782	Switzerland	Gen Practice; n = 13/13	Alc Int	1 d	---	---	+, MF	Overall, Individual	13.5
Cornuz, 200283	Switzerland	Mixed; n = 17/18	Sm Cess	1 d	---	---	+, MF	Sm Cess, Individual	12.5
Feigelman, 201186	US	Peds; n = 47/44	Psy Risk	8 h	---	---	+	Sat Ratings	14
Gielen, 200188	US	Peds; n = 18/13	General	6 h	---	---	+	Overall, Sat Ratings	12
Katz, 200890	US	Int Med; n = 29/36	Phys Act	7.5 h	---	---	+	Provision	13
Liénard, 201092	Belgium	Oncology; n = 46/42	BN, General	30 h	MF	Individual	+	Sat Ratings	14
Marsh, 202194	US	Peds; n = 19/17	BN	---	---	---	+	Overall	14

^aSample sizes are presented as intervention/control.

^bParticipants abbreviations: Int Med, internal medicine; Peds, pediatrics; Family Med, family medicine; OBGYN, obstetrics and gynecology; Gen Practice, general practice.

^cContent abbreviations: EOL, end of life; Testing, diagnostic or unnecessary testing; Anger, anger de-escalation; BN, giving bad news; General, general communication skills; MI, motivational interviewing; PCI, patient-centered interviewing; Sm Cess, smoking cessation; DNR, code status discussions; Alc Int, alcohol intervention; Psy Risk, psychological risk screening; Phys Act, physical activity.

^dFindings abbreviations: NS, no significant findings; +, positive findings (skills improved); MF, mixed findings. If a study reported on multiple skill outcomes within a skill set or domain and 75% or more of those showed an increase, these were coded as positive findings.

^eSkills measured abbreviations: PCC, patient-centered communication; Sat Ratings, satisfaction ratings; Anger, anger de-escalation; Pre-delivery, the first phase of breaking bad news delivery; Individual, individual skills; MI, motivational interviewing; General, general communication skills; 5As, ask, advise, assess, assist, arrange checklist; Overall, total score on assessment tool; Sm Cess, smoking cessation; Provision, whether counseling was provided.

^fA modified version of the MERSQI (Medical Education Research Study Quality Instrument) was used, so scores are on a scale of 7–17, where higher scores indicate better quality.

^gResults based on the mean of outside observer and standardized patient scores on the empathy measure.

^hThis study used unannounced standardized patients.

ⁱAll residents received the intervention for this study, which used a stepped cluster randomized design.

Quality assessment

Each included study’s MERSQI score is noted in Tables 2–4. The mean MERSQI score for all included studies was 12.8 (standard deviation [SD] = 1.4). The single-group, pre-post studies had a mean score of 12.3 (SD = 1.4), the nonrandomized control studies had a mean score of 12.8 (SD = 1.5), and the randomized control studies had a mean score of 13.8 (SD = 0.8).

Positive findings by salient characteristics

We tabulated the results for each of the 4 pre-determined salient characteristics noted above to describe the study results in aggregate (see Supplemental Digital Appendix 2 at [LWW INSERT LINK]).

1. Study design. Of the 44 studies using a single-group, pre-post design, 35 (80%) had positive findings. In contrast, of the 13 studies that used nonrandomized control group designs, 8 (62%) had positive findings, and of the 20 studies that used randomized control group designs, 11 (55%) had positive findings.

2. Evaluation setting. Of the 51 studies that used simulated evaluation settings, 34 (67%) had positive findings. Of the 27 studies that used a clinical evaluation setting, 21 (78%) had positive findings.

3. Evaluator type. Of the 49 studies that used an outside observer as an evaluator, 31 (63%) had positive findings. Of the 42 studies that used an SP, patient, or family member as an evaluator, 27 (64%) had positive findings.

4. Length of training. Finally, 71 (92%) studies listed the number of hours spent in the communication educational activity. Of the 30 studies that reported spending 7 hours or more, 22 (73%) reported positive findings. Of the 41 studies that reported spending less than 7 hours, 31 (76%) reported positive findings.

Discussion

Communication skills education is an important part of GME training. A better understanding of the effectiveness of this education in changing communication behavior is critical to improving training and ultimately improving patient care. In this study, we used rigorous systematic review methods to address our aim of examining the effect of experiential communication skills education on GME trainees’ communication behaviors. We reviewed 77 papers reporting on experiential communication skills education for GME learners. We found that the answer to our aim is complex and confounded by the heterogeneity of study design, settings, and evaluators. However, overall, our synthesis demonstrated that most studies reported some positive communication behavior change. Four clear patterns emerged descriptively in the results.

First, more than half of the studies used a single-group, pre-post design, and positive findings were more common in this type of study design, which is less rigorous than control group study designs. This is in line with an earlier review of postgraduate communication skills education¹⁰¹ that showed that higher-quality studies produced fewer positive effects than lower-quality studies. Bias may be more likely in studies where post-assessment occurs weeks or months after the educational intervention, as there is no way to account for the causal element of time passing on learner communication behaviors, particularly during the formative years of GME training.

Second, across all study design types, positive findings were likely in both simulated settings (i.e., Kirkpatrick’s level 2) and clinical settings (i.e., Kirkpatrick’s level 3). As the Kirkpatrick Model proposes that changing behaviors at higher levels is more complex than at lower levels, we had expected to see more positive results in studies using simulated evaluation than in those using clinical evaluation. There may be several reasons for this result. First, this review summarizes studies that reported findings from either simulated or clinical evaluation settings with only 1 study³¹ reporting findings at both levels. Had there been more studies reporting findings at both levels, we may have noted different results. Another potential reason for this result is that evaluations in simulated and clinical settings both have strengths and limitations. For example, clinical observation only provides a snapshot of learner behaviors and may be influenced by what and how many encounters are able to be observed and evaluated, as compared to the ability to standardize encounters in simulated settings, such as objective structured clinical examinations. However, such simulated settings are critiqued for measuring exam performance rather than actual skills that learners may use in real patient encounters.¹⁰² A final potential explanation is that behavior change may not be more complex in clinical settings than in simulated settings, as Kirkpatrick’s Model would suggest, but instead that these 2 settings allow different insights into residents’ communication behaviors. For example, Goch et al have proposed that simulation provides more insight into challenging and less common situations, while clinical evaluation provides more insight into daily, real-life encounters.¹⁰³

Third, another clear pattern was that positive findings were likely in both studies that used outside observers (i.e., faculty or trained coders) to evaluate trainees and studies that used participants (i.e., SPs or patients or family members) as evaluators. A number of studies comparing types of evaluators in both UME and GME demonstrated that outside observers do evaluate trainees differently than patients, family members, or SPs.^104–107 However, and in support of our findings, there is not a clear consensus on whether outside observers or patients, family members, and SPs provide higher ratings.

Finally, studies that had both shorter (less than 7 hours) and longer (7 hours or more) communication educational activities had at least some positive findings. The question of the necessary length of time (sometimes called “dose”) of communication education is debated in the research literature. A European consensus paper and systematic review indicated that a minimum of 3 days of participation in a communication skills course is necessary for participants to demonstrate the use of skills in a clinical setting;¹⁰⁸ however, our review demonstrated that the majority of studies showed positive results in a clinical setting even though they ranged in length.

Based on the findings of this systematic review, we have 3 recommendations for those who conduct future research on GME communication skills education.

1. Limit the use of single-group studies. Single-group studies are of lower quality and may have more bias than control group studies. The question of whether communication skills education studies can show change in a single-group, pre-post study has been asked and answered many times. Instead, the literature needs more controlled trials to have more rigorous results. Although randomized controlled trials may be difficult, many studies included in this review used wait list or historical controls, which can be a more feasible alternative. However, we also recognize that single-group studies may be useful for filling gaps in the literature related to specific specialties, innovative educational methods, or less frequently examined communication skills content.

2. Measure in multiple settings (i.e., at multiple Kirkpatrick levels). Being able to report findings of one intervention in both simulated and clinical evaluation settings (i.e., Kirkpatrick Levels 2 and 3) is a way of evaluating the transfer of learning from the education setting to the workplace setting. Transfer has been noted by other authors as a challenge at the UME and GME levels.^13,109 Communication skills should be assessed in both simulated and clinical settings whenever possible. Otherwise, it is challenging to understand the factors and conditions that might impact transfer.

3. Share curricula and evaluation measures. More homogeneity in curricula and evaluation measures will allow for a better understanding of how GME communication skills education works through meta-analyses. Rather than developing new models of communication skills education and new evaluation measures for each GME program, oversight groups, such as national professional organizations or educational groups in specialties or subspecialties, could develop curricula and evaluation measures to be used and tested in multiple GME programs. Just as the ACGME Milestone Project¹¹⁰ has delineated a more detailed explanation of what competencies residents need to demonstrate, having a parallel communication skills curriculum for training could be effective and allow for comparison across programs. There are many examples of well-developed curricula and evaluation measures including those described in the articles reviewed in this review, in MedEdPORTAL, and developed by organizations focused on communication in health care, such as EACH: International Association for Communication in Healthcare.

Findings and implications from this study may be even more important given the recent halt of the United States Medical Licensing Examination Step 2 Clinical Skills assessment. As a result of this policy change, GME trainees may have variable levels of experience with communication skills—and, in particular, with their communication skills being assessed—during their medical school training. Implementation of communication skills curricula and assessments in GME programs may help to fill this gap and make sure programs are producing physicians who can effectively communicate, which is increasingly emphasized as important to public interests.

On the other end of the medical education spectrum, it is reasonable to expect that these findings would apply to CME communication skills programs as well. Teaching communication skills to practicing clinicians requires the same experiential approach we have emphasized in this review, since primarily passive didactic approaches have been shown to have little impact on clinicians’ behaviors.^111–113

One limitation of this study is our inability to conduct a meta-analysis or a comparative efficacy evaluation due to the heterogeneity in communication trainings and evaluation measures, even among subgroups. Despite our efforts to analyze by study design, evaluation setting, and evaluator type, there was still significant heterogeneity in key variables, such as length of training. Another limitation is that publication bias may have led to the findings reported being more positive than they would be in real-world settings and/or if we had also included unpublished reports. Finally, because we were not able to meta-analyze effect sizes, the extent to which these studies’ findings have practical significance is unclear.¹¹⁴

Future research could focus on a realist review to determine what types of education (e.g., content, length) work best for positive outcomes.¹¹⁵ Another potentially fruitful area of GME communication skills research could focus on further understanding the finding reported in a couple of the included studies that the impact of communication skills courses is significantly related to trainees’ baseline scores, with those who had lower baseline skill scores improving more.^26,31 Given these findings, focusing more educational efforts on those who begin training with lower skill levels, rather than using a one-size-fits-all approach, may be a better use of limited resources.

Conclusions

The importance of communication skills education for GME trainees is well-accepted and a topic of great interest around the world. Our synthesis of these 77 studies demonstrates strong support that experiential communication skills education can impact GME trainees’ communication behaviors. It also demonstrates that while research has provided a strong foundation for exploring communication skills education in GME, there remain opportunities to deepen our understanding with future research.