Using a computer simulation for teaching communication skills: A blinded multisite mixed methods randomized controlled trial

Abstract

Objectives

To assess advanced communication skills among second-year medical students exposed either to a computer simulation (MPathic-VR) featuring virtual humans, or to a multimedia computer-based learning module, and to understand each group’s experiences and learning preferences.

Methods

A single-blinded, mixed methods, randomized, multisite trial compared MPathic-VR (N=210) to computer-based learning (N=211). Primary outcomes: communication scores during repeat interactions with MPathic-VR’s intercultural and interprofessional communication scenarios and scores on a subsequent advanced communication skills objective structured clinical examination (OSCE). Multivariate analysis of variance was used to compare outcomes. Secondary outcomes: student attitude surveys and qualitative assessments of their experiences with MPathic-VR or computer-based learning.

Results

MPathic-VR-trained students improved their intercultural and interprofessional communication performance between their first and second interactions with each scenario. They also achieved significantly higher composite scores on the OSCE than computer-based learning-trained students. Attitudes and experiences were more positive among students trained with MPathic-VR, who valued its providing immediate feedback, teaching nonverbal communication skills, and preparing them for emotion-charged patient encounters.

Conclusions

MPathic-VR was effective in training advanced communication skills and in enabling knowledge transfer into a more realistic clinical situation.

Practice Implications

MPathic-VR’s virtual human simulation offers an effective and engaging means of advanced communication training.

1. INTRODUCTION

Communication is the most important component of the doctor-patient encounter [1, 2]. Evidence confirms that poor clinician communication skill is associated with lower levels of patient satisfaction, higher rates of complaints, poorer health outcomes, and an increased risk of malpractice claims [3–20]. Failure of empathic communication also results in unnecessary return visits, unnecessary and unwanted somatic treatments, excessive diagnostic testing, missed diagnoses, symptom amplification, and missed opportunities for reassurance and appropriate counseling [21–25].

Communication between and across healthcare teams is also crucial for safe and effective patient care. Among healthcare professionals, communication failures in the hospital setting are consistently the most frequent contributors to sentinel events reported to the Joint Commission [22]. Reducing the potential for adverse patient events requires that interprofessional communication meet the same standard for empathy and respect as clinician-patient communication [23–26].

Acknowledgment that good communication skills are essential for high quality, cost-effective, collegial, and safe medical practice [21, 27–30] has led to widespread support for early introduction and training of communication skills in medical education [31–35]. However, since communication between doctor and patient is a complex phenomenon with many different factors interacting simultaneously, [1, 36] effective communication assessment and training is correspondingly complex. Communication involves both cognitive and affective domains, and is mediated through verbal and nonverbal channels [1, 37, 38]. Over the past 60 years, various coding methods have been developed to analyze the many elements of medical encounters. Although existing methods can provide a detailed understanding of communication dynamics, they are resource-intensive, logistically challenging, and impractical for mainstream education [37, 39–51]. For example, current teaching methods typically include small groups of learners, with a focus on role-playing with each other or with simulated patients. However, this is resource intensive, and with different trainers, discrepancies between groups can appear. Choosing the most suitable trainer for communication skills is difficult, as is the selection and training of simulated patients [52]. Research on clinical communication training demonstrating efficacy and sustained effects is sparse [53]; most studies do not involve a comparison or control condition, and even fewer involve a randomized controlled trial [54]. These challenges underscore the need for the creation and study of practical, innovative methods to help learners master the complexity of healthcare communication, and develop excellent communication skills that will meet current and future competency-oriented accreditation standards [55].

MPathic-VR (an acronym derived from the grant Modeling Professionalism and Teaching Humanistic Communication in Virtual Reality, NIH 5R44TR000360-04/2R44CA141987-02) is a computer-based system designed to address this need. MPathic-VR teaches healthcare learners to handle challenging conversations by enabling them to talk with virtual humans. MPathic-VR’s virtual humans are intelligent conversational agents with human appearance and the capacity to interact using a wide range of communication behaviors that one would expect in face-to-face conversation between humans [56–60]. As learners talk with virtual humans, they are challenged to interpret the virtual humans’ verbal and nonverbal communication, and respond with communication strategies that drive desired outcomes. MPathic-VR records and stores learners’ conversational choices and nonverbal behaviors. Analyses of these data drive assessment and feedback functions, and enable real-time variation of virtual human behavior during the simulation.

1.1 Development Considerations

Creating an effective learning experience required taking many factors into account. These include: building the backbone of the system on specific communication skill learning objectives and techniques identified in the medical literature, creating an experiential-based learning environment sufficiently similar to the real challenges that learners face, providing appropriate feedback in a timely fashion, providing encouragement to the learner, supporting reflection and practice, and considering characteristics that facilitate transfer.

As a foundation, MPathic-VR was designed to provide learners with a toolkit of useful skills [61]. Each conversational exchange between the learners and virtual humans is based on learning objectives directed at specific communication skills including: reflective listening, empathy enhancers, avoiding empathy blockers, appropriate use of facial expression (i.e., brow raises, smiles) or body language (i.e., nodding, body lean), which support the development of rapport [62]. Learning objectives were drawn from established communication protocols, such as SPIKES [63], CRASH [64], and TeamSTEPPS [65, 66]. SPIKES (Set-up, Perception, Invitation, Knowledge, Emotion, Summary) emphasizes principles for breaking bad news, CRASH (Culture, Respect, Assess and Sensitivity and Self-awareness, Humility) emphasizes principles of cultural competence, and TeamSTEPPS (Team Strategies and Tools to Enhance Performance and Patient Safety) emphasizes principles for effective interprofessional communication. These skills align with many of those detailed in the Calgary-Cambridge guide [67, 68], but the MPathic-VR virtual human simulation is not solely skills-based. It also allows for creativity, because learners can view themselves in conversation with virtual humans and repeat interactions, during which they are free to experiment with different dialogue, expressions, and body language [69]. The system also encourages reflection during (reflection-in-action) and after (reflection-on-action) learners' interaction with virtual humans, guided by theories first introduced by Dewey [70] and advanced by Argyris and Shön [71–77], as a means to promote the development of adaptive expertise [78–80]. This acknowledges calls for integrating reflection into communication training [61].

These elements are incorporated within a simulation-based medical education (SBME) framework for effective learning, elements of which include context authenticity, consistent and precise measurement that informs individualized learner feedback, appropriate simulation fidelity, sequence of instruction, and opportunity for deliberate practice [81–84]. The system is grounded in the theory of multimedia learning [85], which holds that people learn better through words and pictures than through either alone. Last, it is further guided by an interactive instructional approach [86, 87] that stresses a dynamic relationship between the learner and the learning system, and integrates system-based elements that have the potential to engage the learner with the behavioral, cognitive, and emotional activities of the learner. This contrasts to other multimedia learning activities that might be termed interactive, but do not consider the integration of these components.

For the Print Version of this Article: To demonstrate MPathic-VR in use, a video component is available. The link to the demonstration video is incorporated into the caption of the image visible below.

For the Electronic Version of this article: To demonstrate MPathic-VR in use, a video component is available and accompanies the electronic version of this manuscript. To access this video component, simply click on the image visible below.

1.2 Research Hypotheses

To examine whether MPathic-VR is useful for teaching advanced communication skills, the investigators developed and tested the following hypotheses: 1) students randomized to learn with MPathic-VR would improve their communication performance after engaging in a communication scenario, receiving feedback on their performance, and then applying the feedback in a second run-through of the scenario; and 2) knowledge acquired through MPathic-VR would be resilient (i.e., students would incorporate learned materials into their manner of communication), and that the performance of MPathic-VR-trained students assessed in a subsequent advanced communication OSCE would be scored higher than students trained with a conventional, widely-used multimedia method, computer-based learning (CBL). The investigators asked the mixed methods research question, how do qualitative findings from students’ reflective comments and responses to an attitudinal survey compare for the MPathic-VR and the CBL experiences?

2. METHODS

2.1 Design

Investigators conducted a single-blinded, mixed methods, randomized controlled trial at three medical schools. Framed by an ethnographic approach, investigators researched students’ experiences when taking the modules. The Institutional Review Boards of all participating medical schools approved this research.

2.2 Setting

The studies were conducted at three US medical schools: Eastern Virginia Medical School (EVMS); the University of Michigan Medical School (UM); and the University of Virginia School of Medicine (UVA).

2.3 Participants

All second-year medical students (N=481) were eligible for the study. While 435 enrolled, 421 (87.5%) completed the trial (Figure 1). At EVMS and UM, the trial was a required activity in the educational curriculum; at UVA, participation was voluntary.

Figure 1.

Participant flow through the MPathic-VR trial

2.4 Intervention Group

Participating students randomized to the MPathic-VR intervention assumed the role of an intern in two thematically linked scenarios. The first scenario emphasized intercultural communication between a young woman with a new diagnosis of acute myelogenous leukemia (Robin, a virtual human), her traditional, El Salvadorian mother (Delmy, a virtual human), and the student learner. The learner had to break bad news to Delmy about Robin’s leukemia in a family meeting, and mediate tensions arising from the different cultural values of mother and daughter. Learning objectives were guided by CRASH principles for cultural competence [64], and by the SPIKES protocol for delivering bad news [63].

The second scenario focused on interprofessional communication. As the student learner left Robin’s room, the oncology nurse caring for Robin (Nicole, a virtual human), signaled the learner to meet her in a nearby conference room. The ensuing discussion involved conflict resolution between the learner and Nicole, who was angered to discover that the learner inadvertently omitted her from the family meeting with Delmy and Robin. Learning objectives for this scenario were developed using the TeamSTEPPS teamwork system developed by the US Department of Defense, in partnership with the US Agency for Healthcare Research and Quality. [66]. Evidence-based cognitive, verbal [88–92], and nonverbal [93–107] communication principals were integrated into both scenarios.

Intercultural and interprofessional communication were selected as the foci of training for multiple reasons. First, they are among ACGME recommended competencies. Second, for both interprofessional and intercultural communication, there is strong evidence supporting these principles. Third, the authors have previous expertise in these areas (e.g., Fetters [108–114] and Marsella [115, 116] in intercultural communication, Scerbo in interprofessional communication [117, 118]). Fourth, these skills were conducive to developing the overarching structure of MPathic-VR’s interactive narrative. Fifth, our medical educator collaborators felt that these skills would add value to their existing curricula.

Figure 2 illustrates student learners’ progress through MPathic-VR. They first viewed a multimedia presentation about general communication principles, and then took a readiness assessment quiz. Students had to achieve a score of 80% or higher to participate. The first scenario addressed intercultural communication. After the virtual humans spoke, students had to choose from three possible responses shown as text on the screen, and speak one of them back to the virtual humans. The best of the three options scored 0 points; the two suboptimal options had higher point values. The pathway through the scenario depended in part on the student’s responses. After completing the first run-through, students received an after-action review (AAR) that included personalized feedback on performance. The AAR encouraged reflective learning by presenting the evidence behind specific verbal choices, the consequences of the choices, and offering suggestions for improvement without specifically indicating which choices were correct or incorrect. In the AAR, students also observed their nonverbal behaviors via a video recording showing them in conversation with the virtual humans, and received feedback on certain nonverbal behaviors detected by the MPathic-VR system, such as nodding, brow-raising, and smiling. Because studies have shown that providing more general information facilitates learning at a deeper level and transfer of knowledge to other contexts, the AAR feedback addressed general principles of effective communication rather than students’ specific choices and behaviors [119, 120]. Students then completed a second run-through of the intercultural communication scenario. Next, they transitioned to the interprofessional communication scenario, which followed the same sequence of an initial run-through, a personalized AAR, and a second run-through to enable students to apply knowledge acquired during the AAR. The intercultural scenario included 16 exchanges (0 to 29 points), and the interprofessional scenario had 13 exchanges (0 to 25 points) that enabled MPathic-VR to calculate performance. A lower score in MPathic-VR reflects better performance, as less optimal choices were penalized with higher scores. Finally, students completed twelve 7-point Likert-format items on an attitudinal survey, and wrote a brief reflective essay to allow investigators to understand the meaning of their experiences.

Figure 2.

Experience flow through the MPathic-VR computer simulation and the Computer-Based Learning control

2.5 Control Group

The computer-based learning (CBL) module used in the control group was a current, open-ware “Introduction to Standardized Communication for Health Professionals” program developed and used at one of the test sites (UVA). The module was chosen to have clinically relevant content and comparable training times for students in the two arms of the study. The CBL module represents the current standard for multimedia training using a self-paced presentation of text, images, and video. The CBL module delivered teaching principles about interprofessional communications to ensure patient safety, including the need for standardized communication based on the principles of SBAR (Situation, Background, Assessment, Recommendation), and other rubrics designed to support health professionals in patient hand-offs. It presented illustrative patient scenarios and videos demonstrating ineffective and effective communication between doctors and nurses. Students in the control group first took and achieved a passing score on the same quiz as the intervention group. After completing the CBL module, students completed the same attitudinal survey questions as the MPathic-VR students, and similarly wrote a short reflective essay (Figure 2).

Regarding the portions of MPathic-VR and CBL dedicated to interprofessional communication training (the skills assessed in a subsequent OSCE (Objective Structured Clinical Examination) – see below), the average time on task was 23 minutes, 46 seconds for MPathic-VR, and 22 minutes, 25 seconds for the CBL module. As noted above, MPathic-VR included an additional module on intercultural communication with an average time on task of 24 minutes, 25 seconds.

2.6 Advanced Communications Skills OSCE

All participants were evaluated on the same OSCE scenario at each of the three schools. The schools coordinated their training efforts to ensure consistency in implementation. In aggregate, thirty-two standardized patient instructors (SPIs) were trained on an interprofessional OSCE scenario in which they portrayed surgical assistant trainees required to scrub in on a surgical case to complete their training requirements. After the surgery, the SPI angrily confronts the learner, who unwittingly took her spot in the operating room. Although the underlying learning objectives were the same as those from the MPathic-VR interprofessional scenario, the story and context were novel. The communication skills required by the OSCE were characterized as advanced because they called for levels of nuance and complexity that are not usually assessed with second-year medical students, although the results (below) indicated that they were indeed capable of learning these materials.

SPIs were blinded to student exposure (intervention and control) during the trial. They rated students from both arms using a 5-point grading format that addressed four domains drawn from interprofessional communication learning objectives: openness/defensiveness, collaborative/competitive, nonverbal communication, and presence (meant here to connote an awareness of others) [65]. In contrast to MPathic-VR scoring, in the advanced communication OSCEs, higher scores represent better performance.

Importantly, students were unaware that their earlier experiences with either MPathic-VR or CBL would be tested in the OSCE station, which was held several days after the intervention. This minimized the opportunity for learners to rehearse information in preparation for the OSCE.

2.7 Data Collection

The medical schools supplied participant demographic information through an Honest Broker system [121] that utilized one unique identifier for the study and one from each medical school. This ensured that participant responses were de-identified.

2.8 Attitudinal Survey

Students in the intervention and control groups answered the same 12 items on an attitudinal survey. The purpose of the attitudinal survey was to obtain participants’ initial reactions and provide feedback about MPathic-VR. It was not measuring a construct or constructs. In addition, the survey was specific to the trial and not intended to be used in any other context. Therefore, a validation study was not indicated. The items were organized into four general areas: clarity, purpose, utility, and likelihood to recommend the learning experience to other. Items used a 7-point Likert response anchored by “Strongly Disagree” to “Strongly Agree.”

2.9 Reflective Essay

Qualitative data were collected from both groups in a short reflective essay written at the end of training [122]. Students were randomized to different questions. CBL questions included, “Reflect on how you think this learning experience in advanced communication skills could be improved” or “Reflect about the three most important things you learned from this interaction.” The MPathic-VR question pool additionally included “Reflect on how interacting with the system has influenced your views about human interactions, e.g., interprofessional, patient-provider, family-provider, patient-family” and “Reflect on how interacting with the system has influenced your understanding about nonverbal communication.” Questions posed to the MPathic-VR and CBL group were comparable with the exception of the question about interactivity, which only applied to MPathic-VR.

2.10 Quantitative Data Analytics

Descriptive statistics were calculated for all demographic items. For the first hypothesis regarding improvement during the MPathic-VR simulation, investigators compared scores for each run-through of the intercultural and interprofessional scenarios with a repeated-measures analysis of variance (ANOVA) to assess learning derived from the additional practice with the system. The MPathic-VR simulation would normally include this repeated measure as an indication of student engagement. For the second hypothesis comparing the MPathic-VR arm and the control arm, investigators conducted both a multivariate analysis of variance (MANOVA), and univariate ANOVAs, on the four OSCE rating scale items with module (intervention or control) as the independent variable. For the final hypothesis regarding student attitudes towards MPathic-VR and CBL learning, investigators compared mean scores for each module aggregated across rating scales with an independent t-test. All analyses were evaluated with an alpha level of .05 unless stated otherwise. Data were analyzed with SAS software, version 9.3 [123].

2.11 Qualitative Data Analytics

All qualitative data were entered into a single file. MAXQDA software facilitated the analysis [124]. Two investigators (LS and MF) read through the text files and developed codes. The analytic approach involved reducing the data into overarching themes [125]. After reading through the entire qualitative database, segments of text were identified and assigned a code based on an emerging coding scheme. This led to an initial codebook. Investigators reviewed and discussed each code to calibrate coding and achieve intercoder agreement [126], then refined, and clarified codes. After coding all text, they then organized related codes into the primary themes [127]. As a validation strategy, a third researcher (TG) then conducted a review of coded data. Here, the analytics focused on students’ experiences while they were taking the MPathic-VR and the CBL modules.

2.12 Mixed Methods Analysis

After completing the qualitative and quantitative analyses, the qualitative findings from learners’ reflections on their experiences were linked [128] with the quantitative results of the attitudinal scale. The purpose of the mixed methods analysis was to compare the two sources of data to gain a more complete understanding of their experiences. The analysis and interpretation are represented in a visual joint display [129].

3. RESULTS

3.1 Demographic Characteristics

The MPathic-VR group (N=210) had a mean age of 25.4 years (SD=2.6) with 104 (49.5%) females, and race distribution of 117 (55.7%) Caucasian-American, 45 (21.4%) Asian-American, 14 (6.7%) African-American, 2 (1%) Native-American/indigenous people, and 32 (15.2%) other/no response. The CBL control (N=211) had a mean age of 25.5 years (SD=2.9), with 94 (44.5%) females, and race distribution of 112 (53.1%) Caucasian-American, 40 (19.0%) Asian-American, 25 (11.8%) African-American, (1) 0.5% Native-American, and 33 (15.6%) other/no response. There were 7 (3.3%) Hispanic/Latinos in each group. There were no statistically significant demographic differences between participants in the two groups.

3.2 Student Learning During MPathic-VR

In MPathic-VR, a lower score reflected better performance. For the intercultural communication scenario, scores decreased significantly from the first run-through (M=11.67, SD=6.26) to the second run-through (M=5.89, SD=5.12), F(1,207) = 166.14, p <.0001, η² = 0.45. For the interprofessional communication scenario, scores decreased significantly from the first run-through (M=7.59, SD=3.96) to the second run-through (M=4.62, SD=2.54), F(1,207) = 104.64, p <0.0001, η² = 0.36. Thus, students successfully learned how to improve their communication skills for both the intercultural and interprofessional scenarios.

3.3 Comparison of Communication Skills on the Advanced Communication OSCE

A higher score on the advanced communication OSCE represented better performance. A MANOVA showed a main effect for module, Pillai’s trace=0.04, F(4, 411)=4.08, p=.003, η² =0.0382. A post hoc univariate analysis was conducted with α=.05/4=.0125, revealing a main effect for the nonverbal communication scale, F(1,414)=13.70, p=.0002, η² =0.0320. However, as can be seen in Table 1, all of the means for the MPathic-VR students were higher than those of the CBL students, and some of the univariate effects approached significance. The investigators therefore created a global composite from the four OSCE rating scale items and conducted an ANOVA on the global composite. A composite represents the individual items as a whole, which is reflected by the correlations between the items. Coefficient alpha [130] for the OSCE items was α=0.82. This analysis indicated a main effect for module, F(1, 414)=6.09, p=.0140, η² =0.0145. Thus, OSCE evaluators rated the communication skills of MPathic-VR-trained students significantly higher (M=.806, SD=.201), than CBL students (M=.752, SD=.198).

Table 1.

Comparison of mean student scores on the advanced communication OSCE between the MPathic-VR computer simulation and the control Computer-Based Learning module

	MPathic-VR Mean (SD)	Control Mean (SD)
Open/Defensive	.830 (.216)	.780 (.232)
Collaborative/Competitive	.757 (.255)	.707 (.276)
Nonverbal Communication	.824 (.256)*	.746 (.277)*
Presence	.811 (.241)	.774 (.238)
Mean (Global)	.806 (.201)**	.752 (.198)**

^*F(1,414)=13.70, p=.0002;

^**F(1, 414)=6.09, p=.0140

3.4 Attitudinal Rankings and Qualitative Assessments of Students’ Experiences

To be thorough, the attitudinal survey was assessed for internal consistency. Coefficient alpha for student experience items was α=0.95. The mean ratings aggregated across the 12 survey items were significantly higher (e.g., more positive) among students with MPathic-VR experience than for students with the CBL experience, t(413)=7.23, p<.0001, r² =0.1123.

Participants’ qualitative assessments noted strengths and potential enhancements for each type of training. For MPathic-VR, codes were grouped into qualitative themes: 1) students felt they learned useful verbal and nonverbal communications skills; 2) students valued the immediate feedback and engagement using video recordings of their interactions with virtual humans; 3) students recognized the value of the system to prepare and practice for emotionally-charged clinical encounters; and 4) students reflected on the clinical utility of communication and need for practice. Qualitative themes for the CBL were: 1) learners valued the system’s presentation of facts based on featured communication strategies, 2) there was a lack of interactivity, and 3) learners experienced information overload. As compared with student comments about the CBL, the MPathic-VR comments reflected a deeper understanding of communication in practice as opposed to memorizing facts, how communication skills develop through interactions, and the utility of practicing communication. Furthermore, students in the MPathic-VR condition praised the application’s interactivity, while CBL students tended to criticize the lack of interactivity in the CBL module.

Quantitative attitudinal scores and qualitative reflections for both the MPathic-VR and CBL groups were consistent as shown in Table 3. The table is organized by survey items, it merges the related attitudinal item results, and provides a typical comment from a student. For example, attitudinal scores towards verbal communication were more favorable for MPathic-VR, a difference also noted in reflective comments, which indicated deeper understanding (e.g., relating skills to their own practice) for MPathic-VR relative to the CBL, where comments focused on mnemonic devices learned. Differences in quantitative attitudinal scores confirmed the qualitative analysis in all domains except the self-assessment of improved clinical skills. Regarding the latter, intervention arm comments suggest the communication practice was perceived as more helpful in preparing for clinical work than the control arm, despite no differences in attitudinal responses.

Table 3.

A comparison of quantitative scores with qualitative reflections for the MPathic-VR intervention and computer based learning control

	MPathic-VR		CBL
Domains	Attitudinal Item Mean (SD)	Qualitative Reflection Illustrative Quotes	Attitudinal Item Mean (SD)	Qualitative Reflection Illustrative Quotes	Interpretation of mixed methods findings
Verbal Communication	5.02 (1.62)	“How to introduce myself without making assumptions about the cultural background of the patient and the family”	3.89 (1.67)	“This educational module was useful for clarifying the use of SBAR and addressing ways that all members of a health care team can improve patient care through better communication skills”	Intervention arm comments suggest deeper understanding of the content than teaching using memorization and mnemonics as in the control, a difference confirmed by higher attitudinal scores.
Nonverbal Communication	4.11 (1.85)	“Effective communication involves non-verbal facial expression like smiling and head nodding”	2.77 (1.45)	None	Intervention arm comments address the value of learning non-verbal communication, the difference confirmed by attitudinal scores.
Training was engaging	5.43 (1.55)	“Reviewing the video review was a great way to see my facial expressions and it allowed me to improve on these skills the second time around”	3.69 (1.62)	“This experience can be improved by incorporating more active participation. For example, there could have been a scenario in which we would have to select the appropriate hand- off information per SBAR guideline”	Intervention arm comments reflect engagement through the after action review while the control comments suggested the need for interaction, the difference confirmed by higher attitudinal scores.
Effectiveness in learning to handle emotionally charged situations	5.13 (1.48)	“I tend to try to smile more often than not in emotionally charged situations and that may result in conveying the wrong message”	2.34 (1.35)	“I anticipate that high-stress situations where time is exceedingly crucial requires modification to the methods presented.”	Intervention arm comments indicate awareness of communication in emotionally charged situations yet control comments indicate the need for additional training, a difference confirmed in attitudinal scores.
Improve clinical skills	4.93 (1.57)	“Practice working through tough situations that are common, but that we don't get to learn how to handle before getting on the wards”	4.62 (1.40)	“This is a useful first step but learning advanced communication skills must involve actually performing them.”	Intervention arm comments suggest the communication practice was more helpful in preparing for clinical work than the control arm, a difference not supported by attitudinal scores on the items used in the survey.

4. DISCUSSION AND CONCLUSIONS

4.1 Discussion

This study assessed the usefulness of a virtual human simulation, as embodied in MPathic-VR, for teaching advanced communication skills to second-year medical students. The investigators’ first hypothesis was that students who interacted with MPathic-VR, received feedback, and immediately applied that knowledge in a second run-through would show an improvement in scores. The results confirm this hypothesis. Students’ scores were nearly halved (i.e., they chose more appropriate statements) and improved in both the intercultural and interprofessional communication scenarios. These results are consistent with the theory that interactive learning better engages students in constructing knowledge and produces better learning outcomes [86].

Secondly, the investigators hypothesized that after an interval of several days, evaluators would rate the interprofessional communication skills of MPathic-VR-trained students higher on a subsequent OSCE station than students in the CBL group. This was confirmed as follows. Significant differences were noted on the nonverbal communication scale, suggesting that MPathic-VR was particularly valuable for acquisition of nonverbal skills. Additionally, MPathic-VR-trained students were rated higher than the control group students on all four ratings scales, and a global composite created from the four OSCE rating scales revealed a significantly higher mean for the MPathic-VR students.

These differences may be attributed to several of the theoretical elements that formed the basis for MPathic-VR’s design. One element is providing an opportunity for active learning and practice [70]. Another is interactivity, in the sense of a dynamic and reciprocal relationship between the learner and a learning system [87]. In MPathic-VR, at each dialogue exchange between the learner and the virtual human, learners were required to consider three different possible responses before being allowed to progress within the scenario. The responses appeared similar on the surface, but differed in key pedagogical constructs, and forced learners to reflect upon the merits of each [71–74]. The AAR reemphasized the communication principles for each scenario, giving students a second opportunity to reflect on their interactions, and apply this knowledge in a second run-through with the virtual humans. Also, some of the suboptimal response choices had significantly higher penalties and produced strong responses from the virtual humans. Participants selected fewer of these “high penalty” choices the second time around. Other possible reasons for the observed differences might include the creation of environments and situations that allowed learners to have experiences through which they were able to construct meaning [131–134], and the provision of individualized, student-focused instruction [86].

Thirdly, the investigators examined how reflective comments and attitudinal survey results compared for students randomized to MPathic-VR or CBL. The results of the mixed methods analysis indicated differential learner experiences between the two conditions. Using two different methodological approaches, Likert item ratings and qualitative reflections [135], further reinforced through the same findings that students’ experiences with MPathic-VR were engaging, and contained valuable features not found in CBL (Table 3).

4.1.1 Knowledge Transfer

The positive results from the OSCE station assessment are noteworthy for two reasons. First, research has shown that learners often have difficulty transferring knowledge from one context to another [136, 137]. Successful transfer depends on the similarity between the training scenario and the novel scenarios; the greater the dissimilarity, the poorer the transfer [138]. The OSCE scenario used in this study was modeled on the MPathic-VR interprofessional scenario. Both scenarios addressed the same learning objectives and required knowledge of communication strategies to empathize and de-escalate tense, high-conflict situations. However, the story lines differed. The MPathic-VR scenario required the learner to resolve a conflict with the oncology nurse who was upset that she had been inadvertently omitted from her patient’s family meeting. In the OSCE scenario, the learner encountered a surgical assistant who was angry that the learner took her spot in an important surgical case. Second, students were unaware that their earlier experiences with either MPathic-VR or CBL would be tested in an OSCE station several days later. Thus, learners had to draw upon knowledge retained in long-term memory and apply it in the OSCE. Together, these two characteristics made for a challenging test of knowledge transfer from MPathic-VR to the OSCE station.

The results therefore suggest that knowledge of communication strategies acquired from MPathic-VR was resilient. Information retained over several days effectively transferred to a clinically realistic and novel scenario. These data further support interactive virtual human simulation, as embodied in MPathic-VR, as an effective means of training advanced communication skills. To the best of our knowledge, there are no comparable data supporting the use of any other simulation methods to develop these advanced communication skills [83, 84, 139–143].

4.1.2 Rationale for the Use of Virtual Humans

There was a deliberate rationale for using virtual humans in MPathic-VR. Research shows similar social effects whether a human is interacting with another human, or with a virtual human agent [138, 144–148]. There can even be instances where human-virtual human communication is preferred over human-human communication, one example being greater willingness to disclose personal information to a virtual human than to a human [149]. Standardization of experience is another reason for using virtual humans. In MPathic-VR, learners interact with a system in which the context of the scenario and the behavior of the virtual human is precisely specified at any moment. This level of control greatly facilitates the task of context-based recognition, and interpretation of learners’ verbal and nonverbal behavior [150]. Also, the level of difficulty encountered can also be controlled and tailored to learners’ abilities and progress. Learners can repeat scenarios and explore the effects of different choices, confident that the virtual human performance will remain consistent with pedagogical design requirements.

4.1.3 Study limitations

First, differing policies among the medical schools resulted in minor differences in their recruitment approaches. However, given the large sample size (N=421) and low nonparticipation rate (n=53, 12.5%), it is unlikely there was any appreciable effect on the results. Second, while MPathic-VR and the CBL module were self-paced, and time on task for the interprofessional communication components were equivalent: 1) MPathic-VR was more interactive by design, and 2) MPathic-VR students performed in two scenarios, but were only assessed on the content from the interprofessional scenario. Thus, differences observed between groups in the OSCE may reflect the joint effects of the content and delivery format. Third, although the OSCE was designed to assess the interprofessional communication training provided, it is possible that different scenarios might yield different results. Fourth, there were minor variations in the interval between exposure to the training modules and OSCE participation within and across the three medical schools. Ideally, the interval would be standardized, but the reality of varying curricular demands and training schedules made it necessary to accept a range of dates. Fifth, it is possible that differences among SPIs could account for variations in scoring, but the randomization procedure of students in the trial most likely would minimize such an effect. Sixth, it is possible that completing the attitudinal survey first may have influenced qualitative reflections about the experience. Nevertheless, concepts not covered in the survey (e.g., interactivity, immediate feedback, information overload) arose only in the qualitative data, suggesting the survey may have primed some, but not all of the qualitative comments. Last, a potential limitation is the small effect sizes (η2) for the comparisons with the OSCE items. One important reason for this result is the 0–1 range of the OSCE items. The effect size is measured in terms of variance and there is little variance between 0–1. Therefore, it is possible to obtain statistical significance, but because of the variability restriction, the effect size will not be large.

Of note, the transfer scenario was performed within an OSCE environment, which is a successive approximation closer, but not an actual clinical setting. Validating long-term retention and application of communication skills learned in MPathic-VR to actual patient encounters remains an area for future research. Further study will also help educators to better understand how the MPathic-VR simulation can maximize training transfer, and how it can combine with existing teaching methods to produce optimal communication training.

4.2 Conclusion

The present study provides initial evidence supporting the use of virtual human simulation for training communication skills. The data demonstrate both improved communication performance with MPathic-VR training, and successful transfer of communication skills acquired from MPathic-VR to a different, clinically realistic communication scenario. Mixed methods evaluation of students’ training experience favored MPathic-VR over traditional CBL. Together, these findings suggest that MPathic-VR might offer educators an effective and engaging means of training advanced communication skills.

4.3 Practice Implications

First, computer simulation with virtual humans appears to hold promise for providing learners with resilient knowledge, a useful toolkit of communication skills, and a safe environment in which to practice, reflect, and become adept in the use of those skills. Second, with further study, MPathic-VR may provide a new standard of training complex communication skills that is consistent over time, and across institutional and disciplinary boundaries.

Table 2.

Medical student attitudes about the MPathic-VR computer simulation and the control Computer-Based Learning (CBL) module

	MPathic-VR (N=210)	CBL (N=205)
The purpose of this training was clear.	5.33 (1.66)	5.20 (1.64)
The content was appropriate for my level of training.	5.83 (1.19)	5.67 (1.27)
This training was engaging.	5.43 (1.55)	3.69 (1.62)
This training was effective for learning verbal communication skills.	5.02 (1.62)	3.89 (1.67)
This training was effective for learning nonverbal communication skills.	4.11 (1.85)	2.77 (1.45)
This training was effective for learning how to handle emotionally charged situations.	5.13 (1.48)	2.34 (1.35)
This training will help me improve my clinical skills.	4.93 (1.57)	4.62 (1.40)
Based on this training, my communication skills improved.	4.60 (1.52)	4.21 (1.43)
The visual media were effective for learning the material.	5.30 (1.52)	4.85 (1.50)
Overall, this was an excellent training experience.	4.72 (1.68)	3.89 (1.61)
I recommend this educational exercise to others at my level of training.	4.82 (1.75)	4.00 (1.67)
I would like to take other educational exercises like this again in the future.	4.46 (1.97)	3.86 (1.68)
Mean Ratings	4.97* (1.34)	4.08* (1.16)

^*t(413)=7.23, p<.0001, r2= .11

Highlights.

• Students improved their communication skill with repeated use of MPathic-VR.

• Knowledge transferred from MPathic-VR to a clinically realistic OSCE scenario.

• Attitudinal ratings were higher for MPathic-VR students than for CBL students.

• Evaluation of students’ training experiences favored MPathic-VR over CBL.

• MPathic-VR may offer an effective and engaging way to train communication skills.