Impact of Simulated Patients on Physiotherapy Students’ Skill Performance in Cardiorespiratory Practice Classes: A Pilot Study

Abstract

Purpose: To date, no evidence exists that high-fidelity simulation improves skill development among physiotherapy students in the university setting. With pressures to reduce costs and maintain or improve the quality of the learning experience, and with pressures on clinical placement, it is essential to investigate methods that might improve students’ skill performance before they undertake clinical practice. Our study set out to investigate (1) the impact of using simulated patients (SPs) in a practical class on physiotherapy students’ skill acquisition and (2) the students’ reflections on the intervention. Method: We devised a pilot study using a single-centre randomized controlled trial. A total of 28 undergraduate physiotherapy students, matched using previous practical examination grades, undertook a 2-hour practical class in which they practised their core cardiorespiratory skills. Pre-session resources were identical. The control group practised on peers; the intervention group practised on SPs. The students’ skill performance was assessed 2 weeks later using the Mini-Clinical Evaluation Exercise (MiniCEX), including gathering qualitative data from the students’ reflections. Twenty-eight students undertook the practical class and subsequent MiniCEX assessment. Results: A statistically significant difference was found for all aspects of the MiniCEX except medical interview (p = 0.07) and physical interview (p = 0.69), and a large effect size was found for all areas except physical interview (0.154) and medical interview (0.378). The students’ reflections focused on three key themes: behaviours and attitudes, teaching the active cycle of breathing technique, and feedback. Conclusions: Our findings suggest that interacting with SPs improves student skill performance, but further research using a larger sample size and an outcome measure validated for this population is required to confirm this.

Periods of supervised clinical practice are a core element of pre-registration physiotherapy education programmes.¹ During these clinical periods, students are responsible for assessing and treating real patients with real conditions or problems. Therefore, their assessment and treatment techniques will have real and visible effects. To ensure that students can undertake this clinical practice safely and effectively, they must achieve deep learning within their university learning.² It is imperative that they understand what they are doing and the underlying rationale for and potential consequences of it. It is also important that they have sufficient ability in performing certain skills to be able to apply the appropriate techniques safely and effectively.

The basic skills that students use during clinical practice are those that they have acquired in the university setting in the first instance through a combination of theoretical and practical learning. Traditionally, practical learning involves students practicing on each other, called peer practice, which is low-fidelity simulation (LFS).¹ However, to achieve the deep and meaningful learning required to transfer that learning to real clinical practice, students need to be exposed to situations that will challenge their knowledge and experience – situations that require them, in essence, to reframe their knowledge, achieving deep learning.³

For effective learning to occur in clinical practice, research has proposed that students must achieve basic levels of the hierarchy of competence: feeling safe and secure, self-efficacy, and knowledge and experience of what to expect in the clinical environment.⁴ Engaging in peer practice (LFS) in university can enable students to feel safe and secure and to gain a level of self-efficacy. Students are also more comfortable with each other; they know what is expected of them with each technique and can react appropriately.⁵ However, peer practice does not impart the knowledge or experience of what to expect in a real clinical situation.

Shulman’s Table of Learning suggests that engagement and motivation are required for deep learning; only once this is achieved can students move forward to the psychomotor domain, the effective performance of skills.⁶ This hypothesis has been supported by Sabus and Macauley, who discussed the circumplex model of affect; students learn more effectively when there is an element of nervousness, tension, or excitement – essentially, when students are alert and engaged.⁷ When students work with peers, it is challenging for them to maintain the level of alertness they need to carry out the focused and repetitive practice required to achieve skill competence, but there is also no pressure to modify or correct techniques if the “patients,” their peers, do things correctly the first time.⁷

Pressures relating to placement provision and sufficient student supervision during clinical practice periods, an issue reported internationally, mean that it is critical that students’ skills be as strong as possible before they are exposed to the real clinical environment.^8,9 This will give them the confidence to learn effectively and minimize the pressure on already stretched clinicians while ensuring patient safety.¹⁰

Because Korpi and colleagues have indicated that students’ expertise develops in real work situations (e.g., clinical placement),¹¹ alternative learning methods may be beneficial, such as high-fidelity simulation (HFS): “Simulation experiences that are extremely realistic and provide a high level of interactivity and realism for the learner; can apply to any mode or method of simulation; for example: human, manikin, task trainer, or virtual reality.”^{12(p. 15)}

Simulation-based learning provides a continuum of complexity and realism, and it can put students into a range of clinical situations from the commonplace to the less frequent but more challenging. This enables students to evaluate the effect of, and modify, interventions as a consequence of patient responses, akin to real situations. Simulated learning already has a strong place in medical and nursing education.¹³ In addition, both Blackstock and colleagues¹⁴ and Watson and colleagues¹⁵ have demonstrated that HFS could replace clinical practice without negatively affecting students’ performance in cardiorespiratory and musculoskeletal physiotherapy.

However, a wider review of the physiotherapy literature shows a small and generally poor-quality evidence base relating to HFS; most studies have focused on investigating its impact on cardiorespiratory teaching. The findings to date have highlighted positive student perceptions^16–19 and possible benefits to developing students’ ability to apply knowledge.^20,21 Only one pilot study appears to have assessed whether HFS improved skill performance among physiotherapy students: Phillips and colleagues compared a group of 37 students who experienced HFS using simulated patients (SPs) with a control group (traditional peer practice) of 36 students in practising their patient assessment skills and ability to mobilize a patient safely.²² Their study found that the HFS group performed skills more poorly than the control group. However, the students had no prior experience of HFS, which may have increased their stress level and inhibited their learning.^7,23

To date, there is no evidence that HFS improves skill development among physiotherapy students compared with traditional teaching and learning approaches. As with health services, universities are experiencing pressure to reduce costs, while maintaining, if not enhancing, the quality of the learning experience.²⁴ Because simulation is a costly method of teaching, evidence to support this method of learning is required if it is to continue to be used.²²

The current lack of evidence indicates that it would be worthwhile to conduct an initial exploratory pilot study to establish whether incorporating SPs into a practical class appeared to have an effect and also to investigate students’ views on that learning approach compared with traditional peer practice (LFS) learning experiences. If so, it would point to the value of undertaking further study in this area and, if appropriate, provide effect sizes for a larger randomized controlled trial.

The aims of this exploratory study, therefore, were to (1) investigate the impact of incorporating SPs into a practical class on physiotherapy students’ performance of core cardiorespiratory skills and (2) gather students’ initial reflections on this learning method.

Methods

Study design

An exploratory pilot study using a single blind randomized controlled trial with an embedded qualitative component was undertaken.²⁵ We used the Mini-Clinical Evaluation Exercise (Mini-CEX) to obtain quantitative measurements of the students’ skill performance and gather qualitative data from the students’ reflections.^26,27

At the study institution, grades are calculated as A–F, so we used a matched pairs design to allocate students with comparable ability into each group. This design also allowed us to more accurately compare the results between the intervention group, which practised on SPs (using HFS), and the control group, which practised on peers (using LFS).²⁸

The study was approved by the School of Health Sciences Research Review group (Ref. No. SHS/16/02); gatekeeper approval was gained from the course leader.

Participants and setting

A convenience sample of students from Year 2 of a 4-year BSc (Honours) physiotherapy programme at one Scottish university was invited to participate in the study. The primary researcher (Walker) explained the study to all the students during a class at the start of their acute care module, which commenced in the second semester. This was followed up with an e-mail invitation and participant information sheet. Although students received an introduction to cardiorespiratory skills in Year 1, in which they briefly practised their skills on each other, previous experience from teaching this module has shown that the students do not retain these skills. The acute care module is the main opportunity students have to develop these core skills before using them in practice.

The students were advised that they would be randomly allocated to either a peer practice group (LFS) or a group undertaking practice on SPs (HFS). They were also advised that their participation was voluntary, non-participation would not disadvantage them in any way, and they could withdraw from the project at any time with no impact on their assessment grade for that module. To reinforce this advice, the module leader was not involved in collecting data for the study. Students who wanted to participate were asked to reply to the invitation email and to provide written informed consent.

The Year 2 students had undertaken three previous clinical placements focusing on care of elderly outpatient musculoskeletal, community, orthopaedic, and neurology patients. They also had experience working with SPs during HFS to practise their assessment skills in these areas, including taking subject histories and objective testing. During these activities, the SPs worked in a case scenario and provided the students with feedback on core professional areas such as communication and handling skills, empathy, and caring.

All 31 students in the Year 2 class agreed to participate in our study, but only 28 attended the practical class and could be included. Blocked randomization was undertaken to enable us to set up a matched pairs design. The students were matched in pairs; the blocking variable was the results from their practical exam from the preceding module (Grades A–F). One of each pair was then randomly assigned to either the HFS group (n = 14) or the LFS group (n = 14). A member of the physiotherapy team, independent of the study and module, used the sealed-envelope method of randomization.

Practical classes

Acute care is taught using a combination of directed study and tutorials or workshops in which students actively apply theory to clinical situations. The aim is to promote deep learning. The students also have access to videos detailing the teaching and application of core respiratory techniques, such as the active cycle of breathing technique (ACBT). Practical classes are traditionally undertaken in a simulated ward environment in which students practise their skills on each other and receive feedback from staff and peers. At the university, the ward environment consists of two six-bed hospital bays, which enabled both groups to be taught simultaneously. Each bed space has a bed, patient locker, and chair and can be separated from the next bed space by curtains; it replicates the environment in which students will work during their clinical placements. The learning outcomes for both groups were the same:

• To practise teaching the three components of ACBT (breathing control, thoracic expansion exercises, and forced expiratory technique).

• To develop skills in modifying the ACBT for patients with breathlessness, sputum retention, or loss of volume.

The LFS group (control group) practised these skills on their peers, working in threes: one patient, one therapist, and one student providing feedback. The peers were advised to put themselves in the position of a patient and to respond appropriately to the students’ instructions – for example, if the instructions were not clear, they were to do what they thought was being asked rather than what they knew they should do. The HFS (intervention) group practised the same skills on seven SPs, with the first student teaching the patient and the other taking notes and providing feedback to them. Other than the models used for practising the technique, both groups received the same experience.

The SPs were members of the volunteer patient bank at the university. They are members of the local community who volunteer to take on this role to facilitate student learning. They have diverse backgrounds, and on joining the patient bank they receive training on the requirements of patients and how to provide constructive feedback. Before each class, they are briefed by the class tutor about what is required of them. When patient scenarios are used, they are sent to the SPs at least 2 weeks in advance of the session. Each volunteer receives a £20 gift voucher for each period of up to 4 hours that they work with students.

For the purposes of this class, the SPs were not performing a specific patient scenario. They were briefed before the class about the purpose of the research and that students would be teaching them various breathing techniques. Because the SPs had not been involved in these practical classes before, the techniques were novel to them, and they were advised to be themselves. If the students did not explain the techniques clearly, they were to do what they thought the instructions meant. If they needed to ask a question, they were to do so in the same way a patient would. Students would have to focus on their explanations and problem solve ways to help the SPs achieve the correct techniques. No attempts were made to standardize how the SPs responded so that students would have a more realistic experience because patients vary in how they respond. However, SPs were advised to behave as though they were breathless, had sputum retention, or had lung volume loss as much as possible after students had practised the basic techniques. Students worked with different SPs so that they would have to modify their explanations depending on the SPs’ different responses.

The practical class lasted 2 hours, during which both groups practised the three components of ACBT, in addition to modifying it for breathlessness, sputum retention, and lung volume loss. The patients were not attached to equipment or wearing costumes, which is typical of our practical classes. The students in the HFS group received feedback from their peers, from the SPs’ responses (and explicit feedback about the clarity of the students’ explanations, handling, and approach), and from the class tutor. This feedback addressed the students’ basic skills before the students and SPs were advised whether the patient had (1) breathlessness, (2) sputum retention, or (3) volume loss. The students then had to explain the problem to the SPs and modify their techniques appropriately. The LFS group received feedback from the class tutor and their peers.

The classes for the HFS and LFS groups ran concurrently; a different tutor facilitated each class to prevent contamination. This approach also ensured that students did not have an opportunity to talk to each other about their in-class experience until it was over. The tutor for the LFS group had 1 year of teaching experience in a university setting and 4 years of clinical experience; the tutor for the HFS group had 14 years of teaching experience in a university setting and 12 years of clinical experience. Before the classes began, the tutors were briefed on the learning outcomes and given a clear teaching plan (see the online Appendix) that outlined the facilitation activities.

Outcomes

No specific validated cardiorespiratory physiotherapy outcome measures were identified;²⁹ consequently, we used the MiniCEX,^26,27 which assesses communication, professionalism, counselling, attitudes, and behaviours and has been shown to be valid and reliable for assessing clinical skills and competence among medical students.³⁰ A student assesses and treats a patient while a clinician rates the student on a Likert scale and provides formative feedback. The reflective component of the MiniCEX gave students the opportunity to provide information on their self-rated competence, confidence, and views of their learning experience. They were asked the following questions:

• What are your perceptions and views of your interactions with your peers in practical and simulation teaching?

• What are your perceptions and views of your interactions with the volunteers in practical and simulation teaching?

The practical class for ACBT was undertaken in the second week of the 6-week module. Data collection was undertaken during the third week. Students undertook the skills assessment on an SP and were assessed by the primary researcher (Walker), who was blind to group allocation and who provided formative feedback on the students’ performance as soon as they completed the MiniCEX. Students completed the self-reflection component of the MiniCex, then returned it to the primary researcher and left the room.

Data analysis

The MiniCEX is used to rate students on a 6-point Likert scale (0 = well below expectation, 1 = below expectation, 2 = borderline, 3 = meets expectation, 4 = above expectation, 5 = well above expectation). These ordinal data enabled use of the non-parametric Wilcoxon signed-rank test to compare the matched pairs’ performance (using IBM SPSS Statistics, Version 25; IBM Corporation, Armonk, NY). Because of the small sample size, we report exact test results. Statistical significance was set at p ≤ 0.05, and effect sizes for the Wilcoxon signed-rank tests were calculated.³¹ Associations between the HFS and LFS groups were investigated using χ² tests.

We analyzed qualitative data from the students’ reflections using modified thematic framework analysis based on that proposed by Spencer and colleagues.³² Both of us are cardiorespiratory specialists, and we each have more than 10 years of clinical experience and more than 7 years’ experience teaching at the university level. In addition, we both have an interest in the use of HFS as a learning method. To prevent bias, we independently identified themes and then met to compare and agree on those that were appropriate. The data for each group were kept separate. Because of the small amount of qualitative data, the descriptive stage of framework analysis was applied, but not the explanatory stage.³²

Results

We collected data on 28 students; the demographic data for the LFS and HFS groups are provided in Table 1. Table 2 shows the results achieved by each matched pair. The Wilcoxon signed-rank test suggests statistically significant higher medians for the HFS group in all aspects except medical interviewing skills (p = 0.07) and physical interviewing skills (p = 0.69; Table 2). The effect sizes for these tests are large in all areas except medical interviewing (medium effect size) and physical interviewing (small effect size), as per Cohen’s d criteria (Table 2). The results of the χ² test indicate a significant association between improved performance and HFS in all aspects except medical and physical interviewing skills (ps = 0.31 and 0.86, respectively).

Table 1 .

Students’ Demographic Information

Characteristic	LFS (n  = 14)	HFS (n  = 14)
Gender, n (%)
Male	3 (21)	5 (36)
Female	11 (79)	9 (64)
Age, mean (SD)	19.9 (1.5)	20.6 (1.9)

LFS = low-fidelity simulation; HFS = high-fidelity simulation

Table 2 .

MiniCEX Results

Pair	Medical interview		Physical interview		Counselling and communication skills		Clinical judgment		Consideration of patient and professionalism		Organization and efficiency		Clinical competence
	LFS	HFS	LFS	HFS	LFS	HFS	LFS	HFS	LFS	HFS	LFS	HFS	LFS	HFS
1	2	3	2	2	3	4	3	4	3	4	3	4	3	4
2	2	3	2	3	3	4	2	4	3	4	3	5	3	4
3	3	3	3	2	4	5	4	5	4	5	4	5	4	5	3	4
4	3	3	3	5	3	5	3	5	3	4	3	5
5	2	3	2	2	3	4	2	4	3	4	3	4	3	4
6	3	3	2	2	3	5	3	5	4	5	3	5	3	5
7	3	2	2	3	3	4	4	5	3	5	3	4	3	5
8	2	2	2	2	3	3	2	4	3	4	3	3	3	3
9	2	2	2	2	3	3	2	3	4	3	3	3	3	3
10	3	2	3	2	3	4	3	5	3	5	4	4	3	5
11	2	3	2	3	5	5	3	5	5	5	5	5	5	5
12	2	4	2	2	3	3	3	4	4	4	3	4	3	4
13	2	3	2	2	3	4	2	3	2	3	3	3	3	3
14	1	3	1	2	3	4	3	4	3	4	3	4	3	4
Mode	2	3	2	2	3	4	3	4/5	3	4/5	3	4	3	5
Median	2	3	2	2	3	4	3	4	3	4	3	4	3	4
Wilcoxon signed-rank test, p-value	0.07		0.69		0.002		0.001		0.005		0.004		0.002
Z-statistic*	−1.999		−0.816		−2.972		−3.035		−2.804		−2.810		−2.889
Effect size	0.378		0.154		0.562		0.574		0.530		0.531		0.546
Cohen’s d criterion	Medium		Small		Large		Large		Large		Large		Large
χ2 test	0.31		0.86		<0.001		0.002		<0.001		<0.001		<0.001

Note: Likert scale ratings: 0 = well below expectation, 1 = below expectation, 2 = borderline, 3 = meets expectation, 4 = above expectation, 5 = well above expectation.

^*Based on negative ranks.

MiniCEX = Mini-Clinical Evaluation Exercise; LFS = low-fidelity simulation (control group); HFS = high-fidelity simulation (intervention group).

Students’ reflections

Our analysis of all 28 student reflections about their initial views of this learning method resulted in three key themes: behaviours and attitudes, teaching ACBT to patients, and feedback from the patients. The subthemes are shown in Figure 1, along with how they interact. Direct quotes are provided. Quotes are from the intervention group (I) and control group (C), and the numbers refer to the pair number.

Figure 1 .

Themes and subthemes developed from the students’ reflections.

Behaviours and attitudes

When working with their peers, students reported that they would “become more distracted in class with our peers” (I3, C9) and would “go off [on] tangents with peers/friends” (C4). They also reported feeling more self-conscious as the therapist and more nervous working with their peers as patients: “more self-conscious with my peers” (C1).

These reflections contrasted with those of the students who worked with the SPs; in this case, students reported that they felt the need to behave more professionally and be more focused: “more professional when looking around the class in the practical session” (I1).

Teaching active cycle of breathing technique to patients

Students who had worked with the SPs reported that they were “better prepared with reading as they put you on the spot” (I11). They also had to focus more on their explanations of techniques and the clarity of their instructions: “Have to explain instructions and modify [them]” (I4). This may relate to the students’ perception that their peers knew the techniques and therefore they did the correct technique without even needing instructions: “peers know what they are asking therefore pre-emptively do it” (I9).

Feedback from patients

Students who worked with their peers reported getting little feedback from them. This contrasted with those who worked with the SPs, who reported that this feedback was much more constructive: “receive more accurate feedback on handling, for example” (I6).

Discussion

The results of this exploratory study suggest that when physiotherapy students practise their core respiratory skills on SPs, they may perform their skills better than when they practise on their peers. A statistically significant difference was found in counselling and communication skills, clinical judgment, consideration of the patient and professionalism, organization and efficiency, and clinical competency. The students also reported behaving more professionally and being better prepared and focused before the class and that the feedback received from the SPs was more constructive.

Although a meaningful difference in scores for the MiniCEX has not been established in the literature, our data analysis indicates a large effect in favour of the HFS activity in key areas. This is further supported by the χ² test results, which indicate a statistically significant number of higher performing students in the HFS group. This suggests that practising these core skills on the SPs improved the students’ skill development and subsequent skill performance. However, this was only evidenced in areas directly related to the class content.

The data provided by the students’ reflections suggest that the difference in performance may be due to gaining improved knowledge before the practical class; increased focus during class; having to modify their instructions to ensure that the SPs understood what was required; and being able to effectively perform the required tasks in a safe, timely, and efficient manner. The increased focus alludes to increased alertness, which Sabus and Macauley argued improves learning.⁷ Moreover, the fact that the students received more detailed and specific feedback from the SPs than from their peers means that they could modify and enhance their technique, thereby enabling them to reframe their knowledge and experience and ultimately promoting deeper learning.

What was not measured was whether the amount of practice that the students undertook differed between the control and intervention groups. In addition to the students reporting improved quality of practice with the SPs, they may have undertaken more deliberate practice, which has been reported to improve skill development.³³ Certainly the comments relating to increased focus and fewer distractions with the SPs may infer improved quality, if not amount of practice. Transfer of learning to practice and retention of learning were also not measured.

These findings contrast with the study by Phillips and colleagues, which is the only other study we found that investigated HFS for skill development among physiotherapy students.²² That study found that students who had practised on peers had fewer safety fails and fails overall compared with students who had practised on SPs. However, a fundamental difference between the studies may be in the use of HFS. Students at our study facility have opportunities to practise undertaking patient assessments on SPs during Year 1 of their course. Consequently, they know what to expect and may have achieved sufficient stimulation in the form of nervousness, tension, or excitement from the activity. This positions the student in the pleasant activation area identified in the circumplex model of affect, enabling effective learning.⁷ The students in the study by Phillips and colleagues²² may have been working in an unpleasant activation area as a result of stress from never having worked with SPs before, and this may have inhibited their learning.⁷ These conflicting findings suggest that further research into the impact of SPs on skill development is warranted.

The two areas we found in which the students did not improve from interacting with the SPs were medical and physical interviewing skills. However, this result is unsurprising because these skills were not a focus of the class that used the SPs; they had been taught previously with peer practice. Results for medical interviewing have suggested that there was a difference in performance between the HFS group and the control group, although this did not reach statistical significance (p = 0.07).

The focus in the interactions with the SPs was on teaching patients a skill. Consequently, the students’ counselling and communication skills, professional skills, and clinical competence would be expected to improve. Teaching ACBT requires students to explain and demonstrate the technique, and the SPs were trained to give feedback on the students’ communication skills and professionalism. Students in the intervention group reported an improvement in their communication skills because they needed to modify their explanations and communicate more effectively with the SPs than with their peers. They were also challenged by the SPs asking questions, and the need to respond appropriately may have influenced their clinical thinking. The results suggest that there may have been some carry-over of generic skills such as communication and patient care from practising with the SPs. This carry-over may have reduced the difference between groups in relation to medical interviewing skills. However, the more specific skills of structuring a patient interview were not addressed in this class, which may have prevented the results from reaching statistical significance. It would be useful to incorporate SPs into the patient assessment skills practical class to evaluate this.

It is possible that the difference in skill performance between the two groups was not influenced by the SPs but by the tutors facilitating the sessions. The LFS group was facilitated by a tutor with less clinical and teaching experience than the tutor for the HFS group, so the HFS group may have benefited from this higher level of clinical and teaching experience. Further studies using a crossover design or using facilitators with similar levels of experience may therefore be beneficial.

Our study had two limitations. The first is that the data were collected from a small sample based at one university; this limited our conclusions about statistical significance and generalizability.

The second limitation concerns the MiniCEX. Although it is validated for use with medical students, it has not been validated for use in assessing practical skills performance with physiotherapy students. It does not provide enough detail about specific aspects of each technique, and this may limit confidence in the results. Using a Likert scale also introduces subjectivity into the evaluation of student performance, although using only one assessor helped control this variable. Unfortunately, there are no validated outcome tools to measure skill performance in physiotherapy practice;²⁹ therefore, the MiniCEX was the most appropriate tool to use.

Conclusion

The findings of this study suggest that interactions with SPs may benefit physiotherapy students’ skill performance. Further research with an adequate sample size, using an outcome measure that has been validated to accurately measure specific physiotherapy skill performance, is required. If it is established that interactions with SPs improve skill performance, it would be beneficial to incorporate SPs into undergraduate physiotherapy practical classes and programmes and investigate whether these enhanced skills are transferred to the clinical environment.

Key Messages

What is already known on this topic

Current evidence indicates positive student perceptions in relation to high-fidelity simulation (HFS) as a learning method with reports of improved confidence. However, limited evidence has been found in relation to the impact of HFS on skill performance.

What this study adds

This study suggests that use of HFS, in the form of standardized patients for practising skills, may help students develop their core cardiorespiratory treatment skills. It also suggests that generic skills such as communication skills may be transferable from one area to another (e.g., from treatment skill to assessment skill)