Comparison of the psychological and physiological effects on students of a video-assisted or text introduction to a simulated task

Abstract

Introduction

The use of different methods for introducing the scenario in simulation-based medical education has not been investigated before and may be a useful element to optimise the effectiveness of learning. The aim of this study was to compare an immersive video-assisted introduction to a minimal text-based one, with regard to emotional assessment of the situation.

Methods

In this pilot study, 39 students participated in a medical simulated scenario. The students were randomly assigned to an experimental group (video-assisted introduction) or a control group (minimal textual introduction) and both were followed by performing surgery on LapSim (Surgical Science, Gothenburg, Sweden). The emotional assessment of the situation, cognitive appraisal, was defined as the ratio of the demands placed by an individual’s environment (primary appraisal) to that person’s resources to meet the demands (secondary appraisal). Secondary outcomes were anxiety (State-Trait Anxiety Inventory), physiological parameters (heart rate, heart rate variability, skin conductance, salivary cortisol), engagement (Game Engagement Questionnaire), motivation (Intrinsic Motivation Inventory) and performance (mean score in percentage calculated by LapSim of predefined levels).

Results

Participants in the immersive video group (n=17) were overloaded in terms of their perceived demands (a ratio of 1.17, IQR 0.30) compared with those in the control group (a ratio of 1.00, IQR 0.42, n=22) (P=0.01). No significant differences were found between the groups in secondary outcomes. Both groups showed an increase of anxiety after the introduction method. In the experimental group, this score increased from 9.0 to 11.0, and in the textual group from 7.5 to 10.5, both P<0.01.

Discussion

This study shows that the method of introducing a simulated scenario may influence the emotional assessment of the situation. It may be possible to make your simulation introduction too immersive or stimulating, which may interfere with learning. Further research will be necessary to investigate the impact and usefulness of these findings on learning in simulation-based medical education.

Introduction

Simulation-based medical education (SBME) is a complex social process, in which trainees interact with each other, manikins, and the simulated environment. Despite several described theories on engagement, simulation fidelity and validity, there is still a gap between matching simulated scenarios with learning outcomes.^1–4 Additional physical fidelity does not directly relate to better attainment of educational goals.¹ Clarification about the psychological and physiological response to reality-inducing and stress-inducing elements in SBME may help to achieve optimal learning goals. One of these elements is the introduction of the simulated scenario.

The introduction of the simulated scenario is part of the prebriefing or also called briefing phase in SBME. One of the goals of it is to create a psychologically safe context, including the establishment of a ’fiction contract' with the participants.¹ In it, the instructor acknowledges that the simulation cannot be exactly like real life, but agrees to make the simulation as real as possible within resource and technology constraints. One of these technology constraints may be the use of different introductory methods. Results from studies in other domains about the use of different story types suggest that video and video plus textual versions, compared with textual versions, lead to higher levels of both engagement as well as the ability to recall information.^{5 6}

Results on performance and learning and the effects on it from emotional and physiological responses among trainees in SBME remain unclear, and some results show contradictions.^7–15 This may be explained by differences in individual cognitive load. The cognitive load theory (CLT), described by Sweller in 1988, emphasises the capacity limitations to process novel information in working memory in instructional or educational contexts, by distinguishing different load categories.¹⁶ One of these load categories is ’germane mental workload'. This workload corresponds to the load induced by conscious application of strategies to solve tasks more efficiently.¹⁶ Individual characteristics, including cognitive appraisal, are involved in this process.¹⁶ Cognitive appraisal is defined as the interaction of demands placed by an individual’s environment (primary appraisal), and that person’s resources to meet the demands (secondary appraisal).^17–19 When resources are perceived as sufficient to meet the demands, the situation is experienced as a challenge. In contrast, when demands are perceived as outweighing the resources, the situation is experienced as a threat.^17–20 The challenge and threat response differ in effects on performance and learning results.

The aim of this study was to compare an immersive video-assisted introduction to a minimal text-based one, with regard to emotional assessment of the situation (threat or challenge response). In this study, we also investigated other psychological and physiological parameters related to arousal, learning and performance, such as feelings of anxiety, physiological parameters, engagement and motivation. The following null hypotheses were explored:

1. An immersive video-assisted introduction leads to the same levels of cognitive appraisal (threat response), compared with a minimal textual introduction.

2. An immersive video-assisted introduction leads to the same levels of anxiety, sympathetic nervous system, engagement, motivation and performance compared with a minimal textual introduction.

Methods

We performed a prospective, randomised controlled pilot study. The study was conducted at a Dutch secondary school in the Southern part of the Netherlands. The accredited Medical Ethics Committee of Máxima Medical Center confirmed this study as consent exempt. Yet, students signed informed consent before participation, which was easy to obtain given the prospective set-up of this study. The participants were 39 healthy secondary school students aged between 17 and 19 years to exclude different levels of medical work experience. They were all final year students with biology as a subject and were asked to voluntarily take part in this investigation.

Preintervention methods

One month prior to performing the tested intervention, a short lecture was given by a medical doctor about basic clinical aspects of patients and laparoscopic surgery. Students were then given the opportunity to practice once, using predefined training levels, on the LapSim (Surgical Science, Gothenburg, Sweden). The LapSim is a simulated video-assisted laparoscopic surgery box. After practising, the students were randomly allocated by computer programming to either the video-assisted introduction group (experimental group) or the textual briefing group (control group).

Briefing methods

The immersive introduction consisted of an introductory video, designed and recorded by a medical simulation centre in the Netherlands (Medsim, Eindhoven) with input from a multidisciplinary team and has been used in various simulated scenarios in obstetric training curricula. The video was recorded from different camera angles and positions and showed a woman involved in a serious car accident, including the actual car crash. The duration of the video was 3.5 min. After the car crash, the video showed the ambulance staff taking care of the patient, calling to doctors in the hospital and transporting the patient to the hospital while the patient’s medical status declined. The second part involved the initiation of a laparoscopic surgery. The video ended after introduction of the instruments into the abdomen through trocars (hollow tubes), ready to start the surgery. The control group received a textual slide where they were supported to perform as a doctor in a surgery room. All students were standing in an upright position when the introduction method was showed. After the method for introducing the scenario, the students were dressed with a surgical apron, gloves, mask and head cap before performing on the LapSim. The location of a surgery room, including manikin, was imitated. This scenario with LapSim was chosen because physical activity, for example, during resuscitation, may influence several parameters. Data on personal characteristics were gathered using self-report questionnaires. After LapSim, students were separately asked how they felt about the situation, how it went, scores were discussed and eventually learning goals were discussed.

Measures

The experiment timeline is shown in figure 1. The primary outcome of the study was cognitive appraisal using the method described by Tomaka et al.^{18 21} This was measured after the introduction method and after LapSim. Cognitive appraisal was calculated as the ratio of the primary appraisal (demands) to the secondary appraisal (resources) (figure 1).^{18 21} The primary appraisal was examined by asking the participants to answer the following questions: "How demanding do you expect the upcoming task to be?" and "How demanding was the task you just completed?"^{18 21} Secondary appraisal was measured by asking the question "How able are/were you to cope with this task?"^{18 21} Answers had to be given on a 10-point Likert scale. The situation was appraised as a ’challenge' if the resources were perceived as equal or greater than the demands (ratio ≤1), and as a ’threat' if the demands were perceived greater than the resources (ratio >1).^{18 21}

Figure 1.

Timeline of experiment. HR, heart rate; HRV, heart rate variability; SC, skin conductance.

Secondary outcomes were anxiety, physiological parameters (heart rate (HR), heart rate variability (HRV), skin conductance (SC), salivary cortisol), engagement, motivation and performance. Anxiety was measured three times: straight after the calibration period (10 min rest period), immediately after the introduction method and directly after LapSim, using the six-item short-form of the state scale of the Spielberger State-Trait Anxiety Inventory (STAI) (figure 1).²² Participants had to indicate their level of agreement about how they felt at the given moment on a 4-point scale. Each of the six questions was given a score of 1–4, generating a total score of 6–24. Physiological parameters (HR, HRV and SC) were continuously measured with a wireless ECG-necklace and a wristband.^{23 24} This equipment was especially designed for measuring stress parameters by Holst Centre/IMEC (Eindhoven, The Netherlands) and has not been used in SBME yet. Mean values for the physiological parameters were calculated for a 10 min rest period (T0, calibration period), during the introduction (T1), and during the performance on the LapSim (T2). HR was determined by direct measurement of normal-to-normal (NN) intervals of QRS complexes resulting from sinus node depolarisations. Measurements of low frequency (LF) component (0.04–0.15 Hz), and high frequency (HF) component (0.15–0.4 Hz) were calculated using the Fast Fourier Transform algorithm. Both results were expressed in absolute values of power (ms²) and normalised units (LFn.u. and HFn.u.). Normalised units represent the relative value of each power component in proportion to the total power of LF plus HF components.²⁵ HF activity has been linked to parasympathetic nervous system activity, LF activity is now accepted to be a mixture of activity of both the sympathetic nervous system and parasympathetic nervous system, and the LF/HF ratio emphasises the behaviour of the autonomic nervous system.²⁵ The SC, the inverse of skin resistance, was defined as the electrical conductance of the skin in microsiemens (µs) and was used for capturing the autonomic nerve responses as a parameter of the sweat gland function.²⁶

Activation of the hypothalamic-pituitary-adrenal axis (HPA axis) was measured using salivary cortisol levels between 13:00 and 16:00 hours, which have shown a close correlation with plasma cortisol levels.²⁷ The HPA axis is a major neuroendocrine system among the hypothalamus, the pituitary gland and the adrenal glands that controls reactions to stress and regulates body processes.²⁷ In response to stress, a cascade of events can occur that culminate in the release of glucocorticoids, such as cortisol. Since this cortisol response is influenced by gender, oral contraceptive use and menstrual cycle phase, salivary cortisol samples were only collected for men.^28–33 Participants were asked to abstain from eating, drinking, consuming caffeine, using drugs, smoking, brushing teeth and doing exercise 1 hour prior to the study period. Salivary samples were collected at the start, and exactly 20 min after the introduction, because previous studies have shown that cortisol response to stressors peaks during this time.^{34 35} Participants chewed on a roll-shaped saliva collector (Salivettes, Sardstedt, Germany) for 30 s. This collector was then placed in a collection tube and frozen until all samples were centrifuged and analysed at the laboratory of the Máxima Medical Center using the ELISA technique.

Three other secondary outcome measures in this study were engagement, motivation and performance. First, engagement was measured by using the 19-item ‘Game Engagement Questionnaire’ (GEQ), with a total range of possible scores of 19–57, straight after LapSim.³⁶ The GEQ provides a psychometrically measure of levels of engagement specifically elicited while playing video games.³⁶ Second, all subjects completed the 22-item Intrinsic Motivation Inventory (IMI) to measure motivation. The subscale ‘interest/enjoyment’ (seven items) is considered the self-report measure of intrinsic motivation. Finally, the performance of the participants on the LapSim was rated using the mean score in percentage calculated by LapSim itself of two predefined levels. This calculation was based on the items such as number of errors, distance of the instruments and time spent to finish the levels.

Statistical analysis

Cognitive appraisal, anxiety and physiological parameters were expressed as absolute and change median values with IQRs. Engagement, motivation and performance were expressed as absolute values. Differences in scores between the two groups were evaluated with the Mann-Whitney U test. For scores over time, we used the Wilcoxon signed rank test. Statistical significance was accepted at a two-sided P value of 0.05. All analyses were performed using SPSS (V.21, IBM, San Jose, California, USA).

Results

Of the 39 participating students, 17 were randomised to the immersive introduction group and 22 to the textual briefing group. The demographics of both groups are shown in table 1.

Table 1.

Demographics of experimental and control group

Variables	Experimental group (n=17)	Control group (n=22)	P value Mann-Whitney U test/Χ2
Age (year), mean (SD)	17.4 (0.2)	17.2 (0.1)	0.57
Male, n (%)	11 (64.7)	7 (31.8)	0.58
Wants to study medicine, n (%)	4 (23.5)	8 (36.4)	0.39

Group medians and IQRs for cognitive appraisal are listed in table 2. A difference was found in cognitive appraisal score between the experimental group (median 1.17, IQR 0.30) and the control group (median 1.00, IQR 0.42) straight after the method for introducing the scenario, P<0.01. Post-LapSim, the cognitive appraisal score was not different between the groups (P=0.68). For anxiety, no difference was found in the total score of the state scale of the STAI (which can range from 6.0 to 24.0) between the two groups straight after introducing the scenario (table 2). Moreover, no differences in physiological parameters were found between the experimental and the control groups on the three different points in time.

Table 2.

Results of psychological and physiological parameters, engagement, motivation and performance in experimental and control group at different points in time

Variables	Points in time	Experimental group		Control group		P value
		N	Median (IQR)	N	Median (IQR)
Cognitive appraisal	After introduction	17	1.17 (0.3)	22	1.00 (0.42)	0.01
	After LapSim	17	1.20 (0.91)	22	1.25 (0.85)	0.68
Anxiety	Before calibration period	17	9 (3.5)	22	7.5 (3.3)	0.32
	After introduction period	17	11 (5)	22	10.5 (4)	0.57
	After LapSim	15	11 (5)	22	10.5 (3.5)	0.72
Heart rate (bpm)	T0	15	86.28 (23.82)	17	88.17 (13.43)	0.93
	T1	14	83.18 (19.62)	12	83.99 (23.65)	0.64
	T2	14	89.29 (17.1)	17	89.28 (12.63)	0.66
Normalised LF of HRV	T0	15	0.82 (0.08)	17	0.87 (0.10)	0.22
	T1	13	0.73 (0.15)	12	0.76 (0.32)	0.70
	T2	13	0.79 (0.16)	17	0.84 (0.15)	0.37
Normalised HF of HRV	T0	15	0.18 (0.08)	17	0.13 (0.10)	0.22
	T1	13	0.27 (0.15)	12	0.24 (0.32)	0.70
	T2	13	0.21 (0.16)	17	0.16 (0.15)	0.37
LF/HF ratio	T0	15	4.64 (2.35)	17	6.72 (5.08)	0.22
	T1	13	2.71 (2.13)	12	3.98 (8.84)	0.70
	T2	13	3.73 (6.01)	17	5.14 (6.34)	0.37
Skin conductance level (μs)	T0	10	0.21 (0.85)	13	0.12 (0.50)	0.50
	T1	10	0.30 (0.82)	14	0.09 (0.71)	0.91
	T2	10	0.08 (1.11)	15	0.76 (0.22)	0.96
Salivary cortisol (only men) (mmol/L)	Before calibration period	7	9.66 (12.90)	11	10.13 (4.72)	0.68
	20 min after introduction	7	11.24 (9.69)	11	9.99 (15.04)	0.93
Engagement		15	30 (6)	22	30 (6)	0.84
Motivation		17	36 (16)	22	36 (16)	0.64
Performance		16	64 (25.3)	21	64 (25.3)	0.79

T0: calibration period; T1:Introductory method.

HF, high frequency; HRV, heart  rate variability; LF, low frequency.

Some parameters changed over time (table 3). In the control group, the cognitive appraisal ratio was higher after performing on the LapSim compared with the moment after the introduction method. Furthermore, in both groups the median anxiety total score did significantly increase from straight after the calibration period to straight after the introduction. Some differences were also found in the physiological parameters between the defined time points for both groups separately. In the experimental group, a decrease was found in HR and LFn.u., and an increase in HFn.u. between the calibration and introduction period. Moreover, in the control group the skin conductance level decreased, while there was no difference in the experimental group.

Table 3.

Results of psychological and physiological parameters between different time points in the experimental and control group

Variables	Interval between points in time	Experimental group			Control group
		Median T0	Median T1	P value	Median T0	Median T1	P value
Cognitive appraisal	After introduction—after LapSim	1.17	1.20	0.55	1.00	1.25	0.02
Anxiety	After calibration period—after introduction	9.0	11.0	<0.01	7.5	10.5	<0.01
Heart rate (bpm)	T0–T1	86.28	83.18	<0.01	88.17	83.99	0.21
Normalised LF of HRV	T0–T1	0.82	0.73	0.01	0.87	0.76	0.06
Normalised HF of HRV	T0–T1	0.18	0.27	0.04	0.13	0.24	0.06
LF/HF ratio	T0–T1	4.64	2.71	0.07	6.72	3.98	0.70
Skin conductance level (μs)	T0–T1	0.21	0.30	0.39	0.12	0.09	0.04
Cortisol (mmol/L) (only men)	Before calibration period—after introduction	9.66	11.24	0.50	10.13	9.99	0.48

T0: calibration period; T1: introductory method.

HF, high frequency; HRV, heart  rate variability; LF, low frequency;

Finally, no differences were found between the groups for engagement, motivation and performance (table 2). The median score on the 19-item GEQ in the experimental group was 30, while this score was 31 in the control group. No difference in performance was found between the two groups.

Discussion

Our findings suggest that an immersive video-assisted introduction causes a higher level of emotional assessment of the situation compared with a minimal textual introduction straight after the method for introducing the scenario. In this study, the immersive introduction resulted in a threat appraisal, while the minimal textual introduction led to a challenge appraisal. This might be caused by more sensory input from the video compared with the textual introduction. Moreover, in both groups the anxiety scores increased from calibration period to introduction period. This result suggest that students in a simulated task are influenced in their learning process from the start of the introduction of the scenario.

However, no differences in physiological parameters were found between the two groups on the three different points in time. Also, no differences were found in levels of engagement, motivation and performance. The median scores on the engagement questionnaire were relatively low. An explanation could be the relatively young age of the participants with low levels of medical experience. Moreover, participation in the study had no consequences for their school results or reputation.

Some differences were found between the time points in both groups, such as a decrease in sympathetic nervous system and an increase in parasympathetic activity. These results are contradictive to the result of the increase in feelings of anxiety. The mismatch of the psychological and physiological modalities is a well-known problem in stress research. An explanation for this mismatch might be the difference in physiological reactivity patterns among persons, which makes a general assessment of physiological stress difficult. Second, measurement errors (eg, recall bias and random fluctuations of physiological features) might influence results. Finally, the translation of feelings in the brain to the peripheral physiological measures is a complex process that involves many intermediate steps that are influenced by various neural and hormonal factors. In this study, the observation of patterns in students' experience of stress and objective physiological parameters may be influenced by other confounders not included in this study design, for example, experience, social support and physical activity.

The final goal of SBME is to improve long-term learning. This study shows that the method of introducing a simulated scenario may influence the emotional assessment of the situation. It may be possible to make a simulation introduction too immersive, which may interfere with learning. With regard to learning, previous studies in SBME and other domains have shown that the effects on memory differ between the threat and the challenge appraisal. A threat response is suggested to impair memory retrieval, tasks that require remembrance of previously learnt information, with no impairments occurring in individuals who have a challenge response.^{18 37 38} This threat response may negatively influence performance during SBME. However, in SBME the final goal is to improve long-term learning. Previous studies also concluded that an elevated stress response, especially if cortisol is increased (threat response), is associated with enhancement of memory consolidation. This is the process when new and fragile memories are converted into more stable and permanent memories.³⁹ Nevertheless, extremely high levels of stress will impair this process.⁴⁰ This suggests that a threat response, without extremely high levels of stress, might result in increased memory consolidation. In this study, we did not find a difference in performance between the groups. This may have to do with differences in task requirements. The effect of the emotional assessment of the situation appears to vary somewhat according to the requirements of the task, for example, decision making, retrieval of learnt material or divided attention. The task on the laparoscopic simulator requires both physical and mental activities, while previous investigated tasks, such as solving drug calculation problems, more focus on mental requirements. Therefore, the threat response may have not resulted in decreased performance in this task. However, it could be that memory consolidation was positively increased. This could become an topic of interest for future studies.

So far, the method of introducing a simulated scenario during the prebriefing phase is a topic that does not receive a lot of attention. This is the first study, which compared a video-assisted introduction to a minimal textual briefing with a simulated task with regard to cognitive appraisal (threat or challenge response), feelings of anxiety, physiological parameters, engagement, motivation and performance. The strength of this study is the randomised design in a homogenous group of young healthy students without experience in simulation-based medical education in a scenario with low physical activity. In scenarios with physical activity, such as resuscitation scenarios, it will be hard to differentiate between the effects of physiological activity and psychological stress on physiological parameters as HR, HRV and SC. The idea of continuously measuring physiological parameters with comfortable materials became a possibility in our study. This made the physiological parameters, which are highly impressionable, more specific and reliable. It is technically complicated to measure HRV continuously without interrupting participants. Moreover, to calculate measurements of HRV we used the Fast Fourier Transform. However, small fluctuations in physiological parameters may not have been recorded. Other methods such as the Wavelet Transform may be better to analyse small fluctuations in physiological parameters because the duration needed to analyse is shorter for this method.

Measuring HR, HRV and SC continuously during the simulated scenario was possible, although these parameters may not be the best method of measuring stress in SBME at all to optimise the learning effectiveness, because the challenge and threat response will result both in increased sympathetic activity. Further research should investigate which methods to measure psychological and physiological stress are the most reliable and usable in simulation-based medical education to optimise the effectiveness of training.

Finally, this study only included a limited group size. This reduced the chance of detecting a true effect.

Moreover, there was a difference in the amount of information initially presented to the two groups. The detailed medical information in the introductory video compared with the textual briefing could have also influenced cognitive appraisal. More evidence is needed to confirm our results and to investigate what specific aspects of the introduction methods may cause a difference in the emotional assessment of the situation, and whether these study results can be generalised to healthcare students or professionals. At this stage, during the set-up of a simulation training trainers must overthink the way of introducing the simulated scenario. The method of the scenario introduction may interfere with learning.

Conclusion

This study suggests that the method of introducing a simulated scenario influences cognitive appraisal and feelings of anxiety. In this study, the immersive introduction resulted in a threat appraisal, while the minimal textual introduction led to a challenge appraisal. It may be possible to make your simulation introduction too immersive or stimulating, which may interfere with learning. Further research will be necessary to investigate the impact and usefulness of these findings on learning in simulation-based medical education.