Medical students' perceptions of using e-learning to enhance the acquisition of consulting skills

Abstract

Background

This study aims to evaluate medical students' perception of the usefulness and effectiveness of an e-learning package developed to enhance the acquisition of consulting skills.

Method

A survey with mixed method data analysis was conducted. Participants were 67 medical students completing their third year primary care rotation as part of a five-year degree at the University of Tasmania. Participants completed a 10 question anonymous online survey after using the elearning package

Results

Of the participants, 92% found it enjoyable and 95% found the e-learning package useful; 75% perceived it to be effective in increasing their performance and 91% believed it increased their knowledge in consulting skills. Benefits for participants' confidence, style and structure of consulting skills were found.

Conclusion

Participants found the e-learning package to be enjoyable and effective. E-learning should be further utilised in a blended learning environment to support face-to-face teaching in consulting skills.

What this study adds:

1. Utilises a blended learning approach.

2. Gives weight to the student voice by specifically evaluating student perceptions.

3. Knowledge of how students learn consulting skills

Background

E-learning has a well-established role in medical education and it has been found to be effective in enhancing learning and is well accepted by students.^1-3 E-learning can be used both as a stand-alone teaching tool and in a blended learning environment where it is linked to face-to-face teaching.

Developing good communication and consulting skills is fundamental for improving health outcomes and developing an effective therapeutic relationship.⁴ Consulting skills have traditionally been taught face-to-face. This study, however, looks at using a blended learning strategy to enhance the acquisition of these important skills and better utilise valuable face-to-face teaching time. There is evidence for the effectiveness of blended learning on results;^1,5 however, there is little literature exploring medical students' perceptions on the usefulness and effectiveness of elearning to complement face-to-face teaching.

A consulting skills programme, utilising standardised patients and video feedback for medical students has been developed at the University of Tasmania (UTAS). This style of teaching has been effectively used in medical education for some time.⁶'⁷ The process and content of consultations focuses on a patient-centred model considering the patient's perspective.⁴ Experience in this teaching and learning environment found students characteristically take the first one or two sessions to understand the concept and standards expected of them, leaving little time for them to develop and improve their skills. In response we have developed and produced a consulting skills e-learning package to complement and support the face-to-face teaching. An e-learning package was chosen for its flexibility and evidence of its quality and effectiveness.^8,9 It is important to create effective e-learning experiences the students perceive as meaningful. Therefore, in evaluating the use of the e-learning package the focus was aimed at the perceptions of the medical students using it.

The aim of this study was to evaluate medical students' perception of the usefulness and effectiveness of an elearning package to enhance the acquisition of consulting skills. This study, therefore, seeks to add to the literature on blended learning by focusing on student perceptions. It also adds to the current understandings of how students learn.

Method

We conducted an anonymous online survey. Eligible participants were medical students completing the third year primary care rotation as part of a five-year degree at UTAS. The study was run in the first semester of both 2009 and 2010. Students were given access to the consulting skills e-learning package to use during their eight week primary care rotation to support face-to-face teaching. After the rotation, all eligible participants were sent an email inviting them to participate in the trial. The email outlined the purpose, the voluntary nature of participation and emphasised the anonymity of the trial. This email also contained a web link to the online survey. This anonymous method of survey collection was chosen to encourage student participation. The survey responses were collected online and only available to the authors in a non-identifiable form. This trial received ethics approval from the UTAS Human Research Ethics Committee. Completion of the survey was taken as implied consent.

Description of e-learning package

The e-learning package includes: an audio-visual lecture recording, outlining the theory and expectations of the consulting skills program; videos of a third year student consulting with a standardised patient; videos of an experienced doctor performing consultations, utilising the methods outlined in the lecture and video examples of the feedback process used in the small group tutorials after each consultation. Students were able to view the elearning package as many times as they wish, at their own pace, whenever they perceive it to be useful. The online lecture runs for approximately one hour and the videos on average for approximately 10 minutes.

Evaluation of the e-learning package

The survey consisted of 10 questions. Three survey questions allowed for free text comments and the remainder had multiple set responses from which participants could choose. Demographic information was limited to age and sex to preserve anonymity. The 10 survey questions were designed by examining the work of Kirkpatrick¹⁰ and the four levels of evaluating training programmes: reaction; learning; behaviour; and results. The reaction of students was measured by asking them to rate their enjoyment in using the e-learning package by endorsing one of four options ranging from ‘extremely enjoyable’ to ‘not enjoyable’ or ‘unsure’. To measure learning, participants were asked to indicate whether they perceived the e-learning package greatly increased, slightly increased or did not affect their knowledge. Behaviour change was measured by asking how they believed their consulting skills would change as a result of using the elearning package in a free text format. Results of using the e-learning package were measured in several ways. Firstly, by asking students to rate their perception of the effect on their preparation for face-to-face consulting skills sessions as greatly increased, slightly increased, no increase or unsure. Secondly, they were asked to indicate whether they thought the e-learning package had improved their performance in consulting skills (yes, no or unsure). Participants were also asked how many times they viewed the e-learning package (one, two, three or more than three times), their purpose in using it and how useful they found it (extremely, very, somewhat, not useful or unsure). An additional question was included in the 2010 survey to obtain information regarding students' evaluation of the effect of using the e-learning package on improving their performance in the end of semester objective structured clinical examination (OSCE) results. We were not able to compare to actual results, as the data was anonymous.

Data analysis

Mixed method data analysis was undertaken. Data were analysed using simple descriptive statistics. Chi square analysis was also used to explore the relationship between demographic variables and measures used in the evaluation of the training package. Fisher's exact test is reported where expected cell counts were below five. Due to small numbers in some of the response categories for some variables categories were collapsed. A p-value of less than 0.05 (two tailed) was considered statistically significant. Statistical analyses were performed using SPSS version 17. Qualitative analysis involved thematic analysis of free text comments, which deepened the understanding of the student experience.¹¹

Results

Of the 108 eligible participants, 67 (62%) responded to the survey and completed the survey in full. Two participants were excluded from data analysis as in free text comments they stated they did not view the e-learning package. Thirty- six responses (55%) were collected in 2009 and 31 in 2010. The majority of participants were female (n=39 (60%)). Of participants, 61.5% (n=40) were aged 21–23 years. Other age ranges of participants were; 15% (n=10) 18–20 years, 14% (n=9) 24–30 years and 8% (n=5) over 30-years-old. One participant selected the option to not state their age. Analysis of the wider UTAS population found 52% of third year medical students in 2009/10 were female and the average age was 23 years.

Most participants (92.3%, n=60) rated the e-learning package as enjoyable. Of these participants, two (3%) found it extremely enjoyable, 29 (48%) found it very enjoyable and 29 (48%) found it somewhat enjoyable. Three (5%) participants did not enjoy using the e-learning package and two (3%) were unsure. Females were much more likely to report overall higher levels of enjoyment than males X² (2, N = 65) = 10.53, p = .004 (Figure 1).

Figure 1. Males and females ratings of enjoyment using the consulting skills e-learning package.

Similarly, a majority of participants (90.8%, n=59) reported a perceived increase in knowledge as a result of using the elearning package. Of these, 21 (36%) reported use greatly increased their knowledge and 38 (64%) reported a slight increase in knowledge. Six (9%) participants reported use of the e-learning package had no effect on their knowledge. Females were more likely to report increases in knowledge that males Χ² (2, N = 65) = 8.63, p = .013 (Figure 2).

Figure 2. Males and females ratings on perceived effects on knowledge of using the consulting kills e-learning package.

When asked about the effect of using the e-learning package on preparation for performing consulting skills, 25 (38%) participants believed the use of the e-learning package had increased their preparation. Of these, 21 (84%) reported use had greatly increased their preparation and four (16%) reported a slight increase in preparation. Thirty- nine (60%) participants reported use of the e-learning package led to no increase in preparation and one participant was unsure. There were no differences between males and females. Overall 75.4% (n=49) of participants reported improvements in their perception of their performance.

Most participants viewed the e-learning package at least three times (n=24, 37%). Twenty-three (35%) participants viewed the learning package once and 18 (28%) twice. There was no statistically significant difference between males and females with regard to the number of times the e-learning package was viewed. The e-learning package was, however, viewed significantly more often in the 2010 cohort when compared with the 2009 cohort Χ² (3, N = 65) = 9.65, p = .019. There were no significant differences between characteristics of the two cohorts that might explain this difference. There was a significant positive association between the number of times the e-learning package was viewed with increased enjoyment Χ² (4, N = 65) = 12.12, p = .008, increased knowledge Χ² (4, N = 65) = 12.11, p = .009, and usefulness Χ² (4, N = 65) = 10.34, p = .016.

Students were asked about their purpose in using the elearning package. They were given eight set options from which they could choose as many as were true for them. Positive selection of four options was the most frequent with a range of 1–7 options. Table 1 lists the responses for each option. Females were significantly more likely to state their purpose for using the e-learning package was to enhance learning in consulting skills Χ² (1, N = 65) = 5.02, p = .025 and also to help prepare for the practical sessions in consulting skills Χ² (1, N = 65) = 4.82, p = .028 (Figure 3). There were no significant differences between males and females with regard to all remaining options.

Table 1. Students' responses to purpose in using the e-learning package.

Purpose (options given)	Yes	No
	N (%)	N (%)
To understand the concept of consulting skills (CS)	35 (54)	30 (46)
To enhance learning in CS	36 (55)	29 (45)
To help prepare for the practical sessions in CS	45 (70)	20 (30)
To learn from watching how a practicing GP performs a consultation	39 (60)	26 (40)
To learn from watching a fellow student perform a consultation	33 (51)	32 (49)
To consider your own style of CS	29 (45)	36 (55)
To learn how to get the most out of a debrief session	12 (18)	53 (82)
To help achieve the learning outcomes in the primary care rotation	16 (35)	49 (75)

Figure 3. Males and females ratings of the e-learning package in enhancing learning and assisting in preparing for practical sessions.

Free text comments could be provided as an alternative to the set options or in addition. Qualitative analysis of free text comments enquiring about students' purpose in using the e-learning package was limited as only four participants provided comments. However, the theme of checking the expected standard and structure required was revealed in three of the four comments.

Qualitative analysis was undertaken of the free text responses to the question asking participants how they perceived their consulting skills changing as a result of using the e-learning package. Sixty-three (97%) participants provided a free text comment. Dominant themes of changes in confidence (8 comments), style (18 comments) and structure (17 comments) were revealed.

In rating the usefulness of the e-learning package the majority of participants (95%, n=62) found it to be useful. Of these, six (10%) participants rated it as extremely useful, 37 (60%) as very useful and 19 (30%) as somewhat useful. Two (3%) participants rated the e-learning package as not useful and one participant was unsure. Free text comments were also an option with this question and 20 participants made a comment. The dominant theme was the ongoing resource the package would provide.

Participants (n=29) in the 2010 survey were asked if they felt the use of the e-learning package would improve their performance in the end of semester OSCE. Twelve (41%) participants chose yes, six (21%) chose no and 11 (38%) were unsure. There was no statistically significant difference between males and females with regard to the usefulness of using the e-learning package on OSCE performance. We analysed age as a variable in determining outcomes from using the e-learning package and no statistically significant differences were noted.

Discussion

This study found medical students perceive the consulting skills e-learning package to be an enjoyable and useful resource. It was also perceived to be effective in increasing knowledge and performance in consulting skills. Females differed from males in reporting greater levels of enjoyment and gains in knowledge. The consulting skills e-learning package has been demonstrated, in the opinion of students to be a worthwhile complement to face-to-face teaching.

This study is consistent with the finding of other studies on the positive effect enjoyment has on learning.¹² It is also consistent with a previous finding of increased enjoyment in female nurses using a similar skill-based e-learning package.¹³ Student engagement with the e-learning package suggests it has allowed for active and deep learning to occur and therefore act as a useful complement to face-to-face teaching.¹⁴ A recent meta-analysis looking at Internet- based learning in health professionals found Internet-based learning to be favourable across a range of learning styles, contexts, topics and learning outcomes and showing a large effect on learning compared with no intervention and a similar effectiveness to traditional methods.¹⁵ An e-learning package was chosen so participants could view it as many times as they wished, whenever it suited them. It was therefore encouraging from a teaching and learning perspective to see most participants viewed the learning package more than once and that increased benefit was obtained by increasing the number of times it was viewed. The fact that it was viewed more frequently in 2010 compared with 2009 was also encouraging to see. A potential explanation for this may be that a favourable perception of the e-learning package in 2009 led students to recommend it to the following year and therefore increase its use.

Given the purpose for developing the e-learning package was to improve students' understanding of the concept and standards required in the face-to-face consulting skills teaching, it was surprising to see only 38% of participants believed it increased their preparation for consulting skills. This perhaps relates to students' overall perception of their sense of feeling prepared. It was also interesting to note 38% of participants were unsure if use of the e-learning package would increase their OSCE performance. This also perhaps tells us more about student confidence and selfperception of how they perform in examinations. These are both areas that could be further explored in a follow-up study with an enhanced qualitative component of analysis. The existing qualitative analysis did certainly add to the quantitative results. For example, exploring changes in consulting skills revealed several dominant themes for confidence, style and structure. The following quotes illustrate this well:

“I have become more confident and now believe I could manage management cases and cases requiring specific skills (e.g. counselling). It was good to see several different styles in practice and draw from them, to refine my own consulting skills method. I especially enjoyed some of the more challenging cases (e.g. grief, overseas patient, sexual history cases etc).”

“It was reassuring to know that I was conducting my consults in a similar way to a practicing GP. It has given me a lot of confidence in taking histories from patients. I have also picked up some tips on the value of asking a couple of additional questions and getting as much information as possible.”

“I think it was useful to see other people doing the consulting and I liked hearing about some different structures for consulting in the (online) lecture. I used this learning to partly inform the construction of my consulting pro-formas I use with patients.”

In contrast some participants did not comment, or were unaware of, any changes:

“Unsure, I doubt there will be any change.”

One quote illustrating the purpose of using the e-learning package highlighted exactly the reason for developing the elearning package:

“To gain some idea of the standard and level expected of us and the construct of sessions.”

Free text comments regarding the usefulness of the elearning package highlighted the benefits of using a blended learning approach and allowing students to make their own evaluations:

“I liked that the student examples were not scripted as perfect. It was helpful to identify what areas I thought the students missed and then see if the debrief agreed.”

Relevance in medical education is important and finding student resources relevant to the actual skills medical students need to be a competent doctor can be challenging. Increasing the relevance and efficacy of medical education warrants our focus and attention and therefore this evaluation of e-learning package adds to the literature in this regard.¹⁶ As a self-directed learning activity it also has promise in delivering high quality learning experiences in the current climate of increasing student numbers. Gormley et al³ have shown students feel quality online learning has a positive impact on their skills. Their study, however, compared e-learning to more traditional forms of clinical skills teaching. It must be remembered the focus of our study was evaluating e-learning in a blended environment, rather than encouraging e-learning as an alternative to face- to-face teaching.

Potential limitations of this study include the small participant numbers and the slightly higher participation in females (60%). This may reflect a preference for e-learning or a possible desire to please the rotation coordinator. This could be explored in greater depth in a follow-up focus group study to further investigate the benefits of e-learning. The ability of this study to detect any significant associations between age and outcomes was limited by the small numbers in several of the age range categories. UTAS accepts both local and international students. It also accepts mature age and postgraduate students as well as school leavers. In 2009 51% of third year medical students were male and the average age was 24 years. The age correlates well with this study's results. These factors increase the generalisability of these finding to a wider medical population. Due to the online, anonymous nature of the study we could not compare effectiveness to an objective measure of results such as their actual performance on the OSCE. This could be incorporated into a future study to further strengthen the results. As the study was undertaken by the coordinator of the primary care rotation, student responses may have been biased towards more favourable evaluations. This potential effect was, however, minimised through the use of an anonymous survey.

Conclusion

Participants found the e-learning package to be enjoyable, useful and effective in increasing knowledge in, and preparation for, consulting skills. E-learning should be further utilised in a blended learning environment to support face-to-face teaching in consulting skills and allow for flexibility in delivery of medical education. These results can also be used to show future students the potential benefits of the e-learning package available to them to stimulate and motivate increased use. Further studies should also explore more fully student experiences via focus groups and link use to OSCE results.