The effect of distance learning via SMS on academic achievement and satisfaction of medical students

Abstract

INTRODUCTION:

Recently, medical education has made significant progress, and medical teachers are trying to find methods that have most impressive effects on learning. One of the useful learning methods is student active participation. One of the helpful teaching aids in this method is mobile technology. The present study aimed to determine the effect of sending educational questions through short message service (SMS) on academic achievement and satisfaction of medical students and compare that with lecture teaching.

SUBJECTS AND METHODS:

In an semi-experimental, two chapters of urology reference book, Smiths General Urology 17^th edition, were taught to 47 medical students of Isfahan University of Medical Sciences in urology course in 2013 academic year. Kidney tumors chapter was educated by sending questions through SMS, and bladder tumors part was taught in a lecture session. For each method, pretest and posttest were held, each consisting of thirty multiple choice questions. To examine the knowledge retention, a test session was held on the same terms for each chapter, 1 month later. At the end, survey forms were distributed to assess student's satisfaction with SMS learning method. Data were analyzed through using SPSS 20.

RESULTS:

The findings demonstrated a statistically significant difference between the two learning methods in the medication test scores. Evaluation of the satisfaction showed 78.72% of participants were not satisfied.

CONCLUSIONS:

The results of the study showed that distance learning through SMS in medical students could lead to increase knowledge, however, it was not effective on their satisfaction.

Introduction

Medical education is changing very fast.[1] Clinical teachers have an important role in the effectiveness of clinical education in supporting learners, encouraging reflections, and providing constructive and regular feedback.[2]

Familiarity to learning theories is a prerequisite for effective education.[3] Different learning strategies have used on different learning situations,[4] and many patterns are designed in according to learning theories.[1] In traditional method, students are faced with a vast amount of information to memorize, much of which seems irrelevant to the world as it exists outside of school. Students usually forget much of what they learned and that which they remember cannot often be useful to the problems and tasks that later face in the business world.[5] Researchers believed that using active methods such as problem-based learning, e-learning, case-based learning, and mobile learning (M-learning), students are becoming more interested and more efficient in learning.[2,6,7,8] M-learning has enormous potential for enhancing student engagement, active learning attitude, and course retention.[9,10] Many researchers have worked on the development of strategies to enhance learning effectiveness using mobile.[11] Many of them have emphasized M-learning engage the students in educational activities without a tightly physical location and are useful when the learner takes advantage of learning opportunities offered by mobile technologies.[12,13,14]

In medical education, the use of mobile devices has been shown to encourage students learning in.[12] M-learning offers versatility in content delivery ranging from text, videos, audios, graphics, animations, pictures, and games to interactive platforms[15] that offers widely available tools to deliver relevant information or learning materials to student. In the information processing theory, short message service (SMS) materials were one method of elaboration when students process medical information.[16,17] In one study, SMS interventions have been found effective in English vocabulary learning.[18] The study on enhancing nursing student's medication knowledge by SMS, the results demonstrated a significant increase in the medication knowledge score among the students.[19] In another study that performed on residents of obstetrical and gynecology has shown learning through text message has much better results on the test questions.[20]

There are remarkably few studies evaluating the efficiency and use of text messaging as an educational tool for medical students training. Hence, we decided to do this study on our medical students in urology course. The aim of this study was to determine the effects of sending educational questions through SMS on academic achievement and satisfaction of medical students and compare that with lecture teaching.

Subjects and Methods

Study design and subjects

We carried out a semi-experimental interventional trial study that conducted to compare medical teaching by sending questions through SMS with lecture method and determine the effects of these educating methods on academic achievement and satisfaction of medical students. Target population was the all 47 9^th semester medical students of Isfahan University of Medical Sciences in 2013 academic year that had chosen the urology course. The inclusion criteria were the medical students that had chosen urology course for the first time. The exclusion criteria were the absence of students on teaching or testing session and the students that did not receive the SMS.

In this study, two chapters of urology reference book, Smiths General Urology 17^th edition were chosen to teach in different methods. The kidney tumors chapter was chosen to be educated by sending questions through SMS, and the bladder tumors part was chosen to be taught in a lecture session in a random selection. The same level of contents of these chapters in difficulty and volume was submitted by three experienced teachers of urology group of the university.

Before the intervention, we held a session for the students to explain about our study and asked them to give us their mobile numbers and we pledged to protect their personal information, student information such as test scores.

At first, a pretest session consisting of thirty multiple-choice questions was held for each chapter. The questions had the educational content that was supposed to be taught, and they were chosen by the teacher from the urology question bank of the group. The chosen questions of two chapters were same at the level of difficulty. In addition, the validity of the questions were submitted by that three experienced teachers of the urology group, and Cronbach's alpha was calculated for reliability (α = 0.84).

To educate the kidney tumors chapter through SMS, the educational content was divided into thirty text questions. We sent one question per day at 8 AM. The students were responsible to find the answers of these thirty questions. The all references such as internet and books were available for them. We considered 1 week for students to study, then, we held a post-test session contain thirty multiple-choice questions. The content of questions was similar to sending questions through SMS. However, their appearance was different. These questions were chosen by the teacher from the urology question bank of the group, too. The level of difficulty of chosen questions was similar to pre-test questions. In addition, the validity and reliability of the questions were submitted by that three experienced teachers of the urology group. We considered a week after the post-test for students to rest.

We taught the other chapter in a lecture method. In this way, 1 week after, the post-test of the first method, the bladder tumors were taught in a lecture and showing slides session by the same teacher at 8 AM. After each 1 h teaching, the students rested for 10 min. At the end of the session, we considered 30 min for students to ask their questions about the topics that were presented. Similar to another teaching method, all references such as Internet and books were available for students. The presence of all students at this session was mandatory. Just like as the previous method, we considered 1 week for students to study and then held the posttest session. The questions of this test contained thirty multiple-choice questions that were chosen by the same teacher from the urology question bank of the group. The level of difficulty and differentiation of chosen questions were similar to pre-test questions. In addition, the validity and reliability of the questions were submitted by our three experienced teachers of the urology group, and Cronbach's alpha was calculated for reliability (α = 0.89).

To examine the knowledge retention of these two training methods, a delay test session was held on the same terms 1 month after the post-test for each chapter. The level of difficulty of questions was similar to post-test questions, and the appearance was different.

In all the pre-test, post-test, and delayed test, 1 point for each correct answer and 0 point for an incorrect or no answer was considered. The minimum and maximum points were 0 and 30.

At the end, the survey forms for SMS intervention were distributed among the participants. It was contained eight items rated on a 5-point Likert scale (1 = very dissatisfied; 5 = very satisfied). The students were asked rate satisfaction with the SMS learning experience, such as “using the SMS learning method in medical education.” The Cronbach's alpha of the satisfaction scale was 0.94. The validity of the questions was submitted by several experienced teachers of the urology group and medical education experts.

Sample size and sampling

This study was done according to the census of incoming students of Isfahan University of Medical Sciences to urology in 2013 academic year, so we did not sample.

Statistical analysis

The data analysis was done using statistical package for social science SPSS (ver 21. IBM Chicago). Data analysis was done in two descriptive and analytical parts. In descriptive statistics, we used tables and suitable statics such as percent and mean to express the data center. In addition, standard deviation (SD) and quarters were used to scattering data. Data analyze in comparison part was done by repeated measurement analysis. More accurate comparisons between two intervention groups due to repeated measurement were done by reliable post-tests like least significant difference.

Results

The study group consisted of 47 medical students with mean age 22.47 years (SD = 0.62). The minimum and maximum were 22 and 24 years, respectively. Most students were male (59.5%).

The average scores of pre-test, post-test, and delayed test are shown in Table 1.

Table 1.

Comparing average (standard deviation) scores of three tests

Independent t-test demonstrated that there was no difference between pretest scores (P = 0.38, t = −0.880), but the post-test and delayed test scores had significant difference in two educational methods, respectively (P < 0.05, t = 2.480) and (P < 0.05, t = 2.24).

Evaluating the satisfaction of teaching through SMS showed that 37 students were not satisfied (78.72%). The mean satisfaction level with SMS learning method was 2.76 (SD = 0.97), and it was significantly lower than the average of society (3) with P < 0.001. Most dissatisfaction of consequences of SMS learning was no impact on enhancing useful study hours. The average scores of every single questions of survey form are shown Table 2.

Table 2.

Comparing frequently distribution and average scores of satisfaction of short message service learning method

Discussion

The results of this study provided that using SMS for sending educational questions to medical students was significantly effective on their educational improvement. Even after 1 month of educating, their scores of the test were higher that of the comparison group who was taught with lecture method. This indicated that medical students can use SMS as a learning tool to enhance their medical knowledge. This informational delivering method also promotes regular study and leads students to self-directed learning. There was no similar intervention focusing on medical education in the literature.

The previous studies have shown positive effects of SMS on English vocabulary learning, but these studies have not evaluate the informational durability or have shown no long-term effective of SMS learning.[18,21] One study on 25 Iranian residents of obstetrics and gynecology that compared traditional teaching methods about breast cancer with learning through SMS, demonstrated that learning through receiving SMS including educational content significantly had better effects on learning.[20] In Chuang and Tsao study on enhancing nursing students’ medication knowledge through SMS, the results found a statistically remarkable increase over time in the medication knowledge score among students who received the SMS learning materials.[19]

Naderi et al. also performed the study that evaluated the effect of M-learning on operating room students’ metacognitive self-regulation and attitude provided learning through SMS was significant impressive on their metacognitive self-regulation and attitude.[22]

In these studies, teaching through mobile phone and SMS was used along with other educational methods such as lecture method, but we used M-learning as an independent educational method and compare that with lecture method.

Shekholeslami et al. performed a study that investigated the effectiveness of M-learning in training and education of the students of a virtual institute for higher education. Unexpectedly, they found no significant difference in learning effectiveness in intervention and control group. Just like our study, SMS learning was used as an independent educational method in their intervention.[23]

Also in our study, the findings supported there was significant difference between two groups on delayed test scores that revealed SMS learning method has strong effects on long-term memory of educational information.

Sending educational questions through SMS and ask students to find the answers as a self-directed learning method might be the factor that increase the durability of educational information. The students had to be more responsible for their learning, so they focused more on what they study and that made information more permanent in the memory.

Similar to our study, Chuang and Tsao found that learning nursing students through SMS might have strong effects on the near post-test scores, such as 1 or 2 weeks after the intervention and could retain the memory for almost 1 month.[19]

Lu's study found that the improvement on the English vocabulary learning was not retained in the delayed testes that were performed 3 weeks after the intervention.[21]

Unexpectedly, the students in SMS learning group reported above average not satisfaction level with this learning method. Teaching by sending educational question through SMS and making students responsible for their learning might be the factor to not be satisfied. Because the students were taught by lecture method in all the academic years, so they were not familiar to this new method and maybe it was hard for them to find the answers of questions.

Doosti et al. did one study that compared the strengths and weaknesses of different educational methods. Their founding showed that student-center teaching could create tension on teacher and student and made them to go back to traditional and familiar teaching methods such as lecture method, whereas the consequences of these new methods have so benefits for both teacher and student, but adopting them may take time.[1] These finding confirmed the dissatisfaction of the students of our study.

In return, in Chuang and Tsao study, the nursing students in the intervention group reported high satisfaction with learning medication materials through SMS,[19] however, in this method, teaching through SMS was used along with lecture method.

The survey results of Shekholeslami et al. study showed despite the lack of confirmation of effectiveness of M-learning, usage of mobile technology in learning can be favorable, and satisfactory.[23] Other researcher demonstrated the text message as useful in teaching anatomy or learning English vocabulary.[19]

In this study despite trying to select two similar chapters, these chapters were not exactly the same and it is suggested that more studies be done by selecting the same content.

In addition, since this study performed just in urology course, generalizing the study in other medical education courses is suggested.

This semi-experimental study was performed for the first time in medical education and showed that sending educational questions through SMS was effective on academic achievement of students, but more experimental studies are needed to prove this impact.

Conclusions

Sending educational questions through SMS can be used along other educational methods in medical education. More experimental and semi-experimental studies for other courses are recommended.

Financial support and sponsorship

This article is the result of a study of the project number 393435 sponsored by Isfahan University of Medical Sciences.

Conflicts of interest

There are no conflicts of interest.