Abstract
INTRODUCTION:
Evaluation of students’ clinical performance is an essential part of nursing education. Mobile technology is one of the new methods of evaluation that has opened a new horizon for nursing professors. Therefore, this study aimed to design, implement, and evaluate mobile health technology in critical care nursing department in four-phase intervention.
METHODS:
This study was a four-stage educational intervention in which all postgraduate students of intensive care nursing in Isfahan University of Medical Sciences participated in the study. The four steps were designing and developing a logbook, needs assessment, designing and implementing an Android app, and evaluating users (students) of the Android app. Subjects’ satisfaction scores were collected using a researcher-made questionnaire with 14 questions. Data were analyzed using SPSS 14 software.
RESULTS:
In this study, a clinical practice evaluation app was designed at four levels of access (student, professor, department manager, and faculty dean). The results showed that more than half of app users reported this technology in terms of an overall response to the tool (87.5%), tool specificity (100%), the amount of information displayed (75%), page layout (62.5%), information rate (75%), recall information (87.5%), and ease of doing instructions (100%) were positive.
CONCLUSION:
Evaluation of clinical practice using technology was associated with increasing students’ satisfaction with evaluation method. In addition, this application enabled the simultaneous interaction between the professor with the higher and lower ranks, including the department manager and student and vice versa and the simultaneous recording of this interaction.
Keywords: Critical care, education, mobile applications, nursing, students
Introduction
Students’ evaluation is an appropriate guideline to predict their clinical skills because there is a significant positive relationship between the self-efficacy and performance of the students,[1] and its objective is to train students and benefit and benefit them from the necessary characteristics of their future careers; therefore, the existence of accurate assessment tools is of utmost importance.[2,3,4] There are several methods for clinical evaluation, of which the logbook is considered as one of the most effective ones.[5,6] Logbook is a booklet that provides content as a study guide and is also a clinical evaluation tool to assess students’ learning and evaluate the faculty's curriculum.[7] It is also worth noting that logbooks, like any other assessment tools, have advantages and disadvantages. Among its advantages are (1) a tool for recording learning experiences, (2) a tool for documenting learning stages, and (3) a tool for evaluating the internship training period. Regarding the disadvantages of this type of evaluation, financial problems and costs (such as printing, photocopying, filing, distributing, collecting, and re-distributing, and loss of the logbooks by the students) for professors and students, the time consuming filling out the logbook, the problem of carrying the logbook booklet and the impossibility to complete the logbook on the same day of the internship, and the lack of timely feedback by the professor can be mentioned. Therefore, it is necessary to use solutions that can cover these limitations.[2,5,6,7] Meanwhile, it is important to note that every new educational technique leads to learning, but the learning depth and sustainability are different in various educational methods, so using new assessment methods can help nursing professors to achieve this goal and learning depth.[8,9]
Among the new assessment methods in the educational system, electronic evaluation using e-learning and computer-based and mobile technology can be mentioned.[10,11] Mobile technology is one of the new assessment methods that has eliminated these limitations and has introduced a new horizon for nursing professors.[11] Mobile technology is one of the new concepts used to describe services supported by mobile devices such as patient monitoring devices, smartphones, and computer tablets. In clinical and educational settings, using mobile technology, like other methods of e-learning, has increased significantly.[12,13] In this regard, Shen et al. in their study reported that nurses were satisfied with the accuracy of administering medical orders and the safety of recording data with the use of personal digital assistants (PDAs).[14] Moreover, in George's study, the results showed that using the PDA in the clinical setting, classroom, and even for the daily schedule resulted in an increase in efficiency of 71% of nursing students. In addition, 100% of students found this tool to be an effective educational tool.[15] Furthermore, the results of a study by Kenny et al. showed that the use of the mobile system in the nursing system has facilitated training and interpersonal interaction. Therefore, it is recommended to use this technology in various fields of education.[16] However, most of the studies have been conducted on medical professors and their students, whereas few studies have been performed on nursing students and their professors. In total, according to the above results and considering the importance of using mobile technology in evaluating clinical performance in graduate students and considering that no study has been conducted in this field in Iran so far, the present study aims to design and promote mobile health education in special care settings.
Methods
The present study is a 4-step quasi-experimental educational intervention in which all master's degree students of Critical Care Nursing Department, School of Nursing and Midwifery Isfahan University of Medical Sciences participated in 2018. After the research was approved by the Ethics Committee of Isfahan University of Medical Sciences, the researcher started sampling by presenting a written introduction letter from Isfahan University of Medical Sciences and obtaining permission from faculty officials. The sample size was calculated by eight individuals based on the limited number of students in the master's degree, which included all students. Convenience sampling method was used in this study. To do so, all fourth-semester master's degree students in the intensive care department were invited to participate in the study. It is worth mentioning that the four steps of the present study are as follows: first step: design and development of a special logbook master of critical care nursing, step 2: performing a needs assessment of professors with the aim of determining the need for virtualization of logbooks, step 3: designing and implementing the Android Logbook Master application in accordance with the criteria and standards of the faculty, and step 4: checking the users’ satisfaction (students) of the Android application.
Inclusion criteria were as follows: the individuals should be willing to participate in the study, they should have access cellphones and Wi-Fi, and be a graduate student of critical care nursing. Exclusion criteria included simultaneous participation in another similar study and being absent for 1 day in internship. Data collection tool in this study was a questionnaire for evaluating users’ satisfaction of the application, which expresses the attitude of users regarding their best description of the application. Logbook Evaluation Questionnaire consists of 14 items and each item was measured on item on a 5-point Likert-type response scale in three dimensions: general reaction, general perception, and general attitude toward the application. The minimum score is 14 and the maximum is 70. Due to lack of availability of a standard questionnaire in this field, so this questionnaire was designed through use of countrywide national project, references, and textbooks, and its validity was confirmed by faculty members of the of Isfahan University of Medical Sciences. Its reliability was confirmed by use of a pilot study (questionnaire was given to 10 nursing students) and calculation of Cronbach's alpha (α = 0.80). Data were analyzed using SPSS 14 (SPSS Inc., Chicago, IL, USA) software and descriptive and analytic tests such as paired t-test and t-test. The significance level of these tests was considered P < 0.05.
Ethical considerations
Ethical aspects of this study were approved by the Nursing and Midwifery Care Research Center (approved research project (code: 297136). All of the subjects were informed about being free to participate in the research and nondisclosure of personal information. They all signed written informed consent.
Results
The present study includes four phases, and the results of the study, respectively, showed that in phase 1 of the study: logbook for master of critical care nursing, the library study method was prepared using the library study method and valid sources of clinical evaluation of students in the original languages including articles and books published in the past 5 years with a care approach and according to the principle of holistic care in nursing. This booklet consisted of five sections. The first section included standards related to general competence (including five dimensions: knowledge and understanding, communication skills, management skills, teaching methods, and observance of rules and regulations), and graduate students of critical care nursing must receive during their internship. In the second section, the special competence of the student in each of the clinical skills in the field of internship in the special intensive care unit of the relevant department was evaluated. In this section, the level of performance of each skill (observation, with assistance and independence) and the frequency of each of the specific skills based on the curriculum were covered. The third section consisted of a comprehensive nursing report using critical thinking/clinical judgment. The fourth section included a case study report, journal club, clinical conferences, presentation, and implementation of clinical guidelines. Finally, it was approved by all the professors and the director of the critical care nursing department, as well as the deputy director of education. In the second phase of the study, according to the announcement of the need of the relevant ministry for using electronic logbooks for master's degree and also the needs assessment done by most members of the Critical Care Nursing Department and also the director of the critical care nursing group, the need to separate the logbooks was announced. In this needs assessment, of 11 professors, 9 professors considered using electronic logbook as one of the educational needs. In addition, in the faculty educational council meeting with the presence of experts, the process for electronization of the master degree's logbook was discussed and with the consensus of the members, it was approved.
In the third phase of the study, the application was launched in collaboration with the professors of the Critical Care Nursing Department as well as the Information Technology Engineering Company. There are four levels of access in this application: the level relating to dean of the faculty, head of the department or director of the college, professors, and students. There were access restrictions at every level except for the head of the department or director of the college and the dean of the faculty. In such a way that at the level of head of the department or director of the college, first, by clicking on the names of the professors, the names of the sections, the dates of the internship, the names of the students, the logbook, and the skills and assessment, the messages from the professors and students are defined as the headlines of the software; however, the head of the department has access to all items. After logging in and entering their username and password, the head of the department can select the appropriate professor and select the activities performed by the students under his/her supervision on the intended date. The procedures and all the actions taken by the student are marked and visible on the same day. Moreover, there is an icon in this application and the head of the department can use it to leave a message for the relevant professor or for the student and vice versa. In addition, in the head of the department section, she/he can enter the internship program for professors during a course of study, and it is possible to prevent the interference of professors’ schedules. Another advantage of this application is the online evaluation of professors after the course, and the results of these evaluations are sent to both the head of the department and the professor. In the third phase of the study and after completing the design and fixing the defects, which lasted about a year, the final product with acceptable educational features was finally approved by the Deputy for Educational Affairs and head of the department, and in the educational meeting of the School of Nursing and Midwifery, the license to run the application was given to the Department of Critical Care Nursing. In the next step, the faculty members were asked to comment on the application, menus, items, and even page layouts. After applying the comments, in the next phase, the application was made available to students and professors of the Critical Care Nursing Department for an internship.
It should be noted that before this application is made available to students and runs, two professors in the field of intensive care nursing and two graduate students of critical care nursing who had completed their internships were asked to install the program on their cellphones and express their problems and comments. All comments were considered and all bugs were fixed, and the application was ready for the final stage. One of these problems was that different cellphones failed to install the application; however, this problem was solved with the support of Padideh Tegarat Company using Blue Stacks software. In addition, the professors complained that completing the application was time consuming; therefore, it was suggested that this program be installed on the computer in the college, and the BlueStacks program was used to solve this problem. At this phase, the students talked to the professors for about 2 days and presented their opinions and suggestions. In the fourth phase, after providing the students and professors with the application, the results showed that most students had a positive evaluation of the application in all aspects. Other points are reported in Tables 1-3.
Table 1.
Frequency percentage of overall feedback on the Logbook app in graduate students
| Variable Overall feedback on the Logbook app | n (%) |
|---|---|
| Terrible | 1 (12.5) |
| Extremely | 7 (87.5) |
| Boring | 2 (25) |
| Satisfactory | 6 (75) |
| Dull | 0 (0) |
| Stimulant | 8 (100) |
| Easy | 2 (25) |
| Hard | 6 (75) |
Table 3.
Frequency percentage of overall feedback attitudes on the Logbook app in graduate students
| Variables | n (%) |
|---|---|
| Reminder information | |
| Problem | 1 (12.5) |
| Easy | 7 (87.5) |
| Ease of doing | |
| Problem | 0 (0) |
| Easy | 8 (100) |
| The instructions provided for completing the logbook were confusing | |
| Never | 0 (0) |
| Always | 8 (100) |
| Tasks can be performed directly and forward | |
| Never | 4 (50) |
| Always | 4 (50) |
| This is a useful evaluation method | |
| Yes | 5 (62.5) |
| No | 3 (37.5) |
| The process of doing a job has a logical sequence | |
| Never | 0 () |
| Always | 8 (100) |
Table 2.
Frequency percentage of overall feedback variables on the Logbook app in graduate students
| Variables | n (%) |
|---|---|
| Tool feature | |
| Hard to read | 0 (0) |
| Easy to read | 8 (100) |
| The amount of information displayed | |
| Insufficient | 2 (25) |
| Enough | 6 (75) |
| Page layout was helpful | |
| Never | 3 (37.5) |
| Always | 5 (62.5) |
| The amount of information | |
| Irrational | 2 (25) |
| Logical | 6 (75) |
Discussion
This article aims to introduce the concept of using mobile technology to evaluate the clinical performance of graduate students of nursing students. It is also important to take into consideration that the design and construction of the mobile application has been done for the first time to evaluate students in Iran. In this regard, the use of mobile-based interventions in high-income countries has shown significant positive results in both education and economic issues.[17] This application does not have the limitations of the booklet logbook. These booklets are no portable and cause problems in transportation, you may lose the booklet, there is a lack of timely feedback and lack of timely registration of teachers’ feedback, registration is sometimes incomplete, etc. In this study, according to the user satisfaction questionnaire, nearly 80% of the research units evaluated that using this application was more useful compared to the booklet.
Inclusive satisfaction is one of the most important factors in using modern teaching methods. In this regard, Zolfaghari et al. (2007) in their study showed using e-learning method with appropriate interactive strategies and attractive virtual environments to motivate students is appropriate.[18] Mazlom, in their study aimed to design and evaluate the electronic software of the nursing process to improve learning and nursing care. The result of their study showed that using software can be a good way to encourage students to perform the clinical nursing. In fact, the main goal of technology is to show acceptance of different educational and assessment methods while at the same time integrate them to achieve effective learning. By focusing on the structure of mobile technology and examining the definitions and theories, in a general view, mobile technology can be considered as a flexible and comprehensive approach in education and facilitate the evaluation and learning process. Therefore, learners search and find the opportunity to choose the time and place of education, identify the desired educational form and tool that suits their specific needs, and can self-study more quickly and easily.[19]
It should be noted that despite the large investments that have been made in the development of mobile technology systems if users do not use the system, the investment will be wasted and useless. Therefore, recognizing the factors influencing the learner s’ satisfaction of these systems is of utmost importance. Mobile technology is one of the most important learning environments in the information era.
The present study showed that using e-learning methods in the clinical environment can be very practical and beneficial and has increased the possibility of simultaneous interaction between teacher and student. However, the present study, like any other study, had its limitations. These include the low-speed Internet access and the inability of learners to install the application. Therefore, this problem was partially controlled by providing an electronic file of trainings and making telephone calls to each learner to use the application. Another limitation is that this application cannot be installed on IOS phones such as Apple. Failing to conduct the study on more graduate students due to the low capacity of this field (eight individuals per year) and time constraints were other limitations.
Conclusion
Evaluating clinical performance using technology has been associated with increased students’ satisfaction with the assessment method. In addition, this application allows simultaneous interaction between the professor and higher and lower ranks, including the head of the department and the student, and vice versa, and simultaneous registration of this interaction. In addition, the system can enhance self-assessment in clinical settings and promote students’ learning of clinical skills in clinical settings. Therefore, nursing schools should conduct more studies to improve the results of learning and clinical evaluation in the context of e-learning.
Financial support and sponsorship
This study was financially supported by Isfahan University of Medical Sciences, Isfahan, Iran.
Conflicts of interest
There are no conflicts of interest.
Acknowledgments
This study was a part of a research project approved in September 2018 (IR.MUI.RESEARCH.REC.1397.443) in IUMS and was financially sponsored by vice chancellery of research in the related university. We greatly appreciate the support and cooperation of this vice chancellery.
References
- 1.Mohamadirizi S, Kohan S, Shafei F, Mohamadirizi S. The relationship between clinical competence and clinical self-efficacy among nursing and midwifery students. Int J Pediatr. 2015;3:1117–23. [Google Scholar]
- 2.Najafi F, Kermansaravi F, Mirmortazavi M, Gheisaranpour H. The Efficacy of logbook in clinical wards from the viewpoints of nursing faculty members and students. RME. 2017;9:64–55. [Google Scholar]
- 3.Jokar F, Haghani F. Nursing clinical education, the challenges facing: A review article. Iran J Med Educ. 2011;10:1153–60. [Google Scholar]
- 4.Bagheri H, Bazghaleh M. Clinical education and its related factors in nursing: A qualitative meta-synthesis study. J Nurs Educ. 2016;4:26–39. [Google Scholar]
- 5.Khalifehzadeh A, Tavasoli AA, Golshahi J, Sanei H, Mirdehghan A, Payehdar Z, et al. Synergy model in clinical teaching of critical care MSc Nursing students and cardiovascular diseases patients. Iran J Med Educ. 2011;10:593–601. [Google Scholar]
- 6.Lotfi M, Zamanzadeh V, Abdollahzadeh F, Davoodi A, Roshangar A. The effect of using logbook on nursing students’ learning. Iran J Med Educ. 2010;10:64–9. [Google Scholar]
- 7.Heidari H, Akbari N. Developing and implementing log book in teaching principles and techniques to nursing and midwifery students: Mixed method study. Future of Medical Education Journal. 2017;7(1):14–8. [Google Scholar]
- 8.Nasr-Esfahani M, Yazdannik A, Mohamadiriz S. Development of nursing students’ performance in advanced cardiopulmonary resuscitation through role-playing learning model. J Educ Health Promot. 2019;8:151. doi: 10.4103/jehp.jehp_125_18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Heydari F, Mohamadirizi S, Nasr-Esfahani M. The effects of multidisciplinary education for nurses and physicians on the management of patients with multiple trauma. Nurs Midwifery Stud. 2019;8:189–94. [Google Scholar]
- 10.Mohamadirizi S, Fahami F, Bahadoran P. The effect of E-Learning education on primipar women's knowledge about neonatal care. Iran J Neonatol. 2013;4:24–7. [Google Scholar]
- 11.Wilkinson A, Forbes A, Bloomfield J, Fincham Gee C. An exploration of four web-based open and flexible learning modules in post-registration nurse education. Int J Nurs Stud. 2004;41:411–24. doi: 10.1016/j.ijnurstu.2003.11.001. [DOI] [PubMed] [Google Scholar]
- 12.Chioh MS, Yan CC, Tang KL, Mustaffa SM, Koh MG, Sim TW, et al. The use of the personal digital assistant by nursing students in the classroom and clinical practice: A questionnaire survey. Singap Nurs J. 2013;40:38–44. [Google Scholar]
- 13.Bagherian A, Sadeghi M. Assessment of the effect of Tell-Demonstration-Observation-Action educational method on dental students’ quality of learning. Iran J Med Educ. 2012;11:764–6. [Google Scholar]
- 14.Shen LQ, Zang XY, Cong JY. Nurses’ satisfaction with use of a personal digital assistants with a mobile nursing information system in China. Int J Nurs Pract. 2018;24:e12619. doi: 10.1111/ijn.12619. [DOI] [PubMed] [Google Scholar]
- 15.George LE, Davidson LJ, Serapiglia CP, Barla S, Thotakura A. Technology in nursing education: A study of PDA use by students. J Prof Nurs. 2010;26:371–6. doi: 10.1016/j.profnurs.2010.08.001. [DOI] [PubMed] [Google Scholar]
- 16.Kenny RF, Van Neste-Kenny JM, Park CL, Burton PA, Meiers J. Mobile learning in nursing practice education: Applying Koole's FRAME model. Int J E Learn Distance Educ. 2009;23:75–96. [Google Scholar]
- 17.Aggarwal NK. Applying mobile technologies to mental health service delivery in South Asia. Asian J Psychiatr. 2012;5:225–30. doi: 10.1016/j.ajp.2011.12.009. [DOI] [PubMed] [Google Scholar]
- 18.Zolfaghari M, Mehrdad N, Parsa Yekta Z, Salmani Barugh N, Bahrani N. The effect of lecture and e-learning methods on learning mother and child health course in nursing students. Iranian journal of medical education. 2007;7:31–9. [Google Scholar]
- 19.Mazlom SR. Development and assessment of computerized software for nursing process: A step toward promotion of nursing education and care. Iran J Med Educ. 2014;14:312–22. [Google Scholar]