Background
Emergence of coronavirus disease 2019 (COVID-19) forced the worldwide higher educational institutes to adopt distance learning mode. Further, remote electronic exams (E−exams) were considered as mode of assessment. Objectives: This cross-sectional study evaluated the students' experience of remote E-exams during the COVID-19 pandemic among Medical Sciences students in Jordan. Materials and Methods: A survey of 29 questions was prepared on Google forms and distributed among students at Faculties of Medical Sciences (Medicine, Dentistry, Pharmacy, Nursing and Applied Medical Sciences) at Jordan University of Science and Technology. The questions include students' demographics, stress experience, and factors contributing to stress as well as behavioral changes related to remote E-exams. Responses were analyzed using descriptive, cross tabulation and Chi-square tests. Results: Among 1019 respondents, 32% reported more stress with remote E-exams. This was associated with academic major and gender. Among students with more stress during remote E-exams, the exam duration, mode of questions navigation and technical problems (exam platform and internet connectivity) appeared as the main factors related to stress in 78%, 76% and >60%, respectively. Other factors include concern regarding the teaching methods, exam environment and students' dishonesty. Remote E-exams had negative impact on students' dietary habits (increase consumption of caffeine and high energy drinks, high sugar food, fast food), sleep (reduction in sleeping hours, more consumption of insomnia medications), physical activity (less exercises) and smoking habits (increase). Conclusion: Results suggested a negative impact of E-exams on students within Medical Faculties. Robust exam platform and remote mock E-exams are recommended to reduce students' potential stress. A stress-free environment is very essential to encourage students to adopt remote E-exams, particularly if the pandemic will take longer. Various awareness programs about students' habits related to dietary, sleep quality, physical activity and smoking are highly valuable for students’ health benefits. This is of particular importance since the current students at Faculties of Medical Sciences are the future health care providers.
Keywords: Covid-19, Remote E-exams, Higher education, Stress, Behavioral changes, Medical sciences
Highlights
-
•
Remote E-exams are more stressful in one third of students from faculties of medical sciences.
-
•
Exam duration, mode of questions navigation and technical problems are major causes for students stress with remote E-exams.
-
•
Remote E-exam experience has negative impact on students' habits related to diet, sleep, physical activity and smoking.
1. Introduction
The emergent coronavirus disease-2019 (COVID-19) pandemic has led to a significant global crisis owing to the rapid spread and high morbidity and mortality statistics of this virus [1]. This pandemic is related to the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that seems to primarily spread by inhalation of respiratory droplets or direct contact with contaminated surfaces [2].
Several countries, including Jordan, adopted strict Non-Pharmaceutical Intervention (NPI) measures in attempts to achieve containment of the disease and to flatten the COVID-19 epidemiologic curve [3]. Globally, higher educational institutions have, fully or partially, closed their campuses to limit the rapid spread of SARS-CoV-2 infection. This caused massive disruption in teaching and learning [4]. A recent survey by the International Association of Universities (IAU) of higher education institutions across the world (N = 424, 109 countries) showed that more than 90% of surveyed institutions have replaced classroom instructions by remote teaching or are in the process of developing solutions to continue distance teaching and learning [5]. Despite the general perception of distance learning as being of less quality compared to face-to-face classroom learning, emerging works showed the opposite [6]. Indeed, the current advents in internet based-communications, videoconferencing applications and educational blogs have facilitated the delivery of the theoretical educational content [7]. However, practical and clinical courses are designed primarily to ensure students’ competence in routine clinical practice. Such skills and experience cannot be optimally gained without a supervised interactive experience [8].
The shift to online teaching raised important challenges for educational institutions for organizing examinations and ensuring the possibility for students to progress in their studies [9]. Electronic examinations (E-exams) were introduced as an effective mode of assessment with particular importance to provide immediate exams feedback, which is considered challenging for academic staff members due to the increase in students numbers [10]. Currently, E-exams are considered a key element in distance education [11].
A health crisis may accelerate the development and adoption of digital and online technologies that open up promising prospects for student assessment [9]. However, even with leading-edge technological infrastructure, the occurrence of technical failures during E-exams remains possible [12]. Although E-exams were adopted by many educational institutes as the mode of assessment, these were based on in-campus E-exams [13]. Students during in-campus E-exams receive similar treatments in terms of suitability of exam environment, technical support as well as exam invigilation [9]. However, taking remote E-exams means that students will undergo the exam at home that holds many other challenges. Although these challenges were described during routine E-exams, they might be exaggerated with remote application [9]. Among these are the possible technical problems that threaten the validity of an examination and the possible increase in dishonest behaviors among students [14]. It is worth mentioning that most in-campus E-exams were designed as mode of assessment for theoretical materials. However, remote E-exams have additional challenges related to difficulty in assessing practical knowledge and skills, all of which can impact achieving the purpose of learning itself [15].
Regarding students in Faculties of Medical Sciences, there was apprehension among students about the mode of assessment [16]. Additionally, there were concerns of how students' mental health will be affected with months of online learning and revision. Further, worries were raised regarding students’ preparedness as qualified health care providers [16]. Of note, the lack of clinical exposure and assessment of clinical skills has increased student stress and willingness to return to placement when time allows [16]. All these factors are expected to influence the future readiness of junior health care providers [16].
Jordan University of Science and Technology (JUST) has adopted remote online teaching as mode of students' education through the COVID-19 crises around mid-March [17]. In addition, remote E-exams were considered as mode of assessment for students' academic performance during the pandemic. Accordingly, this study assessed the experience of students at Faculties of Medical Sciences at JUST with regards to remote E-exams. We have focused on stress experience and factors associated with it as well as the impact of this experience on students' life style and dietary habits. Findings from this study will help to improve the remote E-exams methods and to reduce students' stress experience as well as the potential negative impact on student's health, particularly if traditional teaching activities will remain suspended.
2. Material and methods
2.1. Ethical statement
This study was approved by the Deanship of research and the Institutional Review Board at JUST (IRB number: 12/135/2020). The study was registered with the Research Registry (researchregistry6092) in accordance with the declaration of Helsinki. The study was conducted according to the guidelines of Strengthening the reporting of cohort studies in surgery (STROCSS) 2019 [18].
2.2. Study design
A focus group comprised of 4 faculty members actively involved in education of students at the Faculties of Medical Sciences (Medicine, Dentistry, Pharmacy, Nursing and Applied Medical Sciences) in JUST developed an initial version of the survey. The questionnaire was piloted on 30 students before a final version was developed and approved. Google form was utilized to collect the data using a link that was shared with participants through their e-learning accounts. A follow up reminder was sent after three and six days. The questionnaire was collected from August 22, 2020 to August 31, 2020.
The survey contained 29 questions that aimed to assess the students' experience of remote E-exams, with main focus on (1) determining factors affecting the stress of students during remote E-exams versus in-campus exams and (2) identifying the significant behavioral changes during the remote E-exams period. Factors related to students' stress during exams included constraints (E-exam platform problem, Internet connection problem), exam structure and preparedness (Exam duration, Questions difficulty, Not studying the whole exam material, Whether the exam is one way or two ways, Whether the exam is more than one form), course related factors (Teaching methods have not properly covered the material), and personal factors (The grade is not what student expects, students dishonesty, exam environment at home is not appropriate). Questions about behavioral changes during the period of remote E-exams included students’ habits related to dietary, sleeping hours, physical activity, smoking, social communications and use of medications (analgesics and medications to relief stress or insomnia).
2.3. Calculate minimum sample size for the survey
The minimum number of sample size which is required for this study was determined based on three factors 1) students’ population size 2) margin of error which was set to be ±5% 3) confidence level that was set to be 95% for this study design. The total number of students at Faculties of Medical Sciences in their first to the final year of study who were enrolled in the spring and summer of 2019/2020 academic year are as in Table 1.
Table 1.
Number of enrolled students by academic major.
| Student Faculty of Study | Number of Students |
|---|---|
| Medicine | 3524 |
| Dentistry | 1505 |
| Pharmacy | 2730 |
| Nursing | 1203 |
| Applied Medical Sciences | 2870 |
| Total | 11832 |
Based on the above information, the minimum number of sample size required was 372 surveys [19]. The number of participated students who included in the analysis was 1019.
2.4. Statistical analysis
Statistical packages Minitab version 17.0 was used to analyze the data. Descriptive statistics were used to describe the basic characteristics of the participants such as academic major and gender. Cross tabulation and Chi-square test was done to identify the association between student's experience of stress during exams and the demographic factors as well as association with factors that might contribute to stress during remote E-exams. The association between remote E-exams and students' behavioral changes as well as medications use was also analyzed using Chi-square test.
3. Results
3.1. Students’ characteristics
A total of 1019 students agreed to participate in the study and completed the survey. Approximately, half of the participants were from Faculty of Medicine (51.32%), while the other half in descending order were students from Faculties of Nursing, Dentistry, Pharmacy and Applied Medical Sciences (Table 2). Almost two-thirds of the respondents (65.55%) were females. Participants’ characteristics of are shown in Table 2.
Table 2.
Students’ characteristics.
| Variable | Number (%) |
|---|---|
| Academic major | |
| Medicine | 523 (51.32) |
| Dentistry | 138 (13.54) |
| Pharmacy | 112 (10.99) |
| Nursing | 165 (16.19) |
| Applied Medical Sciences | 81 (7.95) |
| Gender | |
| Female | 668 (65.55) |
| Male | 351 (34.45) |
3.2. Comparison of stress level for remote E-exams
Students were asked about their experience of exam related stress. According to Fig. 1, the majority of students (91%) self-reported exam stress. One third of respondents considered remote E-exams as more stressful in comparison to in-campus exams and one third considered both types of exams are stressful. In comparison, 23.55% reported in-campus exams as being more stressful, while only 8.73% of students did not express stress neither for remote E-exams nor for in-campus.
Fig. 1.
Students' experience of self-reported exam related stress.
Academic major was significantly associated with self-reported students' experience of stress during exams (Table 3). Among students’ major, Pharmacy, Nursing, and Applied Medical Sciences had the highest percent of their students experienced more stress with remote E-exams (66.96%, 43.03%, and 40.74% respectively). In comparison, only 23.33% and 29.71% of students from Faculties of Medicine and Dentistry, respectively, reported stress with remote E-exams.
Table 3.
The association of students’ academic major and gender with exam stress.
| Variable | Self-reported exam related stress |
Total (%) | P valuea | |||
|---|---|---|---|---|---|---|
| Remote E-exams more stressful | In-campus exams more stressful | Both exams are stressful | No stress during exams | |||
| Academic major | 0.000 | |||||
| Medicine (n = 523) | 23.33% | 29.83% | 37.48% | 9.37% | 100% | |
| Dentistry(n = 138) | 29.71% | 20.29% | 42.03% | 7.97% | 100% | |
| Pharmacy (n = 112) | 66.96% | 11.61% | 19.64% | 1.79% | 100% | |
| Nursing (n = 165) | 43.03% | 19.39% | 27.27% | 10.30% | 100% | |
| Applied Medical Sciences (n = 81) | 40.74% | 13.58% | 33.33% | 12.35% | 100% | |
| Gender | 0.000 | |||||
| Female (n = 668) | 38.17% | 20.36% | 36.38% | 5.09% | 100% | |
| Male (n = 351) | 24.79% | 29.63% | 29.91% | 15.67% | 100% | |
Analysis was done using Pearson Chi-square test.
Similarly, gender was significantly associated with self-reported stress during the exam (Table 3). Among female students from all Faculties, 38.17% reported that remote E-exams are more stressful compared with 24.79% of male respondents. The majority of the females experienced more stress during remote E-exams while the majority of males reported higher stress during in-campus exams.
3.3. Factors contributing to stress during remote E-exams
Students were asked about eleven factors that might contribute to stress during remote E-exams. The results of Chi-square analysis of these factors and their possible association with students' experience of stress during exams are shown in Table 4. A significant association was observed between students' experience of stress during exams and all studied factors, except not studying the whole exam material and whether the exam is more than one form. Among these factors, technical problems related to the E-exam platform or internet connections were both reported as factors contributing to stress in approximately two thirds of students who considered remote E-exams more stressful, compared with around 40% of students who considered in-campus exams as being more stressful. A significant association was found between students' experience of stress during exams and exam related factors such as exam duration, question difficulty, and mode of navigation between questions. Exam duration was reported as a factor for stress in 78.07% of students who considered remote E-exams as being more stressful in comparison to 54.17% of students who considered in-campus exams more stressful. The structure of the exam navigation mode between questions was also reported as a factor for stress in 76.32% of students who considered remote E-exams more stressful compared with 50.00% of students considering in campus exam as more stressful. The difficulty of exam questions was reported in 59.06% of students who considered remote E-exams more stressful compared with 37.92% of students considering in-campus exams more stressful. Teaching methods described as not properly covered the material was reported in 52.05% of students considering remote E-exams more stressful. Personal factors including student's grade, colleagues' dishonesty and exam environment at home were reported as important factors that increase stress during remote E-exams.
Table 4.
Factors associated with exams’ stress.
| Variable | Total students (% of Yes) | Self-reported exam related stress |
P valuea | |||
|---|---|---|---|---|---|---|
| Remote E-exams more stressful | In-campus exams more stressful | Both exams are stressful | No stress during exams | |||
| Technical e-learning platform problem | 52.80% | 0.000 | ||||
| Yes | 63.45% | 38.33% | 58.62% | 28.09% | ||
| No | 36.55% | 61.67% | 41.38% | 71.91% | ||
| Internet connection problem | 56.43% | 0.000 | ||||
| Yes | 66.08% | 42.08% | 60.92% | 40.45% | ||
| No | 33.92% | 57.92% | 39.08% | 59.55% | ||
| Exam duration (Time limit) | 70.56% | 0.000 | ||||
| Yes | 78.07% | 54.17% | 79.31% | 51.69% | ||
| No | 21.93% | 45.83% | 20.69% | 48.31% | ||
| Questions difficulty | 52.50% | 0.000 | ||||
| Yes | 59.06% | 37.92% | 61.78% | 30.34% | ||
| No | 40.94% | 62.08% | 38.22% | 69.66% | ||
| Not studying the whole exam material | 23.06% | 0.913 | ||||
| Yes | 23.10% | 22.92% | 23.85% | 20.22% | ||
| No | 76.90% | 77.08% | 76.15% | 79.78% | ||
| Whether the exam is one way or two way (free navigation) | 65.16% | 0.000 | ||||
| Yes | 76.32% | 50.00% | 70.40% | 42.70% | ||
| No | 23.68% | 50.00% | 29.60% | 57.30% | ||
| Whether the exam is more than one form | 16.58% | 0.196 | ||||
| Yes | 19.59% | 13.33% | 16.67% | 13.48% | ||
| No | 80.41% | 86.67% | 83.33% | 86.52% | ||
| Teaching methods have not properly covered the material) | 40.33% | 0.000 | ||||
| Yes | 52.05% | 29.58% | 41.38% | 20.22% | ||
| No | 47.95% | 70.42% | 58.62% | 79.78% | ||
| The grade is not what student expects | 38.96% | 0.000 | ||||
| Yes | 53.51% | 22.92% | 40.80% | 19.10% | ||
| No | 46.49% | 77.08% | 59.20% | 80.90% | ||
| Some students might cheat and affect student’ rank | 32.97% | 0.003 | ||||
| Yes | 39.47% | 30.83% | 31.32% | 20.22% | ||
| No | 60.53% | 69.17% | 68.68% | 79.78% | ||
| Exam environment at home is not appropriate | 30.81% | 0.000 | ||||
| Yes | 45.03% | 17.50% | 28.74% | 20.22% | ||
| No | 54.97% | 82.50% | 71.26% | 79.78% | ||
Analysis was done using Pearson Chi-square test.
3.4. Behavioral changes during remote E-exams period
The results of Chi-square tests for the association between students' stress and behavioral changes during remote E-exams are listed in Table 5. A significant association was found between students' experience of stress during exams and changes in dietary habits. It was noticed that around half of the students who reported more stress during remote E-exams also reported higher consumption of caffeine (47.37%). In addition, these students reported increased consumption of high energy drinks (19.01%), soda drinks (32.16%), fast food (37.13%) and high sugar food (52.34%) as well as reduction in eating healthy food (35.38%) in comparison to other students. Student's experience of stress during exams was also found to be significantly associated with changes in sleeping hours. Among students who reported more stress with remote E-exams, 44.15% had reduction in their sleeping hours and 28.65% reported more consumption of medications to relief insomnia. A significant association was also observed between students' experience of stress during exams and students' physical activity, with around half of students who considered remote E-exams more stressful reported a reduction in sports and exercises and this was less reported by other students. Changes in smoking habits were also found to be affected during remote E-exams period. Although all groups showed comparable percentages in relation to increased smoking, students who reported more stress during in-campus exams had the highest percentage for reduction in smoking (11.67%) in comparison to students reported more stress during remote E-exams (1.75%). Regarding social communications, students' experience of stress during exams was significantly related to time spent with family, with reduction was found in 44.74% of students who reported remote E-exams as being more stressful, while the highest percent of all other groups reported an increase in time spent with family during remote E-exams compared with in-campus exams. In comparison, most students with different experience of stress during exams reported an increase in communication using social media with similar percentage among all groups. Finally, students' experience of stress during exams was significantly associated with analgesics use, with 23.98% of students who reported more stress during remote E-exams had more consumption of analgesics. Of note, no significant association was observed between students' experience of stress during exams and the use of medication to relief stress among all students participated in this survey.
Table 5.
Behavioral changes during remote E-exams period.
| Variable | Self-reported exam related stress |
P valuea | |||
|---|---|---|---|---|---|
| Remote E-exams more stressful | In-campus exams more stressful | Both exams are stressful | No stress during exams | ||
| Caffeine consumption | 0.000 | ||||
| Increased | 47.37% | 22.08% | 35.34% | 38.20% | |
| Decreased | 9.36% | 28.75% | 10.92% | 7.87% | |
| No change | 28.36% | 35.42% | 40.80% | 34.83% | |
| Not applicable | 14.91% | 13.75% | 12.93% | 19.10% | |
| High energy drinks | 0.000 | ||||
| Increased | 19.01% | 12.08% | 15.23% | 10.11% | |
| Decreased | 4.68% | 16.67% | 6.90% | 6.74% | |
| No change | 21.93% | 22.50% | 23.28% | 30.34% | |
| Not applicable | 54.39% | 48.75% | 54.60% | 52.81% | |
| Soda drinks | 0.001 | ||||
| Increased | 32.16% | 18.75% | 24.43% | 13.48% | |
| Decreased | 6.14% | 14.17% | 10.34% | 8.99% | |
| No change | 36.26% | 36.67% | 35.92% | 47.19% | |
| Not applicable | 25.44% | 30.42% | 29.31% | 30.34% | |
| Eating healthy food | 0.000 | ||||
| Increased | 26.61% | 44.58% | 38.22% | 30.34% | |
| Decreased | 35.38% | 20.83% | 28.74% | 17.98% | |
| No change | 38.01% | 34.58% | 33.05% | 51.69% | |
| Eating fast food | 0.000 | ||||
| Increased | 37.13% | 25.83% | 26.44% | 25.84% | |
| Decreased | 27.49% | 40.83% | 41.09% | 31.46% | |
| No change | 29.53% | 26.67% | 28.16% | 25.84% | |
| Not applicable | 5.85% | 6.67% | 4.31% | 16.85% | |
| Eating high sugar food | 0.000 | ||||
| Increased | 52.34% | 25.00% | 39.08% | 26.97% | |
| Decreased | 11.11% | 29.17% | 21.26% | 13.48% | |
| No change | 32.75% | 40.83% | 33.05% | 49.44% | |
| Not applicable | 3.80% | 5.00% | 6.61% | 10.11% | |
| Sleeping hours | 0.000 | ||||
| Increased | 39.47% | 55.42% | 42.82% | 34.83% | |
| Decreased | 44.15% | 19.58% | 34.48% | 26.97% | |
| No change | 16.37% | 25.00% | 22.70% | 38.20% | |
| Exercise/sports | 0.000 | ||||
| Increased | 23.98% | 35.83% | 29.89% | 32.58% | |
| Decreased | 50.58% | 26.25% | 39.37% | 25.84% | |
| No change | 16.96% | 29.58% | 22.41% | 31.46% | |
| Not applicable | 8.48% | 8.33% | 8.33% | 10.11% | |
| Smoking habits | 0.000 | ||||
| Increased | 13.74% | 12.50% | 13.51% | 11.24% | |
| Decreased | 1.75% | 11.67% | 3.45% | 3.37% | |
| No change | 11.70% | 9.17% | 11.21% | 19.10% | |
| Not applicable | 72.81% | 66.67% | 71.84% | 66.29% | |
| Communications using social media | - | ||||
| Increased | 68.42% | 65.00% | 64.37% | 62.92% | |
| Decreased | 14.91% | 15.00% | 12.93% | 7.87% | |
| No change | 15.50% | 18.75% | 22.41% | 26.97% | |
| Not applicable | 1.17% | 1.25% | 0.29% | 2.25% | |
| Time spent with family | 0.000 | ||||
| Increased | 38.89% | 62.08% | 46.84% | 53.93% | |
| Decreased | 44.74% | 19.17% | 29.60% | 21.35% | |
| No change | 14.04% | 13.33% | 18.10% | 19.10% | |
| Not applicable | 2.34% | 5.42% | 5.46% | 5.62% | |
| Analgesics use | 0.000 | ||||
| Increased | 23.98% | 12.08% | 17.24% | 8.99% | |
| Decreased | 7.31% | 18.75% | 7.47% | 7.87% | |
| No change | 35.96% | 32.92% | 40.80% | 42.70% | |
| Not applicable | 32.75% | 36.25% | 34.48% | 40.45% | |
| Have used Medications to relief stress | 0.482 | ||||
| Yes | 20.76% | 19.58% | 20.11% | 13.48% | |
| No | 79.24% | 80.42% | 79.89% | 86.52% | |
| Have used Medications to relief insomnia | 0.045 | ||||
| Yes | 28.65% | 22.08% | 26.72% | 15.73% | |
| No | 71.35% | 77.92% | 73.28% | 84.27% | |
Analysis was done using Pearson Chi-square test.
4. Discussion
E-exams have caused a significant change in the educational history and have been substantially applied in the worldwide higher education institutes [20]. Although E−exams were adopted by many educational institutes as mode of assessment in Jordan, these were based on in-campus E-exams [21]. Emergence of COVID-19 forced the education system worldwide to immediately adopt online teaching as mode of learning. This has raised important challenges for educational institutions and faculties for organizing examinations and ensuring the possibility for students to progress in their studies [16]. This study aimed to explore the relationship between E-exam related stress and potential factors contributing to students’ stress during remote exams as well as with a cluster of behavioral changes among students enrolled in the Medical Sciences Faculties in Jordan.
Remote E-exams appeared as more stressful in almost one third of all students, while in-campus exams were reported as being more stressful by around one quarter of students. Since this is the first experience for students of remote E-exams, the unfamiliarity and the lack of essential properties in the E-exam system might contribute to students’ stress [11,22,23]. Previous studies also reported difficulty with remote E-exams [23]. Of note, the experience of stress among participants was found to be associated with the academic major and gender. Higher proportion of students at Pharmacy, Nursing and Applied Medical Sciences Faculties reported more stress with remote E-exams compared with Medical and Dental students. Limited evidence is available regarding the comparison of exam stress between different faculties. Manandhar et al. reported highest prevalence of stress among nursing students (75%) compared with medical and dental students [24]. This might be attributed to the type of academic assessment system that might vary among different Faculties of Medical Sciences [25]. In line with other studies, female students reported more stress with distance E-exams compared to male students [[26], [27], [28], [29], [30]]. This might be attributed to the way that female students respond to stressful events, while male students being less expressive of their worries [30,31]. Differences in female emotional intelligence, test stress, coping and academic stress were suggested to contribute to these observations [32].
Students' voices were captured about factors that might contribute to stress during remote E-exams. Technical problems (E-exam platform or internet connections) were reported as factors of stress during remote E-exams. A recent study also revealed internet connectivity as a major issue encountered in E-exams during the COVID-19 pandemic among students of Graphic Era Hill University, India [33]. We expect that internet connection might be an issue for students from lower-income families that do not own computer device or internet access [34]. In addition, it may be related to place of residence in Jordan, where internet connectivity is still a challenge in remote areas. The E-exam platform related issues might include problems with accepting the password which can cause a delay to start the exam as reported by previous studies [11,23]. In addition, problems with saving and restoring students’ answers throughout the exam system are suggested as reported previously by Wibowo et al. [11]. Therefore, a more robust platform is required.
In this study, the exam duration and mode of navigation (one way or free) between questions appeared as the major problems in more than 75% of students who considered remote E-exams as being more stressful in comparison to approximately half of students who considered in-campus exams more stressful. Time limit was previously considered as a source of doubts for E-exams [35], with more concern if exams were given remotely. This has been reported by medical students during the pandemic who considered that time might be limited to interpret all materials given and explore every question [16]. Factors that might affect the exam duration include insertion of password and waiting for acceptance as well as time needed to get oriented to the exams [23]. In addition, the navigation between the questions might be confusing and time taking, which add more stress on students [11]. The mode of navigation between questions was previously reported as a student concern, with one-way exams not allowing to access the previous questions is not favorable to students [11].
Other factors reported by students include that questions are not appropriate to teaching given and whether their grade will be affected. This might be due to the unreadiness of both students and instructors for E-learning and teaching, as the usual mode of teaching is in-class lectures, which might add stress on students undergoing remote E-exams. Recent findings revealed that resources, staff readiness, confidence, student accessibility and motivation have valuable roles in achieving integrated learning, particularly during these exceptional circumstances [36,37]. These factors in turn can improve the students’ perception regarding the learning methods that can be reflected as better experience for remote E-exams.
Academic dishonesty was also considered as an issue of concern, mainly among students who considered remote E-exams more stressful. Students during in campus E-exams receive similar treatments in terms of exam invigilation. However, taking distance E-exams means that students will undergo the exam at home that holds many challenges including the possible increase in dishonest behaviors among students and the need to ensure a fair treatment of test takers [14]. Student authentication checking systems, if applied, might influence the assessment process [38]. Since this is the first experience to adopt remote assessment, proctoring/validation systems should be applied to avoid any academic dishonesty [39].
The third part of the survey was about the behavioral changes during the period of remote E-exams in comparison to in-campus exams. This was done in an attempt to identify the impact of remote E-exams experience during the pandemic on students' habits. A significant association was found between students’ experience of stress during exams and changes in dietary habits. Higher consumption of caffeine (47.37%), high energy drinks (19.01%), soda drinks (32.16%), fast food (37.13%), and high sugar food (52.34%) as well as reduction in eating healthy food were reported in students who had more stress during remote E-exams compared with others. Students at Faculties of Medical Sciences are under high stress level with demanding study hours. A recent study among medical students showed that most medical students reported low consumption of caffeine during non-exam times, whereas consumption was considered as moderate during exam period [40]. Caffeine was mainly consumed to keep students more alert for exam preparation and was linked to loss of sleep during exams days [40]. A recent study among Lebanese university students has shown that the total daily intake of caffeine during exam days was alarmingly above the Food and Drug Administration (FDA)-approved daily doses [41]. A study at Universities in Trnava reported better mood, more concentration, increased physical activity and hyperactivity in respondents after the energy drinks consumption, while sleep suppression was reported by the majority [42]. However, consumption of coffee and energy drinks, for the purpose of neuroenhancement was found to be increased during the week before the exams among Bosnian-Herzegovinian university students and was correlated with exam stress level [43].
Stress is recognized to have effect on dietary choices, with increasing consumption of unhealthy food including fat and sweet was reported during stressful times. Nevertheless, some individuals might be more susceptible for the unhealthy choices due to certain psychological factors [44]. A recent study comparing students in the exam conditions versus the regular days, reported effects of stress on appetitive brain function [45]. Self-control was suggested to be affected by a personality trait which might cause overeating in susceptible subjects upon exposure to stressful condition [45]. In addition, a recent study has reported changes in students’ dietary habits during the exams period such as decrease in diet quality, lower fruit and vegetables intake, higher fast food intake and more difficulties to eat healthy in students during the exams period [44]. Students having more stress were at higher risk for exam-induced unhealthy dietary choices. Prevention measures should consider psychological and lifestyle aspects with emphasis on stress control, nutritional education, awareness programs for eating-without-hunger and highlighting the need for health environment [44].
Sleep is an essential circadian cyclic process that has a key role in subjects’ health [46]. In this study, students who reported more stress with remote E-exams, showed the highest percentages for reduction in their sleeping hours and more consumption of medications to relief insomnia compared with other students. Previous studies on the sleep quality of students of Medical Sciences during final exams reported that more than 85% students had sleep disorders [46,47]. The use of sleep medications was also reported among students of medical sciences [46,48]. Using medications to aid in sleep was considered as an approach to manage insomnia caused by many sleep problems [49]. Recently, among university students in Jordan, 9% reported using medication to aid sleep. However, the study has not investigated this during exam stress period [49]. Previous studies reported that students who had less sleep were more likely to forget items studied over short time intervals. These findings in adolescents demonstrate the importance of combining good study habits and good sleep habits to optimize learning outcomes [50].
Evidence from the literature indicates that the experience of stress impairs efforts to be physically active [51]. Around half of students who considered remote E-exams more stressful reported a reduction in sports and exercises and this was less reported by other students. However, results from a two-arm parallel randomized controlled trial showed that exercise can be considered to relief fatigue caused by students' study [52]. In addition, physical activity during the exam period was suggested to counteract the negative impact of stress on students’ health. Therefore, students should be encouraged to increase their activity levels particularly at stressful periods to prevent negative consequences on their sleep and health [53].
Smoking is still prevalent in the medical student population groups [54]. In this study, all smoker students with different stress experience showed comparable percentages in relation to increased smoking, however, students who reported more stress during in-campus exams had the highest percentage for reduction in smoking (11.67%) in comparison to students reported more stress during the remote E-exams (1.75%). Increased tobacco smoking was reported in medical students in Egypt and was related to anxiety [55]. Stress and adaptation mechanisms were main factors for smoking among Malta medical students [54]. Interrelationship among smoking habit and exposure to stressful situations in college students in Iraq showed more nicotine dependence among stressed students [56]. Recently, smoking was reported to have a negative effect on the students' grade point average (GPA) among medical, dental and pharmacy students in Jordan [57]. Therefore, it is necessary to increase awareness among students regarding the potential effects of these factors on their academic performance [57].
During the lockdown when people were forced to stat at homes for a long time, direct interactions with family have a supporting role [33]. In this study, a reduction in time spent with family was found in 44.74% of students who reported remote E-exams as being more stressful, while the highest percent of all other groups reported an increase in time spent with family. A recent study among students to identify activities carried out by students to spend their time apart from study during the lockdown showed that spending time with family was reported in 41% of students [33]. The data revealed that most students spent good amount of time with the family members and they were directly engaged with family, which is recognized as a good indication for appreciation of family values [33]. However, stress related to remote E-exams might have negative impact on students’ communication with their family and thus measures should be taken to reduce stress experience.
The use of self-medication represents a health problem among university students. Students’ experience of stress during exams was significantly associated with analgesics use, with 23.98% of students who reported more stress during remote E-exams had more consumption of analgesics. Recent study reported the use of self-medication in 69.3% of the students from Jordanian universities. The most frequent medications used were analgesics (61.3%) [58]. However, the study did not report whether consumption is increased during exams period. Of note, although students reported stress with remote E-exams, no significant difference was observed regarding the intake of medications to relief stress.
A limitation of this study is lack of generalizability of the findings since it was conducted in a single institution. Suspension of teaching activities during the pandemic might be a stressful experience for students and thus may have affected the responses to this survey.
5. Conclusion
In this study, around one third of students from Faculties of Medical Sciences reported stress with remote E-exams during the Covid-19 pandemic. The main factors related to students' stress include exam duration, navigation mode and technical problems. Other factors include concern that teaching methods have not properly covered the material, the exam environment at home and students’ dishonesty. Therefore, robust exam platforms and conducting remote mock E-exams are recommended to reduce the stress with remote E-exams. A stress-free environment is highly valuable to encourage students to adopt remote E-exams, particularly if the pandemic will take longer. Findings from this study can be considered as an internal audit that may help ameliorate shortcomings observed in conducting remote E-exams during the outbreak period.
In this study, the students' experience of remote E-exams were shown to have negative impact on their dietary habits, sleep, physical activity and smoking habits. Accordingly, awareness programs are highly recommended for students' long lasting well-being. Awareness programs are also needed to encourage students to increase their physical activity during stressful periods which in turn can improve the students sleep quality, reduce stress and affect students’ performance during exams. Smoking is a major problem among university students, which highlights the need for educational national programs for smoking cessation. This is of particular concern that the current students at Faculties of Medical Sciences are the future health care providers.
Funding
No funding received.
Ethical approval
This study was approved by the Deanship of Research and the Institutional Review Board at Jordan University of Science and Technology (IRB number: 12/135/2020).
Consent
This study was approved by the Deanship of Research and the Institutional Review Board at Jordan University of Science and Technology (IRB number: 12/135/2020).
The survey started with the following sentence:
“Please note that your valid participation in this research study is voluntary. You may choose not to participate. This survey is anonymous, and we do not collect identifying information such as your name or email/IP address.
Author contribution
L.E: study concept, design, data collection, data interpretation, writing the paper.
N.A: study design, data analysis and interpretation, writing the paper.
A.J: data Collection, writing the paper
N.O: data collection, editing the manuscript.
A.S: data collection.
K.K: data interpretation, editing the manuscript.
All authors have read and agreed to the final manuscript and responsible for similarity index.
Registration of Research Studies
Name of the registry: Research Registry.
Unique Identifying number or registration ID: researchregistry6092.
Hyperlink to your specific registration (must be publicly accessible and will be checked): https://www.researchregistry.com/browse-the-registry#home/.
Guarantor
Dr. Lina Elsalem.
Declaration of competing intrest
Authors declare no conflict or competing interest.
Acknowledgements
Many thanks for all students participated in this study.
Footnotes
Supplementary data related to this article can be found at https://doi.org/10.1016/j.amsu.2020.10.058.
Appendix A. Supplementary data
The following is the supplementary data related to this article:
References
- 1.Tabish S.A. COVID-19 pandemic: emerging perspectives and future trends. Journal of public health research. 2020;9(1) doi: 10.4081/jphr.2020.1786. 1786-1786. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Sharma A. Severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2): a global pandemic and treatment strategies. Int. J. Antimicrob. Agents. 2020;56(2) doi: 10.1016/j.ijantimicag.2020.106054. 106054-106054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kheirallah K.A. The effect of strict state measures on the epidemiologic curve of COVID-19 infection in the context of a developing country: a simulation from Jordan. Int. J. Environ. Res. Publ. Health. 2020;17 doi: 10.3390/ijerph17186530. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Malee Bassett R., Arnhold N. World Bank Blogs; 2020. COVID-19's Immense Impact on Equity in Tertiary Education.https://blogs.worldbank.org/education/covid-19s-immense-impact-equity-tertiary-education Avaiable from: Last accessed. [Google Scholar]
- 5.Marinoni G., Land H.v.t., Jensen T. International Association of Universities Global Survey Report; 2020. THE IMPACT OF COVID-19 ON HIGHER EDUCATION AROUND THE WORLD.https://www.iau-aiu.net/IMG/pdf/iau_covid19_and_he_survey_report_final_may_2020.pdf Avaiable from: Last accessed. [Google Scholar]
- 6.Hodges C. EDUCAUSE Review; 2020. The Difference between Emergency Remote Teaching and Online Learning.https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning Avaiable from: Last accessed. [Google Scholar]
- 7.Natarajan M. Vol. 26. Information Services & Use; 2006. Use of online technology for multimedia education; pp. 249–256. [DOI] [Google Scholar]
- 8.McCutcheon K., O'Halloran P., Lohan M. Online learning versus blended learning of clinical supervisee skills with pre-registration nursing students: a randomised controlled trial. Int. J. Nurs. Stud. 2018;82:30–39. doi: 10.1016/j.ijnurstu.2018.02.005. [DOI] [PubMed] [Google Scholar]
- 9.OECD . Vol. 6. OECD Publishing; Paris: 2020. Remote Online Exams in Higher Education during the COVID-19 Crisis. (OECD Education Policy Perspectives). [DOI] [Google Scholar]
- 10.Nuha Alruwais G.W. Mike wald, advantages and challenges of using e-assessment. International Journal of Information and Education Technology. 2018;8(1):34–37. doi: 10.18178/ijiet.2018.8.1.1008. [DOI] [Google Scholar]
- 11.Wibowo S. A pilot study of an electronic exam system at an Australian university. J. Educ. Technol. Syst. 2016;45(1):5–33. doi: 10.1177/0047239516646746. [DOI] [Google Scholar]
- 12.Hillier M., Kumar N., Wijenayake N. E-examinations: the impact of technology problems on student experience. In: Brinda T., Passey D., Keane T., editors. Empowering Teaching for Digital Equity and Agency. OCCE 2020. vol. 595. Springer; Cham: 2020. (IFIP Advances in Information and Communication Technology). [DOI] [Google Scholar]
- 13.Dermo J. e-Assessment and the student learning experience: a survey of student perceptions of e-assessment. Br. J. Educ. Technol. 2009;40(2):203–214. doi: 10.1111/j.1467-8535.2008.00915.x. [DOI] [Google Scholar]
- 14.Chirumamilla A., Sindre G., Nguyen-Duc A. Assessment & Evaluation in Higher Education; 2020. Cheating in e-exams and paper exams: the perceptions of engineering students and teachers in Norway; pp. 1–18. [DOI] [Google Scholar]
- 15.Mohmmed A.O. Emergency remote teaching during Coronavirus pandemic: the current trend and future directive at Middle East College Oman. Innovative Infrastructure Solutions. 2020;5(3):72. doi: 10.1007/s41062-020-00326-7. [DOI] [Google Scholar]
- 16.Birch E., de Wolf M. A novel approach to medical school examinations during the COVID-19 pandemic. Med. Educ. Online. 2020;25(1):1785680. doi: 10.1080/10872981.2020.1785680. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Sindiani A.M. Distance education during the COVID-19 outbreak: a cross-sectional study among medical students in North of Jordan. Annals of Medicine and Surgery. 2020 doi: 10.1016/j.amsu.2020.09.036. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Agha R. STROCSS 2019 Guideline: Strengthening the reporting of cohort studies in surgery. Int. J. Surg. 2019;72:156–165. doi: 10.1016/j.ijsu.2019.11.002. [DOI] [PubMed] [Google Scholar]
- 19.Israel G.D. 1992. Determining Sample Size1. Agricultural Education and Communication Department. Available from: http://zulsidi.tripod.com/pdf/DeterminingSampleSizes.pdf. Last accessed. [Google Scholar]
- 20.Fluck A.E. An international review of eExam technologies and impact. Comput. Educ. 2019;132:1–15. doi: 10.1016/j.compedu.2018.12.008. [DOI] [Google Scholar]
- 21.Alsamarai S., Amawi A., Eljinini M.A.H., E-Assessments Systems A comparison of the views of samples of students and professors. Manag. Sci. Eng. 2014;8(3) doi: 10.3968/5566. [DOI] [Google Scholar]
- 22.Da'asin K.A. Attitude of ash-shobak university college students to E-exam for intermediate university degree in Jordan. J. Educ. Pract. 2016;7:10–17. https://files.eric.ed.gov/fulltext/EJ1095708.pdf [Google Scholar]
- 23.Thomas P. Remote electronic examinations: student experiences. Br. J. Educ. Technol. 2002;33(5):537–549. doi: 10.1111/1467-8535.00290. [DOI] [Google Scholar]
- 24.Manandhar S., Pramanik T. Stressors and the levels of stress among the undergraduate medical, dental and nursing students of a medical college in kathmandu. Nepal Medical College Journal. 2019;21(1):21–25. doi: 10.3126/nmcj.v21i1.24842. [DOI] [Google Scholar]
- 25.Ali M. Does academic assessment system type affect levels of academic stress in medical students? A cross-sectional study from Pakistan. Med. Educ. Online. 2015;20(1):27706. doi: 10.3402/meo.v20.27706. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Sreedevi Arepalli. Study on stress among first-year medical students of Kurnool Medical College, Kurnool. Int. J. Med. Sci. Publ. Health. 2016;5(5):852–855. doi: 10.5455/ijmsph.2016.31082015141. [DOI] [Google Scholar]
- 27.Abdulghani H.M. Stress and its effects on medical students: a cross-sectional study at a college of medicine in Saudi Arabia. J. Health Popul. Nutr. 2011;29(5):516–522. doi: 10.3329/jhpn.v29i5.8906. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Eva E.O. Prevalence of stress among medical students: a comparative study between public and private medical schools in Bangladesh. BMC Res. Notes. 2015;8:327. doi: 10.1186/s13104-015-1295-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Hashmat S. Factors causing exam anxiety in medical students. JPMA. J. Pakistan Med. Assoc. 2008;58(4):167–170. https://jpma.org.pk/article-details/1364 [PubMed] [Google Scholar]
- 30.Babar M.G. Perceived sources of stress among Malaysian dental students. Int. J. Med. Educ. 2015;6:56–61. doi: 10.5116/ijme.5521.3b2d. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Al-Saleh S.A. Survey of perceived stress-inducing problems among dental students, Saudi Arabia. Saudi Dent J. 2010;22(2):83–88. doi: 10.1016/j.sdentj.2010.02.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Gordana Stankovska D.D. vol. 16. 2018. Slagana angelkoska, zebide ibraimi & valbona uka, emotional intelligence, test anxiety and academic stress among university students; pp. 157–164.https://eric.ed.gov/?id=ED586176 (Education in Modern Society). [Google Scholar]
- 33.Bisht Raj K., Jasola S., Bisht Ila P. 2020. Acceptability and Challenges of Online Higher Education in the Era of COVID-19: a Study of Students' Perspective.https://www.emerald.com/insight/content/doi/10.1108/AEDS-05-2020-0119/full/html [Google Scholar]
- 34.Hasan N., Bao Y. Impact of “e-Learning crack-up” perception on psychological distress among college students during COVID-19 pandemic: a mediating role of “fear of academic year loss”. Child. Youth Serv. Rev. 2020;118:105355. doi: 10.1016/j.childyouth.2020.105355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Bernik I., Jereb E. Proceedings of the 5th WSEAS International Conference on Education and Educational Technology. 2005. Students ’ readiness for electronic examinations.http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.506.7030&rep=rep1&type=pdf [Google Scholar]
- 36.Hampton D. Self-efficacy and satisfaction with teaching in online courses. Nurse Educat. 2020 doi: 10.1097/nne.0000000000000805. [DOI] [PubMed] [Google Scholar]
- 37.Mishra D.L., Gupta D.T., Shree D.A. Online teaching-learning in higher education during lockdown period of COVID-19 pandemic. International Journal of Educational Research Open. 2020:100012. doi: 10.1016/j.ijedro.2020.100012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Mellar H. Addressing cheating in e-assessment using student authentication and authorship checking systems: teachers' perspectives. International Journal for Educational Integrity. 2018;14(1):2. doi: 10.1007/s40979-018-0025-x. [DOI] [Google Scholar]
- 39.Weiner J.A., Hurtz G.M. A comparative study of online remote proctored versus onsite proctored high-stakes exams. Journal of Applied Testing Technology. 2017;18(1):13–20. https://eric.ed.gov/?id=EJ1142330 [Google Scholar]
- 40.Devi S.S.L., Abilash S.C., Basalingappa S. The rationale of caffeine consumption and its symptoms during preparatory and non-preparatory days: a study among medical students. Biomedical and Pharmacology Journal. 2018;11:1153. doi: 10.13005/bpj/1476. [DOI] [Google Scholar]
- 41.Khalil M., Antoun J. Knowledge and consumption of caffeinated products by university students in Beirut, Lebanon. Clinical Nutrition ESPEN. 2020;37:213–217. doi: 10.1016/j.clnesp.2020.02.014. [DOI] [PubMed] [Google Scholar]
- 42.Stanciak J., Boronova J., Vareckova L. Energy drinks consumption by students at Universities in Trnava. Postmod. Openings. 2020;11(2 Supl 1):16–25. doi: 10.18662/po/11.2Sup1/176. [DOI] [Google Scholar]
- 43.Kusturica J. Neuroenhancing substances use, exam anxiety and academic performance in Bosnian-herzegovinian first-year university students. Acta medica academica. 2019;48(3):286–293. doi: 10.5644/ama2006-124.269. [DOI] [PubMed] [Google Scholar]
- 44.Michels N. Dietary changes and its psychosocial moderators during the university examination period. Eur. J. Nutr. 2020;59(1):273–286. doi: 10.1007/s00394-019-01906-9. [DOI] [PubMed] [Google Scholar]
- 45.Neseliler S. Academic stress and personality interact to increase the neural response to high-calorie food cues. Appetite. 2017;116:306–314. doi: 10.1016/j.appet.2017.05.016. [DOI] [PubMed] [Google Scholar]
- 46.Nasiri K. Sleep quality of students during final exams in ardabil and khalkhal university of medical sciences. Int. Electron. J. Med. 2019;8(1):66–72. doi: 10.31661/iejm893. [DOI] [Google Scholar]
- 47.Aghajanloo A. Sleep quality of students during final examsin zanjan university of medical sciences. Modern Care Journal. 2011;8(4):230–237. https://www.sid.ir/en/journal/ViewPaper.aspx?id=251690 [Google Scholar]
- 48.R A. Sleep quality and some associated factors in kerman students of nursing and midwifery. Health and Development Journal. 2015;4(2):146–150. http://jhad.kmu.ac.ir/article-1-97-en.html [Google Scholar]
- 49.Albqoor M.A., Shaheen A.M. Prevalence and differences in habitual sleep efficiency, sleep disturbances, and using sleep medication: a national study of university students in Jordan. Sleep Breath. 2020 doi: 10.1007/s11325-020-02174-2. [DOI] [PubMed] [Google Scholar]
- 50.Huang S. Sleep restriction impairs vocabulary learning when adolescents cram for exams: the need for sleep study. Sleep. 2016;39(9):1681–1690. doi: 10.5665/sleep.6092. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Stults-Kolehmainen M.A., Sinha R. The effects of stress on physical activity and exercise. Sports Med. 2014;44(1):81–121. doi: 10.1007/s40279-013-0090-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.de Vries J.D. Exercise as an intervention to reduce study-related fatigue among university students: a two-arm parallel randomized controlled trial. PloS One. 2016;11(3) doi: 10.1371/journal.pone.0152137. e0152137-e0152137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Wunsch K., Kasten N., Fuchs R. The effect of physical activity on sleep quality, well-being, and affect in academic stress periods. Nat. Sci. Sleep. 2017;9:117–126. doi: 10.2147/NSS.S132078. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Pizzuto M. vol. 4. MALTA MEDICAL SCHOOL GAZETTE; 2020. https://www.um.edu.mt/library/oar/handle/123456789/60171 (Incidence and Prevalence of Smoking Amongst Local Medical Students). [Google Scholar]
- 55.El-Rasheed A.H. Impact of anxiety on the prevalence of tobacco use among medical students, Ain Shams University, Egypt. Middle East Current Psychiatry. 2018;25(3):107–115. doi: 10.1097/01.XME.0000532208.92454.fc. [DOI] [Google Scholar]
- 56.Ali L.G.A., Sattar H.A.A., Al-Nimer M.S.M. Interrelationship among smoking habit, Drug user, and exposure to stressful situations in college students in baghdad, Iraq. Int. J. Biomed. Eng. 2019;5(2):5–13. http://materials.journalspub.info/index.php?journal=JBMBE&page=article&op=view&path%5B%5D=624 [Google Scholar]
- 57.Aleidi S.M. Factors affecting the academic performance of medical, dental, and pharmacy students in Jordan. Jordan Journal of Pharmaceutical Sciences. 2020;13(2):169–183. https://journals.ju.edu.jo/JJPS/article/view/103185 [Google Scholar]
- 58.Alkhawaldeh A. Assessment of self-medication use among university students. Int. J. Nurs. 2020;7:1–7. http://ijnnet.com/vol-7-no-1-june-2020-abstract-1-ijn [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.