Abstract
Objectives
To assess the feasibility and impact of using a low-cost Android tablet to deliver clinical skills training to third-year medical students in Kenya.
Design
A prospective study using a low cost tablet called ‘connecTAB’, which was designed and manufactured specifically for areas with low bandwidth. Instructional video tutorials demonstrating techniques of cardiovascular and abdominal clinical examinations were pre-loaded onto the tablet.
Setting
Maseno University School of Medicine, Western Kenya.
Participants
Fifty-one third-year medical students from Maseno University School of Medicine were subjects in the study. Twenty-five students were assigned to the intervention group and 26 to the control group.
Main outcome measures
At the start of the study, students from both groups completed an Observed Structured Clinical Examination (OSCE) of the cardiovascular and abdominal evaluations. Students who were allocated to the intervention group then received the connecTAB, whereas students in the control group did not. After a period of three weeks, students from both groups completed a post-study OSCE for both the cardiovascular and abdominal evaluations.
Results
There were significantly higher improvements in the scores for both cardiovascular and abdominal examinations (p < 0.001) within the group who received the e-tablets as compared to the control group.
Conclusion
The study suggests that access to connecTAB improves clinical education and efficacy and holds promise for international training in both medical and allied healthcare professional spheres in resource-limited settings.
Keywords: mHealth, e-learning, clinical examination
Introduction
The World Health Organization (WHO) reports a global healthcare workforce shortage of 7.5 million doctors, pharmacists, nurses, community health workers and other basic healthcare professionals.1 Estimates indicate that this deficit will grow to 12.9 million in the coming decades. While sustainable, high-quality medical education is essential to fulfill the healthcare workforce shortage in low and middle-income countries (LMICs), medical education faces unique challenges in these settings. Government funding has become increasingly limited, forcing universities to require more students to pay larger fees to ensure their program’s financial solvency.2 The decrease in government funding has led to larger class sizes and rising student-to-faculty ratios, with less one-on-one teaching as a result.3 Medical education is further hindered by the migration of highly skilled healthcare professionals to more affluent countries – a phenomenon commonly referred to as the ‘brain drain’.2,4 Consequently, there are fewer experienced faculty available to teach students in LMICs.5 Together, large class sizes and inexperienced faculty have led to rushed and unprepared clinical teaching.3
Recent advances in educational technology offer a potential solution to the on-going challenges facing medical education in LMICs. In the last decade, a largely non-profit industry has spearheaded educational innovations. For example, the Khan Academy and EdX have, and continue to offer, online courses that are utilised by tens of millions of students around the globe.6,7 These video-based courses are high-quality, free to the user, and accessible anywhere via an internet-connected computer, tablet or smart phone. The efficacy of online learning media (known as E-Learning) when integrated into medical curricula has been validated. Studies have concluded that E-learning provides greater educational opportunities and resources for students while also improving faculty efficiency and efficacy.8
The exponential growth of E-learning has led public health and medical education experts to consider it as a potential tool to aid in closing the health-workforce gap in resource-poor countries.9 Mobile broadband subscriptions in LMICs have increased more than 26-fold between the years of 2005 and 2014, and access to computers and smart phones is becoming increasingly common.10 These developments make it possible to extend supplemental educational materials to under-equipped medical educators and their students via tablets and smartphones.11
In resource-poor settings, E-learning may prove to be particularly beneficial in teaching the clinical examination.12,13 Pre-loaded or streamed videos can teach clinical examination skills in a manner similar to the lessons taught by the Khan Academy, and they can be accessed at any time for additional review. Furthermore pre-loaded videos mean that students do not have to rely on broadband internet connection, which is often difficult to find in many LMICs. The effective teaching of examination skills is critical in these regions primarily because many laboratory tests and imaging modalities are unavailable, rendering the clinical examination a healthcare practitioner’s most important diagnostic.
The purpose of this study was to better understand the potential of the connecTAB, a low-cost Android tablet, as an educational aid for the learning of clinical examination skills amongst medical students in Western Kenya. Specifically, the aim of our study was to assess the efficacy of using low-cost Android tablets for video-based clinical-skills training and medical education in resource-limited settings, by comparing Observed Structured Clinical Examination (OSCE) scores between a cohort of students who had connecTABs and a cohort of students who did not.
Methods
Materials
The connecTAB is a low-cost, Android computer tablet that was developed for educational purposes and for this study. It is equipped with Jellybean 4.2 software, a forward and backward facing video camera, microphone, internal speakers, and 1 GB of internal memory with the option to expand it to 32 GB through an external memory card slot. Each connecTAB has the ability to tether to the local 2G and 3G cellular networks and, therefore, works in areas of low bandwidth. It also has the ability to work via a Bluetooth network, allowing local file sharing even in areas with slow internet speeds. For more details of the exact technical specifications of the connecTAB please refer to Figure 1 in the Appendix. Each connecTAB cost $50 USD to produce and for the purposes of this study came preloaded with instructional video tutorials of how to perform basic clinical examinations. This enabled students to access high-resolution instructional videos independent of the internet. In addition, a pre-loaded PDF document with relevant supplementary material was also included.
The videos were developed by Geeky Medics, an organisation specialising in open-access clinical video tutorials.14 The cardiovascular and abdominal video tutorials that were used in this study were each reviewed by two independent medical doctors for accuracy. Both doctors had extensive training in clinical examination assessment and hold Master’s Degrees in Clinical Education.
Ethical approval
The Partners HealthCare Institutional Review Board (Boston, MA, USA) reviewed the study and determined that it did not meet the definition of human subjects research, thus the project was exempted from further review.
Recruitment of medical students
Kenya was selected as the pilot country due to its location; East Africa has one of the lowest density of doctors and medical schools anywhere in the world.15,16 Third-year medical students from Maseno University School of Medicine volunteered to participate in the program in response to an email solicitation from the school’s administration sent to the entire student body. Students were informed that participation was voluntary and that they would be required to complete a pre- and post-study questionnaire, as well as a pre- and post-study clinical assessment. Students were also notified that they would be randomly allocated to either the intervention group or the control group. At the conclusion of the study, students in both the control and intervention group received a connecTAB.
Data collection using OSCE
At the beginning of the program, all students were asked to participate in pre-study OSCEs in which they performed cardiovascular and abdominal physical examinations on a fellow student. These particular examinations were chosen because they are widely regarded as the basic core examinations that should be mastered in the early stages of clinical training.
Each OSCE used a structured 25-point mark scheme, allowing an examiner to objectively score students on their clinical examination performance. A score of 16/25 (64.0%) was deemed a ‘Satisfactory Pass’, and a score of 20/25 (80.0%) or more was deemed ‘Excellent’ and worthy of academic distinction at the medical school. The full mark scheme for both cardiovascular and abdominal examinations can be found in Figure 2 in the Appendix. Students were given 7 min to complete each physical examination. The examinations were observed and assessed by an independent certified doctor. The mark scheme had been previously assessed for accuracy by two independent medically qualified doctors and was also approved and personally reviewed by the Dean of Maseno University School of Medicine.
Students were randomly assigned to either an intervention group (n = 25) or a control group (n = 26). Students in both groups received their usual clinical teaching for the duration of the study. Two students withdrew from the control group during the pre-assessment period and were, therefore, ineligible to continue in the remainder of the study. Each member of the intervention group (n = 25) received the connecTAB with the preloaded clinical examination videos and a PDF document with theoretical information relating to the examinations. The students in the control group (n = 24) did not receive connecTABs during the study but did receive them upon completion. All students involved in the study were reminded regularly that students in the control group should not have access to tablets or videos.
The students in the intervention group were instructed to use their connecTAB to access the instructional videos and to download any other relevant material or medical applications onto the tablet to assist them with their learning. After three weeks, students from both intervention and control groups were asked to return for post-study OSCE assessments in which they were reassessed on the cardiovascular and abdominal examinations.
An independent board certified doctor, who was blinded to whether the students were in the control or intervention group, assessed the OSCE scores for both the cardiovascular and abdominal evaluations.
Statistical analysis
Statistical analysis was conducted using SPSS Version 21 (SPSS Chicago).17 An independent t-test and a paired two-sample t-test were used to determine significance of pre- and post OSCE scores for the control and intervention groups. A p-value of <0.05 was considered statistically significant.
Reliability analysis
Inter-rater reliability was assessed by a second independent medically qualified professional who viewed the same 10 videos as the original assessor and assigned scores based on the standardised 25-point OSCE mark schemes. These scores were then compared with the scores assigned by the original assessor and an alpha coefficient was calculated.
Results
Pre-study OSCE scores
An independent sample t-test revealed no significant differences between the control and intervention groups’ pre-study OSCE scores in either the cardiovascular or abdominal assessment.
Post-study OSCE scores
All 100% (n = 25) of students in the intervention group and 71% (n = 17) of those in the control group returned for the post-study OSCE assessment. An independent t-test revealed statistically significant differences in post-study examination scores between intervention and control groups in both the cardiovascular (p < 0.0001) and abdominal (p < 0.0001) domains, with the intervention group scoring significantly higher than the control group in the post-study assessment (Tables 1 and 2).
Table 1.
Cardiovascular OSCE assessment (total points possible: 25).
| Cardiovascular OSCE | Intervention group: mean score (range, SD) | Control group: mean score (range, SD) | p Value |
|---|---|---|---|
| Pre-study assessment | 5.92 (11, 2.67) | 4.3 (11, 3.07) | p = 0.201 |
| Post-study assessment | 20.48 (11, 2.76) | 9.88 (14, 3.24) | p < 0.001 |
Table 2.
Abdominal OSCE assessment (total points possible: 25).
| Abdominal OSCE | Intervention group: mean score (range, SD) | Control group: mean score (range, SD) | p Value |
|---|---|---|---|
| Pre-study assessment | 5.96 (10, 2.89) | 6.21 (11, 2.95) | p = 0.107 |
| Post-study assessment | 20.63 (12, 3.09) | 9.11 (14, 2.87) | p < 0.001 |
A paired two-sample t-test was also conducted to determine whether the change in scores between pre- and post-study for each of the four ‘domains’ (intervention cardiovascular, control cardiovascular, intervention abdominal, control abdominal) were statistically significant. All four trials had statistically significant increases; additionally, the change in score was an average of 8–10 points higher for both intervention groups as compared to both control groups (Table 3).
Table 3.
p Values for change in score from pre- to post-test in all four domains.
| Domain | Change in score | p Value |
|---|---|---|
| Control cardiovascular | 5.17 | p < 0.001 |
| Control abdominal | 2.34 | p = 0.005 |
| Intervention cardiovascular | 14.66 | p < 0.001 |
| Intervention abdominal | 14.51 | p < 0.001 |
Reliability
The intra-class correlation coefficient (ICC) for inter-rater reliability was 0.960.
Discussion
This study suggests that the introduction of the low-cost Android tablet, connecTAB, with preloaded demonstrational videos, to a cohort of third-year medical students in Kenya was associated with marked improvements in clinical skill examination outcomes; medical students with the connecTAB performed significantly better on two different clinical skills examinations than medical students without the tablet (all p values <0.05).
In addition to supporting existing literature that highlights the potential of E-learning as a useful pedagogic adjunct for clinical training,12 our findings also suggest that having the videos pre-loaded on an inexpensive mobile device can allow for quality clinical learning in areas where financial constraints and poor infrastructure limit dependable internet access and access to technological devices. More than half of the participants did not own an internet-enabled device of any kind prior to the study, and while the convergence of Internet and cell phones has become pervasive in countries like the United States and United Kingdom, the majority of mobile phones sold in Africa are not internet-enabled smartphones. In Kenya, where 82% of the population owns a cell-phone, only 19% of those are smartphones.18 Although these trends are predicted to change over time, smartphones remain a luxury available primarily to high-GDP nations.18 Our innovation of a pre-loaded, low-cost tablet that works in areas of low-bandwidth proved to be beneficial in Maseno, Kenya, and would likely be advantageous for LMICs in general, because of the low-cost of the device.
From a design standpoint, the study revealed a number of interesting concepts. Pre-loading a video onto a tablet or smart-phone device allows viewers to fast forward without having to wait for videos to buffer or load – something that regularly occurs in areas with poor internet. Secondly, pre-loading ensures that all instructional videos demonstrating clinical examination are accurate and standardised. Furthermore, the ability to watch a video anytime, rewind, and view the source multiple times may also give pre-loaded videos a distinct advantage in teaching clinical skills.
In terms of study limitations, the OSCE is by no means a perfect tool for assessing clinical ability. It has been recognised that the OSCE tests the student’s knowledge and skills in compartments, rather than their ability to look at the patient as a whole.19 Despite this disadvantage, the OSCE is still regarded as a valid and reliable method to objectively grade and score clinical skills of junior medical students, and has long been recognised as the gold standard for the assessment of students’ clinical competence.20,21 The nature of an OSCE means that the variables of the examiner and the patient are to a large extent removed.19 The use of a checklist marking criteria results in a more objective examination and means that crucial skills must be demonstrated in order for the student to fulfill the criteria to obtain a pass mark.
A further limitation is that it is difficult to determine the degree to which changes in examination scores can be attributed to the integration of connecTAB in exam preparation as opposed to other outside resources. Anecdotally, we know that there were students who accessed additional materials on the tablets, including e-textbooks, and PowerPoint presentations. The acquisition of these external materials could have led to confounding if the external resources played a greater role in improving the scores than the videos themselves. However, unless the students were all utilising the same outside resources, it seems unlikely that one would observe such a dramatic increase in mean scores. Looking at the browsing history of the tablet could provide a unique opportunity to understand what information the students perceived that they needed that the videos lacked. To utilise these data, we are in the process of developing an analytics program (similar to YouTube analytics) that will allow us to assess factors such as the number of times a video was accessed and the duration of viewing by an individual student. If any of these variables correlate with performance, this knowledge can then be used to inform content development.
Another limitation to the study is the possibility that students in the control group might have accessed the tablets and the videos from the students in the intervention group, despite explicit instructions against doing so. Additionally, because two students dropped out of the study before it began and seven students in the control group failed to return for the post-assessment, there is the potential for skewed data as a result of uneven group sizes. Finally, the duration of the study was only one month and, thus, it is impossible to comment on the long-term retention of the skills acquired.
Integrating smart phones or tablets with pre-loaded instructional video materials into existing teaching methods, especially in resource-limited settings, may have positive implications for learning outcomes, which could be magnified if key personnel, such as course leaders and facilitators, support the program. This buy-in from local stakeholders would both increase ownership of the technology and the results that it yields, which would allow students to receive feedback from in-country or institution senior members of staff.
In terms of future research, the present study was limited to cardiovascular and abdominal examinations. It is possible that these examinations are particularly responsive to video demonstrations. It may be unwise, therefore, to generalise these findings to all clinical skills examinations. We suggest expanding to other, more varied clinical examinations in future studies to test this claim. Furthermore, the potential of the connecTAB program, utilising affordable technology equipped with open-access videos should be explored further in different contexts like the post-graduate training of doctors and with populations such as nurses and community health workers. If connecTAB proves effective in these other environments, then it may well be worth scaling up the project to other LMICs.
Conclusions
The results of this small, experimental study suggest that the use of low-cost tablets or smart phone devices with preloaded instructional videos could offer an innovative and cost-effective pedagogical approach to enhance medical students’ clinical skill competencies and self-confidence levels. To be sure, demonstration videos are not a panacea and need to supplement traditional teaching methods rather than replace them.22,23
Acknowledgements
We would like to thank the faculty at Maseno University School of Medicine for facilitating the study, especially Vice Chancellor Dominic Makawiti and Dean Wilson Odero. We also wish to thank Miss Rosemary Hines, Mr Greg Vigil, Dr Nathalie Laidler-Kylander and Mr Keeran Sivarajah for their guidance and support in helping to design the outcomes measurement tools used in the program. We wish to thank Dr Kiran Sodha for his assistance in data acquisition and Miss Amalia Bersin for drafting the discussion section of the manuscript. Finally, we wish to thank Miss. Kristina Tester and Miss Stephanie Sobek for proofreading and editing the article.
Appendix 1. connecTAB specifications
| Display | 7 inch touchscreen 1024 × 600 resolution at 171 ppi, SD video playback |
|---|---|
| Size | 191 × 115 × 10.6 mm (7.5" × 4.5" × 0.4”) |
| Storage | 8GB (with a microSD card slot for up to 128GB of storage) |
| Battery life | Up to 10 hours |
| Connectivity | Single-band Wi-Fi. Supports public and private Wi-Fi networks or hotspots that use the 802.11 b, 802.11 g or 802.11 n standard with support for WEP, WPA and WPA2 security using password authentication. Also supports Bluetooth connectivity for peer-to-peer networks. |
| Ports | USB 2.0 (micro-B connector) to connect to a PC/Macintosh computer; microSD slot for external storage |
| Software | Android Jelly Bean 4.2.2 |
Declarations
Competing Interests
JO’D is a director of the non-profit organisation that supplies the connecTAB, the tablet used in the study; TB, the senior study author, is a director of the Ujenzi Charitable Trust. The remaining authors have no conflicts of interest to declare.
Funding
The study was funded through an unrestricted grant from the Ujenzi Charitable Trust.
Ethical approval
The Partners HealthCare Institutional Review Board (Boston, MA, USA) reviewed the study and determined that it did not meet the definition of human subjects research, thus the project was exempted from further review.
Guarantor
TB
Contributorship
JO’D conceived the idea for the study, designed the study, acquired and analysed the data, was a lead in writing the manuscript and is accountable for the work; RA assisted in the study design, interpreted the data, critically appraised the manuscript, approved the final version for publication and agrees to be accountable for the work; BDN assisted in interpretation of the data, revising the manuscript, approved the final version for publication and agrees to be accountable for the work; CK assisted in study design, co-authored the manuscript, approved the final version for publication and agrees to be accountable for the work; TFB assisted in study design, critically appraised and revised the manuscript, approved the final version for publication and agrees to be accountable for the work.
Provenance
Not commissioned; peer-reviewed by William Coppola
References
- 1.World Health Organization. A universal truth: no health without a workforce, http://www.who.int/workforcealliance/knowledge/resources/GHWA-a_universal_truth_report.pdf?ua=1 (2013, accessed 07 August 2015).
- 2.Ndetei DM, Mathai M, Khasakhala LI, Mutiso V, Mbwayo AW. University medical education in Kenya: the challenges. Med Teach 2010; 32: 812–815. [DOI] [PubMed] [Google Scholar]
- 3.Aggarwal S, Sharma V. The problems of medical education in a developing economy: the case of India. Ann Trop Med Public Health 2012; 5: 627–629. [Google Scholar]
- 4.Willis-Shattuck M, Bidwell P, Thomas S, Wyness L, Blaauw D, Ditlopo P. Motivation and retention of health workers in developing countries: a systematic review. BMC Health Serv Res 2008; 8: 247–247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Troy PH, Wyness LA, McAuliffe E. Nurses’ experiences of recruitment and migration from developing countries: a phenomenological approach. Hum Resour Health 2007; 5: 15–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.EdX. Free Online Courses from the Worlds Best Universities, https://www.edx.org/ (2015, accessed 2 November 2015).
- 7.Khan Academy. Online Courses, https://www.khanacademy.org/ (2014, accessed 2 November 2015).
- 8.Frehywot S, Vovides Y, Talib Z, et al. E-learning in medical education in resource constrained low- and middle-income countries. Hum Resour Health 2013; 11: 4–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sahu M, Grover A, Joshi A. Role of mobile phone technology in health education in Asian and African countries: a systematic review. Int J Electron Healthc 2014; 7: 269–286. [DOI] [PubMed] [Google Scholar]
- 10.International Telecommunication Union. Key global telecom indicators for the world telecommunication service sector, http://www.itu.int/en/ITU-D/Statistics/Pages/default.aspx (2013, accessed 5 May 2014).
- 11.Istepanian R, Philip N, Wang XH, Laxminarayan S. Non-telephone healthcare: the role of 4G and emerging mobile systems for future m-health systems. Stud Health Technol Inform 2004; 103: 465–470. [PubMed] [Google Scholar]
- 12.O’Donovan J, Bersin A, O’Donovan C. The effectiveness of mobile health (mHealth) technologies to train healthcare professionals in developing countries: a review of the literature. BMJ Innovations 2015; 1: 33–36. [Google Scholar]
- 13.O’Donovan J, Maruthappu M. Distant peer-tutoring of clinical skills, using tablets with instructional videos and Skype: A pilot study in the UK and Malaysia. Med Teach 2014; 37: 1–7. [DOI] [PubMed] [Google Scholar]
- 14.Geeky Medics. Clinical Examination Skills, http://www.youtube.com/geekymedics123 (2014, accessed 5 May 2014).
- 15.Duvivier RJ, Boulet JR, Opalek A, van Zanten M, Norcini J. Overview of the world’s medical schools: an update. Med Educ 2014; 48: 860–869. [DOI] [PubMed] [Google Scholar]
- 16.Kinfu Y, Dal Poz MR, Mercer H, Evans DB. The health worker shortage in Africa: are enough physicians and nurses being trained? Bull World Health Organ 2009; 87: 225–230. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.IBM Corp. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp, Released 2012.
- 18.PewResearch Global Attitudes Project. Emerging nations embrace internet, mobile technology, http://www.pewglobal.org/2014/02/13/emerging-nations-embrace-internet-mobile-technology/ (2014, accessed 11 May 2015).
- 19.Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ 1979; 13: 41–54. [PubMed] [Google Scholar]
- 20.Harden RM. Misconceptions and the OSCE. Med Teach 2015; 37: 1–3. [DOI] [PubMed] [Google Scholar]
- 21.Sloan DA, Donnelly MB, Schwartz RW, Strodel WE. The Objective Structured Clinical Examination. The new gold standard for evaluating postgraduate clinical performance. Ann Surg 1995; 222: 735–742. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Jang HW, Kim KJ. Use of online clinical videos for clinical skills training for medical students: benefits and challenges. BMC Med Educ 2014; 14: 56–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Schreiber BE, Fukuta J, Gordon F. Live lecture versus video podcast in undergraduate medical education: a randomised controlled trial. BMC Med Educ 2010; 10: 68–68. [DOI] [PMC free article] [PubMed] [Google Scholar]