Abstract
Background/Aims
Effective educational tools are important for increasing adherence to asthma guidelines and clinical improvement of asthma patients. We developed a computer-based interactive education program for asthma guideline named the Virtual Learning Center for Asthma Management (VLCAM). We evaluated the usefulness of program in terms of its effects on user awareness of asthma guideline and level of satisfaction.
Methods
Physicians-in-training at tertiary hospitals in Korea were enrolled in a cross-sectional questionnaire survey. The e-learning program on asthma guideline was conducted over a 2-week period. We investigated changes in the awareness of asthma guideline using 35-item self-administered questionnaire aiming at assessing physicians’ knowledge, attitude, and practice. Satisfaction with the program was scored on 4-point Likert scales.
Results
A total of 158 physicians-in-training at six tertiary hospitals completed the survey. Compared with baseline, the overall awareness obtained from the scores of knowledge, attitude, and practice was improved significantly. Participants were satisfied with the VLCAM program in the following aspects: helpfulness, convenience, motivation, effectiveness, physicians’ confidence, improvement of asthma management, and willingness to recommend. All items in user satisfaction questionnaires received high scores over 3 points. Moreover, the problem-based learning with a virtual patient received the highest user satisfaction among all parts of the program.
Conclusions
Our computer-based e-learning program is useful for improving awareness of asthma management. It could improve adherence to asthma guidelines and enhance the quality of asthma care.
Keywords: Asthma, Guideline, Education, E-learning, Virtual patient
INTRODUCTION
Asthma presents as symptoms associated with airway inflammation and bronchial hyper-responsiveness. The prevalence and socioeconomic burden of asthma have increased over the last three decades [1-3]. There have been considerable advances in asthma care, and numerous asthma-control guidelines have been developed internationally. However, several factors make asthma management unsatisfactory [4,5]. In particular, the complexity of asthma guidelines may interfere with their use in clinical practice. Thus, educational tools are necessary to bridge the gap between asthma guidelines and clinical practice. Various educational programs and interventions (e.g., card games, video simulations, and computer software) have been developed to improve adherence to national/global asthma guidelines [6-9]. We devised a new computer-based e-learning program for asthma guideline named the Virtual Learning Center for Asthma Management (VLCAM), which is based on established Korean Asthma Management Guideline [10]. It was designed to increase physicians’ adherence to asthma guidelines through interactive self-learning. This study assessed the usefulness of the program in terms of asthma guideline awareness and user satisfaction.
METHODS
E-learning education program
The VLCAM program provides information on evidence-based asthma management. It consists of four sections: asthma guideline e-book, self-assessment, problem-based learning with a virtual patient, and online reference room (Fig. 1A). The asthma guideline e-book section consists of six parts such as diagnosis, asthma medication, risk factors and prevention, assessment and management, exacerbation management, and special situations in asthma. The interface is presented as pages, which the user can flip through by clicking on the button of each page using the mouse pointer. The learner can select topics and study at their own pace. The self-assessment section consists of 20 questions; feedback is provided to the user after an answer is submitted. This section focuses on improving the participants’ asthma management skills. Problem-based learning with a virtual patient is a unique aspect of the VLCAM program, and it involves the use of virtual patient simulation. The virtual patient’s complaint is presented as text with voice narration. The patient’s photographs, radiographs, laboratory findings, and auscultation sounds are available to the user. Users perform a focused history and physical assessment to determine the diagnosis and formulate a management strategy. Users play the role of a healthcare provider and refine their clinical skills, including making diagnostic and therapeutic decisions, by reviewing virtual asthma patients (Fig. 1B). The reference room section provides asthma guideline documents, management cards, action plans, and patient handouts. These are available at http://www.copd-asthma.co.kr and http://www.kaaf.org.
Figure 1.
(A) Main introduction screen of the Virtual Learning Center for Asthma Management (VLCAM) program and (B) problem-based learning with a virtual patient.
Subjects
A questionnaire-based study was performed between December 2010 and January 2011. Questionnaires were distributed to 170 physicians-in-training working at six tertiary hospitals in South Korea. Participants completed an asthma guideline awareness questionnaire at the beginning of the study and then used the VLCAM program for 2 weeks. Subsequently, participants completed a questionnaire assessed asthma guideline awareness and user satisfaction of the program. A total of 158 subjects were included in the present study. This study was approved by the Seoul National University Hospital Institutional Review Board (No. 1511-095-721).
Format of the questionnaire
The questionnaire was divided into two parts: evaluation of asthma guideline awareness and user satisfaction. Basic personal characteristics were queried, including age, sex, specialty, years of medical practice, experience in asthma treatment, and familiarity with asthma guidelines. Asthma guideline awareness was evaluated from 35 questions regarding knowledge (20 questions), attitude (10 questions), and practice (5 questions). Knowledge, attitude, and practice survey is a representative study of a specific population to collect information on what is known, believed and done in relation to a particular topic. Participants were questioned about their understanding of asthma management and their level of agreement toward outcome expectancy with appropriate care to measure their attitude. Practice refers to the ways in which participants demonstrate their knowledge and attitude in clinical practice (Appendix 1). User satisfaction was assessed by means of questions on 11 items as the following contents: helpfulness, convenience, motivation, effectiveness, physicians’ confidence, improvement of asthma management, and willingness to recommend the VLCAM program.
Asthma guideline awareness
The total score of 100 were weighted as follows: knowledge = 40, attitude = 40, and practice = 20. In the knowledge section, responses received a score of 1 point for a correct answer and 0 points for others (incorrect and “don’t know” responses); the total score was multiplied by 2 to give a final score out of 40. The responses in the attitude and practice sections were scored with 4-point Likert scale to give total scores of 10 to 40 and 5 to 20, respectively. Higher scores of knowledge, attitude, and practice indicated higher levels of awareness of asthma management.
User satisfaction
We were trying to develop an instrument to measure satisfaction of program with using 4-point Likert scale considering racial/ethnic differences in response preferences and separation from the neutral options [11]. The questionnaires regarding helpfulness adopted the following scale: very helpful, 4 points; somewhat helpful, 3 points; somewhat not helpful, 2 points; and not at all helpful, 1 point. Physicians were asked if they agreed or disagreed with a list of statements regarding program: strongly agree, 4 points; somewhat agree, 3 points; somewhat disagree, 2 points; and strongly disagree, 1 point. Participants were invited to provide additional comments on the VLCAM program.
Statistical analysis
A statistical analysis was performed using the SPSS version 18.0 (SPSS Inc., Chicago, IL, USA). All descriptive statistics are presented as mean ± standard deviation (SD). A paired t test was used to compare mean scores before and after using the VLCAM program. A p < 0.05 was interpreted as significant.
RESULTS
Basic characteristics of subjects
A total of 158 physicians-in-training completed the study (response rate, 93%). The mean age of participants was 28.2 ± 3.1 years (range, 21 to 39) and females accounted for 50.6% of the total. Most of the participants (95.6%) were internal medicine physicians; others were family physicians and medical interns. Of these physicians, 82.7% had experience with asthma patient care, but only 64.2% were familiar with asthma guidelines. The baseline characteristics of the subjects are shown in Table 1.
Table 1.
Baseline characteristics of subjects
| Characteristic | Value |
|---|---|
| Subject | 158 |
| Sex, male/female | 78/80 (49.4/50.6) |
| Age, yr | 28.2 ± 3.1 |
| Primary specialty | |
| Internal medicine | 151 (95.6) |
| Othersa | 7 (4.4) |
| Duration of medical practice, yr | |
| < 5 | 155 (98.1) |
| 5–10 | 3 (1.9) |
| Experience of asthma patient care | |
| Yes | 129 (82.7)b |
| No | 27 (17.3)b |
| Missing responses | 2 |
| Familiarity with asthma guidelines | |
| Yes | 97 (64.2)b |
| No | 54 (35.8)b |
| Missing responses | 7 |
Values are presented as number (%) or mean ± SD.
Others: internship, family medicine.
Valid percent.
Awareness of asthma guideline
After the 2-week program, significant improvements were observed in the mean total score from 80.3 to 85.1 out of 100 (p < 0.001). The mean scores were also significantly increased at knowledge (29.2 vs. 30.9 out of 40), attitude (34.9 vs. 36.5 out of 40), and practice (16.2 vs. 17.5 out of 20), respectively (p < 0.001) (Fig. 2).
Figure 2.
Awareness of asthma guideline before and after the intervention. The results are shown as mean ± standard deviation (SD). aIndicate statistically significant differences (< 0.001) between scores before and after using the Virtual Learning Center for Asthma Management (VLCAM) program.
User satisfaction
The VLCAM program received high ratings for all user satisfaction items. Regarding helpfulness to the understanding of asthma guideline, all scores for asthma guideline e-book, self-assessment, and problem-based learning with a virtual patient were more than 3.40 (Fig. 3A). Furthermore, participants showed more favorable responses that the program helped them to understand asthma. The highest mean scores were found for the management of asthma exacerbations at 3.51, followed by asthma medication, assessment and management, special situations, diagnosis, and risk factor and prevention (Fig. 3B). As to questionnaires about the features of program, convenience (i.e., convenient to use for physicians), motivation (i.e., helpful to motivate the physicians to learn guideline), effectiveness (i.e., effective in learning guideline compared to traditional methods), physicians’ confidence (i.e., helpful to build physicians’ confidence about asthma patient care), improvement of asthma management (i.e., helpful to improve our understanding of asthma guideline), and willingness to recommend (i.e., worth recommending to other physicians) received high scores over 3 points (Fig. 4). The section considered to be most attractive and interesting was problem-based learning with a virtual patient (40% of respondents), followed by self-assessment (29%), asthma guideline e-book (28%), and online reference room (3%). Thirty-one participants commented on the need of program based on the content as well as the format. Several participants stated that the VLCAM program should be more concise, and include more images and videos of various clinical situations.
Figure 3.
(A) Participants’ perspectives regarding helpfulness of program components and (B) volume of the contribution to give learners to understand asthma guideline. Data are expressed as percentage of participants and mean ± standard deviation (SD). VLCAM, Virtual Learning Center for Asthma Management.
Figure 4.
User satisfaction regarding the features of the Virtual Learning Center for Asthma Management program (VLCAM). Data are expressed as percentage of participants and mean ± standard deviation (SD).
DISCUSSION
The prevalence of asthma has increased worldwide as communities adopt a Western lifestyle and become urbanized [2]. Moreover, asthma is ranked as the 25th leading cause of disability-adjusted life years lost; this ranking is similar to that of diabetes and one of the highest among all chronic diseases [12]. Due to the increased prevalence and socioeconomic burden of asthma, much effort has focused on improving its management. Indeed, several evidence-based asthma guidelines have been developed in several languages [13,14]. However, there remains a gap between the ideal and actual care of asthma patients. The current level of asthma control and use of inhaled corticosteroids (ICSs) in Korea fall markedly short of Global Initiative for Asthma guidelines for asthma management [4]. Korean asthma guidelines have not had a significant effect on the frequency of prescribed ICSs [15]. Poor asthma control may be due to several factors, including disease severity, presence of comorbidities, a lack of compliance and improper management due to a lack of knowledge about or familiarity with current guidelines [8,9]. Evidence-based practices centered on clinical guidelines achieve reductions in medical errors and mortality rates, healthcare costs, and hospital utilization [16-18]. The translation of asthma guidelines into practice is impeded by their complexity; therefore, novel tools are required to promote the translation of evidence-based guidelines into clinical practice [6,7,19].
Over the last two decades, due to rapid advances in information technology and widespread internet use, e-learning has become a widely used educational tool. E-learning is a computer-based educational tool or system that enables learning anywhere and at any time. Also, e-learning offers educational content based on the needs of learners, which is superior to traditional instructor-led education [20]. General medical practices that rely on computer-guided systems have been shown to produce more favorable clinical outcomes than those that rely on printed articles, manuals, or books [21,22]. Therefore, we developed an e-learning education program for the Korean asthma guideline. The VLCAM program is designed to facilitate the translation of knowledge into clinical practice through case-oriented learning using virtual patients and by providing comprehensive information on the Korean Asthma Management Guideline for Adults 2007 [10].
Our findings showed that the VLCAM program significantly improved awareness of asthma guideline in terms of knowledge, attitude, and practice. All items in user satisfaction questionnaires scored high. These findings are in agreement with those of a previous report on the Distributed Asthma Learning Initiative (DALI) program for pediatricians [7]. Both asthma learning programs had a high degree of learner satisfaction. Our study revealed significant changes in physicians’ knowledge, attitude, and practice of asthma management compared with the reported changes in knowledge and attitude for the DALI program [7]. This might be due to the participants’ characteristics and/or study design. Most of the participants were physicians-in-training, who were thus likely to be interested in learning tools. Therefore, the VLCAM program could encourage physicians to become familiar with current guidelines. However, it cannot be ruled out that participants might remember the correct answers or be conditioned by their awareness that they are being tested due to the short follow-up period. The high user satisfaction scores for understanding the treatment of acute asthmatic episodes and control of chronic symptoms might be attributed to the participants being trained in hospitals, which focus on in-patient rather than out-patient asthma care.
According to systematic reviews, multifaceted and interactive learning methods are more effective than any other intervention reducing gaps between knowledge and practice [21,23]. Interactive education shifts the focus from a passive and teacher-centered model to one that is active and learner-centered, which offers stronger learning motivation and enables the provision of feedback to learners. Interactive tools in various formats, including card games, simulations and computer-based programs such as e-learning, have proven to be effective for adherence to asthma guidelines [6,7,24]. Advances in information technology have facilitated the delivery of learning materials based on learners’ needs and the provision of interactive feedback, which is not feasible in traditional learning. Moreover, technological advancement allows physicians to share their diagnostic skills and treatment information by means of interactive virtual patients. Virtual patients have been defined as interactive computer simulations that reflect clinical scenarios in which the learner not only becomes the healthcare provider but also makes medical decisions. Virtual patients create a potentially richer learning environment by promoting self-directed learning and are employed by multiple disciplines in higher education [25-31]. Respondents stated that the virtual patient simulation was the most attractive and interesting section of the VLCAM program, as it facilitated adherence to guidelines through self-directed learning.
Many participants suggested that the VLCAM program should be more concise and easier to understand. It has been reported that physicians prefer more concise and immediately available guidelines with algorithms or flow charts/diagrams due to a lack of time [32]. Therefore, the full-text format of the asthma guideline e-book section could reduce participants’ motivation. The VLCAM program should be required to design to improve the integration with text, narration, and visual objects (images, graphic novels, and videos), and revised to a format more suitable for physicians. We were limited in our ability to generalize the results to the entire population of physicians for the following reasons. First, the questionnaires were not formally validated although the questionnaires were reviewed by experts in survey development. Second, all participants were trainees at tertiary hospitals; therefore, the results cannot be generalized to primary care physicians. Third, the small sample size and short study duration might have affected the results. In addition, we did not perform a comparative study with other educational interventions. It will be of value to perform further comparative study with other education programs to clarify whether the VLCAM program is effective educational methods, shown as increased adherence to asthma guidelines. Nonetheless, we present, to our knowledge, the first investigation of the usefulness of an asthma guideline-focused interactive educational program for physicians.
E-learning, defined as learning through web-based environment, is as effective as traditional instructor-led methods, and the use of a virtual patient provides comprehensive information on clinical aspects of the underlying disease, providing a reproducible and objective learning environment [33]. In conclusion, the VLCAM program is useful for improving awareness of asthma management. It could improve adherence to asthma guidelines and enhance the quality of care for asthma patients; thereby, reducing gap between knowledge and actual clinical practice. Further studies with larger populations and longer period are warranted to clarify the relationship between VLCAM program use and clinical outcomes in terms of adherence to asthma guidelines.
KEY MESSAGE
1. Effective educational tools are important for bridging the gap in care of patients with asthma.
2. We developed an e-learning education program (Virtual Learning Center for Asthma Management [VLCAM]) for the Korean asthma guideline to facilitate the translation of knowledge into clinical practice through self-directed learning.
3. The VLCAM program could improve adherence to asthma guidelines and contribute to high quality of asthma management.
Acknowledgments
This study was supported by a research grant from the Korea Healthcare technology R&D Project, Ministry of Health and Welfare, Republic of Korea (grant No. A102065). We wish to thank all of our participants at six tertiary hospitals in South Korea, including a Seoul National University Hospital, Seoul National University Bundang Hospital, Asan Medical Center, Chosun University Hospital, Dankook University Hospital, and Pusan National University Hospital.
Appendix 1.
Survey questions for the awareness of asthma guideline
Footnotes
No potential conflict of interest relevant to this article was reported.
REFERENCES
- 1.Braman SS. The global burden of asthma. Chest. 2006;130:4S–12S. doi: 10.1378/chest.130.1_suppl.4S. [DOI] [PubMed] [Google Scholar]
- 2.Song WJ, Kang MG, Chang YS, Cho SH. Epidemiology of adult asthma in Asia: toward a better understanding. Asia Pac Allergy. 2014;4:75–85. doi: 10.5415/apallergy.2014.4.2.75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bahadori K, Doyle-Waters MM, Marra C, et al. Economic burden of asthma: a systematic review. BMC Pulm Med. 2009;9:24. doi: 10.1186/1471-2466-9-24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cho SH, Park HW, Rosenberg DM. The current status of asthma in Korea. J Korean Med Sci. 2006;21:181–187. doi: 10.3346/jkms.2006.21.2.181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rabe KF, Adachi M, Lai CK, et al. Worldwide severity and control of asthma in children and adults: the global asthma insights and reality surveys. J Allergy Clin Immunol. 2004;114:40–47. doi: 10.1016/j.jaci.2004.04.042. [DOI] [PubMed] [Google Scholar]
- 6.Boulet LP, Borduas F, Bouchard J, Blais J, Hargreave FE, Rouleau M. Playing cards on asthma management: a new interactive method for knowledge transfer to primary care physicians. Can Respir J. 2007;14:480–484. doi: 10.1155/2007/504931. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Le TT, Rait MA, Jarlsberg LG, Eid NS, Cabana MD. A randomized controlled trial to evaluate the effectiveness of a distance asthma learning program for pediatricians. J Asthma. 2010;47:245–250. doi: 10.3109/02770900903560209. [DOI] [PubMed] [Google Scholar]
- 8.Moonie SA, Strunk RC, Crocker S, Curtis V, Schechtman K, Castro M. Community Asthma Program improves appropriate prescribing in moderate to severe asthma. J Asthma. 2005;42:281–289. doi: 10.1081/jas-200057900. [DOI] [PubMed] [Google Scholar]
- 9.Ting S. Multicolored simplified asthma guideline reminder (MSAGR) for better adherence to national/global asthma guidelines. Ann Allergy Asthma Immunol. 2002;88:326–330. doi: 10.1016/S1081-1206(10)62016-9. [DOI] [PubMed] [Google Scholar]
- 10.Korean Medical Guideline Information Center . Seoul (KR): Korean Medical Guideline Information Center; c2008. Korean Asthma Management Guideline for Adults 2007 [Internet] [cited 2017 Jul 3]. Available from: http://guideline.or.kr/guideline/search.php?mode=search&-type=2&Msid=28&Fsid=667. [Google Scholar]
- 11.Wang R, Hempton B, Dugan JP, Komives SR. Cultural differences: why do Asians avoid extreme responses? Surv Pract. 2008;1:1–7. [Google Scholar]
- 12.Masoli M, Fabian D, Holt S, Beasley R, Global Initiative for Asthma (GINA) Program The global burden of asthma: executive summary of the GINA Dissemination Committee report. Allergy. 2004;59:469–478. doi: 10.1111/j.1398-9995.2004.00526.x. [DOI] [PubMed] [Google Scholar]
- 13.Lalloo UG, Walters RD, Adachi M, et al. Asthma programmes in diverse regions of the world: challenges, successes and lessons learnt. Int J Tuberc Lung Dis. 2011;15:1574–1587. doi: 10.5588/ijtld.11.0289. [DOI] [PubMed] [Google Scholar]
- 14.Myers TR. Guidelines for asthma management: a review and comparison of 5 current guidelines. Respir Care. 2008;53:751–767. [PubMed] [Google Scholar]
- 15.Kim SH, Cho BL, Shin DW, et al. The effect of asthma clinical guideline for adults on inhaled corticosteroids prescriptiontrend: a quasi-experimental study. J Korean Med Sci. 2015;30:1048–1054. doi: 10.3346/jkms.2015.30.8.1048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Balkrishnan R, Norwood GJ, Anderson A. Outcomes and cost benefits associated with the introduction of inhaled corticosteroid therapy in a medicaid population of asthmatic patients. Clin Ther. 1998;20:567–580. doi: 10.1016/s0149-2918(98)80066-0. [DOI] [PubMed] [Google Scholar]
- 17.Donahue JG, Weiss ST, Livingston JM, Goetsch MA, Greineder DK, Platt R. Inhaled steroids and the risk of hospitalization for asthma. JAMA. 1997;277:887–891. [PubMed] [Google Scholar]
- 18.Suissa S, Ernst P, Benayoun S, Baltzan M, Cai B. Low-dose inhaled corticosteroids and the prevention of death from asthma. N Engl J Med. 2000;343:332–336. doi: 10.1056/NEJM200008033430504. [DOI] [PubMed] [Google Scholar]
- 19.Davis RS, Bukstein DA, Luskin AT, Kailin JA, Goodenow G. Changing physician prescribing patterns through problem-based learning: an interactive, teleconference casebased education program and review of problem-based learning. Ann Allergy Asthma Immunol. 2004;93:237–242. doi: 10.1016/S1081-1206(10)61494-9. [DOI] [PubMed] [Google Scholar]
- 20.Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med. 2006;81:207–212. doi: 10.1097/00001888-200603000-00002. [DOI] [PubMed] [Google Scholar]
- 21.Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ. 1998;317:465–468. doi: 10.1136/bmj.317.7156.465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc. 1999;6:104–114. doi: 10.1136/jamia.1999.0060104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995;274:700–705. doi: 10.1001/jama.274.9.700. [DOI] [PubMed] [Google Scholar]
- 24.Van Sickle D, Singh RB. A video-simulation study of the management of asthma exacerbations by physicians in India. Clin Respir J. 2008;2:98–105. doi: 10.1111/j.1752-699X.2007.00044.x. [DOI] [PubMed] [Google Scholar]
- 25.Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Med Educ. 2009;43:303–311. doi: 10.1111/j.1365-2923.2008.03286.x. [DOI] [PubMed] [Google Scholar]
- 26.Stevens A, Hernandez J, Johnsen K, et al. The use of virtual patients to teach medical students history taking and communication skills. Am J Surg. 2006;191:806–811. doi: 10.1016/j.amjsurg.2006.03.002. [DOI] [PubMed] [Google Scholar]
- 27.Satava RM. The revolution in medical education-the role of simulation. J Grad Med Educ. 2009;1:172–175. doi: 10.4300/JGME-D-09-00075.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Benedict N, Schonder K, McGee J. Promotion of self-directed learning using virtual patient cases. Am J Pharm Educ. 2013;77:151. doi: 10.5688/ajpe777151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Benedict N, Schonder K. Patient simulation software to augment an advanced pharmaceutics course. Am J Pharm Educ. 2011;75:21. doi: 10.5688/ajpe75221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Zary N, Johnson G, Boberg J, Fors UG. Development, implementation and pilot evaluation of a Web-based Virtual Patient Case Simulation environment: Web-SP. BMC Med Educ. 2006;6:10. doi: 10.1186/1472-6920-6-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Davids MR, Chikte UM, Halperin ML. Development and evaluation of a multimedia e-learning resource for electrolyte and acid-base disorders. Adv Physiol Educ. 2011;35:295–306. doi: 10.1152/advan.00127.2010. [DOI] [PubMed] [Google Scholar]
- 32.Kang MK, Kim BK, Kim TW, et al. Physicians’ preferences for asthma guidelines implementation. Allergy Asthma Immunol Res. 2010;2:247–253. doi: 10.4168/aair.2010.2.4.247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med. 2010;85:1589–1602. doi: 10.1097/ACM.0b013e3181edfe13. [DOI] [PubMed] [Google Scholar]