Abstract
Introduction
Latin America is a large and diverse region, comprising more than 600 million inhabitants and one million physicians in over 20 countries. Resistance to antibacterial drugs is particularly important in the region. This paper describes the design, implementation and results of an international bi-lingual (Spanish and Portuguese) online continuing interprofessional interactive educational program on hospital-acquired infections and antimicrobial resistance for Latin America, supported by the American Society for Microbiology.
Methods
Participation, satisfaction and knowledge gain (through pre and post tests) were used. Moreover, commitment to change statements were requested from participants at the end of the course and three months later.
Results
There were 1169 participants from 19 Latin American countries who registered: 57% were physicians and 43% were other health care professionals. Of those, 1126 participated in the course, 46% received a certificate of completion and 54% a certificate of participation. There was a significant increase in knowledge between before and after the course. Of 535 participants who took both tests, the grade increased from 59 to 81%. Commitments to change were aligned with course objectives.
Discussion
Implementation of this educational program showed the feasibility of a continent-wide interprofessional massive course on hospital acquired-infections in Latin America, in the two main languages spoken in the region. Next steps included a new edition of this course and a “New Challenges” course on hospital-acquired infections, which were successfully implemented in the second semester of 2015 by the same institutions.
Keywords: Internet, Continuing education, Interprofessional education, MOOC, Hospital-acquired infections, Latin America
Introduction
Latin America is a large and diverse region, where the predominant languages are Spanish and Portuguese, comprising more than 600 million inhabitants and one million physicians in over 20 countries, from Mexico in the North to Argentina and Chile in the South.
Resistance to antibacterial drugs is particularly important in Latin America. In this regard, a serious public health problem has been identified in the region on account of the emergence of resistance mechanisms and dissemination of these clones – particularly extended spectrum beta-lactamases (ESBLs) of the CTX-M type, and carbapenemases (KPC).1, 2, 3, 4
Among non-fermenting Gram-negative bacilli infections, Acinetobacter baumannii has increased in most of South America and the Caribbean. As a result, in the last decade, resistance of A. baumannii to antibiotics has increased substantially probably due to the relative impermeability of the outer membrane and environmental exposure to a broad group of resistance genes.5, 6, 7
Moreover, in the last few years Clostridium difficile emerged worldwide as the main causative agent of nosocomial diarrhea. The Centers for Disease Control and Prevention (CDC, US Government) report an annual incidence of 453,000 cases in 2011, with 30,000 deaths per year related to “Clostridium difficile infection” (CDI). In Latin America there are few published studies, mainly of outbreaks reported in Brazil, Argentina, and Chile, but the real impact of CDI in this region as an agent of endemic infection is unknown. Only CDI cases caused by hypervirulent strains (ribotype 027) have been reported in Costa Rica and Chile but most likely this is underdiagnosed.8, 9, 10, 11
Gram-positive cocci infections such as methicillin-resistant Staphylococcus aureus (MRSA) and vancomycin-resistant Enterococcus constitute a long-standing problem, but its real impact on hospital-acquired infections (HAI) has not been properly evaluated in our region.12
This complex situation is a challenge health care professionals in the region face everyday as to which is the best antimicrobial strategy – not only to cure their patients, but also to care for the ecosystem. Moreover, contact isolation policies for multi-resistant microorganisms, proper hospital hygiene, and compliance with hand hygiene procedures are highly variable and impact on patient safety.
A challenge of this online course was to address the problem described above, and its main objective was to provide comprehensive continuing interprofessional education about HAI caused by the most frequent microorganisms in Latin America. The target audience was the health personnel involved in this process: professional nurses certified in infection control, members of Committees of Hospital-Acquired Infections, members of Committees for Bacterial Control, medical doctors specialized in infectious diseases, intensive care and internal medicine, microbiologists, bacteriologists, and other health professionals related to this topic in Latin America.
The use of information and communication technologies (ICT) is evolving constantly, with innovations and the potential for more interactive and engaging strategies to be incorporated into educational programming, still allowing for massive participation of physicians, some of them from more remote locations of the countries. Therefore, in 2014, EviMed was invited by the Society for Worldwide Medical Exchange, a non-profit organization based in Miami, to implement together with the American Society for Microbiology and the Department of Infectious Diseases, School of Medicine, Universidad de la República, Uruguay, the integration of ICT into their more traditional formats, based on its record of blended multifaceted programs in Latin America.13, 14, 15, 16
The Faculty coordination, development of multimedia study materials, and tutoring roles were in charge of professors from four universities: Universidad de la República, Uruguay; Universidade Federal de São Paulo – Brazil (UNIFESP); University Clinical Hospital, School of Medicine, University of São Paulo – Brazil (HCFM-USP); School of Medical Sciences, Federal University of Rio de Janeiro – Brazil.
A collaborative work was done together with the American Society for Microbiology (ASM), Centers for Disease Control and Prevention (CDC), Pan American Health Organization (PAHO/WHO) and the Pan American Association of Infectious Diseases (API), which provided information for the needs assessment, international and regional experts to develop teaching materials, speakers for the launching event, and dissemination across the region.
This paper describes the design, implementation and results of a multi-country bi-lingual (Spanish and Portuguese) continuing interprofessional education program on HAI and antimicrobial resistance for Latin America.
Methods
Program content and structure
The course was implemented in the months of July to September 2014. The topic was HAI and antimicrobial resistance. A needs assessment with experts from the region and the target audience was performed through telephone interviews, in order to prioritize the topics to be addressed. The program design was then developed in conjunction between the Uruguayan and Brazilian central Faculty and the ASM liaison expert from Brazil (co-author LT), for which a two-day live workshop was organized.
The content included:
-
•
Current threats of the main antimicrobial resistances: local and global epidemiological aspects.
-
•
Resistance mechanisms.
-
•
Diagnostic approaches in the laboratory: detection methods and interpretation of results.
-
•
Rational use of antimicrobial agents.
-
•
Prevention and surveillance measures.
-
•
Isolation and hygiene measures.
-
•
Outbreaks control.
-
•
Clinical management and new treatments.
The topics above were covered while addressing the following microorganisms: C. difficile, infections caused by extended spectrum beta-lactamase producing Gram-negative bacilli (ESBL-GNB), Acinetobacter, carbapenemase producing enterobacteria, vancomycin resistant Enterococcus (VRE), and MRSA.
Target audience
The target audience was Spanish- and Portuguese-speaking professionals dealing with these conditions across the Latin American region. It could be subdivided into three main groups:
-
(1)
Physicians taking care of individual patients with these infections (e.g., infectious disease specialists, internists, intensive care physicians, nephrologists, and others).
-
(2)
Infection control professionals, both nurses and physicians, working in infection control committees or in hospital wards at health care institutions, dealing with the prevention and management of outbreaks and antimicrobial resistance at the institutional level.
-
(3)
Microbiologists, who deal with this problem from the laboratory perspective, in coordination with the above two groups. In Latin America, microbiologists can be either specialized physicians or biochemists, depending on the country.
There was a challenge involved in having such a diverse audience, in order to produce learning paths meaningful to each of them and to the teams as a whole, and at the same time benefit in the case discussions from the fact that both Faculty and learners had these different backgrounds, resembling real-life settings. The course was then structured in modules, some of them were more focused on solving individual clinical cases, and some of the others were more focused on solving outbreaks of the infections produced by these lead cases, therefore allowing for the different professions to participate and interact with each other, according to their training and experience, thus enriching the discussions.
Health care professionals were invited to register through the different academic networks and with the help of professional congress organizers in some of the countries; promotion was also done to potential sponsors. The Society for Worldwide Medical Exchange coordinated the convening for the course with the different stakeholders and the registration process. There was a registration fee: USD 100 for physicians and USD 75 for other health professionals. Full scholarships were also given by the organizing institutions to national scientific societies and upon request and by commercial sponsors to groups of professionals invited by them.
Implementation of program
EviMed provided a multidisciplinary team of communication and educational experts, clinicians, system engineers, medical information specialists and translators to design and implement the course, together with regional and international domain experts provided by the participating academic institutions. There were 25 expert lecturers, including 21 from Latin America, as well as 2 European and 2 North American experts. Including the tutoring roles, 28 domain experts participated in the design and implementation of this course.
A multifaceted approach was used, in order to maximize opportunities for health care professional participation and ensure that participants could interact and reflect on the material and with each other.
Accordingly, the activities began with on-site and on-line synchronous launching event. The launching event was in Montevideo, Uruguay, on July 25, 2014; experts from CDC and ASM coming from the US, Europe, and Brazil participated onsite. It had simultaneous translation between Spanish and Portuguese at the main room and with two streaming channels for the remote participants, according to their language preference. It was also designed for both remote and onsite participation for large audiences, with voting and questions from the audience.
This was followed by a distance education six-week on-line modality with multiple educational strategies. Reading resources, videos and voice-over-presentations were published, together with pre and post tests, patient aids, and electronic rounds (e-rounds) on clinical cases. Additionally, two clinical simulations using a custom tool developed with the School of Engineering, Universidad de la República, Uruguay17 were used, in order to provide applied learning through knowledge discovery and automatic feedback. They were implemented for two of the six modules; there were two versions of each, designed for either physicians or nurses, and had multiple possible pathways, some better than others, not only right or wrong paths, as in real settings. See Fig. 1.
Fig. 1.
Graph of one of the simulations. Please note different paths can eventually have similar results.
The e-rounds were asynchronous discussion groups in either Spanish or Portuguese, coordinated by tutors who were native speakers of each of the two languages, and were specifically trained for that role; moreover, there was a weekly remote meeting among all tutors and course coordinators to ensure that the feedback provided was consistent among the groups. There were four groups (one in Portuguese and three in Spanish) for the discussions of clinical cases and outbreaks. For the clinical simulations, there were two groups (one in Spanish and another in Portuguese), where people could comment their perspective on the simulated cases. Finally, the introductory and final exchange spaces were shared by all participants, no matter the language; this was particularly important for Brazil, to see the international dimension of the course.
This modality allowed course attendees to participate whenever they could and wished to, lacking a fixed schedule to access the e-round.
A total of 26 h of study workload along two months was estimated, in order to complete the course.
Most of the study materials were developed by the central Faculty, except for the short video lectures by international experts, which were taped remotely from their work or home, then underwent post production and subtitling.
Data collection and analysis
Regarding course evaluation, participation, satisfaction and knowledge gain pre and post tests were used. Moreover, commitment to change statements was requested from participants at the end of the course.
The requirements for the certificate of completion were:
-
-
Access to at least 70% of the study materials.
-
-
Grading of at least 70% of tests in each module.
-
-
Completion of the clinical simulations.
-
-
Participation in at least 50% of discussion forums.
A t test for paired samples was used to compare pre and post test results. Commitment to change statements was categorized by topic.
A sample site with the layout of this course in English can be seen at: http://courses.evimed.net/.
Results
Registration
There were 1169 participants, most of them coming from 19 Latin American Countries, but also from the USA, Spain and Portugal (see Table 1), who registered for the course, of which 824 registered individually, 247 had a sponsored scholarship, and 98 were given a scholarship by the organizing institutions. Of this total, 265 were Portuguese-speaking and 904 were Spanish-speaking.
Table 1.
Individual number of registrations by country.
| Individual registrations by country | Number of respondents |
|---|---|
| Brazil | 264 |
| Argentina | 238 |
| Uruguay | 229 |
| Mexico | 207 |
| Chile | 62 |
| Panama | 24 |
| Paraguay | 24 |
| Colombia | 23 |
| Venezuela | 22 |
| Ecuador | 21 |
| Peru | 15 |
| Costa Rica | 11 |
| Dominican Republic | 7 |
| Salvador | 5 |
| Bolivia | 3 |
| Spain and Portugal | 3 |
| Honduras | 3 |
| USA | 3 |
| Nicaragua | 2 |
| Cuba | 1 |
| Guatemala | 1 |
| Puerto Rico | 1 |
Regarding the characteristics of the audience, 57% were physicians and 43% were other health care professionals.
Participation
Ninety-six percent of registrants enrolled in the course (n = 1128) accessed the online campus, and 99.8% of them (n = 1126) participated in the course. Of this last figure, 46% received a certificate of completion (n = 522) and 54% a certificate of participation (n = 604). The hybrid launching event had a live participation of 225 onsite and 110 online. The lecture recordings from the event were later uploaded in the virtual campus.
The total amount of posts in discussions was 7945, with an average of posts by participant of 6.8. Regarding the interactive case simulations, the numbers were the following: A – Non-Physicians: 409. A – Physicians: 504. B – Non-Physicians: 608. B – Physicians: 533.
Satisfaction
The number of respondents that completed the satisfaction survey was 393. Seventy-three of them (n = 286) defined the course as “very good”, 24% (n = 94) as “good”, 3% as acceptable (n = 13), with no mentions as bad or very bad.
Regarding the relative importance of learning activities, see Table 2. The best and worst aspects of the course are listed in Table 3, Table 4, and the aspects that limited learning in Table 5.
Table 2.
Order of importance for learning activities.
| Order of importance in campus components as a learning contribution (1=contributed less – 6=contributed more) | ||||||
|---|---|---|---|---|---|---|
| Item | Number of people who marked each position |
|||||
| 1 | 2 | 3 | 4 | 5 | 6 | |
| Videos by experts | 19 | 23 | 44 | 52 | 79 | 108 |
| Reading materials | 14 | 15 | 25 | 54 | 115 | 108 |
| Clinical simulations | 6 | 17 | 58 | 108 | 58 | 78 |
| Discussion forums of cases | 23 | 96 | 101 | 65 | 30 | 10 |
| Knowledge tests | 51 | 105 | 82 | 38 | 40 | 9 |
| General exchange forum | 211 | 69 | 15 | 8 | 3 | 19 |
Table 3.
Course evaluation by participants. Best aspects of the course.
| Item | Number of respondents | % |
|---|---|---|
| Topic and approach | 251 | 64 |
| Clinical simulations | 248 | 63 |
| Didactic materials | 204 | 52 |
| Teaching methodology | 173 | 44 |
| Participation of several countries | 120 | 31 |
| Interaction with professors | 104 | 26 |
| Interaction with colleagues | 67 | 17 |
| Learning assessments | 62 | 16 |
| Campus usability | 53 | 13 |
| Users support service | 17 | 4 |
| Nothing in particular | 9 | 2 |
| Other | 6 | 2 |
Table 4.
Course evaluation by participants. Worst aspects of the course.
| Item | Number of respondents | % |
|---|---|---|
| Nothing in particular | 291 | 74 |
| Interaction with colleagues | 17 | 4 |
| Interaction with professors | 14 | 4 |
| Topic and approach | 11 | 3 |
| Learning assessments | 18 | 5 |
| Users support service | 9 | 2 |
| Didactic materials | 9 | 2 |
| Campus usability | 8 | 2 |
| Teaching methodology | 5 | 1 |
| Participation of several countries | 5 | 1 |
| Clinical simulation | 4 | 1 |
| Other | 41 | 10 |
Table 5.
Course evaluation by participants. Aspects which limited learning.
| Item | Number of respondents | % |
|---|---|---|
| Time shortage | 225 | 57 |
| None | 111 | 28 |
| Internet connection | 33 | 8 |
| Interacting requirement | 39 | 10 |
| Distance learning modality | 14 | 4 |
| Challenges with software | 8 | 2 |
| Topic lacked interest | 0 | 0 |
| Other | 37 | 9 |
A special request was made by nurses, to have specific tests for them in next courses, since most of the questions about antibiotic selection were not pertinent for their professional profile.
Knowledge acquisition
There was a significant increase in knowledge between before and after the course. Of 535 participants who took both tests, the grade increased from 59 to 81% (p < 0.001).
Commitment to change
The number of participants who completed the commitment to change was 298, who produced 795 commitments. The topics addressed in these commitments are shown in Table 6, and the follow-up three months later is shown in Table 7.
Table 6.
Commitment to change.
| Indicator | Number of commitments |
|---|---|
| Education of colleagues, patients and self | 204 |
| Improve self professional practice | 131 |
| Improve antimicrobial use | 118 |
| Hygiene measures | 94 |
| Epidemiological surveillance | 76 |
| Improve diagnosis | 54 |
| Creation of infection control committees | 31 |
| Scientific dissemination | 24 |
| Biosecurity | 16 |
Table 7.
Follow-up of commitments.
| Indicator | Number of respondents | % | |
|---|---|---|---|
| Participants who responded three months later | 84 | 28 | Of the total of participants who completed the commitment to change |
| Commitments fulfilled | 101 | 13 | Of the total of commitments |
| Commitments not fulfilled | 62 | 8 | Of the total of commitments |
| Commitments partially fulfilled | 100 | 13 | Of the total of commitments |
Discussion
The results of participation and completion of the course were similar to other courses with this design, and better than most MOOCs.16, 18 In most Latin American countries, professional recertification is not enforced, so participants mostly registered in this course for learning, rather than earning credits. The pretest was not required to obtain the certificate, while the post test was: this limited the number of participants who took both tests. Still, the percentage of participants who took both tests in our experience was similar to other courses in different disciplines.16
It could be debated whether the number of participants in each of the e-round groups should have been lower, such in certificate courses, or larger such as in MOOCs; empirically, we have found that this group size works, as long as peers are stimulated to engage actively in the discussions with each other, and the role of Faculty and tutors is guiding them, rather than interacting on a one-to-one basis.
Regarding the reach as a percentage of the target audience, it is difficult to estimate because of the different professional profiles involved, but in any case, having over 1100 professionals taking a course from most countries in the region implies a massive educational intervention.
Regarding its impact on knowledge, there is a well-defined increase, probably related to some design elements that produced positive results (i.e., a sequential, multifaceted, interactive course).19, 20, 21 Moreover, there are indirect indicators of possible impact on clinical practice, such as the commitment to change statements, aligned with the course objectives.22
Interprofessional education occurs when two or more health and social care professions are interactively learning “about, from and with each other”.23 It focuses on changing practice of health care teams in order to improve health care outcomes. Computer-supported collaborative learning has shown to be more engaging than individual online learning, as a result of immediate feedback from their co-learners and social pressure from them, and has been proposed when deeper learning is sought, particularly in complex and interprofessional educational experiences.24 It may also mimic real-life interactions within the health care team. On the other hand, it needs to be well designed and guided, to avoid negative outcomes.24 In this case, in most instances the learning experience was positive and engaging throughout the course for all professional profiles, and the exchange across professions in the case discussions was fruitful.
Regarding the limitations of this study, it should be noted that it had no control group, and that it lacks information about results on participants who did not complete all the evaluations (satisfaction, knowledge acquisition, among other). However, it provides insight about the cohort who did complete all stages, and it also has objective data on the total cohort (such as participation).
Regarding next steps, a new edition of this course and a “new challenges” course on HAI were implemented during the second semester of 2015 with support from the American Society for Microbiology and the same Universities in Latin America. Requests for improvement from participants in this first edition were taken into account for these courses. Particularly, two different tests were produced for this new edition: for physicians and for non-physicians, to focus evaluation on topics more related to their knowledge and practice.
Conclusions
The implementation of this educational program showed the feasibility of an international and interprofessional massive course on hospital acquired-infections in Latin American, covering the two main languages spoken across the entire region, Spanish and Portuguese.
The inherently different cost structure of online versus onsite educational interventions,25 and the potential reach across and within Latin American countries by using online education as a disruptive innovation,26, 27 shows a possible way of providing quality education to its one million physicians along with the other members of the health care team.
Lessons for practice
Continuing interprofessional education in HAI was challenging but possible, mimicking real-world interactions, thus explaining part of the success of this initiative.
Massive interactive online courses in the regional languages, addressing local needs, with international and local Faculty, may be a possible way to efficiently address the educational needs of one million physicians working in Latin America, as well as the other health professionals and teams considered as a whole.
Funding
This educational program was supported financially by the Society for Worldwide Medical Exchange and the American Society for Microbiology.
Conflicts of interest
The authors declare no conflicts of interest.
Acknowledgements
The authors would like to acknowledge the participation of the international experts and tutors for the regional needs assessment, the production of study materials and the delivery of the course. And in particular, the presence of Dr. Rafael Cantón (Hospital Universitario Ramón y Cajal, Madrid, Spain) and Dr. Denise Cardo (CDC, USA) at the live launching event in Montevideo was very important.
We also thank the enthusiasm and support from Ms. Connie Herndon, Director of Strategic Alliances at the American Society for Microbiology.
References
- 1.Carvalhaes C.G., Cayô R., Gales A.C. Klebsiella pneumoniae carbapenemase-producing Klebsiella pneumoniae in the intensive care unit: a real challenge to physicians, scientific community, and society. Shock. 2013;39(Suppl. 1):32–37. doi: 10.1097/SHK.0b013e31828faf1f. [DOI] [PubMed] [Google Scholar]
- 2.Gales A.C., Castanheira M., Jones R.N., Sader H.S. Antimicrobial resistance among Gram-negative bacilli isolated from Latin America: results from SENTRY Antimicrobial Surveillance Program (Latin America, 2008–2010) Diagn Microbiol Infect Dis. 2012;73:354–360. doi: 10.1016/j.diagmicrobio.2012.04.007. [DOI] [PubMed] [Google Scholar]
- 3.Cordeiro N., Robino L., Medina J., et al. Profloxacin-resistant enterobacteria harboring the aac(6)-Ib-cr variant isolated from feces of inpatients in an intensive care unit in Uruguay. Antimicrob Agents Chemother. 2008;52:806–807. doi: 10.1128/AAC.00444-07. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.de Oliveira M.S., de Assis D.B., Freire M.P., et al. Treatment of KPC-producing Enterobacteriaceae: suboptimal efficacy of polymyxins. Clin Microbiol Infect. 2015;21 doi: 10.1016/j.cmi.2014.07.010. 179.e1–7. [DOI] [PubMed] [Google Scholar]
- 5.Medina-Presentado J.C., Seija V., Vignoli R., et al. Polyclonal endemicity of Acinetobacter baumannii in ventilated patients in an intensive care unit from Uruguay. Int J Infect Dis. 2013;17:e422–e427. doi: 10.1016/j.ijid.2012.12.025. [DOI] [PubMed] [Google Scholar]
- 6.Medina J., Formento C., Pontet J., et al. Prospective study of risk factors in patients with ventilator-associated pneumonia caused by Acinetobacter spp. J Crit Care. 2007;22:18–27. doi: 10.1016/j.jcrc.2006.06.010. [DOI] [PubMed] [Google Scholar]
- 7.Cayô R., Rodrigues-Costa F., Matos A.P., Carvalhaes C.G., Jové T., Gales A.C. Identification of a new integron harboring blaIMP-10 in carbapenem-resistant Acinetobacter baumannii clinical isolates. Antimicrob Agents Chemother. 2015;59:3687–3689. doi: 10.1128/AAC.04991-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lessa F.C., Mu Y., Bamberg W.M., et al. Burden of Clostridium difficile infection in the United States. N Engl J Med. 2015;372:825–834. doi: 10.1056/NEJMoa1408913. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Aguayo C., Flores R., Lévesque S., et al. Rapid spread of Clostridium difficile NAP1/027/ST1 in Chile confirms the emergence of the epidemic strain in Latin America. Epidemiol Infect. 2015;17:1–5. doi: 10.1017/S0950268815000023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Quesada-Gómez C., López-Ureña D., Acuña-Amador L., et al. Emergence of an outbreak-associated Clostridium difficile variant with increased virulence. J Clin Microbiol. 2015;53:1216–1226. doi: 10.1128/JCM.03058-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Paciel D., Medina J. Clostridium difficile: historia de una infección emergente y su manejo actual. An Fac Med (Univ Repúb Urug) 2015;2:1–10. [Google Scholar]
- 12.Merquior V.L., Faria A.R., Rocha F.S., Morais J.M., Neves F.P., Teixeira L.M. Emergence and characterisation of vanB vancomycin-resistant Enterococcus faecalis in Rio de Janeiro, Brazil. Mem Inst Oswaldo Cruz. 2012;107:557–560. doi: 10.1590/s0074-02762012000400020. [DOI] [PubMed] [Google Scholar]
- 13.Cohen H., Margolis A., González N., et al. [Implementation and evaluation of a blended learning course on gastroesophageal reflux disease for physicians in Latin America] Gastroenterol Hepatol. 2014;37:402–407. doi: 10.1016/j.gastrohep.2014.01.004. [DOI] [PubMed] [Google Scholar]
- 14.Llambí L., Esteves E., Martinez E., et al. Teaching tobacco cessation skills to Uruguayan physicians using information and communication technologies. J Contin Educ Health Prof. 2011;31:43–48. doi: 10.1002/chp.20100. [DOI] [PubMed] [Google Scholar]
- 15.Margolis A., Santiago B., Martínez E., et al. CME Congress; Toronto, Canada: 2012. Four-year collaboration in CME among Latin American institutions. [Google Scholar]
- 16.Margolis A., Gonzalez-Martinez F., Noboa O., et al. Online continuing medical education for the Latin American nephrology community. Stud Health Technol Inform. 2015;216:372–375. [PubMed] [Google Scholar]
- 17.Medeiros N, Pérez G, Sniadover N, López A. Generador de casos interactivos. Presented at MoodleMoot Uruguay 2011. Available at: http://www.moodlemoot.org.uy/moodlemoot_2011/moodlemoot/moodlemootuy2011_submission_26.pdf.
- 18.Liyanagunawardena T.R., Williams S.A. Massive open online courses on health and medicine: review. J Med Internet Res. 2014;16:e191. doi: 10.2196/jmir.3439. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Mansouri M., Lockyer J. A meta-analysis of continuing medical education effectiveness. J Contin Educ Health Prof. 2007;27:6–15. doi: 10.1002/chp.88. [DOI] [PubMed] [Google Scholar]
- 20.Lam-Antoniades M., Ratnapalan S., Tait G. Electronic continuing education in the health professions: an update on evidence from RCTs. J Contin Educ Health Prof. 2009;29:44–51. doi: 10.1002/chp.20005. [DOI] [PubMed] [Google Scholar]
- 21.Parboosingh J., Reed V.A., Caldwell Palmer J., Bernstein H.H. Enhancing practice improvement by facilitating practitioner interactivity: new roles for providers of continuing medical education. J Contin Educ Health Prof. 2011;31:122–127. doi: 10.1002/chp.20116. [DOI] [PubMed] [Google Scholar]
- 22.Wakefield J., Herbert C.P., Maclure M., Dormuth C., Wrig J.M., Legare J. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof. 2003;23:81–93. doi: 10.1002/chp.1340230205. [DOI] [PubMed] [Google Scholar]
- 23.Owen J.A., Schmitt M.H. Integrating interprofessional education into continuing education: a planning process for continuing interprofessional education programs. J Contin Educ Health Prof. 2013;33:109–117. doi: 10.1002/chp.21173. [DOI] [PubMed] [Google Scholar]
- 24.MacNeill H., Telner D., Sparaggis-Agaliotis A., Hanna E. All for one and one for all: understanding health professionals’ experience in individual versus collaborative online learning. J Contin Educ Health Prof. 2014;34:102–111. doi: 10.1002/chp.21226. [DOI] [PubMed] [Google Scholar]
- 25.Maloney S., Haas R., Keating J.L., et al. Breakeven cost benefit cost effectiveness and willingness to pay for web-based versus face-to-face education delivery for health professionals. J Med Internet Res. 2012;14:e47. doi: 10.2196/jmir.2040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Christensen C.M., Armstrong E.G. Disruptive technologies: a credible threat to leading programs in continuing medical education? J Contin Educ Health Prof. 1998;18:69–80. [Google Scholar]
- 27.Harris J.M., Jr., Sklar B.M., Amend R.W., Novalis-Marine C. The growth, characteristics, and future of online CME. J Contin Educ Health Prof. 2010;30:3–10. doi: 10.1002/chp.20050. [DOI] [PubMed] [Google Scholar]