Skip to main content
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2018 Jul 23.
Published in final edited form as: Am J Infect Control. 2016 Jul 14;44(11):1360–1364. doi: 10.1016/j.ajic.2016.04.211

Teaching health care workers to adopt a systems perspective for improved control and prevention of health care–associated infections

AR Ruis a, David Williamson Shaffer b, Daniel K Shirley c, Nasia Safdar c,d,*
PMCID: PMC6055227  NIHMSID: NIHMS835567  PMID: 27424302

An estimated 4% of inpatients in U.S. acute care hospitals are diagnosed with preventable health care–associated infections (HAIs).1 Although concerted infection control efforts have achieved reductions in some HAIs,2 the prevalence of HAIs as a whole is growing. This rise in prevalence is occurring despite increasing efforts to improve infection control protocols and implement prevention measures.3

Research4,5 has shown that improvements in infection control and prevention are dependent not only on training health care workers (HCWs) to perform clinical techniques and apply administrative protocols but also on (1) the extent to which such practices are accepted by HCWs as useful or necessary and (2) the identification and removal of barriers to implementation. We argue that procedural approaches alone, even with high levels of adherence, are often insufficient to solve the growing problem of HAIs; it is equally important that interventions address the more complex cognitive aspects of HAI control and prevention. HCWs face many patient care situations for which standard procedures have not been and cannot be developed. In these cases, HCWs must make decisions with incomplete information and a high degree of uncertainty; understand and balance risk versus reward; account for numerous demands, including those of patients, hospital administrators, and insurance providers; coordinate care across multiple contexts and caregivers; and exercise clinical judgment. To implement a reduction in HAIs therefore requires that HCWs follow infection control procedures but also make clinical decisions related to HAI prevention, and the former must be situated within the latter. This suggests that improvements in infection control education and training for HCWs are critical for reducing HAIs.

In this commentary, our aim is to characterize and discuss the affordances of a novel approach to this problem based on cognitive simulation6-8: a practice-based intervention in which participants learn to solve multidimensional problems characterized by incomplete information, multiple stakeholders, and incommensurate demands. This approach, though as yet untried in the context of the complex arena of HAI prevention, has the potential to improve implementation of and adherence to infection control procedures and, more importantly, help HCWs learn to make clinical decisions that promote HAI control and prevention.

There are, of course, many types of cognitive simulation in medical education,6,9-12 and they are used for a wide range of training purposes. According to a recent review by Satish et al,13 however, most cognitive simulations seek to replicate the challenges and demands of clinical practice, providing a safe setting for HCWs to rehearse skills, gain confidence, and develop their abilities in case management, critical thinking, decision-making, and other important aspects of professional practice. However, simulating day-to-day practice does not help HCWs understand the systemic aspects of the problem and the bigger picture of HAI prevention.

IMPORTANCE OF NONPROCEDURAL AND SYSTEMIC FACTORS IN INFECTION PREVENTION

In addition to ensuring that HCWs have mastered and routinely apply basic infection control protocols, HAI reduction is critically dependent on a number of nonprocedural factors, including for example (1) reducing unnecessary and inappropriate use of antibiotics; (2) assessing and managing infection risk at both the individual and institutional levels; (3) managing patient care across clinical microsystems (eg, coordinating care across different clinical teams as a patient is transitioned from surgery to postoperative recovery to rehabilitative support or to another institution); (4) clarifying roles and responsibilities on clinical care teams and therefore reducing role ambiguity; (5) ensuring that a wide range of HCWs fully understand and appropriately implement infection control and prevention measures, both procedural and not; and (6) optimizing infection control processes and prevention measures for different contexts and circumstances.14-18 Furthermore, HCWs must understand the clinical and economic trade-offs among various HAI control measures, such as environmental disinfection, antibiotic stewardship, patient isolation and transport precautions, use of personal protective equipment, hand hygiene, and optimal laboratory testing. For these reasons, clinicians must be able to adopt a systems perspective for patient care, and because trainees are a major part of the frontline clinical workforce, this recommendation is especially applicable to them.

In 2001, the Accreditation Council for Graduate Medical Education began requiring graduate training programs in the United States to incorporate systems-based practice into their curricula, but most have struggled to implement and assess such training.19-21 Although medical and public health education have begun to converge in recent years, reflecting the extent to which clinical practice is increasingly situated within prevention frameworks,22,23 graduate education remains focused on specialization and patient-centered practice.20 This makes it challenging for medical trainees to understand the boundaries, stakeholders, and complex interactions that characterize health care systems, and they view patient care more holistically.

SIMULATION-BASED APPROACH TO TRAINING IN HAI PREVENTION

Educational interventions, particularly graduate and in-service training, comprise a key component of infection control efforts,5,15,16,24-26 but they most involve only training in basic procedures. In a recent review24 of educational interventions to prevent HAIs, the most commonly used tools were “lectures or classes, video presentations, posters, questionnaires and fact sheets, and practical demonstrations.” Online interventions are also based in basic dissemination of information.26 Such approaches effectively convey basic facts and approaches, but they do not help HCWs learn to think broadly about HAI control and prevention.

More involved interventions have been developed in other contexts to encourage systems-based thinking. Englander et al,27 for example, engaged residents in an institutional cost-reduction exercise in which the goal was to lower laboratory testing costs without diminishing the quality of patient care. Because residents request laboratory services extensively, they were able to identify key barriers to the use of point-of-care testing and develop strategies to overcome those barriers, which ultimately saved the institution >$500,000 per year. A key observation is that the residents approached the problem incorporating the view of a health care administrator—that is, someone concerned with management and cost of health care services—rather than through solely the lens of a clinician. In other words, through this exercise, residents learned how diagnostic decision-making can influence hospital economics and that both can be optimized to improve patient care. This is at the core of understanding a health care system.

Although trainees should be included in such exercises whenever feasible, it is difficult to integrate these live programs into already demanding training. Furthermore, such interventions offer little opportunity for supervisors to evaluate trainees, and data cannot be easily collected for assessment of learning and performance.

An online simulation focused on the systems-based approach has numerous advantages over participation in live scenarios such as the one previously described. Among other things, a simulation can (1) be scaffolded to support specific learning objectives and performance outcomes; (2) enable HCWs to consider a complex problem from a different perspective, such as that of a hospital administrator, helping them better understand the systemic aspects of the problem; (3) incorporate large numbers of HCWs who can collaborate even across institutional or geographic barriers; and (4) collect rich data on participant performance, making detailed assessment of learning, attitudes, and practices possible. In addition, research28 suggests that an online simulation may help to overcome barriers to learning such as those related to differences in experience, health care role, or sex.

We propose a simulation-based approach to training in HAI control and prevention based on a particular theory about how people learn and develop mastery in a particular context. According to epistemic frame theory,29-31 expertise in a domain is characterized not by the accumulation of knowledge and skills in isolation but by the particular set and configuration of knowledge, skills, values, habits of mind, and ways of making and justifying decisions, the epistemic frame of that domain. Understanding comes from making connections among different elements of practice, such as the connection that residents made between health care economics and diagnostic decision-making in the exercise previously described. Cognitive simulations are a particularly powerful training tool because they help learners make meaningful and situated connections among skills and knowledge and the decision-making processes that guide action. A broad range of research28,32-36 has shown that cognitive simulations help learners integrate these elements, making them meaningful, actionable, and persistent.

Importantly, a cognitive simulation developed from this theoretical perspective needs to do more than simply recreate a patient care scenario. HCWs develop expertise in HAI control not only understanding basic patient-specific skills, but also through cognitive processes that take into account how patient care decisions have effects beyond the patient in a complex health care system. To develop these cognitive processes, HCWs need to be able to analyze and solve patient care problems by taking into account multiple perspectives, not only theirs, but also the perspectives of patients, other providers, administrators, and those charged with infection control at the institutional or community level.

An online cognitive simulation in which HCWs approach HAI control and prevention from the perspective of a hospital's director of infection control, for example, would help them learn to understand the intricacies of that position outside of patient care. A key element of this approach is to assume a role that is fundamentally different from their own. Health care providers and administrators are interested in the same problem—HAI control and prevention—but they approach that problem from different perspectives: individual patient protection versus protection of patients throughout the health care system. Learning science research6,7,31,37,38 on how professionals are trained suggests that significant learning gains can be achieved when learners are able to assume a role different from their own in a realistic but scaffolded environment in which they can take consequential action. However, this approach does not replace existing procedural training, which remains a necessary component of HAI control protocols; rather, it incorporates or contextualizes that training, making clear the reasons for such policies and procedures and the mechanisms through which they are created and implemented.

In addition, a cognitive simulation can enable HCWs to understand the various—and at times incommensurate—perspectives of various stakeholders. For example, something as simple as decreasing urinary catheter use may be perceived as important for different reasons by different practitioners: for infection control personnel, it decreases the risk for infection and therefore antimicrobial use and multidrug resistance; for urologists, it decreases mechanical complications; for geriatricians, it decreases stress on already frail patients and risk for delirium; for surgeons, it decreases postoperative complications; and for physical therapists, it removes a barrier to mobility, therefore accelerating recovery. From the perspective of a hospital administrator, the push for prevention of HAIs may be driven by increased scrutiny from regulatory agencies or patient safety advocates or financial penalties for high HAI rates imposed by the Centers for Medicare and Medicaid.

COGNITIVE SIMULATION FOR PREVENTION OF HAIS: CLOSTRIDIUM DIFFICILE INFECTION AS AN EXAMPLE CASE

C difficile infection (CDI) is one of the most common HAIs in the United States,39,40 and prevention is particularly challenging. The pathogenesis of CDI is complex, with no device to remove or prevent it. The causative organism, C difficile, is spore-forming, highly antibiotic resistant and may have unknown reservoirs. In addition, the factors that contribute to CDI control and prevention are wide-ranging, as summarized in Figure 1. An online cognitive simulation could therefore provide an effective tool for HCWs to understand the complex, systemic issues that affect CDI rates.

Fig 1.

Fig 1

Diagram illustrating the principal factors involved in control and prevention of CDI. Adapted with permission from Carrico.14 CDI, Clostridium difficile infection; HCW, health care worker; PCR, polymerase chain reaction; PPE, personal protective equipment.

In a simulation designed to improve CDI control and prevention, HCWs might assume the role of infection control specialists on an administrative task force asked to design a CDI reduction strategy for the institution. They are given or asked to find information on effective CDI control and prevention measures, institutional priorities and regulations, and clinical CDI data, along with a budget, timeline, and description of the deliverables expected. Working in small teams and under the guidance of a more experienced mentor, HCWs explore different ways to reduce CDI rates without compromising patient care, exceeding their budget, or unduly increasing the burden placed on clinical staff.

In formulating initial protocols, HCWs must consult with key members of the institution's (virtual) infection control committee, which will ultimately evaluate their proposals. In doing so, they discover that different stakeholders value different aspects of infection control and prevention. For example, the hospital's chief operations officer may prioritize cost and logistical feasibility, whereas the director of clinical care may prioritize clinical scheduling and quality of patient care. Therefore, HCWs learn how to balance the concerns of different stakeholders in the health care system, ultimately crafting a protocol that addresses these concerns and explains the trade-offs made when different demands cannot all be met. Throughout this process, HCWs engage in regular reflection activities, both individually and with their teams, which help solidify the learning. At the end of the simulation, participants describe and justify their CDI reduction strategies.

This online approach to training would be more cost-effective, less time-consuming, and more accessible than a comparable live exercise, but it would also have several other advantages. First, an online system can more efficiently collect rich data on thinking and learning. For example, pre- and postsurveys could be integrated into the exercise as task-force entrance and exit interviews, allowing collection of demographic data and enabling assessment of changes in knowledge, skills, attitudes, and practices. Because all communication occurs through online chat and e-mail, complete transcripts can be logged for subsequent analysis of HCWs’ learning processes. These data can be used to better understand how HCWs think about infection control and prevention, ultimately helping to improve education and training interventions.

Second, the parameters can be easily adjusted to make such a simulation suitable for learners at different levels of training or experience. The problem space in which HCWs develop their CDI reduction plan, for example, could be more or less constrained and more or less scaffolded. Medical students who have fewer relevant experiences and less knowledge to draw on than practitioners could work on a version of the problem that is reduced in difficulty or scope and that contains more resources or guidance.

To our knowledge, such an approach has not been implemented in infection control and prevention, but prior research8,28,32,36,41 has shown it to be an effective method in a number of other contexts, including biomedical engineering design and decision-making in complex ecosocial systems. Online simulations have been developed and widely implemented, for example, in which participants design the ultrafiltration membrane for a hemodialysis unit,8 design an assistive mechanical exoskeleton for mobility enhancement,32 or develop a new land use plan for a city.33 These simulations help participants learn relevant content and practices, but they also help them learn how to identify, explore, and optimize trade-offs in complex systems.

CONCLUSIONS AND RECOMMENDATIONS

An online, cognitive simulation for HCWs that focuses on HAI reduction could ultimately provide a practice-based learning environment in which trainees and clinicians can develop knowledge and skills in infection control and, most importantly, develop the ability to make clinical decisions informed by a systems perspective on HAI control and prevention. Research suggests that this is a critically important challenge in modern health care. A recent review42 in the New England Journal of Medicine, for example, found that only 55% of U.S. patients receive recommended care; the most significant determinant for the 45% who do not is the “gap between actual and optimal performance” of health care systems.

Because infection control and prevention problems are multidimensional and systemic, a cognitive simulation, well designed and appropriately implemented, could enable HCWs not only to learn about best practices but, more importantly, to learn how to think differently about a problem whose solution requires sophisticated and coordinated activity among different actors across individual and institutional levels. To achieve optimal performance, health care systems require education and training programs that are cost-effective, scalable, and require minimal time investment. A cognitive simulation could therefore provide an effective model for addressing a wide range of infection control and patient care problems, ultimately helping to close the gap between actual and optimal performance.

Based on this, we are currently attempting to develop an online cognitive simulation for CDI control and prevention, which we plan to pilot with a small group of trainees to refine the approach and collect pilot data. In the meantime, there are 2 things that health care institutions could do to improve HAI training. First, authoring software that scaffolds the design and development of effective online cognitive simulations in any domain is currently available43 and could be used by health care systems. This software, VIA (Games and Professional Simulations Research Consortium, Madison, WI), ensures that any simulations produced are both pedagogically sound and technologically robust, and the system is designed for use by content experts who do not have experience with computer programming or educational simulation design. This software facilitates development of more effective online training interventions, and it may be especially useful for rapid deployment of training exercises in infection control contexts where live training may be difficult or dangerous, as in the case of the recent Ebola epidemic. Second, as the work of Englander et al27 suggests, there are advantages to including trainees and other clinical staff on administrative committees or task forces, and this could be leveraged in a health care system's HAI prevention efforts. Although this approach does not have the same potential for assessment and scaffolding as online simulation, it can be implemented while suitable online interventions are being developed. Most importantly, new methods are needed for helping HCWs, and especially trainees, understand the systemic nature of HAI control and prevention.

Acknowledgments

Funding/support: This work was funded in part by the National Science Foundation (DRL-0918409, DRL-0946372, DRL-1247262, DRL-1418288, DUE-0919347, DUE-1225885, EEC-1232656, EEC-1340402, and REC-0347000), the MacArthur Foundation, the Spencer Foundation, the Wisconsin Alumni Research Foundation, and the Office of the Vice Chancellor for Research and Graduate Education at the University of Wisconsin–Madison. N.S. is supported by a Veterans Affairs–funded patient safety center and an R03 from AHRQ.

Footnotes

Conflicts of interest: None to report.

Disclaimer: The opinions, findings, and conclusions do not reflect the views of the funding agencies, cooperating institutions, or other individuals.

References

  • 1.Magill SS, Edwards JR, Bamberg W, Beldavs ZG, Dumyati G, Kainer MA, et al. Multistate point-prevalence survey of health care–associated infections. N Engl J Med. 2014;370:1198–208. doi: 10.1056/NEJMoa1306801. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Centers for Disease Control and Prevention Vital signs: central line–associated blood stream infections—United States, 2001, 2008, and 2009. Ann Emerg Med. 2011;58:447–50. doi: 10.1016/j.annemergmed.2011.07.035. [DOI] [PubMed] [Google Scholar]
  • 3.Arnold K, Avery L, Bennett R, Brinsley-Rainisch K, Boyter M, Coffin N, et al. National and state healthcare-associated infections progress report. Centers for Disease Control and Prevention; Atlanta (GA): 2015. [Google Scholar]
  • 4.Saint S, Greene MT, Olmsted RN, Chopra V, Meddings J, Safdar N, et al. Perceived strength of evidence supporting practices to prevent health care-associated infection: results from a national survey of infection prevention personnel. Am J Infect Control. 2013;41:100–6. doi: 10.1016/j.ajic.2012.10.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Aboelela SW, Stone PW, Larson EL. Effectiveness of bundled behavioural interventions to control healthcare-associated infections: a systematic review of the literature. J Hosp Infect. 2007;66:101–8. doi: 10.1016/j.jhin.2006.10.019. [DOI] [PubMed] [Google Scholar]
  • 6.Shaffer DW, Dawson SL, Meglan D, Cotin S, Ferrell M, Norbash A, et al. Design principles for the use of simulation as an aid in interventional cardiology training. Minim Invasive Ther Allied Technol. 2000;10:75–82. doi: 10.1080/13645700152601351. [DOI] [PubMed] [Google Scholar]
  • 7.Dawson SL, Cotin S, Meglan D, Shaffer DW, Ferrell M. Designing a computer-based simulator for interventional cardiology training. Catheter Cardiovasc Interv. 2000;51:522–7. doi: 10.1002/1522-726x(200012)51:4<522::aid-ccd30>3.0.co;2-7. [DOI] [PubMed] [Google Scholar]
  • 8.Chesler NC, Arastoopour G, D'Angelo CM, Bagley EA, Shaffer DW. Design of a professional practice simulator for educating and motivating first-year engineering students. Adv Eng Educ. 2013;3:1–29. [Google Scholar]
  • 9.Pauker SG, Gorry GA, Kassirer JP, Schwartz WB. Towards the simulation of clinical cognition: taking a present illness by computer. In: Reggia JA, Tuhrim S, editors. Computer-assisted medical decision making. Vol. 2. Springer; New York (NY): 1985. pp. 108–38. [Google Scholar]
  • 10.Satish U, Streufert S. Value of a cognitive simulation in medicine: towards optimizing decision making performance of healthcare personnel. Qual Saf Health Care. 2002;11:163–7. doi: 10.1136/qhc.11.2.163. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Mallott D, Raczek J, Skinner C, Jarrell K, Shimko M, Jarrell B. A basis for electronic cognitive simulation: the heuristic patient. Surg Innov. 2005;12:43–9. doi: 10.1177/155335060501200107. [DOI] [PubMed] [Google Scholar]
  • 12.Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med. 2010;85:1589–602. doi: 10.1097/ACM.0b013e3181edfe13. [DOI] [PubMed] [Google Scholar]
  • 13.Satish U, Krishnamurthy S, Dewan M. Role of cognitive simulation in healthcare. In: Riley RH, editor. Manual of simulation in healthcare. 2nd ed. Oxford University Press; Oxford, UK: 2015. pp. 424–38. [Google Scholar]
  • 14.Carrico R. Preventing transmission of Clostridium difficile in healthcare settings. IP Tools. Association for Professionals in Infection Control; Washington (DC): 2011. [Google Scholar]
  • 15.Lo E, Nicolle LE, Coffin SE, Gould C, Maragakis LL, Meddings J, et al. Strategies to prevent catheter-associated urinary tract infections in acute care hospitals: 2014 update. Infect Control. 2014;35:464–79. doi: 10.1086/675718. [DOI] [PubMed] [Google Scholar]
  • 16.Gould CV, Umscheid CA, Agarwal RK, Kuntz G, Pegues DA. Guideline for prevention of catheter-associated urinary tract infections 2009. Centers for Disease Control and Prevention; Atlanta (GA): 2009. [DOI] [PubMed] [Google Scholar]
  • 17.Marschall J, Mermel LA, Classen D, Arias KM, Podgorny K, Anderson DJ, et al. Strategies to prevent central line–associated bloodstream infections in acute care hospitals. Infect Control Hosp Epidemiol. 2008;29(Suppl):S22–30. doi: 10.1086/591059. [DOI] [PubMed] [Google Scholar]
  • 18.Krein SL, Kowalski CP, Hofer TP, Saint S. Preventing hospital-acquired infections: a national survey of practices reported by US hospitals in 2005 and 2009. J Gen Intern Med. 2012;27:773–9. doi: 10.1007/s11606-011-1935-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Varkey P, Karlapudi S, Rose S, Nelson R, Warner M. A systems approach for implementing practice-based learning and improvement and systems-based practice in graduate medical education. Acad Med. 2009;84:335–9. doi: 10.1097/ACM.0b013e31819731fb. [DOI] [PubMed] [Google Scholar]
  • 20.Colbert CY, Ogden PE, Ownby AR, Bowe C. Systems-based practice in graduate medical education: systems thinking as the missing foundational construct. Teach Learn Med. 2011;23:179–85. doi: 10.1080/10401334.2011.561758. [DOI] [PubMed] [Google Scholar]
  • 21.Shortell SM, Singer SJ. Improving patient safety by taking systems seriously. J Am Med Assoc. 2008;299:445–7. doi: 10.1001/jama.299.4.445. [DOI] [PubMed] [Google Scholar]
  • 22.Maeshiro R. Responding to the challenge: population health education for physicians. Acad Med. 2008;83:319–20. doi: 10.1097/ACM.0b013e318166ae9f. [DOI] [PubMed] [Google Scholar]
  • 23.Ruis AR, Golden RN. The schism between medical and public health education: a historical perspective. Acad Med. 2008;83:1153–7. doi: 10.1097/ACM.0b013e31818c6583. [DOI] [PubMed] [Google Scholar]
  • 24.Safdar N, Abad C. Educational interventions for prevention of healthcare-associated infection: a systematic review. Crit Care Med. 2008;36:933–40. doi: 10.1097/CCM.0B013E318165FAF3. [DOI] [PubMed] [Google Scholar]
  • 25.McHugh SM, Hill ADK, Humphreys H. Preventing healthcare-associated infection through education: have surgeons been overlooked? Surgeon. 2010;8:96–100. doi: 10.1016/j.surge.2009.11.009. [DOI] [PubMed] [Google Scholar]
  • 26.Oman KS, Makic MBF, Fink R, Schraeder N, Hulett T, Keech T, et al. Nurse-directed interventions to reduce catheter-associated urinary tract infections. Am J Infect Control. 2012;40:548–53. doi: 10.1016/j.ajic.2011.07.018. [DOI] [PubMed] [Google Scholar]
  • 27.Englander R, Agostinucci W, Zalneraiti E, Carraccio CL. Teaching residents systems-based practice through a hospital cost-reduction program: a “win–win” situation. Teach Learn Med. 2006;18:150–2. doi: 10.1207/s15328015tlm1802_10. [DOI] [PubMed] [Google Scholar]
  • 28.Arastoopour G, Chesler NC, Shaffer DW. Epistemic persistence: a simulation-based approach to increasing participation of women in engineering. J Women Minor Sci Eng. 2014;20:211–34. [Google Scholar]
  • 29.Shaffer DW. Proceedings of the 6th International Conference on Learning Sciences. International Society of the Learning Sciences; Santa Monica, CA: 2004. Epistemic frames and islands of expertise: Learning from infusion experiences. pp. 473–80. [Google Scholar]
  • 30.Shaffer DW. Epistemic frames for epistemic games. Comput Educ. 2006;46:223–34. [Google Scholar]
  • 31.Shaffer DW. Models of situated action: computer games and the problem of transfer. In: Steinkuehler C, Squire KD, Barab SA, editors. Games, learning, and society: learning and meaning in the digital age. Cambridge University Press; Cambridge, UK: 2012. pp. 403–31. [Google Scholar]
  • 32.Chesler NC, Ruis AR, Collier W, Swiecki Z, Arastoopour G, Shaffer DW. A novel paradigm for engineering education: virtual internships with individualized mentoring and assessment of engineering thinking. J Biomech Eng. 2015;137:024701. doi: 10.1115/1.4029235. [DOI] [PubMed] [Google Scholar]
  • 33.Shaffer DW. How computer games help children learn. Palgrave Macmillan; New York, NY: 2007. [Google Scholar]
  • 34.Dede C. Immersive interfaces for engagement and learning. Science. 2009;323:66–9. doi: 10.1126/science.1167311. [DOI] [PubMed] [Google Scholar]
  • 35.Clark D, Linn MC. Designing for knowledge integration: the impact of instructional time. J Learn Sci. 2003;12:451–93. [Google Scholar]
  • 36.Arastoopour G, Shaffer DW, Swiecki Z, Ruis AR, Chesler NC. Teaching and assessing engineering design thinking with virtual internships and epistemic network analysis. Int J Eng Educ. in press. [Google Scholar]
  • 37.Lave J, Wenger E. Situated learning: legitimate peripheral participation. Cambridge University Press; Cambridge, MA: 1991. [Google Scholar]
  • 38.Cotin S, Shaffer DW, Meglan D, Ottensmeyer M, Berry P, Dawson SL. CAML: a general framework for the development of medical simulations. In: Pien H, editor. SPIE. Vol. 4037. Battlefield Biomedical Technologies II; 2000. [Google Scholar]
  • 39.Kyne L, Hamel MB, Polavaram R, Kelly CP. Health care costs and mortality associated with nosocomial diarrhea due to Clostridium difficile. Clin Infect Dis. 2002;34:346–53. doi: 10.1086/338260. [DOI] [PubMed] [Google Scholar]
  • 40.Zimlichman E, Henderson D, Tamir O, Franz C, Song P, Yamin CK, et al. Health care–associated infections: a meta-analysis of costs and financial impact on the US health care system. JAMA Intern Med. 2013;173:2039–46. doi: 10.1001/jamainternmed.2013.9763. [DOI] [PubMed] [Google Scholar]
  • 41.Bagley EA, Shaffer DW. When people get in the way: promoting civic thinking through epistemic game play. Int J Gaming Comput Mediat Simul. 2009;1:36–52. [Google Scholar]
  • 42.Asch SM, Kerr EA, Keesey J, Adams JL, Setodji CM, Malik S, et al. Who is at greatest risk for receiving poor-quality health care? N Engl J Med. 2006;354:1147–56. doi: 10.1056/NEJMsa044464. [DOI] [PubMed] [Google Scholar]
  • 43.Shaffer DW, Ruis AR, Graesser AC. Authoring networked learner models in complex domains. In: Sottilare R, Hu X, Graesser AC, editors. Design recommendations for intelligent tutoring systems: authoring tools. U.S. Army Research Laboratory; Orlando (FL): 2015. pp. 179–91. [Google Scholar]

RESOURCES