Abstract
Objective:
The purpose of this overview (systematic review of systematic reviews) is to evaluate the impact of clinical decision support systems (CDSS) applied to medication use in the care process.
Methods:
A search for systematic reviews that address CDSS was performed on Medline following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Cochrane recommendations. Terms related to CDSS and systematic reviews were used in combination with Boolean operators and search field tags to build the electronic search strategy. There was no limitation of date or language for inclusion. We included revisions that investigated, as a main or secondary objective, changes in process outcomes. The Revised Assessment of Multiple Systematic Reviews (R-AMSTAR) score was used to evaluate the quality of the studies.
Results:
The search retrieved 954 articles. Five articles were added through manual search, totaling an initial sample of 959 articles. After screening and reading in full, 44 systematic reviews met the inclusion criteria. In the medication-use processes where CDSS was used, the most common stages were prescribing (n=38 (86.36%) and administering (n=12 (27.27%)). Most of the systematic reviews demonstrated improvement in the health care process (30/44 – 68.2%). The main positive results were related to improvement of the quality of prescription by the physicians (14/30 – 46.6%) and reduction of errors in prescribing (5/30 - 16.6%). However, the quality of the studies was poor, according to the score used.
Conclusion:
CDSSs represent a promising technology to optimize the medication-use process, especially related to improvement in the quality of prescriptions and reduction of prescribing errors, although higher quality studies are needed to establish the predictors of success in these systems.
Keywords: Decision Support Systems, Clinical; Medication Systems; Drug Information Services; Medical Informatics; Medication Errors; Review Literature as Topic
INTRODUCTION
The volume and complexity of information needed to care for sick patients are immeasurable, especially when the lability of the clinical condition of a hospitalized patient is considered.1 In this context, the medication-use process is considered a key point for the reestablishment and maintenance of health.
The medication-use process comprises five major steps: 1) Prescription of a medicine based on the needs of the patient; 2) Transcription, the transmission of prescription information; 3) Dispensing, providing the medication according to the prescription; 4) Administration or use of the drug; and 5) Monitoring, the continued therapy evaluation of its effects on patients (i.e., effectiveness, safety, therapeutic failures).2,3,4 This process is complex and can be permeated by quality deviations and failures, resulting in medication errors at any stage.5,6,7,8,9
The use of information technology (IT) in the health area is a potentially transformative strategy for the quality of care and patient safety and has been embraced as a public policy in many countries.10,11 In 2009, the American Recovery and Reinvestment Act allocated more than thirty billion dollars to the development of health-related information technology.10,11
In this context, computerized decision support systems (CDSSs) are software programs designed to assist health professionals with daily decisions using complex communication, analysis, synthesis, and presentation of clinical information selected from multiple sources.1,12 These systems can improve the medication-use process by providing information for standardizing clinical practice; clinical alerts regarding drug allergies, drug interactions, and dose ranges appropriate to the needs of the patient; clinical updates on the latest information relevant to pharmacotherapy; dose adjustment calculations based on patient characteristics; and communications on critical changes in the condition of a patient, facilitating appropriate adjustments.13,14
Despite the increasing number of published studies on CDSSs, the effect of these systems on clinical, humanistic and economic outcomes is still contradictory, which highlights the need to evaluate other quality mediator parameters.
Considering health service evaluation, Donabedian outlined three constructs from which inferences can be made regarding their quality: outcomes, process and structure.15 In this definition, the construct process refers to what is done and how the care itself is done. From this perspective, the objective of the present systematic review was to evaluate the impact of CDDSs in the care process, considering the possible mediators of the quality of health care.
METHODS
A search for systematic reviews that addressed clinical decision support systems was performed on Medline on December 1, 2015 following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)16,17 and Cochrane18 recommendations. Terms related to CDSS and systematic reviews were used in combination with Boolean operators and search field tags to build the electronic search strategy that is fully provided in Table 1. There was no limitation of date or language for inclusion. Medline was the search engine selected because of its extensive indexation of health journals and wide use by researchers and health professionals from around the world. A manual search was also performed via triage in the references of the included systematic reviews.
Table 1. Search strategy in Pubmed.
| ((Decision Support Systems, Clinical[MH] OR “computerized decision support systems”[TIAB] OR “Decision Support System*”[TIAB] OR “decision support”[TIAB] OR “computer-assisted decision making”[TIAB] OR “clinical decision support system*”[TIAB] OR “clinical decision support tool*”[TIAB] OR “computerized decision support tool*”[TIAB] OR “clinical decision making tool*”[TIAB]) AND (systematic review[PT] OR meta-analysis[PT] OR systematic review[TIAB] OR meta-analysis[TIAB] OR systematic[SB] OR meta-analysis[MH]) NOT (randomized controlled trial[PT] OR controlled clinical trial[PT] OR random allocation[MH]) AND hasabstract)) |
First, an initial screening from titles and abstracts was performed in duplicate and independently considering, as the main focus, the selection of systematic reviews that applied CDSS to the use of drugs, which reported changes in process outcomes, as defined by Donabedian.15 CDSSs applied to the medication-use process consisted of those applied in the following steps: selection and prescribing, transcribing and verifying, dispensing, administering and therapeutic monitoring.
Second, eligibility of the systematic reviews was assessed by full-text evaluation. Reviews that investigated, as the main or secondary objective, changes in the quality of the medication-use process were included. Narrative revisions, synthesis of secondary sources of data, and descriptive reviews were excluded.
After this step, we proceeded with data extraction that was performed using a previously defined Excel® worksheet. Different process outcomes were grouped by similarity, their impact classified as positive, data insufficient for evaluation, neutral or negative, considering the authors’ report, and the significance statistics performed in each review. Each systematic review included was evaluated using the Revised Assessment of Multiple Systematic Reviews (R-AMSTAR) scale.19 The screening, eligibility and quality evaluation phases were performed in duplicate by two independent reviewers; the disagreements were resolved with a third reviewer.
RESULTS
The search strategy retrieved 954 articles. Five articles were added from manual search, totaling an initial sample of 959 articles. Subsequently, the screening of titles and abstracts resulted in the exclusion of 647 articles. The 309 remaining systematic reviews were read in full; 44 met the inclusion criteria and 267 were excluded. This process is shown in the flow chart provided in Figure 1, which also provides the reasons for excluding the articles after reading in full.
Figure 1. PRISMA flow diagram for the review process.
Most systematic reviews included experimental studies, especially randomized controlled trials. A wide variety of types of CDSSs was observed, integrated or not to other systems, such as computerized provider order entry (CPOE) or electronic health records (EHR) (Online appendix). Considering the setting, most of the reviews (23/44 - 52.27%) included CDSS applied to outpatients and/or inpatients.20-42
There are five stages of the medication-use process: prescribing; transcribing and verifying; dispensing and delivering; administering; and monitoring. As shown in Online appendix, the major process stage, in which the CDSS was used, was prescribing (n=38 (86.36%)), followed by administering (n=12 (27.27%)). The use of CDSS was not observed for transcribing and verifying or dispensing and delivering processes. Additionally, there were systematic reviews that included more than one stage of the medication-use process prescribing and administering (n=12 (18.2%)), prescribing and monitoring (n=6 (13.63%)), and only one study considered three stages: prescribing, administering, and monitoring.43
Quality evaluation
In general, the quality of the included systematic reviews was poor (Table 2), which makes the inference of robust conclusions complicated. The average R-AMSTAR score of the reviews was 24.39 (SD=5.28). Only eight studies performed meta-analysis (18.2%), so the domain 9 was not applicable for the majority of studies, reflecting the low scores found. Seventeen studies scored 22 points or lower, and just five reviews scored higher than 30 points.
Table 2. Quality evaluation of systematic reviews according to R-AMSTAR.
| Author, year | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | Total score |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Bright, 2012 | 3 | 2 | 3 | 2 | 2 | 4 | 4 | 2 | 3 | 2 | 3 | 30 |
| Carling, 2013 | 3 | 3 | 4 | 3 | 4 | 4 | 2 | 3 | NA | 1 | 4 | 31 |
| Chaudhry, 2006 | 2 | 3 | 3 | 1 | 2 | 3 | 1 | 2 | NA | 1 | 3 | 21 |
| Durieux, 2008 | 4 | 3 | 4 | 2 | 4 | 4 | 1 | 2 | NA | 1 | 3 | 26 |
| Eslami, 2007 | 3 | 3 | 3 | 1 | 2 | 1 | 1 | 1 | NA | 1 | 1 | 17 |
| Fathima, 2014 | 3 | 3 | 4 | 2 | 3 | 4 | 2 | 2 | NA | 2 | 3 | 28 |
| Fraccaro, 2015 | 3 | 1 | 4 | 3 | 1 | 3 | 1 | 1 | NA | 2 | 3 | 26 |
| Garg, 2005 | 3 | 3 | 3 | 1 | 1 | 2 | 2 | 4 | NA | 1 | 3 | 23 |
| Georgiou, 2013 | 3 | 3 | 4 | 2 | 2 | 2 | 1 | 1 | NA | 2 | 3 | 23 |
| Gillaizeau, 2013 | 3 | 4 | 4 | 3 | 4 | 4 | 4 | 4 | 4 | 2 | 2 | 38 |
| Hemens, 2011 | 4 | 3 | 2 | 1 | 2 | 4 | 2 | 1 | NA | 2 | 3 | 24 |
| Heselmans, 2009 | 3 | 3 | 4 | 2 | 4 | 4 | 1 | 4 | NA | 1 | 3 | 29 |
| Holstiege, 2015 | 2 | 3 | 4 | 3 | 4 | 4 | 2 | 2 | NA | 2 | 3 | 29 |
| Hunt, 1998 | 2 | 3 | 3 | 3 | 1 | 2 | 2 | 2 | NA | 1 | 2 | 21 |
| Jamal, 2009 | 3 | 1 | 4 | 3 | 1 | 2 | 2 | 4 | NA | 1 | 1 | 22 |
| Jeffery, 2013 | 4 | 4 | 3 | 1 | 1 | 4 | 2 | 2 | 2 | 1 | 4 | 28 |
| Lainer, 2013 | 3 | 3 | 4 | 3 | 2 | 4 | 1 | 1 | NA | 1 | 3 | 25 |
| Lobach, 2012 | 3 | 4 | 4 | 2 | 4 | 4 | 3 | 3 | 4 | 4 | 3 | 38 |
| Marasinghe, 2015 | 3 | 1 | 3 | 1 | 0 | 2 | 3 | 2 | NA | 2 | 3 | 20 |
| Marc, 2013 | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 1 | NA | 1 | 1 | 10 |
| Millery, 2010 | 3 | 3 | 4 | 1 | 1 | 4 | 1 | 1 | NA | 2 | 3 | 23 |
| Montgomery, 1998 | 3 | 1 | 3 | 3 | 1 | 4 | 1 | 1 | NA | 1 | 3 | 21 |
| Nieuwlaat, 2011 | 3 | 3 | 3 | 1 | 2 | 4 | 2 | 2 | 4 | 1 | 3 | 28 |
| Njie, 2015 | 2 | 3 | 1 | 1 | 0 | 3 | 2 | 4 | NA | 2 | 3 | 21 |
| Oluoch, 2012 | 3 | 3 | 4 | 2 | 2 | 4 | 1 | 2 | NA | 1 | 2 | 24 |
| O’Reilly, 2012 | 2 | 2 | 3 | 3 | 0 | 3 | 1 | 1 | NA | 1 | 3 | 19 |
| Pearson, 2009 | 3 | 3 | 4 | 2 | 2 | 4 | 2 | 2 | NA | 1 | 3 | 26 |
| Randell, 2007 | 3 | 3 | 3 | 4 | 1 | 2 | 2 | 1 | NA | 1 | 1 | 21 |
| Robertson, 2010 | 3 | 4 | 4 | 2 | 2 | 4 | 3 | 2 | NA | 1 | 3 | 28 |
| Roshanov, 2011 (1) | 4 | 4 | 4 | 3 | 2 | 4 | 2 | 2 | NA | 2 | 3 | 30 |
| Roshanov, 2011 (2) | 3 | 3 | 3 | 1 | 2 | 4 | 2 | 2 | NA | 2 | 3 | 25 |
| Sahota, 2011 | 3 | 3 | 3 | 3 | 2 | 4 | 3 | 2 | NA | 2 | 3 | 28 |
| Schedlbauer, 2009 | 3 | 3 | 3 | 3 | 2 | 1 | 2 | 1 | NA | 1 | 1 | 20 |
| Shebl, 2007 | 2 | 1 | 3 | 3 | 1 | 4 | 1 | 1 | NA | 1 | 3 | 20 |
| Shojania, 2009 | 3 | 3 | 3 | 4 | 2 | 2 | 1 | 1 | 1 | 1 | 3 | 24 |
| Shojania, 2010 | 3 | 3 | 3 | 4 | 2 | 2 | 1 | 1 | 1 | 1 | 3 | 24 |
| Sintchenko, 2007 | 3 | 1 | 3 | 2 | 1 | 4 | 1 | 1 | NA | 2 | 1 | 19 |
| Souza, 2011 | 3 | 3 | 3 | 3 | 2 | 4 | 2 | 2 | NA | 2 | 3 | 27 |
| Sturzlinger, 2009 | 2 | 1 | 3 | 2 | 1 | 1 | 1 | 1 | NA | 1 | 1 | 14 |
| Tawadrous, 2011 | 3 | 4 | 3 | 4 | 2 | 4 | 2 | 1 | NA | 1 | 3 | 27 |
| Vedel, 2013 | 3 | 3 | 3 | 3 | 0 | 2 | 2 | 1 | NA | 2 | 2 | 22 |
| Welch, 2013 | 2 | 2 | 3 | 2 | 2 | 4 | 1 | 1 | NA | 2 | 3 | 22 |
| Wong, 2010 | 3 | 3 | 3 | 2 | 2 | 4 | 2 | 2 | 4 | 2 | 2 | 29 |
| Yourman, 2008 | 3 | 3 | 4 | 1 | 2 | 4 | 1 | 1 | NA | 2 | 1 | 22 |
NA: not applicable
Impact of the CDSS for the medication-use process
Most of the systematic reviews demonstrated improvement of the health care process (30/44 – 68.2%)27-56, as shown in Online appendix.
Quality of prescriptions
The main positive results were related to the improvement of the quality of prescriptions by the physicians (14/30 – 46.6%). The improvement of the quality of prescription included dose adjustments according to the clinical condition38,43,47 and the inclusion of the pharmacist in the medication review process.56
A review evaluating CDSSs applied to the prescription, guiding dose adjustments for different conditions, indicated benefits in increasing the initial dose (standardized mean difference 1.12), increasing serum concentrations (standardized mean difference 1.12), reducing the time to therapeutic stabilization (standardized mean difference -0.55) and reducing the length of hospital stay (standardized mean difference -0.35).38
A review evaluating the impact of CDSSs on addressing safety concerns (drug interactions, contraindications, dose monitoring and adjustment) and those focusing on medication use in line with guideline recommendations demonstrated that those applied to the safety of prescription had a high success rate, with 91% (10/11) of the studies reporting statistically significant improvements in favor of CDSSs on ≥50% of all outcomes reported.47
Combined analysis of ten studies included in a systematic review, evaluating systems applied to the therapeutic management of asthma and Chronic Obstructive Pulmonary Disease, indicated significant improvements in cases such as the cumulative sparing of prednisone dose adjusted weekly according to the internet-based CDSS, the percentage of children given at least one prescription of corticosteroid, the percentage of visits to the physician leading to medication step up of asthma medication and in the number of contacts with the patients’ physician.43
Garg (2005), in his review evaluating the impact of CDSSs on the performance of health professionals, indicated improvement in professional performance in 62 (64%) of the 97 studies that evaluated this result, including 4 (40%) of 10 health systems, 16 (76%) of the 21 warning systems, 23 (62%) of 37 disease management systems and 19 (66%) of 29 drug administration or prescription systems.55
From another perspective, in their review of CDSSs applied to multi-morbidity, Fracaro and colleagues (2015) emphasized the importance of these systems in the pharmaceutical review of medication, with a higher rate of identification of potential problems than the human experts.56
Lobach et al. (2012), in their review evaluating the impact of three types of CDSSs on clinical, humanistic, economic and process outcomes, indicated that these systems effectively improved health care process measures related to performing preventive services (OR=1.42), ordering clinical studies (OR=1.72), and prescribing therapies (OR=1.57).35
Reduction of prescribing errors
The reduction of prescription errors was another outcome of processes benefitted by the adoption of the CDSS, as reported in five reviews (5/30 - 16.6%).29,39,41,45,54
Georgiou (2013) demonstrated that the use of CPOE with CDSSs was related to significant decreases in prescribing errors (17-201 errors per 100 orders), potential adverse drug events (0.9 per 100 orders), and prescribing of excessive dosages (31%).54
Similarly, Schedlbauer (2009), in his review evaluating the impact of Computerized Alerts and Prompts to Improve Clinicians’ prescribing behavior, indicated that most studies (23 out of 27) demonstrated a benefit in improving prescription behavior and/or reducing the error rates of prescriptions.29 Three other reviews noted the benefits of CDSSs in prescription safety.39,41,45
Lainer (2013), in his review evaluating information technology interventions to improve medication safety in primary care, indicated that only 50% of the included studies revealed a reduction in error rates.51
Other results positively influenced by CDSSs related to the process included preventive care40,44; vaccination28,39,44; adherence to guideline-based care33,35,39,40; and monitoring.32,37,39,48
Chaundry and collaborators conducted a systematic review evaluating the impact of CDSS on the quality, efficiency, and costs of the care process and demonstrated absolute increases of 5 to 66 percentage points in outcomes related to process quality. The primary preventive measures most commonly examined were influenza vaccination rates (improvement from 12 to 18%), pneumococcal vaccines (improvement, 20 to 33%) and laboratory tests for evaluation of fecal occult blood (improvement, 12 to 33%). Benefits related to drug doses were also observed, varying from 12% to 21%. Considering the quality of care, the main benefits were greater adherence to evidence-based care, monitoring of processes and reduction of medication errors. In addition, the authors found benefits in time spent by health professionals, with absolute reductions ranging from 8.5 to 24%.39
Njie, 2015 et al., in their review evaluating the impact of CDSS on cardiovascular prevention, noted an improvement in the follow-up of guidelines for screening and other preventive care services (average global increase of 3.8%), requests for clinical exams (mean global increase of 4.0%) and treatment (overall mean increase of 2.0%).33
Twelve studies21-26,57-62 (27.3%) exhibited insufficient data to prove the efficacy of the medication-use process because some of them included primary studies with inconsistent results or results with no statistically significant difference. Only two reviews (4.5%) revealed no benefit in using CDSSs in process improvement.20,63
DISCUSSION
Most CDSS applied to the medication-use process have shown benefits in the process outcomes (62%), this situation is consistent with results noted in previous overviews.64-69
An overview performed by Cresswell (2012), with inclusion of 41 systematic reviews evaluating CDSSs, indicated that such tools may result in improvements in professional performance, promotion of preventive care, and adherence to clinical guidelines, particularly if specific information is available in real time and systems are effectively integrated into clinical workflows. However, the relationship with clinical outcomes was modest.66
Jasper (2011)70, in his overview, evaluated the impact of CDSSs on the performance of health professionals and patient outcomes. This author selected only systematic reviews with high methodological quality and demonstrated that clinical decision support systems significantly improved the professional’s performance in 52 of 91 primary studies included in 16 systematic reviews that examined this effect (57%). Despite the consonant results regarding benefits in the care process, we note that the overviews cited were not directed at the medication-use process, a critical stage in care.
The methodological difference between the selection of studies of the overview realized by Jasper and ours is noteworthy. Different from this author, who limited the methodological quality reviews to the selection of randomized clinical trial reviews, our intention was to obtain a broader sample that included observational studies and quasi-experimental analyses to simulate the process of using medications in the “real world”.
According to our results, the prescribing and administering stages particularly benefitted by the introduction of clinical decision support systems. These findings are justified, since these steps are associated with high rates of medication errors.
The main positive outcomes provided by CDSS were improvement in the quality of prescriptions and a reduction in the rate of medication errors. Adherence to clinical guidelines for screening, prevention, requests for clinical exams and treatment were also cited.
An overview of 2016, evaluating the impact of CDSS on medication safety69, with the inclusion of 20 systematic reviews, also mentioned that these systems significantly affected the care process by reducing medication errors and improving quality in 75% of the studies. However, the impact on patient outcomes was less consistent.
It is important to note that the change in process outcomes is an important mediator of quality of care; however, it does not always imply a change in clinical, humanistic and economic outcomes. The association between process outcome change and clinical impact has been little explored in the literature and deserves a detailed evaluation in different clinical settings with adequate descriptions of intervention characteristics to consolidate the clinical applicability of CDSSs.
As a limitation, we highlight the low methodological quality and the high heterogeneity reported in the reviews included in this study. This situation may compromise the inference of robust conclusions from the synthesized data. We believe that the broad mapping of the CDSS process outcomes applied to the use of medications allows for an impact survey, considering different settings and intervention methodologies for CDSSs application, which may help health professionals and institutions in the visualization of potential benefits and the implementation of decision making.
CONCLUSIONS
The adequacy of the medication-use process is fundamental to guarantee the quality and safety of care. CDSSs represent promising alternatives to improve prescription quality and reduce medication errors. However, the low methodological quality of the included reviews indicates the need for the standardization of studies and adequate descriptions of the components of each CDSS in order to stratify the determinant characteristics of the success of these systems in their process outcomes.
Footnotes
CONFLICT OF INTEREST
The authors of the article declare that they do not have conflicts of interest.
FUNDING
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes), Brazil.
Contributor Information
Wálleri C. Reis, Department of Pharmacy, Federal University of Paraiba, João Pessoa (Brazil). wallerictr@gmail.com
Aline F. Bonetti, Postgraduate Program in Pharmaceutical Sciences, Federal University of Parana. Curitiba (Brazil). alinefbonetti@gmail.com
Wallace E. Bottacin, Postgraduate Program in Pharmaceutical Sciences, Federal University of Parana. Curitiba (Brazil). wallace.bottacin@gmail.com
Alcindo S. Reis, Jr, Specialist-Postgraduate Program in Pharmaceutical Sciences, Federal University of Parana. Curitiba (Brazil). alcindosrjunior@hotmail.com.
Thaís T. Souza, Postgraduate Program in Pharmaceutical Sciences, Federal University of Parana, Curitiba (Brazil). thaisteles3@gmail.com
Roberto Pontarolo, Professor, Postgraduate Program in Pharmaceutical Sciences, Department of Pharmacy, Federal University of Parana. Curitiba (Brazil). pontarolo@ufpr.br.
Cassyano J. Correr, PhD – Professor, Postgraduate Program in Pharmaceutical Sciences, Department of Pharmacy, Federal University of Parana. Curitiba (Brazil). cassyano@ufpr.br
Fernando Fernandez-Llimos, Institute for Medicines Research (iMed.ULisboa), Department of Social Pharmacy, Faculty of Pharmacy, University of Lisbon. Lisbon (Portugal). f-llimos@ff.ul.pt.
References
- 1.Beeler PE, Bates DW, Hug BL. Clinical decision support systems. Swiss Med Wkly. 2014;144:w14073. doi: 10.4414/smw.2014.14073. [DOI] [PubMed] [Google Scholar]
- 2.Bell DS, Cretin S, Marken RS, Landman AB. A conceptual framework for evaluating outpatient electronic prescribing systems based on their functional capabilities. J Am Med Inform Assoc. 2004;11(1):60–70. doi: 10.1197/jamia.M1374. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Vogenberg FR, Benjamin D. The medication-use process and the importance of mastering fundamentals. P T. 2011;36(10):651–652. [PMC free article] [PubMed] [Google Scholar]
- 4.McKibbon KA, Lokker C, Handler SM, Dolovich LR, Holbrook AM, O'Reilly D, Tamblyn R, Hemens BJ, Basu R, Troyan S, Roshanov PS. The effectiveness of integrated health information technologies across the phases of medication management:a systematic review of randomized controlled trials. J Am Med Inform Assoc. 2012;19(1):22–30. doi: 10.1136/amiajnl-2011-000304. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, Federico F, Goldmann DA. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;285(16):2114–2120. doi: 10.1001/jama.285.16.2114. [DOI] [PubMed] [Google Scholar]
- 6.Takata GS, Mason W, Taketomo C, Logsdon T, Sharek PJ. Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children's hospitals. Pediatrics. 2008;121(4):e927–e935. doi: 10.1542/peds.2007-1779. [DOI] [PubMed] [Google Scholar]
- 7.Miller M.R, Clark J. S, Lehmann C. U. Computer based medication error reporting:insights and implications. Qual Saf Health Care. 2006;15(3):208–213. doi: 10.1136/qshc.2005.016733. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D, Laffel G, Sweitzer BJ, Shea BF, Hallisey R. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA. 1995;274(1):29–34. [PubMed] [Google Scholar]
- 9.Leape LL, Bates DW, Cullen DJ, Cooper J, Demonaco HJ, Gallivan T, Hallisey R, Ives J, Laird N, Laffel G. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995;274(1):35–43. [PubMed] [Google Scholar]
- 10.Catwell L, Sheikh A. Evaluating eHealth Interventions:The Need for Continuous Systemic Evaluation. PLoS Med. 2009;6(8):e1000126. doi: 10.1371/journal.pmed.1000126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.NIH and the American Recovery & Reinvestment Act (ARRA) [acessed 25 May 2017]; Available at: https://recovery.nih.gov/
- 12.Osheroff JA. [accessed 25 May 2017];Improving medication use and outcomes with clinical decision support:a step-by-step guide. Available at: http://www.himss.org/improving-medication-use-and-outcomes-clinical-decision-support-step-step-guide .
- 13.Forni A, Chu HT, Fanikos J. Technology utilization to prevent medication errors. Curr Drug Saf. 2010;5(1):13–18. doi: 10.2174/157488610789869193. [DOI] [PubMed] [Google Scholar]
- 14.Agrawal A. Medication errors:prevention using information technology systems. Br J Clin Pharmacol. 2009;67(6):681–686. doi: 10.1111/j.1365-2125.2009.03427.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Donabedian A. The role of outcomes in quality assessment and assurance. QRB Qual Rev Bull. 1992;18(11):356–360. doi: 10.1016/s0097-5990(16)30560-7. [DOI] [PubMed] [Google Scholar]
- 16.Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015:elaboration and explanation. BMJ. 2015;350:g7647. doi: 10.1136/bmj.g7647. [DOI] [PubMed] [Google Scholar]
- 17.Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. doi: 10.1186/2046-4053-4-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.The Cochrane Collaboration. [accessed 25 May 2017];Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. Available at: http://training.cochrane.org/handbook .
- 19.Kung J, Chiappelli F, Cajulis OO, Avezova R, Kossan G, Chew L, Maida CA. From systematic reviews to clinical recommendations for evidence-based health care:validation of revised assessment of multiple systematic reviews (R-AMSTAR) for grading of clinical relevance. Open Dent J. 2010;4:84–91. doi: 10.2174/1874210601004020084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Wong K, Yu SK, Holbrook A. A systematic review of medication safety outcomes related to drug interaction software. J Popul Ther Clin Pharmacol. 2010;17(2):e243–e255. [PubMed] [Google Scholar]
- 21.Welch BM, Kawamoto K. Clinical decision support for genetically guided personalized medicine:a systematic review. J Am Med Inform Assoc. 2013;20(2):388–400. doi: 10.1136/amiajnl-2012-000892. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009;(3):CD001096. doi: 10.1002/14651858.CD001096.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Shebl NA, Franklin BD, Barber N. Clinical decision support systems and antibiotic use. Pharm World Sci. 2007 Aug;29(4):342–349. doi: 10.1007/s11096-007-9113-3. [DOI] [PubMed] [Google Scholar]
- 24.O'Reilly D, Tarride JE, Goeree R, Lokker C, McKibbon KA. The economics of health information technology in medication management:a systematic review of economic evaluations. J Am Med Inform Assoc. 2012;19(3):423–438. doi: 10.1136/amiajnl-2011-000310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Nieuwlaat R, Connolly SJ, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB CCDSS Systematic Review Team. Computerized clinical decision support systems for therapeutic drug monitoring and dosing:a decision-maker-researcher partnership systematic review. Implement Sci. 2011;6:90. doi: 10.1186/1748-5908-6-90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Hemens BJ, Holbrook A, Tonkin M, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB CCDSS Systematic Review Team. Computerized clinical decision support systems for drug prescribing and management:a decision-maker-researcher partnership systematic review. Implement Sci. 2011;6:89. doi: 10.1186/1748-5908-6-89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Vedel I, Akhlaghpour S, Vaghefi I, Bergman H, Lapointe L. Health information technologies in geriatrics and gerontology:a mixed systematic review. J Am Med Inform Assoc. 2013;20(6):1109–1119. doi: 10.1136/amiajnl-2013-001705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J. Effect of point-of-care computer reminders on physician behaviour:a systematic review. CMAJ. 2010;182(5):E216–E225. doi: 10.1503/cmaj.090578. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Schedlbauer A, Prasad V, Mulvaney C, Phansalkar S, Stanton W, Bates DW, Avery AJ. What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior? J Am Med Inform Assoc. 2009;16(4):531–538. doi: 10.1197/jamia.M2910. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Roshanov PS, You JJ, Dhaliwal J, Koff D, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB CCDSS Systematic Review Team. Can computerized clinical decision support systems improve practitioners'diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implement Sci. 2011;6:88. doi: 10.1186/1748-5908-6-88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Roshanov PS, Misra S, Gerstein HC, Garg AX, Sebaldt RJ, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB CCDSS Systematic Review Team. Computerized clinical decision support systems for chronic disease management:a decision-maker-researcher partnership systematic review. Implement Sci. 2011;6:92. doi: 10.1186/1748-5908-6-92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Oluoch T, Santas X, Kwaro D, Were M, Biondich P, Bailey C, Abu-Hanna A, de Keizer N. The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings:a systematic review. Int J Med Inform. 2012;81(10):e83–e92. doi: 10.1016/j.ijmedinf.2012.07.010. [DOI] [PubMed] [Google Scholar]
- 33.Njie GJ, Proia KK, Thota AB, Finnie RKC, Hopkins DP, Banks SM, Callahan DB, Pronk NP, Rask KJ, Lackland DT, Kottke TE Community Preventive Services Task Force. Clinical decision support systems and prevention:a community guide cardiovascular disease systematic review. Am J Prev Med. 2015;49(5):784–795. doi: 10.1016/j.amepre.2015.04.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Millery M, Kukafka R. Health information technology and quality of health care:strategies for reducing disparities in underresourced settings. Med Care Res Rev. 2010;67(5 Suppl):268S–298S. doi: 10.1177/1077558710373769. [DOI] [PubMed] [Google Scholar]
- 35.Lobach D, Sanders GD, Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux R, Samsa G, Hasselblad V, Williams JW, Wing L, Musty M, Kendrick AS. Enabling health care decisionmaking through clinical decision support and knowledge management. Evid Rep Technol Assess (Full Rep) 2012;(203):1–784. [PMC free article] [PubMed] [Google Scholar]
- 36.Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care:a systematic review. HIM J. 2009;38(3):26–37. doi: 10.1177/183335830903800305. [DOI] [PubMed] [Google Scholar]
- 37.Gillaizeau F, Chan E, Trinquart L, Colombet I, Walton RT, Rège-Walther M, Burnand B, Durieux P. Computerized advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev. 2013;(11):CD002894. doi: 10.1002/14651858.CD002894.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Durieux P, Trinquart L, Colombet I, Niès J, Walton R, Rajeswaran A, Rège Walther M, Harvey E, Burnand B. Computerized advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev. 2008;(3):CD002894. doi: 10.1002/14651858.CD002894.pub2. [DOI] [PubMed] [Google Scholar]
- 39.Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG. Systematic review:impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144(10):742–752. doi: 10.7326/0003-4819-144-10-200605160-00125. [DOI] [PubMed] [Google Scholar]
- 40.Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D. Effect of Clinical Decision-Support Systems. Ann Intern Med. 2012;157(1):29–43. doi: 10.7326/0003-4819-157-1-201207030-00450. [DOI] [PubMed] [Google Scholar]
- 41.Tawadrous D, Shariff SZ, Haynes RB, Iansavichus AV, Jain AK, Garg AX. Use of clinical decision support systems for kidney-related drug prescribing:a systematic review. Am J Kidney Dis. 2011;58(6):903–914. doi: 10.1053/j.ajkd.2011.07.022. [DOI] [PubMed] [Google Scholar]
- 42.Yourman L, Concato J, Agostini JV. Use of computer decision support interventions to improve medication prescribing in older adults:a systematic review. Am J Geriatr Pharmacother. 2008;6(2):119–129. doi: 10.1016/j.amjopharm.2008.06.001. [DOI] [PubMed] [Google Scholar]
- 43.Fathima M, Peiris D, Naik-Panvelkar P, Saini B, Armour CL. Effectiveness of computerized clinical decision support systems for asthma and chronic obstructive pulmonary disease in primary care:a systematic review. BMC Pulm Med. 2014;14:189. doi: 10.1186/1471-2466-14-189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Souza NM, Sebaldt RJ, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB CCDSS Systematic Review Team. Computerized clinical decision support systems for primary preventive care:a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011;6:87. doi: 10.1186/1748-5908-6-87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Sintchenko V, Magrabi F, Tipper S. Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes:a systematic review. Med Inform Internet Med. 2007;32(3):225–240. doi: 10.1080/14639230701447701. [DOI] [PubMed] [Google Scholar]
- 46.Sahota N, Lloyd R, Ramakrishna A, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB CCDSS Systematic Review Team. Computerized clinical decision support systems for acute care management:a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011;6:91. doi: 10.1186/1748-5908-6-91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Robertson J, Walkom E, Pearson SA, Hains I, Williamsone M, Newby D. The impact of pharmacy computerised clinical decision support on prescribing, clinical and patient outcomes:a systematic review of the literature. Int J Pharm Pract. 2010;18(2):69–87. [PubMed] [Google Scholar]
- 48.Pearson SA, Moxey A, Robertson J, Hains I, Williamson M, Reeve J, Newby D. Do computerised clinical decision support systems for prescribing change practice? A systematic review of the literature (1990-2007) BMC Health Serv Res. 2009;9:154. doi: 10.1186/1472-6963-9-154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Montgomery AA, Fahey T. A systematic review of the use of computers in the management of hypertension. J Epidemiol Community Health. 1998;52(8):520–525. doi: 10.1136/jech.52.8.520. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Marasinghe KM. Computerised clinical decision support systems to improve medication safety in long-term care homes:a systematic review. BMJ Open. 2015;5(5):e006539. doi: 10.1136/bmjopen-2014-006539. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Lainer M, Mann E, Sönnichsen A. Information technology interventions to improve medication safety in primary care:a systematic review. Int J Qual Health Care. 2013;25(5):590–598. doi: 10.1093/intqhc/mzt043. [DOI] [PubMed] [Google Scholar]
- 52.Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes:a systematic review. JAMA. 1998;280(15):1339–1346. doi: 10.1001/jama.280.15.1339. [DOI] [PubMed] [Google Scholar]
- 53.Holstiege J, Mathes T, Pieper D. Effects of computer-aided clinical decision support systems in improving antibiotic prescribing by primary care providers:a systematic review. J Am Med Inform Assoc. 2015;22(1):236–242. doi: 10.1136/amiajnl-2014-002886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Georgiou A, Prgomet M, Paoloni R, Creswick N, Hordern A, Walter S, Westbrook J. The effect of computerized provider order entry systems on clinical care and work processes in emergency departments:a systematic review of the quantitative literature. Ann Emerg Med. 2013;61(6):644–653. doi: 10.1016/j.annemergmed.2013.01.028. [DOI] [PubMed] [Google Scholar]
- 55.Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes:a systematic review. JAMA. 2005;293(10):1223–1238. doi: 10.1001/jama.293.10.1223. [DOI] [PubMed] [Google Scholar]
- 56.Fraccaro P, Arguello Casteleiro M, Ainsworth J, Buchan I. Adoption of Clinical Decision Support in Multimorbidity:A Systematic Review. JMIR Med Inform. 2015;3(1):e4. doi: 10.2196/medinform.3503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Stürzlinger H, Hiebinger C, Pertl D, Traurig P. Computerized Physician Order Entry - effectiveness and efficiency of electronic medication ordering with decision support systems. GMS Health Technol Assess. 2009;5:Doc07. doi: 10.3205/hta000069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Randell R, Mitchell N, Dowding D, Cullum N, Thompson C. Effects of computerized decision support systems on nursing performance and patient outcomes:a systematic review. J Health Serv Res Policy. 2007;12(4):242–249. doi: 10.1258/135581907782101543. [DOI] [PubMed] [Google Scholar]
- 59.Marc DT, Khairat SS. Why do physicians have difficulty accepting clinical decision support systems? Stud Health Technol Inform. 2013;192:1202. [PubMed] [Google Scholar]
- 60.Heselmans A, Van de Velde S, Donceel P, Aertgeerts B, Ramaekers D. Effectiveness of electronic guideline-based implementation systems in ambulatory care settings - a systematic review. Implement Sci. 2009;4:82. doi: 10.1186/1748-5908-4-82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Eslami S, Abu-Hanna A, de Keizer NF. Evaluation of outpatient computerized physician medication order entry systems:a systematic review. J Am Med Inform Assoc. 2007;14(4):400–406. doi: 10.1197/jamia.M2238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Carling CL, Kirkehei I, Dalsbø TK, Paulsen E. Risks to patient safety associated with implementation of electronic applications for medication management in ambulatory care--a systematic review. BMC Med Inform Decis Mak. 2013;13:133. doi: 10.1186/1472-6947-13-133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Jeffery R, Iserman E, Haynes RB CDSS Systematic Review Team. Can computerized clinical decision support systems improve diabetes management? A systematic review and meta-analysis. Diabet Med. 2013;30(6):739–745. doi: 10.1111/dme.12087. [DOI] [PubMed] [Google Scholar]
- 64.Lau F, Kuziemsky C, Price M, Gardner J. A review on systematic reviews of health information system studies. J Am Med Inform Assoc. 2010;17(6):637–645. doi: 10.1136/jamia.2010.004838. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Mostofian F, Ruban C, Simunovic N, Bhandari M. Changing physician behavior:what works? Am J Manag Care. 2015;21(1):75–84. [PubMed] [Google Scholar]
- 66.Cresswell K, Majeed A, Bates DW, Sheikh A. Computerised decision support systems for healthcare professionals:an interpretative review. Inform Prim Care. 2012;20(2):115–128. doi: 10.14236/jhi.v20i2.32. [DOI] [PubMed] [Google Scholar]
- 67.Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, McKinstry B, Procter R, Majeed A, Sheikh A. The impact of ehealth on the quality and safety of health care:a systematic overview. PLoS Med. 2011;8(1):e1000387. doi: 10.1371/journal.pmed.1000387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Kilsdonk E, Peute LW, Knijnenburg SL, Jaspers MW. Factors known to influence acceptance of clinical decision support systems. Stud Health Technol Inform. 2011;169:150–154. [PubMed] [Google Scholar]
- 69.Jia P, Zhang L, Chen J, Zhao P, Zhang M. The Effects of Clinical Decision Support Systems on Medication Safety:An Overview. PLoS One. 2016;11(12):e0167683. doi: 10.1371/journal.pone.0167683. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes:a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011;18(3):327–334. doi: 10.1136/amiajnl-2011-000094. [DOI] [PMC free article] [PubMed] [Google Scholar]