The GUIDE-CDSS framework: a comprehensive framework to GUIDE and evaluate the implementation of Clinical Decision Support Systems in healthcare, based on an overview of reviews

Esther R C Janssen; Jesper Knoop; Gianmatteo Farabolini; Nicoló Baldini; Thomas Hoogeboom; Maria Gabriella Ceravolo; Stefano Negrini; Carlotte Kiekens; PREPARE Project Group

doi:10.1093/jamia/ocag080

. 2026 Jun 12;33(9):1811–1821. doi: 10.1093/jamia/ocag080

The GUIDE-CDSS framework: a comprehensive framework to GUIDE and evaluate the implementation of Clinical Decision Support Systems in healthcare, based on an overview of reviews

Esther R C Janssen ^1,^2,^3,^✉, Jesper Knoop ^4,⁵, Gianmatteo Farabolini ⁶, Nicoló Baldini ⁷, Thomas Hoogeboom ^8,⁹, Maria Gabriella Ceravolo ¹⁰, Stefano Negrini ^11,¹², Carlotte Kiekens ¹³; PREPARE Project Group

¹ Radboud Institute for Health Sciences, IQ Health, Radboud University Medical Centre, Nijmegen, 5600HB, The Netherlands

² Musculoskeletal Rehabilitation Research Group, School of Allied Health, HAN University of Applied Sciences, Arnhem/Nijmegen, 6802 EE, The Netherlands

³ Department of Orthopedic Surgery, VieCuri Medical Centre, Venlo, 5912BL, The Netherlands

⁴ Musculoskeletal Rehabilitation Research Group, School of Allied Health, HAN University of Applied Sciences, Arnhem/Nijmegen, 6802 EE, The Netherlands

⁵ Department of Movement Sciences, Faculty of Behavioural and Movement Sciences and Amsterdam Movement Science Research Institute, Vrije Universiteit, Amsterdam, 1081 HV, The Netherlands

⁶ Department of Experimental and Clinical Medicine, Marche Polytechnic University, Ancona, 60121, Italy

⁷ Department of Experimental and Clinical Medicine, Marche Polytechnic University, Ancona, 60121, Italy

⁸ Department of Rehabilitation, Physiotherapy Sciences and Sport, Brain Center, University Medical Center Utrecht, Utrecht, 3508 GA, The Netherlands

⁹ Research Group Innovation of Human Movement Care, HU University of Applied Sciences Utrecht, Utrecht, 3508 AJ, The Netherlands

¹⁰ Department of Experimental and Clinical Medicine, Marche Polytechnic University, Ancona, 60121, Italy

¹¹ Laboratory of Evidence-Based Rehabilitation, IRCCS Galeazzi—Sant’Ambrogio Hospital, Milan, 20157, Italy

¹² Department of Biomedical, Surgical and Dental Sciences, University of Milan “La Statale”, Milan, 20122, Italy

¹³ Laboratory of Evidence-Based Rehabilitation, IRCCS Galeazzi—Sant’Ambrogio Hospital, Milan, 20157, Italy

^✉

Corresponding author: Esther R.C. Janssen, PhD, Department of IQ Health, Radboudumc, PO Box 9101 (160), 6500 HB Nijmegen, The Netherlands (esther.rc.janssen@radboudumc.nl)

Roles

Esther R C Janssen: PhD, Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Visualization, Writing - original draft, Writing - review & editing

Jesper Knoop: PhD, Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing - review & editing

Gianmatteo Farabolini: PhD, Data curation, Formal analysis, Investigation, Methodology, Writing - review & editing

Nicoló Baldini: MD, Data curation, Formal analysis, Investigation, Methodology, Writing - review & editing

Thomas Hoogeboom: PhD, Conceptualization, Funding acquisition, Methodology, Supervision, Writing - review & editing

Maria Gabriella Ceravolo: MD, PhD, Conceptualization, Funding acquisition, Methodology, Supervision, Writing - review & editing

Stefano Negrini: MD, PhD, Conceptualization, Funding acquisition, Methodology, Supervision, Validation, Writing - review & editing

Carlotte Kiekens: MD, Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Supervision, Writing - review & editing

PMCID: PMC13580735 PMID: 42281367

Abstract

Objectives

Despite the significant potential for Clinical Decision Support Systems (CDSSs) to improve care processes and health outcomes, several barriers hinder their widespread implementation in healthcare. While numerous systematic reviews have summarized potential barriers and facilitators for CDSS implementation, a comprehensive framework to guide and evaluate the implementation of CDSSs in healthcare is lacking. This overview of reviews, aims to establish a framework—GUIDE-CDSS—aimed at guiding and evaluating implementation of CDSSs in healthcare.

Materials and Methods

An overview of systematic and scoping reviews was conducted by searching 6 databases. Systematic reviews or scoping reviews that used qualitative research methods to described implementation determinants for CDSSs in the healthcare domain were included. The AMSTAR 2 tool was used to assess the methodological quality. Results were collated into the GUIDE-CDSS framework. This framework describes implementation determinants and elements within those determinants found to impact implementation of CDSSs in healthcare.

Results

Twenty-three reviews were included in the analysis. All reviews had at least 2 critical weaknesses, showing a limited methodological quality of included reviews. Eight determinants and 38 elements for implementation of CDSSs in healthcare and were described in the GUIDE-CDSS framework: perceived relevance, perceived effect, trustworthiness, ease of use, workflow, training and skills, resources, and implementation strategy.

Discussion and Conclusion

This overview provides a comprehensive synthesis of the determinants influencing the implementation of CDSSs in healthcare, collated in the GUIDE-CDSS framework. The findings underscore that for successful CDSS development, implementation and evaluation is multifactorial.

Protocol registration

This study was registered in PROSPERO (No. CRD42024512455).

Keywords: qualitative research, prediction algorithms, precision medicine, artificial intelligence, implementation science, decision support systems

Introduction

To maintain accessible, affordable, and high-quality healthcare, a transition is needed from a population-based, disease-specific approach to a person-centered care that can better address the diverse health needs of a growing global population.¹ Person-centered care focuses on the ideal of whole-person care and shared decision making.² This shift from a reliance on population-level evidence (ie, group averages) and generalized guidelines toward person centered care is ongoing and rapidly accelerating due to a number of factors: (1) the inherently person-centered nature of healthcare, which by definition requires a personalized approach,³ (2) the expanding range of diagnostic and treatment options,⁴^,⁵ (3) the increasing demand for healthcare services amid limited resources,⁶ (4) the growing availability of large-scale, global datasets that remain largely untapped,⁷ (5) rapid development of evidence in healthcare.⁸ As a result Clinical Decision Support Systems (CDSSs) have emerged as critical tools for personalizing healthcare and enhancing clinical decision-making.⁹ These systems are increasingly used at the point of care, supporting individualized diagnostic, prognostic, and treatment decisions.¹⁰ CDSSs leverage advanced data-driven models to make patient-specific inferences based on population data. Early examples of CDSSs, such as the STarT Back Screening Tool¹¹ and the OCTOPUS stratification tool,¹² facilitate stratified healthcare by classifying patients into clinically meaningful subgroups. While these approaches have shown some benefit in improving health outcomes, the field is now shifting toward truly personalized healthcare.³ In this next phase, advanced, AI-driven CDSSs offer considerable promise. By applying machine learning techniques, AI-based systems can generate accurate, individualized insights, providing clinicians and patients with the information necessary to make well-informed treatment decisions.¹³ For example, a CDSS could predict the likely outcomes of various treatment options for a specific patient by analyzing data from similar cases, thereby guiding clinicians and patients toward the most effective treatment plan.

Despite the significant potential for CDSSs to improve care processes and health outcomes, several barriers hinder their widespread implementation. Foremost among these challenges is the limited availability of robust, validated prediction models,¹⁴ which is partly due to the absence of standardized methods for collecting critical data on factors influencing healthcare outcomes, such as personal characteristics (eg, resilience, health literacy, motivation) and socioeconomic variables.⁷ Although multiple frameworks address aspects of technology adoption and implementation, no single framework fully captures the range of determinants specific to CDSS implementation in healthcare, often necessitating the combined use or adaptation of existing models.^15–19 This framework is necessary not only to plan for successful implementation but also to evaluate its effectiveness across diverse healthcare settings. Without such a framework, we cannot systematically identify and address the key determinants—such as organizational culture, technological infrastructure, and clinician engagement—that are crucial for enhancing the successful uptake of CDSSs.

The findings from this study will inform the development, implementation, and evaluation of an advanced Clinical Decision Support System (CDSS) within the EU Horizon-funded PREPARE project, showcasing the value of the proposed framework. PREPARE is a multinational project focused on creating data-driven tools for personalized, holistic rehabilitation and care. To support this aim, we conducted an overview of existing reviews to inform the development, implementation, and evaluation of the CDSS tools.

This overview of systematic and scoping reviews describes the current qualitative evidence on the implementation determinants of CDSSs in healthcare settings and collates them into the GUIDE-CDSS framework.

Methods

An overview of systematic and scoping reviews was performed to summarize published literature related to the implementation of CDSSs in healthcare. This study was registered in PROSPERO (No. CRD42024512455) and reported according to the PRISMA checklist and preferred reporting items for overviews of reviews (PRIOR).²⁰^,²¹ In this overview of reviews, we analyzed the qualitative results of the registered PROSPERO protocol. This decision was driven by the large number of eligible reviews and by the distinct analytical requirements of qualitative versus quantitative outcomes The quantitative results will be reported elsewhere.

Literature search

A systematic search of 6 databases was conducted [MEDLINE (through PubMed), Cochrane, Embase, IEEE Xplore, ACM Digital Library, Episthemonikos] from January 1, 2014 until December 5, 2023. The limited timeframe was chosen to only include state-of-the-art evidence. Reference lists of included reviews were hand-searched for other reviews. Key search terms were related to “clinical decision support” and “systematic review.” The used search string for each database can be found in Appendix S1.

Study selection

Studies were included if they were systematic reviews, meta-analyses or scoping reviews that used individual patient data and some type of statistical model to support clinical decision making (eg, patient profiling, risk assessment, diagnostic support, disease progression modeling, personalized treatment recommendations, outcome prediction, therapy response monitoring, or patient monitoring) within the healthcare context. We excluded rule-based systems, such as alarm systems, computerized reminders, and guideline-based process support tools. While these systems are often categorized under the term CDSS, they primarily provide standardized protocol support rather than enabling data-driven personalization of care, which was the central focus of this overview. Furthermore, studies needed to at least assess qualitative implementation outcomes of implementation of CDSSs. Exclusion criteria were: (1) if the functions of the included CDSS were patient safety monitoring, administrative functions or process improvement (eg, alarm systems, computerized reminders, computerized guidelines, patient data reports, documentation templates, clinical workflow tools), (2) non-English articles, and/or (3) reviews of model-based cost evaluations. Non-English articles were excluded to ensure the overview reflects the state-of-the-art applicable to an international audience. Given that non-English publications often report findings specific to a particular country or region, they were deemed less relevant for the purposes of this review.

Two pairs of reviewers (N.B. and J.K., E.R.C.J. and G.F.) independently conducted the study selection. Each pair screened half of the found studies, using the Rayyan software.²² First, title and abstract were screened for eligibility. Second, the full texts of the eligible articles were retrieved and reviewed. Discrepancies in all phases were solved during a consensus meeting with all 4 reviewers, and when necessary, an expert reviewer (C.K.) resolved final conflicts. If no full text was available a maximum of 3 attempts was made to contact the corresponding author. If the full text could not be retrieved, the study was excluded.

Risk of bias

Four individual reviewers (E.R.C.J., G.F., N.B., and J.K.) applied the AMSTAR 2 tool to assess the methodological quality of the included reviews.²³ The AMSTAR 2 is not intended to generate an overall score, but rather to assist in the identification of high-quality systematic reviews. Through the appraisal of the 16 AMSTAR 2 items, critical and non-critical weaknesses were identified in the context of our overview. Risk of Bias (RoB) assessments of included reviews were assessed to identify methodological weaknesses across reviews. The reviewers then judged the overall confidence in the results of the included reviews, based on combined Risk of Bias assessments and AMSTAR 2 results were possible.²⁴ Disagreement between the reviewers was resolved through a consensus meeting with an expert reviewer (C.K.).

Data extraction

Four reviewers (N.B., G.F., J.K., and E.R.C.J.) independently extracted data using Castor (version 2025.4.0; Castor, Amsterdam, the Netherlands). N.B. and G.F. cross-checked each other’s extractions, as did J.K. and E.R.C.J. The following data on included reviews were extracted: author(s), year of review, aim/scope of review, in- and exclusion criteria, time frame of inclusions, method of review, number of included papers, types of papers included, risk of bias assessment, major findings, recommendations/critical appraisal (eg, level of evidence table/GRADE table). Several dimensions of individual CDSS design features were recorded: type of CDSS, patient population, clinical task addressed, geographical coverage. Barriers and facilitators of CDSS implementation were extracted and tabulated using Excel (version 2408, Microsoft, Redmond, United States) as free text, to reflect implementation determinants.

Synthesis of results

Descriptive statistics were reported for study characteristics and CDSS characteristics. A narrative review was conducted for all implementation outcomes using the SWiM guideline.²⁵ Both the ENTREQ and eMERGe guidelines were considered not applicable as they guide reviews of primary studies. Qualitative data were described as implementation determinants when applicable and synthesized for similar outcomes into overarching implementation determinants and associated elements to improve their coherence. This grouping was performed using a thematic analysis approach, in which we drew initial elements from barriers and facilitators with similarities in semantics and meaning, reported by the reviews (source data). Thereafter, we identified broader implementation determinants reflecting meaningful concepts for implementing a CDSS in practice. All authors reviewed and refined these implementation determinants, during consensus rounds. Determinants were established through discussion with the involved authors until consensus was reached. We did not analyze if there was overlap of primary studies between the included reviews, as this would not influence the outcome of this overview. If overlap between identified barriers and facilitators was found they were comprised into a single element (see Appendix S2).

Results

Characteristics of included reviews

Our search yielded 11 310 papers of which 13 reviews reporting mixed (both quantitative and qualitative) outcomes and 10 reviews reporting qualitative outcomes were eligible, see Figure 1. They included results from 615 primary studies from diverse medical fields. Included reviews reported on studies from Europe (n = 17), North America (n = 17), South America (N = 1), Africa (n = 4), Asia (N = 12) and Oceania (N = 11). We found 5 reviews from the field of general medicine, 3 from nursing, 3 from oncology, 2 from cardiology, 2 from genetics, 2 from infectious diseases, 1 from allied healthcare, 1 from emergency medicine, 1 from hospital care, 1 from internal medicine, 1 on COVID-19, and 1 with a generic hospital perspective. Details on the study characteristics can be found in Table S1. The authors described the use of CDSS in their respective fields in a qualitative manner.

Flowchart of inclusion and exclusion process of the overview of reviews. — PRISMA flowchart.

Assessment of quality

We applied the Amstar 2 to assess the methodological quality of the reviews, see Table S2. The domains on the appropriateness of meta-analysis, RoB on meta-analysis results and assessment of publication bias were not scored for any of the included reviews. As such 2 of the 7 critical domains were not assessed in this overview of reviews. This should be considered when interpreting the overall confidence in the results of our overview. There were no reviews without less than 2 critical weaknesses, there were 2 studies with 2 major weaknesses, 5 with 3 major weaknesses, 4 with 4 major weaknesses and 12 with 5 major weaknesses. Included reviews used varying tools, criteria, or thresholds to judge the RoB in primary studies, as such no synthesis of the risk of bias of primary studies or overall level of confidence assessment was performed.

Implementation determinants

Our synthesis of the reviews identified 8 implementation determinants (ie, barriers and/or facilitators for implementation) from 38 elements. The elements within the implementation determinants describe the prerequisites for implementing CDSS in healthcare practice with more granularity. The implementation determinants and their associated elements are described in Table 1.

Table 1.

Implementation determinants and elements

Implementation determinant	N of reviews	Elements	N of reviews
Perceived relevance	14	Relevance of the outcome of the CDSS	9^26–28^,^30–32^,^35–37
		Relevance within the context	8^26–29^,³²^,³⁴^,³⁶
		Personalization within the context	7²⁶^,²⁸^,²⁹^,³¹^,³³^,³⁸^,³⁹
		Timeliness of information	3²⁸^,²⁹^,³⁴
		Alignment with organizational goals	2²⁶^,²⁸
		Accuracy of the content	2²⁷^,²⁹
Perceived effect	11	Influence on treatment outcome	9²⁶^,²⁹^,³⁰^,³²^,³³^,³⁶^,³⁸^,⁴⁰^,⁴²
		Influence on communication	8²⁶^,²⁸^,²⁹^,³²^,³⁶^,³⁸^,⁴¹^,⁴²
		Influence on care process	8²⁶^,²⁹^,³⁰^,³²^,³³^,³⁸^,⁴⁰^,⁴³
		Influence on patient and professional satisfaction	2²⁸^,³²
Trustworthiness	16	Attitude toward CDSS	11²⁶^,²⁸^,^31–34^,³⁸^,⁴⁰^,^42–44
		Respect for user autonomy	7²⁶^,²⁸^,³²^,³⁶^,³⁸^,⁴¹^,⁴²
		Organizational culture	7²⁶^,²⁸^,²⁹^,³²^,³³^,³⁹^,⁴⁴
		Reliability of information	5²⁶^,²⁸^,³⁰^,³²^,³⁶
		Medicolegal compliance	3²⁶^,²⁸^,⁴¹
		Performance of underlying model	1²⁸
Ease of use	15	Integration within current systems	11^26–29^,³¹^,³²^,³⁴^,³⁶^,³⁹^,⁴¹^,⁴³
		Available functionalities	11^26–29^,^31–33^,³⁶^,³⁸^,⁴⁴^,⁴⁵
		CDSS fit for purpose	9^26–29^,³²^,³³^,³⁶^,³⁹^,⁴⁵
		Technical compatibility	6³⁰^,^32–34^,³⁶^,³⁹
		Look and feel of interface	6²⁹^,³²^,³⁶^,³⁹^,⁴⁴^,⁴⁵
		Standardization of data	2²⁶^,³⁹
Workflow	17	Integration within usual workflow	13^26–32^,³⁴^,³⁶^,³⁹^,⁴³^,⁴⁴
		Time and effort to use CDSS	10^26–29^,^31–33^,³⁶^,⁴¹^,⁴³
		Appropriateness of alerts	8²⁶^,²⁷^,²⁹^,³¹^,³²^,³⁵^,³⁹^,⁴¹
		Providing structure and guidance	4²⁸^,²⁹^,³²^,⁴¹
Training and skills	14	Training and education	12^26–33^,³⁸^,⁴¹^,⁴⁴^,⁴⁵
		Knowledge of CDSS	9²⁶^,²⁷^,^29–31^,³³^,³⁴^,³⁹^,⁴¹
		Supports knowledge	5²⁶^,²⁸^,²⁹^,³²^,³⁸
		Technical skill level	4²⁶^,³²^,³³
Resources	12	Financial	7²⁶^,²⁹^,³¹^,³²^,^37–39
		Technical	5²⁷^,²⁹^,^32–34
		Staff capacity	5²⁶^,²⁸^,²⁹^,³¹^,⁴¹
		Vendors	1²⁶
Implementation strategy	9	Implementation planning	5²⁶^,²⁸^,²⁹^,³³^,³⁹
		Incentives for use	5²⁸^,³¹^,³²^,³⁴^,³⁶
		Local champions	2²⁶^,³⁹
		Advocated by professional/regulatory body	1²⁸

Open in a new tab

Perceived relevance

Perceived relevance refers to the degree to which the CDSS is perceived to align with the clinical context in which it is (intended to be) implemented.^26–39 The 6 elements of perceived relevance were: relevance of the outcome of the CDSS,^26–29^,³²^,³⁴^,³⁶ relevance within the context,^25–28^,³¹^,³³^,³⁵ personalization within the context,^24–27^,³⁰^,³²^,³⁴ timeliness of information,^23–26^,²⁹^,³¹^,³³ alignment with organizational goals,^22–25^,²⁸^,³⁰^,³² and accuracy of the content.^21–24^,²⁷^,²⁹^,³¹
Perceived effect

Perceived effect refers to the degree to which the CDSS is perceived to have an influence on the outcome or process of care and was described as an implementation determinant by 11 reviews.²⁶^,²⁹^,³⁰^,³²^,³³^,³⁶^,³⁸^,^40–42 The 4 elements of perceived relevance were: influence on treatment outcome, influence on communication, influence on care process, and influence on patient and professional satisfaction.
Trustworthiness

Trustworthiness refers to the qualities of the CDSS that make the user (dis)trust the tool they are using, which was reported as an implementation determinant by 16 reviews.²⁶^,^28–30^,^32–36^,^38–44 The 6 elements of trustworthiness were: attitude toward CDSS, respect for user autonomy, organizational culture, reliability of information, medicolegal compliance, and performance of underlying model.
Ease of use

The ease of use of the CDSS refers to the degree to which the system is being user-friendly in practice and was described as an implementation determinant by 15 reviews.^26–29^,^31–34^,³⁶^,³⁹^,⁴¹^,^43–45 The 6 elements of ease of use were: integration within current systems, available functionalities, CDSS fit for purpose, technical compatibility, look and feel of interface, and standardization of data.
Workflow

Workflow refers to the degree to which the implementation of the CDSS interrupts or facilitates the normal care process and was described as an implementation determinant by 17 reviews.^26–36^,^38–41^,⁴³^,⁴⁴ The 4 elements of workflow were: integration within usual workflow, time and effort to use CDSS, appropriateness of alerts, and providing structure and guidance.
Training and skills

Training and skills refers to the ability, knowledge or training of the user required to use the CDSS and was reported as an implementation determinant by 14 reviews.^26–34^,³⁸^,³⁹^,⁴¹^,⁴⁴^,⁴⁵ The 4 elements of training and skills were: training and education, knowledge of CDSS, supports knowledge, and technical skill level.
Resources

Resources refers to the availability of resources in the implementation setting to be able to implement the CDSS and was reported as an implementation determinant by 12 reviews.^26–29^,^31–34^,^37–39^,⁴¹ Elements of resources were: financial, technical, staff capacity, and vendors.
Implementation strategy

Implementation strategy refers to factors influencing the implementation process of a CDSS in healthcare and was described as an implementation determinant by 9 reviews.²⁶^,²⁸^,²⁹^,^31–34^,³⁶^,³⁹ The 4 elements of implementation were: implementation planning, incentives for use, local champions, and advocated by professional/regulatory body.

Other qualitative results

Three studies provided only narrative results on implementation of CDSSs, they reported their findings in a purely narrative format, without predefined categories or structured lists of determinants.^46–48 Additional interpretative coding was used to identify and classify implementation determinants from these reviews prior to inclusion in the synthesis. They described implementation determinants already described in the synthesis: availability of data in clinical practice, generalizability of the CDSS, alert fatigue and integration of the CDSS with the information system. These can be allocated to the following elements: ease of use, perceived relevance and workflow

GUIDE-CDSS

In Figure 2, we provide the GUIDE-CDSS framework: a comprehensive framework to guide and evaluate the implementation of clinical decision support systems in healthcare. The GUIDE-CDSS framework describes the implementation determinants that contain both barriers and facilitators for CDSS implementation in healthcare. This framework identifies the key challenges that must be addressed during the development phase of a CDSS making sure the tool aligns with all determinants for that specific context. During the implementation phase, these determinants should guide the process of integration into clinical practice to ensure successful adoption and functionality. Furthermore, the framework outlines measurable concepts that can be used to assess the implementation of CDSSs and identify potential factors contributing to unexpected outcomes after implementation.

Discussion

The aim of this overview of reviews was to identify the perceived implementation determinants for implementing data-driven CDSSs in healthcare in general. Of the 7825 eligible reviews we included 23 reviews. The majority of these had critical weaknesses indicating an overall high risk of bias. We found a total of 8 concepts representing implementation determinants (ie, barriers and/or facilitators for implementation) of CDSSs in healthcare comprising 38 elements. These implementation determinants and their elements together from the GUIDE-CDSS framework for guiding and evaluating the implementation of CDSSs in healthcare.

In our overview of reviews we found that many of the reported determinants for implementing CDSSs are inverses of one another. For instance, while limited perceived relevance, lack of training, and insufficient trustworthiness act as barriers, their opposites—perceived relevance, adequate training, and sufficient trustworthiness—emerge as facilitators. Interestingly, many reviews reported either barriers or facilitators, but rarely both, potentially reflecting the focus or orientation of individual studies rather than the full spectrum of implementation dynamics. A more balanced approach that concurrently examines both sides of these constructs, as in this overview, provides a more comprehensive framework into how to optimize CDSS adoption.

The multifactorial challenges of CDSS implementation align closely with established theoretical models of technology adoption and implementation. We have made an overview that compares the implementation determinants of these widely applied frameworks in Appendix S3. Models such as the Unified Theory of Acceptance and Use of Technology (UTAUT),¹⁶ the Technology Acceptance Model (TAM),¹⁷ and the Technology Integration model (TIM)¹⁸ offer valuable perspectives on how users’ perceptions of usefulness, ease of use, and social or organizational influence shape adoption behaviors. These frameworks underscore the critical role of individual-level cognitive and motivational processes in determining acceptance of digital health innovations. However, while such models focus primarily on individual and technological determinants, the broader context of healthcare delivery often involves other factors, such as organizational variables, context of patient care or external factors (eg, policy documents, regulatory affairs), that are not fully captured in these technology acceptance theories.¹⁹ Moreover, these models are only partly applicable to the acceptance of technology in the medical field. To account for this, general implementation frameworks such as Proctor’s Implementation Framework provide complementary perspectives.⁴⁹ This frameworks emphasize the importance of implementation strategies, fidelity, sustainability, and measurable implementation outcomes (eg, acceptability, feasibility, adoption, and penetration) to understand and assess embedding within an organization of care. The Consolidated Framework for Implementation Research (CFIR) provides a comprehensive overview of implementation determinants similar to our framework.¹⁵ Integrating insights from both technology acceptance and implementation science models can yield a more comprehensive understanding of successful CDSS implementation. Specifically, technology-focused models help explain why clinicians may or may not choose to use a support system, while implementation models help explain how and under what conditions these systems can be effectively integrated and sustained within complex healthcare environments. Schoville et al proposed the Integrated Technology Implementation Model (ITIM) in which both perspectives are combined¹⁹ and comprises 12 concepts that are central to the process of technology adoption. Many of these concepts fit our findings, although they miss the granularity described in our elements and therefore lack specific guidance for implementation of CDSSs in healthcare. Our framework bridges the gap between technology adoption models and implementation science frameworks, capturing both user-centered and contextual determinants within a single framework.

Despite the extensive theoretical understanding of technology implementation and adoption provided by these theoretical frameworks, the real-world adoption of CDSSs in the complex setting of healthcare remains limited.⁵⁰ Particularly fields such as rehabilitation, due to the individualized person-centered care processes, could benefit from CDSSs, but these are poorly implemented.⁵¹ In such settings, it is paramount to consider all implementation determinants applicable to the specific context of CDSS implementation in healthcare described in this review to facilitate successful implementation.⁵² This means that a thorough implementation plan, taking into account all the implementation determinants in our framework, is crucial for the development of a CDSS as well as for its implementation into routine care. Within PREPARE-rehab we have used this framework to systematically identify relevant determinants during the development of our CDSS tools and planned pilot testing, supporting a more comprehensive implementation approach. For example, we have addressed perceived relevance issues by co-developing our tools with the intended end-users, so it will fit their clinical scope (eg, outcome selection) and communication process with the patient (eg, visualizations of predictions). Moreover, the implementation determinants from this framework can also be used as concepts for measurement instruments in the evaluation of implementation of CDSS in the healthcare setting. Importantly, rather than assuming equal relevance of all determinants across settings, the framework is intended to support the identification of context-specific priorities. For example, “appropriateness of alerts” may only be applicable for disruptive CDSSs. To operationalize this, we are translating the framework into a questionnaire. This instrument will enable healthcare organizations to identify and prioritize key barriers and facilitators, and to tailor implementation strategies accordingly.

We believe addressing these complex and interrelated challenges requires not only organizational and behavioral change, but as a prerequisite for tackling these more context-related challenges, we need to enable interoperability and facilitate trustworthy data analysis.⁵³ In this regard, ongoing international efforts toward data standardization—such as the European Health Data Space (EHDS), and global collaborative initiatives like OHDSI (Observational Health Data Sciences and Informatics)—represent crucial enablers for the effective implementation of CDSSs and other digital health technologies.⁵⁴ Although the OMOP Common Data Model is often associated with secondary use of data, its standardized vocabularies and data structures provide a semantic foundation that enables consistent interpretation of clinical data across systems, which is essential for scalable and interoperable CDSS logic. Emerging architectures that combine OMOP-standardized data with real-time interoperability standards such as FHIR demonstrate how harmonized data models can support point-of-care applications, effectively bridging the gap between retrospective analysis and real-time clinical decision support.⁵⁵^,⁵⁶ These initiatives aim to harmonize data structures, terminologies, and governance mechanisms across healthcare systems, thereby reducing one of the most persistent technical barriers: the fragmentation and inaccessibility of high-quality clinical data across systems.⁷ By facilitating FAIR data (Findable, Accessible, Interoperable and Reusable) and secure data sharing, these frameworks create the preconditions necessary for higher-level adoption processes to take place.⁵⁷ When data infrastructure is FAIR, healthcare organizations are better positioned to integrate decision support tools into clinical workflows, and clinicians are more likely to trust and engage with these systems.⁵⁸ Thus, such data initiatives not only address the technological prerequisites for CDSS implementation but also act as catalysts for overcoming organizational and individual-level barriers.⁵⁹^,⁶⁰

Strengths and limitations

A key strength of this overview of reviews lies in its comprehensive scope and systematic synthesis of existing evidence on determinants for the implementation of CDSSs across diverse healthcare settings. By collating findings from a wide range of reviews, the study provides a broad understanding of the multidimensional factors influencing implementation, capturing both common and context-specific determinants. However, several limitations should be acknowledged. The inclusion of a large number of reviews necessitated the introduction of a post-hoc exclusion criterion to distinguish between reviews focusing on qualitative versus quantitative outcomes. While this aimed to improve analytical clarity, it may have introduced selective reporting and limited the integration of qualitative and quantitative evidence within the synthesis. Consequently, some of the richness and complementarity between different types of data may not have been fully captured. In addition, a RoB assessment of the primary studies was not performed due to the substantial heterogeneity in methods used for risk assessment across the included reviews.

Conclusion

This overview of reviews provides a comprehensive synthesis of the implementation determinants influencing the implementation of CDSSs across healthcare settings. By identifying 8 overarching implementation determinants and their elements, the review highlights how technological, human, and organizational factors are interconnected. The findings underscore that successful CDSS implementation requires consideration of all implementation determinants, already during the development, but more importantly during implementation and evaluation of the CDSSs in healthcare, to increase their impact.

Supplementary Material

ocag080_Supplementary_Data

ocag080_supplementary_data.zip^{(493.7KB, zip)}

Acknowledgments

The following individuals are members of the PREPARE Project Group. Their contributions to the PREPARE project enabled this work: Aghayev Emin (Eurospine), Alepopoulos Vasilis (Centre for Research and Technology Ellas), Banfi Giuseppe (IRCCS Galeazzi—Sant’Ambrogio Hospital; Vita-Salute San Raffaele University), Calcina Erik (Jožef Stefan Institute), Capecci Marianna (Marche Polytechnic University), Ceravolo Maria Gabriella (Marche Polytechnic University), Cordani Claudio (IRCCS Galeazzi—Sant’Ambrogio Hospital), Ferrario Irene (Italian Scientific Spine Institute (ISICO)), Georgiadis Charalampos (Aristotle University of Thessaloniki), Giannios Georgios (Centre for Research and Technology Ellas), Gramatikopoulou Margarita (Centre for Research and Technology Ellas), Herrmann Christian (Eurospine), Hoogendam Lisa (Erasmus MC—University Medical Center Rotterdam), Hoogeboom Thomas (Radboud University Netherlands), Janssen Esther (Radboud University Netherlands), Khijmatgar Shahnawaz (Università degli Studi di Milano), Kiekens Carlotte (IRCCS Galeazzi—Sant’Ambrogio Hospital), Knoop Jesper (HAN University of Applied Sciences), Kompatsiaris Ioannis (Centre for Research and Technology Ellas), Kotsis Vasilios (Aristotle University of Thessaloniki), Mpaltadros Lampros (Centre for Research and Technology Ellas), Negrini Alberto (Italian Scientific Spine Institute (ISICO)), Negrini Stefano (University of Milan; IRCCS Galeazzi—Sant’Ambrogio Hospital), Nikolopoulos Spiros (Centre for Research and Technology Ellas), Novak Erik (Jožef Stefan Institute), Patias George Robert (KIKLO Geospatial Information Technologies), Patias Petros (Aristotle University of Thessaloniki), Pennestrì Federico (IRCCS Galeazzi—Sant’Ambrogio Hospital), Poli Arianna (Linköping University), Roustanis Themistocles (KIKLO Geospatial Information Technologies), Selles Ruud (Erasmus MC—University Medical Center Rotterdam), Staal Bart (HAN University of Applied Sciences), Tsaopoulos Dimitrios (Centre for Research and Technology Ellas), Tartaglia Gianluca Martino (University of Milan), Zaharatos Haris (Cellock), Helena Burger (University Rehabilitation Institute of Slovenia—Soča), Zala Kuret (University Rehabilitation Institute of Slovenia—Soča), Gaj Vidmar (University Rehabilitation Institute of Slovenia—Soča), Zlatko Matjačić (University Rehabilitation Institute of Slovenia—Soča).

Contributor Information

Esther R C Janssen, Radboud Institute for Health Sciences, IQ Health, Radboud University Medical Centre, Nijmegen, 5600HB, The Netherlands; Musculoskeletal Rehabilitation Research Group, School of Allied Health, HAN University of Applied Sciences, Arnhem/Nijmegen, 6802 EE, The Netherlands; Department of Orthopedic Surgery, VieCuri Medical Centre, Venlo, 5912BL, The Netherlands.

Jesper Knoop, Musculoskeletal Rehabilitation Research Group, School of Allied Health, HAN University of Applied Sciences, Arnhem/Nijmegen, 6802 EE, The Netherlands; Department of Movement Sciences, Faculty of Behavioural and Movement Sciences and Amsterdam Movement Science Research Institute, Vrije Universiteit, Amsterdam, 1081 HV, The Netherlands.

Gianmatteo Farabolini, Department of Experimental and Clinical Medicine, Marche Polytechnic University, Ancona, 60121, Italy.

Nicoló Baldini, Department of Experimental and Clinical Medicine, Marche Polytechnic University, Ancona, 60121, Italy.

Thomas Hoogeboom, Department of Rehabilitation, Physiotherapy Sciences and Sport, Brain Center, University Medical Center Utrecht, Utrecht, 3508 GA, The Netherlands; Research Group Innovation of Human Movement Care, HU University of Applied Sciences Utrecht, Utrecht, 3508 AJ, The Netherlands.

Maria Gabriella Ceravolo, Department of Experimental and Clinical Medicine, Marche Polytechnic University, Ancona, 60121, Italy.

Stefano Negrini, Laboratory of Evidence-Based Rehabilitation, IRCCS Galeazzi—Sant’Ambrogio Hospital, Milan, 20157, Italy; Department of Biomedical, Surgical and Dental Sciences, University of Milan “La Statale”, Milan, 20122, Italy.

Carlotte Kiekens, Laboratory of Evidence-Based Rehabilitation, IRCCS Galeazzi—Sant’Ambrogio Hospital, Milan, 20157, Italy.

PREPARE Project Group:

Aghayev Emin, Alepopoulos Vasilis, Banfi Giuseppe, Calcina Erik, Capecci Marianna, Maria Gabriella Ceravolo, Cordani Claudio, Ferrario Irene, Georgiadis Charalampos, Giannios Georgios, Gramatikopoulou Margarita, Herrmann Christian, Hoogendam Lisa, Thomas Hoogeboom, Esther Janssen, Khijmatgar Shahnawaz, Carlotte Kiekens, Jesper Knoop, Kompatsiaris Ioannis, Kotsis Vasilios, Mpaltadros Lampros, Alberto Negrini, Stefano Negrini, Nikolopoulos Spiros, Novak Erik, Patias George Robert, Patias Petros, Pennestrì Federico, Poli Arianna, Roustanis Themistocles, Selles Ruud, Staal Bart, Tsaopoulos Dimitrios, Tartaglia Gianluca Martino, Zaharatos Haris, Helena Burger, Zala Kuret, Gaj Vidmar, and Zlatko Matjačić

Author contributions

Esther R.C. Janssen (Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Visualization, Writing—original draft, Writing—review & editing), Jesper Knoop (Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing—review & editing), Gianmatteo Farabolini (Data curation, Formal analysis, Investigation, Methodology, Writing—review & editing), Nicoló Baldini (Data curation, Formal analysis, Investigation, Methodology, Writing—review & editing), Thomas Hoogeboom (Conceptualization, Funding acquisition, Methodology, Supervision, Writing—review & editing), Maria Gabriella Ceravolo (Conceptualization, Funding acquisition, Methodology, Supervision, Writing—review & editing), Stefano Negrini (Conceptualization, Funding acquisition, Methodology, Supervision, Validation, Writing—review & editing), and Carlotte Kiekens (Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Supervision, Writing—review & editing)

Supplementary material

Supplementary material is available at Journal of the American Medical Informatics Association online.

Funding

Funded by the European Union. UK participants in Horizon Europe Project PREPARE are supported by UKRI grant number 10086219 (Trilateral Research). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or European Health and Digital Executive Agency (HADEA) or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them. Grant Agreement 101080288 PREPARE HORIZON-HLTH-2022-TOOL-12-01. The work of CK and SN was supported by the Italian Ministry of Health—Ricerca Corrente.

Conflicts of interest

S.N. declares that he owns stock of ISICO (Milan, Italy).

Data availability

The data underlying this article are available in the article and in its online supplementary material.

References

1. Goetz LH, Schork NJ. Personalized medicine: motivation, challenges, and progress. Fertil Steril. 2018;109:952-963. [DOI] [PMC free article] [PubMed] [Google Scholar]
2. Grover S, Fitzpatrick A, Azim FT, et al. Defining and implementing patient-centered care: an umbrella review. Patient Educ Couns. 2022;105:1679-1688. [DOI] [PubMed] [Google Scholar]
3. Negrini S, Selb M, Kiekens C, et al. ; 3rd Cochrane Rehabilitation Methodology Meeting Participants. Rehabilitation definition for research purposes. A global stakeholders’ initiative by Cochrane Rehabilitation. Eur J Phys Rehabil Med. 2022;58:333-341. [DOI] [PMC free article] [PubMed] [Google Scholar]
4. Fayyaz A, Haqqi A, Khan R, et al. Revolutionizing cancer treatment: the rise of personalized immunotherapies. Discov Oncol. 2024;15:756. [DOI] [PMC free article] [PubMed] [Google Scholar]
5. Mehmood F, Mumtaz N, Mehmood A. Next-generation tools for patient care and rehabilitation: a review of modern innovations. Actuators. 2025;14:133. [Google Scholar]
6. Boniol M, Kunjumen T, Nair TS, Siyam A, Campbell J, Diallo K. The global health workforce stock and distribution in 2020 and 2030: a threat to equity and 'universal’ health coverage? BMJ Glob Health. 2022;7:e009316. [DOI] [PMC free article] [PubMed] [Google Scholar]
7. Janssen ERC, Punt IM, van Soest J, et al. Operationalizing and digitizing person-centered daily functioning: a case for functionomics. BMC Med Inform Decis Mak. 2024;24:184. [DOI] [PMC free article] [PubMed] [Google Scholar]
8. González-Márquez R, Schmidt L, Schmidt BM, Berens P, Kobak D. The landscape of biomedical research. Patterns. 2024;5:100968. [DOI] [PMC free article] [PubMed] [Google Scholar]
9. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. [DOI] [PMC free article] [PubMed] [Google Scholar]
10. Bright TJ, Wong A, Dhurjati R, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157:29-43. [DOI] [PubMed] [Google Scholar]
11. Traeger A, McAuley JH. STarT Back Screening Tool. J Physiother. 2013;59:131. [DOI] [PubMed] [Google Scholar]
12. Knoop J, Dekker J, van Dongen JM, et al. Stratified exercise therapy does not improve outcomes compared with usual exercise therapy in people with knee osteoarthritis (OCTOPuS study): a cluster randomised trial. J Physiother. 2022;68:182-190. [DOI] [PubMed] [Google Scholar]
13. Ouanes K, Farhah N. Effectiveness of Artificial Intelligence (AI) in clinical decision support systems and care delivery. J Med Syst. 2024;48:74. [DOI] [PubMed] [Google Scholar]
14. Sittig DF, Wright A, Osheroff JA, et al. Grand challenges in clinical decision support. J Biomed Inform. 2008;41:387-392. [DOI] [PMC free article] [PubMed] [Google Scholar]
15. Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17:75. [DOI] [PMC free article] [PubMed] [Google Scholar]
16. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Q. 2003;27:425-478. [Google Scholar]
17. Davis FD, Bagozzi RP, Warshaw PR. Technology acceptance model. J Manag Sci. 1989;35:982-1003. [Google Scholar]
18. Shaw H, Ellis DA, Ziegler FV. The Technology Integration Model (TIM). Predicting the continued use of technology. Comput Human Behav. 2018;83:204-214. [Google Scholar]
19. Schoville RR, Titler MG. Guiding healthcare technology implementation: a new integrated technology implementation model. CIN Comput Inf Nursing. 2015;33:99–107. [DOI] [PubMed] [Google Scholar]
20. Moher D, Shamseer L, Clarke M, et al. ; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. [DOI] [PMC free article] [PubMed] [Google Scholar]
21. Gates M, Gates A, Pieper D, et al. Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. BMJ. 2022;378:e070849. [DOI] [PMC free article] [PubMed] [Google Scholar]
22. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5:210. [DOI] [PMC free article] [PubMed] [Google Scholar]
23. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. [DOI] [PMC free article] [PubMed] [Google Scholar]
24. Arienti C, Patrini M, Negrini S, Kiekens C. Overview of cochrane systematic reviews for rehabilitation interventions in persons with spinal cord injury: a mapping synthesis. Arch Phys Med Rehabil. 2023;104:143-150. [DOI] [PubMed] [Google Scholar]
25. Campbell M, McKenzie JE, Sowden A, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. 2020;368:l6890. [DOI] [PMC free article] [PubMed] [Google Scholar]
26. Abell B, Naicker S, Rodwell D, et al. Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review. Implement Sci. 2023;18:32. [DOI] [PMC free article] [PubMed] [Google Scholar]
27. Borum C. Barriers for hospital-based nurse practitioners utilizing clinical decision support systems: a systematic review. Comput Inform Nurs. 2018;36:177-182. [DOI] [PubMed] [Google Scholar]
28. Chan SL, Lee JW, Ong MEH, et al. Implementation of prediction models in the emergency department from an implementation science perspective-determinants, outcomes, and real-world impact: a scoping review. Ann Emerg Med. 2023;82:22-36. [DOI] [PubMed] [Google Scholar]
29. Chen W, O’Bryan CM, Gorham G, et al. Barriers and enablers to implementing and using clinical decision support systems for chronic diseases: a qualitative systematic review and meta-aggregation. Implement Sci Commun. 2022;3:81. [DOI] [PMC free article] [PubMed] [Google Scholar]
30. Van Dort BA, Carland JE, Penm J, Ritchie A, Baysari MT. Digital interventions for antimicrobial prescribing and monitoring: a qualitative meta-synthesis of factors influencing user acceptance. J Am Med Inform Assoc. 2022;29:1786-1796. [DOI] [PMC free article] [PubMed] [Google Scholar]
31. Lee TC, Shah NU, Haack A, Baxter SL. Clinical implementation of predictive models embedded within electronic health record systems: a systematic review. Informatics (MDPI). 2020;7:25. [DOI] [PMC free article] [PubMed] [Google Scholar]
32. Meunier PY, Raynaud C, Guimaraes E, Gueyffier F, Letrilliart L. Barriers and facilitators to the use of clinical decision support systems in primary care: a mixed-methods systematic review. Ann Fam Med. 2023;21:57-69. [DOI] [PMC free article] [PubMed] [Google Scholar]
33. Araujo SM, Sousa P, Dutra I. Clinical decision support systems for pressure ulcer management: systematic review. JMIR Med Inform. 2020;8:e21621. [DOI] [PMC free article] [PubMed] [Google Scholar]
34. Peters LJ, Torres-Castaño A, van Etten-Jamaludin FS, Perestelo Perez L, Ubbink DT. What helps the successful implementation of digital decision aids supporting shared decision-making in cardiovascular diseases? A systematic review. Eur Heart J Digit Health. 2023;4:53-62. [DOI] [PMC free article] [PubMed] [Google Scholar]
35. Rittmann B, Stevens MP. Clinical decision support systems and their role in antibiotic stewardship: a systematic review. Curr Infect Dis Rep. 2019;21:29. [DOI] [PubMed] [Google Scholar]
36. Yung A, Kay J, Beale P, Gibson KA, Shaw T. Computer-based decision tools for shared therapeutic decision-making in oncology: systematic review. JMIR Cancer. 2021;7:e31616. [DOI] [PMC free article] [PubMed] [Google Scholar]
37. Sebastian A, Carroll JC, Oldfield LE, et al. Effect of genetics clinical decision support tools on health-care providers’ decision making: a mixed-methods systematic review. Genet Med. 2021;23:593-602. [DOI] [PubMed] [Google Scholar]
38. Mazo C, Kearns C, Mooney C, Gallagher WM. Clinical decision support systems in breast cancer: a systematic review. Cancers (Basel). 2020;12:369. [DOI] [PMC free article] [PubMed] [Google Scholar]
39. Johnson D, Del Fiol G, Kawamoto K, et al. Genetically guided precision medicine clinical decision support tools: a systematic review. J Am Med Inform Assoc. 2024;31:1183-1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
40. Akbar S, Lyell D, Magrabi F. Automation in nursing decision support systems: a systematic review of effects on decision making, care delivery, and patient outcomes. J Am Med Inform Assoc. 2021;28:2502-2513. [DOI] [PMC free article] [PubMed] [Google Scholar]
41. Bradley PT, Hall N, Maniatopoulos G, Neal RD, Paleri V, Wilkes S. Factors shaping the implementation and use of Clinical Cancer Decision Tools by GPs in primary care: a qualitative framework synthesis. BMJ Open. 2021;11:e043338. [DOI] [PMC free article] [PubMed] [Google Scholar]
42. Burns A, Donnelly B, Feyi-Waboso J, et al. How do electronic risk assessment tools affect the communication and understanding of diagnostic uncertainty in the primary care consultation? A systematic review and thematic synthesis. BMJ Open. 2022;12:e060101. [DOI] [PMC free article] [PubMed] [Google Scholar]
43. Fletcher E, Burns A, Wiering B, et al. Workload and workflow implications associated with the use of electronic clinical decision support tools used by health professionals in general practice: a scoping review. BMC Prim Care. 2023;24:23. [DOI] [PMC free article] [PubMed] [Google Scholar]
44. Wosny M, Strasser LM, Hastings J. Experience of health care professionals using digital tools in the hospital: qualitative systematic review. JMIR Hum Factors. 2023;10:e50357. [DOI] [PMC free article] [PubMed] [Google Scholar]
45. Maes-Carballo M, Martín-Díaz M, García-García M, et al. Decision aids for decision making about locally advance breast cancer: a systematic review. Cancer Invest. 2023;41:1-13. [DOI] [PubMed] [Google Scholar]
46. Tebeje TH, Klein J. Applications of e-health to support person-centered health care at the time of COVID-19 pandemic. Telemed J E Health. 2021;27:150-158. [DOI] [PubMed] [Google Scholar]
47. Toffaha KM, Simsekler MCE, Omar MA. Leveraging artificial intelligence and decision support systems in hospital-acquired pressure injuries prediction: a comprehensive review. Artif Intell Med. 2023;141:102560. [DOI] [PubMed] [Google Scholar]
48. Skullbacka S, Airaksinen M, Puustinen J, Toivo T. Risk assessment tools for QT prolonging pharmacotherapy in older adults: a systematic review. Eur J Clin Pharmacol. 2022;78:765-779. [DOI] [PMC free article] [PubMed] [Google Scholar]
49. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38:65-76. [DOI] [PMC free article] [PubMed] [Google Scholar]
50. Greenhalgh T, Wherton J, Papoutsi C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19:e367. [DOI] [PMC free article] [PubMed] [Google Scholar]
51. Olsen S, Lim XML, Alder G, Stavric V, Signal N. Factors influencing adoption and sustained use of rehabilitation technologies: a scoping review and qualitative analysis. Disabil Rehabil Assist Technol. 2025;20:804-822. [DOI] [PubMed] [Google Scholar]
52. Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14:103. [DOI] [PMC free article] [PubMed] [Google Scholar]
53. van Velzen M, de Graaf-Waar HI, Ubert T, et al. 21st century (clinical) decision support in nursing and allied healthcare. Developing a learning health system: a reasoned design of a theoretical framework. BMC Med Inform Decis Mak. 2023;23:279. [DOI] [PMC free article] [PubMed] [Google Scholar]
54. Hallinan CM, Ward R, Hart GK, et al. Seamless EMR data access: Integrated governance, digital health and the OMOP-CDM. BMJ Health Care Inform. 2024;31:e100953. [DOI] [PMC free article] [PubMed] [Google Scholar]
55. Unberath P, Prokosch HU, Gründner J, Erpenbeck M, Maier C, Christoph J. EHR-independent predictive decision support architecture based on OMOP. Appl Clin Inform. 2020;11:399-404. [DOI] [PMC free article] [PubMed] [Google Scholar]
56. Lichtner G, Schiefenhövel F, Gashi B, et al. Multicenter Evaluation of an Interoperable System for Automated Guideline Adherence Monitoring in CUs.ICrit Care Med. 2026;54:119–128. 10.1097/CCM.0000000000006961 [DOI] [PubMed] [Google Scholar]
57. Wilkinson MD, Dumontier M, Aalbersberg IJ, et al. The FAIR guiding principles for scientific data management and stewardship. Sci Data. 2016;3:160018. [DOI] [PMC free article] [PubMed] [Google Scholar]
58. Queralt-Rosinach N, Kaliyaperumal R, Bernabé CH, et al. ; COVID-19 LUMC Group. Applying the FAIR principles to data in a hospital: challenges and opportunities in a pandemic. J Biomed Semantics. 2022;13:12. [DOI] [PMC free article] [PubMed] [Google Scholar]
59. Bashir M, ed. FAIR and beyond: evolving principles for modern data ecosystems. In: Designing the Conceptual Landscape for a XAIR Validation Infrastructure; Springer Nature; 2025. [Google Scholar]
60. Gebler R, Reinecke I, Sedlmayr M, Goldammer M. Enhancing clinical data infrastructure for AI research: comparative evaluation of data management architectures. J Med Internet Res. 2025;27:e74976. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

ocag080_Supplementary_Data

ocag080_supplementary_data.zip^{(493.7KB, zip)}

Data Availability Statement

The data underlying this article are available in the article and in its online supplementary material.

[ocag080-B1] 1. Goetz LH, Schork NJ. Personalized medicine: motivation, challenges, and progress. Fertil Steril. 2018;109:952-963. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B2] 2. Grover S, Fitzpatrick A, Azim FT, et al. Defining and implementing patient-centered care: an umbrella review. Patient Educ Couns. 2022;105:1679-1688. [DOI] [PubMed] [Google Scholar]

[ocag080-B3] 3. Negrini S, Selb M, Kiekens C, et al. ; 3rd Cochrane Rehabilitation Methodology Meeting Participants. Rehabilitation definition for research purposes. A global stakeholders’ initiative by Cochrane Rehabilitation. Eur J Phys Rehabil Med. 2022;58:333-341. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B4] 4. Fayyaz A, Haqqi A, Khan R, et al. Revolutionizing cancer treatment: the rise of personalized immunotherapies. Discov Oncol. 2024;15:756. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B5] 5. Mehmood F, Mumtaz N, Mehmood A. Next-generation tools for patient care and rehabilitation: a review of modern innovations. Actuators. 2025;14:133. [Google Scholar]

[ocag080-B6] 6. Boniol M, Kunjumen T, Nair TS, Siyam A, Campbell J, Diallo K. The global health workforce stock and distribution in 2020 and 2030: a threat to equity and 'universal’ health coverage? BMJ Glob Health. 2022;7:e009316. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B7] 7. Janssen ERC, Punt IM, van Soest J, et al. Operationalizing and digitizing person-centered daily functioning: a case for functionomics. BMC Med Inform Decis Mak. 2024;24:184. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B8] 8. González-Márquez R, Schmidt L, Schmidt BM, Berens P, Kobak D. The landscape of biomedical research. Patterns. 2024;5:100968. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B9] 9. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B10] 10. Bright TJ, Wong A, Dhurjati R, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157:29-43. [DOI] [PubMed] [Google Scholar]

[ocag080-B11] 11. Traeger A, McAuley JH. STarT Back Screening Tool. J Physiother. 2013;59:131. [DOI] [PubMed] [Google Scholar]

[ocag080-B12] 12. Knoop J, Dekker J, van Dongen JM, et al. Stratified exercise therapy does not improve outcomes compared with usual exercise therapy in people with knee osteoarthritis (OCTOPuS study): a cluster randomised trial. J Physiother. 2022;68:182-190. [DOI] [PubMed] [Google Scholar]

[ocag080-B13] 13. Ouanes K, Farhah N. Effectiveness of Artificial Intelligence (AI) in clinical decision support systems and care delivery. J Med Syst. 2024;48:74. [DOI] [PubMed] [Google Scholar]

[ocag080-B14] 14. Sittig DF, Wright A, Osheroff JA, et al. Grand challenges in clinical decision support. J Biomed Inform. 2008;41:387-392. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B15] 15. Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17:75. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B16] 16. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Q. 2003;27:425-478. [Google Scholar]

[ocag080-B17] 17. Davis FD, Bagozzi RP, Warshaw PR. Technology acceptance model. J Manag Sci. 1989;35:982-1003. [Google Scholar]

[ocag080-B18] 18. Shaw H, Ellis DA, Ziegler FV. The Technology Integration Model (TIM). Predicting the continued use of technology. Comput Human Behav. 2018;83:204-214. [Google Scholar]

[ocag080-B19] 19. Schoville RR, Titler MG. Guiding healthcare technology implementation: a new integrated technology implementation model. CIN Comput Inf Nursing. 2015;33:99–107. [DOI] [PubMed] [Google Scholar]

[ocag080-B20] 20. Moher D, Shamseer L, Clarke M, et al. ; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B21] 21. Gates M, Gates A, Pieper D, et al. Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. BMJ. 2022;378:e070849. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B22] 22. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5:210. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B23] 23. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B24] 24. Arienti C, Patrini M, Negrini S, Kiekens C. Overview of cochrane systematic reviews for rehabilitation interventions in persons with spinal cord injury: a mapping synthesis. Arch Phys Med Rehabil. 2023;104:143-150. [DOI] [PubMed] [Google Scholar]

[ocag080-B25] 25. Campbell M, McKenzie JE, Sowden A, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. 2020;368:l6890. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B26] 26. Abell B, Naicker S, Rodwell D, et al. Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review. Implement Sci. 2023;18:32. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B27] 27. Borum C. Barriers for hospital-based nurse practitioners utilizing clinical decision support systems: a systematic review. Comput Inform Nurs. 2018;36:177-182. [DOI] [PubMed] [Google Scholar]

[ocag080-B28] 28. Chan SL, Lee JW, Ong MEH, et al. Implementation of prediction models in the emergency department from an implementation science perspective-determinants, outcomes, and real-world impact: a scoping review. Ann Emerg Med. 2023;82:22-36. [DOI] [PubMed] [Google Scholar]

[ocag080-B29] 29. Chen W, O’Bryan CM, Gorham G, et al. Barriers and enablers to implementing and using clinical decision support systems for chronic diseases: a qualitative systematic review and meta-aggregation. Implement Sci Commun. 2022;3:81. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B30] 30. Van Dort BA, Carland JE, Penm J, Ritchie A, Baysari MT. Digital interventions for antimicrobial prescribing and monitoring: a qualitative meta-synthesis of factors influencing user acceptance. J Am Med Inform Assoc. 2022;29:1786-1796. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B31] 31. Lee TC, Shah NU, Haack A, Baxter SL. Clinical implementation of predictive models embedded within electronic health record systems: a systematic review. Informatics (MDPI). 2020;7:25. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B32] 32. Meunier PY, Raynaud C, Guimaraes E, Gueyffier F, Letrilliart L. Barriers and facilitators to the use of clinical decision support systems in primary care: a mixed-methods systematic review. Ann Fam Med. 2023;21:57-69. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B33] 33. Araujo SM, Sousa P, Dutra I. Clinical decision support systems for pressure ulcer management: systematic review. JMIR Med Inform. 2020;8:e21621. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B34] 34. Peters LJ, Torres-Castaño A, van Etten-Jamaludin FS, Perestelo Perez L, Ubbink DT. What helps the successful implementation of digital decision aids supporting shared decision-making in cardiovascular diseases? A systematic review. Eur Heart J Digit Health. 2023;4:53-62. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B35] 35. Rittmann B, Stevens MP. Clinical decision support systems and their role in antibiotic stewardship: a systematic review. Curr Infect Dis Rep. 2019;21:29. [DOI] [PubMed] [Google Scholar]

[ocag080-B36] 36. Yung A, Kay J, Beale P, Gibson KA, Shaw T. Computer-based decision tools for shared therapeutic decision-making in oncology: systematic review. JMIR Cancer. 2021;7:e31616. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B37] 37. Sebastian A, Carroll JC, Oldfield LE, et al. Effect of genetics clinical decision support tools on health-care providers’ decision making: a mixed-methods systematic review. Genet Med. 2021;23:593-602. [DOI] [PubMed] [Google Scholar]

[ocag080-B38] 38. Mazo C, Kearns C, Mooney C, Gallagher WM. Clinical decision support systems in breast cancer: a systematic review. Cancers (Basel). 2020;12:369. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B39] 39. Johnson D, Del Fiol G, Kawamoto K, et al. Genetically guided precision medicine clinical decision support tools: a systematic review. J Am Med Inform Assoc. 2024;31:1183-1194. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B40] 40. Akbar S, Lyell D, Magrabi F. Automation in nursing decision support systems: a systematic review of effects on decision making, care delivery, and patient outcomes. J Am Med Inform Assoc. 2021;28:2502-2513. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B41] 41. Bradley PT, Hall N, Maniatopoulos G, Neal RD, Paleri V, Wilkes S. Factors shaping the implementation and use of Clinical Cancer Decision Tools by GPs in primary care: a qualitative framework synthesis. BMJ Open. 2021;11:e043338. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B42] 42. Burns A, Donnelly B, Feyi-Waboso J, et al. How do electronic risk assessment tools affect the communication and understanding of diagnostic uncertainty in the primary care consultation? A systematic review and thematic synthesis. BMJ Open. 2022;12:e060101. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B43] 43. Fletcher E, Burns A, Wiering B, et al. Workload and workflow implications associated with the use of electronic clinical decision support tools used by health professionals in general practice: a scoping review. BMC Prim Care. 2023;24:23. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B44] 44. Wosny M, Strasser LM, Hastings J. Experience of health care professionals using digital tools in the hospital: qualitative systematic review. JMIR Hum Factors. 2023;10:e50357. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B45] 45. Maes-Carballo M, Martín-Díaz M, García-García M, et al. Decision aids for decision making about locally advance breast cancer: a systematic review. Cancer Invest. 2023;41:1-13. [DOI] [PubMed] [Google Scholar]

[ocag080-B46] 46. Tebeje TH, Klein J. Applications of e-health to support person-centered health care at the time of COVID-19 pandemic. Telemed J E Health. 2021;27:150-158. [DOI] [PubMed] [Google Scholar]

[ocag080-B47] 47. Toffaha KM, Simsekler MCE, Omar MA. Leveraging artificial intelligence and decision support systems in hospital-acquired pressure injuries prediction: a comprehensive review. Artif Intell Med. 2023;141:102560. [DOI] [PubMed] [Google Scholar]

[ocag080-B48] 48. Skullbacka S, Airaksinen M, Puustinen J, Toivo T. Risk assessment tools for QT prolonging pharmacotherapy in older adults: a systematic review. Eur J Clin Pharmacol. 2022;78:765-779. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B49] 49. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38:65-76. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B50] 50. Greenhalgh T, Wherton J, Papoutsi C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19:e367. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B51] 51. Olsen S, Lim XML, Alder G, Stavric V, Signal N. Factors influencing adoption and sustained use of rehabilitation technologies: a scoping review and qualitative analysis. Disabil Rehabil Assist Technol. 2025;20:804-822. [DOI] [PubMed] [Google Scholar]

[ocag080-B52] 52. Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14:103. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B53] 53. van Velzen M, de Graaf-Waar HI, Ubert T, et al. 21st century (clinical) decision support in nursing and allied healthcare. Developing a learning health system: a reasoned design of a theoretical framework. BMC Med Inform Decis Mak. 2023;23:279. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B54] 54. Hallinan CM, Ward R, Hart GK, et al. Seamless EMR data access: Integrated governance, digital health and the OMOP-CDM. BMJ Health Care Inform. 2024;31:e100953. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B55] 55. Unberath P, Prokosch HU, Gründner J, Erpenbeck M, Maier C, Christoph J. EHR-independent predictive decision support architecture based on OMOP. Appl Clin Inform. 2020;11:399-404. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B56] 56. Lichtner G, Schiefenhövel F, Gashi B, et al. Multicenter Evaluation of an Interoperable System for Automated Guideline Adherence Monitoring in CUs.ICrit Care Med. 2026;54:119–128. 10.1097/CCM.0000000000006961 [DOI] [PubMed] [Google Scholar]

[ocag080-B57] 57. Wilkinson MD, Dumontier M, Aalbersberg IJ, et al. The FAIR guiding principles for scientific data management and stewardship. Sci Data. 2016;3:160018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B58] 58. Queralt-Rosinach N, Kaliyaperumal R, Bernabé CH, et al. ; COVID-19 LUMC Group. Applying the FAIR principles to data in a hospital: challenges and opportunities in a pandemic. J Biomed Semantics. 2022;13:12. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocag080-B59] 59. Bashir M, ed. FAIR and beyond: evolving principles for modern data ecosystems. In: Designing the Conceptual Landscape for a XAIR Validation Infrastructure; Springer Nature; 2025. [Google Scholar]

[ocag080-B60] 60. Gebler R, Reinecke I, Sedlmayr M, Goldammer M. Enhancing clinical data infrastructure for AI research: comparative evaluation of data management architectures. J Med Internet Res. 2025;27:e74976. [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

The GUIDE-CDSS framework: a comprehensive framework to GUIDE and evaluate the implementation of Clinical Decision Support Systems in healthcare, based on an overview of reviews

Esther R C Janssen, PhD

Jesper Knoop, PhD

Gianmatteo Farabolini, PhD

Nicoló Baldini, MD

Thomas Hoogeboom, PhD

Maria Gabriella Ceravolo, MD, PhD

Stefano Negrini, MD, PhD

Carlotte Kiekens, MD

Roles

Abstract

Objectives

Materials and Methods

Results

Discussion and Conclusion

Protocol registration

Introduction

Methods

Literature search

Study selection

Risk of bias

Data extraction

Synthesis of results

Results

Characteristics of included reviews

Figure 1.

Assessment of quality

Implementation determinants

Table 1.

Other qualitative results

GUIDE-CDSS

Figure 2.

Discussion

Strengths and limitations

Conclusion

Supplementary Material

Acknowledgments

Contributor Information

Author contributions

Supplementary material

Funding

Conflicts of interest

Data availability

References

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases