Barriers and Facilitators to the Use of Clinical Decision Support Systems in Primary Care: A Mixed-Methods Systematic Review

Abstract

PURPOSE

To identify and quantify the barriers and facilitators to the use of clinical decision support systems (CDSSs) by primary care professionals (PCPs).

METHODS

A mixed-methods systematic review was conducted using a sequential synthesis design. PubMed/MEDLINE, PsycInfo, Embase, CINAHL, and the Cochrane library were searched in July 2021. Studies that evaluated CDSSs providing recommendations to PCPs and intended for use during a consultation were included. We excluded CDSSs used only by patients, described as concepts or prototypes, used with simulated cases, and decision supports not considered as CDSSs. A framework synthesis was performed according to the HOT-fit framework (Human, Organizational, Technology, Net Benefits), then a quantitative synthesis evaluated the impact of the HOT-fit categories on CDSS use.

RESULTS

A total of 48 studies evaluating 45 CDSSs were included, and 186 main barriers or facilitators were identified. Qualitatively, barriers and facilitators were classified as human (eg, perceived usefulness), organizational (eg, disruption of usual workflow), and technological (eg, CDSS user-friendliness), with explanatory elements. The greatest barrier to using CDSSs was an increased workload. Quantitatively, the human and organizational factors had negative impacts on CDSS use, whereas the technological factor had a neutral impact and the net benefits dimension a positive impact.

CONCLUSIONS

Our findings emphasize the need for CDSS developers to better address human and organizational issues, in addition to technological challenges. We inferred core CDSS features covering these 3 factors, expected to improve their usability in primary care.

INTRODUCTION

Achieving best practice in primary care is a challenge because primary care professionals (PCPs) face a variety of health care issues and cannot always identify and access all the relevant information within the timeframe of the consultation.^1,2 Clinical decision support systems (CDSSs) are software designed to be a direct aid to clinical decision making, in which an inference engine matches the features of an individual patient to a computerized clinical knowledge base or a machine learning algorithm and then presents patient-specific assessments or recommendations to the clinician or the patient for a decision.^3,4 Clinical decision support systems are intended to improve the quality, safety, and efficiency of care.^5-7 In primary care, they have not yet proven effectiveness on clinical outcomes, such as morbidity or mortality.^8,9 However, according to a large recent meta-analysis of controlled trials in any settings, CDSSs increase the proportion of patients receiving the desired element of care by 5.8% overall, with a trend toward a worse outcome in the outpatient setting.¹⁰

Qualitative evaluations are needed to obtain a comprehensive understanding of the barriers and facilitators to CDSS use, which are key to their implementation success.^11,12 For this purpose, several systematic reviews focused on specific types of CDSS (knowledge-based CDSSs,¹³ clinical reminders¹⁴), specific processes of care (drug prescription,^15,16 diagnosis¹⁷), or specific health issues (antibiotics prescription,¹⁸ HIV management¹⁹), without any restriction to their context of use. However, health information systems implementation or evaluation models such as the HOT-fit framework (Human, Organization, Technology, Net Benefits²⁰) and others^21-25 emphasize the influence of context-specific factors in the use of health information systems. This is even more important for the primary care setting owing to the unique combination of the diversity and complexity of health issues managed (often multiple in the same consultation), its patient-centered care approach, and its particular decision-making context,²⁶ which may generate specific needs for decision support systems.

The objective of the present systematic review was therefore to identify and quantify the barriers and facilitators to the use of CDSSs by PCPs. From a qualitative synthesis based on the HOT-fit framework, we derived a quantitative synthesis assessing the mean impacts of the human, organizational, technological factors, and the net benefits dimension on CDSS use by PCPs.

METHODS

The present study was a mixed-methods systematic review that followed the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) reporting guideline. The study protocol was registered on PROSPERO on July 14, 2020 (CRD42020185199).

Search Strategy

The search strategy was built in cooperation with medical librarians. We searched PubMed/MEDLINE, PsycInfo, Embase, CINAHL, and the Cochrane library for relevant studies. We tracked citations from included records to identify additional relevant references. The search was performed on July 5, 2021 without date limitation. The complete search strategy for each database is available in Supplemental Appendix 1.

Eligibility Criteria

We included all qualitative, quantitative, and mixed methods studies and only original articles for which the primary or secondary objectives were to identify barriers and facilitators to the use of CDSSs in primary care. For studies involving various professions or specialties, we only considered those that had at least 50% PCPs in the study sample. We included CDSSs that provided recommendations to PCPs (and possibly to patients) and were intended for use during the consultation.

We excluded publication types that were posters, dissertations or theses, conference proceedings, commentaries, letters, or editorials. We excluded the following decision supports that were not considered as being CDSSs: drug-drug interaction alert systems, risk assessment tools that provided assessments but not recommendations, and clinical decision supports without inference engine. We excluded the following CDSSs: decision aids only used by patients, CDSSs described as concepts or prototypes, and CDSSs evaluated with simulated clinical scenarios.

Selection Process

The selection process was performed using Covidence software (Veritas Innovation Ltd).²⁷ After automatic removal of duplicates, 2 authors (P-Y.M. and C.R.) independently screened titles and abstracts and excluded irrelevant records. They independently screened potentially relevant articles in full text while documenting reasons for exclusion. The concordance () was 0.62 for title and abstract screening and 0.72 for full text screening. The disagreements were resolved by seeking consensus between the 2 authors.

Quality Appraisal

The quality of the included studies was independently appraised by 2 authors (P-Y.M. and C.R.) using the Covidence software. We applied the QuADS tool that has been designed to appraise the methodological and reporting quality of qualitative, quantitative, and mixed-methods studies in systematic reviews, based on 13 common criteria.²⁸ Each criterion is assessed according to the 4 following proposals: no mention at all, very slightly, moderately, complete. The concordance () was 0.39. The disagreements were resolved by seeking consensus between the 2 authors.

Data Extraction

We used a structured data collection form to extract CDSS features (Supplemental Table 1) and methodological features of the included studies. The extraction process was performed independently by 2 authors (P-Y.M. and C.R.); disagreements were resolved by seeking consensus between them. We contacted the main authors of the included studies to obtain data on CDSS features not reported in the published article.

The HOT-fit Framework

The HOT-fit framework describes the interdependent human, organizational, and technological factors related to health information system adoption. These 3 factors are described through 7 dimensions: system use and user satisfaction related to the human factor; environment and structure related to the organizational factor; system, information and service quality related to the technological factor. The framework is complemented by an additional dimension, net benefits, which captured the positive and negative effects of CDSS recommendations on PCPs (Figure 1).²⁰ Each HOT-fit dimension includes several evaluation measures. The HOT-fit framework was chosen as it assesses barriers and facilitators to the use of health information systems from a pragmatic, user-centered approach.

Figure 1:

HOT-fit framework (derived from Yusof et al²⁹).

HOT-fit = human, organization, technology, net benefits.

Note: The HOT-fit framework describes the interdependent human, organizational, and technological factors related to health information system adoption. A fit between the human, organizational, technological factors and the net benefit dimension is required for the adoption of these systems.

Data Synthesis

The review followed a sequential synthesis design, using a framework synthesis to inform a quantitative synthesis (Figure 2).^30-32 First, a convergent integrated approach built up a common set of qualitative data from the included qualitative and quantitative data. It involved data transformation through a narrative interpretation of quantitative data.³¹ The resulting framework synthesis was based on the HOT-fit framework. Second, we performed a quantitative synthesis by calculating the difference between barriers and facilitators categorized according to the HOT-fit framework.

Figure 2.

Mixed-methods synthesis design.

CDSS = clinical decision support system; HOT-fit = human, organization, technology, net benefits.

The framework synthesis was performed using NVivo (QSR International), released in March 2020.³³ Barriers and facilitators to the use of CDSSs were coded inductively as concepts and then classified into the appropriate HOT-fit evaluation measures.³² This was done independently by 2 authors (P-Y.M. and E.G.), who reviewed together every coded citation and reached consensus on the concepts and the associated HOT-fit evaluation measures. We extended the HOT-fit framework to include a few concepts we were unable to classify in existing evaluation measures. Due to the various level of detail in the description of the barriers and facilitators to CDSS use in the included studies, certain codes were considered as explanatory elements of higher concepts that we named main barriers and facilitators.

From the framework synthesis results, we first quantified the individual impacts of the 3 HOT-fit factors (human, organization, technology) and the net benefits dimension on the use of each CDSS. In practice, facilitators to CDSS use were quantified as +1 and barriers as −1. For example, if a given CDSS had X barriers and Y facilitators classified in the human factor, this factor’s impact on the use of this CDSS was calculated as Y-X. For each CDSS, barriers and facilitators were considered only once, regardless of the number of their occurrences in the included studies. Then, we calculated the mean impact of each HOT-fit factor and the net benefits dimension on the use of the whole sample of CDSSs included, with a confidence interval (CI).

RESULTS

A total of 48 studies assessing 45 CDSSs were included in the review (Figure 3).^34-81 Of these, 29 were mixed-methods studies, 15 were qualitative studies, and 4 were quantitative studies. Included studies were published from 1998 to 2021 (median year: 2016). Three CDSSs (EBMeDS, NHGDoc, and PRIMA-EDS) were assessed in 2 studies.

Figure 3.

PRISMA flow chart.

CDSS = clinical decision support system; PRISMA = Preferred Reporting Items for Systematic Review and Meta-Analysis.

Description of the CDSSs

Clinical decision support systems were mainly developed (n = 18) and used (n = 17) in the United States. The main users were primary care physicians (n = 37) and nurses (n = 22). Clinical decision support systems were used for preventive care (n = 27), treatment (n = 21), management of chronic disease(s) (n = 13), and/or diagnosis (n = 7). All CDSSs were knowledge based; none integrated a machine learning algorithm. Updates to the knowledge database were made in 11/33 CDSSs for which this information was available. The strength of evidence of CDSS recommendations was provided in 4/19 studies for which the information was available. Twenty-three CDSSs provided a direct link to the source of recommendations. Thirteen CDSSs provided educational materials to help patients with shared decision making. Primary care professionals were trained to the use of 36 CDSSs. Most CDSSs were partially or fully integrated in the electronic health record (EHR) (n = 28). The main features of the included CDSS are described in Table 1. Detailed features are presented in Supplemental Appendix 2.

Table 1.

Main Features of the 45 Identified CDSSs

CDSS n = 45 in 48 Studies	Name	Country of Use	Care Procedures	Targeted Health Issues
North America
Alagiakrishnan et al,36 2016	SMART-CDS	Canada	Prevention (iatrogenesis)	Adaption of medication to renal function from the patient’s EHR
Ash et al38 2011		USA	Diagnosis, Therapeutics (prescribing, vaccination)	Drug-drug, drug-condition, and drug-allergy interaction checking, patient care plan dashboard with reminders, nearly 3,000 condition specific point-and-click templates for documentation
Curry et al42 2011	Decision Support Server	Canada	Prevention (disease, test ordering)	Prescription of diagnostic imaging
Dixon et al43 2013		USA	Prevention (disease, test ordering), Management of chronic disease(s)	Diabetes mellitus type II, hypertension, coronary artery disease
Doerr et al44 2014	My Family	USA	Prevention (disease, test ordering)	Cancer risk management
Edelman et al45 2014	The Pregnancy and Health Profile (PHP)	USA	Prevention (disease, test ordering)	Prenatal genetic screening
Feldstein et al46 2013	Patient Panel-Support Tool (PST)	USA	Prevention (iatrogenesis, disease, test ordering), Management of chronic disease	Graphically displays ‘‘care gaps’’ (eg, for screening, medication use, monitoring, risk-factor control, vaccination)
Guenter et al47 2019	McMaster Pain Assistant (MPA)	Canada	Diagnosis, Therapeutics (prescribing, vaccination)	Neuropathic pain
Jenssen et al51 2016		USA	Prevention (disease, test ordering) Therapeutics (prescribing, vaccination)	Smoking cessation
Kempe et al52 2017	Immunization information systems	USA	Prevention (disease, test ordering)	Vaccination
Lam Shin Cheung et al54 2020	eAMS	Canada	Therapeutics (prescribing, vaccination), Management of chronic disease(s)	Asthma
Lemke et al55 2020	GWA	USA	Prevention (disease, test ordering)	Genetic risk assessment
Litvin et al. (2012)	ABX-TRIP CDSS	USA	Prevention (disease, test ordering), Diagnosis, Therapeutics (prescribing)	Acute respiratory infections
Litvin et al56 2016		USA	Prevention (disease, test ordering)	Identification and management of chronic kidney disease
Minian et al62 2021		Canada	Prevention (disease, test ordering)	Alcohol cessation
Montini et al63 2013		USA	Prevention (disease, test ordering) Therapeutics (prescribing, vaccination)	Tobacco cessation
Price et al67 2017		Canada	Prevention (iatrogenesis)	Potentially inappropriate prescriptions in the elderly
Richardson et al68 2019		USA	Therapeutics (prescribing, vaccination)	Sore throat, upper respiratory tract infections
Rubin et al72 2006		USA	Therapeutics (prescrbiing, vaccination), Diagnosis	Acute respiratory tract infections
Trafton et al76 2010	ATHENA-OT	USA	Therapeutics (prescribing, vaccination)	Opioid therapy for chronic, noncancer pain
Trinkley et al77 2021		USA	Therapeutics (prescribing, vaccination)	Heart failure
Williams et al79 2016		USA	Prevention (disease, test ordering)	Pediatric cardiovascular risk
Zheng et al81 2005	CRS	USA	Prevention (iatrogenesis, disease, test ordering), Therapeutics (prescribing, vaccination), Management of chronic disease(s)	Diabetes mellitus type II, hyperlipidemia, steroid-induced osteoporosis, influenza, pneumonia, breast cancer, cervical cancer
Europe
af Klercker et al35 1998		Sweden	Diagnosis	Ear, nose, throat diseases
Arts et al37 2018		The Netherlands	Management of chronic disease(s)	Diabetes mellitus type II, atrial fibrillation, hypertension, medication prescriptions relating to care of older adults
Bindels et al41 2003	GRIF Automated Feedback System	The Netherlands	Prevention (disease, test ordering)	Comments on the appropriateness of diagnostic tests ordered by general practitioners
Helldén et al48 2015	The renal button	Sweden	Prevention (iatrogenesis)	Adaption of medication to renal function from the patient’s EHR
Heselmans et al49 2020 and Koskela et al53 2016	EBMeDS	Belgium, Estonia, Finland, Italy	Prevention (iatrogenesis, disease, test ordering), Therapeutics (prescribing, vaccination), Management of chronic disease	> 1,000 NICE-accredited international guidelines
Lugtenberg et al58,59 2015 (2 articles)	NHGDoc	The Netherlands	Prevention (disease, test ordering, iatrogenesis), Therapeutics (prescribing, vaccination), Management of chronic disease(s)	Diabetes mellitus type II, cardiovascular risk management, asthma/COPD, thyroid disorders, viral hepatitis and other liver diseases, atrial fibrillation, subfertility
Pannebakker et al64 2019		England	Prevention (disease, test ordering)	Pigmented skin lesions
Rieckert et al68,69 2018, 2019	PRIMA-EDS	Germany, Austria, Italy, England	Prevention (iatrogenesis)	Polypharmacy in older and chronically ill people
Rousseau et al71 2003		England	Therapeutics (prescribing, vaccination), Management of chronic disease(s)	Asthma and angina in adults
Toth-Pal et al75 2008	Evibase	Sweden	Management of chronic disease(s)	Congestive heart failure
Australia
Abimbola et al34 2019	Health Tracker	Australia	Prevention (disease, test ordering)	Cardiovascular risk management
Bandong et al39 2019	My Whiplash Navigator	Australia	Therapeutics (prescribing, vaccination)	Whiplash-associated disorders
Peiris et al65 2014		Australia	Diagnosis, Therapeutics (prescribing, vaccination)	Back pain management
Wan et al78 2012		Australia	Management of chronic disease(s)	Diabetes mellitus type 2
Wilson et al80 2007	EMPOWER	Australia	Prevention (disease, test ordering), Therapeutics (prescribing, vaccination)	Cardiovascular risk management, hypertension
South America
Maia et al60 2016		Brazil	Management of chronic disease(s)	Diabetes mellitus type II
Marcolino et al61 2021		Brazil	Prevention (disease, test ordering), Management of chronic diseases	Diabetes mellitus type II, hypertension, cardiovascular risk treatment
Silveira et al73 2019	TeleHAS	Brazil	Prevention (disease, test ordering), Therapeutics	Cardiovascular risk management, hypertension
Africa
Bessat et al40 2019	REC	Burkina Faso	Diagnosis, Therapeutics (prescribing, vaccination)	Follow-up and treatment of children under the age of 5 years in developing countries
Jensen et al50 (2019)	eIMCI	South Africa	Prevention (disease, test ordering), Therapeutics (prescribing, vaccination)	Management of childhood illness
Sukums et al74 2015	QUALMAT	Burkina Faso, Ghana, Tanzania	Therapeutics (prescribing, vaccination)	Antenatal and intrapartum care
Asia
Praveen et al66 2014		India, Indonesia, Thailand	Management of chronic disease(s)	Cardiovascular risk management

CDSS = clinical decision support system; COPD = chronic obstructive pulmonary disease; EHR = electronic health record; NICE = National Institute for Health and Care Excellence.

Quality Assessment

The rationale for the choice of the data collection tool was not mentioned at all in 18 studies and very slightly in 6 studies. The format and content of the data collection tool was estimated as completely appropriate to meet the stated research objectives in 27 studies, and moderately appropriate in 19 studies. The appraisal of the 13 quality criteria of the QuADS tool is presented in Supplemental Table 2. The methodological features of the included studies are presented in Supplemental Table 3.

Framework Synthesis

In total, 186 main barriers and facilitators and 69 explanatory elements were identified. All CDSSs reported technological barriers or facilitators, but 4 CDSSs did not assess the human factor, 4 did not assess the organizational factor, and 3 did not assess the net benefits dimension. Among the 186 main barriers and facilitators, 43 were classified as human factors, 49 as organizational factors, 70 as technological factors, and 24 in the net benefits dimension. The full list is presented in Supplemental Appendix 3, and barriers and facilitators (or their pairs) identified in at least 7 CDSSs are described in Tables 2 and 3 with their explanatory elements. The mean number of main barriers and facilitators identified per CDSS was 28.9; this ranged from 8 to 86 (Supplemental Table 4). We complemented the HOT-fit framework by adding an evaluation measure called “hardware” in the dimension “structure” of the factor “organization” to describe barriers and facilitators related to hardware issues in PCPs facilities.

Table 2.

Main Facilitators Reported in More Than 7 CDSSs, and Their Explanatory Elements, Classified According to the HOT-Fit Framework

HOT-Fit Framework		Main Facilitatorsa (No. CDSSs Concerned)	Explanatory Elements
Factors and Dimensions (No. CDSSs concerned)	Evaluation Measures
Human (n = 41)     User satisfaction (n = 31)	Perceived usefulness Training Software satisfaction Motivation to use Overall satisfaction	Perceived usefulness of the CDSS (n = 23) Training before use is appreciated (n = 10) PCPs would continue to use the CDSS (n = 9) Patients’ perceived usefulness of the CDSS increases PCPs motivation to use it (n = 7) CDSSs increase PCPs satisfaction (n = 7)
Organization (n = 41)     Structure (n = 39)	Clinical process	Natural integration of the CDSS in the clinical workflow (n = 13)
	Autonomy	Producing reports of quality measures through collected data increases the value from the CDSS’s use in clinical practice (n = 7)	Expansion of skill set and roles in assisting physicians and patients in meeting care needs
	Teamwork	Other professionals ease physician’s increased workload with the CDSS (n = 6)
Technology (n = 45)     System quality (n = 45)	Ease of use	The CDSS is user-friendly (ergonomic) (n = 30) CDSS recommendations are easy to understand (n = 9)
	Usefulness of system features and functions	Reminders (n = 8)
	Ease of learning	Easy to use after a short learning period (n = 9)
Information quality (n = 40)	Usefulness	Information provided is useful for the targeted process of care (n = 13) Educational materials for patients are valuable (n = 7)	General agreement with the validity of recommendations
	Format	Pleasing visual layout [n = 12]
	Relevance	Recommendations are relevant (n = 11)
	Reliability	Recommendations are reliable (n = 9)
Service quality (n = 11)	Technical support	Satisfaction with the CDSS service support (n = 7)	CDSS technical staff availability
Net benefits (n = 42)	Effectiveness	Potential to improve the quality of care (n = 23)	Brings preventive care to the forefront
			Helps to systematize assessment of every patient
			Facilitates patient care management
		CDSS helps PCPs to improve guideline adherence (n = 11)
	Efficiency	Using CDSS saves time (n = 22)	Shortening documentation time
			Giving a quick patient evaluation from relevant data in patients’ EHRs
	Decision-making quality	CDSS facilitates decision making (n = 22)	CDSS is facilitating decision making about referral
	Communication	CDSS helps focus on patient education (n = 18) CDSS eases patient-PCP communication (n = 13)	CDSS helps increase patient engagement
	Clinical practice	CDSS is a way to update PCP’s knowledge (n = 17) CDSS leads to better teamwork in primary care (n = 7) CDSS increases PCPs’ self-confidence (n = 7)
	Error reduction	CDSS helps PCPs to identify unrecognized information needs (n = 17)

CDSS = clinical decision support system; EHR = electronic health record; HOT-fit = human, organization, technology, net benefits; PCP = primary care professional.

^aMain facilitators are ranked by the number of CDSSs concerned.

Table 3.

Main Barriers Reported in at Least 7 CDSSs, and Their Explanatory Elements, Classified According to the HOT-Fit Framework

HOT-Fit Framework		Main Barriersa (No. CDSSs Concerned)	Explanatory Elements
Factors and Dimensions (No. CDSSs Concerned)	Evaluation Measures
Human (n = 41)     System use (n = 39)	Resistance or reluctance	Conflicts between CDSS recommendations and PCP expertise or beliefs (n = 18)	CDSS recommendations do not reflect the complexity of the situation
	Report acceptance	Alert fatigue (n = 13) Information overload (n = 8)	Lack of a concise synthesis of the CDSS recommendation
	Training	Training before use is needed (n = 11)	The training session to the CDSS is inadequate or too short
	Attitude	PCPs don’t need help with the targeted health issue (n = 8)
		Lack of engagement from PCPs (inertia of previous practice) (n = 8)
	Knowledge and expertise	Lack of computer skills (n = 7)
	Motivation to use	Ask for financial compensation to use the CDSS (n = 7)
Organization (n = 41)	Clinical process	Using CDSS disrupts usual workflow (n = 25)
Structure (n = 39)	Teamwork	Need of more teamwork with other PCPs to help physicians with CDSS’s increased workload (n = 13)	Physicians fear more the CDSS workload than assistants or nurses
	Hardware	Lack or computers or tablets (n = 7)
Environment (n = 18)	Inter-organizational relationship	Difficulty to use CDSSs for patients comanaged by other specialists (n = 11)	Information is sometimes missing or not integrated from external sources
Technology (n = 45)     System quality (n = 45)	Ease of use	The CDSS is not user-friendly (n = 21)	Need to switch windows in the EHR while using CDSSs
			Location of CDSS recommendations should be changed
			Need to switch windows between the EHR and the CDSS
	Turnaround time	CDSS slowness (n = 16)	CDSS’s slowness impairs the interaction with the patient and increases the consultation time
	Usefulness of system features and functions	CDSS not fully integrated in the EHR (n = 14)	A CDSS not fully integrated in the EHR is time consuming and disrupts workflow
			The most current information collected in the EHR is sometimes not updated in the CDSS
	Database contents	The CDSS should target more health issues (n = 11)
		Questioning validity of CDSS’s knowledge database (n = 7)	Concerns about the CDSS’s independence from pharmaceutical industry
	Flexibility	Need of customization options (n = 8)
Information quality (n = 40)	Format	Format of recommendations (length, structure, font colors) (n = 13)
	Reliability	Doubtful reliability of the recommendations (n = 12)	The reliability of the recommendations depends on the quality and completeness of the information collected
	Relevance	Recommendations are not relevant (n = 11)	Conflicts between patient complaints and unrelated CDSS recommendations General recommendations are often irrelevant
	Usefulness	Recommendations are not helpful (n = 8)
Net benefits (n = 42)	Efficiency	Increased workload during the consultation (n = 33)	Lack of time to use the CDSS during the consultation Structured data collection takes too much time Duplication of data collection Coping strategies: increased consultation time, need of additional time to use the CDSS outside the consultation, scheduling follow-up consultations
		Negative effect on patient-PCP communication (n = 7)

CDSS = clinical decision support system; EHR = electronic health record; HOT-Fit = human, organization, technology, net benefits; PCP = primary care provider.

^aMain barriers are ranked by the number of CDSSs concerned.

From the human perspective, PCPs valued CDSSs for which they were trained and perceived to be useful. Conversely, they did not appreciate CDSS that provided recommendations conflicting with their beliefs or expertise. They frequently experienced alert fatigue or information overload.

From the organizational perspective, PCPs appreciated that the CDSS was well integrated into the clinical workflow and physicians appreciated that other professionals eased their workload. Conversely, PCPs expressed difficulties in using CDSSs with patients managed by other specialists because of disagreements between CDSS recommendations and specialists’ prescriptions.

From the technological perspective, PCPs appreciated fully integrated and easy-to-use CDSSs, providing reminders for PCPs and educational materials to patients, and relevant and reliable recommendations. Conversely, they disliked CDSSs that were slow, or targeting only a few health issues. They sometimes questioned the reliability of the recommendations, which they attributed to the quality and completeness of the information collected. PCPs frequently requested CDSS customization features.

From the net benefits perspective, PCPs felt that using CDSSs increased workload during the consultation for 33 of the 45 included CDSSs. Despite this major barrier, PCPs largely agreed on the benefits of CDSSs in terms of their potential to improve quality of care, particularly for preventive care. Primary care clinicians also felt that CDSSs improved the automatic identification of all useful information that they did not systematically recognize at the moment the clinical decision is made, as well as shared medical decision making through the provision of educational material to patients and their adherence to guidelines.

Quantitative Synthesis

Organizational and human HOT-fit factors had an overall negative impact on CDSS use by PCPs. The technological factor had a neutral overall impact, and the net benefits dimension an overall positive impact on CDSS use (Table 4). Individual impacts of the 3 HOT-fit factors and the net benefits dimension on the use of each CDSS are presented in Supplemental Table 5.

Table 4.

Mean Impacts of HOT-Fit Categories on CDSS Use by PCPs

	Human	Organization	Technology	Net Benefits
Mean impact of HOT-fit categories on CDSS use	Slightly negative	Slightly negative	Neutral	Positive
Mean difference between barriers and facilitators (95% CI)	−1.5 (−2.2 to −0.8)	−1.9 (−2.6 to −1.1)	−0.5 (−1.5 to 0.5)	+ 3.1 (2.2 to 3.9)

CDSS = clinical decision support system; HOT-fit = human, organization, technology, net benefits; PCP = primary care professional.

Note: In the human factor, there was 1.5 additional barriers per CDSS than there were facilitators. In the net benefits dimension, there was 3.1 additional facilitators per CDSS than there were barriers.

DISCUSSION

All barriers and facilitators to CDSS use by PCPs were distributed across the 3 HOT-fit factors and the net benefits dimension, with a predominance of the technological factor, that was the only factor explored in all studies. However, the overall impact of the technological factor on the use of CDSSs by PCPs was neutral, which indicates a balance between technological barriers and facilitators, while the human and organizational factors had overall negative impacts and the net benefit dimension an overall positive impact. The net benefits reported by PCPs support the potential effectiveness of CDSSs in improving quality and safety of care. However, they seem unable to improve care efficiency since they are believed to increase PCP workload.

Comparison With Other Studies

This review is the first to identify and quantify barriers and facilitators to the use of CDSSs specifically in the primary care setting. In 2017, Kilsdonk et al conducted a systematic review and gap analysis of barriers and facilitators to the use of knowledge-based CDSSs in any setting (including hospitals) according to the HOT-fit framework, including CDSS evaluations based on simulated clinical scenarios.¹³ The quantitative gap analysis revealed the predominance of technological and human factors, and a knowledge gap regarding the organizational factor and the net benefits dimension. However, no conclusion could be drawn on the relative impact of these factors on CDSS use. In the present review, the organizational factor was not less frequently identified than the human factor, and the quantitative synthesis assessed for the first time the relative impact of the HOT-fit factors on CDSS use. In 2013, Moxey et al analyzed both health care providers’ general views on, and use of, CDSSs including computerized guidelines and risk assessment tools, in various settings.¹⁵ These systematic reviews identified barriers and facilitators mostly concerning time consumption, workflow, integration in the EHR, user friendliness, and relevance of the recommendations. Moxey et al conducted a subgroup analysis that identified the lack of CDSS integration into the EHR and patient negative opinion as barriers specific to the ambulatory care setting.¹⁵ Poor CDSS integration was confirmed as a main technological barrier in our review. The diversity of EHRs developed for primary care may explain the persistence of this barrier over time.⁸² Patient negative opinion was less frequently reported in our review, presumably because of the increasing acceptance over time of the use of health information systems during consultations.⁸³

Other barriers and facilitators identified in the present review are original as compared with previous reviews.^13,15 First, teamwork needs for and benefits of using CDSSs were frequently reported. The importance of teamwork could not be identified previously since it is just emerging in primary care in many health care systems, contrarily to the hospital setting.^84,85 Second, PCPs expressed difficulties in using CDSSs with patients co-managed by specialists, due to discrepancies between specialist and CDSS recommendations or to outdated patient information in the EHR. Third, PCPs expected CDSSs covering a large array of conditions in agreement with the diversity of the health issues they manage. Fourth, the contribution of CDSSs to reporting on quality measures was valued by PCPs, in a context of evaluation programs implemented in primary care.

Strengths and Weaknesses

The scope of this review was limited to strictly defined CDSSs providing recommendations to PCPs based on individual patient characteristics, according to a medical decision-making perspective. Two previous systematic reviews^13,15 also included decision supports providing individualized assessments (risks) or general recommendations (not personalized to the patient), which presumably increased the heterogeneity of the barriers and facilitators identified. They also have included decision supports used in any care setting (including hospitals), and those evaluated using simulated clinical scenarios. In the present review, only CDSSs used in clinical settings and in the specific context of primary care were included, which further strengthens the external validity of the results. It may, however, not be fully representative of the various CDSSs developed for primary care, as some of them were probably not studied regarding barriers and facilitators to their use.

Frequency-based indicators of the impact of HOT-fit categories on CDSS use were useful for comparison purposes; they were, however, limited by the heterogeneity of CDSS barriers and facilitators and by the difficulty to weight them individually according to their perceived importance by PCPs. In addition, PCPs reported some barriers such as conflicts between CDSS recommendations and their own expertise or lack of computer skills, while the first barrier is more likely related to the use of guidelines⁸⁶ and the second to the uptake of EHRs.^87,88 The classification of barrier and facilitators in the evaluation measures was sometimes subjective because of similarities between, or lack of clear definition of some HOT-fit evaluation measures, as already reported by other authors.¹³ For instance, the HOT-fit evaluation measure “clinical process” in the dimension “structure” of the organizational factor is close to the evaluation measure “clinical practice” of the net benefits dimension.

Implications

The present systematic review highlights barriers and facilitators to the use of CDSSs related to its feasibility (eg, increased workload), acceptability (eg, conflicts with PCPs expertise or beliefs), meaningfulness (eg, relevance of recommendations), and effectiveness (net benefits dimension). These different forms of evidence refer to the feasibility, appropriateness, meaningfulness, and effectiveness (FAME) evidence-based model,⁸⁹ which is useful to understand complex interventions such as implementing CDSSs. Based on these findings, we inferred an operational list of 11 intrinsic and contextual CDSS features expected to make them more feasible, acceptable, meaningful, and effective in primary care (Table 5). They are spread across the 3 interdependent human, organizational, and technological factors. Among intrinsic features, the expectation of decision support for preventive care is consistent with the great importance of prevention in primary care practice. The expectation of a large array of conditions covered by CDSSs is explained by the preference of PCPs for a single comprehensive system rather than several CDSSs with limited clinical coverage displaying recommendations in separate windows, each requiring a specific training. Information overload refers to PCPs facing more information than they have the time or cognitive ability to process.^90,91 Clinical decision support systems aim at both rationalizing patient management while avoiding overwhelming PCPs with information, which can be achieved by CDSSs providing concise recommendations and prioritizing the most appropriate interventions recommended for each patient.⁹² In addition, the feature of providing patients with educational material supports shared decision making within a patient-centered approach. Among contextual features, developing CDSSs in close collaboration with PCPs according to a bottom-up approach is needed to improve their perceived usefulness and user-friendliness and the relevance of their recommendations. Providing the rationale for selecting the sources of the CDSS knowledge base is expected to increase CDSS reliability. This seems critical, even more for future non–knowledge-based CDSSs, as health care professionals are exposed to automation bias, which consists of over-relying on automated advice.⁹³ Teamwork for data collection and use of CDSSs may ease physician workload and make EHR patient data more complete and reliable. Several of these intrinsic and contextual features may allow the leading barrier, increased workload, to be overcome. Since the findings of the present review show that human and organizational factors are the most impeding to CDSS use, we recommend that CDSS developers investigate the human and organizational requirements in the early stages of CDSS development and that evaluation studies of CDSS use systematically evaluate these factors. In addition, quantitative studies are particularly needed to assess the weight of the main barriers and facilitators identified through the present framework synthesis.

Table 5.

Expected Features of a CDSS for Primary Care

Intrinsic features

Including preventive care

Covering a large array of conditions

Providing reminders personalized to the patient

Minimizing information overload and alert fatigue

Providing educational materials to patients

Integrated in the EHR, with the fewest possible duplicate data entries

Fast processing

Contextual features

Developed in close collaboration with PCPs

Providing the rationale for the selection of sources of its knowledge base

With teamwork for data collection and use of the CDSS

With systematic training for its use

CDSS = clinical decision support system; EHR = electronic health record; PCPs = primary care professionals.

CONCLUSION

Although benefits reported by PCPs support the potential effectiveness of CDSS use in improving quality and safety of care, they also highlight its lack of efficiency due to increased workload. Our findings emphasize the need for CDSS developers to better address human and organizational issues in addition to technological stakes. Covering theses 3 factors, we inferred core intrinsic and contextual CDSS features expected to improve their usability in primary care.