Table 1.
Data extraction table
| Authors | Origin | Aims | Context | Methods | Outcome measures | Key findings of interest | Impact on time/workload |
|---|---|---|---|---|---|---|---|
| Ahmad et al. 2010 [26] | Canada | To enhance understanding about computer-assisted health-risk assessments (HRA) from physicians’ perspectives regarding benefits and concerns/challenges |
Condition of focus: Domestic violence Setting: 1 family practice clinic Tool: Health Risk Assessment tool - Embedded/linked with EMR: Yes - Interruptive alert: No, guides visit - User-driven: Physician - Risk score: No |
Qualitative interviews |
(1) What do you think of your experience with the HRA? (2) How would you describe its potential across various risks and visits? (3) Would you recommend such HRA in a family practice setting? 4) What factors are important for its implementation in a family practice setting? |
Theme 2: perceived risks & challenges (subtheme: length of visit) - Some expressed concern about the increase in length of the visit - Others managed the time pressure by offering follow-up visits or viewed the task of risk review as a professional obligation even if it meant increases in the consultation time - Follow-up visit offered in order to avoid “taking time away” from other waiting patients - ‘Patient readiness'—not always the right time to address in the visit |
Perception Impact on time: Mixed views |
| Arts et al. 2017 [27] | Netherlands | To investigate the effectiveness of a CDSS as measured by GPs' adherence to the Dutch GP guideline for patients with Atrial Fibrilation |
Condition of focus: Stroke prevention in AF Setting: General Practice Tool: CDS using antithrombotic guidelines - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Physician - Risk score: Yes |
Quantitative RCT |
Primary: Difference in the proportion of patients with AF treated in accordance with the guideline between the intervention and control groups Secondary: reasons GPs provided for deviating from the guideline and how they responded to required justification |
Limited evidence for effectiveness, attributed to low usage Reasons for low usage discussed in a separate qualitative evaluation, but included barriers relating to lack of time, too many alert notifications and tool functionality limitations |
Perception Impact on time: Increase Driving perception: lack of time, too many alerts |
| Arts et al. 2018 [28] | Netherlands | To identify remediable barriers which led to low usage rates seen in RCT |
Condition of focus: Stroke prevention in AF Setting: General Practice Tool: CDS using antithrombotic guidelines - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Physician - Risk score: Yes |
Mixed: quantitative survey + focus group | Barriers and facilitators for CDSS use |
More than 75% of GPs answered the question: "What were the most important reasons for not opening the recommendations?" citing reasons relating to lack of time Many felt there was lack of time during the appointment to perform the suggested actions. Some GPs scheduled follow up appointments for this purpose |
Perception Impact on time: Increase |
| Baron et al. 2017 [29] | USA | To evaluate the value and feasibility of three examples of CDS relating to occupational health in five primary care group practices |
Condition of focus: Occupational Health Setting: Primary Care Tool: CDS using guidelines - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: Physician - Risk score: No |
Qualitative:- interviews + observations |
Interviews: - physicians' daily work patterns - experience with EMRs and CDS - attitudes and practice regarding consideration of health factors encountered in a patient’s job - how patients' work data is collected in the EMR Observations: - workflow data through observations of clinic staff |
The amount of clinical time the CDS tools would require was a prominent concern 1 of 7 themes: clinician acceptance is affected by whether CDS adds or saves time |
Perception Impact on time: No conclusion |
| Bauer et al. 2014 [30] | USA |
To examine the attitudes and opinions of paediatric users’ toward the Child Health Improvement through Computer Automation (CHICA) system |
Condition of focus: Child Health Setting: Community paediatric clinics Tool: Child Health Improvement (CHICA) CDS for paediatric visits - Embedded/linked with EMR: Yes - Interruptive alert: No, guides visit - User-driven: Physician & patient - Risk score: No |
Quantitative: survey + free text | General acceptability and satisfaction |
Critical opinions of CHICA were that it wasted time and money. This perception persisted in spite of informal time-flow studies in one of the clinics showing that CHICA did not create significant delays Approximately half of respondents reported that it did slow down clinic |
Perception and objective measure showed conflict Impact on time: Increase (perception), no impact (objective) Driving perception: lack of time, workflow disruption |
| Carlfjord et al. 2011 [31] | Sweden | To explore how staff at 6 Primary Health Care units experienced implementation of a tool for lifestyle intervention in primary health care |
Condition of focus: Preventive care Setting: PHC units Tool: CDS for lifestyle intervention & preventive services - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: Patient-completed - Risk score: Unclear |
Qualitative: focus groups + interviews |
- Overall working situation coinciding with the implementation process - Experiences of implementation activities and the tool - How to address lifestyle issues with patients - Openness to innovations |
GPs' perception of workload already being too heavy and accommodating never-ending changes such as the lifestyle intervention tool may threaten independence and bring extra work |
Perception Impact on time: Increase Driving perception: workload already heavy |
| Carlfjord et al. 2012 [32] | Sweden | Qualitative evaluation of the 2011 study to explore staff’s perceptions of handling lifestyle issues, including the consultation as well as the perceived usefulness of the tool |
Condition of focus: Preventive care Setting: PHC units Tool: CDS for lifestyle intervention & preventive services - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: Patient-completed - Risk score: Unclear |
Qualitative: focus groups |
Staff responses to two scenarios: - How to handle the patient/consultation - Advice to give to another clinic considering implementing the tool |
Lifestyle issues tend to be forgotten when the workload is increasing, despite interest and awareness of its importance Many staff members found it difficult to initiate a conversation about lifestyle, particularly alcohol consumption, when the patient is seeking care for something else Time is too pressured to be focused on lifestyle/prevention especially in acute times/low resources |
Perception Impact on time: Increase Driving perception: no time for preventive care |
| Chiang et al. 2017 [33] | Australia | To test the acceptability and feasibility of the Treat to Target CVD (T3CVD), an EMR-based CDS tool, for the evaluation of Absolute CVD Risk in general practice |
Condition of focus: Cardiovascular Disease Setting: 1 general practice Tool: CDS for CVD risk assessment - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Automatic, based on EMR data - Risk score: Yes, for patients with > 10% risk of CVD |
Qualitative interviews |
Acceptability and feasibility of the T3CVD tool, including GPs’ experiences with the tool in real-world clinical practice, under a framework of 3 themes: - patient control - clinical quality of care - technology capability/capacity |
GPs described how the ACVDR assessment pop-up necessitated additional time, often needing to arrange a follow-up visit if there was no time to discuss While the tool had capacity to save time by automating information provision rather than GPs manually accessing the existing CVD risk tool, it is potentially disruptive and adds to many existing pop-ups |
Perception Impact on time: Mixed views |
| Chiang et al. 2015 [15] | Australia | To explore the use of a cancer risk tool, which implements the QCancer model, in consultations and its potential impact on clinical decision making |
Condition of focus: Cancer Setting: General Practice Tool: QCancer risk tool - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: GP - Risk score: Yes, for each of 10 cancers |
Qualitative: simulated consultations + interviews |
1. Coherence 2. Cognitive participation 3. Collective action 4. Reflexive monitoring |
Tool was easy and quick to use, but introducing the emotive topic of cancer caused anxiety. Risk output 'too confronting' to use in a consultation and could lead to a loss of control over the consultation and time being used to reassure which may lead to overrunning by 20-30 m GPs thought pop-up alerts would add to alert fatigue |
Perception Impact on time: Increase Driving perception: impact on conversation |
| Crawford et al. 2011 [34] | Scotland | To understand primary care practitioners’ views towards screening for diabetic foot disease and their experience of the SCI-DC system |
Condition of focus: Diabetic foot disease Setting: General Practice Tool: CDS for screening - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: GP - Risk score: Unclear |
Qualitative interviews | Views on and use of decision support systems, specifically SCI-DC | The duplication of effort to complete the SCI-DC and then the GP IT system through which the practice receives remuneration is unnecessarily time consuming. Integration into GP IT systems is central to its adoption |
Perception Impact on time: Increase Driving perception: stand-along system, double data entry needed |
| Curry et al. 2011 [35] | Canada |
To explore two issues in the implementation of CDS for Diagnostic Imaging: - Will physicians incorporate decision-support technology into their clinical routines? - Will physicians follow clinical advice when Provided? |
Condition of focus: Diagnostic imaging Setting: Family Medicine clinic Tool: CDS to guide decisions to order imaging - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: GP - Risk score: No |
Mixed: quantitative analysis of useage data + qualitative interviews |
Quantitative—usage by clinicians Qualitative—perceived effects of taking part in the study and challenges |
The largest challenge was perceived interference with usual work flows, specifically the interactivity between EMR and the CDS (perceived to be too slow, although measured as less than 1 s). The time for physicians to interact with CDS was also perceived to be too long Time taken to use tool (described in Methods) 2 min 15 s as follows: - 1 min to enter data, 5 s for CDS to check if appropriate - < 1 min for clinician to look at recommendation if not appropriate - 10 s to complete DI order if still required |
Perception and objective measure of time to use tool showed conflict Impact on time: Increase Driving perception: slow software, workflow disruption |
| Dikomitis et al. 2015 [36] | UK | To collect data on the (non)use of electronic risk assessment tools (eRATs) and on the experiences of using the tool in everyday practice |
Condition of focus: Cancer Setting: General Practice Tool: eRATs - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: Yes |
Qualitative interviews |
Normalisation Process Model: (1) interactional workability (2) relational integration (3) skill-set workability (4) contextual integration |
Interactional workability—GPs' reactions to the on-screen prompts was influenced by different factors: - the approach of the doctor - the GP’s clinical experience - time pressures in specific consultations Contextual integration—most issues related generally to new interventions being implemented: - GPs expected to multi-task within one consultation - constant time pressures - prompt fatigue is a risk if added to an already 'busy screen' and alerts may not be responded to - concern over workload for secondary care |
Perception Impact on time: Increase Driving perception: lack of time |
| Duyver et al. 2010 [37] | Belgium |
To explore GPs' perceptions of feasibility and added value of the MDS-HC as a geriatric assessment tool and to investigate potential barriers and facilitating factors regarding the implementation of this tool in an ambulatory community setting |
Condition of focus: Geriatric care Setting: General Practice Tool: Home care assessment tool - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: GP - Risk score: No |
Quantitative: survey + free text |
Four assessment areas: (1) technical acceptability (2) clinical relevance of the tool (3) management and optimization of health care planning (4) valorisation of the role of the GP |
Free comments from GPs: - Long and fastidious encoding - CDS was too long, added admin workload - Excellent concept, worth making easier and shorter to use |
Perception Impact on time: Increase Driving perception: slow software |
| Eaton et al. 2012 [38] | USA |
To examine Abdominal Aortic Anneurysm (AAA) screening ordering in an academic primary care internal medicine clinic that uses physician reminders based on real-time CDS for preventive screening and to identify why screening ordering rates vary among providers |
Condition of focus: Abdominal Aortic Aneurysm (AAA) Setting: Primary Care clinic Tool: CDS for AAA screening - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: No |
Quantitative: records review |
Features of the first visit during the study: - visit date - visit type (general medical examination vs. other type) - provider role (staff physician vs. other) - provider gender - was AAA screening ultrasound was ordered during the visit |
Visit time (based on the fixed length of different visit types) is an important determinant for preventive screening Patients more likely to have screening ultrasound ordered during longer medical examinations, which usually has more time allotted (40 min) and often has a disease-prevention component During longer medical examinations, 24% of eligible patients had the recommended AAA screening ordered, compared with only 6% during shorter visits |
Objective measure of time for whole consultations (via proxy of visit type) Impact on time: Increase |
| Laforest et al. 2019 [39] | UK |
To review the tools available, clinician attitudes and experiences, and the effects on patients of genetic cancer risk assessment in general practice |
Condition of focus: Cancer Setting: General Practice Tool: range of risk assessment tools, including web-based, risk-stratification, and paper-based - Embedded/linked with EMR: Range - Interruptive alert: Range - User-driven: GP - Risk score: Range |
Systematic review |
1. What tests/tools are available to identify increased genetic risk of cancer in general practice? 2. What are clinicians’ attitudes towards Screening/testing population groups for genetic cancer risk? 3. What are the levels of patient knowledge, satisfaction, and anxiety in relation to tests and communication by a GP about cancer risk? 4. What are patients’ risk perceptions following screening/testing for genetic cancer risk in primary care? 5. What are the outcomes of referrals following genetic cancer risk identification in general practice? |
5 studies examined GP attitudes: - Owens et al. (43): some providers concerned over the time needed to counsel patients who were newly determined as at high risk, and regarding liability for not successfully providing risk counselling - Wu et al. (45): physicians at two primary care clinics felt they were already collecting high-quality family histories and that the tool would negatively impact their workflow. Physicians believed that patients would redirect discussions away from physician priorities and, instead, focus on tool recommendations. However, post-implementation, 86% of physicians found the tool improved their practice, and none reported adverse effects on workflow GPs were worried about the impact of risk assessment on patient anxiety, particularly if discussions with whole families would be required. GPs were concerned about their ability to explain risk and implications in short, routine appointments |
Perception Impact on time: Mixed views |
| Fathima et al. 2014 [40] | Australia |
To systematically review randomized controlled trials evaluating effectiveness of CDS in the care of people with asthma and COPD and to identify the key features of those systems that have the potential to overcome health system barriers and improve outcomes |
Condition of focus: Asthma & COPD Setting: Primary care Tool: range of CDS systems, including prevention & management, providing guidelines - Embedded/linked with EMR: Range - Interruptive alert: Range - User-driven: Clinician - Risk score: No |
Systematic Review (qualitative syntheses) |
Assessment of intervention effects 1. Type of CDS intervention 2. Effectiveness of CDS: - Clinical outcomes - Healthcare process measures - User workload and efficiency outcomes - Relationship-centred outcomes - Economic outcomes - Use and Implementation outcomes |
Workload and efficiency outcomes assessed included asthma documentation by ED doctors, consultation time, time for disposition decision in the ED, and user knowledge These were assessed as the primary outcome by two trials, of which one trial showed significant improvement in rate of asthma documentation. The other trial did not show any effect from the use of CDS on the time taken to make a disposition decision 83% of studies of CDS tools which were stand-alone in design had favourable clinical outcomes, compared with 38% of embedded designs. This may be due to alert fatigue and too low a threshold for alerts being generated Low evidence provided by studies re user workload & efficiency. One RCT in a hospital ED measured consultation time as a marker of workflow efficiency, finding no significant difference in consultation time between the intervention group compared with control |
Objective measure of time for whole consultations (1 study in a systematic review) Impact on time: neither increase nor decrease |
| Finkelstein et al. 2017 [41] | USA | To implement a comprehensive informatics framework to promote breast cancer risk assessment and chemoprevention in primary care that was informed by potential user feedback (usability testing to determine barriers and facilitators affecting the toolbox use by providers) |
Condition of focus: Cancer Setting: Primary care Tool: Breast cancer risk assessment & chemoprevention - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: Clinician & patient - Risk score: Yes |
Qualitative interviews | Ease of use, content, navigation | Ease of use: notifications were noted to be too time consuming to process |
Perception Impact on time: Increase Driving perception: Lack of time |
| Fox et al. 2014 [42] | USA | To evaluate adherence to an evidence-based Chronic Kidney Disease computer decision-support checklist in patients treated by Primary Care Physicians compared with usual care at a single site |
Condition of focus: Chronic Kidney Disease Setting: Primary care clinic Tool: CDS checklists for CKD - Embedded/linked with EMR: Unclear - Interruptive alert: No, guides visit - User-driven: Clinic staff - Risk score: No |
Quantitative records review | Clinical measures of CKD management |
Comment that the checklist was used to create a priority and incorporated into workflow so that CKD was treated appropriately. This is a step above a simple alert at the point of care and circumvented alert fatigue The 'extra time' needed for PCP to improve CKD care did not seem to adversely affect other areas of preventive care |
Perception Impact on time: Neither increase nor decrease Driving perception: did not take time away from other areas of preventive care |
| Gill et al. 2019 [43] | USA | To examine the impact of Point Of Care CDS on diabetes management in small- to medium- sized independent primary care practices that had adopted the PCMH model of care |
Condition of focus: Diabetes Setting: Primary care practices Tool: CDS for diabetes management - Embedded/linked with EMR: Yes - Interruptive alert: No, guides visit - User-driven: Automatic - Risk score: No |
Mixed: quantitative analysis of EMR data + qualitative interviews |
Quantitative: Clinical measures of diabetes management Qualitative: Barriers to and facilitators of successful implementation of CDS that achieved optimal diabetes management |
Barriers impeding implementation of the CDS included time and reimbursement in light of the need for time to implement team-based care, not specifically regarding the impact of the CDS on visit lengths |
Perception Impact on time: No conclusion |
| Green et al. 2015 [36, 44] | UK | To explore GPs’ experiences of incorporating Risk Assessment Tools (RATs) for lung and bowel cancers into their practice and to identify constraints and facilitators to the wider dissemination of the tools in primary care |
Condition of focus: Cancer Setting: General Practice Tool: paper-based RATs - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: Clinician - Risk score: Yes |
Qualitative interviews | GPs' experiences of the implementation process and their use of the RATs in practice | A minority of participants did not feel RATs added to practice: "I’m not sure it fits into the consultation in a natural way of making a decision about the management of that patient. It’s one more thing to fit into a busy ten minute consultation” |
Perception Impact on time: Increase Driving perception: workload already heavy |
| Gregory et al. 2017 [45, 46] | USA | To examine asynchronous alert-related workload in the EMR as a predictor of burnout in primary care providers (PCPs), in order to inform interventions targeted at reducing burnout associated with alert workload |
Condition of focus: Generic Setting: Primary care Tool: inbox-style EMR alerts - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: Clinician - Risk score: No |
Mixed: quantitative survey + focus groups |
Subjective alert workload (perception of time available to complete tasks) Objective alert workload (actual hours spent) Burnout (scale) |
Quantitative: subjective alert workload was positively related to 2/3 dimensions of burnout. Subjective alert workload was also generally predictive of burnout, whereas objective alert workload was not This suggest that it is the perception of alert burden that predicts burnout, rather than the actual amount of time spent attending to alerts Qualitative: time spent managing alerts was a major theme in focus group discussion and survey comments |
Perception and objective measure of workload showed conflict Impact on workload: Mixed views |
| Harry et al. 2019 [47] | USA | To identify adoption barriers and facilitators before implementation of CDS for cancer prevention in primary care |
Condition of focus: Cancer Setting: Primary care clinics Tool: CDS for cancer prevention & screening - Embedded/linked with EMR: Yes - Interruptive alert: No, guides visit - User-driven: Clinician - Risk score: Unclear |
Qualitative interviews |
1. Factors that facilitate or hinder key informant support for the intervention 2. Key informant knowledge and beliefs about the intervention and tension for change 3. The relative advantage(s) of the intervention compared with other interventions currently available in the EMR 4. Relevant organizational culture norms and values related to cancer prevention and screening 5. Factors that may foster adoption from a key informant perspective 6. Related external policies and incentives 7. Implementation climate |
PCP time limitations were a major concern. PCPs are being asked to do more with less time, including seeing more patients in a day, making some PCPs wonder how to fit the CDS into the visit It was perceived to be 5–10 min to use the tool, which would add time pressure as appointments are usually 'already 20 min behind' However, some same informants who mentioned time constraints also said that this would only be a limitation until PCPs learned the CDS tools |
Perception Impact on time: Increase Driving perception: Lack of time |
| Hayward et al. 2013 [48] | UK | To understand how GPs interact with prescribing CDS in order to inform deliberation on how better to support prescribing decisions in primary care |
Condition of focus: Prescribing Setting: General Practice Tool: CDS for prescribing - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: No |
Mixed: quantitative analysis of useage data + conversation analysis |
Timing of computer tasks and utterances Prescribing alerts and responses Conversation analysis |
Total mean duration of consultation: 9 min 3 s - time before prescribing = 5 min 47 s - during prescribing = 1 min 47 s - time after prescribing = 1 min 30 s Timing of alerts was problematic as they interrupt in order to correct decisions already made rather than to assist earlier deliberations. By the time an alert appears the GP will have potentially spent several minutes considering, explaining, negotiating, and reaching agreement with the patient, possibly given instructions, and printed information about treatment. An alert in the final seconds of the task increases the probability of it being ignored |
Objective measure of time of whole consultations Impact on time: neither increase nor decrease |
| Henderson et al. 2013 [49, 50] | UK |
To determine uptake of online diagnostic CDS and impact on clinical decision-making and patient management and to elicit users’ views of utility |
Condition of focus: Generic Setting: General Practice Tool: online diagnostic CDS - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: GP - Risk score: No |
Mixed: focus group + survey | Whether and how well the system had been embedded in everyday practice, based on the evidence available from the focus groups and post-use survey |
Low usage reported There was conflict at the organisational level, as agreement to participate in the study had been primarily by practice managers, while CDS use was to be by clinicians. This was linked to the major issue of no time having been identified for clinicians to use the system during a consultation Searches took 4 min within a 10 min consultation: ‘We have so many things thrown at us…the PCT telling us to do this and that you can get a little overwhelmed.’ (GP) |
Perception and objective measure of time to use tool Impact on time: Increase Driving perception: workload already heavy |
| Heselmans et al. 2012 [51] | Belgium | To assess users’ perceptions towards the implemented EBMeDS, the investigation of user interactions with the system and possible relationships between perceptions and use |
Condition of focus: Generic Setting: Family Practice Tool: CDS for a range of conditions - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: No |
Mixed: quantitative survey + qualitative interviews |
Qualitative: factors that may account for acceptance and use of EBMeDS Quantitative: computer-recorded user interactions with the system over evaluation period of 3 months to assess the actual use of the system |
Although majority of FPs were positive about the system, the most important reasons to neglect reminders related to the number of reminders and lack of time to read them (44%). However, 35% reported they could perform their tasks faster using the system Quantitative analysis of physicians’ log files:- Study measured number of seconds for a reminder to be closed after appearing, but this is only referred to in the discussion: - reminders open for less than 3 s were assumed to have been 'clicked away' (ie ignored) - 49% of alerts were open for < 2 s and 32.5% open for < 3 s which suggests that sensitivity threshold may have been too low |
Perception Impact on time: Mixed views |
| Hirsch et al. 2012 [52] | Germany | To evaluate the uptake of an interactive, transactional, and evidence-based library of decision aids and its association to decision making in patients and physicians in the primary care context |
Condition of focus: Range Setting: Primary Care Tool: CDS for a range of conditions, including CVD, AF, CHD, diabetes and depression - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: GP - Risk score: No |
Quantitative survey |
Which module was used and how detailed the steps of the shared decision making process were discussed using a four point scale Physicians were asked who made the decision at the end of the consultation, and for a subjective appraisal of consultation length (“unacceptably extended”, “acceptably extended”, “neither nor”, “shortened”) |
Subjective appraisal of consultation length: in 8.9% of consultations physicians said that they were “unacceptably extended” by the CDS, 76.3% of consultations were “acceptably extended”, 14.2% “neither nor”, and 0.5% were “shortened” Majority of physicians stated that the consultation length was either not extended or ‘acceptably’ extended Log files analysis reported average consulting time was 8 min, so use of CDS was therefore not extending the usual 10 min appointment slot |
Perception Impact on time: Mixed views |
| Holt et al. 2018 [53] | UK |
To identify the barriers to automated stroke risk assessment linked to invitations and screen reminders in primary care (AURAS-AF) |
Condition of focus: Stroke prevention in Atrial Fibriliation Setting: General Practice Tool: stroke risk assessment - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: Yes |
Mixed: quantitative analysis of useage data + interviews |
Quantitative: coded data indicating the responses to the screen prompts Qualitative: Researcher-led issues around AURAS-AF, and allowed people to express their own experiences and priorities |
Time available and the patient's own agenda dictated whether the alert was used to introduce the topic into the consultation. In some cases, GPs recognised that the timing was not right to initiate a discussion GP estimated the alert added 5–10 min more on the consult ('so you leave it') |
Perception Impact on time: Increase Driving perception: impact on conversation |
| Hoonakker et al. 2012 [54] | USA | To examine barriers, and possible improvements to a tool, HeartDecision (HD) |
Condition of focus: Cardiovascular disease Setting: Primary care Tool: cardiac risk assessment - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: GP - Risk score: Yes |
Mixed: quantitative time study + survey + qualitative interviews + observations |
A stop watch was used to measure the time the physicians spent on the different pages of the tool Survey: additional information about the need for such tools, use of the tool, barriers against its use, facilitators, and possible improvements |
The time study showed that on average, physicians spent 13 min using the tool, which is 'too long' for a regular patient visit, which lasts on an average 10 min |
Objective measure of time to use tool Impact on time: Increase |
| Kortteisto et al. 2012 [55] | Finland | To assess and describe in depth the specific reasons for HPs using or not using the eCDS in primary care |
Condition of focus: Generic Setting: Primary care clinic Tool: CDS for a range of conditions - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: No |
Mixed: focus group + survey |
Focus groups: general ideas about the eCDS, experiences of the use, practical problems, advantages /disadvantages for work, barriers to use and facilitators, and development issues Survey: system’s capacity and quality, as well as its perceived usefulness and ease of use |
Common barrier was busy practice in primary care ‘When I am busy, I don’t look for anything really.’ ‘Nothing more than simply doing what I have to do.’ Within 'functionality', majority of clinicians reported it was 'rapid enough' Within ‘usefulness’, drug alerts 'motivate' but 'take time' and 'requires more time for paperwork' |
Perception Impact on time: Mixed views |
| Krog et al. 2018 [56] | Denmark | To explore facilitators and barriers to using the eMDI in psychometric testing of patients with symptoms of depression in Danish general practice |
Condition of focus: Depression Setting: General Practice Tool: CDS for diagnosis/monitoring of patients with depression - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: GP - Risk score: Yes |
Qualitative interviews | Determinants for using the eMDI in relation to the GPs’ capability, opportunity and motivation to change clinical behaviour |
eMDI was a 'timesaver' compared to the paper version because of cutting out need for data entry or printing and scanning which frees up time for other tasks, e.g. more time for dialogue in the consultation (i.e. 'better' consultations through improved use of consultation time and prioritisation of GPs' time) However, for some interviewees, time and efficiency aspects have worked as a barrier because of lack of time to change routines and experiences with the eMDI as being too time-consuming when filled in during the consultation |
Perception Impact on time: Mixed view |
| Lafata et al. 2016 [57] | USA |
To evaluate the association of exam room use of EMRs, HRA tools, and self-generated written patient reminder lists with patient–physician communication, recommended preventive health service delivery, and visit length |
Condition of focus: Generic Setting: Primary care Tool: range of tools, including EMR tools, risk assessment tools and written patient lists - Embedded/linked with EMR: Range - Interruptive alert: Range - User-driven: Range - Risk score: n/a |
Quantitative: observational |
1. Visit length (face-to-face interaction time in minutes between patients and physicians); 2. Patient engagement communication behaviour; 3. Physician–patient-centred communication behaviour; and 4. Physician delivery of evidence-based preventive health services |
On average, physicians spent almost 27 min with the patient (SD = 10 min) Mean visit length was longer for patients who used a self-generated written reminder list compared to patients who did not use such a list (30.0 vs. 26.5 min). Visit length was also significantly longer when the EMR was accessed in the exam room compared to those visits in which the EMR was not accessed in the exam room (27.7 vs. 23.9 min) Visits that included exam room–based use of the EMR lasted, on average, just over 3 min more than visits in which the EMR was not accessed in the exam room The use of a HRA instrument was not associated with increased visit length, but was not associated with decreased length either |
Objective measure of time of whole consultations Impact on time: neither increase nor decrease |
| Litvin et al. 2012 [58] | USA |
To describe use of the CDS, as well as facilitators and barriers to its adoption, during the first year of the 15-month intervention |
Condition of focus: Prescribing Setting: Primary care practices Tool: CDS for antibiotic prescribing for Acute Respiratory Infections - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: Physician - Risk score: No |
Mixed: quantitative analysis of EMR data + qualitative interviews and observations |
Using EMR data, CDS use was calculated at the practice level as the number of encounters at which an ARI diagnosis (or multiple diagnoses) using the CDS was made divided by the number of all encounters at which an ARI diagnosis was made, regardless of CDS use Qualitative data were recorded during practice site visits using a structured site visit field note Group interviews with staff elicited facilitators and barriers of CDS adoption |
Organisational factors: - Barrier: 'use of CDS alters workflow' - Facilitator: perception that CDS speeds up the visit and shortens documentation time. Others felt that the CDS did not affect the length of the visit. None reported that the CDS slowed the visit |
Perception Impact on time: Mixed view |
| Lugtenberg et al. 2015 [59] | Netherlands | To investigate the exposure to and experiences with the CDS quality improvement intervention, to gain insight into the factors contributing to the intervention’s impact |
Condition of focus: Generic Setting: General Practice Tool: CDS for range of activities, including patient data registration, prescribing and management - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: No |
Mixed: quantitative analysis of usage data + survey + qualitative interviews |
Quantitative: - NHGDoc data to measure exposure to the intervention in both study groups - Survey data on exposure to and experiences with the CDS intervention Qualitative: range of barriers that GPs and PNs perceive in using NHGDoc or similar CDS in practice |
Survey: - Limited time available during and after consultation (60% of GPs and 16% of PNs) - Too much additional work required during and after consultation (60% of GPs, 27% of PNs) |
Perception Impact on time: Increase Driving perception: Lack of time |
| Lugtenberg et al. 2015 [59–61] | Netherlands |
To identify perceived barriers to using large-scale implemented CDS, covering multiple disease areas in primary care |
Condition of focus: Range Setting: General Practice Tool: CDS for range of conditions, including CVD, asthma/COPD, diabetes, thyroid disorders, viral hepatitis, AF, subfertility, gastro protection and chronic renal failure - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: No |
Qualitative focus groups | Value of CDS in a primary care setting, CDS in an ideal world, experiences with using CDS with the example of NHGDoc, perceived advantages and disadvantages, and barriers to using them in practice |
The system’s responsiveness was a problem, with the loading of alerts taking too long Many physicians mentioned that using CDS has a negative effect on patient communication during consultation and is considered a barrier to their use Discrepancy between a patient’s reason for visiting and the alert content was a reason not to use it Limited time during consultations made it difficult to use the CDS, as well as the additional work it requires |
Perception Impact on time: Increase Driving perception: slow software, workload already heavy |
| Pannebakker et al. 2019 [62] | England |
To understand GP and patient perspectives on the implementation and usefulness of the eCDS |
Condition of focus: Cancer Setting: General Practice Tool: - Type: CDS for melanoma - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: GP - Risk score: No |
Qualitative interviews | GP and patient perspectives | Some reflected on how using CDS did not intrude in a consultation, and that it could help with saving time during or after a consultation |
Perception Impact on time: Decrease Driving perception: efficiency, reduced time needed for data entry |
| Peiris et al. 2009 [63] | Australia | To develop a valid CDS tool that assists Australian GPs in global CVD risk management, and to preliminarily evaluate its acceptability to GPs as a point-of-care resource for both general and underserved populations |
Condition of focus: Cardiovascular disease Setting: General Practice Tool: CDS for CVD risk management - Embedded/linked with EMR: No - Interruptive alert: Yes - User-driven: GP - Risk score: Yes |
Mixed: quantitative survey + qualitative interviews |
Survey: GP attitudes about the tool and management provided Interviews: general attitudes about the tool and its impact on the consultation; a review of specific tool outputs; recommendations for future tool development |
Challenges: - Time pressures introduced by incorporating CVD risk management into routine care - Extra work seen in cases where the GP didn't expect CVD risk to be high, but these instances were few - Future automation of the tool (e.g. pre-population with data) seen as important Recommendations - ‘Too wordy' to read whilst with patient. For a consultation, 'you've got 15 min at most' |
Perception Impact on time: Neither increase nor decrease Driving perception: work was increased only where risk was unexpectedly high, but this was not often |
| Rieckert et al. 2018 [64] | Germany |
To examine how GPs experienced the PRIMA-eDS tool, how GPs adopted the recommendations provided by the CMR, and explore GPs’ ideas on future implementation |
Condition of focus: Prescribing Setting: General Practice Tool: CDS to prevent inappropriate medication in older populations - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: GP - Risk score: No |
Qualitative interviews |
1. Polypharmacy in everyday practice 2. Using the eCRF 3. General overview of the comprehensive medication review 4. Output of the CMR and how GPs responded to the recommendations 5. Implementation of the tool into daily practice routine |
Entering patient data into the eCRF was time-consuming. After a period of familiarisation utilisation became easier and faster ‘For the first one I took 45 min I think and in the end it took me ten minutes’. (GP 14) Retrieving additional information provided by the tool was perceived as being too time-consuming |
Perception Impact on time: Mixed views |
| Rickert et al. 2019 [65] | Germany |
To examine how GPs experienced the PRIMA-eDS tool, how GPs adopted the recommendations provided by the CMR, and explore GPs’ ideas on future implementation |
Condition of focus: Prescribing Setting: General Practice Tool: CDS to prevent inappropriate medication in older populations - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: GP - Risk score: No |
Quantitative survey | Use of and attitudes toward the CMR, its recommendations, and future use |
Prerequisites for the future use of the PRIMA-eDS tool: Technical limitations were rated by 93% of GPs as important for future use of PRIMA-eDS, data security by 86%, and time requirement by 85% DISCUSSION: Previous research has shown that physicians ignored alerts when this was not the reason for the patients’ visit, as often there was not enough time to deal with both |
Perception Impact on time: Unclear |
| Robertson et al. 2011 [66] | Australia | To determine GPs’ access to and use of electronic information sources and CDS for prescribing |
Condition of focus: Prescribing Setting: General Practice Tool: CDS for prescribing - Embedded/linked with EMR: - Interruptive alert: - User-driven: GP - Risk score: No |
Qualitative interviews |
Electronic resources/CDS: - advantages and disadvantages of electronic over paper-based resources, - valued features of electronic decision support systems, - features of alerts and reminders (content, presentation and perceived usefulness), - support and training needs |
GPs mentioned the pressures of a 10- to 15-min consultation, that their information needs were immediate at the point of care. GPs wanted relevant information presented concisely, easily searchable, integrated in the workflow and embedded in clinical software (the need to logon or go outside the main programme was seen as a burden and time waster) |
Perception Impact on time: Unclear |
| Sperl-Hillen et al. 2018 [67] | USA |
To evaluate whether the CDS intervention can improve 10-year CVD risk trajectory in patients in primary care setting |
Condition of focus: Cardiovascular disease Setting: Primary Care Tool: assessment of CV risk - Embedded/linked with EMR: Yes - Interruptive alert: - User-driven: GP - Risk score: Yes |
Quantitative: analysis of EMR + survey |
Primary: CV risk values and clinical impact of the CDS system Secondary: Pre- and post (18 m) survey of Primary Care physicians: - Confidence and preparedness to address CV risk with patients - Satisfaction and perceptions with the CDS |
PCPs reported that the CDS helped them to initiate discussions about CV risk (94%), improved CV risk factor control (98%), saved time when talking about CV risk with patients (93%), enabled efficient elicitation of patient treatment preferences (90%), supported shared decision making (95%), and influenced treatment recommendations (89%) |
Perception Impact on time: Decrease Driving perception: saved time in conversations with patients |
| Sperl-Hillen et al. 2019 [68, 69] | USA | To evaluate improvements to clinical outcomes, impact on clinic workflow, use of CDS and satisfaction among clinicians |
Condition of focus: Chronic disease Setting: Primary Care clinics Tool: CDS for chronic disease management & preventive care - Embedded/linked with EMR: Yes - Interruptive alert: - User-driven: GP - Risk score: |
Quantitative: analysis of EMR, CDS useage data, survey of clinicians |
Clinical outcomes Impact on clinic workflow CDS use rates Clinician satisfaction |
93 percent reported it saved time when talking to patients about CV risk factor control |
Perception Impact on time: Decrease Driving perception: saved time in conversations with patients |
| Sukums et al. 2015 [70, 71] | Sub-Saharan Africa | To describe health workers’ acceptance and use of the eCDS for maternal care in rural primary health care (PHC) facilities of Ghana and Tanzania and to identify factors affecting successful adoption of such a system |
Condition of focus: Antenatal and intrapartum care Setting: Primary Health Care clinics Tool: - Type: CDS for antenatal and intrapartum care - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: Clinician - Risk score: No |
Mixed: quantitative survey + interviews |
Perceived challenges affecting the eCDS use through a mid-term- and post- survey at 10 months (midterm) and 18 months (final) after implementation Interviews with the care providers were conducted to explore their views and experiences with the eCDS |
Perceived increase in workload due to the eCDS use reported About one third of providers indicated a lack of time to use the eCDS Reasons given for these challenges: inadequate computer skills, inadequate staffing during busy periods Perceived workload also increased due to simultaneous manual and electronic documentation, which some providers felt to disrupt their work |
Perception Impact on time: Increase Driving perception: stand-alone data entry, workload already heavy |
| Trafton et al. 2010 [72, 73] | USA | To evaluate the usability of ATHENA-OT, and to identify key needs of clinicians for both integrating the CDSS into their workflow and for opioid prescribing in general |
Condition of focus: Prescribing Setting: Primary Care Tool: CDS for use of opioid therapy for chronic, non-cancer pain - Embedded/linked with EMR: - Interruptive alert: - User-driven: Clinician - Risk score: No |
Mixed: quantitative and qualitative observations, survey, interviews and usage data |
Usability of ATHENA-OT Key needs of clinicians |
Qualitative: Many competing time constraints limit use of a CDS for OT. While the CDS streamlines and facilitates practices recommended in the CPG, they still require time to complete Quantitative survey: ATHENA-OT system was rated lowest on expectations that it would save time in visits Provider Shadowing: Clinic visits varied from 13 to 59 min and averaged 31 min. 10/35 visits involved the ATHENA-OT. In these 10 visits, the time ATHENA-OT was used ranged from 3 s to 10 min Clinicians appeared to have reasonable time to use the system. This contradicts clinicians’ self-reported lack of time during visits, reflecting either non-representativeness of the visits observed, or exaggeration of time constraints by clinicians |
Perception and objective measure of time showed conflict Impact on time: mixed views |
| Trinkley et al. 2019 [74] | Canada |
To describe current clinician perceptions regarding beneficial features of CDS for chronic medications in primary care |
Condition of focus: Prescribing Setting: Primary Care Tool: CDS for prescribing chronic medications - Embedded/linked with EMR: - Interruptive alert: - User-driven: Clinician - Risk score: No |
Qualitative focus groups |
Beneficial CDS features for chronic medication management in primary care Participants' ideal CDS for chronic medications Potential unintended consequences of the CDS |
Main beneficial features of alerts: (1) non-interruptive alerts; (2) clinically relevant and customisable support; (3) summarisation of pertinent clinical information and (4) improving workflow Alerts were “one more thing to get through” and a barrier to completing tasks. Clinicians reported ‘alert fatigue’, with an overwhelming number of alerts for ‘every patient’ While not universally endorsed, some indicated they liked one alert for lung cancer screening and found it helpful, because it interrupted workflow at the right time No consensus regarding best timing of an interruptive alert. Roughly equal numbers preferred the to alert at: (1) opening of an encounter; (2) ordering or reviewing a medications; (3) entering a diagnosis or (4) at the end of the encounter |
Perception Impact on time: Unclear |
| Voruganti et al. 2015 [75] | Canada | To investigate current practices for assessing risk, awareness and use of risk assessment tools in primary care, and to assess PCPs’ perspectives regarding the usefulness, usability and feasibility of implementing computer-based health risk assessment tools into routine clinical practice |
Condition of focus: Chronic disease Setting: Primary Care Tool: risk assessment for chronic diseases - Embedded/linked with EMR: - Interruptive alert: - User-driven: Clinician - Risk score: |
Qualitative focus groups | PCPs' awareness of risk assessment tools, and views on their usefulness, usability and feasibility of routinely using them in clinical practice |
Perceived benefits and shortcomings of tools: - beneficial for initiating discussion, engaging patients in risk discussions, and guiding decision-making by physicians and patients - concern about impact on workflow (“it might bring up a lot more other issues that they [patients] weren’t originally aware of and the discussion might actually… be less directed”) - some felt differently, that “it usually stops a lot of the meandering dialogue that you’d otherwise engage in” Expectations of an ideal risk assessment tool: - number of steps to complete a risk assessment should be minimised to a few clicks |
Perception Impact on time: Mixed view |
| Walker et al. 2017 [76, 77] | Australia |
To examine useability and acceptability of a prototype tool ‘CRISP’ (Colorectal cancer RISk Prediction tool), identify barriers and enablers to implementing CRISP in Australian general practice, and optimize the design of CRISP prior to an RCT |
Condition of focus: Cancer Setting: General Practice Tool: risk assessment for colorectal cancer - Embedded/linked with EMR: - Interruptive alert: - User-driven: Clinician - Risk score: |
Qualitative: simulated consultations + interviews |
Acceptability, usability and implementation strategies were explored at an individual level (GP, PN and PM) and organizational level (the practice) using the four domains of NPT: - Coherence - Cognitive participation - Collective action - Reflexive monitoring |
Collective action - GPs, PNs and PMs all agreed that lack of GP consultation time would limit the use of CRISP by GPs - Consensus that nurses have the capacity, time and expertise to complete the risk assessment as part of routine preventive health consultations - Opinions about who would take responsibility for the final decision about screening advice was split between GPs and PNs; many GPs feared missing a diagnosis |
Perception Impact on time: Increase Driving perception: lack of time |
| Zazove et al. 2017 [78] | USA |
To develop a model electronic alert that integrates into system 1 thinking (thinking that is fast and intuitive, often occurring without much conscious thought), which family medicine clinicians would use to improve identification of individuals at risk for HL |
Condition of focus: Hearing loss Setting: Family Medicine clinic Tool: risk assessment for hearing loss - Embedded/linked with EMR: - Interruptive alert: - User-driven: Clinician - Risk score: |
Mixed: cognitive task analysis interviews |
How often various issues were identified, the root causes of use and non-use of the electronic alert, sample quotes highlighting major issues, and any potential solutions mentioned |
Time pressure with electronic prompts: - Clinicians felt visits were already overloaded, limiting their ability to handle additional alerts. Addressing all recommendations for complex patients requires more than the typical 15-min office visit - Alerts intrude on the doctor-patient relationship, since they rarely address the primary reason for the visit, and the added workload contributes to clinician stress due to falling further behind in the schedule Hearing loss is not easily addressable: - the “time pressure” was due to clinicians being uncomfortable and having to take time to think about how to address HL (ie, being forced into slow and effortful system 2 thinking), which they did not have to do with other conditions |
Perception Impact on time: Increase Driving perception: workload already heavy |
| Murdoch et al. 2015 [79, 80] | UK | To use conversation analysis to assess the interactional workability of using CDS for telephone triage |
Condition of focus: Same-day appointment requests (range of conditions) Setting: General Practice Tool: CDS to guide nurse-led telephone triage - Embedded/linked with EMR: Yes - Interruptive alert: No, guides triage call - User-driven: Nurse - Risk score: No |
Qualitative: conversation analysis |
1. Structuring of patient's problem 2. making sense of/managing pt's symptoms in the CDS 3. where pts' experience misaligned with CDS requirements 4. nurse accountability within CDS 5.Consequences of not using CDS on Qs and answers |
Use of CDS impacted call 'trajectory' and caused disruptions 'interactional workability' of using the CDS for telephone triage Eg. Operational problems such as mistyping a symptom or condition led to a 'prolonged pause' while nurse attempted to correct/search around for another term and explain the delay to the patient |
Objective measure of time of whole consultations but timings data not reported Impact on time: unclear |
| Jetelina et al. 2018 [81] | USA |
Proof of concept study: before and after implementation of e-tool |
Condition of focus: Behavioural Health Setting: Primary care clinics Tool: suite of e-tools for Behavioural Health Clinicians - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: Clinician - Risk score: No |
Mixed: quantitative surveys + qualitative interviews and observations |
Clinical outcomes and patient experience Acceptability of the e-tools Factors influencing implementation |
Acceptability: - Tool was acceptable and easy to use - Tool added 1 to 2 min to the initial visit but time during follow-up visits by automatically populating the history of the presenting illness and patient instructions at subsequent visits |
Perception Impact on time: Decrease Driving perception: efficiency, reduced time needed for data entry |
| McGinn et al. 2013 [82] | USA | To examine the effect of the tools on diagnostic and treatment patterns and to assess adoption of the tool for each condition |
Condition of focus: Upper Respiratory Tract Infections Setting: Primary care clinics Tool: CDS with clinical prediction rules for 2 URTIs - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: Yes |
Quantitative: analysis of EMR and usage data |
Usage data: number of visits involving: - tool being opened once triggered - calculator being completed - viewing of recommendations and - following of recommendations |
Time not measured or commented on Regarding diagnostic/treatment/management patterns, no significant differences between arms in proportions of visits resulting in patient returning to ED/Outpatient clinic for follow-up High adoption rates reported |
Objective workload measure of follow-up visits Impact on workload: neither increase nor decrease |
| Litvin et al. 2016 [83] | USA | To assess impact on CKD clinical quality measures and facilitators/barriers to use of tools |
Condition of focus: Chronic Kidney Disease Setting: Primary care clinics Tools: CDS tools for CKD, including risk assessment - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: Yes |
Mixed: quantitative analysis of EMR data + qualitative observations and interviews |
Quantitative: CKD clinical quality measures at baseline & 2 years Qualitative: Barriers/facilitators to using tools |
Barriers included mention by some staff that CDS tools required 'extra clicks' and additional steps outside of existing workflow |
Perception Impact on time: Increase Driving perception: workflow disruption |
| Linder et al. 2009 [84] | USA | To assess effect of CDS on antibiotic prescribing rates for ARI visits |
Condition of focus: Acute Respiratory Infections Setting: Primary care clinics Tool: CDS for ARIs to help reduce inappropriate prescribing - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: Yes |
Quantitative: analysis of EMR data |
Primary: Rate of antibiotic prescribing for ARI visits Secondary: 30-day re-visit rates attributable to ARIs |
Recorded duration of Smart Form use (assume mean) 8.1 m, sd 5.8 m. However, whole visit length not captured/reported and not compared with Usual Care Re-visit rate attributable to ARIs 8% in intervention and 9% in control but not remarked upon in Discussion or quantified as significant or not significant Remarked in discussion that to be effective, CDS must fit as seamlessly as possible into existing workflow and allow clinicians to manage unanticipated interruptions |
Objective measure of time to use tool and revisit rates Impact on time: Unclear Revisit rates not remarked upon |
| Ranta 2013 [85] | New Zealand | To assess feasibility of introducing risk tool to help timely management of TIAs |
Condition of focus: TIA & Stroke Setting: General Practice Tool: CDS for TIA & Stroke risk - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: GP - Risk score: Yes |
Quantitative: analysis of usage data + survey |
Usage of the tool and advice rendered/actions taken by GPs Post-pilot satisfaction of GP users |
GP survey interviews reported 'no major concerns' regarding the time required to enter data (Methods report this to be 3-5 min per TIA patient) Time not formally measured and no indication given of whether time is added to the consultation (perhaps 'acceptably') |
Perception Impact on time: Unclear |
| Price et al. 2017 [86, 87] | Canada | To examine how 40 STOPP rules could be implemented as alerts into practice and the impact on prescribing |
Condition of focus: Prescribing Setting: Family Practices Tool: Screening tool of older people's prescriptions (STOPP) - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Physicians - Risk score: No |
RCT with mixed methods: quantitative analysis of EMR data + interviews |
Quantitative: change in rate of potentially inappropriate prescriptions (PIPs) between arms Qualitative: views on barriers and facilitators of implementation |
Qualitative interviews - 'workflow' cited as a barrier to implementation, along with 'location' of alert on-screen - Workflow not expressed in terms of time (increase or decrease) or sequence of activity and not quantified or measured as time |
Perception Impact on time: Unclear |
| Wan et al. 2010 [88] | Australia | To explore GPs and patients' views of implementing CVAR assessment, including issues regarding identifying patients at risk and the timing and context for assessment |
Condition of focus: Cardiovascular disease Setting: General Practice Tool: Range of electronic and paper-based tools for cardiovascular absolute risk assessment - Embedded/linked with EMR: Unclear - Interruptive alert: Unclear - User-driven: Unclear - Risk score: Yes |
Qualitative interviews | Views of barriers and facilitators of implementing CVAR assessment | No specific e-tool examined and time not mentioned, but a general comment in Discussion that GP workload pressure is an important barrier to increasing preventive activity in general |
GP workload pressure Impact on time/workload: no conclusion |
| Hor et al. 2010 [89] | Ireland | To assess prevalence and use of EMR and any form of CDS for prescribing and to explore perceived benefits of future introduction of CDS-eP, barriers to implementation and presumptive responses to prescribing alerts |
Condition of focus: Prescribing Setting: General Practice Tool: Hypothetical CDS for e-prescribing - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Unclear - Risk score: No |
Quantitative: survey + free text |
Prevalence and use of EMR and any form of CDS for prescribing GPs' perceived benefits of future introduction of CDS-eP, barriers to implementation and presumptive responses to prescribing alerts |
Time not measured but mentioned in GPs' general comments: hypersensitive interruptive alerts whilst prescribing and causing a delay to prescribing by e.g. 20 s would be frustrating |
Perception Impact on time: Increase Driving perception: workload already heavy |
| Troeung et al. 2016 [90] | Australia | To develop and evaluate performance of the e-screening tool against practice EMR data |
Condition of focus: Familial hypercholesterolaemia Setting: General Practice Tool: screening tool to identify patients with familial hypercholesterolaemia - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: GP - Risk score: Yes |
Quantitative: analysis of EMR and tool data | Performance of the e-screening tool | Time reported (10 min) to run the e-screening search, not consultation length |
Objective measure of time to use tool Impact on time: Unclear |
| Jimbo et al. 2013 [91] | USA | To identify PCPs' perceived barriers and facilitators to implementation |
Condition of focus: Cancer Setting: General Practice Tool: CDS for patients regarding their colorectal cancer screening preferences, linked to computerised reminder alerts for clinicians - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Patient - Risk score: No |
Qualitative focus groups | Barriers and facilitators to implementation |
Clinicians (majority) identified principal barriers to patients' access to colorectal cancer screening and patients' concerns regarding costs, but also time constraints during visits to discuss the screening options (e.g. could mean a 5–10 min conversation)—the web-based tool completed by patients prior to their visit could potentially save time at the visit Some concerns over how clinician reminder alerts would fit into the usual visit workflow, but not expanded upon |
Perception Impact on time: mixed view |
| Akanuwe et al. 2020 [92] | UK | To explore the views of service users and primary care practitioners on how best to communicate cancer risk information when using QCancer, a cancer risk assessment tool, with symptomatic individuals in primary care consultations to enable them be involved in decisions on referral and cancer investigations |
Condition of focus: Cancer Setting: General Practice Tool: Qcancer risk assessment tool - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: Yes |
Qualitative interviews and focus groups |
Personalising risk information Informing and involving patients Being open and honest Providing time for listening, explaining and reassuring in the context of a professional approach |
‘Talking about risk is quite difficult’ - GP may be reluctant to inform the patient about cancer risk when they themselves were uncertain about the risk calculated or how to communicate this Patients: - GPs should take time to talk to patients to gain their confidence and show they care: 'You wouldn't want to feel that you've been rushed’ - ‘GPs would need more time to use the tools in consultations' GPs: GPs expressed the need to provide more time to provide explanations to patients |
Perception Impact on time: Increase Driving perception: time needed to use the tools, convey information and show caring |
| Bangash et al. 2020 [93] | USA | To develop a CDS tool for Familial Hypercholesterolemia based on physician feedback from qualitative interviews, usability testing and an implementation survey |
Condition of focus: Familial Hypercholesterolemia Setting: Primary Care Tool: CDS for FH - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: GP - Risk score: Yes |
Mixed: Qualitative interviews, usability testing + survey |
The most common barrier is the increasing cognitive burden on providers due to EMR complexity and limited time during clinical encounters Most physicians are receptive towards CDS. The only survey item where the majority of the physicians gave either a neutral response or disagreed was regarding the CDS tool ‘not’ increasing time spent with a patient This response reiterated the need for CDS to be designed to increase e ciency and not add to provider burden |
Perception Impact on time: Increase Driving perception: workflow disruption, limited time during consultations |
|
| Bradley et al. 2021 [94] | UK | To synthesise qualitative data of GPs’ attitudes to and experience with a range of CDs to gain better understanding of the factors shaping their implementation and use |
Condition of focus: Cancer Setting: Primary Care Tool: Range of CDS tools for cancer - Embedded/linked with EMR: Range - Interruptive alert: Range - User-driven: Range - Risk score: Yes |
Systematic review—qualitative synthesis |
Impact of CDS on role of GP - communicating risk - collaboration with secondary care/guidelines - nature of training provided Elements determining GPs' use - clinical acumen v protocol - medicolegal issues Impact on GPs work - increasing awareness of cancer - prompt fatigue - impact of IT integration (time) - time as a resource GPs' reflections - unintended consequences - investigation and referral patterns - 'think cancer' |
Prompt fatigue - Prompt fatigue was mentioned by several studies. The interruptions impacted on the flow of the consultation The prompts were regarded, in some studies, as making work more difficult; another commented on the usefulness of prompts for future consultations Impact of IT integration - ‘It was not easy accessing the tools during patient consultation’ Time as a resource - Recognition of the benefits of using a CDS was essential to justify the additional time required for its use. This impacts on consultation, time required to train users and the additional effort to continue using the CCDT. Time is at a premium in general practice in the UK: the pressures of the 10 min appointment, to keep up to date and to attend training |
Perception Impact on time: Mixed views Driving perception: time consuming to complete, limited time during consultations |
| Breitbart et al. 2020 [95] | Germany | To assess how CDS vs standard consultations affect patient satisfaction, diagnostic accuracy and length of consultations |
Condition of focus: Skin conditions Setting: General Practice Tool: Visual CDS for dermatology consultations - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: GP - Risk score: No |
Randomised feasibility study |
Patient experience and satisfaction Diagnostic accuracy Consultation length |
The median duration of the consultations using the CDS was 10 min, similar to that in the standard arm In the CDSS arm, overall patient satisfaction correlated negatively with increased duration of consultation (P = 0.02) Across both arms younger patients (20-40y) were more bothered about consultation length relative to the older patients Overall, the CDSS increased aspects of patient satisfaction, improved diagnostic accuracy without influencing the duration of the consultation |
Objective measure of time of whole consultations Impact on time: neither increase nor decrease |
| Byrne et al. 2015 [96] | Ireland | To benchmark the awareness and use of Risk Assessment tools and CVD prevention guidelines along with barriers to their use among a sample of Irish GPs |
Condition of focus: CVD Setting [16, 97]: General Practice Tool: CVD risk assessment tools - Embedded/linked with EMR: - Interruptive alert: - User-driven: GP - Risk score: |
Cross-sectional survey of GPs |
Demography Risk assessment CVD guideline use Perception of barriers to use of Risk Assessment tools and CVD guidelines |
Main three barriers to use of Risk Assessment tools: 1) patients focused on a single risk factor and not global picture (32.9%) 2) time constraints (30.6%) 3) not being used to using a risk calculator (18.4%) Barriers to implementation of CVD prevention guidelines: - lack of remuneration (40.8%) - too many CVD guidelines (38.9%) - time constraints (35.7%) |
Perception Impact on time: Increase Driving perception: limited time during consultations |
| Caturegli et al. 2020 [98] | South Africa | To pilot a prescribing tool |
Condition of focus: Tuberculosis Setting: Primary Care clinics Tool: prescribing tool for TB preventive therapy - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: Clinician - Risk score: No |
Mixed methods |
Prescribing rates Perceived barriers to prescribing Workload Stock-outs Prescription guidelines Intervention impact Cognitive load Documentation |
Workload - Reduces the writing one has to do Documentation - According to five of eight providers, time spent documenting medications and contraindications was reduced with the tool |
Perception Impact on time: Decrease Driving perception: reduced time spent documenting medications and reduced cognitive load |
| Chadwick et al. 2017 [99] | UK | To evaluate feasibility and acceptability of a prototype application of a risk stratification algorithm incorporated into a CPOE and triggering a prompt to offer an HIV test when the healthcare worker is ordering other tests |
Condition of focus: HIV Setting: Hospitals and general practices Tool: HIV testing prompt within a Computerised Physician Order Entry system - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: No |
Qualitative: Interviews and focus groups |
Frequency and appropriateness of the prompt The prompt in the context of the consultation Reactions of patients to the prompt Impact of the prompt on HIV testing |
Frequency and appropriateness of the prompt - Little evidence of “prompt fatigue”. This particular prompt, compared with other prompts, was considered simple to understand and easy to manage The prompt in the context of the consultation - Most discussed blood tests and submitted an order with the patient present. Some GPs ordered the test after the patient had left and were faced with the dilemma of whether to bring the patient back to discuss HIV testing - Many hospital-based and general practice HCWs felt the prompt was too late in the ordering process and disrupts the consultation, potentially opening up a new topic, causing irritation |
Perception Impact on time: Mixed views Driving perception: Either: - no mention of time - prompt causes test to be ordered after consultation - disruptive if prompt comes too late in the ordering process - potentially opens up a new topic right at the end of the consultation |
| Chadwick et al. 2021 [100, 101] | UK | To evaluate a prototype application designed to prompt in real-time, BBV testing in previously untested higher risk individuals attending primary care |
Condition of focus: Blood-borne viruses Setting: General practice Tool: CDS to identify patients at risk of blood-borne viruses - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: Yes |
Prospective cohort study |
Number of 'hard prompts' and clinicians' responses BBV tests ordered Survey of GPs |
Clinicians’ perceptions of the prompt system were positive with average additional time required for BBV test discussion in consultations estimated at 2 min Nineteen percent of clinicians reported having to make an additional appointment after a BBV test prompt because of insufficient time during a consultation and 15% had to make an additional appointment to discuss test results Free-text answers stressed the lack of time available Median additional consultation time varied from 0.25 min when the clinician ignored the prompt to 2 min when the prompt was accepted or declined |
Perception Impact on time: Increase Driving perception: limited time during consultations |
| Chima et al. 2019 [102] | UK |
To summarise existing evidence on the effects of eCDSTs on decision making for cancer diagnosis in primary care, and determine factors that influence their successful implementation |
Condition of focus: Cancer Setting: General practice Tool: Range of CDS tools to support cancer diagnosis - Embedded/linked with EMR: Range - Interruptive alert: Range - User-driven: Range - Risk score: Range |
Systematic review |
Appropriateness of care (n = 5); Diagnostic accuracy (n = 1); Time to diagnosis (n = 1); Cost-effectiveness (n = 1); Process measures (n = 1); and Qualitative (n = 4) |
Introducing the tool disrupted the consultation, with GPs reporting feeling a loss of control Additional tasks and time pressures impacted clinical flow For eCDSTs designed for use during consultation, there were challenges due to disruption of the usual workflow and the generation of additional tasks in an already-busy appointment |
Perception Impact on time: Increase Driving perception: limited time during consultations, disruption, loss of control, additional tasks and time pressures, impact on clinical flow |
| Dobler et al. 2019 [103] | USA | To determine clinician outcomes in RCTs of encounter decision aids for Shared Decision Making |
Condition of focus: Range Setting: General practice Tool: Range of CDS tools to support shared decision making of screening/treatment options in consultations - Embedded/linked with EMR: Range - Interruptive alert: Range - User-driven: Range - Risk score: Range |
Systematic review |
Clinician satisfaction - clinical encounter - decision-making process - the decision aid - the decision made Efficiency - consultation length Personal and professional well-being - mood and burnout - satisfaction with the practice of clinical care |
Clinician satisfaction - Communication was enhanced by providing visual representations of choices, reduced clinicians’ burden to produce accurate representations, giving clinicians more time to engage in meaningful discussions with patients - Clinicians' concerns included that decision aids would add time to their clinics if they were not simple Efficiency - One study showed that 77% of clinicians in the decision aid group thought the decision aid was not disruptive and even potentially beneficial, 15% found it neither disruptive nor beneficial and 8% found it potentially disruptive - 9/13 studies measuring consultation length found no significant difference in time between intervention and control groups. Three studies reported a longer and one study a shorter consultation time in the decision aid group - One study evaluating why some of the clinicians in the intervention group did not use the decision aid in consultations, showed that clinicians’ perception that they do not have enough time was the main reason for not using it. Length of consultation was not measured, so it is unclear if using the decision aid did prolong the consultation time |
Objective measure of time of whole consultations Impact on time: neither increase nor decrease Driving perception: limited time during consultations, perception that it will add time |
| Fiks et al. 2015 [104] | USA | To characterize patterns of adoption of the CDS system, assess the impact of performance feedback on CDS adoption by primary care clinicians, and measure the impact of CDS use on guideline adherence |
Condition of focus: Otitis Media in children Setting: Primary care Tool: CDS for Otitis Media - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: No |
Adoption of CDS Impact of feedback on adoption - use of documentation or order entry panels Adherence to guidelines Visit-level covariates: - visit type - type of OM |
Clinicians concerned regarding the number of “clicks” needed to use the system, which was perceived as inefficient Clinician enthusiasm for the tool was decreased because of the change in workflow that was required, especially for visits with multiple problems Clinicians ignored the tool at 80 percent of eligible visits. Two percent of clinicians never used the CDS, and 11 percent used the tool during a trial period but not again |
Perception Impact on time: Increase Driving perception: inefficient to use and causes changes in workflow |
|
| Ford et al. 2021 [105] | UK |
To support and optimise the design of future CDSs by identifying factors that influence how or why GPs use these tools, looking specifically into aspects of CDSs they find useful and problematic, both individually and in the wider context of their practice |
Condition of focus: Dementia Setting: General Practice Tool: Hypothetical CDS for dementia risk prediction - Embedded/linked with EMR: Range - Interruptive alert: Range - User-driven: Range - Risk score: Range |
Qualitative interviews |
Trust in individual CDS Usability of CDS in consultation context Usability of CDS in broader practice context |
Intrusiveness Perceived as unhelpful where CDS raised an issue which the GP felt to be unimportant within that particular consultation; where the alert does not relate to a topic of importance for either GP or patient, it may be perceived as undermining of the GP’s professional expertise - Self-population of CDS fields using previously- recorded data (e.g. in QRISK) was viewed as a benefit which reduced intrusiveness and time pressures Alert fatigue - Negative impact on patient-doctor rapport: “It really is like an interruption”, “no GP wants somebody to just burst in with the door opening or the phone ringing. In the same vein no GP really wants a big thing to just pop up on the screen that they didn’t call up.” - proliferation of alerts led to participants becoming desensitised to alerts, which could cause GPs to miss important safety alerts |
Perception Impact on time: No conclusion Driving perception: limited time in consultations, workload already heavy |
| Henshall et al. 2017 [106] | UK |
To explore the views of clinicians, patients and carers.on feasibility and acceptability |
Condition of focus: Psychiatric disorders (schizophrenia) Setting: General Practice Tool: Cloud-based CDS algorithm providing information on interventions - Embedded/linked with EMR: Unclear - Interruptive alert: Range - User-driven: Clinician - Risk score: No |
Qualitative focus groups |
Applications in clinical practice Communication Conflicting priorities Record keeping and data management |
Applications in clinical practice - the tool did not reflect the complex clinical assessment process, being unable to capture detailed information about patient characteristics, time pressures, anxiety, influence of carers, clinician experience and organisational factors - “When we see someone… it changes the conversation…Depending on how much time you have” Communication - some clinicians and patients/carers highlighted that receiving too much information might cause unnecessary worry and result in clinicians spending considerable time reassuring patients - risk of entering into a ‘minefield of discussion’ |
Perception Impact on time: Mixed views Driving perception: limited time in consultations, discussion will add time |
| Holmstrom et al. 2019 [107] | Sweden | To describe factors affecting the use of a decision support tool and experiences among Telephone Nurses in Swedish primary health care |
Condition of focus: Telephone nursing (range of conditions) Setting: Primary Care Tool: CDS providing guidelines/information and documentation in patient records - Embedded/linked with EMR: Yes - Interruptive alert: No, guides telephone call - User-driven: Nurse - Risk score: No |
Qualitative: observations and interviews |
Factors that decrease or cause deviation from CDSS Positive factors CDS complicates work |
Long working experience, time pressure, lack of training, and non-native callers decreased CDS use Because of time constraints, the TN sometimes chose to rely on their own professional knowledge instead of the CDS |
Perception Impact on time: Increase Driving perception: limited time in consultations, reading text in the tool takes time |
| Kostopoulou et al. 2017 [108, 109] | UK | To measure the prototype’s effectiveness, usability, and potential impact on the consultation and patient satisfaction |
Condition of focus: Generic Setting: General Practice Tool: CDS for diagnostic support for a range of conditions - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: No |
Simulated patient study |
Vision IT system recorded length of time patient record was open GP and patient survey(s), including length of consultation |
Mean length of baseline consultation = 13.73 min (2.96 SD) Mean length of CDS consultation = 14.42 (5.28 SD) Neither the number of investigations nor the length of consultation differed significantly between the baseline and CDS sessions Patient satisfaction re consultation length similar at baseline and CDS consultations |
Objective measure of time of whole consultations Impact on time: neither increase nor decrease |
| Laka et al. 2021 [110] | Australia |
To identify the different individual, organisational and system level factors that influence the adoption and use of CDS |
Condition of focus: Antibiotic management Setting: Hospital and general practices Tool: Range of CDS for antibiotic management - Embedded/linked with EMR: Range - Interruptive alert: Range - User-driven: Range - Risk score: Range |
Quantitative survey |
Survey - Perceived benefit - Perceived barriers - Perceived facilitators Free-text comments - Lack of flexibility - Information overload - Information accuracy |
Perceived barriers - Clinicians in primary care more likely than those in hospital to believe that factors such as time limitation restrict the use of CDS Information overload: - Time and workload pressures make it difficult for clinicians to distinguish important information from irrelevant data |
Perception Impact on time: Increase Driving perception: limited time in consultations |
| Lemke et al. 2020 [111, 112] | USA | To assess PCPs’ views of the tool and genomics-based CDS in clinical practice |
Condition of focus: Geonomics (family health history screening) Setting: Primary Care Tool: Geonomics-based CDS to identify patients at risk - Embedded/linked with EMR: Yes - Interruptive alert: No - User-driven: Patient completes tools, which alerts clinician based on answers - Risk score: No |
Qualitative interviews |
Benefits to clinical care Challenges in practice CDS issues Physician-recommended solutions |
Time: - Adding another topic to the patient’s annual visit, such as the alert recommendations, was difficult because of time constraints due to discussing other recommended screens and agenda items. Discussing the family history tool findings sometimes created difficulties in time management. “Because [physicians are] rushed and overburdened, if they know it’s going to take a lot of time, they’re going to ignore it.” Workflow: Many clinicians reported having tight schedules and felt that adding another component to the visit, would tax existing processes |
Perception Impact on time: Increase Driving perception: limited time in consultations, adds burden |
| Li et al. 2012 [113] | USA |
To assess how providers interact with the CDS while interviewing a simulated patient and to identify barriers to use prior to the implementation of a randomized controlled trial |
Condition of focus: Upper Respiratory Tract Infections Setting: Primary Care Tool: CDS providing clinical prediction rules for Strep or Pneumonia - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: Yes |
Usability testing and simulated patient study |
Usability issues: - Usability - Navigation - Content - Workflow |
Overall perception of the CDS had a positive-to-negative commentary ratio of 0.86 favouring the negative; the categories of ‘Navigation’ ‘Workflow’ were associated with the largest volume of negative comments Average encounter duration: 2.03 min (5.11–18.35 min) In 71% of cases (n = 17) the CDS was triggered after an average of 51% of the visit had elapsed. Clinicians spent on average 12.2% of encounter time using the CDS Timing of trigger- (1) Visits where CDS accessed at the beginning of the visit lasted on average 13:26 min (2) Visits where CDS accessed at the end of the visit lasted on average 5:09 min |
Objective measure of time of whole consultations Impact on time: no conclusion as no comparison with control |
| Lo et al. 2018 [114] | Australia | To assess the usability and acceptability of the iPrevent prototype |
Condition of focus: Cancer Setting: Primary Care Tool: Breast cancer risk assessment, providing tailored risk management information - Embedded/linked with EMR: Yes - Interruptive alert: No, guides visit - User-driven: Clinician - Risk score: Yes |
Quantitative usability testing Piloting using both simulated and real patients |
Usability Acceptability Risk perception Knowledge Time spent completing tool Consultation time |
The median time taken for clinician consultations in which iPrevent data were discussed was 20 (range 5–45) minutes Majority of clinicians felt the length of the tool was 'too long' |
Perception and objective measure of time Impact on time: mixed picture Driving perception: length of time to complete tool too long |
| Margham et al. 2018 [115] | UK | To evaluate the impact of the electronic trigger tool, including acceptability to clinicians, ease of use, and rates of finding patient safety events |
Condition of focus: Range Setting: Primary Care Tool: Trigger tools to identify patients at risk of safety-related incidents, including diagnostics, medication and communication, across a range of conditions - Embedded/linked with EMR: Yes - Interruptive alert: No, audit tool - User-driven: Clinician - Risk score: Yes |
Mixed methods |
Quantitative - numbers of patients identified and reviewed - rate of identification of patient safety events Qualitative - barriers and benefits to implementation, - ease of use - value of the trigger tool in the context of a busy GP surgery |
GPs all expressed concern that the tool might identify too many patients at risk of harm, place further demands on GP time, and require additional resources to manage properly - ‘Heart said “good idea”. Head said “hope it doesn’t significantly increase my workload”!’ |
Perception Impact on time: Increase Driving perception: limited time in consultations, workload already heavy |
| North et al. 2016 [116] | USA | To examine clinician time involved in risk calculation and decision making. This was done in a setting to estimate the minimum time it might take a provider at the point of care |
Condition of focus: Cardiovascular disease, Atrial Fibrilation, Diabetes and Heart Failure Setting: Primary Care Tool: CVD risk assessment (within Ask Mayo Expert) - Embedded/linked with EMR: Yes - Interruptive alert: Unclear - User-driven: Clinician - Risk score: Yes |
Case scenarios Morae® Recorder software used to collect timing and usage |
Risk calculation time Combined risk calculation and clinical decision making time |
AF CHADSVASC risk calculation 36 s (9 s) AF total management time 85 s (18 s) Lipids AHA-ASCVD risk calculation 45 s (12 s) Lipids total management time 110 s (32 s) HF SHFM risk calculation 171 s (42 s) HF total management time 347 s (89 s) |
Objective measure of time to use tool Impact on time: authors concluded that time spent on risk calculation can be reduced by using automated algorithms |
| Olakotan et al. 2021 [117] | Malaysia | To identify factors affecting the appropriateness of CDSS alerts in supporting clinical workflow based on the proposed evaluation measures for CDSS alert |
Condition of focus: Range Setting: Primary Care Tool: Range of CDS tools generating alerts to support clinical workflow - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: No |
Systematic review |
Technology factors Human factors Organisational factors Process factors |
Technology: 5 studies reported that alert overload increases the mental workload of clinicians Human - 11 studies showed that EMR-embedded asynchronous alerts increase clinicians’ workload - Providing clinicians with protected time to respond to alerts reduces alert-related workload and improves patient safety - Other instances of workload involve clinicians documenting clinical data into EMR before an alert can be triggered and selecting reasons for bypassing alerts |
Perception and objective measure of workload Impact on workload: Increase Driving perception: alert overload increases mental workload, physical and cognitive weariness, no time to respond to alerts, time needed for documenting clinical data and selecting reasons for bypassing alerts |
| Porat et al. 2017 [108, 109] | UK | To identify facilitators and barriers to future DSS adoption |
Condition of focus: Range Setting: General Practice Tool: CDS tools to support diagnosis - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: No |
Mixed: Qualitative interviews with GPs + quantitative survey of patients |
Perception of GPs Satisfaction of patients |
Impact on consultation style and GP-pt interaction: - ‘You need to get used to it…I do my consultations in a different way, but it works quite quick’ - Eight GPs (23%) were concerned that typing during the consultation would interfere with doctor-patient communication: “I normally chat and look at the patients, it throws my normal thing.” “I usually don’t code during the consultation, less contact with patient, my style is to listen for a long time.” Time concerns - Thirteen GPs (38%) felt consultation took longer with the tool than without. Without the tool, using only the EMR, GPs wrote mainly free text, which they perceived to be faster: “It will be hard to use it in a 10-min. consultation.” - Average consultation time did not significantly differ between baseline and CDS sessions - The GPs who expressed concerns about time took longer when using the CDS (mean time 15.45 min) than in the baseline session (mean time 13.53 min), paired samples t-test: 2.13, df = 12, P = 0.055, but this was not the case for the whole GP sample - Despite concerns about time, GPs believed that they could become better using the CDS |
Perception and objective measure of time Impact on time: neither increase nor decrease Driving perception: searching and selecting symptom codes takes longer than typing free text in patient notes as usual, limited time in consultations |
| Richardson et al. 2017 [118] | USA | To understand the determinants of usability of two CDS tools for lessons and themes that could be generalizable to all forms of CDS |
Condition of focus: Upper Respiratory Tract Infections Setting: General Practice Tool: CDS for antibiotic ordering for URTIs, using a clinical prediction rule for risk assessment of either A Streptococcus, pharyngitis or pneumonia - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: Yes |
Observational study (1) Think aloud testing using written case scenario (2) Near live testing using simulated patients |
Visibility Workflow Content Understandability Navigation |
Duration of each think aloud/near live scenario: 25—45 min - the automatic order set, automatically generated documentation and communication with nurses and patients decreased workload and saved time Passive alerts triggered at the time of decision making allow clinicians to use tools without disturbing their natural workflow “I much prefer to have stuff in the background that doesn’t force me to have hard stops… There may be a whole series of other things I’m dealing with." |
Perception Impact on time: Decrease Driving perception: can be used in such a way as to fit in with workflow and decision making |
| Richardson et al. 2019 [119] | USA | To further understand the barriers and facilitators of meaningful CDS usage within a real clinical context |
Condition of focus: Upper Respiratory Tract Infections Setting: General Practice Tool: CDS for antibiotic ordering for URTIs, using a clinical prediction rule for risk assessment of either A Streptococcus, pharyngitis or pneumonia - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: Yes |
Qualitative observational study |
Tool Interruptions Workflow Tool Applicability Patient-Tool interaction Provider-Computer-Patient Interaction Ease of Use Missed Opportunities |
- Of 6 patient encounters, 5 were acute or follow-up visits that lasted about 15 min each, 1 was a complete physical exam that was about 30 min in length - Clinicians spent 0%-3% of visit time listening to the patient without engaging with the computer - Clinicians completed the tool quickly; however, during half of the visits, hard stops and fixed elements in the tool created barriers. Clinicians spent about 1 min of the visit time completing the CDS tool |
Objective measure (estimate) of time of whole consultations Impact on time: neither increase nor decrease |
| Rubin et al. 2021 [16] | UK |
To establish the tool's acceptability and collect relevant data to inform the design of a subsequent definitive trial |
Condition of focus: Cancer Setting: General Practice Tool: CDS risk assessment tool for oesophageal-gastric cancer - Embedded/linked with EMR: No - Interruptive alert: No, audit tool - User-driven: Clinician - Risk score: Yes |
Randomised feasibility study - Quantitative usage data - Qualitative interviews |
Quantitative - Data related to tool use (symptoms entered and risk score generated) - Individual patient data from EMR 6 months after the index consultation, including data on secondary care procedures and diagnoses Qualitative GP interviews re facilitators and constraints influencing implementation of eCDS in routine practice |
Use of eCDS by GPs was very low and only loosely consistent with use claimed during interviews Problems with use were identified by all GPs interviewed ‘lack of integration of the software with the clinical systems’ (n = 7) ‘slow to access and/or use’ (n = 6) ‘You had to open up something completely separate to the clinical system that you’re working in, and when you’ve got very very limited time that was a negative almost pushing you to not using it’ S Not enough time within consultations (n = 5) - ‘Patients never come with one symptom or issue, they come with a few different things, and we won’t automatically think, out of three problems, one of them is related to a gastric or oesophageal cancer, erm, I’m not necessarily going into the tool’. (GP5) - ‘No way on this planet any of the GPs under the pressure we were under(…)was going to use a separate program’ |
Perception Impact on time: Increase Driving perception: limited time in consultations, separateness from the clinical system, time needed for coding, and complexity of consultations |
| Scheitel et al. 2017 [120] | USA |
To assess the impact of our clinical decision support tool on the efficiency and accuracy of clinician calculation of cardiovascular risk and its effect on the delivery of guideline-consistent treatment recommendations |
Condition of focus: Cholesterol management Setting: Primary Care Tool: Cardiovascular risk scores and guideline-based treatment recommendations - Embedded/linked with EMR: Yes - Interruptive alert: No, audit tool - User-driven: Clinician - Risk score: Yes |
Quantitative: - usage data - survey data |
Time spent making calculation and recommendation Efficiency of clicks and key strokes making calculation and recommendation Accuracy of calculation and recommendation Survey Results |
Without the tool, clinicians spent an average of 4 min and 21 s to calculate ASCVD score and a total of 5 min and 8 s to additionally determine care/treatment With the tool, the clinicians spent 39 s to calculate ASCVD score and a total of 1 min and 31 s determine a recommendation for patient care The clinicians saved 3 min and 42 s in calculating ASCVD score and a total of 3 min and 38 s in determining care/treatment. The time savings were statistically significant |
Objective measure of time to use tool Impact on time: Decrease of 3 min 38 s |
| Seol et al. 2021 [121] | USA | To assess the effectiveness and efficiency of intervention via CDS on pertinent asthma outcomes in a real-world primary care setting |
Condition of focus: Asthma Setting: Paediatric Primary Care Tool: CDS for asthma guidance and prediction to predict risk of adverse events (A-GPS) - Embedded/linked with EMR: Yes - Interruptive alert: No, audit tool - User-driven: Clinician - Risk score: Yes |
Quantitative: notes review and survey data |
Primary: adverse event (AE) within 1 year (ED visit/hospitalisation for asthma or unscheduled visit for asthma requiring oral corticosteroid) Secondary:- - Clinician burden for reviewing and collecting clinical data from EMR for making a clinical decision - Healthcare cost - Asthma control status - Timeliness of asthma follow up care after AE |
A-GPS significantly reduced clinician burden for chart review for asthma management by 67%, with an estimated median time to review patient’s medical records of 3.5 min (IQR: 2–5) with A-GPS intervention vs. 11.3 min (IQR: 6.3–15) without A-GPS (P < 0.001). Average decrease within a person with A-GPS (vs. without A-GPS) was 7.3 min |
Objective measure of time to use tool Impact on time: Decrease of 7.3 min |
| Shillinglaw et al. 2012 [97] | USA |
To examine US physicians’ awareness, use, and attitudes regarding global CHD risk assessment in clinical practice, and how these vary by provider specialty |
Condition of focus: Chronic Heart Disease Setting: Primary Care Tool: CHD risk assessment tools - Embedded/linked with EMR: Range - Interruptive alert: No - User-driven: Clinician - Risk score: Yes |
Quantitative survey |
Awareness of tools available to calculate CHD risk Method and use of CHD risk assessment Attitudes towards CHD risk assessment Frequency of using CHD risk assessment to guide recommendations of aspirin, lipid-lowering and blood pressure (BP) lowering therapies for primary prevention |
Reasons for not using CHD risk assessment: - Among physicians who reported not using CHD risk assessment (N = 492), the reason with the highest mean importance rating was, “It is too time consuming" - Family physicians rated this reason higher than general internists and cardiologists |
Perception Impact on time: Increase Driving perception: limited time in consultations |
| Siaki et al. 2021 [122] | USA | To determine the feasibility of a web-based clinical decision support tool (CDST) using a renin-aldosterone system (RAS) classification matrix and drug sequencing algorithm to assist providers with the diagnosis and management of uncontrolled hypertension |
Condition of focus: Hypertension Setting: Primary Care Tool: CDS using a renin-aldosterone system classification matrix and drug sequencing algorithm to support diagnosis and management of uncontrolled HTN - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: Clinician - Risk score: No |
Quantitative data on hypertension clinical measures + survey of clinicians |
Primary: 1) BP rates of control 2) clinician management time using an electronic logbook 3) Satisfaction with the CDS |
The fastest clinician averaged 10 min per patient. The slowest took 20.56 min The overall average was 16.59 min (17.05% less time spent), saving 3.41 min per office visit avoided |
Objective measure of time of whole consultations Impact on time: Decrease |
| Takamine et al. 2021 [123, 124] | USA | To understand providers’ views on the opportunities, barriers, and facilitators of incorporating risk prediction to guide their use of cardiovascular preventive medicines |
Condition of focus: Cardiovascular disease Setting: Primary Care Tool: CVD risk assessment tools - Embedded/linked with EMR: Range - Interruptive alert: No - User-driven: Clinician - Risk score: Yes |
Qualitative interviews |
Attitudes toward adoption of an ASCVD risk prediction-based approach Key provider concerns - Quantified risk goals vs a “whole patient” approach - Validity of risk prediction - Compatability with workflow - Does adopting risk prediction add value? The role of performance measurement |
Compatability with workflow - Given time pressures, concerns about workflow were common. 'You’re going to create a reminder in there, what else are you going to take off my plate?’ Cognitive burden - A few mentioned cognitive burdens associated with switching to this novel method: it’s easier to treat an A1C down to a certain value than, “Well, if this person’s A1C is 7.5, the cardiovascular risk is a certain number, and if we get it down to 6.8, the cardiovascular risk is a different number.”’ |
Perception Impact on time: Increase Driving perception: limited time in consultations, time needed to calculate or search for risk numbers, adding work without reducing anything else |
| Wan et al. 2012 [125] | Australia | To evaluate the uptake and use of the CDS tool as well as to describe the impact of the EDS tool on the primary care consultation for diabetes from the perspectives of general practitioners and practice nurses |
Condition of focus: Diabetes Setting: General Practice Tool: CDS to support management of patients with type 2 diabetes - Embedded/linked with EMR: Yes - Interruptive alert: Yes - User-driven: Clinician - Risk score: Yes |
Qualitative interviews |
Use of the CDS tool Impact of the tool on the consultation process Impact of the tool on diabetes care Barriers to the use of the tool Suggestions on its improvement |
Impact on the consultation process - GPs’ perceptions varied on the impact of the tool on consultation times. Many felt they tended to spend longer with patients when using the tool compared to usual consultations, but that this was because they were using the tool to provide better quality of care for their patients - One PN felt the consultation was lengthened, and another reported no change - Problems with tool functionality were perceived by some GPs as costing time. Some thought the tool itself slowed down their IT systems - Some felt the tool was distracting when the patient wasn’t attending for diabetes care |
Perception Impact on time: mixed views Driving perception: limited time in consultations, tool is time consuming and distracting, better quality care takes longer |
| Wright et al. 2020 [126] | New Zealand | To assess the acceptability and feasibility of the Maternity Case-finding Help Assessment Tool (MatCHAT), a tool designed to provide e-screening and clinical decision support for depression, anxiety, cigarette smoking, use of alcohol or illicit substances, and family violence among pre- and post-partum women under the care of midwives |
Condition of focus: Perinatal mental health Setting: Midwifery Tool: CDS for screening for antenatal and postnatal depression, anxiety, substance use and partner violence - Embedded/linked with EMR: No - Interruptive alert: No - User-driven: Clinician - Risk score: Unclear |
Mixed methods: Quantitative usage data + qualitative interviews with midwives |
MatCHAT usage data - numbers of screens completed - positive cases - participants who wanted help and the level of care recommended - survey ratings of acceptability, feasibility and utility Interviews - the MatCHAT prototype - midwives' knowledge - barriers to implementation |
Barriers to implementation - midwives unanimous that MatCHAT was ‘one more thing’ in their hectic work schedule, and this influenced the low uptake - The midwives who did not use MatCHAT thought that it would increase the length of appointments: ‘I was in conflict because I know I needed to ask those questions and MatCHAT would have been useful for that, but in another way, it was going to take up a large chunk of time.’ - midwives also worried that screening might ‘open a can of worms’ |
Perception Impact on time: mixed views Driving perception: - increase: heavy workload and limited time, worry that tool would increase time, might 'open a can of woms' - decrease: if used efficiently can cut overall time spent screening |
| Wu et al. 2013 [127] | US | To examine physicians' experiences of using MeTree, a computerized Family Health History CDS tool which includes risk stratification |
Condition of focus: Family health history Setting: Primary Care Tool: a CDS tool to collect data on family health history and risk stratification for various conditions |
Mixed methods: Quantitative survey + qualitative interviews with physicians |
Survey and interviews: Ease of integration of MeTree into clinical practice at the two intervention clinics |
Physicians initially felt that they were already collecting high-quality family histories and that MeTree would negatively impact workflow and redirect prioirities However, post MeTree introduction, 86% of physicians believed that the tool improved the way they practised medicine, thereby making practice easier, and none reported that it adversely affected their workflow |
Perception Impact on workflow: No negative impact Driving perception: the tool improved the way they practised, making practice easier |
| Dexheimer et al. 2013 [128] | US | To examine whether an automatic disease detection system increases clinicians’ use of paper-based guidelines and decreases time to a disposition decision |
Condition of focus: Asthma Setting: Primary Care Tool: a computerized asthma detection system combined with a paper-based asthma care protocol in the pediatric ED to help standardize care and reduce time to disposition decision |
Quantitative data on time to disposition decision | Workload and efficiency outcomes assessed: time for disposition decision in the ED as the primary outcome | No effect from the use of eCDS on the time taken by the ED physicians to make a disposition decision |
Objective measure of time Impact on time: Neither increase or decrease |
| Moffat et al. 2014 [129] | UK | An evaluation my CRUK of cancer CDS tools |
Condition of focus: Cancer Setting: General Practice Tool: eCDS using cancer risk algorithms with an interruptive risk score alert, risk calculator and audit function |
Mixed methods: Quantitative usage data of tool and referrals + qualitative interviews with GPs |
Quantitative data: use of the tools in practice, referrals Qualitative: impact on practice and the management of patients, and considerations and implications for further work in this area |
Interviews with GPs highlight the varying impact of the tools on practice, ranging from no impact at all, to increasing knowledge, to influencing the management, including referral or investigation, of patients GPs were concerned about the level at which the prompt was set (i.e. at what level of risk a prompt appeared on their screen) and the potential for ‘prompt fatigue’ Some GPs expressed concerns that a 10-min consultation was a barrier to use of the symptom checker function within the tool |
Perception Impact on time: mixed views Driving perception: Increase: concern regarding risk threshold of the alert and lack of time within a 10-min appointment to use the tool Decrease/no impact: Use of the tool did not influence the decision to investigate or refer in the majority of cases |
| Murphy et al. 2012 [130] | US | To measure the time spent by physicians managing asynchronous alerts |
Condition of focus: generic Setting: Primary Care Tool: Asynchronous alerts during the day regarding a range of conditions |
Quantitative: time spent processing alerts | On average, clinicians received over 56 alerts per day and spend 49 min responding to asynchronous alerts. These alerts burden clinicians in terms of physical fatigue and cognitive weariness. Providing clinicians with protected time to respond to alerts reduces alert-related workload and improves patient safety |
Objective measure of time spent processing alerts Impact on time: No conclusion |