Automation in nursing decision support systems: A systematic review of effects on decision making, care delivery, and patient outcomes

Abstract

Objective

The study sought to summarize research literature on nursing decision support systems (DSSs ); understand which steps of the nursing care process (NCP) are supported by DSSs, and analyze effects of automated information processing on decision making, care delivery, and patient outcomes.

Materials and Methods

We conducted a systematic review in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement. PubMed, CINAHL, Cochrane, Embase, Scopus, and Web of Science were searched from January 2014 to April 2020 for studies focusing on DSSs used exclusively by nurses and their effects. Information about the stages of automation (information acquisition, information analysis, decision and action selection, and action implementation), NCP, and effects was assessed.

Results

Of 1019 articles retrieved, 28 met the inclusion criteria, each studying a unique DSS. Most DSSs were concerned with two NCP steps: assessment (82%) and intervention (86%). In terms of automation, all included DSSs automated information analysis and decision selection. Five DSSs automated information acquisition and only one automated action implementation. Effects on decision making, care delivery, and patient outcome were mixed. DSSs improved compliance with recommendations and reduced decision time, but impacts were not always sustainable. There were improvements in evidence-based practice, but impact on patient outcomes was mixed.

Conclusions

Current nursing DSSs do not adequately support the NCP and have limited automation. There remain many opportunities to enhance automation, especially at the stage of information acquisition. Further research is needed to understand how automation within the NCP can improve nurses’ decision making, care delivery, and patient outcomes.

INTRODUCTION

Electronic decision support systems (DSSs) are computer programs that help clinicians make informed decisions based on existing knowledge and individual patient characteristics. Their functions can vary from presenting data to users, generating alerts to implementing actions on behalf of users.¹ For nurses, DSSs can play a vital role in facilitating the nursing care process (NCP), which is an important framework that helps nurses put their knowledge into practice. The NCP is a crucial care planning and problem-solving process in nursing practice that helps determine the needs of patients. It is generally taught to nursing students in early years of their education program and consists of five sequential steps: assessment, problem identification, planning, intervention, and evaluation.^2,3 At each of these steps, nurses are required to use critical thinking and clinical judgment to make complex decisions about patient care. Additionally, the NCP is a dynamic process often requiring multiple iterations as the patient’s condition changes (Figure 1). For instance, a patient with knee pain may have a fluctuating pain score requiring nurses to frequently revise the care plan during their hospital stay. Failure to update care plans can lead to a mismatch between care arrangements and patient’s needs.

Figure 1.

The nursing care process with examples of decisions taken at each step for a hospitalized patient who is experiencing knee pain (after Toney-Butler and Thayer).²

Use of the NCP for continuous care planning means that nurses are constantly making decisions, and for each decision made, they go through the four distinct stages of human information processing: information acquisition, information analysis, decision selection, and action implementation.⁴ For example, to prevent falls, a nurse must formulate a care plan by acquiring information about the patient’s history, physical and mental health, and current functional status. Using this information, the nurse must analyze the patient’s risk for falls and select the most suitable actions. They can then implement the selected actions to ensure best patient outcomes. Within each step of NCP, nurses may be required to make multiple decisions, and for each of those decision, they may go through one or more stages of human information processing.

To make the process of care planning more robust, several DSSs such as assessment forms, risk score calculators, and risk management plans are used in nursing practice. These systems can help nurses in two ways: by facilitating one or more steps of NCP and by automating the stages of human information processing, ie, by executing information acquisition, analysis, decision and action selection, and action implementation on behalf of nurses.⁴ Little is known about the capabilities of existing DSSs to automate these stages and their effects on decision making, care delivery, and patient outcomes.

Previous reviews of nursing DSSs have been limited to describing system features⁵ or examining use patterns,⁶ or have been focused on specific clinical problems.⁷ A recent overview of systematic reviews addressed DSSs among other tools but was limited to nursing practice measures such as time, and professional satisfaction.⁸ To the best of our knowledge, no previous review has examined the effects of automation in nursing DSSs. To address this gap, we sought to summarize the research literature describing DSSs used by nurses, and to analyze effects of automated information processing on decision making, care delivery, and patient outcomes.

MATERIALS AND METHODS

The review was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta Analyses) statement.⁹ As the NCP is unique to nursing practice, we focused on electronic DSSs used solely by nurses and looked for studies about the effects of DSSs on decision making, care delivery, and patient outcomes. We chose these categories of impact using an already established framework called the information value chain, which shows that multiple steps are necessary from using DSSs to impacting patient outcomes including interacting with DSSs, receiving new information that then alters decisions, the care delivery process, and outcomes.¹⁰ For instance, a nurse may interact with a DSS to receive important new information based on which they decide to implement (decision making) an intervention (care delivery) that prevents a patient from falling during their inpatient stay (patient outcome). Thus, effects of a DSS can be measured by examining changes in decision making, care delivery, and patient outcomes.

Search strategy

We searched bibliographic databases including PubMed, CINAHL, Cochrane, Embase, Scopus, and Web of Science in October 2019 and updated the search in April 2020. Our search included the following themes based on the review objectives: decision support, nursing, care delivery, and patient outcomes. With the help of a research librarian, the authors developed search queries consisting of a comprehensive set of keywords and MeSH (Medical Subject Headings) terms relating to the chosen themes (Supplementary Appendix A). The retrieval set was limited to articles published in 2014 or later. We decided to restrict the start date of our search to 2014 because our focus was on contemporary DSSs in nursing. Previous systematic reviews published in 2007⁵ and 2013¹¹ included similar studies and reviewed features or effects of DSSs. Moreover, the volume of nursing DSS studies has significantly increased since 2014, with an average of 154 publications indexed in PubMed per year (as compared with 70 per year from 2000 to 2013).

Study selection

Our initial database search retrieved 1019 results (Figure 2) After duplicate entries were removed, titles and abstracts of 568 articles were screened by two reviewers (S.A. and J.A.C.). Studies about electronic DSSs used by multidisciplinary teams were excluded. Non-English articles and conference abstracts were also excluded. Full-length articles were retrieved from 69 abstracts identified for inclusion and were assessed independently against the inclusion criteria by the same reviewers. Forty-one of them were excluded because the full-text was not available, the DSS was not electronic, exclusively used by nurses, the desired impact was not studied, or the DSS was tested with undergraduate nursing students. We sought and received further information from the authors of three studies to aid eligibility assessment. Any disagreements about inclusion or exclusion were resolved by consensus. Interrater agreement was moderate (Cohen’s k = 0.565, n = 69).¹²

Figure 2.

Article search and retrieval process. DSS: decision support system.

Data extraction and synthesis

For each included study, descriptive information about the DSS, study design, user group, setting, implementation approach, stages of automation, nursing care process, and effects on decision making, care delivery, and patient outcomes were extracted. The quality of included studies was assessed using the Cochrane risk-of-bias tool¹³ for randomized controlled trials and the Joanna Briggs Institute’s checklists for nonrandomized controlled trials.^14,15 Owing to heterogeneity of interventions and outcomes, a meta-analysis was not attempted. Instead, a narrative synthesis was conducted.⁹

User group and setting

We extracted information about participant credentials. They were mainly registered nurses, midwives, and nurse practitioners.¹⁶ While both registered nurses and nurse practitioners have a degree in nursing, and work closely with patients, nurse practitioners have further specialized training and experience. We categorized settings into short-term care in which patient encounters with the healthcare system are for a limited period of time (eg, emergency departments) and long-term care (eg, nursing homes).¹⁷

Nursing care process

DSS functions were mapped to one or more steps of the NCP: assessment, problem identification, planning, intervention, and evaluation.²

Stages of automation

DSSs were examined using a previously published framework that describes the automation of human information processing into 4 distinct stages⁴:

1. Information acquisition: Sensing or registering of data, without the user having to input information manually, eg, a telehealth DSS that collects data from home monitoring devices.

2. Information analysis: Cognitive functions and inferential processes, such as calculations, eg, a bedside DSS that calculates insulin dose.

3. Decision selection: Recommendations about possible actions, eg, a prehospital DSS that generates triage recommendations for transfer to aged care.

4. Action implementation: Actual execution of an action choice, eg, an oncology DSS that automatically documents nursing interventions directly in the medical record.

DSS functions could map to one or more stages of the framework.

Effects on decision making, care delivery, and patient outcomes

Using the information value chain two reviewers (S.A., F.M.) independently examined free text descriptions to identify DSS effects on decision making, care delivery, and patient outcomes. The interrater reliability for the classification was almost perfect (Cohen’s k = 0.867, n = 56).¹² The effects of DSSs were then categorized as positive, negative, or no impact.

A narrative synthesis then integrated findings into descriptive summaries for each variable examined.

RESULTS

DSS users and settings

In total, 28 studies were included in our review (Table 1). Most of the DSSs studied (n = 24, 86%) were used by registered nurses (Table 2). Three (11%) were used by nurse practitioners to manage a variety of clinical conditions including obesity, tobacco use, and depression,¹⁸ back pain,¹⁹ and complex comorbidities in homebound patients.²⁰ One DSS that guided antenatal care plans was used by both registered nurses and midwives.²¹

Table 1.

Studies reporting effects of nursing DSSs on decision making, care delivery, and patient outcomes (n = 28)

Author, year	Study design	Study dates	User group	Setting	Sample size	Target population	Purpose of DSS	Stages of human information processing automated				Steps of nursing care process
								IA	IAn	DS	AI	A	PI	P	I	E
Ajay et al, 201639	Quasi experimental	1 Dec 2012 to 21 Aug 2014	RNs	Long term: community health centers	22 009 patients 5 CHCs	Patients above the age of 30 y, who attended participating community health centers	Nursing care plan for noncommunicable diseases		Y	Y		Y	Y	Y	Y	Y
Bakken et al, 201418	RCT	Not mentioned	NPs	Both	363 NP students, 34 349 unique encounters	Patients attending NP clinics	Reminds nurses to screen patients, formulate nursing care plans.		Y	Y		Y	Y	Y	Y
Barken et al, 201738	Descriptive	Oct 2015 to Feb 2016	RNs	Short term: telemedicine	3 nurses 60 h observation	Telemedicine nurses	Triage assessment and recommendation	Y	Y	Y		Y	Y		Y
Bennett et al, 201628	Quasi experimental	Preimplementation: 1 Apr 2009 to 31 Mar 2010 Postimplementation: 1 Apr 2011 to 31 Mar 2012	RNs	Short term: emergency department	800 patient records (400 pre and 400 post)	Patients with confirmed neutropenia who attended emergency department	Triage assessment and recommendation		Y	Y		Y	Y	Y	Y
Boltin et al, 201829	Quasi experimental	April 2017 Simulated event	RNs	Short term: emergency department	13 registered nurses 296 simulated patient encounters	Mass casualty victims (all age)	Triage assessment and recommendation	Y	Y	Y		Y			Y
Bowles et al, 201522	Quasi experimental	Preimplementation (control phase): 3 wk Postimplementation (experiment): 4 wk	RNs	Short term: adult inpatient	6 hospital units Control phase: 2816 assessments Experiment phase: 3450 assessments	Adult patients admitted in and discharged alive from participating units	Risk assessment and recommendation for discharge outcomes		Y	Y		Y	Y	Y	Y
Chunmei et al, 201823	Quasi experimental	Year 2017. Duration not mentioned	RNs	Short term: adult inpatient	600 patient records (300 pre implementation, 300 post imp)	Adult inpatients	Risk assessment and management recommendations		Y	Y			Y	Y	Y
Ciqueto Peres et al, 201524	Descriptive	1 Jan 2011 to 31 Dec 2012	RNs	Short term: adult inpatient	337, 056 admissions 18 RNs	Patients admitted to medical and surgical units	Nursing problem assessment		Y	Y		Y	Y	Y
Cortez and Wells, 201636	RCT	11 wk (dates not mentioned)	RNs	Short term: adult outpatient	4 oncology clinics 50 nurses (24 control. 26 intervention)	Adult oncology nurses	Recommends interventions for common symptoms		Y	Y	Y			Y	Y
Dallaire and Cossi, 201537	RCT	3 mo (dates not mentioned)	RNs	Short term: ambulance	806 patients (396 control, 410 intervention)	Older adults who called for ambulance	Triage assessment and recommendation		Y	Y		Y			Y
Dehgani Soufi et al, 201830	Quasi experimental	Not mentioned	RNs	Short term: emergency department	10 nurses (5 in each group) 537 patients	Triage patients in emergency department	Triage assessment and recommendation		Y	Y		Y		Y	Y
Febretti et al, 201443	Quasi experimental	Not mentioned	RNs	Long term: palliative care	24 nurses	End-of-life care patients	Recommends intervention for pain management.		Y	Y		Y	Y		Y
Geurts et al, 201731	RCT	May 2010 to Dec 2012	RNs	Short term: emergency department	222 patients (103 intervention, 109 control)	Children aged between 1 mo and 5 y, presenting in ED with acute vomiting and/or diarrhea	Assess and recommends rehydration interventions		Y	Y		Y	Y	Y	Y	Y
Harless, 201619	Quasi experimental	Preimplementation start date: 15 Jun 2015 Postimplementation start date: 15 Sep 2015	NPs	Short term: adult outpatient	31 patients 2 NPs	Patients over age 18 y, who presented at NP clinic with complain of nonspecific low back pain	Assessment and recommendations for functional outcomes		Y	Y		Y	Y	Y	Y
Horner and Coleman, 201421	Quasi experimental	9 mo (Dates not mentioned)	Registered nurses and midwives	Short term: Community health centres	3 clinics 900 patients 3 RNs	Antenatal patients	Perinatal risk assessment and management recommendations		Y	Y		Y	Y	Y	Y
Kihlgren et al, 201632	Quasi experimental	Not mentioned	RNs	Short term: Emergency Department	281 patients assessed with DSS	Older people	Triage older people with health deterioration		Y	Y		Y			Y
Lytle et al, 201525	Quasi experimental	Preimplementation: Oct 2012 to May 2013 Postimplementation: Aug 2013 to Jan 2014	RNs	Short term: Adult inpatient	2-16 hospital units 60 patients per unit	Adult inpatients in medical and surgical units	Alerts nurses to complete fall risk assessments and documentation		Y	Y		Y	Y	Y	Y
Mahabee-Gittens et al, 201833	Quasi experimental	3 mo starting Nov 2015	RNs	Short term: Urgent care	35 nurses 14, 218 patient visits	Pediatric urgent care nurses	Screening and counseling for caregivers who smoke		Y	Y		Y	Y		Y
McDonald et al, 201641	RCT	Not mentioned	RNs	Long term: home care	7919 patients 500 nurses (335 control, 165 intervention)	Home health patients with high medication regimen complexity	Identify patients with high medication regimen complexity and offers management module	Y	Y	Y		Y	Y		Y
McLeod et al, 202034	Observational	Preimplementation: Jul to Sep 2016 Postimplementation: Jun 2017 to Dec 2018	RNs	Short term: Emergency Department	7 Eds 1491 triage assessments	Triage patients in emergency department	Triage assessment and recommendation		Y	Y		Y	Y	Y	Y
Olsho et al, 201444	Quasi experimental	12 mo	RNs	Long term: Nursing homes	12 nursing homes in experiment group 13 nursing homes in control group	Nursing home residents	Risk assessment	Y	Y	Y			Y	Y
Reynolds et al, 201926	Quasi experimental	1 y	RNs	Short term: Intensive care unit	64 nurses, 2 sites	Pediatric nurses	Medication dose calculation		Y	Y					Y
Tang et al, 201945	Quasi experimental	Jan 2017 to Jun 2017	RNs	Long term: Nursing homes	60 admission nurses, 30 patients.	Elderly patients with chronic diseases admitted to nursing homes	Formulate nursing care plans		Y	Y		Y		Y	Y
Telford et al, 201827	Observational	1 May 2015 to 15 Dec 2015	RNs	Short term: Intensive care unit	48 patients 1640 blood glucose values	ICU patients receiving insulin infusion	Medication dose calculation		Y	Y					Y	Y
Thoma-Lurken et al, 201840	RCT	Jan to May 2017	RNs	Long term: Community	67 nurses	Care coordinator nurses in community based dementia care	Risk assessment and management recommendations		Y	Y		Y	Y	Y
Topaz et al, 201842	Quasi experimental (Pilot)	Apr-May 2016	RNs	Long term: Home care	176 patients (90 control, 86 experimental)	Patients transitioning from hospital to home care	Generates priority score	Y	Y	Y		Y			Y
Vetter, 201520	Quasi experimental	3 mo	NPs	Long term: Home care	7 NPs 39 patient records	Chronically ill homebound patients with complex, comorbid conditions	Recommends diagnosis and ICD code		Y	Y		Y	Y
Wouters et al, 202035	Descriptive	Jul 2016 to Jul 2018	RNs	Short term: Emergency Department	24 nurses	Telephone triage nurses	Telephone triage assessment and recommendation		Y	Y		Y	Y	Y	Y	Y

A: assessment; AI: action implementation; CHC: community health centre; DS: decision selection; dSS: decision support system; E: evaluation; I: intervention; IA: information acquisition; Ian: information analysis; ICD: International Classification of Diseases; ICU: intensive care unit; NP: nurse practitioner; P: plan; PI: problem identification; RCT: randomized controlled trial; RN: registered nurse; Y: yes.

Table 2.

Characteristics of the 28 studies reporting effects of nursing DSSs

Characteristic	Studies
Study design
Quasi-experimental	18 (64)
Randomized controlled trial	6 (21)
Descriptive	3 (11)
Observational	1 (4)
Year of publication
2014	3 (11)
2015	5 (18)
2016	7 (25)
2017	2 (7)
2018	7 (25)
2019	3 (11)
2020 (until March)	1 (4)
DSS users
Registered nurses	24 (86)
Nurse practitioners	3 (11)
Nurses and midwives	1 (4)
Setting
Short-term-care units	19 (68)
Emergency or urgent care	8 (29)
Inpatient wards	4 (14)
Outpatient clinics	3 (11)
Critical care units	2 (7)
Ambulance	1 (4)
Remote consultation	1 (4)
Long-term-care units	8 (29)
Nursing homes	3 (11)
Home health care	3 (11)
Community	2 (7)
Both	1 (4)
Nursing care process
Assessment	23 (82)
Problem identification	19 (68)
Planning	17 (61)
Intervention	24 (86)
Evaluation	3 (11)
Automated functions
Information acquisition	5 (18)
Information analysis	28 (100)
Decision selection	28 (100)
Action implementation	1 (4)

Values are n (%).

dSS: decision support system.

^aCategories are not mutually exclusive (ie, DSSs may have target multiple nursing care process steps and stages of automation).

Nineteen DSSs were implemented in short-term care units including inpatient wards,^22–25 critical care units,^26,27 emergency or urgent care,^28–35 outpatient clinics,^19,21,36 ambulance,³⁷ and remote consultation.³⁸ Eight DSSs were used in long-term-care facilities including community,^39,40 home health care,^20,41,42 and nursing homes.^43–45 One DSS was used in both short- and long-term units.¹⁸

Nursing care process

Our mapping to the NCP showed that most DSSs were concerned with assessment, ie, they provided a list of questions or symptoms (n = 23, 82%), or supporting implementation like incident management plans (n = 24, 86%). Overall, 71% solely supported these two steps. The least commonly supported step was evaluation (n = 3, 11%). Only 2 DSSs supported all the steps of the NCP.

Automation in nursing DSSs

Our examination of stages of automation using the automation of human information processing framework showed that the stages of information analysis and decision selection were automated by all DSSs (n = 28, 100%). Examples include DSSs for triage in emergency departments^{29,30,32,34,35,37,38}; risk management such as falls, pressure injuries, and medication errors^{18,21–23,25,40–42,44}; dose calculation^26,27; symptom management for pain, diarrhea, and fever^{19,31,36,39,41,43}; and nursing problem identification.^20,24,45 Five DSSs automated information acquisition by collecting information from medical records^41,42,44 or patient input.^29,38 Only one DSS automated action implementation by documenting nursing interventions directly into patients’ records.³⁶

Effects of DSSs

The 28 included studies reported 56 outcome measures in total. Of these, baseline data to examine direction of effects were reported in 45 (Supplementary Appendix C). Twenty-two measures were related to decision making (reported in 16 unique studies), 11 were linked to care delivery (reported in seven unique studies), and 12 were related to patient outcomes (reported in seven unique studies). Overall, DSSs were demonstrated to have a positive effect on nurses’ decision making in 18 of the 22 outcome measures reported (82%, Table 3).

Table 3.

Effect of DSS on decision making, care delivery, and patient outcomes in studies that provided comparable baseline data (eg, 7 studies examined effects of DSS on care delivery reporting 11 different outcome measures demonstrating a positive effect in 8 measures)

	Number of studies	Total number of outcome measures reported	Positive effect of DSS	Negative effect of DSS	No significant effect of DSS
Decision making 18 , 20–23 , 25 , 26 , 28 , 30 , 32–34 , 36 , 40 , 43 , 45	16	22	18 (82)	0 (0)	4 (18)
Care delivery 26 , 28 , 31 , 33 , 37 , 39 , 42	7	11	8 (73)	1 (9)	2 (18)
Patient outcomes 19 , 22 , 23 , 25 , 31 , 33 , 41 , 42 , 44	9	12	6 (50)	1 (8)	5 (42)
Total		45	32 (71)	2 (7)	10 (22)

Values are n or n (%).

dSS: decision support system.

Effects of DSSs on decision making

The effects of DSSs on decision making was reported in 23 studies (82%, n = 28). Nurses’ agreement with DSS recommendations ranged widely between 20% and 87%.^{21,24,25,29,31–33,39,43} Several studies reported improvements in accuracy of nursing problem identification ,^18,20,23 triage prioritization,^28,29,32,34 implementation of evidence-based practice,^{18,21,25,28,43} and nursing documentation.^30,36 Reports of reduction in time required to make a decision^34,45 and care plan variations among nurses^19,40,45 were also found.

Other studies reported no significant effect on nursing interventions^2,18,22 or cognitive load.²⁶ One study found that documentation and plan of care compliance decreased after DSS implementation.²⁵ Another study reported that improvement in nurses’ decision making was not consistent. While decision making was improved initially when DSS was implemented, it gradually deteriorated within three months.³³ Finally, four studies found that nurses’ decisions were influenced by factors such as their work environment, experience, and clinical judgment.^24,26,35,38

Effects of DSSs on care delivery

Only seven (25%) studies reported effects of DSSs on care delivery. Two RCTs reported that high risk patients received appropriate care sooner than usual when DSS was used for prioritization,^37,42 while another RCT found that patients received more oral rehydration than intravenous when DSS that recommended treatment was implemented, without any effect on cost of treatment.³¹ An experimental study found that more patients were screened for and diagnosed with hypertension and diabetes with the use of DSSs.³⁹ Additionally, a DSS that addressed child tobacco smoke exposure resulted in caregivers receiving education material 83.5% of the time.³³ One DSS that calculated medication dose improved detection of dosage errors,²⁶ and another study that evaluated an emergency department DSS found that the rate of appropriate analgesic administration increased but that the use of intravenous antibiotics decreased.²⁸

Effects of DSSs on patient outcomes

Eleven (39%) studies reported effects of DSSs on patient outcomes. Four of these reported reduction in readmission when DSS was used to manage patient care.^22,31,41,42 Two examined patient’s length of stay, which was increased⁴² or remained unchanged.³¹ Functional outcomes of patients with lower back pain also remained unchanged with the use of DSSs.¹⁹

Two studies that evaluated DSSs for risk management found that falls and pressure injuries decreased postimplementation.^23,44 However, a similar study found that there was no effect on falls postimplementation.²⁵ Another study of a DSS to manage intravenous insulin infusion reported one event of mild hypoglycaemia.²⁷ A DSS that screened and managed patients with hypertension and diabetes reported significant reduction in mean systolic blood pressure and fasting blood glucose.³⁹ Another DSS to manage patients with high medication regimen complexity reported 8% risk reduction with DSSs, as compared with 4.5% without DSSs,⁴¹ and finally, a DSS that supported caregiver education to quit smoking reported that 67% of those who were educated showed willingness to quit in the next 30 days.³³

Quality of studies and risk of bias

Of the 28 studies included, most common study designs were quasi-experimental (n = 18, 64%)^{19–26,28–30,32,33,39,42–45} and randomized controlled trials (n = 6, 21%)^{18,31,36,37,40,41} (Table 2). Risk of bias assessment for the six RCTs showed some concerns (Supplementary Appendix B[i]), including not enough information about measures taken to conceal allocation sequence,^31,40,41 blinding of personnel,^36,37,41 and prespecified analysis plan.^{18,36,37,40,41} Similarly, in quasi-experimental and observational studies (Supplementary Appendices B[ii] and B[iii]), information about participants was either unclear or missing.^{23,28,30,39,45} Confounding factors were not addressed,²⁷ and measurement methods and statistical analysis were limited.^{20,21,23,29,32,45}

DISCUSSION

Main findings and implications

Despite increasing use of DSSs in clinical practice and awareness of the benefits of automation,⁴⁶ their use in nursing is understudied. Previous reviews have mainly considered nursing DSSs alongside other information and communication technologies.^8,47 To the best of our knowledge, ours is the first to apply the stages of human information processing⁴ to examine automation within nursing DSSs in relation to the NCP. We also report about DSS support for nurses’ decision making, care delivery, and patient outcomes.

Consistent with previous reviews, we found considerable heterogeneity in DSS functions, study design, and outcome measures that prevented comparative assessments.^11,48,49 The majority of the included DSSs supported patient assessments and assisted nurses to identify correct interventions. However, evaluation of the efficacy of those interventions was only supported by three DSSs. These findings are consistent with a previous review that found few DSSs supported nurses to evaluate clinical interventions.⁵

We also found that most DSSs automated two of the four stages of human information processing stages, information analysis and decision selection. These stages are likely targeted because they allow for logical, rule-based processing, which makes them more feasible from a system design perspective. Technological feasibility is a key consideration for automation.⁵⁰ Examples of tasks within these stages include risk scores calculation and recommendation of actions based on scores. The remaining two stages, information acquisition and action implementation, are more complex to automate because they require nurses’ clinical expertise and involve subjective assessment as well as actions that require physical handling. From a nursing practice perspective, information analysis and decision selection cover most of the cognitive work. However, the absence of support for the remaining stages can lead to negative outcomes. When information acquisition is manual, errors, delays, and missed data are more likely. Erroneous input can subsequently result in faulty analysis and decision selection. Similarly, manual action implementation can also be easily missed or delayed. Here, an important implication is that nurses and DSSs need to work together as they each may perform different stages of the information processing task.

In terms of the effects of DSSs on decision making, care delivery, and patient outcomes, evidence in all three outcome domains was mixed. While some DSSs were reported to have an overall positive impact on decision making in the form of improved compliance with recommendations^19,28,43 and reduced time to make decisions,^34,45 others had no sustainable impact. Two studies even reported negative outcomes.^25,28 Interestingly, one study reported that the long term impact of DSSs was not impressive, ie, compliance and improvements in decision making and care delivery increased initially but decreased with use of DSSs over a longer period of time.³³ A recent review of clinical DSSs describes in detail the long-term effect of DSS use including users’ clinical skills, education, and overreliance.⁵¹ As there was only one study that measured DSS effects over time, it was not possible to conduct a formal analysis of long-term effects in this review. Further research is required to explore whether effects are sustained or changed in the long term.

We found many reports about improvement in evidence-based practice as a result of DSS implementation, meaning that more patients were appropriately diagnosed, prioritized, educated, or treated.^{28,29,32–34,37,39,42} However, the impact of these improved practices on patient outcomes was mixed. While there were several reports about reduced safety incidents such as readmissions^22,31,41,42 and falls,^23,44 other DSSs had no effect^19,31 or increased patient harm.^25,27 There are four possible explanations for this disparity in impact. First, the way DSSs are integrated with existing information systems can affect outcomes. In our review, DSSs that were integrated with electronic health records (EHRs) had better outcomes,^{21,34,42,44,45} as compared with those that were standalone tools.^26,35 Integration with existing information systems can help address gaps in automation, reduce data duplication, and improve usability.^52,53 Second, most existing DSSs do not usually support more than one step of the NCP, which may affect outcomes. DSSs are mostly designed to focus on singular tasks such as medication dose or risk score calculation. It is important here to understand that these tasks are part of wider care processes. For example, dose calculation is a task within medication management and may not reduce errors if it is not directly linked to medication administration.²⁶ Third, nurses’ decision making is a complex process with multiple variables influencing outcomes such as their clinical experience, judgment, personal values, autonomy, situation awareness, and organizational context.^54–56 Their decisions are not always straightforward or mathematical in nature. Nurses often collaborate with team members, consider available resources, and prioritize tasks before coming to a decision. Therefore, the effects of DSSs may vary. Finally, we observed a wide variety of methods to implement and study DSSs. Some studies allowed time for interventions to be embedded postimplementation, and others did not. The impact of different implementation strategies is a well-understood phenomenon in the health informatics literature.^57,58 We also found that evaluations were conducted in different settings and over different timeframes. Some tested DSSs in a controlled environment, eg, a computer laboratory, and others were situated in clinical settings.

Recommendations for DSS developers, nurses, and researchers

Based on our findings, we make several recommendations for DSS developers, nurses, and researchers. While DSS design is usually based on clinical tasks that must be supported, developers need to consider nursing tasks as components of the NCP where each step is interconnected. For example, a DSS that addresses knee pain management (Figure 1) should not be limited to pain assessment, but rather should support decisions about relevant potential problems, planned expected outcomes, individualized interventions, and evaluation. It is also important to realize that for many nursing care problems, DSSs alone cannot be the solution but an essential component of a group of strategies that may be needed. For example, a fall prevention toolkit combining decision support with patient and family education materials reported a significant reduction in fall rates.⁵⁹ Another study implemented 2 DSSs along with training and patient education materials to improve the safety of medication use in elderly patients.⁶⁰ Such use of multiple interventions is not a new concept in nursing practice. The importance of designing multisystematic models that include elements of the physical environment, culture, and technology has been highlighted in previous systematic reviews.^61,62 Such an approach can conform to the way nurses perform their clinical duties, engage patients and other healthcare team members in care delivery, and address various care management factors at the same time to enhance overall impact.

There is also a need to make nursing DSSs more intelligent, ie, instead of being hard-coded, systems should be adaptive and learn from patient data and user behaviour.⁶³ Over the last decade, advancements in artificial intelligence techniques have proven effective in disease diagnosis,⁶⁴ which suggests that it is certainly possible to automate problem identification based on quantifiable data from patient medical records. Additionally, evidence suggests that using automatically deriving data from EHRs and making decisions based on those data can reduce errors and improve care quality.⁶⁵ However, opportunities to incorporate such technologies within nursing DSSs and better support the NCP are yet to be realized.⁶⁶

It is also important to make thoughtful decisions about which NCP steps and clinical tasks to automate. One way of doing this is through workflow analysis that nurse leaders can perform. Workflow analysis considers all physical and cognitive factors involved in decision making.⁶⁷ It requires detailed documentation of each step of the decision-making process, including activities, environments, patients, organizational culture, policies, and procedures involved.⁶⁸ For example, workflow analysis of resident monitoring in nursing home can capture available resources, decisions about who to monitor and when to intervene, and actions such as identifying location of harm and setting alarms.⁶⁹ Another example is inpatient risk assessment, workflow that starts with initial assessment at the time of admission, and regular reassessments depending on patients’ condition. Using outcomes of workflow analysis, nurse leaders can select the system that best fits their processes. Ultimately, the decision about which NCP step and nursing tasks that can be automated should depend on added benefits to care delivery, and patient outcomes.¹⁰

Frontline staff, who are often the users of DSSs, should be involved in design and implementation processes.⁷⁰ They can participate at several stages such as the needs analysis stage by sharing their perspectives, verbalizing expectations and inherent requirements of nursing practice; testing stage by providing use cases or helping developers and leaders explore effects of system prior to implementation⁷¹; and finally in the evaluation stage by offering meaningful insights about impact on decision making, care delivery, and outcomes.

Further research on the development and testing of automated nursing DSSs is required. While there are many studies examining nurses’ decision making^54,72,73 and their interaction with health technology,^74,75 there is a need for primary studies to examine the role of automation in nursing DSSs in relation to the NCP. Indeed, safe and effective automation of DSSs will guide the future landscape of nursing practice to improve care quality and patient safety.

Limitations

This review has several limitations. It was restricted to DSSs used solely by nurses and did not include multidisciplinary implementation studies that may involve nurses as part of a diverse user group. Moreover, our search was limited to studies published from January 2014 to March 2020. The use of DSSs is not new in nursing and several studies published before and after our search duration have evaluated their effects.^{59,60,62,76–78} It is thus possible that our examination of automation in nursing DSSs and its effects is not exhaustive. Heterogeneity in DSS functions, study design, and outcome measures prevented quantitative examination of effects of automation on decision making, care delivery, and patient outcomes.

CONCLUSION

This review confirms that current nursing DSSs do not adequately support the NCP and have limited automation. There remain many opportunities to enhance automation, especially at the stage of information acquisition such as by allowing DSSs to acquire data from other sources such as EHRs. Further research is required to understand how automation within the NCP can improve nurses’ decision making, care delivery, and patient outcomes.

FUNDING

This research was supported by doctoral scholarship to SA provided by Macquarie University (MQRES 20192700, 20201711); the National Health and Medical Research Council Centre for Research Excellence in Digital Health (APP1134919), and a Macquarie University Safety Net grant.

Author contributions

SA conceived this study and designed and conducted the analysis with advice and input from FM and DL. SA drafted the manuscript with input from all authors. All authors provided revisions for intellectual content. All authors have approved the final manuscript.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.