Impact of Clinical Decision Support System Assisted prevention and management for Delirium on guideline adherence and cognitive load among Intensive Care Unit nurses (CDSSD-ICU): Protocol of a multicentre, cluster randomized trial

Shan Zhang; Shu Ding; Wei Cui; Xiangyu Li; Jun Wei; Ying Wu

doi:10.1371/journal.pone.0293950

. 2023 Nov 28;18(11):e0293950. doi: 10.1371/journal.pone.0293950

Impact of Clinical Decision Support System Assisted prevention and management for Delirium on guideline adherence and cognitive load among Intensive Care Unit nurses (CDSSD-ICU): Protocol of a multicentre, cluster randomized trial

Shan Zhang ^1,^#, Shu Ding ^1,^2,^#, Wei Cui ¹, Xiangyu Li ¹, Jun Wei ³, Ying Wu ^1,^*

Editor: Sascha Köpke⁴

PMCID: PMC10684021 PMID: 38015867

Abstract

Background

Adherence to the delirium bundle intervention is sub-optimal in routine practice, and inappropriate use of the instructional design of interventions may result in higher cognitive load among nurses. It remains unclear whether the Clinical Decision Support System (CDSS) Assisted Prevention and Management for Delirium (CDSS-AntiDelirium) results in the improvement of adherence to delirium intervention and the reduction of extraneous cognitive load, as well as improving adherence to delirium intervention, among nurses in the intensive care unit (ICU).

Methods

This study (named the CDSSD-ICU) is a multicentre, prospective, cluster randomized controlled clinical trial. A total of six ICUs in two hospitals will be randomized in a 1:1 ratio to receive either the CDSS-AntiDelirium group or the delirium guidelines group. The CDSS-AntiDelirium consists of four modules: delirium assessment tools, risk factor assessment, a nursing care plan, and a nursing checklist module. Each day, nurses will assess ICU patients with the assistance of the CDSS-AntiDelirium. A total of 78 ICU nurses are needed to ensure statistical power. Outcome assessments will be conducted by investigators who are blinded to group assignments. The primary endpoint will be adherence to delirium intervention, the secondary endpoint will be nurses’ cognitive load measured using an instrument to assess different types of cognitive load. Repeated measures analysis of variance will be used to detect group differences. A structural equation model will be used to clarify the mechanism of improvement in adherence.

Discussion

Although the CDSS has been widely used in hospitals for disease assessment, management, and recording, the applications thereof in the area of delirium are still in infancy. This study could provide scientific evidence regarding the impact of a CDSS on nurses’ adherence and cognitive load and promote its further development in future studies.

Clinical trial registration

ChiCTR1900023711 (Chinese Clinical Trial Registry).

Introduction

Delirium, a common complication among patients in the intensive care unit (ICU), with incidence ranging from 70 to 87% [1, 2], is associated with a longer length of hospital stay and increased mortality [3, 4]. Therefore, the Pain, Agitation, Delirium, Immobility, and Sleep (PADIS) guidelines recommend the use of bundle interventions as a way to prevent and manage ICU delirium, which focuses on eliminating delirium risk factors [5]. However, adherence to the delirium bundle intervention is sub-optimal in routine clinical care [6]. Previous studies have demonstrated that various barriers may hinder adherence to implementing the bundle intervention, such as heavy workload and burden of maintaining nursing care records [7], complexity algorithm of assessment tools [8], as well as numerous risk factors that must be collected through multiple channels [9]. These factors may contribute to lowering the speed for receiving and processing information, decreased capacity of working memory, and a greater cognitive load [10, 11].

Cognitive load refers to the total amount of cognitive resources that a person needs to process cognitive activities [12] and comprises three load types: intrinsic, extraneous, and germane cognitive [13, 14]. Intrinsic cognitive load refers to the demand for working memory resources in processing information elements and their interactions in related cognitive activities [15]. Extraneous cognitive load is produced by inappropriate task presentation other than the cognitive activity itself [16]. Germane cognitive load promotes schema construction, thereby facilitating the execution of cognitive activities [15]. Higher extraneous cognitive load has been identified as one of the most important problems in providing intensive care [17, 18]. This negatively impacts nurses, such as by reducing their capacity to acquire knowledge [18, 19].

Considering the negative consequences of a higher cognitive load owing to the implementation of delirium bundle intervention, it is important to provide ICU nurses with a tool that can reduce extraneous cognitive load. Clinical decision support systems (CDSSs) have been widely used in most hospitals worldwide for disease assessment, management, and recording [20, 21]. Several studies have shown that a CDSS can help medical staff by reducing the amount of clinical information that must be recalled, which leads to a significant reduction in cognitive load [22, 23]. Therefore, we developed the Clinical Decision Support System (CDSS) Assisted Prevention and Management for Delirium (CDSS-AntiDelirium) based on the Cognitive Load Theory and Human Factors Engineering (HFE) [24], which were used to guide the division of functional modules, process design, and user interface (UI) design throughout the development process. There were five steps to designing and developing CDSS-AntiDelirium, including (1) assessing the reasons for non-adherence to the PADIS guidelines and the needs of users regarding functionality, workflow, and UI of the CDSS-AntiDelirium; (2) designing the functional modules following PADIS guidelines and nursing processes; (3) iteratively designing the system structure and UI of the CDSS-AntiDelirium; (4) improving the CDSS-AntiDelirium based on end-user feedback in pilot testing; and (5) usability testing of the CDSS-AntiDelirium. Development of the CDSS-AntiDelirium was completed by a multidisciplinary team, including healthcare professionals, nursing researchers, nursing informaticians, and information technology engineers. The CDSS-AntiDelirium was tested and showed good usability. However, the impact of CDSS-AntiDelirium on adherence and cognitive load in ICU nurses is still unclear. Therefore, in this study, we aim to conduct a multicentre, cluster randomized trial (RCT) to investigate the effectiveness of the CDSS-AntiDelirium on improving adherence to delirium intervention and reducing cognitive load among ICU nurses as well as reveal the mechanism of adherence improvement in delivering delirium intervention.

Materials and methods

The protocol is developed according to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT), see S1 File [25].

Study design

The study is a multicentre, cluster RCT. A total of six ICUs in two hospitals will be randomized in a 1:1 ratio to receive either a CDSS-AntiDelirium group or an individual intervention group. Fig 1 shows the design of the study.

Fig 1 — *Al-AntiDelirium*: the Artificial Intelligence Assisted Prevention and Management for **Delirium** *PADIS guideline*: the Pain, Agitation, Delirium, Immobility, and Sleep (PADIS) Guidelines.

Study setting

Six ICUs in two tertiary hospitals are selected based on a high incidence of delirium as well as variety in the samples, including two surgical ICUs (SICUs), two respiratory ICUs (RICUs), and two cardiac ICUs (CICUs). Hospital A is a 1000-bed university-affiliated teaching hospital in Beijing, with 52 ICU beds in three participating ICUs. Hospital B has 730 beds and three eligible ICUs with 46 beds, in which the annual admission rate is around 850 patients. These two hospitals have both fully implemented hospital informatization.

Participants

Registered nurses eligible for the study (1) have a minimum of 1 year of experience working in the ICU; (2) work full-time in the unit; and (3) consent to participate in this study. Nurses will be excluded from the study if they are on leave for various reasons during the study period.

Recruitment

All recruitment will be carried out by trained research staff who will not be involved in the intervention and will be blinded to the nurses’ group assignments. To identify eligible nurses, researchers will administer a questionnaire among all nurses to screen nurses based on the inclusion and exclusion criteria. Written informed consent will be obtained from all eligible nurses. To retain participating nurses, research staff will explain the benefits for nurses if this study is successfully implemented.

Randomization and blinding

The risk of between-group contamination is reduced with cluster randomization. Therefore, ICUs will be randomized 1:1 to receive either CDSS-AntiDelirium care or PADIS guideline care. The allocation sequence is based on computer-generated random numbers and allocation is performed by a statistician who is independent of the data analyses and not involved in data collection. All eligible nurses will be recruited until the sample size is sufficiently large.

Investigators who will enroll participants will not be made aware of the randomization list. Once the ICUs (cluster) that will take part in the study have been determined, randomization will be performed, considering the size of each ICU and the type of disease treated. The following criterion will be considered in this process: whether the center is classified as having a high incidence of delirium. Investigators will recruit participants following predefined inclusion and exclusion criteria. Baseline data and endpoint measures will be obtained by data collectors or outcome assessors who have no role in the intervention and are blinded to the allocations. However, this will not be possible for blind nurses who implement the interventions to the allocation owing to the nature of the intervention.

Intervention group

The appropriate subset of interventions from the PADIS guideline (such as the ABCDEF bundle) will be tailored to each patient’s specific risk factors using the CDSS-AntiDelirium. The intervention is targeted to eliminate various modifiable risk factors of ICU delirium, including hearing impairment (e.g., speaking loudly, slowly, and patiently with the patient; assisting the patient to correctly wear hearing aids), visual impairment (e.g., assisting the patient to wear eyeglasses correctly), pain (e.g., providing non-pharmacological interventions, such as distraction and relaxation therapy), use of anesthetic or sedatives (e.g., implementation of the spontaneous awakening trial), mechanical ventilation (e.g., conducting spontaneous breathing trials according to the doctor’s orders), indwelling catheter (e.g., removing the catheter as soon as possible), infection (e.g., reducing invasive operations and avoiding unnecessary catheterization), immobility (e.g., helping patients to do passive range-of-motion exercises, 10 times for each joint), sleep deprivation (e.g., assisting patients to wear earplugs or anti-noise equipment), and no visits from family members (e.g., encourage family visits). A set of IF-THEN rules are stored in the CDSS-AntiDelirium, which are designed to trigger the nursing care plan based on the identified delirium risk factors. These interventions are delivered 7 days a week primarily by nurses in a face-to-face mode, until the patient’s discharge. Nursing care activities will vary among specific frequency, duration, intensity, or dose.

Nurses in the intervention group will provide delirium assessment and intervention with the assistance of the CDSS-AntiDelirium (Fig 2). The CDSS-AntiDelirium consists of four modules: delirium assessment tools, risk factor assessment, nursing care plan, and a nursing checklist module. Before the study, an educational program is to be delivered by researchers (Fig 1), including knowledge regarding ICU delirium risk factors, assessment tools such as the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU), and bundle interventions. Additionally, eligible ICU nurses will be trained in how to operate the CDSS-AntiDelirium. The CDSS-AntiDelirium will be installed on a personal digital assistant, for which nurses must complete the necessary registration information including hospital, department, name, and password. Nurses can then log into the system and (1) enter the Delirium assessment tools module to perform ICU delirium assessment at least two times (8:00–10:00 a.m., 4:00–6:00 p.m.) per day; (2) enter the Risk factor assessment module to assess patient’s existing risk factors (e.g., hearing impairment, visual impairment, pain, and mechanical ventilation), in which dynamic delirium prediction rule for ICU patients (DYNAMIC-ICU) will be utilized to classify patients into different risk levels [26]; (3) enter the Nursing care plan module to confirm the plan, which is automatically tailored based on the results of risk factor assessment; (4) enter the Nursing checklist module to check the specific execution timetable of each intervention based on the results of the nursing care plan. The data transformation algorithms are predetermined.

Each day, the nurse will assess ICU patients with the assistance of the CDSS-AntiDelirium (Table 1).

Table 1. Safety and quality pathway for ICU nurses in implementing delirium intervention.

Step	Actions	Intervention group	Control group
Step 1	Delirium diagnosis	The nurse completes the item one by one according to the prompts in CDSS-AntiDelirium, and the system will automatically present whether the patient has ICU delirium or not (See S2 File).	The nurse will use the paper-based delirium assessment tools to identify delirium (See S3 File).
Step 2	Identify delirium risk factors	The nurse fills out a brief risk factor assessment in CDSS-AntiDelirium, including three categories, predisposing, disease-related, iatrogenic, and environmental factors; the system will automatically report personalized risk factors of each patient and show the risk prediction value based on a dynamic ICU delirium prediction rule (See S2 File).	The nurse will complete a paper-based risk factors evaluation list to identify the risk factors of each patient, including three categories same as to intervention group (See S3 File). In addition, the nurse will calculate the delirium risk prediction value based on the delirium prediction rule.
Step 3	Tailor nursing care plan	The nurse will receive the personalized delirium prevention or management care plan automatically which is tailored based on the results of the risk factors assessment (See S2 File).	The nurse needs to tailor an appropriate subset of interventions from the PADIS guidelines that target patients’ specific risk factors (See S3 File).
Step 4	Implement nursing checklist	The nurse can check the nursing checklist, which automatically displays the specific execution time of each intervention based on the results of the nursing care plan (See S2 File).	The nurse needs to make sure of the execution time of each intervention from the ICU delirium prevention and treatment interventions sheet (See S3 File).

Open in a new tab

Control group

Nurses in the control group will provide delirium intervention based on a paper-based delirium assessment tool (CAM-ICU), risk factor assessment, and bundle interventions. Before the study, an educational program is to be delivered by researchers (Fig 2); the content is the same as that in the intervention group. In addition, nurses are trained to use paper-based individual interventions. During the study, the intervention will be provided by nurses (see Table 1).

Outcome measures

Outcome assessment will be implemented by trained research staff who are not involved in the clinical nursing care of patients.

The primary outcome is adherence to delirium intervention among ICU nurses, which is defined as adherence to implementing delirium and risk factor assessment, delirium prevention, and management intervention. The reasons for non-adherence will be recorded daily by the intervention staff. A complete adherence rate is recorded as 100%, and a complete non-adherence rate is recorded as 0%. The adherence rate is between 0 and 100, according to the following formulae.

Adherence to delirium interventions is calculated as (the number of interventions implemented/the total number of interventions expected) × 100%. The total number of interventions that should be implemented is in accordance with the nursing care plan. For example, an individualized delirium care activity checklist with 10 interventions is tailored by the CDSS-AntiDelirium based on the results of the risk factor assessment. However, a bedside nurse only completes six interventions during the whole shift; therefore, the adherence to delirium intervention is (6/10) × 100% = 60%. During a whole shift, a bedside nurse will take care of three to five ICU patients; therefore, the total adherence to delirium intervention per day is calculated as (number of interventions implemented for all patients cared for/total number of interventions expected for all patients cared for) × 100%.
Adherence to risk factor assessment is defined as (the number of risk factors assessed/the total number of risk factor assessments expected) × 100%. The total number of risk factors that should be assessed in accordance with the risk factor assessment sheet, including hearing impairment, visual impairment, pain, use of anesthetic or sedatives, mechanical ventilation, indwelling catheter, infection, immobility, sleep disorders, and no visits from family members. The total adherence to risk factor assessment per day is calculated as (the number of risk factors assessed for all patients cared for/the total number of risk factor assessments expected for all patients cared for) × 100%.
Adherence to ICU delirium assessment is defined as (the number of ICU delirium assessments/the total number of ICU delirium assessments expected) × 100%. The total number of ICU delirium assessments for each patient is calculated two times (8 to 10 A.M. and 4 to 6 P.M.) per day. The total adherence to ICU delirium assessment per day is calculated as (number of ICU delirium assessments for all patients cared for/total number of ICU delirium assessments expected for all patients cared for) × 100%.

The secondary outcomes include overall and three types of cognitive load for nurses. Cognitive load is to be assessed using an instrument for measuring different types of cognitive load (MDT-CL) with 10 items [27]. Each item is scored from 0 to 10; the higher the score, the greater the cognitive load. The overall cognitive load is equal to the average score of the three cognitive loads. The Chinese version retains sensitivity and specificity in measuring cognitive load [28]. Cronbach’s α of the Chinese version of the MDT-CL is 0.818; Cronbach’s α value in the measurement of intrinsic cognitive load (Items 1, 2, and 3) is 0.879; Cronbach’s α value in the measurement of extraneous cognitive load (Items 4, 5, and 6) is 0.878; and Cronbach’s α in the measurement of germane cognitive load (Items 7, 8, 9, and 10) is 0.946.

Level of knowledge refers to nurses’ knowledge of ICU delirium assessment tools, risk factors, prevention, and management intervention and is measured using the questionnaire on ICU delirium knowledge. A 20-item single-choice questionnaire has been developed [29], with Cronbach’s α for the questionnaire of 0.82. The higher the score, the higher the knowledge level regarding ICU delirium.

Participant timeline

Enrollment and data collection started in November 2022. Recruitment will continue until the target population (78 nurses) is enrolled, which is expected to end in June 2023. Afterward, the data analysis will be carried out for subsequent publication.

Data collection

Before the study, uniform training will be delivered by research staff to data collectors, including theoretical knowledge and assessment skills for delirium, to maintain high consistency of delirium assessment in all included units. To minimize error and maximize reliability, the project director will perform the following: providing intensive training to the assessors, including a review of the procedure outlined based on a standardized patient to ensure high inter-rater reliability (kappa > 0.9); meeting with the assessors every week to review procedures and check the quality of assessment for the primary outcome. Data collectors will recruit nurses based on the inclusion and exclusion criteria, and informed consent will be obtained from all eligible nurses before allocation (Fig 1). Then, basic demographic data of ICU nurses will be collected, and baseline cognitive load will also be assessed.

During the study, adherence to delirium intervention and the cognitive load of nurses will be measured at the end of each shift on each day (Fig 1).

Data management

All data will be collected on a printed, pre-coded form that will be double entered into an electronic database and go through extensive checking for errors and data completeness. The ICUs involved in the study will only have access to their own data. We will perform the following to ensure data quality. (1) Before the study, all investigators and data collectors will be trained to master the procedures of data collection. (2) All participating nurses will be trained to apply the tools, but different groups of nurses will attend a different education session: intervention group nurses will learn how to use the CDSS-AntiDelirium, and control group nurses will learn how to apply paper-based individual interventions. (3) Data cleaning and reviewing will be ongoing to detect missing data and inconsistencies. The investigator will solve any problems promptly.

Data analysis

PASS version 11.0.7 (NCSS, LLC. Kaysville, Utah, USA) will be used to estimate the sample size for the current study. This cluster RCT aims to improve adherence to delirium intervention among ICU nurses using the CDSS-AntiDelirium as compared with PADIS guidelines. We assume that the adherence is 80% in the CDSS-AntiDelirium group and 50% in the paper-based individual intervention group, based on a similar study that explored adherence to delirium intervention [30]. Sample size calculations show that 12 nurses in each cluster will provide 80% power and a two-sided significance level (α) of 0.05, with an intra-cluster (within-unit) correlation of 0.002 [31]. In considering a possible attrition rate of 10%, we will enroll 13 nurses in each ICU setting; therefore, the final sample size for the study is 78 (13 × 6) ICU nurses.

All data will be analyzed using IBM SPSS version 21.0 (IBM Corp., Armonk, NY, USA) with an intention-to-treat (ITT) principle. To avoid the reduction in statistical efficiency and bias caused by missing data, multiple imputations will be performed to create a complete dataset, assuming that data are lost at random. Statistical analyses will be done by a statistician who is blinded to the intervention allocations. Continuous variables (e.g., age, years of ICU experience) will be described as mean and standard deviation for normally distributed data and as median and interquartile range for non-normally distributed data. Comparisons between groups will be performed with the Mann–Whitney U-test (analysis of variance) or Wilcoxon test, including cognitive load, and adherence. Categorical variables (e.g., sex, education level) will be expressed as frequency and percentage. The chi-square test or Fisher’s exact test will be used to examine between-group differences according to demographics.

Considering the correlation with repeated measures in the same individual, repeated measures analysis of variance (RMANOVA) will be used to analyze data to assess the effect of time and group on adherence and different types of cognitive load that are collected every day during the study period. Mauchly’s W statistic will be used to assess the estimated sphericity and Greenhouse–Geisser correction will be used if the data do not conform to the sphericity test (P < 0.05). The influence of potential covariates, e.g., age, sex, educational level, marital status, department, years of ICU experience, and level of professional title, will be assessed by including their interaction with the intervention in RMANOVA. Bonferroni post hoc tests will be used to assess within-group changes over time, as well as between-group differences during the study. In addition, adherence and cognitive load will also be analyzed using a linear mixed model for repeated measures (MMRM); the combination of RMANOVA and MMRM will allow us to draw more reliable and valid conclusions. Subgroup analyses for the primary endpoint (adherence to delirium interventions) will be performed in the following groups: age (< 30 years vs. ≥ 30 years), years of ICU experience (< 10 years vs. ≥ 10 years), education level, professional title, baseline intrinsic cognitive load, extraneous intrinsic cognitive load, and germane cognitive load (median value as the cut-off point).

AMOS 21.0 statistical software for path analysis will be used to analyze the relationship among task presentation (different groups), cognitive load, and adherence. The acceptable model goodness of fit is as follows: (1) χ²/degrees of freedom (df) ratio (χ²/df) < 3; (2) goodness-of-fit index (GFI) > 0.85; (3) comparative fit index (CFI) > 0.90; (4) Tucker–Lewis Index (TLI) > 0.90; and (5) root mean square error of approximation < 0.05 [32]. Effect sizes are measured using the standardized coefficients and are evaluated at the following levels: 0.1–0.3 = small; 0.3–0.5 = medium; > 0.5 = large [33]. A bootstrap approach with 95% bias-corrected confidence intervals (CIs) and 95% Percentile CIs (based on 2,000 bootstrap samples) will be used to test the significance of the mediation effect. All tests are two-tailed, and a P-value < 0.05 is considered statistically significant.

Data monitoring

Supervisors will adopt a safety role to monitor the quality and completeness of data in each center. They will audit the original data and clarify any problems with the data in the collection (for example, insufficient enrolment or retention of participants, inadequate researchers, and missing data). The supervisor has the right to terminate the trial for reasons of patient safety. There is no interim analysis in this trial, and the study will continue until the target sample size is reached.

Harms

There is no harm expected to nurses participating in this study. In the present study, the patient interventions will not pose an additional risk to patients. However, adverse events can occur even during normal care that can result in patient dysfunction, signs of discomfort, prolonged hospitalization, as well as life-threatening events. Any adverse events (e.g., fall, pressure ulcer, unplanned extubation, bradycardia, tachycardia) will be recorded by data collectors and reported to the Ethics Committee of the hospital as soon as possible.

Patient and public involvement

Patients and/or the public (including clinical professionals) will be involved in the study design, or conduct, or reporting of this research. A participatory workshops carried out in October 2022 focused on collecting their opinions and solving problems in the design of the protocol. Information about the study (CDSSD-ICU) is used to inform patients and public representatives. The findings of the CDSSD-ICU will be summarized for publication.

Ethical considerations and dissemination

The Ethics Committee of the two participating hospitals approved the study protocol (#XW082 and #CY1225-33). Approval from the ethics committee must be obtained for any modification to this protocol. The trial was registered in the Chinese Clinical Trial Registry (www.chictr.org.cn) with the identifier ChiCTR1900023711 on June 8, 2019. During the study, guidelines of Good Clinical Practice will be fully followed to guarantee the rights of participants.

All study data are anonymized and treated confidentially and will not be disclosed to anyone other than members of the research group. For the duration of the study, all paper documents for each participant will be stored in locked cabinets, and electronic data will be entered into a password-protected database. The results of the study will be submitted for publication in peer-reviewed journals. All data underlying the findings will be shared through the Baidu network (https://pan.baidu.com/s/1qVeHCHE23n75eGWJBOz1ZQ). All authors will take part in reviewing drafts of the manuscript and solving any other publication problems in this study. In addition, we propose disseminating the results of this study at nursing or delirium-related academic conferences; all academic posters and papers will be written by research team members.

Ancillary and post-trial care

Because the ICUs involved in this study will incorporate the intervention protocol into routine care, no additional compensation is received by ICU nurses.

Validity and reliability/Rigour

A cluster RCT with practical maneuverability will be conducted to implement the trial, and high-quality evidence will be provided from the results of this study. This study adopts a rigorous methodology to minimize the risk of potential bias. For instance, before the study, all data collectors blinded to the group allocation will be trained to use the assessment tools to proficiently and effectively evaluate the variables to minimize errors and improve reliability. The operational definitions of variables involved in this study will be predefined by reviewing the literature and guidance to minimize information bias. The outcome assessors will receive intensive training in terms of validated and reliable instruments to assess adherence, cognitive load, and other outcomes, which can minimize observer bias.

Results and discussion

During the implementation of delirium intervention, ICU nurses require greater cognitive resources to maintain constant attention, recall information, and conduct real-time monitoring [22]. ICU nurses also must conduct complex decision-making, implying a high cognitive load, which may result in poor adherence to the clinical guidelines. However, a CDSS can automatically collect, sort, and classify patient information and provide immediate feedback to healthcare providers during disease diagnosis, treatment, and nursing activities [34]. Therefore, ICU nurses can enhance their adherence to delirium intervention with the assistance of a CDSS. However, no complete CDSS exists to assess, prevent, and intervene in ICU delirium, and the impact of a CDSS on nurses’ outcomes remains unclear. The CDSS-AntiDelirium was designed to identify delirium and modifiable risk factors in patients and to automatically provide personalized delirium preventive and management interventions. Use of the CDSS-AntiDelirium may encourage ICU nurses to be more involved in implementing delirium interventions.

To the best of our knowledge, this would be the first multicentre, cluster randomized trial focused on improving understanding of whether the CDSS-AntiDelirium is effective and beneficial for adherence to delirium intervention and reducing cognitive load among ICU nurses, as well as revealing the mechanism of adherence improvement in delivering delirium intervention. We hypothesize that: (1) the CDSS-AntiDelirium can reduce the extraneous cognitive load of ICU nurses and promote adherence to implementing delirium interventions, and (2) extraneous cognitive load is a mediating variable between task presentation and adherence to delirium intervention. The results of this study will have a profound impact on the research field regarding Cognitive Load Theory and CDSSs and will promote further development in future studies.

Several limitations will exist in the proposed study. First, this study will only be carried out in two hospitals in a single region, which will limit the generalization of our results. We should conduct further research in more hospitals in more regions. Second, the potential for the risk of contamination involves nurses from different groups in the same hospital who may communicate with each other during their non-working hours. Therefore, before the study, we will carry out a training program and ask nurses in the two groups to avoid discussing this study during breaks. Finally, long-term outcomes are not designed to be assessed in our study. Despite challenges, these limitations will be reduced using rigorous methodology. The issues in this study should be addressed in future studies to further improve efficiency and effectiveness.

Conclusions

The findings from this study will highlight the effectiveness of the CDSS-AntiDelirium for adherence to delirium intervention and cognitive load among ICU nurses and will encourage hospital administrators to adopt a CDSS for complex activities. In addition, this study will clarify the mechanism of the CDSS-AntiDelirium in improving adherence to delirium intervention among ICU nurses and will provide a theoretical and methodological basis for improving adherence to guidelines.

Supporting information

S1 File. SPIRIT 2013 checklist.

(PDF)

Click here for additional data file.^{(94.6KB, pdf)}

S2 File. Clinical Decision Support System Assisted prevention and management for Delirium (CDSS-AntiDelirium).

(PDF)

Click here for additional data file.^{(685.4KB, pdf)}

S3 File. Paper-based individual interventions.

(PDF)

Click here for additional data file.^{(328.8KB, pdf)}

S4 File. Research project submitted to the ethics committee (English).

(PDF)

Click here for additional data file.^{(610.3KB, pdf)}

S5 File. Research project submitted to the ethics committee (Chinese).

(PDF)

Click here for additional data file.^{(475.7KB, pdf)}

S6 File

(PDF)

Click here for additional data file.^{(474.8KB, pdf)}

Acknowledgments

We extend our sincere thanks to Professor Meihua Ji (School of Nursing, Capital Medical University in Beijing, China) for her editing assistance.

Data Availability

No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.

Funding Statement

This research was supported by Grant 72304196 from the National Natural Science Foundation of China. The funders had and will not have a role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

1.Ali M, Cascella M. ICU Delirium: Treasure Island (FL): StatPearls Publishing. 2022. [PubMed] [Google Scholar]
2.Martinez FE, Ho VW, Brogan G, Pickering C, Leung G, Parrey J, et al. Period-Prevalence and Risk Factors for Delirium in a Tertiary Referral ICU: A prospective cohort study. Acute Med. 2021;20(3):168–173. [PubMed] [Google Scholar]
3.Stollings JL, Kotfis K, Chanques G, Pun BT, Pandharipande PP, Ely EW. Delirium in critical illness: clinical manifestations, outcomes, and management. Intensive Care Med. 2021;47(10):1089–1103. doi: 10.1007/s00134-021-06503-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Devlin JW, Needham DM. Long-Term Outcomes after Delirium in the ICU: Addressing Gaps in Our Knowledge. Am J Respir Crit Care Med. 2021;204(4):383–385. doi: 10.1164/rccm.202104-0910ED [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Devlin JW, Skrobik Y, Gelinas C, Needham DM, Slooter A, Pandharipande PP, et al. Clinical Practice Guidelines for the Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption in Adult Patients In the ICU. Crit Care Med. 2018;46(9):e825–e873. doi: 10.1097/CCM.0000000000003299 [DOI] [PubMed] [Google Scholar]
6.Zhang S, Han Y, Xiao Q, Li H, Wu Y. Effectiveness of Bundle Interventions on ICU Delirium: A Meta-Analysis. Crit Care Med. 2021;49(2):335–346. doi: 10.1097/CCM.0000000000004773 [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Balas MC, Burke WJ, Gannon D, Cohen MZ, Colburn L, Bevil C, et al. Implementing the awakening and breathing coordination, delirium monitoring/management, and early exercise/mobility bundle into everyday care: opportunities, challenges, and lessons learned for implementing the ICU Pain, Agitation, and Delirium Guidelines. Crit Care Med. 2013;41(9 Suppl 1):S116–27. doi: 10.1097/CCM.0b013e3182a17064 [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Alhaidari AA, Allen-Narker RA. An evolving approach to delirium: A mixed-methods process evaluation of a hospital-wide delirium program in New Zealand. Australas J Ageing. 2017;36(2):E20–E26. doi: 10.1111/ajag.12406 [DOI] [PubMed] [Google Scholar]
9.Hosie A, Lobb E, Agar M, Davidson PM, Phillips J. Identifying the barriers and enablers to palliative care nurses’ recognition and assessment of delirium symptoms: a qualitative study. J Pain Symptom Manage. 2014;48(5):815–30. doi: 10.1016/j.jpainsymman.2014.01.008 [DOI] [PubMed] [Google Scholar]
10.Vogels J, Demberg V, Kray J. The Index of Cognitive Activity as a Measure of Cognitive Processing Load in Dual Task Settings. Front Psychol. 2018;9:2276. doi: 10.3389/fpsyg.2018.02276 [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Kirschner PA, Sweller J, Kirschner F, Zambrano RJ. From Cognitive Load Theory to Collaborative Cognitive Load Theory. Int J Comput Support Collab Learn. 2018;13(2):213–233. doi: 10.1007/s11412-018-9277-y [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Sweller J. Cognitive load during problem solving: Effects on learning. Cogn Sci. 1988;12:257–285. doi: 10.1016/0364-0213(88)90023-7 [DOI] [Google Scholar]
13.Paas F, Renkl A, Sweller J. Cognitive Load Theory and Instructional Design: Recent Developments. Educ Psychol (Lond). 2003;38(1):1. doi: 10.1207/S15326985EP3801_1 [DOI] [Google Scholar]
14.Paas FG, Van Merrienboer JJ, Adam JJ. Measurement of cognitive load in instructional research. Percept Mot Skills. 1994;79(1 Pt 2):419–30. doi: 10.2466/pms.1994.79.1.419 [DOI] [PubMed] [Google Scholar]
15.Wilby KJ, Paravattil B. Cognitive load theory: Implications for assessment in pharmacy education. Res Social Adm Pharm. 2020; doi: 10.1016/j.sapharm.2020.12.009 [DOI] [PubMed] [Google Scholar]
16.Blissett S, Rodriguez S, Qasim A, O’Sullivan P. Learning Echocardiography in the Workplace: A Cognitive Load Perspective. Acad Med. 2020; doi: 10.1097/ACM.0000000000003789 [DOI] [PubMed] [Google Scholar]
17.Gomez TL, Henao-Castano AM, Troche-Gutierrez IY. Prevention and treatment of delirium in intensive care: Hermeneutics of experiences of the nursing team. Enferm Intensiva (Engl Ed). 2021; doi: 10.1016/j.enfi.2021.05.001 [DOI] [PubMed] [Google Scholar]
18.Ceballos-Vasquez P, Rolo-Gonzalez G, Hernandez-Fernaud E, Diaz-Cabrera D, Paravic-Klijn T, Burgos-Moreno M. Psychosocial factors and mental work load: a reality perceived by nurses in intensive care units. Rev Lat Am Enfermagem. 2015;23(2):315–22. doi: 10.1590/0104-1169.0044.2557 [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Yu C, Jiang J, Zhong M, Zhang H, Duan X. Training load of newly recruited nurses in Grade-A Tertiary Hospitals in Shanghai, China: a qualitative study. BMC Nurs. 2023;22(1):9. doi: 10.1186/s12912-022-01138-z [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. doi: 10.1038/s41746-020-0221-y [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Kindle RD, Badawi O, Celi LA, Sturland S. Intensive Care Unit Telemedicine in the Era of Big Data, Artificial Intelligence, and Computer Clinical Decision Support Systems. Crit Care Clin. 2019;35(3):483–495. doi: 10.1016/j.ccc.2019.02.005 [DOI] [PubMed] [Google Scholar]
22.Dal Sasso GM, Barra DC. Cognitive Workload of Computerized Nursing Process in Intensive Care Units. Comput Inform Nurs. 2015;33(8):339–45; quiz E1. doi: 10.1097/CIN.0000000000000157 [DOI] [PubMed] [Google Scholar]
23.Wu P, Nam MY, Choi J, Kirlik A, Sha L, Berlin RJ. Supporting Emergency Medical Care Teams with an Integrated Status Display Providing Real-Time Access to Medical Best Practices, Workflow Tracking, and Patient Data. J Med Syst. 2017;41(12):186. doi: 10.1007/s10916-017-0829-x [DOI] [PubMed] [Google Scholar]
24.Salvemini AV. Improving the human-computer interface: a human factors engineering approach. MD Comput. 1998;15(5):311–5. [PubMed] [Google Scholar]
25.Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586. doi: 10.1136/bmj.e7586 [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Fan H, Ji M, Huang J, Yue P, Yang X, Wang C, et al. Development and validation of a dynamic delirium prediction rule in patients admitted to the Intensive Care Units (DYNAMIC-ICU): A prospective cohort study. Int J Nurs Stud. 2019;93:64–73. doi: 10.1016/j.ijnurstu.2018.10.008 [DOI] [PubMed] [Google Scholar]
27.Leppink J, Paas F, Van der Vleuten CP, Van Gog T, Van Merrienboer JJ. Development of an instrument for measuring different types of cognitive load. Behav Res Methods. 2013;45(4):1058–72. doi: 10.3758/s13428-013-0334-1 [DOI] [PubMed] [Google Scholar]
28.Zhang S, Wu Y, Fu Z, Lu Y, Wang Q, Mingxuan L. Psychometric properties of the Chinese version of the instrument for measuring different types of cognitive load (MDT-CL). J Nurs Manag. 2020;28(2):277–285. doi: 10.1111/jonm.12919 [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Shan Z, Yuan H, Ying WU. Development and psychometric testing of ICU Delirium Knowledge Questionnaire. Chinese Nursing Management. 2020;20(1):120–124. doi: 10.3969/j.issn.1672-1756.2020.01.025 [DOI] [Google Scholar]
30.Trogrlic Z, van der Jagt M, Lingsma H, Gommers D, Ponssen HH, Schoonderbeek J, et al. Improved Guideline Adherence and Reduced Brain Dysfunction After a Multicenter Multifaceted Implementation of ICU Delirium Guidelines in 3,930 Patients. Crit Care Med. 2019;47(3):419–427. doi: 10.1097/CCM.0000000000003596 [DOI] [PubMed] [Google Scholar]
31.Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, et al. Fall prevention in acute care hospitals: a randomized trial. JAMA. 2010;304(17):1912–8. doi: 10.1001/jama.2010.1567 [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1–55. [Google Scholar]
33.Cohen J. Statistical power analysis for the behavioral sciences. J Am Stat Assoc. 1989;84:19–74. [Google Scholar]
34.Wang PF, Shen LQ, Zhang HJ, Li BH, Ji H. A Nursing Pain Assessment and Record Information System: Design and Application in the Oncology Department. Comput Inform Nurs. 2017;35(12):647–652. doi: 10.1097/CIN.0000000000000373 [DOI] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0293950.r001

Decision Letter 0

Sascha Köpke

25 Jun 2023

PONE-D-23-02468Impact of Artificial Intelligence Assisted Prevention and Management for Delirium on cognitive load in Intensive Care Unit nurses (AID-ICU): protocol of a multicenter, cluster randomized trialPLOS ONE

Dear Dr. Wu,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The protocol has been assessed by two peer reviewers and both point out important aspects to be addressed in a revision. Importantly, these include the rationale of the study, the reporting of outcome measures and the rationale and the nature of the intervention. Please make sure that these aspects are clearly addressed and clarified.

Please submit your revised manuscript by Aug 05 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sascha Köpke

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

3. Thank you for stating the following in your Competing Interests section:

NO authors have competing interests

Please complete your Competing Interests on the online submission form to state any Competing Interests. If you have no competing interests, please state "The authors have declared that no competing interests exist.", as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now

This information should be included in your cover letter; we will change the online submission form on your behalf.

4. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

5. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please delete it from any other section.

Reviewers' comments: Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: No

Reviewer #2: Yes

**********

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: No

Reviewer #2: Partly

**********

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: No

Reviewer #2: No

**********

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

6. Review Comments to the Author

Reviewer #1: Dear authors,

thank you very much for the opportunity to review the submitted manuscript. The manuscript includes a protocol of a RCT assessing the impact of an artificial intelligence delirium-tool on cognitive load of critical care nurses.

I have reviewed hundreds of scientific manuscripts, but I have never read a manuscript that insults and belittles nurses in intensive care units in such a way. The very first sentence, "Higher cognitive load of nurses is a common problem in the intensive care units, which is associated with nurses’ poor performance of activities, as well as related to poor patient outcomes," implies that nurses are often cognitively overloaded, unable to do their jobs cognitively, and therefore harm patients. It is an insult to all nurses who work hard, take justifiable pride in their work, and study and gain further qualifications on the side. For ethical and professional policy reasons, this manuscript cannot be approved for publication.

The authors' reasoning is inconsequential. The rationale cites literature in which physicians as well as nurses have been studied in cognitive load in simulation centres, with no differences. Other literature cites consensus plans and curricula. There is no evidence of cognitive overload in nurses.

Further, primary and secondary outcome parameters are confounded for this study, which is actually a QI project.

Reviewer #2: Thank you very much for the very interesting protocol. Please bear in mind that my suggestions are meant to improve the manuscript. I hope you feel your work is appreciated.

You would have to clarify why the outcome cognitive load is described in the title of the manuscript, although it is described as a secondary outcome in the text.

Intervention AI-AntiDelirium:

• Z.85: You write that the AI AntiDelirium was developed on the basis of Cognitive Load Theory. Please describe this process.

• Please briefly describe the basic operation of AI. If necessary, a conceptual differentiation from clinical decision support systems (CDSS) would be necessary.

• Are data transformation algorithms predetermined or adaptive?

• Who was involved in the development of AI?

• Some information is still missing to describe the intervention. You could orientate yourself on the TiDier checklist. In particular, it would be interesting to know whether the intervention takes place in the form of an APP on a tablet or possibly integrated into a patient data management system.

• The algorithm linking the identified delirium risk factors to the Nursing Care Plan should be described.

Dynamic ICU Delirium Prediction Rule

• At what APACHE-II score are 3 points assigned? At a cut-off point above 7.5 (Fan et al. 2019)?

• The weighting of a very high score for Hearing Deficits (9 points) and Indwelling Catheter (8 points) can be derived from the preliminary study. How empirically reliable is this information in general and in particular with very wide confidence intervals?

Control Group

• You mention the PADIS guideline, but the description of the individual interventions or bundle interventions would be useful.

Adherence:

• Perhaps it would be useful to describe the calculation of adherence (a-c), with an example and/or a formula.

• Z.177: Is actual delirium risk factors assessment meant here?

• Z.172: Is this about adherence to the delirium risk factors assessment in general, or to the assessment of individual risk factors?

• Z.225: You assume an adherence of 80% for the AI-AntiDelirium group and 50% for the paper-based PADIS group and refer to the study by Trogrlic et al. 2018. This assumption should be explained.

PLoS One. 2023 Nov 28;18(11):e0293950. doi: 10.1371/journal.pone.0293950.r002

Author response to Decision Letter 0

11 Jul 2023

Reviewers' comments: Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: No

Reviewer #2: Yes

Response: We modified the relevant sentence in the introduction to make it more clear. As following:

However, the impact of AI-AntiDelirium on adherence and cognitive load in ICU nurses is still unclear. Therefore, this study aims to conduct a multicenter, cluster randomized trial (RCT) to investigate the effectiveness of AI-AntiDelirium on improving adherence to delirium intervention and reducing cognitive load of ICU nurses and reveal the mechanism of adherence improvement in delivering the delirium intervention.

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

Reviewer #1: No

Reviewer #2: Partly

Response: We added the relevant sentence in the Materials and methods. As following:

Before the study, uniform training will be delivered by research staff to data collectors, including theoretical knowledge and assessment skills for delirium, in order to maintain high consistency of delirium assessment in all included units. To minimize error and maximize reliability, the project director performed the following: providing intensive training to the assessors including a review of the procedure outlined based on a standardized patient to ensure high inter-rater reliability (Kappa > 0.9); meeting with the assessors every week to review procedures and check the quality of the assessment for the primary outcome.

In order to avoid the reduction in statistical efficiency and bias caused by missing data, multiple imputations was performed to create a complete dataset assuming that data is lost at random.

The influence of potential covariates, e.g., age, gender, educational level, marital status, departments, years of ICU experience, and level of professional title, will be collected by including their interaction with the intervention in RMANOVA.

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Reviewer #1: No

Reviewer #2: No

Response: We provided more detailed information in the revised manuscript to allow the work to be replicable based on the Template for Intervention Description and Replication (TIDieR) checklist, please see the detailed information in the “Intervention group and control group, Table 1, Figure 2, S3 File, S4 File and sample size calculations”.

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

Reviewer #1: No

Reviewer #2: No

Response: All data underlying the findings will be shared through the Baidu network (https://pan.baidu.com/s/1qVeHCHE23n75eGWJBOz1ZQ). We added this sentence in the “dissemination” section.

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: No

Reviewer #2: Yes

Response:

To present the manuscript in an intelligible fashion and written in standard English, Professor Meihua Ji (United States Registered Nurse) guided us in English writing and had good writing skills. She graduated from the University of Pittsburgh School of Nursing in 2018 and worked in the School of Nursing, Capital Medical University in 2019 through the Overseas Talent Introduction Program. We extend our thanks to Professor Meihua Ji in the Acknowledgments section. As following:

We extend our sincere thanks to Professor Meihua Ji ( School of Nursing, Capital Medical University in Beijing, China) for her editing assistance.

6. Review Comments to the Author

Reviewer #1: Dear authors,

Further, primary and secondary outcome parameters are confounded for this study, which is actually a QI project.

Response: Thank you very much for pointing out this key issue.

Firstly, we are very sorry for the misunderstanding caused by our expression, as we all know, nursing care in the intensive care units is characterized by elevated demand on workloads and increased cognitive load. ICUs require higher levels of nurse staffing than other health care environments, because of the demands of close monitoring and observation of patients and necessity of managing technological life support equipment for severely ill patients, which can ensure the quality of care and patient safety. We have great respect for ICU nurses, and we are part of the team of nurses. We modified the relevant sentence in the abstract and introduction. As following:

[Abstract]

Adherence to the delirium bundle intervention is sub-optimal in routine practice, and inappropriate use of the instructional design of interventions may result in higher cognitive load among nurses.

[Introduction]

Higher extraneous cognitive load has been identified as one of the most important problems in providing intensive care. It negatively impacts nurses, such as reduced learning capacity for acquiring knowledge.

Secondly, as for the cites literature about the evidence of cognitive overload in nurses, we substituted more suitable literature. As following:

Higher extraneous cognitive load has been identified as one of the most important problems in providing intensive care[17,18]. It negatively impacts nurses, such as reduced learning capacity for acquiring knowledge[17,19].

Reference

[17] B. Han, D. B. Gal, M. Mafla, L. D. Sacks, A. T. Singh, and A. Y. Shin. 2021. Role of Texting as a Source of Cognitive Burden in a Pediatric Cardiovascular ICU. Hosp Pediatr (2021), e253-e257. DOI 10.1542/hpeds.2021-005869

[18] Tovar LO Gomez, A. M. Henao-Castano, and I. Y. Troche-Gutierrez. 2021. Prevention and treatment of delirium in intensive care: Hermeneutics of experiences of the nursing team. Enferm Intensiva (Engl Ed) (2021). DOI 10.1016/j.enfi.2021.05.001

[19] C. Yu, J. Jiang, M. Zhong, H. Zhang, and X. Duan. 2023. Training load of newly recruited nurses in Grade-A Tertiary Hospitals in Shanghai, China: a qualitative study. BMC Nurs. (2023), 9. DOI 10.1186/s12912-022-01138-z

Thirdly, we are very sorry that our expressions in the title and the methodology were not clear, which caused reviewer to be confused about the primary outcome and the secondary outcome. We have made the corrections in the Title. As following:

“Impact of Artificial Intelligence Assisted Prevention and Management for Delirium on guideline adherence and cognitive load in Intensive Care Unit nurses (AID-ICU): protocol of a multicenter, cluster randomized trial”.

Reviewer #2: Thank you very much for the very interesting protocol. Please bear in mind that my suggestions are meant to improve the manuscript. I hope you feel your work is appreciated.

You would have to clarify why the outcome cognitive load is described in the title of the manuscript, although it is described as a secondary outcome in the text.

Response: Thanks a lot for your very helpful comment. In our study, the primary outcome will be adherence to delirium guideline intervention, the secondary endpoint will be the cognitive load. We revised the title into Impact of Artificial Intelligence Assisted Prevention and Management for Delirium on guideline adherence and cognitive load in Intensive Care Unit nurses (AID-ICU): protocol of a multicenter, cluster randomized trial.

Intervention AI-AntiDelirium:

• Z.85: You write that the AI AntiDelirium was developed on the basis of Cognitive Load Theory. Please describe this process.

Response: We added more information about the development of AI-AntiDelirium in the introduction section, as following:

Therefore, we developed the Artificial Intelligence Assisted Prevention and Management for Delirium (AI-AntiDelirium) based on the Cognitive Load Theory and Human factors engineering (HFE), which were used to guide the division of functional modules, process design, and user interface design throughout the development process. There are five steps to design and develop AI-AntiDelirium, including (1) assessing the reasons for non-adherence to the PADIS guidelines and the needs of users for the functionality, workflow, and UI of AI-AntiDelirium; (2) designing the functional modules following PADIS guidelines and nursing processes; (3) iteratively designing the system structure and the UI of AI-AntiDelirium; (4) improving the AI-AntiDelirium based on end-users feedback in a pilot testing; (5) usability testing of AI-AntiDelirium.

• Please briefly describe the basic operation of AI. If necessary, a conceptual differentiation from clinical decision support systems (CDSS) would be necessary.

Response: We added the information about the basic operation of the AI-AntiDelirium in the “Intervention group” section, and also in Figure 1 and Table 1, as following:

AI-AntiDelirium is installed on Personal Digital Assistant (PDA), where nurses need to complete necessary registration information including hospital, department, name, password, etc. And then nurses can log into the system and (1) Enter the delirium assessment tools module to perform ICU delirium assessment at least two times (8:00-10:00 a.m., 4:00-6:00p.m.) per day; (2) Enter the risk factor assessment module to assess patient's existing risk factors, and a dynamic delirium prediction rule for ICU patients (DYNAMIC-ICU) would be utilized to classify patients into different risk levels; (3) Enter the nursing care plan module to confirm the plan, which is automatically tailored based on the results of risk factors assessment; (4) Enter the nursing checklist module to check the specific execution timetable of each intervention based on the results of the nursing care plan.

• Are data transformation algorithms predetermined or adaptive?

Response: The data transformation algorithms are predetermined.

• Who was involved in the development of AI?

Response: The design and development of the AI-AntiDelirium were completed by a multidisciplinary team, which included healthcare professionals (clinical experts, ICU nurses) as end-users, nursing researchers, nursing informaticians, and IT engineers (UI designers, architects, programmers) who specialized in developing clinical nursing information systems and the health-related mobile applications. The team members continued to discuss online daily and meet weekly to discuss any revisions required throughout the whole process. As for the detailed information about the development of the AI-AntiDelirium was described in another article, therefore, we provide the information in this response letter.

Response: Thanks a lot for your professional and helpful comment. We added more information based on the Template for Intervention Description and Replication (TIDieR) checklist. As following:

The appropriate subset of interventions from the PADIS guideline (such as the ABCDEF bundle) will be tailored to the patient's specific risk factors by AI-AntiDelirium. The intervention is targeted at eliminating various modifiable risk factors of ICU delirium, including hearing impairment (speaking loudly, slowly, and patiently with the patient; assisting the patient to wear hearing aids correctly, etc.), visual impairment (assisting the patient to wear eyeglasses correctly, etc.), pain (provide non-pharmacological interventions, such as distraction and relaxation therapy, etc.), use of anesthetic or sedatives (implementation of the spontaneous awakening trial, etc.), mechanical ventilation (conduct spontaneous breathing trials according to the doctor's order, etc.), indwelling catheter (remove the catheter as soon as possible, etc.), infection (reduce invasive operations and avoid unnecessary catheterization, etc.), immobility (help patients to do passive range-of-motion exercises, 10 times for each joint, etc.), sleep deprivation (assist patients to wear earplugs or anti-noise equipment, etc.), and no family members visit (family visits are encouraged, etc.). These interventions are delivered 7 days a week primarily by nurses in a face-to-face mode, until the patient’s discharge. Nursing care activities varied with a specific frequency, duration, intensity, or dose.

AI-AntiDelirium is installed on Personal Digital Assistant (PDA), where nurses need to complete necessary registration information including hospital, department, name, password, etc.

• The algorithm linking the identified delirium risk factors to the Nursing Care Plan should be described.

Response: Thank you very much for the comment. We described the algorithm linking the identified delirium risk factors to the nursing care plan, as following:

A set of IF-THEN rules in AI-AntiDelirium were stored and designed to trigger the nursing care plan based on the identified delirium risk factors.

Dynamic ICU Delirium Prediction Rule

• At what APACHE-II score are 3 points assigned? At a cut-off point above 7.5 (Fan et al. 2019)?

Response: We revised the sentence to make it easier to understand in Dynamic ICU Delirium Prediction Rule section in the S3 File, as following:

Predictors included in the model including the APACHE-II score above 7.5, history of chronic disease, sleep deprivation, use of anesthetic or sedatives, infection, indwelling catheter, and hearing impairment. The simplified score for each predictor was 3, 3, 3, 3, 4, 8, and 9 points.

Response: In our previous article[1], a dynamic delirium prediction rule in

patients admitted to the Intensive Care Units (DYNAMIC-ICU) were constructed.

DYNAMIC-ICU has the highest predictive value of stratifying ICU patients into different levels of risk in delirium development with the C statistic of 0.907 based on seven risk factors and 0.888 based on total risk scores of seven factors in the derivation cohort (n=336) and 0.900 in the validation cohort (n=224). In order to allow us to draw more reliable and valid conclusions, firstly, all patients were undergone extensive assessment with interviews and a review of the medical record to collect necessary information on possible risk factors. Secondly, given that our study was an observational cohort study, other confounding factors would bias the model results. To control bias as much as possible, we conducted univariate analysis and logistic regression, and the final model performed well in both the derivation and external validation cohorts. Thirdly, we applied mean imputation and bootstrap resampling procedures to augment data for deriving predictors in our prediction rule. These advanced statistical methods may minimize the effects of missing data and limit model overfitting to ensure more stable variables are included for prediction rule establishment and validation. In the future, we can further examine the role of different risk factors in the dynamic delirium prediction rule.

[1]Fan H，Ji M，Huang J，et al. Development and validation of a dynamic delirium prediction rule in patients admitted to the Intensive Care Units (DYNAMIC-ICU): A prospective cohort study. Int J Nurs Stud, 2019, 93: 64-73.

Control Group

• You mention the PADIS guideline, but the description of the individual interventions or bundle interventions would be useful.

Response: Thanks very much for your professional suggestion. We revised the sentence to make it clear. As following:

“In addition, nurses are trained to use paper-based individual interventions.”

And the description of the control group (paper-based individual interventions) is uniformly modified in the revised manuscript.

Adherence:

• Perhaps it would be useful to describe the calculation of adherence (a-c), with an example and/or a formula.

Response: Thank you very much for your kind suggestion. We described the calculation of adherence (a-c) with a formula. As following:

a.Adherence to delirium interventions is calculated as (the number of interventions implemented / the total number of interventions expected) × 100%. The total number of interventions that should be implemented in accordance with the nursing care plan. For example, an individualized delirium care activity checklist with 10 interventions is tailored by AI-AntiDelirium based on the results of the risk factors assessment. However, the bedside nurse only completes 6 interventions during the whole shift, therefore, the adherence to delirium interventions is (6/10) × 100% = 60%. During a whole shift, a bedside nurse will take care of three to five ICU patients, therefore, the total adherence to delirium interventions per day is calculated as (the number of interventions implemented for all cared patients/ the total number of interventions expected for all cared patients) × 100%.

b.Adherence to risk factors assessment is defined as (the number of risk factors assessed / the total number of risk factors assessments expected) × 100%. The total number of risk factors that should be assessed in accordance with the risk factors assessment sheet, including hearing impairment, visual impairment, pain, use of anesthetic or sedatives, mechanical ventilation, indwelling catheter, infection, immobility, sleep disorders, and no family members visit. The total adherence to risk factors assessment per day is calculated as (the number of risk factors assessed for all cared patients/ the total number of risk factors assessments expected for all cared patients) × 100%.

c.Adherence to ICU delirium assessment is defined as (the number of ICU delirium assessed / the total number of ICU delirium assessment expected) × 100%. The total number of ICU delirium assessments for each patient is calculated two times (8 to 10 A.M. and 4 to 6 P.M.) per day. The total adherence to ICU delirium assessment per day is calculated as (the number of ICU delirium assessed for all cared patients/ the total number of ICU delirium assessments expected for all cared patients) × 100%.

• Z.177: Is actual delirium risk factors assessment meant here?

Response: Yes, it is the actual number of delirium risk factors assessed. We revised the sentence more specifically to make it clear how to calculate the adherence to risk factors assessment. As following:

• Z.172: Is this about adherence to the delirium risk factors assessment in general, or to the assessment of individual risk factors?

Response: In this revised manuscript, we described both adherence to the assessment of individual risk factors and adherence to the delirium risk factors assessment in general. In the finally statistical analysis, adherence to the delirium risk factors assessment in general is used to represent adherence to the delirium risk factors assessments.

• Z.225: You assume an adherence of 80% for the AI-AntiDelirium group and 50% for the paper-based PADIS group and refer to the study by Trogrlic et al. 2018. This assumption should be explained.

Response: A sample size calculation should specify a minimum treatment effect that is both clinically important and likely to be achievable based on evidence from previous trials of similar interventions. In the study by Trogrlic et al. (2018), they designed an e-learning program for delirium screening (Phase II) and delirium guideline (Phase III). The primary outcome was adherence changes to delirium guidelines recommendations, and the study showed improved health professionals’ adherence to delirium guidelines. The average adherence to delirium guidelines (delirium screening, sedation assessments, and light sedation; Top 3 adherence) after the e-learning program for guideline implementation was about 80%, and the average adherence to delirium guidelines for the baseline group was about 50%. Therefore, we assume an adherence of 80% for the AI-AntiDelirium group (intervention group) and 50% for the paper-based individual interventions (control group).

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(25.4KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0293950.r003

Decision Letter 1

Sascha Köpke

25 Aug 2023

PONE-D-23-02468R1Impact of Artificial Intelligence Assisted Prevention and Management for Delirium on guideline adherence and cognitive load in Intensive Care Unit nurses (AID-ICU): protocol of a multicenter, cluster randomized trialPLOS ONE

Dear Dr. Wu,

Please submit your revised manuscript by Oct 09 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Sascha Köpke

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #2: Partly

**********

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

Reviewer #2: Yes

**********

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Reviewer #2: No

**********

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: Dear authors,

I am very pleased that you have implemented my suggestions. There are still some ambiguities that are probably easy to implement.

Z.72 - 75: The cited literature by Han et al. 2021 describes communication via text messaging in an intensive care unit .

Z.86 With regard to the development of the AI-AntiDelirium, you refer to another publication as well as to 5 steps of the development of the AI-AntiDelirium. Were the results of the development process published? If not, would a separate publication be interesting?

Is the AI-AntiDelirium really artificial intelligence? Is it not rather a Clinical Decision Support System (CDSS) that acts on the basis of predefined IF-Then rules? You might want to reconsider the term Artificial Intelligence in the title and in the acronym "AI-AntiDelirium" and in other places.

203: The calculation of adherence is now more comprehensible. However, the adherence cannot be between 0 and 1 if the calculation of the formulas a, b, c is done according to a different systematic.

257: The description of the level of knowledge of the nurses rather belongs to the chapter Outcomes.

243: You assume that the recruitment of nurses will be completed by the end of June 2023. What is the current status of recruitment? Are changes to the protocol still feasible?

Best regards

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Nov 28;18(11):e0293950. doi: 10.1371/journal.pone.0293950.r004

Author response to Decision Letter 1

29 Aug 2023

Reviewer #2: Dear authors,

I am very pleased that you have implemented my suggestions. There are still some ambiguities that are probably easy to implement.

Z.72 - 75: The cited literature by Han et al. 2021 describes communication via text messaging in an intensive care unit .

Response: We replaced the cited literature by Han et al. 2021 as Ceballos-Vasquez et al:

Ceballos-Vasquez P, Rolo-Gonzalez G, Hernandez-Fernaud E, Diaz-Cabrera D, Paravic-Klijn T, Burgos-Moreno M. Psychosocial factors and mental work load: a reality perceived by nurses in intensive care units. Rev Lat Am Enfermagem. 2015;23(2):315-22. doi: 10.1590/0104-1169.0044.2557.

Response: We appreciate the reviewer very much for your positive comments and interest in the development of the AI-AntiDelirium. Detailed information about the Development of an Artificial Intelligence-Assisted Prevention and Management for Delirium (AI-AntiDelirium) is under review in a peer-reviewed journal.

Response: Thanks a lot for your very helpful comment. As you suggested, we modified the Title as following:

Impact of Clinical Decision Support System Assisted Prevention and Management for Delirium on guideline adherence and cognitive load in Intensive Care Unit nurses (CDSSD-ICU): protocol of a multicenter, cluster randomized trial.

In addition, we revised the the acronym "AI-AntiDelirium" into "CDSS-AntiDelirium".

203: The calculation of adherence is now more comprehensible. However, the adherence cannot be between 0 and 1 if the calculation of the formulas a, b, c is done according to a different systematic.

Response: Thanks a lot for your kindly comment. We revised the calculation of adherence as following:

Complete adherence rate is recorded as 100%, and complete non-adherence rate is recorded as 0%. Adherence rate is between 0 and 100 according to the following formula.

257: The description of the level of knowledge of the nurses rather belongs to the chapter Outcomes.

Response: As you suggested, we moved the level of knowledge of the nurses into the Outcome measures section.

243: You assume that the recruitment of nurses will be completed by the end of June 2023. What is the current status of recruitment? Are changes to the protocol still feasible?

Response: The current status of recruitment is finished, between November 2022 and June 2023, 38 nurses (mean [SD] age, 33.29 [0.88] years; 33 [86.8%] female) are included in the intervention group, and 42 nurses (mean [SD] age, 33.52 [0.79] years; 39 [92.9%] female) are recruited in the control group. Changes to the protocol are still feasible, we continue to collect the outcomes among recruiting ICU nurses.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(17.2KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0293950.r005

Decision Letter 2

Sascha Köpke

11 Sep 2023

PONE-D-23-02468R2Impact of Clinical Decision Support System Assisted Prevention and Management for Delirium on guideline adherence and cognitive load in Intensive Care Unit nurses (CDSSD-ICU): protocol of a multicenter, cluster randomized trialPLOS ONE

Dear Dr. Wu,

Please submit your revised manuscript by Oct 26 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Sascha Köpke

Academic Editor

PLOS ONE

Journal Requirements:

Additional Editor Comments:

The paper has again been reviewed and the reviewer is mostly happy with the manuscript.

However, before publication,there are three more aspects to be addressed:

(1) Most importantly, please apply another language check preferably by a professional language editor as there are still a number of inadequate expressions and language use is often incorrect.

(2) In l.139, you state that “Investigators who will enroll participants are not made aware of the randomization list.” Please be clearer about this process. As “post-randomization recruitment bias” is a common risk of bias in cluster randomized studies, it is important to know, who is recruiting patients and determining their eligibility.

(3) Under “Patient and public involvement” (l.347ff.), please be more specific. Information is not involvement. So if there is no active role in the sense of PPI, please state this clearly.

[Note: HTML markup is below. Please do not edit.]

PLoS One. 2023 Nov 28;18(11):e0293950. doi: 10.1371/journal.pone.0293950.r006

Author response to Decision Letter 2

10 Oct 2023

Additional Editor Comments:

The paper has again been reviewed and the reviewer is mostly happy with the manuscript.

However, before publication,there are three more aspects to be addressed:

(1) Most importantly, please apply another language check preferably by a professional language editor as there are still a number of inadequate expressions and language use is often incorrect.

Response: Thanks a lot for your comment. As you suggested, we polished the manuscript by a professional language expert.

Response: We provided more information about this process, as following:

(3) Under “Patient and public involvement” (l.347ff.), please be more specific. Information is not involvement. So if there is no active role in the sense of PPI, please state this clearly.

Response: We provided more detailed information in the “Patient and public involvement”, as following:

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(13KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0293950.r007

Decision Letter 3

Sascha Köpke

24 Oct 2023

Impact of Clinical Decision Support System Assisted Prevention and Management for Delirium on guideline adherence and cognitive load among Intensive Care Unit nurses (CDSSD-ICU): protocol of a multicentre, cluster randomized trial

PONE-D-23-02468R3

Dear Dr. Wu,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sascha Köpke

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

The authors have adequately responded to the final recommendations.

Reviewers' comments:

PLoS One. doi: 10.1371/journal.pone.0293950.r008

Acceptance letter

Sascha Köpke

13 Nov 2023

PONE-D-23-02468R3

Dear Dr. Wu:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Professor Sascha Köpke

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 File. SPIRIT 2013 checklist.

(PDF)

Click here for additional data file.^{(94.6KB, pdf)}

S2 File. Clinical Decision Support System Assisted prevention and management for Delirium (CDSS-AntiDelirium).

(PDF)

Click here for additional data file.^{(685.4KB, pdf)}

S3 File. Paper-based individual interventions.

(PDF)

Click here for additional data file.^{(328.8KB, pdf)}

S4 File. Research project submitted to the ethics committee (English).

(PDF)

Click here for additional data file.^{(610.3KB, pdf)}

S5 File. Research project submitted to the ethics committee (Chinese).

(PDF)

Click here for additional data file.^{(475.7KB, pdf)}

S6 File

(PDF)

Click here for additional data file.^{(474.8KB, pdf)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(25.4KB, docx)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(17.2KB, docx)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(13KB, docx)}

Data Availability Statement

No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.

[pone.0293950.ref001] 1.Ali M, Cascella M. ICU Delirium: Treasure Island (FL): StatPearls Publishing. 2022. [PubMed] [Google Scholar]

[pone.0293950.ref002] 2.Martinez FE, Ho VW, Brogan G, Pickering C, Leung G, Parrey J, et al. Period-Prevalence and Risk Factors for Delirium in a Tertiary Referral ICU: A prospective cohort study. Acute Med. 2021;20(3):168–173. [PubMed] [Google Scholar]

[pone.0293950.ref003] 3.Stollings JL, Kotfis K, Chanques G, Pun BT, Pandharipande PP, Ely EW. Delirium in critical illness: clinical manifestations, outcomes, and management. Intensive Care Med. 2021;47(10):1089–1103. doi: 10.1007/s00134-021-06503-1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref004] 4.Devlin JW, Needham DM. Long-Term Outcomes after Delirium in the ICU: Addressing Gaps in Our Knowledge. Am J Respir Crit Care Med. 2021;204(4):383–385. doi: 10.1164/rccm.202104-0910ED [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref005] 5.Devlin JW, Skrobik Y, Gelinas C, Needham DM, Slooter A, Pandharipande PP, et al. Clinical Practice Guidelines for the Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption in Adult Patients In the ICU. Crit Care Med. 2018;46(9):e825–e873. doi: 10.1097/CCM.0000000000003299 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref006] 6.Zhang S, Han Y, Xiao Q, Li H, Wu Y. Effectiveness of Bundle Interventions on ICU Delirium: A Meta-Analysis. Crit Care Med. 2021;49(2):335–346. doi: 10.1097/CCM.0000000000004773 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref007] 7.Balas MC, Burke WJ, Gannon D, Cohen MZ, Colburn L, Bevil C, et al. Implementing the awakening and breathing coordination, delirium monitoring/management, and early exercise/mobility bundle into everyday care: opportunities, challenges, and lessons learned for implementing the ICU Pain, Agitation, and Delirium Guidelines. Crit Care Med. 2013;41(9 Suppl 1):S116–27. doi: 10.1097/CCM.0b013e3182a17064 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref008] 8.Alhaidari AA, Allen-Narker RA. An evolving approach to delirium: A mixed-methods process evaluation of a hospital-wide delirium program in New Zealand. Australas J Ageing. 2017;36(2):E20–E26. doi: 10.1111/ajag.12406 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref009] 9.Hosie A, Lobb E, Agar M, Davidson PM, Phillips J. Identifying the barriers and enablers to palliative care nurses’ recognition and assessment of delirium symptoms: a qualitative study. J Pain Symptom Manage. 2014;48(5):815–30. doi: 10.1016/j.jpainsymman.2014.01.008 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref010] 10.Vogels J, Demberg V, Kray J. The Index of Cognitive Activity as a Measure of Cognitive Processing Load in Dual Task Settings. Front Psychol. 2018;9:2276. doi: 10.3389/fpsyg.2018.02276 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref011] 11.Kirschner PA, Sweller J, Kirschner F, Zambrano RJ. From Cognitive Load Theory to Collaborative Cognitive Load Theory. Int J Comput Support Collab Learn. 2018;13(2):213–233. doi: 10.1007/s11412-018-9277-y [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref012] 12.Sweller J. Cognitive load during problem solving: Effects on learning. Cogn Sci. 1988;12:257–285. doi: 10.1016/0364-0213(88)90023-7 [DOI] [Google Scholar]

[pone.0293950.ref013] 13.Paas F, Renkl A, Sweller J. Cognitive Load Theory and Instructional Design: Recent Developments. Educ Psychol (Lond). 2003;38(1):1. doi: 10.1207/S15326985EP3801_1 [DOI] [Google Scholar]

[pone.0293950.ref014] 14.Paas FG, Van Merrienboer JJ, Adam JJ. Measurement of cognitive load in instructional research. Percept Mot Skills. 1994;79(1 Pt 2):419–30. doi: 10.2466/pms.1994.79.1.419 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref015] 15.Wilby KJ, Paravattil B. Cognitive load theory: Implications for assessment in pharmacy education. Res Social Adm Pharm. 2020; doi: 10.1016/j.sapharm.2020.12.009 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref016] 16.Blissett S, Rodriguez S, Qasim A, O’Sullivan P. Learning Echocardiography in the Workplace: A Cognitive Load Perspective. Acad Med. 2020; doi: 10.1097/ACM.0000000000003789 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref017] 17.Gomez TL, Henao-Castano AM, Troche-Gutierrez IY. Prevention and treatment of delirium in intensive care: Hermeneutics of experiences of the nursing team. Enferm Intensiva (Engl Ed). 2021; doi: 10.1016/j.enfi.2021.05.001 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref018] 18.Ceballos-Vasquez P, Rolo-Gonzalez G, Hernandez-Fernaud E, Diaz-Cabrera D, Paravic-Klijn T, Burgos-Moreno M. Psychosocial factors and mental work load: a reality perceived by nurses in intensive care units. Rev Lat Am Enfermagem. 2015;23(2):315–22. doi: 10.1590/0104-1169.0044.2557 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref019] 19.Yu C, Jiang J, Zhong M, Zhang H, Duan X. Training load of newly recruited nurses in Grade-A Tertiary Hospitals in Shanghai, China: a qualitative study. BMC Nurs. 2023;22(1):9. doi: 10.1186/s12912-022-01138-z [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref020] 20.Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. doi: 10.1038/s41746-020-0221-y [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref021] 21.Kindle RD, Badawi O, Celi LA, Sturland S. Intensive Care Unit Telemedicine in the Era of Big Data, Artificial Intelligence, and Computer Clinical Decision Support Systems. Crit Care Clin. 2019;35(3):483–495. doi: 10.1016/j.ccc.2019.02.005 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref022] 22.Dal Sasso GM, Barra DC. Cognitive Workload of Computerized Nursing Process in Intensive Care Units. Comput Inform Nurs. 2015;33(8):339–45; quiz E1. doi: 10.1097/CIN.0000000000000157 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref023] 23.Wu P, Nam MY, Choi J, Kirlik A, Sha L, Berlin RJ. Supporting Emergency Medical Care Teams with an Integrated Status Display Providing Real-Time Access to Medical Best Practices, Workflow Tracking, and Patient Data. J Med Syst. 2017;41(12):186. doi: 10.1007/s10916-017-0829-x [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref024] 24.Salvemini AV. Improving the human-computer interface: a human factors engineering approach. MD Comput. 1998;15(5):311–5. [PubMed] [Google Scholar]

[pone.0293950.ref025] 25.Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586. doi: 10.1136/bmj.e7586 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref026] 26.Fan H, Ji M, Huang J, Yue P, Yang X, Wang C, et al. Development and validation of a dynamic delirium prediction rule in patients admitted to the Intensive Care Units (DYNAMIC-ICU): A prospective cohort study. Int J Nurs Stud. 2019;93:64–73. doi: 10.1016/j.ijnurstu.2018.10.008 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref027] 27.Leppink J, Paas F, Van der Vleuten CP, Van Gog T, Van Merrienboer JJ. Development of an instrument for measuring different types of cognitive load. Behav Res Methods. 2013;45(4):1058–72. doi: 10.3758/s13428-013-0334-1 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref028] 28.Zhang S, Wu Y, Fu Z, Lu Y, Wang Q, Mingxuan L. Psychometric properties of the Chinese version of the instrument for measuring different types of cognitive load (MDT-CL). J Nurs Manag. 2020;28(2):277–285. doi: 10.1111/jonm.12919 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref029] 29.Shan Z, Yuan H, Ying WU. Development and psychometric testing of ICU Delirium Knowledge Questionnaire. Chinese Nursing Management. 2020;20(1):120–124. doi: 10.3969/j.issn.1672-1756.2020.01.025 [DOI] [Google Scholar]

[pone.0293950.ref030] 30.Trogrlic Z, van der Jagt M, Lingsma H, Gommers D, Ponssen HH, Schoonderbeek J, et al. Improved Guideline Adherence and Reduced Brain Dysfunction After a Multicenter Multifaceted Implementation of ICU Delirium Guidelines in 3,930 Patients. Crit Care Med. 2019;47(3):419–427. doi: 10.1097/CCM.0000000000003596 [DOI] [PubMed] [Google Scholar]

[pone.0293950.ref031] 31.Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, et al. Fall prevention in acute care hospitals: a randomized trial. JAMA. 2010;304(17):1912–8. doi: 10.1001/jama.2010.1567 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0293950.ref032] 32.Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1–55. [Google Scholar]

[pone.0293950.ref033] 33.Cohen J. Statistical power analysis for the behavioral sciences. J Am Stat Assoc. 1989;84:19–74. [Google Scholar]

[pone.0293950.ref034] 34.Wang PF, Shen LQ, Zhang HJ, Li BH, Ji H. A Nursing Pain Assessment and Record Information System: Design and Application in the Oncology Department. Comput Inform Nurs. 2017;35(12):647–652. doi: 10.1097/CIN.0000000000000373 [DOI] [PubMed] [Google Scholar]

PERMALINK

Impact of Clinical Decision Support System Assisted prevention and management for Delirium on guideline adherence and cognitive load among Intensive Care Unit nurses (CDSSD-ICU): Protocol of a multicentre, cluster randomized trial

Shan Zhang

Shu Ding

Wei Cui

Xiangyu Li

Jun Wei

Ying Wu

Roles

Abstract

Background

Methods

Discussion

Clinical trial registration

Introduction

Materials and methods

Study design

Fig 1. Schedule of enrolment, interventions, and assessments.

Study setting

Participants

Recruitment

Randomization and blinding

Intervention group

Fig 2. User interface of four modules of the CDSS-AntiDelirium.

Table 1. Safety and quality pathway for ICU nurses in implementing delirium intervention.

Control group

Outcome measures

Participant timeline

Data collection

Data management

Data analysis

Data monitoring

Harms

Patient and public involvement

Ethical considerations and dissemination

Ancillary and post-trial care

Validity and reliability/Rigour

Results and discussion

Conclusions

Supporting information

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Sascha Köpke

Roles

Author response to Decision Letter 0

Decision Letter 1

Sascha Köpke

Roles

Author response to Decision Letter 1

Decision Letter 2

Sascha Köpke

Roles

Author response to Decision Letter 2

Decision Letter 3

Sascha Köpke

Roles

Acceptance letter

Sascha Köpke

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases