NOTICE‐ED: Nurse or Technician Insights Into Cognitive Evaluations in the Emergency Department

ABSTRACT

Background

Several strategies have been proposed to increase chronic cognitive impairment (CI) screening in the emergency department (ED). Our goal was to assess the feasibility and acceptability of implementing specific CI screening tools and strategies in the ED from an ED registered nurse and technician perspective.

Methods

We performed a qualitative study using semi‐structured interviews with a purposive sample of ED nurses and ED technicians (EDTs). Participants worked at an urban academic hospital and were interviewed between November 2023 and March 2024. Interviews assessed participants' opinions on the feasibility and acceptability of CI screening and the use of machine learning (ML) tools to identify high‐risk patients for targeted CI screening, tablet‐based screenings, and two validated CI screenings: the Ottawa 3DY (O3DY) and Short Blessed Test (SBT). We used the Consolidated Framework for Implementation Research (CFIR) to develop our interview guide and performed a rapid analysis with deductive and inductive codes based on CFIR constructs.

Results

Four major themes related to CI screening tools arose: (1) Benefits of CI screening; (2) feasibility of integrating screening tools into existing workflows; (3) professional role limitations; and (4) implementation requirements. Participants perceived CI screening as important for allocating limited ED resources. Shorter, less specific testing, including the O3DY, was seen as feasible during triage, while longer, more specific screening, including the SBT, was seen as more feasible in roomed care areas. Both ED nurses and EDTs identified the need for electronic health record tools and dedicated screening teams to facilitate implementation.

Conclusion

ED nurses and EDTs support chronic CI screening if screening techniques and clinical teams can be optimized to make workflows feasible.

Nurses and ED technicians proposed steps for the screening process that incorporated machine‐learning‐based screening and two stages of CI screening, requiring EHR support and team‐based care. CI, cognitive impairment; ED, emergency department; O3DY, Ottawa 3DY.

Summary.

• Key points

  • ○
    Emergency department (ED) nurses and technicians viewed chronic cognitive impairment (CI) screening as useful for distributing limited ED resources.
  • ○
    Risk‐stratifying patients for CI screening through machine learning algorithms may be helpful if such tools prompt next steps in the electronic health record.
  • ○
    Two‐stage chronic CI screening may increase screening feasibility in the ED.
• Why does this paper matter?

  • ○
    The under‐recognition of chronic cognitive impairment (CI) in older adults visiting the emergency department (ED) can lead to unsafe discharge planning and other poor patient outcomes. This study can guide the implementation of CI screening in a way that limits the burden placed on ED clinicians through novel screening strategies, such as machine learning‐based tools.

1. Introduction

In the United States, a growing number of older adults (those ages 65 years and older) receive care in emergency departments (EDs) [1]. Over one in four older adults who visit EDs will have some form of cognitive impairment (CI), including delirium, mild cognitive impairment (MCI), and dementia [2]. Identification of CI in the ED can prompt testing for reversible causes of CI [3, 4], improve care through specific interventions for patients with CI [5], and increase referrals to primary care and memory centers for definitive diagnosis and evidence‐based treatment [6]. Yet, emergency clinicians miss up to 80% of CI diagnoses [7, 8]. Undetected delirium is associated with increased mortality within 6 months following an ED visit [9], and older adults with acute and/or chronic CI are at high risk of ED revisits, hospital readmissions, and functional decline [10, 11]. Consequently, the Geriatric ED Guidelines recommend CI screening for all older ED patients, including a two‐step delirium screening, followed by an ED‐validated dementia/MCI screening [12].

Despite these recommendations, several barriers have limited the adoption of CI screening across EDs [13, 14]. For chronic CI screening, one large barrier to adoption is the variety of existing screening tools. Dozens of validated chronic CI screening tools exist [15, 16], but there is no consensus on which tools are most useful to ED clinicians [12, 17]. For example, the Short Blessed Test (SBT) and the Ottawa 3DY (O3DY) have similar sensitivity for detecting CI in the ED, but the SBT is a longer, more specific test [15]. To date, there has been little exploration of how differences in length and test performance characteristics might influence clinician preferences for and adherence to CI screening and intervention. In addition, emergency nurses and physicians have cited high patient volumes, time constraints, unclear follow‐up steps, and existing screening responsibilities as barriers to implementation [5, 13, 18, 19, 20].

As a result, new strategies have been proposed to make CI screening more feasible in the ED. These include decreasing the screening burden by involving ED technicians (EDTs) in the screening process [21, 22], using machine learning (ML) based on electronic health record (EHR) data to identify high‐risk patients for targeted CI screening [23], and using tablet computer applications for patient‐administered screenings [24, 25].

There has been limited investigation of ED clinician perspectives on the feasibility of incorporating these new solutions into ED workflows. Emergency nurses have extensive clinical training, see older adults routinely in the ED, and already lead numerous screenings, including suicide and fall risk assessments [26]. EDTs work with nurses and often monitor patients with behavioral concerns as one‐to‐one sitters [27]. Despite having been proposed as implementers of CI screenings, the EDT perspective is not yet present in the literature [22]. Overall, EDTs and ED nurses can provide insights into the challenges and pragmatic details necessary to implement CI screenings in the ED.

This study sought to assess the feasibility and acceptability of implementing specific chronic CI screening tools and strategies in the ED from a nursing and EDT perspective, before health system adoption. This included exploring and comparing the utility of the SBT and O3DY. We also addressed novel but existing and validated technological solutions in two areas: machine‐learning‐based tools to predict CI risk and tablet‐based screenings.

2. Methods

2.1. Study Design

This was a qualitative study using a case‐study based approach [28] and content analysis [29]. We performed individual semi‐structured interviews with ED nurses and EDTs to elicit opinions on the above CI screening tools and strategies, and we followed the Consolidated Criteria for Reporting Qualitative Research (COREQ) in reporting this study (see Supplementary Appendix S1). The University of Pennsylvania Institutional Review Board provided ethical approval for this study (Protocol 854674).

2.2. Study Setting

We interviewed ED nurses and EDTs employed at an urban teaching hospital that has level 1 trauma services and sees approximately 50,000 ED visits per year. Approximately one in four ED visits is among older adults, defined as adults ages 65 years and older. Patients are currently screened in ED triage for fall risk, elder abuse, sepsis, and suicide screenings. The ED does not have designated spaces for older adults or geriatric ED accreditation and does not screen patients for CI.

2.3. Participant Recruitment

Eligibility included nurses and EDTs who spoke English fluently and who were ages 18 years and older. Additionally, we required participants to have completed a minimum of five triage shifts between November 2022 and November 2023 and therefore had experience with patient screening processes. Using the software R, version 4.3.2 (R Core Team, Vienna, Austria), we randomly selected 75 nurses and EDTs to sequentially invite by email and in person to participate in the study.

2.4. Conceptual Framework

To create the interview guide (Supplementary Appendix S2), a medical student (SJN), a nurse (KJM), two ED physicians (ANC, ABF), and an anthropology student (ARB) performed a literature review of CI screening implementation [18, 22, 30] and developed questions based on the Consolidated Framework for Implementation Research (CFIR) [31]. CFIR consists of constructs to evaluate barriers and facilitators to implementing evidence‐based practices. We focused on the following CFIR constructs to develop questions, which were most relevant to our research goals, including participant perspectives on: (1) Relative Priority (importance of recognizing CI compared to other screenings and tasks), (2) Relative Advantage (effectiveness of CI screening vs. current practice), (3) Compatibility (fit of CI screening within ED workflows), (4) Available Resources (resources exist to carry out CI screening), and (5) Relational Connections (teams and networks to facilitate CI screening).

Interviews with nurses focused on the above constructs in relation to the SBT, O3DY, and ML tools to identify at‐risk patients, and self‐administered tablet screenings (without specifying one tablet tool, as we hoped to elicit broad thoughts on the feasibility of using this technology in the ED). We provided descriptions of each tool and showed participants the SBT and O3DY. Interviews with EDTs focused on CI screening, in general, and the perceived feasibility of EDTs contributing to a screening process. We pilot‐tested the interview guide with two nurses and two EDTs, refining our questions based on their interviews and feedback.

2.5. Data Collection

SJN obtained informed consent from participants, then carried out the interviews via phone or videoconference. Interviews took place from November 2023 to March 2024 and lasted between 35 and 70 min. The audio‐recorded interviews were transcribed verbatim, then de‐identified and verified for accuracy. Participants self‐reported their demographic information after their interviews and received $30 gift cards. We interviewed participants about each screening tool until thematic saturation had been reached (i.e., no new themes emerged) [32], as determined by consensus discussion among research team members.

2.6. Data Analysis

We performed rapid analysis of interview transcripts [33] using a deductive approach to coding based on the pre‐selected CFIR constructs listed above, as well as an inductive approach based on common topics and other CFIR constructs that arose during interviews, such as Communications (a CFIR construct) and the role of caregivers. To determine additional topics included in the analysis, SJN, ARB, ANC, KJM, and ABF read the first two transcripts together and discussed emerging themes beyond the pre‐selected CFIR constructs. SJN and ARB then read all transcripts independently, finalized the codes collaboratively, and created a matrix in Microsoft Excel that included all deductive and inductive codes; see Table 1 for code definitions.

TABLE 1.

Code definitions and mapping final study themes.

Code	Definition	Final theme
Relative advantage—current ED practices versus CI screening	Comparing the effectiveness of CI screening to current ED practices	Benefits of cognitive impairment screening
Relative priority	Importance of recognizing CI compared to current screenings and tasks
Tension for change	Need to change current ED processes
Caregiver involvement	Mention of caregivers impacting CI recognition, including mention of CI screening prompting caregiver involvement
Compatibility	Fit or lack of fit of CI screening within current ED processes, including location, timing, and stage of ED visit	Feasibility of integrating screening tools into existing workflows
Relative advantage—SBT versus O3DY	Comparing the feasibility of SBT screening to that of O3DY screening
Relational connections	Mention of who could implement CI screening, how different clinicians could or could not contribute	Professional role limitations
Available resources	Existing or needed resources to carry out CI screening	Implementation requirements
Communications	Existing or needed information sharing practices for CI screening

Note: Final themes on the right were established from the corresponding codes to the left. Code definitions were developed from: Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32‐item checklist for interviews and focus groups. International Journal for Quality in Health Care. 2007. Volume 19, Number 6: pp. 349–357.

SJN read through interview transcripts and summarized participants' responses for each code in Excel. ARB then reviewed the transcripts, verifying and adding details to the matrix. The team met to resolve any differences in analysis and to discuss key patterns in participant responses. To develop final themes, the research team iteratively discussed, grouped, and named response patterns. For example, under our code “Relative Priority,” response patterns that arose included the importance of recognizing CI to determine ED bed allocation and identify the need for social work involvement. We then grouped these responses under the larger theme, “Benefits of cognitive impairment screening” (see Table 1 for mapping of themes with codes).

3. Results

3.1. Participant Characteristics

Of 50 clinicians approached, 13 nurses and 7 EDTs participated in interviews. Demographic characteristics are reported in Table 2.

TABLE 2.

Participant characteristics.

Characteristic	N (%)
Age, mean years (SD)	34 (8)
Gender
Woman	15 (75)
Man	5 (25)
Race
White	12 (60)
Black	5 (25)
Asian	2 (10)
Multi‐racial	1 (5)
Ethnicity
Hispanic or Latino	1 (5)
Professional role
Nurse	13 (65)
Technician	7 (35)
Years working in an ED (range)
Nurses	2–30
Technicians	1–10
Number of triage shifts worked over 1 year, mean (SD)	24 (9)

Abbreviations: ED, emergency department; SD, standard deviation.

3.2. Overview of Themes

We identified four themes related to implementing CI screening: (1) Benefits of cognitive impairment screening, (2) Feasibility of integrating screening tools into existing workflows, (3) Professional role limitations, and (4) Implementation requirements. We summarize our themes, subthemes, and illustrative quotes in Supplementary Appendix S3. We note below where responses differed between nurses and EDTs.

3.3. Benefits of Cognitive Impairment Screening

3.3.1. Improving Diagnosis

Both ED nurses and EDTs explained that patients with CI can often converse well over short periods of time, such as a triage encounter, and therefore clinicians might not recognize cognitive deficits early on in an ED visit. Eighteen participants (90%) therefore thought that a formal screening tool would be useful, but the rest felt that they already easily recognized CI and would not benefit from an additional screening tool. Participants noted that recognizing CI in triage can prompt clinicians to seek collateral information from caregivers, as patients with CI might not be able to provide accurate histories. They described that without recognizing the need for collateral information, clinicians may miss critical diagnoses and fail to distinguish between acute and chronic CI.

3.3.2. Resource Allocation

Participants also explained that recognizing CI in triage was essential for determining how to allocate limited ED resources, including ED rooms. All participants felt that acute CI should factor into patients' acuity levels and lead to priority in rooming patients, rather than assigning them to common care spaces. Some participants felt that chronic CI should lead to patient placement closest to nurses' stations for easier monitoring for falls, agitation, and wandering. Participants also felt that identifying acute delirium and baseline CI would help the ED team recognize the need for involving sitters to prevent unsafe behaviors, such as pulling out intravenous lines, and social work to ensure safe discharge planning and the provision of resources for memory care.

3.4. Feasibility of Integrating Screening Tools Into Existing Workflows

3.4.1. Ottawa 3DY

When considering formal screening tools, participating nurses expressed that the O3DY was similar to questions they currently ask informally in triage and would fit well into their triage workflows. Their concerns about the O3DY were about: (1) how to administer the spelling portion of the screening for patients with limited English proficiency or illiteracy, and (2) that anyone, including those without CI, might forget the day or date when waiting in the ED, leading to a high number of false‐positive screenings, unnecessary testing, and wasted resources. As a result, participants thought that the brevity of the O3DY was preferable for triage, but a positive O3DY screening should be followed by a longer, more specific test in roomed care areas.

3.4.2. Short Blessed Test

Participants thought the SBT would fit best during roomed care in the ED, when clinicians follow fewer patients than during triage care. Compared to the O3DY, participants appreciated the level of detail and the weighted scoring system the SBT provided. In addition to an initial assessment in the ED, they felt that the SBT could also be helpful in reevaluating mental status before and after admission. Due to the length of the SBT, however, nurses felt they would not be able to memorize the test, which might slow down its administration due to the need for locating a computer or a printout of the screening.

3.4.3. Tablet‐Based Screenings

The additional equipment and set‐up required for tablet‐based screenings made their use in triage seem less feasible to participants. Participants expressed that many older adults struggle to use new technology or have decreased vision, so would likely require assistance and more clinician time, despite the tests being self‐administered. Furthermore, they identified the risk of theft of tablets as a barrier to implementation. If used, participants felt this technology would be best performed in ED rooms or in the waiting room with staff to assist in screenings.

3.4.4. Machine Learning to Identify At‐Risk Patients

Given ED time constraints, participants felt that having a built‐in EHR tool to identify at‐risk patients for CI screening in triage would be helpful. They noted that alerts, such as fall and sepsis risk flags, already exist in the EHR. Participants also identified potential limitations of ML tools. They noted that new patients do not always have information present in their electronic charts, so a program based on EHR data could miss these patients. Participants also described that some patients may not neatly fit into an algorithm, resulting in over or under screening in some cases. Nine nurses (69.2%) felt that even without a risk notification, they should still screen patients who seem to have CI, based on clinical intuition.

3.5. Professional Role Limitations

While nurses felt that CI screening was within their scope of practice, their high workloads limited the feasibility of adding the longer CI screenings to their responsibilities. EDTs also expressed having limited time to perform these screenings. Participating EDTs felt they had the clinical experience necessary to perform CI screening, but did not think all their colleagues would feel comfortable administering these screenings due to the range of professional backgrounds that exist in the EDT role. Both nurses and EDTs suggested having students, volunteers, or geriatric specialists perform more detailed CI screenings (i.e., the SBT) in roomed care areas.

3.6. Implementation Requirements

To facilitate the implementation of any possible screening tools, participants outlined various necessary conditions. They expressed the need for training on administering the screenings, which should include an explanation of how CI screening might improve patient care. For the O3DY and SBT, participants also emphasized the importance of having the screening questions built into the EHR, as well as reminders to perform the screenings. For the tablet tools, participants stated that the results should automatically populate in patients' charts.

Participants also stated that there should be an order set or clear pathway to guide next steps if the ML tool identifies someone as at‐risk for CI and if someone screens positive for CI. Participants' suggested steps are outlined in Figure 1 and included: (1) the triage nurse identifies the CI risk flag in a patient's EHR, (2) performs the O3DY, and (3) informs the charge nurse and covering provider of a positive screen. Next, (4) the patient is prioritized for an ED room with acute CI or an easily observable ED bed for chronic CI, (5) a designated team or clinician performs longer, more specific CI testing (SBT), (6) social work is automatically consulted to assist with discharge planning, and (7) the ED team assesses the need for further clinical services.

FIGURE 1.

Potential screening process. Nurses and ED technicians proposed steps for the screening process that incorporated machine‐learning‐based screening and two stages of CI screening, requiring EHR support and team‐based care. CI, cognitive impairment; ED, Emergency Department; O3DY, Ottawa 3DY.

4. Discussion

This study provides insight into how ED nurses and EDTs perceived the importance of CI screening and into applications of technology to increase the feasibility and acceptability of screening for CI. We found that patient volume and time constraints limited the feasibility of universal CI screening for older patients. However, nurses and EDTs identified early recognition of CI as important for improving patient safety and resource allocation. Nurses endorsed the utility of technology to reduce the number of patients screened and to facilitate screening and subsequent steps. Participants highlighted further screening considerations to limit the burden placed on ED clinicians with already high workloads.

First, our participants highlight the possible benefits of adding an initial, automated risk‐stratification step to reduce the number of patients screened manually for chronic CI. Recently, there has been increased development of ML tools to determine CI risk [23, 34, 35] but the optimal implementation of these tools in an acute care space has not been explored. Our participants described that EHR‐based risk identification should prompt clinical action. Alert fatigue, which describes insufficient responses to care alerts due to the frequency of notifications, is a known barrier to acting upon EHR‐embedded risk notifications [35, 36]. To ensure that a positive ML risk assessment results in subsequent screening, our participants suggested a mandatory EHR‐based CI screening (“hard stop”). However, hard stops may have unintended consequences, such as delaying care, so options to opt out of screening under high‐acuity circumstances would be necessary [37, 38].

Participants also highlighted concerns that an ML tool would miss patients with minimal EHR data and those with characteristics not captured within the original algorithm. Some ML tools based on EHR data have relied on incomplete information [39], perpetuated bias [40], and underrepresented older adults [35, 41]. Future research should evaluate strategies to mitigate bias in identifying CI risk based on EHR data, such as ensuring training on balanced and representative datasets [42]. Our participants also suggest that there should be protocols on screening patients who show clinical signs of CI, but whom an ML tool may fail to identify. Screening for CI in adults younger than 65 who have cognitive symptoms has also occurred in primary care settings [43], and expanding ML‐based CI screening to this younger population warrants exploration.

Following ML‐based risk identification, participants propose CI screening in two stages: an initial O3DY screen in triage, followed by more specific testing in roomed care areas. In primary care settings, others have investigated two‐stage chronic CI screening, including validating the Dual‐Stage Cognitive Assessment [44]. A two‐step process for dementia and delirium screening is also generally recommended in Australian healthcare organizations, consisting of the 4‐A's Test for CI and then a Confusion Assessment Method for delirium or Abbreviated Mental Test score for CI [45]. Overall, ED‐based two‐stage screenings remain poorly explored and may reduce the screening burden for ED clinicians.

Regardless of the screening process, the lack of clear next steps after a positive screen has limited the impact of a positive screen on care [42, 46]. Participants recommended an ED track board icon in the EHR, automatic messaging and in‐person communication with the covering provider, and an automated consult to the clinician or team who will perform more specific testing during the ED visit. These results add to findings from a prior study, which also proposed developing order sets for patients with CI [47]. Ultimately, implementing automated communication, consults, and orders would require significant buy‐in from hospital leadership and informatics.

In addition, both ED nurses and technicians suggested having a dedicated team to perform more specific CI screenings in roomed‐care areas. Our results augment studies demonstrating that time constraints in the ED limit the feasibility of CI screening for nurses and physicians [17, 24]. We add perspectives from EDTs, who expressed that their competing responsibilities and variations in training would also make widespread implementation by EDTs challenging. Yet, the enthusiasm of some EDTs for screening suggests that motivated EDTs could help form a specialized CI screening team. For example, the Geriatric Emergency Room Innovations for Veterans program trained a group of care technicians to focus on administering geriatric screenings in the ED [22]. Trained medical students have also previously performed geriatric syndrome screenings in the ED and could serve as other screening team members [21]. Future studies could explore the impact of these roles on patient safety and healthcare costs to motivate establishing new care teams and increased staffing.

Another finding was that tablet‐based screening in the ED is infeasible. This aligns with a previous study showing that less than half of sampled older adults in the ED were willing to answer clinical questions on a tablet [24]. While the assumption that older adults cannot interact with technology is rooted in ageism, involving older adults in the design and implementation of tablet‐based tools may improve patient buy‐in [47]. Time and assistance needed to complete web‐based questionnaires has also been correlated with cognitive impairment risk, serving as a potentially useful measure [48]. As an alternative, ML algorithms trained on voice recordings have also detected cognitive impairment and could serve as a lower‐barrier screening tool in the ED [49]. For any screening equipment, participants emphasized the importance of secure storage, durability, user friendliness, and adaptation to the ED.

Lastly, participants often discussed acute and chronic CI as independent processes, which may have influenced their views on the utility of chronic CI screening. There is strong evidence that chronic CI increases the risk of acute CI and vice versa [50]. In care settings outside of the ED, recognizing chronic CI has prompted delirium prevention protocols, and this practice could be applied to acute care settings [51]. Our participants proposed training clinicians on the administration and importance of chronic CI screening, which could also include education on the relationship between acute and chronic CI. Overall, communicating the purpose of care processes, such as prompting preventative steps, can increase efficiency and work satisfaction in the ED [52].

5. Limitations

This study has various limitations. First, we interviewed ED clinicians from a single urban academic hospital, so their perspectives may not apply to other practice settings. Most participants were women and white, and their experiences might differ from those of clinicians from other demographic groups. Perspectives on screenings are also limited without experience using the specific tools in an ED setting. Though we sought initial and general opinions on the feasibility of tablet‐based screenings, participant perspectives may have also shifted had they explored specific applications.

6. Conclusions

Integrating an ML‐based risk stratification tool, two‐stage chronic CI screening, and automated communication of screening results into the EHR could facilitate chronic CI screening in the ED. Dedicated screening teams who receive training on the importance of chronic CI recognition could also improve the wellbeing of ED patients and clinicians. Future studies should explore post‐implementation outcomes and perspectives on participants' proposed screening solutions.

Author Contributions

Sarah J. Nessen: concept and design, acquisition of subjects and data, analysis and interpretation of data, preparation of manuscript. Anita N. Chary: concept and design, analysis and interpretation of data, preparation of manuscript. Annika R. Bhananker: analysis and interpretation of data, preparation of manuscript. K. Jane Muir: concept and design, analysis and interpretation of data, preparation of manuscript. Kyra O'Brien: preparation of manuscript. Ari B. Friedman: concept and design, analysis and interpretation of data, preparation of manuscript.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Supplementary Appendix S1. Supporting Information.

Nessen S. J., Chary A. N., Bhananker A. R., et al., “ NOTICE‐ED: Nurse or Technician Insights Into Cognitive Evaluations in the Emergency Department,” Journal of the American Geriatrics Society 73, no. 8 (2025): 2503–2511, 10.1111/jgs.19578.