Assessing Delirium Severity in Dementia: Item Development and Psychometric Validation for a Delirium Severity Instrument

Abstract

PURPOSE:

Despite the importance of measuring delirium severity in patients with Alzheimer’s disease and related dementias (ADRD), no validated instrument currently exists. Our goal was to generate items for use in such an instrument.

METHOD:

An interdisciplinary expert panel developed items to assess seven domains and 21 subdomains of delirium severity. Nursing experts provided input on items.

RESULTS:

Experts achieved consensus on 21 items, including best respondents for each item, and appropriate look-back periods. Nursing experts emphasized the need for educating nursing staff and obtaining information from caregivers.

CONCLUSION:

Careful, nuanced identification of distinguishing features is key for generating items for measuring delirium severity in persons with ADRD. Once developed, engaging nurses will be essential to facilitate adoption and relevance of the tool.

Delirium and dementia are common, distressing causes of cognitive impairment worldwide, which frequently occur together. Delirium is an acute disturbance in attention and awareness that represents a change from baseline cognitive function (Oh et al, 2017) and is a serious clinical problem for older adults that has been associated with increased health care costs and poor clinical outcomes (Gou et al, 2021; Oh et al, 2017). Alzheimer’s disease and related dementias (ADRD) is a progressive loss of cognitive skills and refers to various related types of progressive cognitive decline (Ajrouch et al, 2023). Delirium and ADRD are highly interrelated, with each representing an important risk factor for the other (Fick et al, 2005). ADRD has been identified as the leading risk factor for development of delirium (Fick, Kolanowski, et al, 2013; Fick, Steis, et al, 2013), whereas delirium has been well documented to accelerate the trajectory of cognitive decline in ADRD (Fong & Inouye, 2022). Individuals with ADRD often experience poorer outcomes associated with delirium, including longer lengths of hospitalization, higher risk of institutionalization and mortality, and greater cognitive and functional decline (Han et al, 2022; Hshieh et al, 2020). Moreover, delirium in persons with ADRD remains frequently unrecognized and undiagnosed (Fong et al, 2012; Fong et al, 2009; Morandi et al, 2014; Morandi et al, 2015).

Despite its frequency and clinical importance, diagnosing delirium and rating delirium severity in the presence of ADRD is highly challenging. Delirium and dementia have overlapping symptoms, and the presence of ADRD can result in overestimation or underestimation of delirium severity (Fick et al, 2005). Delirium severity has been recognized as the preferred outcome measure for evaluating effectiveness of delirium treatments, for correlating with clinical outcomes in a dose-response fashion, and for pathophysiological and biomarker studies, all of which benefit from a more graded rather than binary outcome.

A systematic review of 228 articles, including 42 delirium severity instruments (Jones et al, 2019), found that pre-existing cognitive impairment and/or ADRD confounded measurement of delirium and its severity (Marcantonio et al, 2014). Notably, none of the 42 instruments in the prior study were specifically developed or validated for use in persons with ADRD (Vasunilashorn et al, 2016). We designed our work to help address this gap.

The current study builds on previous work conducted as part of the National Institutes of Health (NIH)– funded Better Assessment of Illness (BASIL) study (Hshieh et al, 2019), where we used expert panel input and psychometric data analysis to develop a new delirium severity instrument (DEL-S). We subsequently validated the DEL-S against reference standard ratings, then streamlined the instrument for efficiency while maintaining high content validity and reliability across a broad range of delirium symptom domains (Vasunilashorn et al, 2020).

As a next step, we wanted to develop a delirium severity measure for use among individuals with mild to moderate ADRD. Because of the challenges and complexity of identifying delirium in persons with ADRD, we realized that simply applying the DEL-S in persons with ADRD would not be optimal. Rather, we needed to carefully re-evaluate the appropriate domains and subdomains, and carefully consider the items to be included. Thus, we applied the thoughtful and rigorous approach for instrument development established by the Patient Reported Outcomes Measurement Information System initiative (Figure 1), which fulfills the US. Food and Drug Administration guidance for outcome measures to be used in clinical trials (Rothrock et al, 2011; Turner et al, 2007). In a prior study, we worked with an interdisciplinary international expert panel on Construct Definition, that is, to defi ne the domains and subdomains relevant to delirium superimposed on ADRD (Schulman-Green et al, 2022). Following a modified Delphi process, the expert panel reached consensus that the DEL-S content domains of Cognition, Level of Consciousness, Inattention, Psychiatric-Behavioral, Emotional Dysregulation, Psychomotor Features, and Functional Domains remain important in rating delirium severity in ADRD (Schulman-Green et al, 2022). However, the subdomains of Cognitive Impairment, Anxiety, Fear, and Depression were considered less useful for rating delirium severity in the context of ADRD because these symptoms may occur in both conditions and their relative attribution to each is challenging in the clinical setting.

Figure 1. Instrument for delirium severity in Alzheimer’s Disease and Related Disorders (ADRD) – developmental approach.

In the development approach of the delirium severity in ADRD instrument, this manuscript reflects the Item Generation step in this process (shaded).

The current study describes the next step in our instrument development, Item Generation, where we develop the specific items to represent the domains and subdomains identified as part of the Construct Definition step above. Our first aim was to generate the items, which is an involved, multi-step process that includes refining the specific wording to describe each content domain, identifying the respondent or rater for each item, and specifying the reference time-frame (ie, look-back period) for the rating (e.g., change from 1 to 2 months ago, or acute change over the past day). Because item generation involves intensive work over a prolonged period, we opted to use a local interdisciplinary expert panel to develop the set of items to capture the instrument domains. Our second aim, after the items were generated, was to seek input from a nursing expert panel on the face and content validity of the items developed, their feasibility of administration, and importantly, their real-world applicability in clinical practice. Input at this stage was considered important as nurses are likely to be involved in using the instrument.

Rigorous item generation is rarely described for new clinical measures. Given the importance and novelty of this measure for delirium severity in persons with ADRD, we hoped that this process would help build an optimal instrument for fi eld-testing.

METHOD

Item Generation

Overview. Item generation was conducted by a local interdisciplinary expert panel assembled and overseen by the BASIL study principal investigators (PIs; SKI, RNJ). Th e local expert panel comprised 10 clinicians drawn from the Greater Boston area, each with clinical and/or research expertise in delirium, dementia, and/or delirium in ADRD, including professionals in the fields of geriatrics, general medicine, neurology, psychiatry, neuropsychology, nursing, social work, epidemiology, and psychometrics (Table A). All expert panel procedures followed recommended practices for achieving consensus through a modified Delphi process (Hshieh et al, 2019; Vasunilashorn et al, 2020). Six expert panel meetings (60 to 90 minutes each) were held between January 8 and February 12, 2021, with additional working sub-group meetings and homework assignments completed in between. The expert panel was advised that they would be developing items for an instrument that would be used with patients with a range of cognitive abilities as defined by the Clinical Dementia Rating Scale (CDR), from intact (CDR = 0) to very mild (CDR = 05), mild (CDR = 1), and moderate (CDR = 2) ADRD (Morris et al, 1997).

Table A.

Description of BASIL Local Expert Panelists (n=10)

Characteristic	Mean (range)
Years in clinical practice	19.4 (0–45)
Years caring for patients with delirium	19.4 (0–45)
Years caring for patients with dementia	19.4 (0–45)
Years conducting research with patients with delirium	17.6 (4–35)
Years conducting research with patients with dementia	20.5 (2–35)
Years in the field of delirium superimposed on dementia (clinical or research)	15.6 (1–35)
Clinical work setting †	n
Hospital	6
Nursing Home/Home-based care/Other	3
Not applicable	2
Frequency of contact with patients with delirium
Frequently	7
Seldom/None	3
Frequency of contact with patients with dementia †
Frequently	-
Seldom/None	-
Ever a family caregiver to a patient with delirium
Yes	6
No	4
Ever a family caregiver to a patient with dementia
Yes	6
No	4
Race/Ethnicity
Non-Hispanic White	6
None-white or Hispanic	8
Gender
Female	6
Male	4
Profession †
Geriatrics	3
Measurement	2
Neuropsychology	1
Neurology	1
Nursing	1
Psychiatry	1
Social Work	1

Developing Item Descriptions. Th e PIs assigned the 10 expert panel members into three working groups covering two content domains each: (1) Cognitive Impairment and Disorganized Thinking; (2) Emotional Dysregulation and Psychomotor Features; and (3) Functional and Sleep Subdomains (Table B). Each working group met independently to generate item descriptions for their assigned content domains. In an iterative multi-step process, each working group presented its recommended wording to the full expert panel. Revisions and edits were suggested and incorporated across multiple rounds until consensus was achieved for all items in the six subdomains. Redundancies were identified and eliminated. This process was repeated in a sequential fashion until consensus was achieved with each subdomain. For these procedures, consensus was defined as complete agreement (100%) by all expert panel members on the fi nal decisions.

Table B.

Working Groups for Delirium Severity in ADRD Item Development by Subdomains

Group 1	Group 2	Group 3
Cognitive impairment and disorganized thinking	Emotional Dysregulation and Psychomotor Features	Functional and Sleep Items
Level of consciousness	Inappropriate behavior	Insomnia
Inattention	Apathy	Hypersomnolence
Disorientation	Withdrawal	Sleep-wake cycle reversal
Disorganized thinking	Anger/hostility/irritability	Poor postural control
Cognitive Impairment	Psychomotor agitation	Poor oral intake
Delusions	Psychomotor retardation	Decline in self-care
Hallucinations		Transferring
		Gait/Walking
		Incontinence

Refining Items, Identifying Respondents, and Look-Back Periods. Once a consensus description of each subdomain item was developed, the expert panel added examples of the symptom or behavior to help guide clinicians in scoring the items and train research staff. Next, the expert panel selected appropriate respondents/raters for each item (ie, asking the patient directly, asking the nurse or family caregiver, or making ratings based on observing the patient). Th e respondent or rater could vary according to clinical setting. Finally, the expert panel reached consensus on the look-back period for each item to best capture acute change from baseline. Given that the goal was to rate delirium severity in persons with ADRD, three look-back periods were identified by the expert panel to be most relevant: change from baseline (with baseline being weeks to months before); acute changes over the past 24 hours; and immediate observations made at the bedside.

Face and Content Validity, Feasibility, and Applicability

Nursing perspective was considered critical to ensure that the delirium severity measure would be clinically relevant and well-grounded in the reality and experiences of clinicians who would ultimately use the instrument in practice (Mallinckrodt & Tekie, 2016). Thus, the nurse expert panel reviewed the fi nal items for face and content validity, feasibility of administration, and real-world applicability. The nurse expert panel, drawn from the Northeast (New York and Boston) region, comprised nurses from various clinical work settings, including hospital, nursing home, and home- and community-based settings (Table C). We purposefully sampled nurse experts who worked with patients with delirium and dementia who were diverse in terms of race/ethnicity, socioeconomic status, and other characteristics. Th e majority of nurse experts had significant clinical and personal caregiving experience with delirium or dementia.

Table C.

Delirium Severity in ADRD Nurse Expert Panel: Participant Demographic and Clinical Information (N=10)

Characteristic	Mean (range)
Years of nursing experience	15 (0–37)
Years caring for patients with delirium	11.1 (0–37)
Years conducting research on patients with delirium	1.6 (0–13)
Years cared for patients with dementia	12 (0–30)
Years conducting research on patients with dementia	5.5 (0–23)
Clinical work setting †	n
Hospital	6
Nursing Home/Home-based care/Other	6
N/A	4
Frequency of contact with patients with delirium
Frequently	5
Seldom/None	5
Been a family caregiver to a patient with delirium
Yes	6
No	4
If yes, # years	1.75 (range 0.5–5)
Frequency of contact with patients with dementia
Frequently	6
Seldom/None	4
Been a family caregiver to a patient with dementia
Yes	4
No	6
If yes, # years	4.25 (range 1–6)
Race/Ethnicity
Non-Hispanic white	5
Non-white or Hispanic	6
Gender
Female	8
Male	2
Highest nursing degree †
MSN	1
PhD	7
DNP	2

Upon electronic consent, we emailed a copy of the potential delirium severity items and a review form to the nurse expert panel members. This form was completed as advance “homework” by panel members. Subsequently, a 90-minute, on-line, video-recorded meeting with the nurse expert panel was held on March 3, 2021, moderated by a study co-investigator with extensive focus group experience (DS-G). Th e goal of the meeting was to review and suggest edits to the potential delirium severity items to optimize face and content validity, as well as feasibility, ease of use, and clinical applicability across settings. Using the review form and a concurrent discussion guide during the session, we asked nurse experts to provide input on the scoring instructions, item content/wording, response options, respondent/rater options, appropriateness for persons with ADRD, and general feedback. Nursing experts were helpful in identifying the most important items and approaches to identify delirium in persons with ADRD.

Following the session, our focus group expert (DS-G) compiled data from nurse experts’ review forms, meeting notes, and the video recording into the categories of instructions, item content, item wording, response options, examples, and appropriateness of use for persons with ADRD. We added overarching discussion points from the session to create a report of comments and suggested edits for the delirium severity measure.

The PIs (SKI, RNJ) reviewed the report of nurse expert panel comments and recommendations, incorporated edits to refine delirium severity items, and presented the most substantial issues to the local expert panel for discussion and consensus.

RESULTS

Item Generation

In extensive discussion, local experts acknowledged that the main challenges for the new delirium severity measure in patients with ADRD are in identifying key delirium symptom domains and quantifying the acuity of change in these symptoms so that the multiple facets of delirium severity can be captured accurately (Schulman-Green et al, 2022). Addressing these challenges required developing careful descriptions of each item, along with choosing the best respondent/rater and look-back period to capture acute change superimposed on chronic cognitive impairment.

Table D provides details of the consensus process of the local expert panel on the final items selected for inclusion. Initial work focused on achieving consensus on the specific wording of the item descriptions. Through multiple iterations, local experts developed a consensus description for each subdomain item and included examples of each symptom or behavior to facilitate actual patient assessments and research staff training. Key discussion points are also captured in Table D. For example, for disorganized thinking, the expert panel emphasized the need to be able to objectively document a change from baseline, as cognition is likely to be impaired at baseline in patients with ADRD. Thus, the expert panel recommended that brief cognitive items be included in the assessment, such as orientation and attention questions (i.e., digit span test), which could be compared to baseline ratings and monitored over time.

Table D.

Local Expert Panel Discussion of Final Delirium Severity in ADRD Items

Items (Domain)	Description	Example	Discussion and Notes from Expert Panel
Inattention (INA)	Reduced ability to maintain attention to external stimuli and shift attention to new external stimuli	Questions must be frequently repeated because attention wanders	• Multiple raters to determine who are the best raters for this item • Some overlap with other items capturing inattention (e.g, easily distracted, changes in thinking or concentration, etc.)
Easily distracted (INA)	More easily distracted by external stimuli than would be considered normal or expected	Patient cannot focus on any one person or task; attention is darting around the room to every noise or stimulus	• Rate based on formal and informal assessment. Formal includes brief cognitive screening; informal includes observer ratings at bedside
Disorientation (COG)	Impaired ability to locate oneself in one’s environment, in reference to time, place or person.	Patient thinks it is night, during the day	• Often present with AD/ADRD. Included because reflects delirium severity well if increased or new disorientation present
Disorganized thinking (COG)	Disorganized thinking, as indicated by rambling, irrelevant or incoherent speech	You ask the respondent if they needed help with eating and receive nonsense response: “Let’s go get the sailor suits!	• Rate based on cognitive assessment by trained interviewer • Noted may be difficult to capture in time-limited encounter
Level of consciousness (LOC)	Rating of level of alertness	Alert, lethargic, stupor, coma (unarousable)	• Clear instructions required regarding how much stimuli given and how much the patient responded, to guide scoring • Considered asking caregiver but decided to focus on observation
Judgment (JUD)	Decline in higher-level problem solving or judgment.	Patient succumbed to a financial scam Patient made a large inappropriate purchase	• Originally part of Cognitive domain; because of the importance of assessing baseline judgment, a specific question was developed for this item • Goal is to assess whether there has been a recent change in the patient’s ability handle higher-level decisions • Asked of family member or long-term caregiver
Delusions (PSY)	Evidence of a thought disorder or paranoid ideation.	Insisting other people are trying to harm him/her or steal from him/her	• While multiple raters considered, chose to rate based on trained interviewer rating
Hallucinations (PSY)	Visual, auditory, or tactile hallucinations	Patient says he thought the pile of laundry was a person in his room	• May be hard to observe; experts recommended this be patient-reported • Hallucinations can be an important part of delirium but only present in about 15% cases. If acute onset, helpful indicator of delirium
Inappropriate behavior (PSY)	Uncooperative or socially inappropriate behavior	Resisting care, undressing inappropriately, or unexpected behavior	• New onset is important as this could be baseline symptom of ADRD • Changed term from “bizarre” to “socially inappropriate” behavior
Psychomotor agitation (PSM)	Greatly increased level of motor activity as compared with the norm	Constant shifting of position; increased speed of motor responses, repetitive movements (e.g., picking behaviors)	• Stressed that older adults can have a broad range of motor activity, so need to emphasize that this is outside the norm • Purposeless and repetitive nature of activity is important to note.
Psychomotor retardation (PSM)	Greatly reduced or slowed level of activity as compared with the norm	Moves extremely slowly, such as to pick up a glass	• Experts discussed range of motor activity, and this needs to be shown by extreme slowing of motor processes and speech (responses)
Apathy (EMO)	Dull, flat, listless affect; lacking emotion	Little interest in activities or conversation, no smiling or laughing	• New onset is important to determine as this could be baseline symptom of ADRD • May be challenging to rate if this feature is mild or marked
Withdrawn (EMO)	Detached or aloof, not engaged with daily activities	Paucity of conversation, answers in very few words	• Discussed combining with apathy as difficult to differentiate; often go hand in hand.
			• Decided to keep both in order to assess if they contribute information independently
Irritability or anger (EMO)	New hostility or anger that is outside the norm for the patient	Patient shouting, expressing anger and hostility	• New onset is important as this could be baseline symptom of ADRD
Gait and walking (FUN)	Change in the patient’s walking or balance	Patient newly requiring assistance with walking from a person or device	• Experts noted that becoming unstable is a sensitive sign of mental status change • Focus on new loss of balance or falls • Ask as nursing observation for the past day
Urinary or fecal incontinence (FUN)	New or change in urinary or fecal incontinence	New wetting or soiling of bedding; or need for use of pads/diapers	• Wanted to clarify this is incontinence, urinary or fecal, not constipation which can be common in the hospital • Wanted to focus on new inability due to mental status or inattention • If chronic, note that this can be a symptom of ADRD and other medical issues • Ask as nursing observation for the past day
Poor oral intake (FUN)	New decline in oral intake-- either food or fluids or both.	Nursing notes poor appetite, eating <50% meals	• Expert panel noted reduced oral intake can be an early warning sign of delirium • Wanted to focus on new inability due to mental status or inattention • If chronic, noted that this can be symptom of ADRD or other medical conditions • Noted cannot assess with use of feeding tube at baseline • Ask as nursing observation for the past day
Poor postural control (FUN)	New inability to maintain their usual postural control	Slumping in bed or chair, unable to maintain upright posture, unable to control body positioning	• New onset is important as this could be baseline symptom of ADRD • Goal is to assess whether the patient cannot maintain their usual postural control, which can be another early warning sign for delirium • Ask as nursing observation for the past day, and interviewer observation at bedside
Transfer (FUN)	Decline in ability to transfer between surfaces	Needing help to move from bed to chair, or bed to commode/toilet	• Assess whether there has been a recent loss or decline in ability to transfer • Emphasized acute difficulties (due to mental status) and not due to recent surgery, injury or chronic conditions • Ask as nursing observation for the past day
Self-care, appearance (FUN)	Decline in appearance or hygiene	Unkempt appearance which is new (hair, face, clothing)	• Experts noted that decline in self-care is often an early warning sign of delirium • Experts changed term from “grooming” to “self-care, appearance” • Ask as nursing observation for the past day, and interviewer observation at bedside
Hypersomnolence (FUN)	New daytime drowsiness, which is excessive	Repeatedly dozes off while you are asking questions	• New onset is important as this could be baseline symptom of ADRD • Ask as nursing observation for the past day, and interviewer observation at bedside
Sleep-wake cycle reversal (FUN)	New sleep-wake cycle reversal	Evidence of a reversed cycle, manifested by frequent napping during the day and insomnia at night	• New onset is important as this could be a baseline symptom of ADRD • Decided to include insomnia in the examples and not as its own independent item because insomnia is common in the hospital • Ask as nursing observation for the past day

The expert panel emphasized the importance of change in function to evaluate delirium severity in the face of ADRD. Given this change, local experts opted to include three additional functional subdomains (i.e., gait, incontinence, and transferring) in the new measure, which were not included in the DEL-S. Th us, new items were developed for these subdomains. For these items, the expert panel wanted to distinguish new-onset changes that may be associated with delirium from chronic symptoms associated with ADRD or chronic conditions. For example, for the subdomain of poor oral intake, the local expert working group recommended highlighting change in intake due to poor attention and mental status, rather than reduced appetite alone.

Respondents and Look-Back Periods

Table E (available in the on-line version of this article) provides the panel consensus regarding who would be the best respondent/rater for each item, across each of the three look-back period timeframes. Th e respondent/rater varied according to setting (i.e., home, hospital, nursing home) and timeframe. For example, to rate change from baseline in items such as “easily distracted” as part of the Inattention domain, the preferred reporters are knowledgeable family caregivers at home or nursing home staff (nurse or certified nursing aide [CNA]) who are familiar with the patient. To rate acute change over the past day in the hospital, the preferred reporter is the patient’s nurse. A trained research assistant formally assessing the patient was recommended for rating immediate observations at the bedside. For other items, such as mild versus marked apathy, which is a subjective measure and difficult to determine, the panel ultimately decided that for the field test, all three possible respondents/raters across all three look-back periods should be included to compare and determine the optimal respondent.

Table E.

Decisions about Respondent/Rater, Look-Back Period for Delirium Severity in ADRD Across Settings

Domain	Item	Respondent/Rater, Look-Back Period
		Establish baseline, change from baselinea	Report acute changes over past 24 hours	Immediate observations at bedside
INA	Difficulty focusing attention	FM / NH		RA
INA	Easily distracted	FM / NH		RA
COG	Disorientation			RA
COG	Disorganized thinking	FM / NH		RA
LOC	Level of consciousness			RA
PSY	Delusions			RA
PSY	Hallucinations		PT
PSY	Inappropriate behavior	FM / NH	RN	RA
PSM	Psychomotor agitation	NH	RN	RA
PSM	Psychomotor retardation	NH	RN	RA
EMO	Apathy	FM / NH	RN	RA
EMO	Withdrawal	FM / NH	RN	RA
EMO	Anger/Hostility/Irritability	FM / NH	RN	RA
FUN	Performance - Gait and walking	NH	RN
FUN	Performance - Urinary or fecal incontinence	NH	RN
FUN	Performance - Poor oral intake	NH	RN
FUN	Performance - Poor postural control	NH	RN	RA
FUN	Performance - Transferring	NH	RN
FUN	Performance - Self-care, appearance	NH	RN	RA
FUN	Sleep disorder - hypersomnolence	NH	RN	RA
FUN	Sleep disorder - sleep-wake cycle reversal	NH	RN

Note. INA = Inattention domain; COG = Cognitive; LOC = Level of Consciousness; JUD = Judgment; PSY = Psychiatric-Behavioral; PSM = Psychomotor features; EMO = Emotional Dysregulation; FUN = Functional; FM = Family Member; RA = Research Assistant; NH = Nursing Home staff (registered/licensed nurse or certified nurse aide); PT = Patient; RN = Registered or licensed hospital nurse. Note: Family/NH staff asked to assess change from baseline with each item. RN asked for observations during shift or from report for past 24 hours. RA asked to directly observe behavior at bedside before, during, and after interview with cognitive screening tests.

^aBaseline prior to acute illness: Weeks to months, depending on setting and logistics

Because episodic symptoms such as delusions and hallucinations are difficult to capture, there was substantial discussion regarding the best respondent/rater for each of these items, as well as the optimal look-back period. Local experts decided that delusions would be best assessed via bedside observations by research staff and hallucinations would be best self-reported by patients over the past 24 hours. Hallucinations are difficult to observe during a brief bedside encounter and relatively rare in delirium (although highly specific). Thus, hallucinations are the one patient-reported item included in the delirium severity measure for persons with ADRD.

Table F presents the consensus decisions on domains and items to be rated by each rater/ respondent, along with the rationale. Knowledgeable family members were considered the optimal reporters for change from baseline status across settings. These individuals are best able to report chronic and superimposed acute changes in cognitive abilities in persons with ADRD. In the nursing home, staff (including nurses and CNAs) who know the patient well would be able to provide information on acute changes superimposed on dementia. The expert panel recognized that the acute care nurse in the hospital may not be familiar with the patient’s pre-hospital status, but would be in the best position to report acute changes over the past day in cognitive and functional status. Trained research assistants would be preferred to conduct cognitive testing and make observational ratings of potential delirium symptoms at the bedside; this assessment could also be done by clinical nursing staff with training. Finally, patient self-report was recommended for hallucinations in the past 24 hours. Th e expert panel noted that the clinical setting in which the instrument would be administered should be taken into consideration, and the wording of items (e.g., look-back period, verb tenses) may need to be adapted for optimal use, such as when using the instrument in the acute care hospital versus nursing home.

Table F.

Local Expert Panel Decisions on Items and Rationale for Respondents/Raters for Each Item

	Rationale	Domain	Item
Family	Ability to report on patient status long-term, and to provide	INA	Difficulty focusing attention
member	information on acute (vs. chronic) change from baseline.	INA	Easily distracted
		PSY	Inappropriate behavior
		EMO	Apathy
		EMO	Withdrawal
		EMO	Anger/Hostility/Irritability
Nursing home	Ability to report on patient’s status over long-term, and to	INA	Difficulty focusing attention
staff	provide information on acute (vs. chronic) change from	INA	Easily distracted
	baseline. Nursing home staff report on long-term observations	COG	Disorganized thinking
	similar to both family and nurse in hospital.	PSY	Inappropriate behavior
		PSM	Psychomotor agitation
		PSM	Psychomotor retardation
		EMO	Apathy
		EMO	Withdrawal
		EMO	Anger/Hostility/Irritability
		FUN	Performance – Gait and walking
		FUN	performance – Urinary or fecal incontinence
		FUN	Performance – Poor oral intake
		FUN	Performance – Poor postural control
		FUN	Performance – Transferring
		FUN	Performance – Self-care, appearance
		FUN	Sleep disorder – Hypersomnolence
		FUN	Sleep disorder – Sleep-wake cycle reversal
		FUN
Nurse in	Ability to report on patient’s status throughout their shift and	PSY	Inappropriate behavior
hospital	on any changes from nursing report during past day. Can rate	PSM	Psychomotor agitation
	acute, short-term changes. Nursing observations during care	PSM	Psychomotor retardation
	key to rate acute change in functional domains.	EMO	Apathy
		EMO	Withdrawal
		EMO	Anger/hostility/irritability
		FUN	Performance – Gait and walking
		FUN	performance – Urinary or fecal incontinence
		FUN	Performance – Poor oral intake
		FUN	Performance – Poor postural control
		FUN	Performance – Transferring
		FUN	Performance – Self-care, appearance
		FUN	Sleep disorder – Hypersomnolence
		FUN	Sleep disorder – Sleep-wake cycle reversal
		FUN
Research	Makes direct observations based on cognitive testing, and also	INA	Difficulty focusing attention
Assistant	informal observations before, during, and after interview.	INA	Easily distracted
		COG	Disorientation
		COG	Disorganized thinking
		LOC	Level of consciousness
		PSY	Delusions
		PSY	Inappropriate behavior
		PSM	Psychomotor agitation
		PSM	Psychomotor retardation
		EMO	Apathy
		EMO	Withdrawal
		EMO	Anger/hostility/irritability
		FUN	Performance – Poor postural control
		FUN	Performance – Self-care, appearance
		FUN	Sleep disorder – Hypersomnolence
Patient	Self-report in last 24 hours. Patient might be observed, but experts recommended self-report as best approach.	PSY	Hallucinations

Note. INA = Inattention domain; COG = Cognitive; LOC = Level of Consciousness; JUD = Judgment; PSY = Psychiatric-Behavioral; PSM = Psychomotor features; EMO = Emotional Dysregulation; FUN = Functional

Verifying Face and Content Validity, Feasibility, and Real-World Applicability

The nurse expert panel helped verify the best respondents for each item, and refined item descriptions and examples to facilitate easy operationalization across clinical settings. Crucially, the nurse expert panel made specific suggestions to streamline the number and wording of items to minimize the length of the instrument. Nurse experts also recommended edits to ensure that the potential delirium severity items would be appropriate for individuals on the continuum of mild to moderate dementia in terms of item content and wording (e.g., taking questions literally), examples (e.g., applicability across clinical settings), and length (e.g., considering the patient’s shortened attention span). The subdomain of insomnia was ultimately removed from the instrument based on the nurse expert panel recommendation, because they believed insomnia would be difficult to measure in the hospital or nursing home setting. Other sleep subdomains—hypersomnolence and sleep-wake cycle reversal—were retained.

Nurse experts emphasized the need for a trained assessor to administer the delirium severity measure. A nurse expert explained, “It is hard for bedside clinicians to know there is delirium in the [patient] unless they know [the patient’s] baseline, which in hospital is infrequent. In other settings it may be more frequent” Another nurse expert remarked that interviewing nurses and/or family caregivers would be important:

When I learned what delirium looked like, it was through spending many hours with patients and watching them change. I question the observation abilities of research assistants to pick up on symptoms and grade them as mild versus marked [without speaking with nurses or family caregivers].

Nurse experts noted the importance of obtaining information on patients from family caregivers who can provide detailed and nuanced information about patients’ baseline cognitive functioning and acute changes with delirium.

DISCUSSION

We describe the process undertaken by our local expert panel and nurse expert panel in developing and refining items to assess delirium severity in patients with ADRD. Th e final items include root descriptions for each content subdomain, respondents/raters for each item, and the look-back period for assessment. Functional questions thought to have more relevance in a patient population with underlying ADRD were added that had not been included in the DEL-S. The challenges of look-back period and varying clinical settings (e.g., home, hospital, nursing home) were noted to add complexity to this measure, and would need to be carefully addressed through consideration of appropriate respondents and reference timeframes.

STRENGTHS AND LIMITATIONS

The strengths of this work include the multidisciplinary and iterative Delphi-based consensus process used by the local expert panel. Local experts have a history of collaboration and flexibility with receptiveness to feedback. As another strength, the item generation process addresses two critical issues in nurses’ assessment of delirium. First, nurses have reported assessment of delirium as challenging due to its wide range of non-specific symptoms, atypical presentations, and misattribution of delirium when the patient’s baseline is unknown (El Hussein & Hirst, 2016). The incorporation of items, such as acute change in function, that help distinguish features of delirium can help navigate nuances in clinical presentation. Second, instruments that comprehensively assess delirium have been cumbersome to administer during the course of patient care. The perspectives of our nurse expert panel were invaluable ensuring face and content validity of the items, and enhancing the feasibility of the final delirium severity measure. In addition, we found that the nurse experts’ comments mirrored many of those of the external expert panel from our prior study, highlighting overarching issues in research and clinical care of patients with ADRD, including caregiver knowledge, different clinical settings, and ADRD disease progression.

We note some limitations to this work. There remain unresolved issues regarding look-back periods (i.e., time reference) and best respondents/raters of information; thus, we are planning to include all respondents across all look-back periods for some of the items. In addition, current items for the delirium severity measure focus on patients with mild to moderate dementia. Severe ADRD is not addressed and will be an important area for future study. Future refinement of the items should also incorporate feedback from other raters or respondents, including family caregivers, CNAs, and other clinical staff. We plan to address these limitations in a future field-test of the delirium severity items. Finally, it is important to note that although items have been generated, the delirium severity instrument is not yet ready for clinical application. It has not yet been field-tested, or validated against reference standard ratings.

CONCLUSION

The current study presents important work to help advance the fi eld of delirium assessment. Th rough a systematic approach, the study has identified and operationalized items, raters, and look-back periods to determine delirium severity in ADRD. This work has contributed in substantial and practical ways to advancing our understanding of delirium severity in the context of ADRD, a novel area that has not been well-studied. Moreover, this work highlights the importance of rigorous item generation in effective instrument development. Completion of this study lays the groundwork for future fi eld testing including additional refinement of items, improved feasibility for use in clinical practice, and validation of the items against reference standard ratings.

It is clear that careful and nuanced identification of distinguishing features is key to helping better measure delirium severity in patients with dementia (Hshieh et al, 2020; Schulman-Green et al, 2022). This is a challenging yet essential area, as patients with dementia are at high risk for delirium. Ultimately, the availability of a reliable and validated delirium severity measure for persons with ADRD will have a number of implications for clinical care and research, from improved diagnosis to advancing clinical trials aimed at the prevention and treatment of delirium in ADRD, and for use as a valuable outcome measure. A robust delirium severity measure for patients with ADRD recognizes the family as an integral part of the health care team and uses a multidisciplinary team approach in which nursing plays a central role in assessment. Furthermore, the use of a delirium severity measure for ADRD will strengthen transitions of care and facilitate effective communication of care needs across multiple care settings.