Screening for Delirium via Family Caregivers: Convergent Validity of the Family Confusion Assessment Method (FAM-CAM) and Interviewer-Rated CAM

Abstract

Background/Objectives

As the population ages, delirium superimposed on dementia is becoming a common problem. Family caregivers may provide critical information to assist with early detection. The purpose of this study was to explore agreement between the Family Confusion Assessment Method (FAM-CAM) for delirium identification and interviewer-rated CAM delirium ratings.

Design

Exploratory analysis of agreement.

Setting

Community.

Participants

52 family caregivers and 52 elders with pre-existing impairment on standardized cognitive testing.

Measurements

The interviewer-rating for delirium was determined by fulfillment of the Confusion Assessment Method (CAM) algorithm.

Results

The total sample included 52 paired CAM:FAM-CAM assessments completed across 52 dyads of elders with pre-existing cognitive impairment and family caregivers. The point prevalence of delirium was (7/52)13%. Characteristics did not differ significantly between the delirium and non-delirium groups. The FAM-CAM questions that mapped directly to the original four-item CAM algorithm had the best overall agreement with the interviewer-rated CAM, kappa=0.85 (95% confidence interval, CI 0.65–1.0), sensitivity 88% (CI 47–99%) and specificity 98% (CI 86–100%).

Conclusion

The FAM-CAM is a sensitive screening tool for detection of delirium in elders with cognitive impairment utilizing family caregivers, with relevance for both research and clinical practice.

INTRODUCTION

Delirium superimposed on dementia (DSD) is increasingly problematic as the population ages. The prevalence of DSD ranges from 18% to 89% in hospitalized and community dwelling older adults (1, 2). Dementia is a major risk factor for the development of delirium (3, 4). Persons with dementia who develop delirium have poor functional outcomes, increased rates of re-hospitalization and mortality, and an accelerated downward trajectory of their cognitive impairment (2, 5, 6). Lowered levels of cognition and function often contribute to the decision to institutionalize (5, 7).

Although a relationship exists between delirium and dementia, there are several features of delirium that do not overlap with dementia and are potentially more easily recognized by care providers; these include an acute change in mental status, impaired attention, symptom fluctuation, and altered level of consciousness. Acute change and inattention are hallmarks of delirium. Except for persons at the end of life, level of consciousness should not be impaired in older adults with dementia and is an important indicator of a change in mental status (8–10). Early detection and treatment of DSD could slow its progression and prevent complications, both critical to remaining in the community.

Family caregivers provide 80% of the care for the 3.57 million community-dwelling elders with dementia in America (11). Typically, family caregivers spend at least 40 hours each week caring for their family member with dementia (11). Family caregivers are present, often on a 24-hour basis, and may make critical observations about mental status changes in persons with dementia that, when shared, can result in earlier identification and treatment of delirium.

The Confusion Assessment Method (CAM) is widely used to screen for the presence of delirium (12, 13). The only instrument available for family screening for delirium, the Family Confusion Assessment Method (FAM-CAM), was developed by Inouye and colleagues to screen for delirium by interviewing family caregivers (14–16). Derived from the original 10-item CAM instrument (12), the FAM-CAM was adapted to maximize detection of delirium (i.e., acute onset and fluctuating course, inattention, disorganized thinking, altered level of consciousness, disorientation, perceptual disturbances, and psychomotor agitation) from the observations of family caregivers. While relatively uncommon in delirium, “inappropriate behavior” and perceptual disturbances such as hallucinations were included in the FAM-CAM to maximize sensitivity and specificity.

The specific aim of this study was to explore the convergent validity (17, 18) of the FAM-CAM, completed by family caregivers, and the CAM, completed by trained interviewers who directly assessed elders with pre-existing cognitive impairment. A second aim was to identify the FAM-CAM items that maximize sensitivity and specificity for delirium screening in older persons with cognitive impairment.

METHODS

Study Sample

The sample included 88 paired CAM and FAM-CAM assessments completed across 58 dyads of family caregiver-elder with pre-existing cognitive impairment and their family caregivers. The dyads were drawn from two separate primary studies: the “eCare for Eldercare” pilot study conducted in three central Pennsylvania communities (N=13) (19) and the “Hospital to Home: Cognitively Impaired Elders/Caregivers” study in Philadelphia (N= 45) (20). Records were dropped from analysis if the caregiver indicated he/she had not seen the elder participant in two or more days prior to FAM-CAM completion (n=6 records/6 caregivers).

Inclusion and exclusion criteria were similar in both studies. Older adults with pre-existing cognitive impairment included those who were 65 years or older, community-dwelling, and English-speaking. Cognitive impairment was determined using validated methods in both studies: Modified Blessed Dementia Rating Scale (21) score of >3 and symptoms of dementia documented over a 6 month period (eCare for Eldercare study) or, in the Hospital to Home study, either previous history of dementia or deficits in orientation, recall or executive function (≤4 on Six Item Screen or CLOX1=≤10) (22–24). Exclusion criteria for both parent studies were any terminal condition (i.e., <6 months prognosis). Additional exclusion criteria for the Hospital to Home study included undergoing active cancer treatment, active substance abuse, and recent cerebrovascular accident.

For family caregivers, inclusion criteria were agreement to participate, English-speaking, and daily interaction with the elder. Additional inclusion criteria for caregivers in the eCare for Eldercare study included daily internet access via a personal computer or study-provided smart phone.

Measures

Interviewer-Rating for Delirium

The original CAM instrument was a 10-item instrument validated against expert raters (12). The 4-item CAM diagnostic algorithm for delirium was developed for this study, and has gained widespread use for identification of delirium. By the CAM diagnostic algorithm, a positive screen for delirium is indicated by an acute change in mental status from the person’s baseline as well as fluctuation of symptoms, the presence of inattention, and either disorganized thinking or an altered level of consciousness (12). Wei and colleagues (25) conducted a systematic review evaluating the performance of the CAM. Based on seven high quality studies (N=1,071), the combined sensitivity was 94% (95% CI = 91–97%) and specificity was 89% (95% CI = 85–94%). Recent studies documented the CAM maintains high sensitivity and specificity when used appropriately (26, 27). Additionally, several studies documented that use of the CAM can facilitate identification of delirium in the face of dementia (12, 28–30).

For the present study, the interviewer-rating for delirium was determined by fulfillment of the CAM diagnostic algorithm (12), and was based on direct assessments of the cognitively impaired elders made by trained research assistants (RAs). In both parent studies, the CAM was completed based on results of cognitive screening tools.

All RAs had relevant educational preparation, training, and experience working with patients and their families. The RAs received further training in administration of the CAM and other instruments through self-study, didactic sessions, paired mock interview sessions with inter-rater reliability assessment, and paired ratings of older adults observed by an expert interviewer. During standardization, the inter-rater reliability statistic for the overall delirium rating between the RAs and the expert interviewer was Kappa=0.95 in 19 paired observations. FAM-CAM Rating

The presentation of the FAM-CAM to the family caregivers was similar in both parent studies. Caregivers were instructed in individualized face-to-face sessions about the use of the FAM-CAM that included education about each symptom and instructions in how to score each item. In addition, a one-sentence introduction on the FAM-CAM form requested the family caregiver to “please answer each of the following questions about the family member you are caring for at home (16).” In the eCare for Eldercare study, RAs also taught the family caregivers how to access the study website, and how to complete the FAM-CAM form using either their personal computer or the smart phone. Participants in the Hospital to Home study were provided a paper copy of the FAM-CAM and received similar didactic instructions. In both studies, the RAs were available to provide additional guidance to clarify symptoms and scoring instructions on an ongoing basis. Family caregivers were encouraged to provide written comments on the delirium symptoms; either in a free text comment field in the eCare for Eldercare study or directly to the RA in the Hospital to Home study, the caregiver was also able to ask questions about completing the FAM-CAM.

Study Procedures

Demographic information was collected by the RA from interviews with the family caregivers during the initial visit. RA visits with participating elders with cognitive impairment were pre-scheduled according to the parent study protocols. In the eCare for Eldercare study, the FAM-CAM ratings were completed and transmitted electronically by family caregivers daily but paired ratings were completed by RAs visiting the home to conduct the interviewer-ratings once a week (N=40). When a FAM-CAM rating was positive for delirium on any daily rating, an RA was dispatched within 24 hours (n=8 of 40) to conduct a paired rating. RAs were blinded to the caregiver responses to individual FAM-CAM questions but not to the overall conclusion (delirium/no delirium). In the Hospital to Home study, the interviewer-rating (paired with FAM-CAM) was done on post hospital discharge visits (N=48) scheduled at 2, 6, 12, 24 and 52 week intervals. The CAM was administered as part of a battery of scales assessing the study participant followed by an interview using standardized instruments with the caregiver for the entire study. The FAM-CAM administration was added to the battery of scales administered to the caregiver for the final 28 week period. By design, the RA was blinded to the caregiver FAM-CAM responses. While the caregiver was completing the FAM-CAM, the RA was assessing the elder participant with pre-existing cognitive impairment.

Statistical Analysis

Descriptive statistics are presented for basic demographic variables. FAM-CAM sensitivity, specificity, and kappa were calculated against a interviewer-rated CAM delirium rating established by carefully trained RAs, along with their 95% confidence intervals. For these FAM-CAM calculations, screening “delirium positive” indicates being classified as impaired by the CAM criteria which were specified a priori. True positives (TP) were those who screened positive for delirium by FAM-CAM and had delirium by interviewer-ratings; false positives (FP) were those who screened positive by FAM-CAM but were not delirious by the interviewer-rating; true negatives (TN) were those who screened negative by FAM-CAM and were not delirious by the interviewer-rating; and false negatives (FN) were those who screened negative by FAM-CAM but were delirious by the interviewer-rating. Sensitivity was calculated as TP/(TP + FN); specificity was calculated as TN/(TN + FP). Sensitivity and specificity according to the operational definitions were compared using the McNemar test. Cohen’s Kappa is a measure of chance-corrected agreement between raters. It is based on the percent of data values on the main diagonal of the table, adjusted for the amount of agreement that could be expected due to chance alone.

Analyses using different combinations of the FAM-CAM questions were performed beginning with questions which mapped to the original CAM algorithm and then 11 other possible combinations of the FAM-CAM questions to assess their impact on sensitivity, specificity and kappa of the FAM-CAM. Based on these preliminary analyses, the focus was on 3 features which had the maximal impact on kappa (i.e., disorientation, perceptual disturbances and “inappropriate behavior”), individually and in combination.

RESULTS

The final sample (Table 1) included 52 paired assessments from 52 dyads. No family caregivers were dropped from the study due to inability to follow instructions regarding FAM-CAM completion.

Table 1.

Demographic Characteristics: Elders with Pre-existing Cognitive Impairment (N=52)

Variable	Overall N=52 Mean (SD and range) or (%)	Delirium Positive n=7 Mean (SD and range) or (%)	Delirium Negative n=45 Mean (SD and range) or (%)	p Value Comparing Delirium Positive to Negative
Age, years	82 (8, 83 66–103)	(9, 81 70–103)	(8, 66–99)	0.503
Education, years	12 (4, 10 6–25)	(2, 12 6–12)	(4, 7–25)	0.113
Female	35 (67%)	5 (71%)	30 (67%)	1.000
Male	17 (33%)	2 (29%)	15 (33%)
Race:
Caucasian	26 (50%)	3 (43%)	23 (51%)	0.743
African-American	25 (48%)	4 (57%)	21 (47%)
Caregiver relationship:
Son/Daughter	30 (58%)	3 (28%)	27 (61%)	0.461
Spouse	12 (23%)	3 (38%)	9 (20%)
Other relative	8 (15%)	2 (25%)	6 (14%)
Friend	2 (4%)	0 (0%)	2 (5%)

NOTE: Delirium – positive and negative based on interviewer-rating.

Table 1 shows the demographic characteristics of the participants and the relationship of their caregivers. The point prevalence of positive cases of delirium was 13% (7/52) by the interviewer-rating. Sample characteristics did not differ significantly between the delirium and non-delirium groups, or between the sub-samples from the two parent studies.

Results were similar for the analyses completed using all records that include multiple paired ratings (not shown). The numbers of paired ratings/dyad were: 1 rating for 41 participants; 2 for 3 participants; 3 for 3 participants, and 4 or more for 5 participants for a total of 82 paired observations. The results were remarkably similar comparing multiple paired ratings to those containing only the first record per patient. Therefore, the findings from this study are based on the data analysis restricted to the first paired rating from each of the study dyads.

Table 2 demonstrates the performance of the FAM-CAM items using the first paired record per dyad compared with the CAM interviewer-ratings for delirium. This table begins with the original CAM algorithm and then sequentially adds additional items that impacted the sensitivity, specificity and kappa of the FAM-CAM in our preliminary analyses. The FAM-CAM sequences are listed in descending order by their kappa value. The FAM-CAM question combination that has the highest kappa (0.85) is the sequence of questions that maps directly to the original CAM algorithm with no additional questions.

Table 2.

Sensitivity, Specificity and Kappa for FAM-CAM Questions (first record/dyad, N=52)

FAM-CAM Questions	Sensitivity % (95% CI)*	Specificity % (95% CI)**	Kappa (95% CI)
Original CAM Algorithm: Acute	88 (7/8)	98 (43/44)	0.85
Onset + Fluctuation + Inattention + Disorganized thinking or Altered consciousness	(47, 99)	(86, 100)	(0.65, 1.00)
Added: Disorientation	75 (6/8) (42, 100)	98 (43/44) (85, 100)	0.77 (0.51, 1.00)
Added: Perceptual Disturbances	50 (4/8)	100 (44/44)	0.63
(Hallucinations)	(40, 100)	(80, 98)	(0.30, 0.95)
Added: “Inappropriate behavior”	38 (3/8) (10, 74)	100 (44/44) (90, 100)	0.50 (0.14, 0.86)
Added: Disorientation + Perceptual	13 (1/8)	100 (44/44)	0.19
Disturbances (Hallucinations) + “Inappropriate behavior”	(<1, 53)	(90, 100)	(−0.13, 0.52)

NOTE: CAM=Confusion Assessment Method; FAM-CAM=Family Confusion Assessment Method;

^*Sensitivity=proportion screened as delirious using the FAM-CAM among those screened as delirious using the CAM;

^**Specificity=proportion screened as not delirious using the FAM-CAM among those not screened as delirious using the CAM.

Analysis was repeated without the eCare for Eldercare data to address any concerns about the RAs in that study who were not blinded to the FAM-CAM results while assessing participants using the CAM. Results were very similar to the analysis that included both sources of participants.

FAM-CAM questions that target the delirium features disorientation, perceptual disturbances and “inappropriate behavior” were included to determine their value, if any, in various combinations with other FAM-CAM questions for the screening of delirium symptoms. No combination including any or all of these questions improved the kappa of the FAM-CAM. However, these additional questions did slightly improve specificity from 98 to 100%.

In this exploratory study, overall agreement between the CAM and FAM-CAM was 96% (50/52). There was one FAM-CAM that was false-positive where the RA rated inattention and change in level of consciousness as absent while the caregiver rated the same items as present. There was one FAM-CAM that was false-negative where the only feature that the RA and the caregiver rated similarly was disorientation. This raises the possibility of either over- or under-identification of delirium symptoms by family caregivers, who sometimes weighted the significance or severity of symptoms differently than the trained reviewers. The disagreement in ratings between the family caregiver and the interviewer rating prompted re-training of the family caregiver on the FAM-CAM; thereafter, paired ratings had complete agreement.

Caregivers in the eCare for Eldercare study had the opportunity to add comments in a free-text field accompanying each FAM-CAM question. None of the caregivers indicated problems, and uncertainties were only raised on a few questions with comments, as indicated in Table 3. The question garnering the most comments was the initial overall question that asks whether the caregiver has noticed any recent problems with memory, concentration, attentiveness, confusion, inappropriate behavior or extreme sleepiness. The other question that generated more comments was regarding excessive drowsiness in the daytime, which generated four different comments.

Table 3.

FAM-CAM Free Text Comment Field Entries

FAM-CAM Question Topics	Caregiver Comments
Overall question: memory, concentration, attentive, confusion, inappropriate behavior, extreme sleepiness	“slightly”
	“forgot day of the week”
	“concentration”
	“confused”
	“behaving inappropriately, bad day”
	“last night when out to dinner”
Inattention	“normal”
Speech disturbance	“rambling”
Disorientation	“time”
Inappropriate behavior	“kept referring to wanting to go to bed”
Excessively drowsy in daytime	“don’t know”
	“when he comes home from day care”
	“sometimes”
	“only on days he stays home because he’s bored or just watching TV”

DISCUSSION

This exploratory study is the first we are aware of to examine the agreement of a family caregiver administered delirium instrument, the FAM-CAM, with the interviewer-based CAM rating administered by a trained interviewer. Agreement between the two assessments was excellent with Kappa=0.85 (95% confidence interval=0.65, 1.00). Since identifying the presence of delirium as early as possible is vital to the outcomes of the person experiencing delirium, the use of the FAM-CAM may have great clinical relevance. The algorithm of FAM-CAM questions with the highest kappa (0.85) used in this study included positive responses to the features of the original CAM algorithm. These findings lend support for the proposed approach for the FAM-CAM. The level of agreement found in this study should be adequate for screening purposes.

These findings hold important potential implications. First, substantial level of agreement between the caregiver completed FAM-CAM and the interviewer CAM assessment indicates that caregivers can provide accurate information to determine if delirium is present. We do not propose or recommend that caregivers can “diagnose” delirium with the FAM-CAM, rather, the instrument may be useful to identify symptoms early which can then be brought to the attention of healthcare providers. Although this study had a relatively small sample, the point prevalence of delirium was 13% in elders with cognitive impairment. This finding is similar to point prevalence rates in two previous study populations of community dwelling persons with dementia, which were reported as 13% (31) and 18% (2).

Second, six of the FAM-CAM questions mapped to the items assessing acute onset, fluctuating course, inattention, disorganized thinking, and altered level of consciousness and had maximal agreement. Further testing of these questions is recommended in larger samples including participants who are fully delirious, those with and without dementia, those artificially sedated, and normal controls.

Strengths of this study include a high level of agreement of the FAM-CAM with an interviewer CAM rating of delirium by trained RAs. The sample included community dwelling older adults with pre-existing cognitive impairment as well as both white and non-white races. The few comments generated by caregivers in the eCare for Eldercare parent study indicate no difficulty among caregivers about answering the questions independent of study personnel.

Major limitations of this study include the lack of a true external gold standard for delirium, and the small homogenous sample, that is, participants with pre-existing cognitive impairment only. In addition, different administration methods were used for the FAM-CAM across the two parent studies (paper and pencil, personal computer and smart phone) which may have influenced the overall results. In the eCare for Eldercare study, the RAs were not blinded to FAM-CAM results when assessing participants with the CAM. Further, close temporal proximity of assessments between the family caregiver and RA was not always achieved. On two occasions, positive FAM-CAM ratings for delirium were transmitted late in the evening hours; although the RA visit was made within 24 hours, it was not on the same day. Additionally, only limited family caregiver demographics were collected, and educational level was unknown. RAs were blinded to caregiver answers to individual FAM-CAM questions in both parent studies but RAs were not blinded to the overall conclusion (delirium/no delirium) to the FAM-CAM in the eCare for Eldercare study. Finally, a major problem with the current version of the FAM-CAM instrument is the insufficient assessment of acute onset. An adapted question was utilized, asking: “in the last day or so, have you noticed [FAM-CAM symptom]…” Although that lead-in specifies a recent time period, it does not explicitly ask whether the observed change had an acute onset. This can be avoided by using the official FAM-CAM provided at the website www.hospitalelderlifeprogram.org.

The FAM-CAM is a screening tool for delirium utilizing the observations of family caregivers. Use of this tool may help to engage the family caregiver in alerting health care professionals about a change in mental status that is worthy of further evaluation, thus, heightening likelihood of earlier identification of delirium. The FAM-CAM may be useful to establish the presence of delirium at hospital and emergency department admission, and may facilitate longitudinal studies of delirium for homebound or community-based elders. Future work may include evaluation of different modes of FAM-CAM assessment, validation in larger, diverse samples against external gold standards, tested against other delirium and cognitive scales (not the CAM) and investigating the most effective means to communicate FAM-CAM results to health care practitioners.