Factor Analysis of the Short-Form Cohen-Mansfield Agitation Inventory and the Measurement Invariance by Gender

Abstract

Background and Purpose:

The Cohen-Mansfield Agitation Inventory (CMAI) is a widely used measure of agitation. The purpose of this study was to test the internal consistency, reliability, and validity of short-form CMAI in a sample of nursing home residents with cognitive impairment and examine if it is invariant across gender.

Methods:

This study utilized baseline data from a randomized trial including 553 residents from 55 nursing homes. Data was analyzed using structural equation modeling.

Results:

Confirmatory factory analysis supported the three-factor structure including aggressive (α = .794), physically nonaggressive (α = .617), and verbally agitated (α = .718) behaviors. Invariance testing confirmed that the shortened measure is invariant across gender.

Conclusions:

Findings provide validity evidence of short-form CMAI to assess agitation and gender differences in agitation in nursing home population.

The assessment and management of behavioral and psychological symptoms of dementia (BPSD) is important for the clinical care of residents in long-term care settings. BPSD interferes with daily personal care, puts an enormous strain on the caregivers (Rabinowitz et al., 2005; Sommer et al., 2010) and may contribute to staff and resident injuries during care (Galindo-garre, 2015; Lachs et al., 2011; Pillemer et al., 2016). This further affects the quality of life of the residents (Davison et al., 2017; Livingston et al., 2017) and the work ability of the nursing staff (Brett et al., 2017).

Agitation, defined as “inappropriate verbal, vocal or motor activity that is not judged by an outside observer to result directly from the needs or confusion of the agitated individual” (Cohen-Mansfield, 1991; Cohen-Mansfield & Billig, 1986), is one of the most frequently observed behavioral symptom among older adults with cognitive impairment or dementia in nursing home and residential care settings (Livingston et al., 2017; Zuidema et al., 2007). Some examples of the agitated behaviors demonstrated by older adults include cursing or verbal aggression, screaming, kicking, pushing, general motor restlessness, disruptive vocalization, and repetitive behaviors. These behaviors are prevalent in approximately 50%–90% of nursing home residents with cognitive impairment or dementia (Arai et al., 2017; Livingston et al., 2017; Pelletier & Landreville, 2007; Zuidema et al., 2007). Physically aggressive behaviors such as hitting and kicking are more common among the residents with very severe cognitive decline who are more likely to exhibit agitated behaviors, and irritability and verbally agitated behaviors are common among those with moderate to severe cognitive decline (Livingston et al., 2017; Zuidema et al., 2009). Existing literature presents mixed findings in regard to gender differences in the occurrence of agitated behaviors (Cohen-Mansfield, 1996; Ho et al., 2016; Schreiner, 2001; Tao et al., 2018; Vance et al., 2003). Particularly, assessment of agitation based on Cohen-Mansfield Agitation Inventory (CMAI; Tao et al., 2018), Computer-Assisted Behavioral Observation System (CABOS; Burgio et al., 1994), and Multidimensional Dementia Assessment Scale (MDDAS; Lövheim et al., 2009) yielded that women exhibit higher number of agitated behaviors (Tao et al., 2018; Vance et al., 2003) and are more likely to pace/wander, complain, hide/hoard things, and experience anxiety and irritability than their male counterparts (Lövheim et al., 2009; Tao et al., 2018). Conversely, the comparison of the dimensions of agitated behaviors, as opposed to item-by-item comparison, yielded no significant differences. Specifically, there was no significant difference in physically aggressive, physically nonaggressive, and verbally aggressive behaviors between men and women based on CMAI; although women were more likely to exhibit verbally nonaggressive behaviors than men in the unadjusted model, this was no longer significant in the adjusted model with covariates (Tao et al., 2018). In another study, Maji et al. (2012) similarly did not find significant gender differences except in verbal agitation; women were more likely to exhibit verbal agitation. Agitation was assessed using 29-item CMAI (Cohen-Mansfield, 1991). In examining the gender differences in the clusters of agitated resident behaviors, Ho et al. (2016) also reported the presence of fewer diverse clusters in men and women. Agitation was assessed using Dutch version CMAI-D (Zuidema et al., 2011). The exploratory factor analysis yielded eight clusters in men and seven in women and, majority including physically aggressive behavior, physically nonaggressive behavior, verbally nonaggressive behavior, sexual behavior, and vocal agitation were identical in both groups. The remaining clusters although not identical were similar, for example, “hiding/hoarding” observed in men was similar to “hiding/hoarding and inappropriate handling of objects” observed in women.

The Cohen-Mansfield Agitation Inventory (CMAI) is one of the most frequently used measures to assess and evaluate agitated behaviors among older adults (Chenoweth et al., 2009; Laybourne et al., 2019; Rabinowitz et al., 2005; Suh, 2004; Tao et al., 2018). It was originally developed to assess the frequency of agitated behaviors among older adults in nursing home settings (Cohen-Mansfield, 1991; Cohen-Mansfield et al., 1989). It has since been validated and used in community as well as acute care settings (Cohen-Mansfield, 1991; Kupeli et al., 2018; Laybourne et al., 2019; Tractenberg et al., 2002). Additionally, the measure has been adapted and tested for psychometric properties in various cultures. Some examples include the Dutch version CMAI-D with evidence of reliability based on correlation coefficient ranging from 0.61 to 0.73 for three factors (Zuidema et al., 2011; Zuidema et a., 2006); the Korean version CMAI-K with evidence of reliability based on Cronbach’s alpha of 0.88 and intraclass correlation coefficient of 0.96 (Suh, 2004); the Turkish version CMAI-T with evidence of reliability based on Cronbach’s alpha ranging from 0.80 to 0.87 for three factors (Altunöz et al., 2015; Ozel-Kizil et al., 2010); the Chinese version CCMAI with evidence of reliability based on Cronbach’s alpha of 0.75 and correlation coefficient of 0.85 (p < .001) (Choy et al., 2001); and the Norwegian version Brief Agitation Rating Scale (BARS) with evidence of reliability based on Cronbach’s alpha of 0.76 and spearman’s rho of 0.64 (p < .001) (Sommer et al., 2010).

Both long and short versions of CMAI are available for use in long-term care settings. The long version consists of 29 indicators (see Table 1) of agitated behaviors among older adults each of which are rated on a 7-point Likert scale (1: never; 2: less than once a week; 3: once or twice a week; 4: several times a week; 5: once or twice a day; 6: several times a day; and 7: several times an hour) based on the caregiver’s recall of resident behavior over the past 2 weeks (Cohen-Mansfield, 1991). Prior testing provided evidence of reliability of the long-form CMAI based on high (α = .86 to 0.91) internal consistency for the overall scale and good inter-rater agreement rate (based on 0- or 1-point discrepancy) ranging from .88 to .92 (Cohen-Mansfield, 1991; Finkel et al., 1992; Rabinowitz et al., 2005). Convergent validity was based on the significant relationships between CMAI and other measures of BPSD (Finkel et al., 1992), resident quality of life (Livingston et al., 2017) and a significant change in agitation following an intervention (Ballard et al., 2018). Subscales within the long version of the CMAI have been most commonly noted to include three factors: aggressive behavior, physically nonaggressive behavior, and verbally agitated behavior in nursing home settings (Altunöz et al., 2015; Choy et al., 2001; Cohen-Mansfield et al., 1989; de Jonghe & Kat, 1996; Zuidema et al., 2006). Each of these factors have evidence of acceptable to good internal consistency with Cronbach’s alpha close to or greater than .7 (Choy et al., 2001; Rabinowitz et al., 2005).

TABLE 1.

Items in Order of the Factor Structure of Long-Form CMAI and Short-Form CMAI

Factor structure of long-form CMAI based on existing research		Factor structure considered for short-form CMAI based on factor structure of long-form CMAI
Factor I—Aggressive behavior		Factor I—Aggressive behavior
3.	Spitting (include at meals)a	1.	Cursing or verbal aggression
4.	Cursing or verbal aggression	2.	Hitting (including self), kicking, pushing, biting, scratching, aggressive spitting (include at meals)
7.	Hitting (including self)
8.	Kicking
9.	Grabbing onto people
10.	Pushing	3.	Grabbing onto people
11.	Throwing thingsa	14.	Screaming
13.	Screaminga
14.	Bitinga
15.	Scratchinga
21.	Hurt self or other (cigarette, hot water etc.)a
25.	Tearing things or destroying propertya
Factor II—Physically nonaggressive behavior		Factor II—Physically nonaggressive behavior
1.	Pace, aimless wandering	4.	Other aggressive behaviors or self-abuse including intentional falling, making verbal or physical sexual advances, eating/drinking/chewing inappropriate substances, Hurts self of other with inappropriate substance
2.	Inappropriate dress or disrobinga
6.	Repetitive sentences or questions
16.	Trying to get to a different place (e.g., out of the room, building)
22.	Handling things inappropriately
26.	Performing repetitious mannerismsa
20.	Eating/drinking inappropriate substancesa	5.	Pace, aimless wandering, trying to get to a different place (e.g., out of the room/building)
27.	Making verbal sexual advancesa
29.	General restlessnessa
		6.	General restlessness, performing repetitious mannerisms, tapping, strange movements
		7.	Inappropriate dress or disrobing
		8.	Handling things inappropriately
		13.	Hiding things, hoarding things
Factor III—Verbally agitated behavior		Factor III—Verbally agitated behavior
5.	Constant unwarranted request for attention or help	9.	Constant request for attention or help
12.	Strange noises (weird laughter or crying)a	10.	Repetitive sentences, calls, questions, or words
18.	Complaining
19.	Negativism	11.	Complaining, negativism, refusal to follow directions
23.	Hiding thingsb
24.	Hoarding thingsb	12.	Strange noises (weird laughter or crying)
17.	Intentional fallingc
28.	Making physical sexual advancesc

Note. CMAI = Cohen-Mansfield Agitation Inventory.

^aItems with mixed findings, excluded in some and reported in others due to low frequency (<10% or <5%), failure to load on any factors, or due to weak factor loadings (i.e., <0.40).

^bItems loaded in separate factor (Factor IV—Hiding/Hoarding) in some studies.

^cItems excluded across studies due to low frequency (<10% or <5%).

The short-form CMAI, based off of the factor structure of long-form CMAI, includes 14 items (see Table 1) rated on a reduced 5-point Likert scale (1: never; 2: less than once a week; 3: once or several times a week; 4: once or several times a day; 5: a few times an hour or continuous for half an hour or more) based on caregiver interview (Werner et al., 1994). This condensed scale is less time consuming to complete. While existing research has provided evidence of the inter-rater reliability of short-form CMAI based on 81.8%–92.3% agreement between two raters, there has been less psychometric testing of the short-form CMAI particularly with regard to the factor structure of this shorter measure. There is a need to assess validity and confirm factor structure of the short-form CMAI which offers the assessment of agitation with condensed items and reduced response scale.

The purpose of this study was to test the reliability and validity of the 14-item CMAI based on the three-factor structure. The three factors include aggressive behavior, physically nonaggressive behavior, and verbally agitated behavior. The factors and item selection for each of these factors (see Table 1) was based on existing work (Altunöz et al., 2015; Choy et al., 2001; Cohen-Mansfield et al., 1989; de Jonghe & Kat, 1996; Rabinowitz et al., 2005; Schreiner et al., 2000; Suh, 2004; Zuidema et al., 2006). Specifically, it was hypothesized that consistent with the long-form CMAI, the short-form CMAI would have three factors with acceptable reliability. In addition, it was hypothesized that there would be no difference in factor structure and factor means across gender.

METHODS

Design

This was a cross-sectional analysis utilizing baseline data from an ongoing cluster randomized trial entitled, “Testing the Implementation of EIT-4-BPSD.” Briefly, this trial focuses on person-centered management of the BPSD in nursing homes through the implementation of the evidence-based intervention known as Evidence Integration Triangle for Behavioral and Psychological Symptoms of Dementia (EIT-4-BPSD). Study protocol and the details of the intervention have been previously published (Resnick et al., 2018). The study obtained approval from a University-based Institutional Review Board.

Sample.

The sample included 553 residents from 55 nursing homes. Nursing homes were eligible to participate if they were in Maryland or Pennsylvania; had 100 or more beds; were able to access email and websites via electronic devices like phone, tablet, or computer; agreed to actively partner with the research team in an initiative to change practice at their institution; and were able to identify a nursing staff as an internal champion to assist the research team with the implementation process (Resnick et al., 2018).

The residents from the participating facilities were recruited upon meeting the following six criteria: (a) residing at the facility at the time of recruitment; (b) 55 years or older; (c) exhibited at least one BPSD in the past month; (d) had evidence of cognitive impairment as indicated by Brief Interview of Mental Status (BIMS); (e) not enrolled in hospice; and (f) not in the facility for short-stay rehabilitation care. Upon receiving the list of eligible residents from a designated staff member, residents were approached for recruitment by EIT-4-BPSD staff and research evaluators. The resident ability to consent was evaluated using the Evaluation to Sign Consent (ESC) form (Resnick et al., 2007). If the resident was deemed unable to consent based on ESC, assent was obtained from the resident and consent for their participation was obtained from the Legally Authorized Representative (LAR). A total of 1,100 residents were approached and 572 (consent rate 52%) were consented. Forty-three (4%) of the 1,100 approached were noncommunicative, did not understand English, died, or were transferred before they could be consented, 156 (14%) refused to assent or consent to participate, 221 (20%) LARs were unavailable, and 90 (8%) LARs refused to consent. Of those that consented, 19 residents were not eligible as they had a BIMS score greater than 12. There were 11 residents who were ineligible as they were on Hospice and 7 were ineligible as they were younger than 55. Of the 572 that were consented, 553 residents were enrolled into the study. Of these, there were 17 individuals that died, were hospitalized, or were transferred out of the facility prior to the collection of baseline data.

Data Collection.

Data collection for EIT-4-BPSD was completed by trained research evaluators with prior experience working with residents with cognitive impairment and their caregivers in long-term care. The research evaluators collected data at baseline, 4, and 12 months although only baseline data was used for these analyses.

Measures

Descriptive resident variables included age, gender, race, marital status, and assessment of cognitive status. Demographic characteristics were obtained from resident charts/electronic records at the facility. Cognition was assessed using BIMS, a short cognitive screener with three-item recall and orientation questions. Scores on the BIMS range from 0 to 15 indicating severe impairment (0–7), moderate impairment (8–12), and cognitively intact (13–15) status of the residents (Chodosh et al., 2008; Saliba et al., 2012). Prior research has provided the evidence of reliability and validity of BIMS (Chodosh et al., 2008; Mansbach et al., 2014).

Measurement of Agitation.

Agitation was assessed using the short-form, that is, 14-item CMAI described above. The 14-item CMAI is a caregiver rated questionnaire using a 5-point Likert scale to rate the frequency of disturbing behaviors in dementia such as verbal/physical aggression, general restlessness, strange noises, and so on. Scores range from 14 to 70 with lower scores suggesting less frequent agitated behaviors. It has an evidence of good reliability with inter-rater agreement of .82 (Cohen-Mansfield, 1991). Validity was based on the decrease in agitation following an intervention (Resnick et al., 2016).

Data Analysis

Statistical analyses were completed using Statistical Package for the Social Sciences, SPSS version 26.0, and Mplus version 8 (Muthén & Muthén, 2017). First, SPSS was used for data check (e.g., normality, missing) and descriptive statistics. Data was missing on approximately 7% of the participants. Analysis was limited to those with complete data on CMAI. Following data check and descriptive statistics, Mplus 8 was used next to test the factor structure of short-form CMAI with weighted least square mean and variance adjusted (WLSMV) estimators for the categorical and ordinal items since the item responses on CMAI were ordinal. Specifically, the three-factor structure of short-form CMAI was tested using confirmatory factor analysis (CFA). Model fit was estimated using Chi-square/degrees of freedom (χ²/df), comparative fix index (CFI), Tucker–Lewis index (TLI), Steigers root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). The accepted cutoff values for both CFI and TLI are close to 0.95 and higher values are better (Hu & Bentler, 1998). For RMSEA and SRMR, the acceptable thresholds are less than 0.06 and 0.08, respectively and lower values are better (Hu & Bentler, 1998). Cronbach’s alpha (α) was also assessed to determine the internal consistency, that is, reliability of the CMAI and each factor.

The invariance tests and examination of the latent mean differences were completed next. The study looked at configural, metric, and scalar invariance consecutively as recommended in the literature (Vandenberg & Lance, 2000) to determine if the factor structure is invariant by gender. The configural invariance test provides free estimation of factor loadings and intercepts; establishment of a configural invariance implies similar structure and conceptual framework of the measure across the groups. The metric invariance test is a weak invariance test where factor loadings are constrained to be the same across the groups; establishment of metric invariance implies that each group responded to the items in the same way. The scalar invariance test, where both factor loadings and intercepts are constrained to be the same across the groups, is stronger than configural and metric tests; scalar invariance allows the latent means between groups to be compared. The invariance tests were evaluated with model fit indices described above and, the assessment of change in CFI value at metric and scalar levels. A CFI change greater than 0.01 indicates a lack of the evidence of metric and scalar invariance and, therefore, a value less than 0.01 is desired (Cheung & Rensvold, 2002).

Once the measurement invariance testing was completed and scalar invariance was established the latent mean differences were examined for all three factors across gender. To examine the latent mean differences across gender, each of the factor means was constrained to zero for men and set to free estimation for women.

RESULTS

Sample Descriptive

A total of 553 residents were enrolled in the study. The mean age of the participants was 83.88 years (standard deviation [SD] = 10.44). The majority were White (n = 419, 76%) and female (n = 398, 72%). Almost half of the participants were widowed (n = 260, 47%), with some individuals reported as married (n = 99, 17.78%), never married (n = 97, 17.78%), and divorced (n = 57, 10.3%). The mean score on the BIMS was 4.31 (SD = 3.50) indicating that overall the participants had severe cognitive impairment. There was an evidence of low levels of agitation among the participants with the mean score of 21.02 (SD = 8.36).

Descriptive Analysis of Short-Form CMAI

The CMAI was completed with 536 (of553) participants. All 536 participants had complete response on all the items. Table 2 lists all 14 items of the scale and the mean score and SD for each item based on the total sample and by gender. As seen in Table 1, the mean score was less than 2 (range = 1–5) for each individual item that suggests that there was limited evidence of agitation.

TABLE 2.

Items From Short-Form CMAI, Mean (SD)

CMAI items	Mean (SD)
	Overall (N = 536)	Men (n = 149)	Women (n = 387)
1. Cursing or verbal aggression	1.97 (1.25)	2.0 (1.29)	1.96 (1.23)
2. Hitting (including self), kicking, pushing, biting, scratching, aggressive spitting (include at meals)	1.54 (1.006)	1.50 (0.96)	1.56 (1.02)
3. Grabbing onto people, throwing things, tearing things, or destroying property	1.40 (0.94)	1.42 (0.98)	1.39 (0.93)
4. Other aggressive behaviors or self-abuse including intentional falling, making verbal or physical sexual advances, eating/drinking/chewing inappropriate substances, hurts self of other with inappropriate substance	1.18 (0.67)	1.32 (0.89)	1.13 (0.55)
5. Pace, aimless wandering, trying to get to a different place (e.g., out of the room/building)	1.66 (1.23)	1.66 (1.27)	1.66 (1.22)
6. General restlessness, performing repetitious mannerisms, tapping, strange movements	1.46 (1.11)	1.43 (1.07)	1.48 (1.13)
7. Inappropriate dress or disrobing	1.21 (0.64)	1.27 (0.68)	1.18 (0.63)
8. Handling things inappropriately	1.19 (0.67)	1.18 (0.65)	1.19 (0.68)
9. Constant request for attention or help	1.66 (1.26)	1.60 (1.22)	1.69 (1.28)
10. Repetitive sentences, calls, questions, or words	1.75 (1.32)	1.61 (1.22)	1.80 (1.36)
11. Complaining, negativism, refusal to follow directions	1.80 (1.22)	1.85 (1.24)	1.78 (1.21)
12. Strange noises (weird laughter or crying)	1.27 (1.17)	1.22 (0.72)	1.29 (0.87)
13. Hiding things, hoarding things	1.31 (0.91)	1.21 (0.72)	1.35 (0.98)
14. Screaming	1.62 (1.17)	1.54 (1.06)	1.65 (1.21)

Note. CMAI = Cohen-Mansfield Agitation Inventory; SD = standard deviation.

Internal Structure

Factor Structure of the Short-Form CMAI or CMAI-14.

Table 3 shows the model results of the three-factor solution of short-form CMAI with respective fit indices. The χ²/df ratio was 2.43 providing evidence of good model fit to the data. There was also an evidence of adequate model fit based on other fit indices. As shown in Table 3, the CFI (0.933) and TLI (0.917) were close to 0.95. Similarly, the RMSEA (0.052) was within the acceptable threshold of <0.06 and the SRMR (0.064) was also within the acceptable threshold of <0.08.

TABLE 3.

Model Results of the Three-Factor Solution of Short-Form CMAI With Fit Indices

Model	χ 2	df	CFI	TLI	RMSEA (90% CI)	SRMR
3-factor model	179.931*	74	0.933	0.917	0.052 (0.042, 0.061)	0.064

Note. N = 536. χ² = Chi-square; CFI = comparative fix index; CMAI = Cohen-Mansfield Agitation Inventory; CI = confidence interval; df = degrees of freedom; RMSEA = Steigers root mean square error of approximation; SRMR = standardized root mean square residual; Tucker–Lewis Index.

^*p < .001.

The standardized factor loadings for each item are presented in Table 4. All standardized factor loadings were close to or greater than 0.60 indicating that the item was significantly associated with the factor as hypothesized.

TABLE 4.

Three-Factor CFA Solution of Short-Form CMAI (α = .841) With Standardized Factor Loadings

Factors	CMAI Items	Item Results
		B	SE	p	β	R 2
Aggressive behavior (α = .794)	1. Cursing or verbal aggression	1.000	0.000	-	0.813	0.661
	2. Hitting (including self), kicking, pushing, biting, scratching, aggressive spitting (include at meals)	1.024	0.035	.000	0.832	0.692
	3. Grabbing onto people, throwing things, tearing things, or destroying property	1.129	0.036	.000	0.918	0.842
	14. Screaming	0.967	0.041	.000	0.786	0.617
Physically nonaggressive behavior (α = .617)	4. Other aggressive behaviors or self-abuse including intentional falling, making verbal or physical sexual advances, eating/drinking/chewing inappropriate substances, hurts self of other with inappropriate substance	1.000	0.000	-	0.683	0.467
	5. Pace, aimless wandering, trying to get to a different place (e.g., out of the room/building)	0.833	0.112	.000	0.569	0.324
	6. General restlessness, performing repetitious mannerisms, tapping, strange movements	1.205	0.119	.000	0.823	0.678
	7. Inappropriate dress or disrobing	0.816	0.111	.000	0.558	0.311
	8. Handling things inappropriately	1.097	0.122	.000	0.749	0.562
	13. Hiding things, hoarding things	0.736	0.110	.000	0.503	0.253
Verbally agitated behavior (α = .718)	9. Constant request for attention or help	1.000	0.000	-	0.762	0.581
	10. Repetitive sentences, calls, questions, or words	1.033	0.060	.000	0.788	0.620
	11. Complaining, negativism, refusal to follow directions	1.089	0.071	.000	0.830	0.689
	12. Strange noises (weird laughter or crying)	0.885	0.099	.000	0.674	0.455

Note. N = 536; α = Cronbach’s alpha; B = unstandardized factor loadings; CMAI = Cohen-Mansfield Agitation Inventory; SE = standard error of the estimate; p = p-value; β = standardized factor loadings; R² = variance.

All three factors were correlated with each other supporting the construct validity of the shortened CMAI. The strongest correlation was between aggressive behavior and verbally agitated behavior (r = .802; p < .0001) followed by aggressive behavior and physically nonaggressive behavior (r = .780; p < .0001) and, physically nonaggressive behavior and verbally agitated behavior (r = .744; p < .0001).

Measurement Invariance across Gender.

The findings from the measurement invariance test across the two groups (i.e., men and women) are presented in Table 5. The configural invariance test demonstrated an acceptable fit with CFI (0.937) and TLI (0.922) close to 0.95 and RMSEA (0.059) less than 0.06 suggesting similar structure and conceptual framework for the short-form CMAI across genders. The metric invariance test also demonstrated an acceptable fit suggesting that both men and women respond to short-form CMAI in similar manner. The fit indices slightly improved for metric model with slightly greater CFI (0.940) and TLI (0.931) values and comparatively lower RMSEA (0.055) than the configural model. Additionally, the change of CFI between configural and metric models was 0.003 which is within the threshold of 0.01 supporting metric invariance across the two groups. Finally, the scalar invariance test also demonstrated an adequate fit with improved model indices suggesting that the item intercepts are equivalent across gender and hence, latent factor means of the short-form CMAI can be meaningfully compared between men and women. Despite a slight increase in the chi-square value, CFI, TLI, and RMSEA significantly improved for the scalar model with CFI (0.944) and TLI (0.948) most close to 0.95 and RMSEA (0.048) less than 0.06. The CFI change between the scalar and metric models, 0.004 also stayed below. 01 supporting scalar invariance in the two groups. As shown in Table 5, the chi-square difference tests comparing the configural, metric, and scalar invariance models were also nonsignificant.

TABLE 5.

Measurement Invariance Tests of Short-Form CMAI by Gender and Comparison of Measurement Invariance Models

Model	χ 2	df	CFI	TLI	RMSEA (90% CI)	SRMR	χ2 diff	p < .05
Configural invariance	285.439*	148	0.937	0.922	0.059 (0.049, 0.069)	0.081	Scalar vs. Configural	No
Metric invariance	289.545*	159	0.940	0.931	0.055 (0.045, 0.065)	0.082	Metric vs. Configural	No
Scalar invariance	318.651*	197	0.944	0.948	0.048 (0.038, 0.057)	0.085	Scalar vs. Metric	No

Note. N = 536 (Male = 149; Female = 387); χ² = Chi-square; χ² diff = chi-square different test comparison models; CFI = comparative dix index; CI = confidence interval; df = degrees of freedom; RMSEA = Steigers root mean square error of approximation; SRMR = standardized root mean square residual; TLI = Tucker–Lewis Index.

^*p < .001.

Latent Factor Means Differences.

The comparison of the latent factor means between men and women in the current study showed that there was no significant difference in aggressive (z = .084; p = .933), physically nonaggressive (z =.291; p =.771), and verbally agitated (z = −.617; p = .537) behaviors between the two groups.

Reliability

Internal Consistency of the Factors and Reliability Based on R2.

The internal consistency of the items based on Cronbach’s alpha provided evidence of internal consistency for the short-form CMAI. As shown in Table 4, the Cronbach’s alpha for aggressive behavior, physically nonaggressive behavior, and verbally agitated behavior was 0.794, 0.617, and 0.718 respectively suggesting acceptable internal consistency (Taber, 2018; Ursachi et al., 2015); the overall scale also had a good internal consistency with Cronbach’s alpha of 0.841.

There was also some evidence of the reliability of items loading on respective factors based on R² (Bollen, 1989) as the majority of all the items had R² values of greater than .5. As shown in Table 4, for aggressive behavior, the R² of the items ranged between 0.6 and 0.9. Likewise, for verbally agitated behavior, the R² ranged between 0.5 and 0.7 for most items; only “strange noises” had an R² less than 0.5 suggesting greater error and a slightly lower reliability of this item. For physically nonaggressive behavior, the R² of the items ranged between 0.3 and 0.7; the three items with low reliability included “pace, aimless wandering, trying to get to a different place,” “inappropriate dress or disrobing,” and “hiding things, hoarding things.”

DISCUSSION

The findings from this study provide some additional support for the short-form CMAI or CMAI-14. There was sufficient evidence of internal consistency, the majority of the items had R² values of greater than .5 suggesting little error, and the model fit the data as hypothesized. In addition, there was evidence of measurement invariance such that the short-form CMAI with a three-factor solution was invariant across genders.

The three-factor solution of short-form CMAI showing that agitation among the residents with cognitive impairment could be evaluated in three behavioral dimensions: aggressive behavior, physically nonaggressive behavior, and verbally agitated behavior is in accordance with existing research. Most studies focused on a three-factor structure for CMAI (Altunöz et al., 2015; Choy et al., 2001; Cohen-Mansfield et al., 1989; de Jonghe & Kat, 1996; Zuidema et al., 2006) in nursing home settings. There were a few studies, however, that reported a four-factor structure with hiding/hoarding as the additional fourth factor (Rabinowitz et al., 2005; Suh, 2004). We considered the three-factor structure over the four-factor structure because the fourth factor hiding/hoarding would have included only one item. Consistent with our findings, others have noted that “hiding things” and “hoarding things” loaded significantly on the physically nonaggressive behavior subscale in exploratory factor analysis of long-form CMAI (Altunöz et al., 2015; de Jonghe & Kat, 1996; Zuidema et al., 2006).

In further support of the three-factor measure additional internal consistency of the three factors was supported in this study. The alpha coefficient was close to or greater than .7 indicating acceptable internal consistency for aggressive, physically nonaggressive, and verbally agitated behaviors. This is comparable to the findings from prior work examining the factor structure of long-form CMAI where, the alpha coefficient ranged from 0.81 to 0.86 for aggressive, 0.62 to 0.83 for physically nonaggressive, and 0.59 to 0.80 for verbally agitated behaviors in different cultures (Altunöz et al., 2015; Choy et al., 2001; Rabinowitz et al., 2005).

In examining the reliability of the items loading on the factors, low reliability for some items was noted on the physically nonaggressive behaviors subscale. Particularly, “pace, aimless wandering, trying to get to a different place,” “inappropriate dress or disrobing,” and “hiding things, hoarding things” had R² close to 0.3 suggesting higher amount of error and low item reliability. As each behavior is quite different in these individual items, it is likely that the error is due to lack of clarity of items creating confusion in completion of the scale. For example, inappropriate dressing is different than disrobing. Also, pace or aimless wandering seems different than trying to get to a different place; it might even reflect things other than agitation such s boredom or anxiety. Rewording of these items or separating out the individual items might help to improve item reliability. It is also possible that some of the items are not relevant to physically nonaggressive behaviors. Specifically, “hiding things, hoarding things” might not be relevant to physically nonaggressive behaviors and could be separated out. This item had been described as a fourth factor in some studies previously with factor loading close to or greater than 0.8 when separated out as fourth factor with two items “hiding things” and “hoarding things” (Rabinowitz et al., 2005; Suh, 2004).

In existing research, gender was found to predict agitation (Vance et al., 2003) and this might have prompted the prior researchers to examine gender differences in agitation. For example, a study examined gender differences in neuropsychiatric symptoms and reported that women had comparatively broader range of CMAI symptoms compared to men (Tao et al., 2018). Additionally, women were more likely to exhibit certain behaviors (e.g., pacing or aimless wandering, complaining, refusal to follow directions, and hiding/hoarding things) than men. Another study compared the clustering of neuropsychiatric symptoms in a sample of nursing home residents by gender and found predominantly identical clusters between men and women (Ho et al., 2016). In examining the gender differences, it was typically assumed that the CMAI operates similarly across men and women. It is important that the examination of aforementioned gender differences be based on some evidence supporting that the measure is appropriate for use for males and females and, any differences in scores across genders is real and not because an item is most likely to be endorsed for males and least likely to be endorsed for females or vice versa. The findings from in variance testing in the current study suggests that there is no difference in the structure or meaning of short-form CMAI across genders allowing for a meaningful and accurate comparison between the two groups using the shortened measure.

In comparing the latent factor means across gender, the current study did not find significant differences in aggressive, physically nonaggressive, and verbally agitated behaviors between men and women. This was comparable to the findings in existing research in which there was no significant differences noted in behaviors between men and women (Lövheim et al., 2009; Tao et al., 2018). Future research should continue to explore gender differences in behavior to better understand how the different dimensions of agitation may vary between men and women. In doing so it would help clinicians and scholars to target specific behavioral dimensions relevant for each group while formulating care plans, and while designing and delivering interventions.

LIMITATIONS

This study was limited by the virtue of being a secondary data analysis. Consequently, the findings are limited by sample selectivity meaning that the findings are more reflective of the cognitively impaired population in selected nursing home settings and cannot be generalized to all long-term care residents. Further research is needed to confirm these findings with a diverse population of residents. Although the model fit indices and alpha coefficients were adequate, some values were lower than desired. Since this is one of the first studies to examine the factor structure of short-form CMAI and the measurement invariance by gender, other studies could corroborate these findings in similar or diverse population. Also, this study only included baseline data collected at a single point in time and therefore did not provide information on changes over time. Future studies could assess longitudinal invariance to ensure that the repeated measurement of agitation with short-form CMAI represent the same construct in the same metric over time. Additionally, the participants in this study had severe cognitive impairment on average and hence, invariance by the level of cognitive impairment was not assessed. Future studies could consider assessment of measurement invariance of short-form CMAI across the levels of cognitive impairment using cognitively diverse population.

CONCLUSION

The short-form CMAI used extensively in long-term care research to evaluate one of the common BPSD among older adults, agitation has validity evidence based on CFA and the measurement invariance by gender. The measure can be used to facilitate assessment of agitation among older adults and the assessment of change in agitation following a treatment plan or behavioral intervention. Additionally, the assessment of measurement invariance by gender provides support that the measure can be used for both males and females.

Relevance to Nursing Practice, Education, or Research

There has been less psychometric testing of the short-form CMAI particularly regarding the factor structure of this shortened measure. Findings from the current study provide evidence of validity of short-form CMAI and suggest that it can be used reliably to measure agitation. In addition, the findings also suggest that the shortened scale is appropriate for use across genders. With these findings, this study supports the assessment of agitated behaviors and the differences in agitated behaviors among males and females in long-term care settings using the short-form CMAI. Additionally, this study also raises important questions focused on reevaluation of some items with low reliability in this shortened measure and, the examination of measurement invariance of short-form CMAI across the levels of cognitive impairment.