Validation of the clutter image rating in older adults with hoarding disorder

Abstract

Background

The Clutter Image Rating (CIR) was created to meet a gap in the research on compulsive hoarding: how to ascertain clutter level in an individual’s home without a home visit, as not all clinicians have the ability to conduct a home visit. The CIR has proven itself to be both reliable and valid for use in adults with compulsive hoarding symptoms. However, there is currently a dearth of information on performance of the CIR in older adults diagnosed with hoarding disorder (HD). Because older adults have increased medical issues, including fall risks, evaluating the level of clutter in the house is especially critical in geriatric populations.

Method

The current study was an investigation of the reliability and validity of the CIR in assessing late life HD. The internal consistency, convergent and divergent validity, and norms of the CIR were investigated in a large geriatric HD sample and compared with a midlife sample of individuals with HD. Criterion validity of the CIR was investigated through the comparison of participant ratings conducted in the clinic and clinician ratings conducted in the home.

Results

The current study found similar levels of reliability and validity in a late life sample as in previous studies conducted in mid-life adults.

Conclusions

Unlike previous studies, the current study did not find a significant relationship between the CIR and the non-clutter related subscales of the Savings Inventory-Revised. The CIR appears to be both reliable and valid for assessing clutter levels in older adults diagnosed with HD.

Introduction

Hoarding Disorder (HD) is a recent addition to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013); however, researchers have chronicled the existence of individuals suffering from hoarding symptoms independent of the context of other disorders for several decades (e.g., Frost and Hartl, 1996). Because HD has only recently been codified, much of the literature on hoarding involves samples in which the inclusion criteria used was compulsive hoarding (typically as defined by Frost and Hartl, 1996) or hoarding symptoms with or without other OCD symptoms. Where appropriate, we will refer to each study sample (HD, compulsive hoarding, hoarding behaviors, or hoarding symptoms) as defined by the authors.

Hoarding disorder has been associated with increased health and safety risks in geriatric samples, including multiple medical and psychiatric co-morbidities (Ayers and Dozier, 2014) and impairment in activities of daily living (Ayers et al., 2012). Finding assessment tools that can accurately assess hoarding symptoms is important for this particularly vulnerable HD age group.

Hoarding, both in its present diagnostic criteria in the DSM-5 and in its original definition as compulsive hoarding, involves an excessive amount of clutter in the home such that living spaces can no longer be utilized for their intended purposes. The amount of clutter in a home is directly associated with the functional impairment experienced by older adults with HD (Ayers et al., 2012), and thus provides a powerful indicator of the consequences that may be experienced by afflicted individuals.

The Clutter Image Rating (CIR; Frost et al., 2008) was created to meet a gap in the research on compulsive hoarding: how to ascertain the amount of clutter in an individual’s home. Home visits are often difficult to conduct due to travel time, institutional rules, cost of travel, safety concerns, and patients unwilling to allow others into the cluttered home. The CIR consists of nine photographs of increasing clutter severity for the three primary rooms of a house (living room, kitchen, and bedroom), for a total of 27 pictures of clutter. Individuals are instructed to select the picture from each set that best matches the appearance of the corresponding room in their home. The three scores (one for each room) are averaged to create a total score.

As discussed by Frost and colleagues (2008), the use of a pictorial scale for assessing proportional clutter volume eliminates the issue of under- or over- reporting clutter based on false associations with the word, which can limit the overall construct validity of a measure. For example, a person who would verbally report “all or nearly all” or their home is “difficult to walk through because of clutter” on the Savings Inventory-Revised (SIR; Frost et al., 2004) might be over-reporting due to a mistaken perception of what qualifies as such clutter. Alternatively, a low-insight individual who perceives his or her house as easy to walk though because they have become accustomed to the pathways through the clutter might under-report his or her clutter levels. When given a visual depiction of what different levels of clutter look like, individuals with low levels of insight into the severity of their hoarding symptoms are provided with a visual comparison to optimize the accuracy of their self-reported clutter.

The CIR has proven itself to be both reliable and valid in mid-life samples. Test-retest reliability has been established through wait-list controls in treatment outcome studies of mid-life individuals (Frost et al., 2012). Internal consistency has been demonstrated repeatedly to be large in mid-life samples (αs ranging from 0.77 to 0.91; Dimauro et al., 2013; Tolin et al., 2010; Nordsletten et al., 2013).

Concurrent criterion validity in mid-life samples has been demonstrated by strong correlations between clinician and participant ratings of the participant’s home (Frost et al., 2008; Tolin et al., 2010; DiMauro et al., 2013), suggesting that participants’ ratings on the CIR are able to accurately capture participants’ home clutter levels without a clinician home visit. The CIR has been proven to discriminate between mid-life individuals with and without compulsive hoarding (Grisham et al., 2010; Gordon et al., 2013; Nordsletten et al., 2013; Drury et al., 2014). CIR scores for mid-life individuals with compulsive hoarding or HD range from 3.64 (SD: 1.55; Tolin et al., 2010) to 4.5 (SD: 1.5; Frost et al., 2011). CIR scores for mid-life individuals without compulsive hoarding range from 1.2 (SD: 0.3; Gordon et al., 2013) to 1.46 (SD: 0.71; Tolin et al., 2010).

Although the evidence cited above provides an adequate validation for the use of the CIR in mid-life samples, symptoms of hoarding may change or increase with age (Grisham et al., 2006; Ayers et al., 2010), underlying the importance to explore the performance of the CIR in geriatric HD samples in addition to mid-life samples. Because older adults with HD may have a different symptom presentation than midlife adults with HD, hoarding severity measures need to be evaluated in geriatric samples to determine if they have comparable psychometric properties. The CIR may be especially susceptible to age-related changes in its properties due to its construction: the images used for each room were determined to be “equidistant” in clutter level by a small sample of college undergraduates (Frost et al., 2008), whose perceptions of clutter may differ from older adults.

The CIR has been proven to discriminate between small samples of older adults with and without compulsive hoarding (Ayers et al., 2012). There is also some initial evidence of the convergent validity of the CIR in geriatric samples: Ayers and colleagues (2012) found that the CIR was a significant predictor of the functional impairment caused by hoarding at levels consistent with results from middle-aged samples (Frost et al., 2013).

There is precedence for evaluating psychiatric assessments for use in geriatric samples (Beck et al., 1995; Kabacoff et al., 1997; Segal et al., 2008); however, to date, there have been no large-scale examinations of the psychometric properties of hoarding severity measures in older adults meeting DSM-5 criteria for HD. Validation of the CIR for use in late life is especially critical due to the brief nature of the measure. Because administration of the CIR takes under 5 minutes, the CIR has the potential to function as a brief screening tool for HD in older adults, who may feel burdened by long self-report or clinician-administered assessments.

The aim of the current study is to conduct the first systematic investigation of the validity of the CIR as an assessment of hoarding severity in older adults with HD. We hypothesize that the CIR will demonstrate similar levels of reliability and validity as have been previously witnessed in samples of mid-life adults. Understanding the psychometric properties of the CIR in geriatric HD samples increases the efficacy of assessment of an important aspect of HD in older adults.

Methods

Participants

The current study looked at baseline CIR scores for 81 older adults and 24 mid-life adults with HD recruited from the community between July 2008 and April 2014 for either two individual intervention studies for late-life HD (n = 73) or for a group intervention study for HD that recruited participants from across the lifespan (n = 32). Participants were recruited primarily through flyers posted throughout San Diego County. Additional recruitment efforts included postings on craigslist (a free classified advertisement website) and referrals from healthcare providers.

All study protocols were approved by the Institutional Review Board of the University of California, San Diego and by the VA San Diego Healthcare System. Assessments were conducted at the VA San Diego Healthcare System. No monetary compensation was provided for participation, and all participants provided written informed consent. In addition to data from participants’ baseline assessments, a subset of the clinician ratings of the CIR during home-visits were included to examine criterion validity in the geriatric sample.

Participants were required to meet the proposed DSM-5 HD symptom severity criteria, determined by a consensus diagnosis supervised by a licensed clinical psychologist using a clinician-administered semi-structured interview and self-report measures. In line with DSM-5 criteria, participants in the current study were required to have difficulty discarding possessions, urges to save items and distress when forced to discard objects, impairment in use of living areas of the home due to clutter levels, and clinically significant distress or functional impairment due to the hoarding symptoms. Further, participants were excluded if it was determined that the hoarding symptoms were caused by a medical condition or were the result of a separate psychiatric diagnosis. Thus, participants who were identified as having comorbid OCD were required to have HD as the primary diagnosis and for their hoarding symptoms to not be related to their OCD.

Measures

Hoarding symptom severity was assessed using the CIR, the SI-R and the UCLA Hoarding Severity Scale (UHSS; Saxena et al., 2007). Clinician ratings of the CIR were available for 35 of the geriatric participants. The home visits during which clinician ratings of the CIR were conducted occurred within two weeks of participants’ baseline assessments.

The SI-R is 23-item self-report Likert-type scale of hoarding severity that measures hoarding symptoms on three dimensions: difficulty discarding, excessive acquisition, and clutter. Items on the SI-R are summed for a total possible score of 92; scores over 40 on the SI-R total score are indicative of clinically severe hoarding symptoms (Frost et al., 2004).

The UHSS is a 10-item semi-structured, clinician-administered assessment of hoarding symptom severity. The UHSS assesses for the presence and severity of clutter, urges to save items, excessive acquisition, difficulty discarding, social and occupational impairment„ perfectionism, indecisiveness, and procrastination. Items on the UHSS are summed for a total possible score of 40, with scores greater than 20 indicated clinically severe levels of hoarding symptoms.

Anxiety and depressive symptoms were assessed using the Hospital Anxiety and Depression Scale (HADS; Zigmond and Snaith, 1983), a 14-item self-report measure of mental health. Seven of the items are summed to create an anxiety subscale and seven of the items are summed to generate a depression subscale. Two of the items on the HADS depression subscale are reverse scored. Scores of 0–7 on either of the HADS subscales indicate no clinically severe mental health symptoms; scores of 8–10 indicate borderline severity; and scores of 11 or higher indicate clinically severe symptoms.

Impairment from hoarding behaviors was measured using the Activities of Daily Living in Hoarding (ADL-H; Frost et al., 2013), a 15-item self-report assessment. Items are rated from 1 (“can do easily”) to 5 (“unable to do”), and participants have the option of indicating that a particular item is “not applicable.” All items rated by the participant are averaged to create a mean score. The ADL-H has been found to discriminate between hoarding and non-hoarding samples (Frost et al., 2013).

Data analysis

All analyses were performed using Stata version 13.0 (StataCorp, 2013). The CIR was examined in the older adult sample (n = 81) for internal consistency, as measured by Cronbach’s α and inter-item correlations, and convergent validity, as measured by the relationship between the CIR, the SI-R, the UHSS, and the ADL-H. Divergent validity was assessed through an examination of the relationship between the CIR and the HADS Anxiety and Depression scales and the differential relationship between the CIR and the SI-R subscales. Concurrent criterion validity for the CIR was assessed by comparing participant ratings conducted in the clinic with clinician ratings assessed in participants’ homes. All analyses in the current study were run in both a late life and a midlife sample of adults with HD in order for comparisons between groups and possible age-related differences were examined for all variables, with the exception of the comparison between participant and clinician ratings. An alpha level of p < 0.05 was used to indicate significant findings.

Results

Demographics

The mean age of the late life sample was 68 (SD = 6.4, range 60–87) and the sample was mostly female (69%) and Caucasian (84%). The majority of older participants were high school graduates (mean years of education: 16.0, SD = 2.1, range: 10–20) and reported living alone (72% either divorced, separated, single, or widowed; 28% either married or living with partner). Scores on the CIR in the late life sample did not differ by gender (t (73) = 0.75, p = 0.23) and were not correlated with age (r = 0.05, p = 0.64); however, scores on the CIR were negatively correlated with years of education (r = −0.36, p < 0.01).

The mean age of the midlife sample was 52.5 (SD = 5.4, range 40–59) and had similar gender and ethnic breakdowns as the late life sample (75% female and 83% Caucasian). All midlife participants reported having completed at least 12 years of education (mean years of education: 15.2, SD = 2.2, range 12–20). The mean number of years of education did not differ significantly between the late life and the midlife samples (t (37.8) =1.49, p = 0.07). The majority of midlife participants reported living alone (71% either divorced, separated, single, or widowed; 29% either married or living with partner). Scores on the CIR in the midlife sample also did not differ by gender (t (22) = 0.63, p = 0.27), age (r = 0.21, p = 0.32) and were not correlated with years of education (r = 0.09, p = 0.67).

Scores on the CIR were not correlated with age for the full sample (r = 0.02, p = 0.81; age range 40–87).

Internal consistency

Internal reliability and descriptive statistics from both samples are provided for all variables in Table 1. The CIR demonstrated a high internal consistency in both samples (late life: α = 0.86; midlife: α = 0.84). There were significant correlations among all CIR items in the late life sample; however, in the midlife sample the Kitchen item was not significantly correlated with the Bedroom item (p > 0.05) (see Table 2). All items were significantly correlated with the CIR in both the late life sample (Living Room: r = 0.91, p < 0.0001; Kitchen: r = 0.88, p < 0.0001; Bedroom: r = 0.87, p < 0.0001) and in the midlife sample (Living Room: r = 0.91, p < 0.0001; Kitchen: r = 0.75, p < 0.0001; Bedroom: r = 0.77, p < 0.0001).

Table 1.

Descriptive statistics and age comparisons of all measures in adults with HD

		LATE LIFE ADULTS (AGED 60+; N = 81)		MIDLIFE ADULTS (AGED <60; N = 24)		T =
		α	MEAN (SD)	α	MEAN (SD)
CIR	CIR	0.86	4.0 (1.8)	0.84	4.3 (1.7)	0.60
	Living room	–	4.3 (2.1)	–	4.7 (1.9)	0.76
	Kitchen	–	3.6 (1.9)	–	3.5 (2.2)	−0.21
	Bedroom	–	4.1 (2.2)	–	4.6 (2.2)	0.94
SI-R	Total	0.9	57.4 (12.7)	0.9	59.7 (13.1)	0.74
	Clutter	0.92	24.5 (7.2)	0.93	24.2 (8.3)	−0.16
	Difficulty discarding	0.83	19.1 (4.3)	0.87	19.2 (4.4)	0.03
	Acquisition	0.81	13.8 (4.9)	0.88	16.3 (5.3)	2.05*
UHSS		0.80	27.5 (5.9)	0.81	30.42 (6.0)	2.07*
HADS	Anxiety	0.78	9.7 (4.1)	0.79	10.4 (4.3)	0.67
	Depression	0.86	7.8 (4.4)	0.89	8.8 (5.5)	0.83
ADL-H		0.91	2.1 (0.71)	0.95	2.1 (0.88)	0.34

^*p < .05

CIR = Clutter Image Rating; SI-R = Savings Inventory-Revised; UHSS = UCLA Hoarding Severity Scale; HADS = Hospital Anxiety and Depression Scale; ADL-H = Activities of Daily Living in Hoarding

Table 2.

CIR inter-item correlations in adults with HD

	LATE LIFE ADULTS (AGED 60+; N = 81)			MIDLIFE ADULTS (AGED <60; N = 24)
	LIVING ROOM	KITCHEN	BEDROOM	LIVINGROOM	KITCHEN	BEDROOM
CIR	0.91****	0.88****	0.87****	0.91****	0.75****	0.77****
Living room	–	–	–	–	–	–
Kitchen	0.76****	–	–	0.59**	–	–
Bedroom	0.67****	0.62****	–	0.63**	0.23	–

^*p < .05,

^**p < .01,

^***p < .001,

^****p < .0001

Convergent/divergent validity

There were no significant differences at the p < 0.05 level between the two samples for the CIR total or any of the items (see Table 1). There were also no significant differences at the p < 0.05 level between the two samples for the HADS Anxiety and Depression scales, the ADL-H, the SI-R Total, or the SI-R Difficulty Discarding and Clutter subscales (see Table 1). However, there was a significant mean difference between the two age samples on the SI-R Acquisition subscale (p < 0.05) and on the UHSS (p < 0.05), with mid life adults scoring significantly higher on both measures (see Table 1).

The CIR was significantly correlated with the SIR Total and the SI-R Clutter subscale in both the late life sample (SI-R Total: r = 0.47, p < 0.0001; SI-R Clutter: r = 0.68, p < .0001) and in the mid life sample (SI-R Total: r = 0.49, p < 0.05; SI-R Clutter: r = 0.50, p < 0.05). In both samples, the CIR was not significantly correlated with the SI-R Difficulty Discarding or Acquisition subscales at the p < 0.05 level (see Table 3). In the late life sample, all three rooms were significantly correlated with the SI-R Total (Living Room: r = 0.046, p < 0.0001; Kitchen: r = 0.34, p < 0.001; Bedroom: r = 0.46, p < 0.0001) and the SI-R Clutter subscale (Living Room: r = 0.66, p < 0.0001; Kitchen: r = 0.53, p < 0.0001; Bedroom: r = 0.53, p < 0.0001). The CIR items were not significantly correlated with the SI-R Difficulty Discarding or Acquisition subscales at the p < 0.05 level in the geriatric sample (see Table 3). In the midlife sample, the SI-R Total was significantly correlated with only the kitchen (r = 0.47, p < 0.05) and the SI-R clutter subscale was significantly correlated with only the Living Room (r = 0.41, p < 0.05) and the kitchen (r = 0.42, p < 0.05). In the late life sample, the CIR and all items were significantly correlated with the UHSS (CIR: r = 0.60, p < 0.0001; Living Room: r = 0.54, p < 0.0001; Kitchen: r = 0.48, p < 0.0001; Bedroom: r = 0.58, p < 0.0001). In the mid-life sample, the UHSS was significantly correlated with the CIR (r = 0.41, p < 0.05), but was not significantly correlated with any of the individual CIR items at the p < 0.05 level (see Table 3). The CIR and all items were significantly correlated with the ADL-H in both the late life sample (CIR: r = 0.62, p < 0.0001; Living Room: r = 0.58, p < 0.0001; Kitchen: r = 0.55, p < 0.0001; Bedroom: r = 0.51, p < 0.0001) and in the midlife sample (CIR: r = 0.72, p < 0.001; Living Room: r = 0.63, p < 0.01; Kitchen: r = 0.67, p < 0.001; Bedroom: r = 0.56, p < 0.01).

Table 3.

Correlations among the CIR and all other variables in adults with HD

		LATE LIFE ADULTS (AGED 60+; N = 81)					MIDLIFE ADULTS (AGED <60; N = 24)
		CIR	LIVING ROOM	KITCHEN	BEDROOM	CIR	LIVING ROOM	KITCHEN	BEDROOM
SI-R	Total	0.47****	0.46****	0.34**	0.46****	0.49*	0.29	0.47*	0.4
	Clutter	0.68****	0.66****	0.53****	0.62****	0.50*	0.41*	0.42*	0.37
	Difficulty discarding	0.14	0.12	0.1	0.15	0.11	−0.06	0.2	0.11
	Acquisition	0.11	0.12	0.03	0.15	0.33	0.13	0.34	0.31
UHSS		0.60****	0.54****	0.48****	0.58****	0.41*	0.29	0.37	0.32
HADS	Anxiety	0.19	0.06	0.18	0.26*	0.06	−0.09	0.1	0.11
	Depression	0.15	0.04	0.08	0.28*	0.41*	0.24	0.44*	0.31
ADL-H		0.62****	0.58****	0.55****	0.51****	0.72***	0.63**	0.67***	0.56**

^*p < .05,

^**p < .01,

^***p < .001,

^****p < .0001.

The CIR was not significantly correlated with either the HADS Anxiety Subscale in both samples (see Table 3). The HADS Depression Subscale was significantly correlated with the CIR in the midlife sample (r = 0.41, p < 0.05), but not in the late life sample (p > 0.05). The bedroom item was significantly correlated with both the HADS Anxiety (r = 0.26, p < 0.05) and Depression (r = 0.28, p < 0.05) subscales in the late life sample; however, in the midlife sample only the kitchen item was significantly correlated with the HADS Depression Subscale (r = 0.44, p < 0.05) and no items were significantly correlated with the HADS Anxiety Subscale at the p < 0.05 level (see Table 3).

Criterion validity

Geriatric participant ratings were significantly correlated with clinician ratings for the CIR (r = 0.54, p = .0008) and all items (Living Room: r = 0.60, p = 0.0001; Kitchen: r = 0.44, p = 0.008; Bedroom: r = 0.48, p = 0.0028).

Discussion

Excessive clutter is one of the core diagnostic criteria of hoarding disorder (American Psychiatric Association, 2013); therefore correct diagnosis of HD without a clinician home visit requires a measure of clutter that can be completed outside of a patient’s home. The CIR provides a standardized self-report method for assessing proportional clutter volume. The current study represents the largest investigation of the psychometric properties of the CIR in older adults meeting DSM-5 criteria for HD. The results indicate that the CIR has similar properties of reliability and validity in older adults as in mid-life adults, suggesting that the CIR is a valid and reliable tool for assessing clutter levels in older adults with HD.

The CIR demonstrated adequate convergent reliability in older adults with HD: the CIR was significantly correlated with a clinician administered measure of hoarding severity (UHSS), a self-report measure of clutter (SI-R Clutter subscale), and hoarding-related impairment in activities of daily living (ADL-H). Accurate evaluation of impairment in activities of daily living (ADLs) may be especially critical in geriatric hoarding samples, who report moderate to substantial ADL impairment (Kim et al., 2001; Ayers et al., 2012; Steketee et al., 2012). Because clutter severity is associated with increased impairment in ADLs, clinicians may want to use the CIR to obtain a quick evaluation of the potential functional impairment of geriatric HD patients.

In general, reliability and validity outcomes were similar to the original validation of this instrument (Frost et al., 2008). However, unlike the original validation studies, the current study did not find that CIR scores (from any age group) were significantly correlated with either the Difficulty Discarding or the Acquisition subscales (Frost et al., 2008). Thus, our results suggest that the CIR is accurate at teasing out clutter problems from other HD symptoms.

The CIR demonstrated divergent validity in older adults with HD as the CIR was not significantly correlated with measures of anxiety and depression (HADS) as well as hoarding symptoms not directly correlated with clutter (SIR Acquisition and Difficulty Discarding subscales). These results are similar to those found by Frost and colleagues (2008), who concluded that there was a modest correlation of the CIR and anxiety symptoms and no significant correlation between the CIR and depression symptoms. Frost and colleagues (2008) utilized different measures of anxiety and depression and only examined correlations with the CIR; therefore, no comparisons can be made with regards to the individual items of the CIR. These results imply that clutter levels in HD are not a result of lack of motivation or behavioral inactivation associated with depressive symptoms.

Finally, the CIR also demonstrated equivalent levels of criterion validity, operationalized as significant correlations between clinician and participant ratings of the CIR, in older adults as in middle-aged compulsive hoarding and HD samples (Frost et al., 2008; Tolin et al., 2010; DiMauro et al., 2013). This indicates that treatment seeking HD patients across the lifespan are accurate reporters of their clutter levels using the CIR. Thus, when evaluating an older adult for a potential HD diagnosis, clinicians have the option of administering the CIR in the clinic instead of visiting an individual’s home to determine clutter level.

A limitation of the current study was the low number of midlife adults with HD to compare against the sample of older adults with HD, which may have impacted the atypical trends seen in the midlife sample, such as the lack of correlation of the kitchen and bedroom CIR items (see Table 2). Although it is possible that the clutter levels in the two rooms are not correlated in mid-life HD samples, a more likely explanation is that our sample size of midlife adults (n = 24) was too small to accurately characterize the presentation of hoarding symptoms in this population.

Further, these participants were from a self-referred, treatment-seeking sample that had some level of insight into their behaviors and thus more awareness of their clutter. An exploration of this measure in a variety of settings (Adult Protective Services, Primary Care Clinics, etc.) would provide information on the optimal environment or patient population in which to utilize the CIR. Future work should utilize independent ratings and multiple time points for accuracy of ratings and test-retest reliability.

The CIR is particularly useful with older adults due to the ability to quickly garner information on living conditions that may hinder activities of daily living. Further, the simplicity of this measure makes it easy for both clinicians to administer and patients to complete. This measure is not taxing for those with limited physical and cognitive abilities. Thus, this could be utilized as a relatively fast assessment of clutter levels. With HD being thrust into the spotlight with its addition to the DSM-5 (American Psychiatric Association, 2013) and our aging population, clinicians need a tool such as the CIR to provide accurate information on HD symptoms.