Validity and Utility of the Center for Epidemiological Studies Depression Scale for Detecting Depression in Family Caregivers of Persons with Dementia

Abstract

Background/Aims

The psychometric properties of Center for Epidemiological Studies Depression Scale (CES-D) have never been substantively investigated in caregivers of persons with dementia (PwD). We evaluated the validity and reliability of CES-D for detecting caregiver depression in dementia, and assessed whether CES-D could provide added utility beyond Zarit Burden Interview (ZBI).

Method

Family caregivers of community-dwelling PwD (n = 394) completed self-administered questionnaires containing CES-D. Factorial validity was evaluated with confirmatory factor analysis; convergent and discriminant validity with Pearson's correlation coefficient; known-group validity by comparing across key variables; and internal consistency reliability with Cronbach's α. Cohen's κ was used to compare the agreement between those with depression (CES-D ≥16) and those with high caregiver burden (ZBI >60).

Results

CES-D demonstrated convergent, discriminant and known-group validity, consistent with a priori hypotheses. The original four-factor model of CES-D produced the best model-fit indices. Internal consistency reliability was good for the CES-D total scale (α = 0.92), but lower for the Positive affect and Interpersonal problems subscales (α = 0.70–0.74). Forty-five percent of the caregivers had depression as identified by CES-D (95% CI 40–50%), but most of them were not identified by high ZBI scores (κ = 0.16).

Conclusions

CES-D is a valid and reliable scale for detecting caregiver depression in dementia. It has added utility, beyond that of a caregiver burden scale, in identifying a subgroup of caregivers with depression but not burden. However, two subscales (Positive affect and Interpersonal problems) may require caution in interpretation among non-Caucasian caregivers.

Introduction

Depression occurs in at least 1 in 3 caregivers of persons with dementia (PwD) (as demonstrated by a recent meta-analysis) [1], a prevalence which is comparatively higher than that in the general population or in the caregivers of persons with other physical or mental illnesses [1]. Depression can cause a variety of psychological and somatic problems and increase the risk of caregivers contemplating suicide [2]. It compromises caregivers' physical health [3], lowers caregivers' quality of life [4], and has been shown to cause the caregivers to place PwD in an institutional care facility more rapidly [5, 6]. Depression in caregivers can also impact the PwD adversely, as it has been associated with more rapid cognitive decline in PwD [7]. Considering the significance of this problem, it is pertinent to focus our efforts on case finding of caregiver depression with a validated scale.

While there are many scales to measure depression (such as 9-item Patient Health Questionnaire [PHQ-9] [8] and Geriatric Depression Scale [9]), the Center for Epidemiological Studies Depression Scale (CES-D) is one of the most widely-used scales in caregivers of PwD [1, 10, 11] due to its simplicity of self-administration and the non-proprietary nature. In the literature, CES-D has demonstrated acceptable validity for both Western [12, 13, 14, 15, 16, 17] and Asian populations [18, 19, 20, 21], as well as for both general [14, 16] and clinical populations (such as in patients with cancer [15], diabetes [20], and schizophrenia [17]). The diagnostic accuracy of CES-D has also been confirmed in a recent meta-analysis [22], where it achieved an area under the curve of 0.87 on the summary receiver operating characteristic curve for the diagnosis of depression.

Notwithstanding the widespread use of CES-D, its psychometric properties have never been substantively investigated in caregivers of PwD. This is pertinent because the experience of depression in dementia caregiving can be qualitatively distinct from that of the other populations, often precipitated and perpetuated by the unique caregiving context related to the PwD. For example, in one study, caregivers of PwD reported that the sense of isolation as well as the physical and emotional demands of caring for PwD compounded their mental health issues [23]; while in another study, the caregivers reported the prominent theme of relational deprivations and losses, as well as the sentiments of powerlessness and helplessness [24]. These reports in the literature provided some examples on the unique experiences in caregiving which are less often endured by non-caregivers with depression.

Moreover, the literature has been unclear whether depression scales (such as CES-D) may have any added utility in dementia services. This is especially relevant considering that scales to capture caregiver stress/burden have already been routinely used in dementia services, and that depression and stress/burden are commonly conceptualized as falling within the same spectrum (with depression being the manifestation of high and significance stress/burden, as suggested by the diathesis-stress model [25]). As such, it remains uncertain whether the construct of depression would have been sufficiently measured by caregiver stress/burden scales, which essentially renders depression scales redundant in dementia caregiving.

In this study, we sought to assess the validity (factorial validity, convergent and discriminant validity, and known-group validity) and reliability (internal consistency reliability) of CES-D for detecting depression in family caregivers of PwD. Essentially, our evaluations would include, among others, confirming the factor structure of CES-D in dementia caregivers, correlating CES-D with other scales of related constructs and elucidating the differences in CES-D scores across key demographic information (stage of dementia and behavioral problems of PwD). As a secondary aim, we also sought to assess whether CES-D provides any added utility beyond a commonly used caregiver burden scale (Zarit Burden Interview, ZBI) in identifying caregivers with significant duress.

Methods

Participants and Procedures

We recruited participants from the dementia clinics of two tertiary hospitals (Institute of Mental Health and Khoo Teck Puat Hospital). These two hospitals are the only two dementia services that serve the population in the North-East of Singapore, and hence provide a reasonably representative sampling of the population (given that majority of PwD in Singapore still received dementia care from tertiary hospitals) [26, 27]. At Institute of Mental Health, we also recruited participants from the inpatient wards of the geriatric psychiatry unit. These wards provide care for PwD with severe behavioral problems which are difficult to manage in the community settings. We used a consecutive sampling method and achieved a response rate of 87.8% in our recruitment.

The inclusion criteria comprised: (a) spouses or children of PwD; (b) caring for community-dwelling PwD (that is, the PwD were not dwelling in nursing homes); and (c) able to read in English or Chinese. The participants completed self-administered questionnaires which included a depression scale (CES-D) and a caregiver burden scale (ZBI). Participants from Khoo Teck Puat Hospital also completed an additional scale assessing caregiving gains (Gain in Alzheimer Care Instrument, GAIN).

This study received the ethical approval from the Domain Specific Review Board of National Healthcare Group, Singapore (Ref. No.: 2014/00127). It also received informed consent from all the participants.

Measures

CES-D is a 20-item, self-administered scale which measures depressive symptoms in the previous 1 week [14]. The original author recommended a cut-off score of ≥16 in CES-D to identify those with depression [14]. This cut-off score (≥16) has also been validated in a recent meta-analysis, which demonstrated its sensitivity of 0.87 and specificity of 0.70 in detecting depression in the general population [22]. At the same time, an alternative cut-off score of ≥20 was also proposed by the meta-analysis to balance between sensitivity and specificity (0.83 and 0.78, respectively). CES-D has 4 subscales – Depressed affect, Somatic symptoms, Interpersonal problems, and Positive affect [14, 28].

ZBI is a 22-item scale which measures the perceived caregiver stress/burden (caregiver burden has previously been defined as the physical, psychological, social, and financial problems of caring for an adult with a disabling condition) [29]. It has a total score ranging from 0 to 88, with scores above 60 indicating high caregiver burden [30]. ZBI has demonstrated good reliability and validity for assessing caregiver burden in various countries [31, 32, 33], including in Singapore [33, 34]. ZBI was shown to contain five subscales – Burden in the relationship, Emotional well-being, Social and family life, Finances, and Loss of control[35].

GAIN is a scale to measure caregiving gains in dementia. Higher scores denote higher gains. GAIN has previously been validated in Singapore [36, 37].

Demographic information related to caregivers and PwD was collected, based on self-reports by the caregivers or obtained from the medical records when the caregivers were uncertain. This information included age, gender, ethnic, marital status, employment status, highest education level, relationship with PwD, duration and frequency of caregiving, stage of dementia, and severe behavioral problems in PwD. The stage of dementia was obtained using a brief measure based on the three staging concept described in the revised third edition of Diagnostic and Statistical Manual of Mental Disorders (DSM-III-R) [38]. From the three options, participants chose the description that best described the PwD – still capable of independent living (mild stage), needs some assistance with daily living (moderate stage), or needs round-the-clock supervision (severe stage). This brief measure was previously shown to have reasonable agreement with the Clinical Dementia Rating Scale (κ 0.56–0.6) [39, 40, 41]. The presence of severe behavioral problems was indirectly measured through the need for admission to the geriatric psychiatry ward. When PwD needed admission to the geriatric psychiatry ward, it indicated that PwD had behavioral problems that were too severe to be managed in the community setting.

Statistical Analyses

Overall, we conducted the following analyses: factorial validity, convergent and discriminant validity, known-group validity, and internal consistency reliability. The details of each analysis are described in the paragraphs below.

Factorial validity was assessed with confirmatory factor analysis [42, 43]. We compared the model fit between the original four-factor model [14, 28] and other alternative models in the literature [44, 45, 46, 47, 48]. We evaluated the goodness-of-fit of the models using root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and the Tucker-Lewis index (TLI). Good model fit was considered if the values of RMSEA and SRMR were below 0.08, and the values of CFI and TLI were above 0.9 [42, 43]. We also used the Akaike information criterion in model comparison.

Convergent and discriminant validity was evaluated by examining the correlations among CES-D, ZBI, and their respective subscales, using the Pearson's correlation coefficient (r). Correlation coefficients >0.50 were considered strong [49].

Known-group validity was assessed by comparing the mean scores of CES-D, using one-way analysis of variance. We pre-selected two variables (stage of dementia and behavioral problems of PwD) to contrast CES-D scores in the assessment of known-group validity, because more advanced dementia and the presence of behavioral problems in PwD have been reported as key risk factors for caregiver depression in a recent meta-analysis [10].

Internal consistency reliability of CES-D and its subscales was assessed using Cronbach's α. The reliability indices of ≥0.70 represent the minimally acceptable reliability to allow group comparisons in research settings [50, 51], while values ≥0.90 indicate suitability for individual-level measurements in clinical care [50, 51, 52]. For each item in the subscales of CES-D, we also tested the item-rest correlation and the Cronbach's α of the subscale if that item was removed from its subscale. The item-rest correlation of at least 0.40 [51] indicates the consistency of responses of each item with its counterparts within the same subscale. The Cronbach's α of a subscale should become lower if an item was deleted from the subscale, indicating the necessity of the item in maintaining internal consistency reliability.

As part of the secondary aim, we assessed whether high scores in a caregiver burden scale (ZBI) would have sufficed to identify those with significant burden and concomitant depression. Using Cohen's κ, we compared the agreement between those with ZBI scores of >60 (high caregiver burden) [30] and those with CES-D scores of ≥16 (depression) [14, 22]. We also conducted sensitivity analysis to assess the changes in Cohen's κ if the alternative cut-off score for CES-D (≥20) [22] was used instead.

All analyses were performed using the STATA software version 13.

Results

We recruited 394 participants, with 54.6% of the participants from Institute of Mental Health while the rest from Khoo Teck Puat Hospital. The demographic information is shown in Table 1.

Table 1.

Demographic information on the caregivers and the persons with dementia (n = 394)

Variables related to caregivers
Age, years	53.0±10.7
Female gender	236 (59.9)
Ethnicity
Chinese	341 (86.6)
Malay	25 (6.3)
Indian	18 (4.6)
Others	10 (2.5)
Marital status
Married	271 (68.8)
Single	94 (23.9)
Widowed/divorced/separated	29 (7.3)
Employment status
Not working	123 (31.2)
Working part-time	52 (13.2)
Working full-time	219 (55.6)
Highest education
Primary or no formal education	41 (10.4)
Secondary	228 (57.9)
Tertiary	125 (31.7)
Relationship with PWD
Child	340 (86.3)
Spouse	54 (13.7)
Staying with PWD	264 (67.0)
Duration of caregiving in years	6.8±6.7
Frequency of caregiving
Daily, for at least 4 h a day	211 (53.6)
Daily, but less than 4 h a day	79 (20.0)
At least once a week	84 (21.3)
Less than once a week	20 (5.1)
Primary caregiver role	279 (70.8)
Variables related to PWD
Age, years	79.5±8.2
Female gender	278 (70.6)
Age at dementia diagnosis, years	75.6±8.5
Duration of dementia diagnosis, years	4.5±3.5
Stage of dementiaa
Mild	62 (15.7)
Moderate	163 (41.4)
Severe	169 (42.9)
Severe behavioral problemsb	22 (5.6)

Data are presented asn (%) or mean± standard deviation. PWD, persons with dementia.

^aThe stage of dementia was obtained using a brief measure based on the three dementia severities described in the revised third edition of Diagnostic and Statistical Manual of Mental Disorders (DSM-III-R). From the three options, participants chose the description that best described the PWD - still capable of independent living (mild stage), needs some assistance with daily living (moderate stage), or needs round-the-clock supervision (severe stage).

^bThe presence of severe behavioral problems was indirectly measured through the need for admission to a geriatric psychiatry ward, indicating a behavioral problem that was too severe to be managed in the community setting.

In confirmatory factor analysis, the original four-factor model of CES-D produced the best model fit indices (Table 2). This was followed closely by the three-factor model which had marginally poorer fit.

Table 2.

Fit indices of various models in confirmatory factor analysis (n = 394)

Model	χ2 (df)	RMSEAb	SRMRb	CFIb	TLIb	AICc
One-factor modela	895.00 (170)	0.104	0.071	0.828	0.808	16,503.58
Two-factor modela	605.02 (l69)	0.081	0.051	0.896	0.884	16,215.59
Three-factor modela	557.54 (l67)	0.077	0.049	0.907	0.894	16,172.12
Four-factor model (original)a	545.45 (164)	0.077	0.049	0.909	0.895	16,166.02

The model which fulfilled the criteria of good fit is highlighted in bold. df, degree of freedom; RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual; CFI, comparative fit index; TLI, Tucker-Lewis index; AIC, Akaike Information Criterion.

^aThe original four-factor model consisted of the subscales of Depressed affect, Somatic symptoms, Interpersonal problems, and Positive affect. The three-factor model comprises the Interpersonal problems subscale, Positive affect subscale, and a subscale that combines Depressed affect and Somatic symptoms. The two-factor model distinguishes the positive affect from the rest of the scale items. The one-factor model indicates CES-D as a unidimensional scale.

^bCriteria for good model-fit: RMSEA <0.08, SRMR <0.08, CFI >0.90, TLI >0.90.

^cSmaller AIC values indicate better model fit.

In the assessment of convergent and discriminant validity, the findings were consistent with what we had expected (Table 3) [36, 53, 54, 55]. CES-D correlated strongly with caregiver burden scale ZBI (r = 0.71) and most of the subscales of ZBI (r = 0.60–0.70), but not with the Finances subscale of ZBI (r = 0.46) or the caregiving gains scale (GAIN) (r = −0.16). Most of the subscales of CES-D correlated strongly with ZBI (r = 0.70–0.71), with the exception of the Interpersonal problems subscale and Positive affect subscale of CES-D, which correlated less strongly with ZBI (r = 0.48 and 0.29, respectively).

Table 3.

Construct validity as shown by the correlation among various scales, using Pearson's correlation coefficient

	CES-D and its subscales
	CES-D total score	Depressed affect subscale	Somatic symptoms subscale	Interpersonal problems subscale	Positive affect subscale
ZBI total scale	0.71a	0.70	0.71	0.47d	–0.29e
Burden in the relationship subscale	0.60	0.59	0.63	0.39	–0.20
Emotional well-being subscale	0.66	0.65	0.66	0.45	–0.25
Social and family life subscale	0.65	0.62	0.64	0.44	–0.30
Loss of control subscale	0.70	0.69	0.67	0.44	–0.34
Finances subscale	0.46b	0.47	0.45	0.29	–0.18
GAIN	–0.16c	–0.11	–0.04	–0.07	0.30

CES-D, Center for Epidemiological Studies Depression Scale; ZBI, Zarit Burden Interview; GAIN, Gain in Alzheimer Care Instrument.

^aCES-D correlated strongly (r >0.50) with a caregiver burden scale, ZBI. This is expected because the literature has reported the correlation between depression and burden in caregivers of those with other medical or psychiatric conditions [54, 55].

^bCES-D correlated less strongly (r <0.50) with the Finances subscale of ZBI. The Finances subscale of ZBI, which comprises item 15 (do you feel that you don't have enough money to take care of your relative), is not expected to show concordance with the depressive symptoms measured by CES-D.

^cCES-D correlated less strongly (r <0.50) with the GAIN scale. This is because GAIN, which measures positive outcomes in caregiving, is a different construct from the burden-depression spectrum and has only been shown to correlate weakly with ZBI [36].

^dThe Interpersonal problems subscale of CES-D correlated less strongly (r <0.50) with ZBI. The Interpersonal problems subscale of CES-D describes the feeling of critical reactions from others. Such feeling may be experienced, though less often, among caregivers who are burdened [53].

^eThe Positive affect subscale of CES-D correlated less strongly (r <0.50) with ZBI because each of them measures a distinct construct.

In the assessment of known-group validity (Fig. 1), the findings were also consistent with what we had expected. The mean CES-D scores were significantly different among those caring for PwD at various stages of disease (p = 0.007). In pairwise comparisons, the mean CES-D scores were significantly higher among those caring for PwD at severe stage compared to the mild stage (p = 0.035 after Bonferroni adjustment) or the moderate stage (p = 0.023 after Bonferroni adjustment). However, the mean scores were not significantly different between the mild and moderate stage (p = 1.000 after Bonferroni adjustment). Similarly, caring for PwD with behavioral problems resulted in higher CES-D scores (p < 0.001).

Fig. 1.

Comparison of the mean scores of Center for Epidemiological Studies Depression Scale (CES-D) between groups which are known to affect the depressive symptoms in caregivers of persons with dementia in the assessment of known-group validity. a Mean scores of CES-D across the stages of dementia. b Mean scores of CES-D in the presence and absence of behavioral problems. Whiskers indicate the 95% confidence interval of the scores.

Generally, CES-D and its subscales demonstrated good reliability. However, two of the subscales showed relatively lower reliability (Cronbach's α of 0.69 for the Interpersonal problems subscale; and 0.74 for the Positive affect subscale). Table 4 provides details on the internal consistency reliability of the individual items within each subscale. Most of the Cronbach's α were lowered if an item was deleted from a subscale, indicating the necessity of the item in maintaining internal consistency reliability.

Table 4.

Internal consistency reliability of the subscales of CES-D

CES-D items within their original factors	Item-rest correlationa	Cronbach's a if item deletedb
Subscale 1: Depressed affect (Cronbach'sα = 0.91)
3. I felt that I could not shake off the blues even with the help from my family or friends	0.69	0.90
6. I felt depressed	0.82	0.88
9. I thought my life had been a failure	0.65	0.90
10. I felt fearful	0.79	0.89
14. I felt lonely	0.66	0.90
17. I had crying spells	0.67	0.90
18. I felt sad	0.77	0.89
Subscale 2: Somatic symptoms (Cronbach'sα = 0.85)
1. I was bothered by things that usually don't bother me	0.66	0.82
2. I did not feel like eating; my appetite was poor	0.61	0.83
5. I had trouble keeping my mind on what I was doing	0.71	0.82
7. I felt that everything I did was an effort	0.57	0.84
11. My sleep was restless	0.62	0.83
13. I talked less than usual	0.47	0.85
20. I could not get going	0.67	0.82
Subscale 3: Positive affect (Cronbach'sα = 0.74)
4. I felt I was just as good as other people	0.40	0.76
8. I felt hopeful about the future	0.52	0.69
12. I was happy	0.61	0.64
16. I enjoyed life	0.63	0.63
Subscale 4: Interpersonal problems (Cronbach'sα = 0.69)
15. People were unfriendly	0.53	NA
19. I felt that people dislike me	0.53	NA

CES-D, Center for Epidemiological Studies Depression Scale; NA, not applicable.

^aCorrelation between the item and the summated score for all other items from the same factor.

^bCronbach's a of the respective subscale if the item was deleted from the subscale.

The original cut-off score of CES-D (≥16) identified 45% of the caregivers as having depression (95% CI 40–50%), while the alternative cut-off score (≥20) identified a lower prevalence of 31% (95% CI 27–36%). As shown in Figure 2, many caregivers with depression (CES-D ≥16) did not have high ZBI scores (>60). The agreement between high CES-D scores and high ZBI scores was modest (κ = 0.16). The agreement remained modest even when the alternative CES-D cut-off score of ≥20 was used (κ = 0.25).

Fig. 2.

Scatterplot between a commonly used caregiver burden scale (Zarit Burden Interview, ZBI) and Center for Epidemiological Studies Depression Scale (CES-D) (n = 394). ZBI scores above 40 are considered moderate to high caregiver burden (vertical line), while CES-D scores above 16 are considered significant depression (horizontal line).

Discussion

Although CES-D has frequently been used to detect depression in caregivers of PwD, this is the first study, to our knowledge, that substantively explores the psychometric properties (factorial validity, convergent and discriminant validity, known-group validity, and internal consistency reliability) of CES-D in family caregivers of PwD. Similar to studies in non-caregivers [19, 20], the results give assurance to the validity and reliability of CES-D for detecting depression in caregivers of PwD. Our findings also support the original four-factor structure of CES-D as has been reported by previous studies involving caregivers of PwD [56, 57, 58, 59, 60]. Our reported prevalence of depression, in 31–45% of caregivers, was comparable with findings from the extant literature [1], which further strengthens our hypothesis that CES-D indeed measures the construct of depression in caregivers of PwD.

CES-D and all its subscales had reliability indices of near to or above 0.70, indicating that they can be reliably used for group comparisons in research settings [50, 51]. However, two subscales (Positive affect and Interpersonal problems) had relatively lower reliability (Cronbach's α 0.69–0.74) compared to the rest (Cronbach's α 0.85–0.92). With reliability indices which were far below the value of 0.90, these two subscales may be less suitable for individual-level measurements in clinical care [50, 51, 52]. The more modest reliability results of these two subscales could possibly be due to the effects of culture, and suggest a need for more caution in interpreting these two subscales especially in non-Caucasian caregivers. The Positive affect subscale asks caregivers whether they feel good, hopeful, happy, and enjoy life. The items in this subscale might not be endorsed consistently in the oriental culture which emphasizes modesty and emotional restraint [20, 61, 62]. Such issue of internal consistency in the Positive affect subscale has been reported in a few studies involving Asian populations [20, 61, 62]. For instance, one study assessed CES-D among Korean immigrants in America and found that immigrants with lower adoption of the American culture reported less positive affect, with a noticeable dose-response relationship [61]. The influence of culture could also be seen with the Interpersonal problems subscale. The Interpersonal problems subscale focuses on the emotion of negative reactions from others. It is possible that the expression of such feelings may not be consistently endorsed in some populations, such as Singapore, where social harmony and respect for differences in personal beliefs are highly valued [63].

Conventionally, caregiver stress/burden and depression are perceived to mediate each other and fall within the same entity of duress as a result of caregiving [64]. As such, one would expect good agreement between those with high stress/burden and those with depression, and that a caregiver burden scale should suffice to identify those with depression. Unexpectedly, our findings suggest otherwise. While it is true that most of the caregivers with high burden also had depression (Fig. 2), the converse was not necessarily true. In this study, many caregivers with depression did not exhibit high burden. It is possible that this group of caregivers may have been depressed due to reasons beyond their caregiving experience. Another plausible explanation may be that these caregivers with depression are liable to self-blaming rather than attribute their predicament to caring for the PwD, which is not always socially sanctioned. This observation can benefit from future research to explore the caregiving experience of caregivers with depression in the absence of high burden, to ascertain the impact of depression on caregiving. The findings also demonstrate the added utility of CES-D beyond that of a caregiver burden scale, and reinforce the need to include assessment for depression in routine practice. This is important because the interventions for caregiver burden and depression can differ. Caregiver burden can be ameliorated by interventions that focus on improving the caregiving experience, such as psychoeducation, caregiver training, and respite care [65], while caregiver depression may require more intensive interventions (such as cognitive behavioral therapy) to target the mood symptoms of the caregiver [66].

Some limitations of this study should be mentioned. First, the scales were self-administered in this study; hence, caregivers with lower literacy might have been under-represented. Second, we did not directly measure the degree of behavioral problems in PwD and had only used the indirect measure of the need for admission to a geriatric psychiatry ward to capture PwD with the more severe degree of behavioral problems not manageable in the community setting. Third, this was a cross-sectional study, and we did not have the opportunity to assess longitudinal properties such as test-retest reliability or responsiveness. We are currently planning a separate longitudinal study to address this gap.

In conclusion, CES-D can be used in clinical care, as a valid and reliable scale, to detect caregiver depression in dementia. It has also demonstrated its usefulness, beyond that of a caregiver burden scale, in identifying a subgroup of caregivers with depression. However, two subscales of CES-D (Positive affect and Interpersonal problems) show more modest reliability, possibly due to the influence of culture, and may require more caution in interpretation when used among non-Caucasian caregivers.

Statement of Ethics

This study received the ethical approval from the Domain Specific Review Board of National Healthcare Group, Singapore.

Disclosure Statement

There is no conflict of interest to declare.

Funding Sources

This work was supported by the Singapore Ministry of Health's National Medical Research Council under the Centre Grant Program (Grant No.: NMRC/CG/004/2013). It also received pilot funding from the National University of Singapore. Separately, the last author (T.M.L.) was supported by a research fellowship from the Singapore Ministry of Health's National Medical Research Council (Grant No.: NMRC/Fellowship/0030/2016 and NMRC/CSSSP/0014/2017). The funding sources had no involvement in any part of the project.

Author Contributions

T.M.L. designed the study, planned the statistical analyses, collected the data, performed statistical analyses, interpreted the results, and wrote the manuscript. J.Y. searched the literature, interpreted the results, and contributed to the manuscript writing and revision. P.Y. advised on the study design, contributed to data collection, interpreted the results, and revised the manuscript. M.G. performed statistical analyses, interpreted the results, and reviewed the manuscript. All authors approved the final version of the manuscript for submission.