The McCusker Subjective Cognitive Impairment Inventory (McSCI): a novel measure of perceived cognitive decline

Abstract

Background

Subjective cognitive decline (SCD), i.e. self/other-reported concerns on one’s cognitive functioning without objective evidence of significant decline, is an indicator of dementia risk. There is little consensus on reliability and validity of the available SCD measures. Therefore, introducing a novel and psychometrically sound measure of SCD is timely.

Objective

The psychometric properties of a new SCD measure, the McCusker Subjective Cognitive Impairment Inventory–Self-Report (McSCI-S), are reported.

Methods

Through review of previously published measures as well as our clinical and research data on people with SCD, we developed a 46-item self-report questionnaire to assess concerns on six cognitive domains, namely, memory, language, orientation, attention and concentration, visuoconstruction abilities and executive function. The McSCI-S was examined in a cohort of 526 participants using factor analysis, item response theory analysis and receiver operating characteristic (ROC) curve.

Results

A unidimensional model provided acceptable fit (CFI = 0.94, TLI = 0.94, RMSEA [90% CI] = 0.052 [.049, 0.055], WRMR = 1.45). The McSCI-S internal consistency was excellent (.96). A cut-off score of ≥24 is proposed to identify participants with SCDs. Higher McSCI-S scores were associated with poorer general cognition, episodic verbal memory, executive function and greater memory complaints and depressive scores (P < .001), controlling for age, sex and education.

Conclusions

Excellent reliability and construct validity suggest the McSCI-S estimates SCDs with acceptable accuracy while capturing self-reported concerns for various cognitive domains. The psychometric analysis indicated that this measure can be used in cohort studies as well as on individual, clinical settings to assess SCDs.

Key Points

• We address the urgent need for psychometrically sound measures of subjective cognitive decline (SCD) by proposing a novel measure.

• The McCusker Subjective Cognitive Impairment Inventory–Self-Report (McSCI-S) measures cognitive concerns with excellent reliability and validity.

• The McSCI-S can identify individuals with above average levels of SCD at 99.9% accuracy.

• This SCD measure can reliably be used in research and clinical settings for clinical decision-making at individual and group levels.

Introduction

Subjective cognitive decline (SCD) refers to self or informant-reported cognitive decline relative to previous abilities that were perceived as normal [1]. Subjective memory complaint (SMC), a more specific component of SCD, includes self or others’ reported concerns about one’s memory abilities. SCD/SMC, in the absence of concurrent cognitive impairment, has been linked to future cognitive decline [1–3] and Alzheimer’s disease (AD) [4].

Available measures of SCD/SMC have major limitations, including small sample sizes used in the test validation phase, less than optimal psychometric properties, lack of a research-informed cut-off score(s) and most importantly, limited content validity as represented by the number of cognitive domains sampled. The existing SCD measures also have low sensitivity in detecting subtle subjective changes at the individual, compared to the cohort level [5], thus limiting their application in clinical settings. For instance, in individual-level decision-making about a patient, a test should have a reliability of greater than 0.95 [6]; a condition that many of the existing SCD measures do not meet [7–10]. Further, a review of 34 SCD measures used in 19 studies, noted that 61.8% (n = 21) of these measures had items that were not conceptually related to specific cognitive functions (e.g. items assessing emotional and psychological conditions or physical and motor functioning) [11].

The McCusker Subjective Cognitive Impairment Inventory (McSCI; pronounced: Mak·see) is developed with the goal of providing a clinically informed and psychometrically sound measure of SCD. Here, we report on the psychometric properties of the McSCI–Self-Report (McSCI-S), including its factor structure, reliability and construct validity.

Methods

Participants were recruited from several ongoing studies in Western Australia (WA) and New South Wales (NSW), Australia. These studies included (i) the Western Australia Memory Study (WAMS), a longitudinal study of the neuropsychological and biological markers of ageing among community-dwelling individuals aged 30+ without a history of psychiatric or neurological conditions [12, 13]; (ii) The Kerr Anglican Retirement Village Initiative in Ageing Health (KARVIAH) Study (NSW), a completed clinical trial that examined the efficacy of curcumin to prevent future risk of dementia in retirement village residents aged 65–90 (see [14]); (iii) The Australian Imaging, Biomarkers and Lifestyle (AIBL) Study of Ageing, an ongoing longitudinal observational study recruiting participants aged 60+ with preclinical, prodromal and clinical stages of AD [15]; (iv) The 56-week, Double-Blind, Randomised Study to Evaluate the Efficacy of Testosterone, With and Without DHA Supplementation on Cerebral Amyloid Load in Known Brain Amyloid-PET Positive Men with Subjective Memory Complaints [TotAL Study; (ACTRN12618000761268)]; and (v) The Dominantly Inherited Alzheimer Network (DIAN), an observational study of individuals aged 18 years and over from a family with a known mutation for AD [16].

Interested participants were provided with a unified, study-specific information and consent pack to sign prior to enrolling into the McSCI Study. This study was approved by the Ramsay Health Care WA/SA (Western Australia and South Australia) Human Research Ethics Committee.

Across all studies, participants were eligible for inclusion into the McSCI Study if they were living independently; were able to read and write fluent English and provide consent; had no major medical, psychiatric or neurological condition (e.g. dementia) affecting their cognitive abilities; had normal hearing and vision with or without correction, as per the parent studies’ inclusion/exclusion criteria; and were 30 years old or over at the time of recruitment.

Measures

The measures used in this study included the following.

The McCusker Subjective Cognitive Impairment Inventory–Self-Report¹

In January 2013, we conducted a general PubMed search with key words including ‘subjective cognitive complaints,’ ‘subjective cognitive impairment,’ ‘subjective memory complaints,’ ‘self-reported cognitive problems,’ ‘self-reported memory problems,’ and ‘dementia,’ ‘mild cognitive impairment’ or ‘MCI’ and ‘Alzheimer’s disease’ or ‘AD’. We identified 205 papers on SMC/SCD, with ~30 that have used published questionnaires (our unpublished data). We also examined literature reporting cognitive decline in preclinical AD, MCI and dementia to determine cognitive domains/functions affected that should be included in the McSCI [17].

An initial bank of 96 items was developed. Two of the authors, with the assistance of their PhD students and research assistants as well as three independent psychologists (see the acknowledgement), rated these items for assessing specific cognitive domains, item clarity, appropriateness and face validity. Forty-six items were selected representing six cognitive domains, namely, language skills (LS, 6 items), orientation (O, 6 items), attention and concentration (AC, 6 items), visuoconstruction abilities (VC, 6 items), executive function (EF, 9 items) and memory (MA, 13 items) (see Table 1 for the McSCI-S Questionnaire, and Appendices for cognitive domains, items allocated to each cognitive domain and descriptive details) [18]. These 46 items were then given to participants to comment on wording, and where appropriate, we have revised the items to improve their comprehensibility. The McSCI-S items are scored on a 5-point Likert scale (0–4) with responses ranging from ‘Almost always true’ (scored: 4) to ‘Almost never true’ (scored: 0). The possible total McSCI score ranges from 0 to 184, with higher scores representing more concerns.

Table 1.

The McCusker subjective cognitive impairment inventory-self report (McSCI-S)

This questionnaire asks about gradual changes in memory, language, concentration, and other mental abilities that you might have experienced or noticed during the last two years, as compared to five years ago. Please read each statement carefully and choose the answer (by a tick ✔or cross ✗) that describes your current mental abilities, irrespective of your physical health (e.g. arthritis, hearing or eyesight problems). For those phrases that you cannot choose an answer, please make your best guess of how good you will perform in such a given condition.
In the last two years		Almost Always True	Usually True	Occasionally True	Usually Not True	Almost Never True
1	I have noticed more difficulties with my language and speech abilities.
2	I have significant problems with my memory abilities.
3	I have more difficulty concentrating on different tasks.
4	I have more difficulty solving everyday problems that come up around the house.
5	I cannot remember where I have put things.
6	I lose track of the date, more often.
7	I may forget the name of the person I am talking to even when I know them.
8	It is difficult for me to repeat back something that I have just heard (e.g. phone number, address etc.).
9	I forget details of an event that happened a couple of weeks ago.
10	I am not as organised as I used to be.
11	I get distracted quickly and cannot follow a conversation or a movie plot.
12	I am slower at writing or typing.
13	I forget things that I intend to do in the near future.
14	I do things earlier or later than the time, they are expected to be done.
15	I keep forgetting where I have parked my car in a carpark.
16	I have trouble planning things ahead and carrying out these plans on time.
17	I often forget basic things that I learned at school when I was young.
18	Sometimes, I don’t understand what others say, regardless of my hearing ability.
19	I am more likely to bump into objects or people.
20	I have more trouble making decisions on everyday matters (e.g. which clothes to wear; which item to buy).
21	Others have told me that I repeat myself (e.g. telling them the same story or asking the same questions).
22	When driving/walking back to my home, I may pass my house without realising it.
23	I fail to recognise well-known individuals (e.g. actor, singer, TV character) that I knew.
24	It is difficult for me to manage a household emergency (e.g. a leaking tap; a spider or a small non-poisonous lizard in the bedroom etc.).
25	I forget the name of everyday items (e.g. kitchen utensils; familiar household objects).
26	I have to try harder to remember things that I have heard, read, or seen.
27	I cannot keep my mind focused on a task even if I enjoyed it.
28	I cannot do two things at once (e.g. washing dishes and talking to someone; driving and listening to the radio).
29	More often, I forget what day of the week it is.
30	I often forget how to use a gadget/tool that I have recently learned to use (e.g. computer; mobile/smart phones).
31	I cannot find a friend’s/relatives’ address on the map.
32	I can no longer play well in a game of skill (e.g. Bridge, Chess, Cards and Golf) or doing something I was good at.
33	When speaking, I have difficulty finding the right words.
34	My handwriting has dramatically changed.
35	When I look at old photos, I cannot recognise some of the people I used to recognise.
36	Nowadays, I have to read something a few times to understand it.
37	I fail to pay attention to what is going on around me.
38	I may not be able to accurately copy a drawing (e.g. a tree; a house, etc.).
39	I can no longer do simple arithmetic calculations ‘in my head’.
40	Sometimes, I get confused about how to get to a specific room (e.g. bathroom, bedroom, or lounge) within my home.
41	If interrupted, I cannot pick up the conversation from where I stopped.
42	When I see famous places or buildings, I may not recognise them.
43	I have had difficulty giving directions to someone because I get the routes confused.
44	It is difficult for me to stop something that I have just started doing.
45	I forget my close relatives’ names or dates of birth (e.g. grandchildren, nieces and nephews).
46	I have difficulty using things that I have previously used with confidence (e.g. electronic toothbrush; microwave; calculator).

Other measures

To examine the validity and relationship between the McSCI-S and established cognitive measures, we analysed the WAMS data collected at the same study visit. Global cognitive function was assessed using the Montreal Cognitive Assessment (MoCA) [19]. Verbal episodic memory was examined using the California Verbal Learning Test-II (CVLT-II) including total learning trials, short and long delay free recalls and recognition hits [20]. The Trail Making Test B (time: seconds) [21] was used to assess attention/processing speed and executive function [22]. The SMC was assessed using the Memory Assessment Clinic- Questionnaire (MAC-Q), a measure of perceived decline in memory that has six items with total scores ranging from 7 to 35. A cut-off score of ≥25 represents SMCs [8]. The Depression Anxiety Stress Scale-21 (DASS-21) is a 21-item self-report, and each of its item is rated on a 0–3 scale and the total score is multiplied by 2 to create a score range of 0–126 [23].

Data analysis

The psychometric properties of the McSCI-S were evaluated using item response theory (IRT) methods. This allows both the item difficulty parameters and the individual trait-level estimates to be placed on the same scale. Due to the polytomous nature of the McSCI-S items, the graded response model (GRM) introduced by Samejima [24] was used to fit the model and estimate item parameters.

Factor structure

Because the McSCI-S asks questions about concerns of decline across six different cognitive domains, we sought to determine whether the scores were better explained by a unidimensional or multidimensional factor structure. Using Mplus version 8 [25], we specified two competing models: a single-factor confirmatory factor analysis (CFA) model and a six-factor CFA model. Model fit was judged using standard fit criteria, including the comparative fit index (CFI), Tucker–Lewis index (TLI), root mean square error of approximation (RMSEA) and weighted root mean square residual (WRMR). The superiority of the multidimensional model would be indicated by an absence of estimation problems, superior model fit statistics, sufficiently dissociable factors (i.e. factor intercorrelations of < 0.9) and standardised factor loadings of sufficient magnitude (i.e. > 0.40) and sign (i.e. positive). The best model was then further evaluated using McDonald’s omega, calculated using the psych package (version 1.8.12) for R version 3.5.2 [26].

Graded response model

The unidimensional graded response IRT model was analysed using the mirt package (version 1.29) for R version 3.5.2. We estimated item discrimination (a) parameters and up to four threshold (b) parameters for each item. These parameter estimates were used to evaluate the McSCI-S scale-level psychometric properties, including total test information, standard error and reliability across the range of the underlying trait (θ).

Cut-off score calculation

We used Youden’s index [27] to identify a relatively sound cut-off point. Youden’s index has a score range of 0 to 1 and the cut-off points with higher scores are considered to perform better for screening or diagnostic purposes. Youden’s index was calculated using the following formula: Youden’s J = (sensitivity + specificity) − 1 [28, 29].

Results

The CFA results and associations between the McSCI-S and other measures are provided here. Where necessary, further information or results are provided in the Appendices, under Appendices.

Demographics

Table 2 provides demographic data for the CFA cohort (n = 526). Participants’ age ranged between 39 and 97 years (M = 71.47; SD = 7.28), and 35.4% were female. Years of education ranged from 6 to 24 (M = 13.06; SD = 3.08). In the IRT sample (n = 385), 269 participants (69.8%) and in the larger sample with full data (n = 503), 347 participants (68.9%) were memory complainers using MAC-Q cut-off ≥25.

Table 2.

Demographic data for the CFA cohort (n = 526).

Variable	Overall	Range
Age, M (SD)	71.47 (7.28)	39–97
Female sex, N (%)	186 (35.4)	–
Education years, M (SD)	13.06 (3.08)	6–24
McSCI-S Total, M (SD)	36.96 (22.53)	0–122

McSCI-S, The McCusker Subjective Cognitive Impairment Inventory- self report

Factor analysis, validity and reliability

McCusker Subjective Cognitive Impairment Inventory–Self-Report factor structure

The first CFA model tested the hypothesis that the McSCI-S item scores could be explained by a six-factor model, with each correlated factor corresponding to a unique cognitive domain (i.e. language, memory, attention and concentration, executive functioning, orientation and visuoconstruction). Although this appeared to fit well (CFI = 0.94, TLI = 0.94, RMSEA [90% CI] = 0.051 [.048, 0.054], WRMR = 1.41), there was insufficient discrimination between the factors. The estimated correlations between all the pairwise factor correlations exceeded 0.87, and 13 of the 15 factor correlations were g > 0.90, (95% CI [.99, 1.04]). These results strongly suggest redundancy among the factors and a more parsimonious model was warranted.

A unidimensional model also fit the data well, CFI = 0.94, TLI = 0.94, RMSEA [90% CI] = 0.052 [.049, 0.055], WRMR = 1.45, with almost no decrement in model fit relative to the six-factor model. In addition, McDonald’s omega for the unidimensional model was 0.96, which strongly supports a single-factor model.

Graded response model

After verifying that the unidimensionality assumption could be satisfied, Samejima’s GRM was used to estimate item and scale parameters. Discrimination and threshold estimates are shown in Table 3. All discrimination parameters performed well, ranging from a minimum of 1.04 to a maximum of 2.68. Threshold parameters for the first (lowest) threshold ranged from a low of −1.97 (Item 5) to a high of 2.00 (Item 40), suggesting that very low trait levels (q < −2) of SCD may not be reliably measured by the McSCI-S items. In contrast, the threshold parameters for the last (highest) threshold ranged from a low of 1.92 (Item 25) to a high of 4.77 (Item 44), suggesting that high and very high trait levels (q > 2) of SCD can be estimated precisely by the McSCI-S. Figure 1 shows the relationship between the underlying trait level of SCD and the expected score on the McSCI-S. The scale information and standard errors are shown in Appendices Figure-Supp File-A. As illustrated in Figure 1, the McSCI-S has high information (³ 10) across a wide range of the SCD trait, roughly 2 SD below the mean to 4 SD above the mean. Consequently, the standard errors of estimated SCD within this interval are low (<0.31), meaning that the McSCI-S is capable of precisely estimating SCD for most levels of the trait. In addition, a consequence of the high information is the resulting high reliability across most of the latent ability continuum, as shown in Figure Supp-File (Appendices). In particular, reliability estimates of 0.90 or greater, indicative of excellent reliability, were found for ability levels ranging from −1.73 to 4.54. Reliability estimates of 0.95 or greater, a highly desirable measurement property, are possible within the range of ability levels from −0.89 to 3.72. In other words, for 99.99% of the population of individuals with above-average (i.e. q > 0) SCD, the precision of the McSCI-S is excellent (i.e. SEs < 0.23) and its reliability is in the most desirable range for individual- level SCD screening decision-making [6]. In contrast, the psychometric properties of the McSCI-S in the lower half of the SCD trait continuum was less desirable, but not inadequate, meaning that SCD estimates are less precise and reliable when individuals are very low on this trait (i.e. not reporting high levels of cognitive decline). Further details at the item level of the McSCI-S can be found in Table Supp-2 (providing the latent trait z-scores corresponding to observed scores). Figure Supp-2 shows item response category characteristic curves for each of the 46 McSCI items, providing a visual depiction of how each of the items’ response options operate as a function of the underlying trait being measured.

Table 3.

Parameter estimates for the graded response model

Item	a	b1	b2	b3	b4
1	1.27	−0.59	0.53	2.44	3.92
2	1.28	−1.96	−0.37	1.68	3.14
3	1.91	−0.77	0.50	1.70	3.02
4	1.96	−0.20	1.24	2.46	–
5	1.24	−1.97	−0.45	2.11	3.91
6	1.46	−0.89	0.58	2.09	3.49
7	1.22	−0.91	0.34	2.65	3.94
8	1.58	−1.45	0.08	1.63	2.94
9	1.51	−1.20	0.36	2.09	3.70
10	1.85	−0.46	1.01	2.22	3.17
11	1.99	−0.45	0.92	2.25	–
12	1.83	−0.30	0.95	1.86	3.06
13	1.72	−0.88	0.76	2.37	–
14	2.02	−0.03	1.51	2.78	–
15	1.48	0.07	1.83	3.77	–
16	1.90	0.29	1.73	3.05	–
17	1.54	−0.37	1.02	2.37	–
18	2.13	−0.16	1.04	2.50	–
19	1.40	0.48	1.73	3.61	–
20	1.93	0.27	1.46	2.66	–
21	1.15	−0.36	1.06	3.52	–
22	1.92	1.69	3.12	–	–
23	1.04	−0.35	1.39	3.96	–
24	1.41	1.24	2.67	3.26	–
25	1.31	0.60	1.92	–	–
26	1.92	−0.69	0.56	1.96	3.06
27	2.68	0.23	1.25	2.19	–
28	1.41	0.45	1.57	2.75	3.67
29	1.42	0.41	1.92	3.20	–
30	1.48	−0.59	0.52	1.98	3.26
31	2.00	0.95	2.22	–	–
32	1.56	0.19	1.61	2.81	–
33	1.51	−0.94	0.52	2.59	–
34	1.30	0.08	1.35	2.37	3.65
35	1.68	0.48	1.76	3.25	–
36	1.80	−0.75	0.50	1.97	3.36
37	1.74	−0.13	1.20	2.82	–
38	1.19	0.79	2.50	3.35	–
39	1.50	0.55	1.64	2.88	–
40	2.00	2.00	–	–	–
41	1.30	−0.43	1.17	3.21	–
42	1.57	0.50	2.04	–	–
43	1.97	0.36	1.61	2.67	–
44	1.09	−0.16	1.39	2.85	4.77
45	1.26	0.06	1.23	2.47	3.88
46	1.33	1.28	2.80	–	–

Note. a = discrimination parameter; b = threshold parameter.

Figure 1.

The relationship between SCD at trait level and expected score on the McSCI-S. Expected Total McSCI-S Score as a Function of Theta (Latent SCD). The x-axis represents the level of the latent trait (SCCs), with higher scores representing more complaints. The y-axis shows the expected total McSCI-S sum score. The s-shaped curve demonstrates the relationship between the latent trait and the sum score on the McSCI-S that would be expected on the basis of the underlying trait.

Association with other measures

A preliminary study was conducted with a cohort of 383 individuals aged 39–97 years old from our larger longitudinal study of ageing, the WAMS [18]. The demographic data for this sub-cohort are presented in Table Supp-3. Age was not significantly associated with the McSCI-S total or its IRT Factor scores (P > .05). The McSCI-S total score was negatively associated with global and specific cognitive functions including the MoCA, the CVLT-II learning trials 1–5 total score and CVLT-II short and long delay free recalls (r = −.14, −.21, −.13, −.13, respectively; P < .05). The McSCI-S total score was also associated with TMT-B and DASS depression score (P < .05; Table 4).

Table 4.

Associations between McSCI and other measures (n = 382)

	Age	Edu.	MoCA	MAC-Q	DASS-D (n = 69)	CVLT-L1-5	CVLT- SD-FR	CVLT- LD-FR	CVLT-Recog.	TMT-B
McSCI-S Total	.033	−.14a	−.18b	. 53b	.45b	−.21b	−.13a	−.13a	−.041	.24b
McSCI-S IRT Factor Score	0.06	−.14a	−.10	.55b	.25b	−.15a	−.10	−.09	−.01	.19a

Pearson correlation coefficient (two-tailed) was used; CVLT, California Verbal Learning Test-II; CVLT-L1–5, CVLT-II learning trials 1–5 total score; CVLT-SD-FR, CVLT-II short delay free recall score; CVLT-II LD-FR, CVLT-II long delay free recall; CVLT-Recog., CVLT-II recognition Hits; DASS-total, Depression Anxiety Stress Scale; Edu, education years; M-Q, The Memory Complaint Questionnaire; McSCI, McCusker Subjective Cognitive Impairment Inventory (McSCI-S); McSCI-T, McSCI-S Total Score; MoCA, The Montreal Cognitive Assessment; TMT-B, Trail making Test-B.

^a P < .05,

^b P < .01.

The MAC-Q cut-off ≥25 was used to group participants into SMCs and noncomplainers. The two groups performed significantly differently on the McSCI-S total score, [t (df = 376) = −7.29, P < .001, (Figure Supp-3)]. Using the McSCI-S Factor scores, this significant difference between the MAC-Q-derived SMCs and noncomplainers was confirmed [t (df = 197.6) = 6.11, P < .01]. The association between the MAC-Q and McSCI-S total score or Factor score was significant (r = .53 and .55, respectively; P < .001); however, as 73% of the variance was not shared, the MAC-Q and McSCI-S appear to capture different aspects of SCD.

Cut-off score

To identify an optimal cut-off score for the McSCI-S, ROC curves analysis was conducted. The participants (n = 503) were classified into two groups based on the absence or presence of SMC using the MAC-Q cut-off score. This model resulted in an area under the curve (AUC) = 0.70; 95% CI = 0.65–0.75. Using the Youden index, a cut-off score of 24 and above resulted in sensitivity = 0.79 and specificity = 0.50 (data on ROC graphs, sensitivity, specificity and Youden’s J are presented in the Appendices: Figure Supp-4 and Table Supp-4). This cut-off was chosen for its higher sensitivity.

Discussion

The results clearly indicated that the McSCI-S is a reliable measure of SCD. In addition, the McSCI-S total scores (both total score and the IRT-based factor score) were significantly related to measures of cognitive function and memory complaints providing evidence of good concurrent validity.

The McSCI-S items were developed to capture concerns on most cognitive domains that have been previously identified as important by the SCD Initiative (SCD-I) Working Group including memory, attention and working memory, language, executive function, orientation and visuospatial skills [11]. However, the CFA findings on the McSCI-S showed that a single factor and not a six-factor model was the best fit for the data. While this finding indicates that a single total score may best represent the SCDs for a given respondent, it does not imply that domain specific scores of the McSCI-S should not be used in future research when individuals with higher risk or at confirmed (e.g. via biomarkers) prodromal and clinical stages of dementia or other neurological/psychiatric conditions are concerned. That is, in clinical cohorts, the patterns of responding may more clearly align with a multifactor model. Of note, recent research has provided increasingly convincing evidence that age-related cognitive decline progresses in a relatively uniform fashion across most cognitive domains, rather than in a domain-specific fashion [30]. Such findings, indirectly support the unidimensional factor structure of the McSCI-S.

Compared to previously published measures of SCD (e.g. [7–10]), the McSCI-S has shown significantly higher reliability and better validity statistics. For example, the McSCi’s omega coefficient was much higher than similar measures and we have previously reported strong reliability findings for the McSCI, using Cronbach’s alpha (.96; n = 367) in a smaller sample [18]. Furthermore, the IRT analyses added nuance to the reliability estimates because they do not assume that a single reliability value can describe the entire test. The IRT results for the McSCI-S showed that reliability was highest for average and above levels of SCD, but less reliable at lower levels of SCD. This is an important finding because it indicates that the McSCI-S can estimate SCD when it is high, which is the case in preclinical stage of AD, as compared to when it is low, as in the clinical stages of dementia [31], potentially due to decreased insight. Therefore, McSCI-S can serve better in detecting those at risk rather than those with dementia. These psychometric properties imply that the McSCI, as compared to other measures, performs better at detecting SCDs related to neurodegenerative processes, although this has not been assessed by us and requires further research to be supported.

Content validity was supported by the finding that the McSCI-S was significantly associated with another measure of SCD (MAC-Q) and there were significant differences between those with and without such complaints. Evidence for concurrent validity came from the finding that higher McSCI-S scores were associated with poorer performance on objective measures of cognition (e.g. the MoCA and CVLT-II), as it was expected. Previous studies on the relationship between SCD and objective measures of cognition have mostly been unsuccessful in stablishing convincing associations between the two or providing evidence of sufficient validity for SCD measures [32–35].

The cut-off score ≥24 is proposed for the McSCI-S, following the ROC analysis. This cut-off was chosen for better sensitivity at the cost of specificity, which was not the primary objective of this study. Of note, Youden’s index is based on the results of the ROC, which relies on the accuracy of the gold standard in identifying true-positive from true-negative cases. Here, we used the MAC-Q as it was the primary measure of SMC available across several cohort studies that we recruited participants from. Because of the less-than-ideal reliability data available for the MAC-Q (Cronbach’s α = .57) [36], future research with more robust SCD measures may result in slightly different cut-off scores for the McSCI-S.

Higher McSCI-S scores were also associated with higher depression scores. There is a wealth of evidence on the relationship between depression severity and SCD [37, 38]. In addition, depressive symptoms have been proposed as a risk factor for cognitive decline as well as dementia and can indicate those at higher risk of dementia [12]. However, a strong relationship between depression and SCD may also result from overlapping or unclear items. For example, depression measures often contain items that ask about memory and other cognitive concerns [39] or items that are confusing to complete. With this in mind, during the development phase of the McSCI, we excluded mood items and sought feedback from our participants in finalising the wording of the items to minimise such sources of bias [39]. Future research can determine how well the McSCI-S predicts the rate of cognitive decline and conversion to MCI and dementia after controlling for the effects of depression.

Conclusions

Research evidence suggests that SCDs are related to a wide range of conditions and represent various underlying aetiologies. Therefore, it is important to accurately capture these self-reported concerns about cognition to identify the patterns of SCD that are predictive of dementia as opposed to those patterns that are indicative of depression, personality or other factors. The McSCI-S, as compared to other measures, has shown powerful psychometric properties including very high reliability and validity and is, therefore, an appropriate measure to assess SCDs at both individual and group levels.

Data Availability:

The data presented in this study are available from corresponding authors upon written, formal request. The data are held in a password protected, secure cloud environment, based at our universities.

Declaration of Conflicts of Interest:

B.G., M.W., C.S. and R.B. report no conflict of interest directly or indirectly associated to this manuscript. H.R.S. and R.N.M. report being Directors of SMarT Minds WA, Australia. H.R.S. and R.N.M. have had or are receiving personal reimbursements or research support from the Pharmaceutical and Nutraceutical companies including Alector, Alnylam Pharmaceuticals, CWEK PTY LTD, WA, Australia, and Biogen pharmaceuticals.

Declaration of Sources of Funding:

The Western Australia Memory Study (WAMS), the primary study for this manuscript, has been supported by a grant from the National Health and Medical Research Council-Australia (Grant Number: 324100 awarded to RNM), an Early Career Researcher Grant (HRS; G1001512-2014) from Edith Cowan University, the Dementia Australia, the Cecilia Margaret Hudson Dementia Research Grant to H.R.S. (2014; Dementia Australia Research Foundation), the McCusker Charitable Foundation grant, the Alzheimer’s Research Australia (previously known as the Australian Alzheimer’s Research Foundation Inc.), the Hollywood Private Hospital and the Charlies Foundation Western Australia, Australia. The sponsors of this study had no contribution to the design, data collection, statistical analysis, interpretation of the results or the preparation and final submission of the manuscript.