Abstract
Introduction
To assess the reliability of short versions of the Australian National University Alzheimer's Disease Risk Index (ANU-ADRI).
Methods
A short form of the ANU-ADRI (ANU-ADRI-SF) was developed by assessing risk and protective factors with single questions where possible and with short forms of sub-questionnaires where available. The tick box form of the ANU-ADRI (ANU-ADRI-TB) was developed with unique questions for each risk and protective factor for Alzheimer's disease. The short versions were evaluated in an independent community sample of 504 participants with a mean age of 45.01 (SD = 14.85, range = 18–81).
Results
The short versions demonstrated high reliabilities when compared with the ANU-ADRI. However, the proportion of misclassification was high for some risk factors and particularly for the ANU-ADRI-TB.
Discussion
The ANU-ADRI-SF may be considered if less reliable questions from the ANU-ADRI-SF can be replaced with more reliable questions from the ANU-ADRI for risk/protective factors with high misclassification.
Keywords: Risk assessment, Alzheimer's disease, Short versions, Screening
1. Introduction
At present, there is no cure or effective treatment for dementia [1], [2], [3] and existing pharmacological treatments do not modify the course of the disease [4]. Prevention is therefore one of the key objectives of current dementia research [5], and increased attention has been paid to identifying risk and protective factors for dementia. It is also essential that people understand and address their risk profile for dementia as early as possible before the development of the pathologic processes which lead to unrecoverable neurodegeneration. Easily accessible methods of risk assessment are an important tool for facilitating population-level dementia risk awareness.
A questionnaire-based risk assessment tool, the Australian National University Alzheimer's Disease Risk Index (ANU-ADRI), was developed to assess the presence of 11 risk and 4 protective factors for Alzheimer's disease (AD) [6]. These risk and protective factors have reliable scientific evidence and can be measured by self-report. The ANU-ADRI was validated on three independent cohorts [7] against the Cardiovascular Risk Factors, Aging and Dementia index and has been proposed as the key risk assessment tool in large-scale dementia prevention trials. ANU-ADRI has already been used in the first online dementia risk reduction trial in Australia [8], [9] and is currently available to the general public through the ANU-ADRI website (http://anuadri.anu.edu.au/). The website has attracted over 11,700 visitors since its launch early 2014.
The original ANU-ADRI takes approximately 15–20 minutes to complete, although longer time of up to 1 hour has been reported for older adults who were not familiar with the computer or Internet. The length of the ANU-ADRI did not affect the participation of a dementia prevention trial [9] or completion of publicly available ANU-ADRI through the website. However, a shorter version of the ANU-ADRI may have a role on public websites and online applications, enabling the ANU-ADRI to reach wider audiences in more diverse settings and in turn, save time in assessing an individual's risk level. Given that the average general practice (GP) consultation time is 14.6 minutes (95% CI, 14.1–15.0) [10], the length of the time taken to complete the current ANU-ADRI can be considered long. Hence, by providing shorter versions of the ANU-ADRI, health professionals such as GPs for example may be able to assess a client's risk level and provide them with relevant advice in a timely manner. The ANU-ADRI can also be used to screen individuals to identify those at high risk of developing AD who would benefit most from intervention programs. We therefore developed and evaluated two brief alternatives (a short form and a tick box form) to the original longer version of the ANU-ADRI while aiming to preserve its content coverage.
2. Methods
2.1. Participants and procedure
One thousand and seventy three people from the community were approached by Qualtrics, a survey company. Five hundred and four of them completed the original ANU-ADRI as well as one of the short versions to be validated. Sample size calculations were estimated using G*Power (version 3.1.3). To detect a medium difference (0.5) between two independent sample means, with a 5% risk of type I error (α), 95% power, and two equal groups (1:1), 105 persons in each group was required. Our sample size was more than double the required size, and half of the participants (group 1; n = 251) completed the original and the short form of the ANU-ADRI. The other half (group 2; n = 253) completed the original and the tick box form of the ANU-ADRI. Participants were randomly assigned to one of two groups by the Qualtrics survey program. Inclusion criteria were being ≥18 years, being proficient in English, having internet access, and having no psychiatric or neurological diagnoses.
2.2. Measures and procedures
2.2.1. Short form
A short form of the ANU-ADRI (ANU-ADRI-SF; see Supplementary A) which included the validated short forms of questionnaires used in the original ANU-ADRI (e.g. International Physical Activity Questionnaire short form [11] and 10 item Community Epidemiological Study-Depression scale (CESD-10) [12]) while keeping single-item questions (e.g., gender, age, and smoking status) was developed. The risk and protective factors covered by more than one item were also simplified into single-item questions by adding questions together. For example, two questions on diabetes “Have you ever been told by a doctor or other health professional that you have diabetes?” and “Have you ever been told by a doctor or other health professional that you have high sugar levels in your blood or urine?” became a single question “Have you ever been told by a doctor or other health professional that you have diabetes or have high sugar levels in your blood or urine?”. These single items intended to reduce the number of questions while keeping the contents. These include education, diabetes, cholesterol, traumatic brain injury, cognitive activity, and fish intake. The short form was estimated to take approximately 5 minutes to complete.
2.2.2. Tick box form
The tick box form of the ANU-ADRI (ANU-ADRI-TB; see Supplementary B) was built with single-item questions for each AD risk and protective factors and was estimated to take approximately 2 minutes to complete. The ANU-ADRI-TB was developed with an assumption that a single-item scale may be as sensitive and reliable as multi-item scales. This assumption was supported in previous research such as in depression [13] and physical activity [14]. It was found that a single-item interview for depressed mood provides a reliable and accurate screen that can be used in clinical settings that permit direct patient interview.
One of the short versions followed by original long version of the ANU-ADRI was administered on the same day.
2.3. Analysis
To analyze the reliability of the short versions, intra class correlation coefficients (ICC) were used between the original scores and short versions' scores. The relative measure of risk points attributed to each risk and protective factor assessed in the ANU-ADRI-SF was the same as that used in the original ANU-ADRI which ranged between 0–6 for risk factors and −7 to 0 for protective factors. ANU-ADRI-TB, however, only had binary variables (having risk/protective factors vs not) and was compared against binary variables of the original ANU-ADRI. Although the original ANU-ADRI did not evaluate misclassifications, as the tick box form can only be measured as binary outcomes (having risk/protective factors vs not), we felt it was worth examining misclassification in addition to reliability tests. The number of risk and protective factors misclassified in the short compared to the long form were calculated using binary variables (having risk/protective factors vs not). Percentages of misclassification of each risk and protective factor were also calculated to examine false-positive (being identified as having a risk factor when this is not the case) and false-negative misclassifications (not being identified as having a risk factor when this is not the case). Reliability and misclassification were not investigated for age, education, and BMI as exactly the same questions were used to assess these variables in all three versions. The data were analyzed using SPSS 22.
3. Results
Table 1 shows the participants' demographic characteristics and dementia risk exposure. Two hundred and forty nine (49.4%) of 504 participants were males. Participants were aged between 18 and 81 (M = 45.01, standard deviation [SD] = 14.85) years and were evenly spread across age groups of ≤30, 30–39, 40–49, 50–59, and ≥60 years. The mean age of the original validation sample was higher than the present study. However, ANU-ADRI is currently available to the general public as we are not restricting user's age. The statistics from the ANU-ADRI website demonstrated that the mean age of website users was 48.71 years (SD = 18.90). Therefore, the current sample was a better representation of the users of the ANU-ADRI.
Table 1.
Demographic characteristics of participants in the short form and tick box form group
| Characteristics | Short form | Tick box form | χ2 | P value |
|---|---|---|---|---|
| Age (M, SD) | 44.96 (15.03) | 45.06 (14.69) | t = −0.072 | .943 |
| Gender | ||||
| Females | 51.0% | 50.2% | 0.032 | .858 |
| Education (M, SD) | 14.54 (3.51) | 14.94 (4.02) | t = −1.216 | .225 |
| BMI | ||||
| Overweight | 29.5% | 29.6% | 0.069 | .966 |
| Obese | 28.3% | 27.3% | ||
| High cholesterol | 23.1% | 26.1% | 0.603 | .437 |
| Diabetes | 13.9% | 7.9% | 4.727 | .030 |
| TBI | 10.8% | 9.9% | 0.104 | .747 |
| Depression | 37.1% | 35.2% | 0.192 | .661 |
| Physical activity | ||||
| Medium | 30.7 % | 33.6% | 3.037 | .219 |
| High | 47.0% | 50.2% | ||
| Cognitive activity | ||||
| Middle | 3.2% | 3.2% | 0.204 | .903 |
| Highest | 31.5% | 29.6% | ||
| Social activity | ||||
| Low | 40.6% | 48.6% | 5.120 | .163 |
| Fish consumption | ||||
| 0.25–2 p p/wk | 44.2% | 49.4% | 4.166 | .244 |
| 2–4 p p/wk | 12.0% | 7.5% | ||
| >4 p p/wk | 6.4% | 8.3% | ||
| Alcohol consumption | ||||
| Light to moderate | 71.5% | 73.2% | 0.181 | .913 |
| Smoking | ||||
| Current smoker | 24.3% | 24.9% | 2.204 | .332 |
| Past smoker | 26.3% | 31.6% | ||
| Pesticide | 10.5% | 9.2% | 0.246 | .620 |
Note. Figures are based on the original ANU-ADRI. Bold text indicates significance.
The samples from group 1 (short form) and group 2 (tick box form) were not significantly different across most characteristics. However, those who completed ANU-ADRI-SF were significantly more likely to have diabetes than those who completed ANU-ADRI-TB.
3.1. Reliability
ICC were computed to study the reliability of the short versions of the ANU-ADRI (see Table 2). ICCs for ANU-ADRI-SF suggested moderate to strong agreement with coefficients ranging from 0.772 to 0.992. All were statistically significant (P < .001) except for cognitive activity where the coefficients was small and not significant (ICC = .031, P = .285). The ANU-ADRI-TB also showed small to strong agreement for most risk factors with correlations ranging from 0.444 to 1 except for cognitive activity demonstrating extremely small and insignificant (ICC = .024, P = .287) coefficient value.
Table 2.
Inter-class correlation (ICC) coefficients between short forms and long form
| Risk/Protective factors | Short form |
Tick box form |
|---|---|---|
| ICC (95% confidence interval) | ICC (95% confidence interval) | |
| Cholesterol | 0.926 (0.905–0.942) | 0.979 (0.973–0.983) |
| Diabetes | 0.909 (0.882–0.930) | 1 |
| TBI | 0.776 (0.568–0.878) | 0.702 (0.509–0.819) |
| Depression | 0.890 (0.858–0.914) | 0.789 (0.725–0.838) |
| Physical activity | 0.796 (0.737–0.841) | 0.444 (0.197–0.605) |
| Cognitive activity | 0.031 (−0.084 to 0.147) | 0.024 (−0.065 to 0.118) |
| Social activity | 0.937 (0.919–0.951) | 0.822 (0.684–0.889) |
| Fish consumption | 0.772 (0.631–0.849) | 0.751 (0.667–0.812) |
| Alcohol consumption | 0.990 (0.987–0.992) | 0.640 (0.536–0.721) |
| Smoking | 0.992 (0.990–0.994) | 0.996 (0.995–0.997) |
| Pesticide | 0.977 (0.971–0.982) | 0.826 (0.774–0.865) |
Bold text indicates significance.
Internal consistency was also examined for CESD-10, measuring depression, and it showed a high level of internal consistency (Cronbach's alpha of 0.879), which was comparable with the internal consistency of CESD-20 (Cronbach's alpha of 0.934).
3.2. Misclassification
Misclassification of someone having or not having risk and/or protective factors was also investigated using binary variables. Fig. 1 demonstrated that those who completed the ANU-ADRI-TB tended to have higher rates of misclassifications than those who completed the ANU-ADRI-SF.
Fig. 1.
Number of risk and protective factors misclassification.
Proportions of misclassification on risk and protective factors were examined (Table 3). The false-positive misclassification represented cases where a risk or protective factor for AD was identified as present when it was not. The false-negative misclassification represented cases where a risk or protective factor for AD was identified as absent when it was not. The factor with the highest false-positive misclassification rate was cognitive activity (66.1%), whereas fish consumption (13.9%) and physical activity (13.1%) had the highest false-negative misclassification rates for the ANU-ADRI-SF. For the ANU-ADRI-TB, cognitive activity (68.4%) followed by fish consumption (11.9%) and alcohol consumption (11.5%) had the highest false-positive misclassification rates, whereas social activity level (34.4%) and physical activity level (28.1%) had the highest false-negative misclassification rates. Overall, more risk and protective factors were misclassified with ANU-ADRI-TB than ANU-ADRI-SF. Both short versions overclassified protective cognitive activity and underclassified social and physical activity.
Table 3.
Percentage of misclassification
| Risk/Protective factors | Short form |
Tick box form |
Base rate‡ | ||
|---|---|---|---|---|---|
| False positive∗ | False negative† | False positive∗ | False negative† | ||
| Cholesterol | 2.8% | 2% | 1.2% | 0.4% | 23.4% |
| Diabetes | 3.6% | 0% | 0% | 0% | 9.1% |
| TBI | 4.0% | 0% | 4.3% | 0.8% | 6.3% |
| Depression | 3.2% | 6.0% | 3.2% | 11.9% | 36.1% |
| Physical activity | 4.4% | 13.1% | 2.0% | 28.1% | 81.1% |
| Cognitive activity | 66.1% | 0% | 68.4% | 0.4% | 29.6% |
| Social activity | 7.6% | 8.8% | 2.0% | 34.4% | 44.6% |
| Fish consumption | 0.4% | 13.9% | 11.9% | 1.6% | 17.1% |
| Alcohol consumption | 0.4% | 0.8% | 11.5% | 2.4% | 71.2% |
| Smoking | 1.2% | 0.4% | 0.4% | 0.8% | 53.6% |
| Pesticide | 0% | 0.8% | 0% | 4.0% | 9.7% |
Misclassification as having risk/protective factors when they do not.
Misclassification as not having risk/protective factors when they do.
Based on the original ANU-ADRI.
4. Discussion
The present study developed and evaluated two short versions of the ANU-ADRI as possible alternatives to the original ANU-ADRI to be used in timely restricted settings. The short form of the ANU-ADRI demonstrated moderate to good reliability as the ANU-ADRI scores between short and long versions were similar. This suggests that when there are time constraints, the ANU-ADRI-SF may replace the original ANU-ADRI.
The ANU-ADRI-TB on the other hand had lower reliability and higher occurrence of misclassification of risk and protective factors than the ANU-ADRI-SF. Risk and protective factors that were simplified into single items or used shorter versions of existing questionnaires were likely to lead to misclassification. This suggests that factors that require comprehensive measures cannot be replaced by a single question. This appears to be especially true for factors that require a wide range of actions and/or activities (e.g., cognitive engagement) as well as frequencies and/or quantity (e.g., frequency and duration of weekly physical activity). Another possible explanation for low reliability could be the ceiling effect and that the original version had a wider range of options. For example, most adults may be engaging in at least one of the activities listed for cognitive activities every day or nearly every day. It is not surprising to find high misclassifications. Caution should therefore be exercised when using the short versions of the ANU-ADRI and other dementia risk assessment scales that do not have adequate measures of risk factors.
Translation of epidemiological research on risk factors for dementia requires the development of widely accessible assessment tools. The unique contribution of this study is to develop and evaluate a brief, practical AD risk assessment tool that can be used by clinicians or self-administered. The strength of the study was that we tested short versions of assessment tools against an established, valid tool that had been applied in practical settings. In addition, the scores were drawn from scientific evidence. However, there are some limitations to the study. The validation of the scales has not been examined as this would require following participants from when they did not have AD to when they develop AD in large populations over long periods of time to determine how well they perform in predicting AD decades later. The present study demonstrates the reliability of short versions of the ANU-ADRI when compared to the longer version. Lack of similar short versions of assessment tools also made it impossible for us to evaluate validity of the scales. Future research should investigate the validity of the short versions with large population-based samples. We also did not assess participants' cognitive functioning at the time of the assessment as this was not the aim of the present study. However, this should be considered in a future study to ensure that participants who are involved do not have any cognitive dysfunction at the time of assessment.
Overall, the present study showed that the shorter versions of the ANU-ADRI, especially the ANU-ADRI-SF have limitations but are of practical use. Short versions were least accurate in assessing lifestyle activities because these involve more complex assessment algorithms (physical, cognitive, and social activity). However, the short form is acceptable for assessing medical and demographic risk factors. Therefore, the use of the short versions should only be recommended as a second choice, when individuals have limited time and need a quick indication of an AD risk level or where lifestyle activity is not the focus of the assessment. The original ANU-ADRI cannot be fully replaced by the short versions and when using the short form, less reliable questions from the ANU-ADRI-SF should be replaced with more reliable questions from the original ANU-ADRI for those risk and protective factors with low reliability and high misclassification. We recommend using the tick box version of the ANU-ADRI only where there is no possibility of using the short form. One future possible use of the tick box form may be administration using a stepped approach, with participants being asked to complete longer questions according to the tick box version results.
Research in context.
-
1.
Systematic review: We conducted a systematic review of the literature to identify a brief, evidence based, validated, risk assessment tool for Alzheimer's disease. No other tool that assesses level of risk for Alzheimer's or dementia has been developed that is convenient and based on self-report.
-
2.
Interpretation: Our study results show that short form of the ANU-ADRI may be considered where the longer original version is not practical or when there are time constraints. However, the original ANU-ADRI remains the preferred choice. The tick box form of the ANU-ADRI is not recommended for use unless there is no other option. ANU-ADRI-TB is too brief and insufficient for measuring the complexity of cognitive, physical, and social activities.
-
3.
Future directions: Further research is required to examine the usability of the ANU-ADRI-SF in various settings and feedback on effectiveness of the scale should be collected for future improvement.
Acknowledgments
The research was funded by the Dementia Collaborative Research Centres (DCRC) Knowledge Translation Small Grant. K.J.A. is funded by National Health and Medical Research Council fellowship 1102694, and N.C. is funded by an ARC future fellowship 120100227. The funders had no role in this research. S.K. had full access to all the data in the study and takes responsibility for the integrity of the data and accuracy of the data analysis.
Footnotes
The authors declare no conflicts of interest.
Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.trci.2016.03.001.
Supplementary data
References
- 1.Chung S., Mehta K., Shumway M., Alvidrez J., Perez-Stable E.J. Risk perception and preference for prevention of Alzheimer's disease. Value Health. 2009;12:450–458. doi: 10.1111/j.1524-4733.2008.00482.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Polidori M.C., Nelles G., Pientka L. Prevention of Dementia: Focus on Lifestyle. Int J Alzheimers Dis. 2010;2010 doi: 10.4061/2010/393579. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Rockwood K., Wallack M., Tallis R. The treatment of Alzheimer's disease: Success short of cure. Lancet Neurol. 2003;2:630–633. doi: 10.1016/s1474-4422(03)00533-7. [DOI] [PubMed] [Google Scholar]
- 4.Rodda J., Carter J. Cholinesterase inhibitors and memantine for symptomatic treatment of dementia. BMJ. 2012;344:e2986. doi: 10.1136/bmj.e2986. [DOI] [PubMed] [Google Scholar]
- 5.Mangialasche F., Kivipelto M., Solomon A., Fratiglioni L. Dementia prevention: current epidemiological evidence and future perspective. Alzheimers Res Ther. 2012;4:6. doi: 10.1186/alzrt104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Anstey K.J., Cherbuin N., Herath P. Development of a new method for assessing global risk of Alzheimer's disease for use in population health approaches to prevention. Prev Sci. 2013;14:411–421. doi: 10.1007/s11121-012-0313-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Anstey K.J., Cherbuin N., Herath P.M., Qiu C., Kuller L.H., Lopez O.L. A self-report risk index to predict occurrence of dementia in three independent cohorts of older adults: the ANU-ADRI. PLoS One. 2014;9:e86141. doi: 10.1371/journal.pone.0086141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Anstey K.J., Bahar-Fuchs A., Herath P., Rebok G.W., Cherbuin N. A 12-week multidomain intervention versus active control to reduce risk of Alzheimer's disease: study protocol for a randomized controlled trial. Trials. 2013;14:60. doi: 10.1186/1745-6215-14-60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Anstey K.J., Bahar-Fuchs A., Herath P., Kim S., Burns R., Rebok G.W. Body brain life: A randomized controlled trial of an online dementia risk reduction intervention in middle-aged adults at risk of Alzheimer's disease. Alzheimer's & Dementia: Translational Research & Clinical Interventions. 2015;1:72–80. doi: 10.1016/j.trci.2015.04.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Sayer G.P., Britt H., Horn F., Bhasale A., McGeechan K., Charles J. Australian Institute of Health and Welfare; Canberra: 2000. Measures of health and health care delivery in general practice in Australia. [Google Scholar]
- 11.Bertakis K.D., Azari R., Helms L.J., Callahan E.J., Robbins J.A. Gender differences in the utilization of health care services. J Fam Pract. 2000;49:147–152. [PubMed] [Google Scholar]
- 12.Green C.A., Pope C.R. Gender, psychosocial factors and the use of medical services: A longitudinal analysis. Soc Sci Med. 1999;48:1363–1372. doi: 10.1016/s0277-9536(98)00440-7. [DOI] [PubMed] [Google Scholar]
- 13.Chochinov H.M., Wilson K.G., Enns M., Lander S. “Are you depressed?” Screening for depression in the terminally ill. Am J Psychiatry. 1997;154:674–676. doi: 10.1176/ajp.154.5.674. [DOI] [PubMed] [Google Scholar]
- 14.Jackson A.W., Morrow J.R., Jr., Bowles H.R., FitzGerald S.J., Blair S.N. Construct validity evidence for single-response items to estimate physical activity levels in large sample studies. Res Q Exerc Sport. 2007;78:24–31. doi: 10.1080/02701367.2007.10599400. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.