Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Apr 1.
Published in final edited form as: Clin Neuropsychol. 2017 Oct 9;32(3):524–529. doi: 10.1080/13854046.2017.1378371

Consistency of traumatic brain injury reporting in older adults with and without cognitive impairment

Kristin Wilmoth a,iD, Christian LoBue a, Matthew A Clem a, Rajadhar Reddy a, Linda S Hynan b,a, Nyaz Didehbani a, Kathleen Bell c, Kyle B Womack d,a, John Hart Jr d,a, Hunt Batjer e, C Munro Cullum a,d,e
PMCID: PMC5851823  NIHMSID: NIHMS924555  PMID: 28988512

Abstract

Objective

Medical history information regarding prior traumatic brain injury (TBI) usually relies on self-report, although little is known about the reliability of this information with regard to injuries sustained years or decades earlier. Even less is known about the reliability of self-reported medical history information in older individuals with cognitive impairment. To this end, we assessed the test-retest reliability of self-reported TBI history in a large, national sample.

Methods

Participants (n = 4309) were older adults with intact cognition, mild cognitive impairment (MCI) and Alzheimer’s disease (AD) from the National Alzheimer’s Coordinating Center. Subjects provided TBI history information at baseline and one annual follow-up visit. Consistency of self-reported history of TBI with <5 minutes loss of consciousness (mLOC) and TBI with ≥5 mLOC reported at time 1 and 2 was analyzed across diagnostic groups.

Results

Overall, subjects provided reports of TBI history at follow-up that were highly consistent with baseline reports (97.8–99.6% agreement), and Cohen’s kappa coefficients were all larger than .80 and statistically significant, maximum p < .001. Furthermore, level of cognitive impairment was not a significant predictor of consistency in reporting.

Conclusions

These data are some of the first to suggest that self-report may be a consistent method of obtaining remote TBI history in the absence of medical records for older individuals, regardless of cognitive impairment.

Keywords: Traumatic brain injury, concussion, Alzheimer’s, disease, cognitive, impairment, dementia

Introduction

Increasing attention has been devoted to the potential later-in-life consequences of traumatic brain injuries (TBI). Although additional research is needed before conclusions can be drawn, TBI may initiate and/or accelerate neurodegenerative processes in some at-risk individuals as they age (Gardner et al., 2014; LoBue, Denney, et al., 2016; LoBue, Wadsworth, et al., 2016; LoBue, Wilmoth, et al., 2016). As with other medical information, TBI history is usually based upon retrospective reporting, often years or decades following the event, though little is known about the reliability of such reports.

Despite reliance upon medical histories, the reliability of patient reports has not often been studied. With regard to TBI reporting, Kerr, Marshall, and Guskiewicz (2012) found 62% consistency in the number of life-time injuries reported in 899 former NFL players based on surveys nine years apart. A similar study in young athletes ages 12–18 found 80% concordance between TBI reports, approximately two years apart (Wojtowicz et al., 2017). To our knowledge, only two studies have examined the impact of cognitive impairment on the reliability of self-reported medical history in older adults, with mixed results. For example, the Mini-Mental State Examination (Folstein, Folstein, & McHugh, 1975) was found to be unrelated to reliability of self-reports of 14 specific diagnoses in 1002 older women with moderate to severe impairments in community functioning (Simpson et al., 2004). However, impaired clock drawing was related to lower agreement between patient self-report and medical records in patients asked about colonoscopy history, with 76 and 88% agreement in the impaired and unimpaired groups, respectively (Daly, Levy, Joshi, Xu, & Jogerst, 2010)

Because health professionals rely heavily on self-reported information to formulate diagnoses and treatments, research into the reliability of such reports is important. The purpose of this investigation was to examine the test-retest reliability of self-reported history of remote TBI in older individuals with and without cognitive impairment in a large, national sample.

Methods

The National Alzheimer’s Coordinating Center (NACC) Uniform Data-Set (UDS) compiles data from all Alzheimer’s Disease Centers (ADCs) funded by the National Institute on Aging (NIA). The UDS was queried for subjects aged 50+ with initial and follow-up visits completed between September 2005 and June 2015. Subjects included those with a primary diagnosis of mild cognitive impairment (MCI), possible or probable Alzheimer’s disease (AD), or were cognitively intact based on standard NACC procedures (Morris et al., 2006).

As part of each examination, UDS subjects (often with a study partner present) are asked whether they have ever sustained a TBI with: (a) brief (<5) minutes loss of consciousness (mLOC) or (b) longer (≥5) mLOC. Each TBI question is recorded as having never occurred (‘Absent’), occurred within the past year (‘Recent/Active’), or occurred more than a year prior to the visit (‘Remote/Inactive’). In order to control for TBI events during the follow-up period, which would confound reliability analyses, we excluded subjects reporting a recent TBI at either visit. Clinical Dementia Rating Scale (CDR) Sum of Boxes were used to determine level of gross impairment, with higher scores (range = 0–18) indicating more severe impairment (Morris, 1993).

Chi-square or t-tests evaluated TBI history in relation to factors, such as age, gender, race (Caucasian, African American, and other), education level, diagnosis, dementia severity. Cohen’s kappa was used to assess agreement (i.e. yes-yes or no-no responses) to the presence/absence of remote TBI across the first and second UDS visits. Analyses were repeated for the <5 and ≥5 mLOC TBI variables in the same sample, and kappa coefficients were calculated separately for participants with normal cognition, MCI, and AD to compare diagnostic groups. To control for Type I error, a Bonferroni correction for multiple comparisons was performed (α = .008). Statistical assumptions of each comparison were examined to ensure the appropriateness of analyses. Specifically, homogeneity of variances was checked using Levene’s test. IBM SPSS V23 was used to perform all analyses.

Results

Of the 31,792 cases returned by the UDC query, 5820 presented for follow-up within one year. Of those, 715 no longer met diagnostic inclusion criteria by follow-up, 59 reported a recent TBI history and were excluded, and TBI data for 24 cases was ‘Unknown’ or missing. Thus, our sample consisted of 2024 participants with normal cognition, 760 with MCI, and 1525 with AD (see Table 1 for sample details).

Table 1.

Demographic characteristics and dementia rating for subjects with normal cognition, mild cognitive impairment, and Alzheimer’s Disease.

Cognitively intact n = 2024 MCI n = 760 AD n = 1525
Age, y, M (SD) 73.5 (9.1) 74.4 (8.6) 75.2 (9.4)
Education, y, M (SD) 15.6 (2.9) 15.1 (3.2) 14.2 (3.6)
CDR-SB score, M (SD) .1 (.3) 1.3 (1.1) 6.4 (4.0)
% Caucasian 83.3 81.1 84.6
% Female 63.7 45.9 53.6
Open in a new tab

Notes: MCI = Mild Cognitive Impairment; AD = Alzheimer’s disease; CDR-SB = Clinical Dementia Rating Scale Sum of Boxes.

Analyses revealed excellent response consistency across groups. Specifically, 1986 (98.1%) cognitively intact, 743 (97.8%) MCI, and 1493 (97.9%) AD subjects consistently reported TBI history with <5 mLOC at baseline and follow-up, with corresponding kappa coefficients of .819, .832, and .820, respectively. Regarding history of TBI with ≥5 mLOC, 2011 (99.4%) of cognitively intact, 752 (98.9%) MCI, and 1519 (99.6%) AD participants provided consistent responses to the TBI questions at time 1 and 2, with kappa coefficients of .844, .861, and .904, respectively. In these analyses, Cohen’s kappa coefficients were statistically significant (maximum p < .001; see Table 2), indicating a level of agreement beyond chance, and in fact, kappas met criteria for almost perfect agreement (Landis & Koch, 1977).

Table 2.

Reliability of self-reported history of traumatic brain injury.

Cognitively intact n = 2024 MCI n = 760 AD n = 1525
Self-reported history of TBI with LOC <5 min
  Consistent time 1 and 2, n (%) 1986 (98.1) 743 (97.8) 1493 (97.9)
  Reported TBI time 1, n (%) 109 (5.4) 55 (7.2) 99 (6.5)
  Reported TBI time 2, n (%) 116 (5.6) 54 (7.1) 91 (6.0)
  Kappa .819 .832 .820
  p <.001 <.001 <.001
Self-reported history of TBI with LOC ≥5 min
  Consistent time 1 and 2, n (%) 2011 (99.4) 752 (98.9) 1, 519 (99.6)
  Reported TBI time 1, n (%) 43 (2.1) 31 (4.1) 32 (2.1)
  Reported TBI time 2, n (%) 42 (2.1) 29 (3.8) 32 (2.1)
  Kappa .844 .861 .904
  p <.001 <.001 <.001
Open in a new tab

Notes: MCI = Mild Cognitive Impairment; AD = Alzheimer’s disease; TBI = Traumatic Brain Injury; LOC = Loss of Consciousness.

No significant differences between those with and without consistent TBI history reporting were found for age, gender, race, years of education, diagnosis (normal cognition, MCI, or AD), or CDR Sum of Boxes score (p > .05). Among the 263 subjects (6%) reporting a history of TBI with <5 mLOC at time 1, agreement was only slightly lower (82.5%). A total of 106 subjects (2%) reported a history of TBI with ≥5 mLOC during their initial visit. For those subjects, agreement was again only slightly lower (85.8%).

Discussion

Self-reported remote history of TBI collected during consecutive clinic visits appears to be reliable among older individuals with and without cognitive impairment due to MCI and AD in this national sample. This finding was consistent for self-reported mild TBI with brief (<5 min), as well as longer periods of unconsciousness following TBI. Individuals with and without consistent TBI self-reports at follow-up did not significantly differ in terms of demographics or level of cognitive impairment. Overall, these findings indicate that self-report methods are consistent over time in obtaining a history of TBI, much like other medical conditions (Donato, Boffetta, Fazioli, Gelatti, & Porru, 1998; Simpson et al., 2004; Daly et al., 2010).

Despite the positive findings, there are limitations regarding the sample that may limit generalizability. Although we used a national data-set, the sample was not population-based, as subjects were volunteers recruited through ADCs and were predominantly Caucasian and generally well-educated. Second, those with AD demonstrated only mild dementia severity; thus, these findings may not generalize to more severely impaired populations.

Another limitation involves the minimal TBI information available, though this is commonplace with many longitudinal databases that are not designed with TBI in mind. In the case of the NACC UDS, the cut-off of <5 mLOC and ≥5 mLOC used to collect TBI history is rather arbitrary and leaves open a wide range of severities. Further, a history of TBI was dependent on self-reported information, and most TBIs are mild in nature and do not result in unconsciousness. Thus, it is possible other acute symptoms leading to an inability to recall events following the injury, such as post-traumatic amnesia, may have been misinterpreted as loss of consciousness (Ruff et al., 2009). In addition, an unknown proportion of subjects may have received assistance with history reporting from a study partner. Detailed collateral information and medical documentation verifying patient TBI histories were not available in this data-set, potentially restricting generalizability of the findings.

The UDS includes participants’ responses to TBI history questions only in terms of presence or absence, without estimates of number or dates of injuries. Thus, it is unknown whether repetitive injuries occurred or when they occurred, as they could have happened as recently as almost one year prior, or have been decades earlier. Because medical documentation was unavailable to verify patients’ histories, we were without a means of verifying the accuracy of reports. Thus, the focus of this study was the test-retest reliability of self-reported TBI histories provided at two consecutive clinic visits, rather than agreement with other information sources. Last, follow-up data were restricted to one-year in order to control for the possible confound of a recent TBI event, which may limit the generalizability of these findings to longer test-retest intervals.

The literature examining TBI as a risk factor for later-in-life cognitive decline has been mixed, with the use of self-reported history sometimes cited as a limitation (Deutsch, Mendez, & Teng, 2015; Dikmen et al., 2009; LoBue, Wadsworth, et al., 2016; Sivanandam & Thakur, 2012; Xu et al., 2015). Our study is one of the first supporting the consistency of self-reported history of TBI in older adults, including those with documented MCI or AD. Histories of mild and more serious TBI were reported with high agreement across subjects between time points roughly one year apart. Overall, these results are promising and offer support for the reliability of retrospective self-report in clinical and research settings.

Acknowledgments

This work was supported in part by the Texas Institute for Brain Injury and Repair at the University of Texas Southwestern Medical Center and UT Southwestern’s Alzheimer’s Disease Center [NIA grant number P3012300-19]. Additional support was provided by the Texas Alzheimer’s Research and Care Consortium (TARCC) and the Darrell K Royal Texas Alzheimer’s Initiative, directed by the Texas Council on Alzheimer’s Disease and Related Disorders. The NACC database is funded by NIA/NIH grant number U01 AG016976. NACC data are contributed by the NIA-funded ADCs: P30 AG019610 (PI Eric Reiman, MD), P30 AG013846 (PI Neil Kowall, MD), P50 AG008702 (PI Scott Small, MD), P50 AG025688 (PI Allan Levey, MD, PhD), P50 AG047266 (PI Todd Golde, MD, PhD), P30 AG010133 (PI Andrew Saykin, PsyD), P50 AG005146 (PI Marilyn Albert, PhD), P50 AG005134 (PI Bradley Hyman, MD, PhD), P50 AG016574 (PI Ronald Petersen, MD, PhD), P50 AG005138 (PI Mary Sano, PhD), P30 AG008051 (PI Thomas Wisniewski, MD), P30 AG013854 (PI M. Marsel Mesulam, MD), P30 AG008017 (PI Jeffrey Kaye, MD), P30 AG010161 (PI David Bennett, MD), P50 AG047366 (PI Victor Henderson, MD, MS), P30 AG010129 (PI Charles DeCarli, MD), P50 AG016573 (PI Frank LaFerla, PhD), P50 AG005131 (PI James Brewer, MD, PhD), P50 AG023501 (PI Bruce Miller, MD), P30 AG035982 (PI Russell Swerdlow, MD), P30 AG028383 (PI Linda Van Eldik, PhD), P30 AG053760 (PI Henry Paulson, MD, PhD), P30 AG010124 (PI John Trojanowski, MD, PhD), P50 AG005133 (PI Oscar Lopez, MD), P50 AG005142 (PI Helena Chui, MD), P30 AG012300 (PI Roger Rosenberg, MD), P30 AG049638 (PI Suzanne Craft, PhD), P50 AG005136 (PI Thomas Grabowski, MD), P50 AG033514 (PI Sanjay Asthana, MD, FRCP), P50 AG005681 (PI John Morris, MD), P50 AG047270 (PI Stephen Strittmatter, MD, PhD).

Footnotes

Disclosure statement

No potential conflict of interest was reported by the authors.

References

  1. Daly JM, Levy BT, Joshi M, Xu Y, Jogerst GJ. Patient clock drawing and accuracy of selfreport compared with chart review for colorectal cancer (CRC) screening. Archives of Gerontology and Geriatrics. 2010;50:341–344. doi: 10.1016/j.archger.2009.05.017. [DOI] [PubMed] [Google Scholar]
  2. Deutsch MB, Mendez MF, Teng E. Interactions between traumatic brain injury and frontotemporal degeneration. Dementia and Geriatric Cognitive Disorders. 2015;39:143–153. doi: 10.1159/000369787. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Dikmen SS, Corrigan JD, Levin HS, Machamer J, Stiers W, Weisskopf MG. Cognitive outcome following traumatic brain injury. Journal of Head Trauma Rehabilitation. 2009;24:430–438. doi: 10.1097/HTR.0b013e3181c133e9. [DOI] [PubMed] [Google Scholar]
  4. Donato F, Boffetta P, Fazioli R, Gelatti U, Porru S. Reliability of data on smoking habit and coffee drinking collected by personal interview in a hospital-based case-control study. European Journal of Epidemiology. 1998;14:259–267. doi: 10.1023/a:1007463620130. [DOI] [PubMed] [Google Scholar]
  5. Folstein MF, Folstein SE, McHugh PR. ‘Mini-mental state’. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research. 1975;12:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
  6. Gardner RC, Burke JF, Nettiksimmons J, Kaup A, Barnes DE, Yaffe K. Dementia risk after traumatic brain injury vs nonbrain trauma: The role of age and severity. JAMA Neurology. 2014;71:1490–1497. doi: 10.1001/jamaneurol.2014.2668. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Kerr ZY, Marshall SW, Guskiewicz KM. Reliability of concussion history in former professional football players. Medicine and Science in Sports and Exercise. 2012;44:377–382. doi: 10.1249/MSS.0b013e31823240f2. [DOI] [PubMed] [Google Scholar]
  8. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. [PubMed] [Google Scholar]
  9. LoBue C, Denney D, Hynan LS, Rossetti HC, Lacritz LH, Hart J, Jr, Cullum CM. Self-reported traumatic brain injury and mild cognitive impairment: increased risk and earlier age of diagnosis. Journal of Alzheimer’s Disease. 2016;51:727–736. doi: 10.3233/JAD-150895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. LoBue C, Wadsworth H, Wilmoth K, Clem M, Hart J, Jr, Womack KB, Cullum CM. Traumatic brain injury history is associated with earlier age of onset of Alzheimer disease. The Clinical Neuropsychologist. 2016;31:85–98. doi: 10.1080/13854046.2016.1257069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. LoBue C, Wilmoth K, Cullum CM, Rossetti HC, Lacritz LH, Hynan LS, Womack KB. Traumatic brain injury history is associated with earlier age of onset of frontotemporal dementia. Journal of Neurology, Neurosurgery, and Psychiatry. 2016;87:817–820. doi: 10.1136/jnnp-2015-311438. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Morris JC. The Clinical Dementia Rating Scale (CDR): Current version and scoring rules. Neurology. 1993;12:2412–2414. doi: 10.1212/wnl.43.11.2412-a. [DOI] [PubMed] [Google Scholar]
  13. Morris JC, Weintraub S, Chui HC, Cummings J, DeCarli C, Ferris S, Kukull WA. The Uniform Data Set (UDS): Clinical and cognitive variables and descriptive data from alzheimer disease centers. Alzheimer Disease and Associated Disorders. 2006;20:210–216. doi: 10.1097/01.wad.0000213865.09806.92. [DOI] [PubMed] [Google Scholar]
  14. Ruff RM, Iverson GL, Barth JT, Bush SS, Broshek DK, Policy NAN Committee, Planning. Recommendations for diagnosing a mild traumatic brain injury: A National Academy of Neuropsychology education paper. Archives of Clinical Neuropsychology. 2009;24:3–10. doi: 10.1093/arclin/acp006. [DOI] [PubMed] [Google Scholar]
  15. Simpson CF, Boyd CM, Carlson MC, Griswold ME, Guralnik JM, Fried LP. Agreement between self-report of disease diagnoses and medical record validation in disabled older women: factors that modify agreement. Journal of the American Geriatrics Society. 2004;52:123–127. doi: 10.1111/j.1532-5415.2004.52021.x. [DOI] [PubMed] [Google Scholar]
  16. Sivanandam TM, Thakur MK. Traumatic brain injury: A risk factor for Alzheimer's disease. Neuroscience and Biobehavioral Reviews. 2012;36:1376–1381. doi: 10.1016/j.neubiorev.2012.02.013. [DOI] [PubMed] [Google Scholar]
  17. Wojtowicz M, Iverson GL, Silverberg ND, Mannix R, Zafonte R, Maxwell B, Berkner PD. Consistency of self-reported concussion history in adolescent athletes. Journal of Neurotrauma. 2017;34:322–327. doi: 10.1089/neu.2016.4412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Xu W, Tan L, Wang HF, Jiang T, Tan MS, Tan L, Yu JT. Meta-analysis of modifiable risk factors for Alzheimer’s disease. Journal of Neurology, Neurosurgery, and Psychiatry. 2015;86:1299–1306. doi: 10.1136/jnnp-2015-310548. [DOI] [PubMed] [Google Scholar]

RESOURCES