Correspondence of the Boston Assessment of Traumatic Brain Injury-Lifetime (BAT-L) and the VA Comprehensive TBI Evaluation (CTBIE)

Abstract

Objective

To compare the diagnosis of positive versus negative for mild traumatic brain injury (mTBI) using the Boston Assessment of TBI-Lifetime (BAT-L), a validated forensic clinical interview used to identify TBI in research, to the diagnosis of mTBI in the clinical Polytrauma Service using the Comprehensive TBI Evaluation (CTBIE).

Participants

OEF/OIF/OND Veterans who were enrolled in the TRACTS longitudinal cohort study and received a CTBIE at a Veterans Health Administration (VHA) healthcare facility (n=104).

Main Measures

BAT-L, CTBIE, Neurobehavioral Symptom Inventory.

Results

There was poor correspondence between the BAT-L and CTBIE mTBI diagnoses (kappa=0.283). The Second Level Evaluation showed moderate sensitivity but poor specificity relative to the BAT-L. The agreement did not improve after removing individuals who had failed symptom validity measures, as assessed by the Validity 10 scale of the NSI.

Conclusions

This lack of correspondence highlights the difficulties in diagnosing mTBI in Veterans using retrospective self-report. Future work is needed to establish a reliable and valid method for identifying military mTBI both for the care of our Veterans and for appropriate distribution of benefits.

Introduction

Mild traumatic brain injury (mTBI) in Veterans returning from Operation Enduring Freedom (OEF), Operation Iraqi Freedom (OIF) and Operation New Dawn (OND) has been the focus of intense research over the last decade; however, there is still debate regarding how to best assess military concussion retrospectively. The detection of TBI, and mTBI in particular, in returning Veterans is primarily reliant on self-report of the injury months to years after the event occurred. Combat injuries often occur in stressful contexts and involve disrupted mental status and/or memory. Recall of the events is therefore vulnerable to distortion, which could lead to a misdiagnosis if based solely on self-report.

The current Department of Veteran’s Affairs (VA) TBI protocol, in place since 2007, mandates that all returning Veterans deployed after September 11^th, 2001 receive the VA TBI Clinical Reminder Screen (1). Individuals who screen positive on the VA TBI screen are referred for further evaluation known as the VA Comprehensive Traumatic Brain Injury Evaluation (CTBIE) (1). The CTBIE follows a specific template; however, there are no explicit guidelines offered to clinicians to help them determine whether the Veteran should receive a positive diagnosis of TBI (2). The lack of systematic definitions and procedures leaves open the potential for inaccuracies and/or inconsistencies.

We have recently validated the Boston Assessment of TBI-Lifetime (BAT-L) (3), which could be used adjunctively to improve the validity of the CTBIE. The BAT-L TBI diagnosis is determined by the Department of Defense criteria which include the presence of at least one of the following: altered mental status (AMS), posttraumatic amnesia (PTA), or loss of consciousness (LOC) following trauma to the head (4). The BAT-L has demonstrated high inter-rater reliability and high correspondence with other validated TBI detection methods (3) including the VA TBI screen when administered in a research setting, but poor correspondence with the VA TBI screen when administered clinically (5). There is an empirical base of evidence demonstrating a correlation between TBI diagnosis and blast exposure as documented by the BAT-L and objective measures of neurological and neuropsychological outcomes that are representative of TBI sequelae (6, 7). We argue the BAT-L can serve as a “gold standard” TBI diagnostic tool to evaluate the efficacy of the CTBIE.

The present study is an extension of previous work that compared the BAT-L to the VA TBI screen (5). This study compared the TBI diagnoses as determined by the CTBIE obtained in a VA Polytrauma Network Site to the diagnoses obtained using the BAT-L in a research setting. Reasons for discrepant diagnoses are explored and the impact of possible symptom exaggeration is considered.

Methods

Participants

All participants were enrolled in a longitudinal cohort study at the Translational Research Center for TBI and Stress Disorders (TRACTS) National Research Center (NRC) (8) at the VA Boston Healthcare System. The present study evaluated the first 109 consecutively enrolled participants who had also received a CTBIE at a VA healthcare facility. On average participants received their CTBIE 1.09 years before their TRACTS visit, and the exact time of the CTBIE ranged from 5.27 years before to 3.09 years after their TRACTS research visit. Five Veterans were excluded from statistical analyses because they experienced a military TBI outside of deployment (e.g., stateside or during a training mission), leaving a final sample size of 104 (96M/8F). These 5 Veterans were excluded because the CTBIE does not account for TBIs that occur outside of deployment, whereas the BAT-L considers these military-related injuries. Thus, including these individuals would have exaggerated the discrepancy between measures.

Participants ranged in age from 21–61 years (M=31, SD=8) and on average received 13.6 years of education (M=13.6, SD=1.7) (Table 1). The TRACTS longitudinal cohort and study procedures have been well-characterized elsewhere (8). Most Veterans received their CTBIE at a VA Boston Healthcare System hospital. Others received their CTBIE at other VA Polytrauma clinics located throughout the country, allowing for better representation of general US VHA procedures.

Table 1.

Demographic Information

	Mean (SD)
Age, mean (SD) years	31 (8.0)
	range= 21–61
Education, mean (SD) years	13.6 (1.7)
	range= 12–19
Gender
Female	8 (7.7%)
Male	96 (92.3%)
Ethnicity
White	78.1%
Hispanic/Latino	14.3%
African American	3.8%
American Indian	1.0%
Unknown	2.9%

Procedures

TBI diagnoses were obtained from the BAT-L interview and CTBIE.

BAT-L

The BAT-L employs a forensic approach to determine presence and duration of acute TBI signs at the time of each possible TBI event (e.g., AMS, PTA, LOC). There are multiple queries and probes for each event to help the interviewer develop a timeline to determine duration of these signs. Other factors that might be interpreted as alterations of consciousness (e.g., chaos and confusion due to explosions, sensory changes, psychological response, substance use) are queried. Details specific to injuries in military settings are queried such as use of protective gear, reports of medics if known, and changes to duty post-injury. Presence and duration of post-concussive symptoms (PCS) post-event are recorded. Veterans are also queried about blast exposure in addition to blast TBI. The number of blasts they have been exposed to at close (0–10m), medium (11–25m) and far (25–100m) distances are documented. Duration of acute signs of TBI (AMS, PTA, LOC) is used to determine the injury severity.

CTBIE

The CTBIE does not ask for a detailed description of the injury, but it does inquire about AMS, PTA, and LOC at the time of the event. Veterans are also asked if they are currently experiencing PCS such as headaches, dizziness, memory impairment, etc. This is followed by a physical examination and brief assessment of current psychological functioning such as orientation, social interaction, and communication.

For a complete description of the BAT-L or CTBIE see (3) or (1) respectively.

Symptom Validity Measures

NSI

Participants completed the NSI as part of their TRACTS longitudinal assessment and as part of their CTBIE assessment. Symptom Validity was coded as pass or fail based on an individual’s Validity 10 score, as per Vanderploeg and colleagues (9).

Statistical Analyses

Participants’ diagnoses from the BAT-L and the CTBIE were coded as positive or negative and compared using χ² in SPSS. Results are reported as the CTBIE diagnosis compared to the BAT-L diagnosis.

Results

Correspondence between the BAT-L and the CTBIE (Table 2)

Table 2.

OEF/OIF/OND Deployment Injuries: Comparison of CTBIE TBI diagnosis to the BAT-L TBI diagnosis in 104 OEF/OIF/OND Veterans

VA CTBIE	BAT-L for military TBI during OEF/OIF/OND deployment(s)
	Positive for TBI	Negative for TBI	Total
Positive for TBI	58 (true positive)	13 (false positive)	71
Negative for TBI	18 (false negative)	15 (true negative)	33
Total	76	28	104

Overall, the BAT-L positively diagnosed 73% of Veterans in this sample with TBI compared to the Secondary Evaluation that diagnosed 68% of Veterans with TBI. To assess agreement between the BAT-L and CTBIE, the Cohen κ coefficient and the Kendall τ coefficient were calculated. Both measures indicated only fair consistency between the two assessments (Cohen κ=0.283; Kendall τ −b= 0.285; Table 2). The CTBIE demonstrated diagnostic agreement with BAT-L TBI diagnosis for 73 of 104 individuals (Positive TBI diagnosis agreement = 58 Veterans; negative TBI diagnosis agreement = 15 Veterans). Disagreements between the two instruments occurred for approximately 30% of cases. Specifically, 18 individuals received a positive diagnosis for TBI based on the BAT-L interview, but were not diagnosed with a TBI based on their CTBIE (false negatives). Thirteen Veterans were not diagnosed with a TBI based on the BAT-L interview, but received a positive diagnosis based on the CTBIE (false positives). Using the BAT-L as the validated gold standard, these data suggest that the Secondary Evaluation has moderate sensitivity to detect TBI (76.3% sensitivity) but poor specificity (53.6%).

Description of disagreements in diagnosis

Based on the injury description provided in the reports for both measures, we could verify that the BAT-L and CTBIE assessed the same index injury in 13 of 13 of the false positives and 14 of 18 false negatives. Disagreement in diagnoses fell into four categories: Errors, Inconsistent Reporting, Confounding Factors, and TBI diagnosis made from report of PCS versus acute TBI diagnostic criteria (Table 3). Many discrepancies were due to inconsistent reporting by Veterans, which may be confounding the results. Analyses with these individuals excluded found that the correspondence between the two measures increased, but there were still significant discrepancies between measures (χ² = −19.08, p <.01; Cohen κ=0.453 Kendall τ −b= 0.453).

Table 3.

Descriptions of Diagnosis Disagreement between the BAT-L and CTBIE

Category	Number of Cases	Description	Example(s)
Errors	8	Explicit factual mistake in one or the other assessment	Two cases received positive diagnoses based on the CTBIE despite the fact that the CTBIE report explicitly stated that those individuals did not experience the signs necessary for diagnosis according to the DoD criteria. Six cases were negative for TBI diagnosis despite that the CTBIE specifically noted that the patient experienced the signs associated with a concussion.
Inconsistent Reporting	11	The Veteran reported different signs across assessments	Veterans denied any TBI signs at one assessment, but reported them at the other (3 false positives and 8 false negatives).
Confounding Factors	5	Circumstances surrounding the injury may have clouded the Veteran’s interpretation of his/her symptoms	One Veteran reported feeling confused after a blast woke him up in the middle of the night. This was considered AMS based on the CTBIE; however when further queried on the BAT-L, he reported that he was able to respond appropriately/perform duties as expected within seconds of awakening. His confusion was thought to be related to being awoken during a chaotic situation rather than related to acute AMS caused by a TBI.
PCS vs TBI Diagnostic Criteria	3	Veterans reported PCS, but did not report acute AMS, PTA, or LOC at the time of the injury	Veterans denied any signs of TBI at the time of their injury, but reported symptoms such as headaches, dizziness, and memory problems persisting after the incident. These symptoms are often secondary to other causes, such as the stress or chaos of the combat situation, dehydration, lack of sleep, etc.

Symptom Validity Analyses

To determine the role of symptom exaggeration in the poor correspondence between the BAT-L and the CTBIE in clinical and research visits, we conducted additional χ² comparisons after excluding 36 individuals who were missing or had failed the NSI symptom validity measure at either visit. Only 2 of the Veterans who had failed the symptom validity measures had discrepant research versus clinical diagnoses. The poor correspondence between the two measures remained (Cohen κ=0. 234; Kendall τ −b=0.238).

Discussion

This study found poor correspondence of TBI diagnosis between the research administration of the BAT-L and the clinically administered CTBIE. Correspondence did not improve when Veterans who failed the embedded symptom validity measure at either assessment were removed from analyses. The disagreement between the BAT-L and CTBIE was therefore not thought to be due to issues with engagement or symptom exaggeration, indicating that the potential for monetary incentive (service connection status) that may be present during the CTBIE did not seem to affect Veterans’ report of TBI signs.

These findings suggest that the addition of key aspects of the BAT-L approach to TBI assessment to the CTBIE could help structure clinicians’ interviews and improve the latter’s sensitivity to detect mTBI and reject possible exposures that did not result in mTBI. Clinicians can continue to use the CTBIE in its valid format, but they may benefit from incorporating additional probes to obtain a detailed timeline for each possible injury and evaluate functioning immediately after injury (e.g., duration of acute TBI signs such as AMS, PTA, and LOC) to avoid confounding the diagnosis by considering symptoms that may have developed later due to other factors (such as long-standing nonspecific PCS). Clinicians could also be trained to use probes to discern AMS from common potential military confounds (e.g., chaos secondary to combat). It may be beneficial for the CTBIE examiners to query about blast exposure (without resultant TBI) and non-combat related mTBIs, since these injuries can impact the Veterans’ current functioning (10, 11).

There were two primary factors at play that contributed to the disagreement between assessments. First, a great deal of the inconsistency between the two assessments may be attributed to human error, most likely due to the lack of clear guidelines for TBI diagnosis in the CTBIE. The CTBIE computer template now “flags” these inconsistencies, which should reduce examiner error in diagnosis. The additional clinical training described above may also reduce these errors. Second, incongruity was also due to inconsistent reporting by the Veterans. Reliance of TBI diagnosis on self-report of TBI signs long after an event is an inherent problem in TBI research (12). However, even when individuals who reported inconsistent signs between the two assessments were removed, there were still meaningful differences between the two measures.

The primary limitation of this study was that both assessments of TBI referred to events that are remote, in some cases months to years prior to the assessment. As such, responses may have been influenced by individuals’ memory of the injury. Despite this limitation, self-report via semi-structured interview by trained clinicians remains the gold standard for TBI diagnosis (12) and assessment of remote TBI is commonly required in actual clinical practice. Another limitation is the inherent difficulty of diagnosing TBI remotely when it occurred in a confusing and stressful combat setting. It is possible that trauma associated with the incident injury itself, or the surrounding combat, and not a direct injury to the brain better accounts for the Veterans’ altered mental status both at the time of the event and their recurring problems with function and self-reported symptoms.

Future research is necessary to compare the clinical outcomes of Veterans diagnosed with TBI on each of these measures to determine which measure is more accurate in identifying Veterans at risk for future functional decline. Part of our mission at TRACTS is to examine the relations between TBI indices (as assessed by the BAT-L) as well as other co-occurring conditions and functional outcome post-deployment. Such longitudinal information will provide the necessary data needed to set clinical goals to improve service members' overall functionality. TBI does not occur in isolation; therefore, the many co-occurring conditions and their effects on function must also be considered when assessing the impact of TBI on outcome.