Pre-injury assessment of everyday executive function in moderate to severe traumatic brain injury

Abstract

Executive dysfunction is frequently observed in moderate to severe traumatic brain injury (TBI) and is commonly assessed with objective measures or subjective rating scales. Given the variability in executive function in the normal population, a reliable measure of pre-injury executive function would be of considerable value. In this study we examined pre-injury self and collateral (relative or friend) ratings on the Frontal Systems Behavior Rating Scale (FrSBe). Fifty-one persons with moderate to severe TBI and their collaterals provided before- and after-TBI ratings at 3 months post injury. A subset of 36 dyads were retested at 6 and 12 months; 26 neurologically intact controls and their collaterals also provided FrSBe ratings. At 3 months post injury, the difference between patient and collateral ratings of current status was larger than the difference relating to premorbid status, suggesting that patients were able to rate themselves from a pre-injury perspective. However, pre-injury ratings from collaterals were more reliable over time compared to patients’ self-ratings.For all sets of ratings—before injury, after injury, and status of healthy controls—collateral ratings indicated more abnormality, overall, than comparable self-ratings. Evaluating one’s own executive behaviour may be a difficult task even without TBI, with the difficulty exacerbated by the effects of injury.

Introduction

Moderate to severe traumatic brain injury (TBI) causes cognitive and behavioural deficits that contribute to long-standing limitations in societal participation. Deficits in executive functions, the regulatory mechanisms needed to enable appropriate goal-directed behaviour, are frequently observed but notoriously difficult to measure, for several reasons. First, executive functions encompass a broad range of control operations that have proved difficult to label and categorize. Second, the structured setting of a typical neuropsychological assessment may not allow for accurate assessment of the very functions that allow us to adapt to the changing demands of everyday social and physical environments. Third, there is evidence that executive functions are quite variable across healthy individuals, and that such variability is related to differences in life outcomes (Diamond, 2013). This variation probably has advantages for the species—for example, we benefit from having some people who initiate behaviour in risky situations and others who are more prone to stay back and “hold the fort.” With regard to measuring the status of an individual with a TBI, however, it can be difficult to estimate the degree of change in executive function if one cannot confidently assign a normative value to all individuals prior to injury.

Alternative and supplemental methods of measuring executive function have been developed in response to these challenges. There now exist standardized measures that attempt to capture some of the complexity of real-world cognitive demands (e.g., the Multiple Errands Test; (Clark, Anderson, Nalder, Arshad, & Dawson, 2015). Rating scales such as the Dysexecutive Questionnaire (Burgess, Alderman, Evans, Emslie, & Wilson, 1998) and the Frontal Systems Behaviour Scale (FrSBe; (Grace & Malloy, 2001) are designed to quantify the frequency of everyday behaviours aligned with executive function or its impairment. The FrSBe further distinguishes three aspects of disordered cognition and behaviour associated with executive dysfunction in subscales derived from factor analytic research: Apathy and Disinhibition, which capture diminished and excessive/ inappropriate behaviour, respectively, and Executive Dysfunction, which captures the cognitive aspects of the construct as expressed in everyday behaviour. These scales may be completed by the patient and/ or a collateral respondent, with the discrepancy between them often interpreted as impaired self-awareness on the patient’s part (Niemeier et al., 2014). Measures of everyday executive behaviours appear to capture unique components of executive function, as their association to neuropsychological or laboratory executive measures may be modest (Lengenfelder, Arjunan, Chiaravalloti, Smith, & DeLuca, 2015). In one TBI study, self-reported scores on the Apathy and Executive Dysfunction subscales were inversely associated with extent of community integration, whereas objective neuropsychological measures failed to show a similar relationship (Reid-Arndt, Nehl, & Hinkebein, 2007).

Interestingly, the FrSBe has also proven to be capable of capturing meaningful variations in “normal” executive function. In an unselected sample of 127 community dwelling adults, the self-reported Executive Dysfunction subscale score was a significant predictor of credit card debt (Spinella, Yang, & Lester, 2004). In another study, the Total score and the Apathy subscale discriminated healthy participants from those affected by substance abuse (Pluck et al., 2012); notably, some of the healthy controls scored within the clinically elevated range. Discrepancies between self- and other-ratings on the FrSBe may be meaningful in normal populations, as well: healthy adults who under-reported problems on the FrSBe compared to relatives’ ratings performed more poorly on a sustained attention task compared to “accurate” responders (Hoerold et al., 2008).

The FrSBe has other potentially advantageous features for research on executive function. Its favourable psychometric characteristics led to inclusion in the TBI Common Data Elements outcome measures (Wilde et al., 2010). Moreover, for each item on the rating form, respondents have the option to rate the patient both currently, and prior to the injury or onset of the condition. Given the variability in executive capability in the normal population, comparing a patient to his or her pre-injury profile might be more relevant than using population norms, and offers a new approach to comparing patient and collateral reports. One study showed that a similar degree of change from before to after TBI was reported by patients and collaterals at 1 year post injury (Lengenfelder et al., 2015). Both sets of after-injury ratings were generally in the abnormal range, indicating that by this time patients were aware of dysexecutive problems that were also reported by relatives.

When considering a set of ratings about an individual before injury, the question arises: How reliable are such ratings? It is well known that retrospective assessments, either of oneself or of other people, may be affected by factors known collectively as response shift. Response shift occurs when a significant life event, such as an injury or illness, leads to re-calibration of one’s internal standards of measurement or re-prioritization of values (Schwartz et al., 2013). For example, quality of life might be rated as improved after a debilitating illness if one has gained an appreciation of different aspects of life. With respect to executive function, premorbid ratings could be shifted toward the positive by a tendency to view the person as better-behaved than he or she really was (the “good old days” phenomenon). Conversely, if the person exhibits executive dysfunction currently, evaluation of pre-injury status could shift toward the negative to downplay the significance of the problem (“I was always like that”). Of course, the accuracy of pre-injury ratings could be affected by other factors, such as lack of familiarity with the person in the case of collateral raters, or cognitive disturbance in the case of self-ratings.

The current study had several objectives. We wished to provide descriptive data on the pre-injury FrSBe ratings provided by people with moderate to severe TBI, as well as those of collateral raters who knew them well, as such data have appeared infrequently in the literature. We collected pre-injury data relatively soon after injury in an attempt to minimize distortions due both to memory failure and to response shift, which can evolve over time. Another objective was to examine the reliability of the premorbid FrSBe scores, which we approached in two ways. First, we examined the intra-rater consistency of each set of before-injury scores (patient, collateral) across 3 time points during the first year after the TBI. We reasoned that high consistency would confer more trust in the “accuracy” and thus, the utility of the ratings to help determine changes due to TBI. Secondly, we compared patients and collateral raters on both before- and after-injury ratings at the earliest time point. We hypothesized that the difference between raters soonafter injury, when many patients would be expected to exhibit limited awareness of executive dysfunction (Hart, Sherer, Whyte, Polansky, & Novack, 2004; Niemeier et al., 2014), would exceed the difference in the pre-injury scores, providing indirect validation of the pre-injury ratings. We also predicted that the after-injury difference between raters, but not the before-injury difference, would exceed the rater differences derived from healthy controls and their collateral raters, recruited from socioeconomic backgrounds similar to those of the patient sample.

Method

Participants

51 persons with moderate to severe TBI and 26 uninjured controls participated in this study. Participants with TBI were drawn from 2 studies that recruited from an inpatient rehabilitation unit, for testing sessions scheduled to occur 2–3 months post injury. The first study, which contributed 36 participants, had the following inclusion criteria: (1) TBI with at least one of the following: Glasgow Coma Scale (GCS) score <13 in the Emergency Department (not due to sedation, paralysis, or intoxication), documented loss of consciousness for ≥ 12 hours, or prospectively documented post-traumatic amnesia (PTA) ≥ 24 hours; (2) age 18–64; (3) fluent in English; (4) able to participate in testing and interviewing (i.e., not grossly confused, restless, or unresponsive). Participants were excluded for: (1) history of prior TBI, CNS disease, seizure disorder, schizophrenia, or bipolar disorder; (2) history of long-term abuse of alcohol or stimulants (e.g., cocaine) likely to have resulted in neurologic sequelae. The remaining 15 participants were drawn from the second study, which had similar criteria except that participants were included with loss of consciousness > 1 hr or any positive neuroimaging related to trauma; there was no upper limit to age; and participants with history of drug or alcohol abuse were not excluded.All participants had cleared PTA prior to testing.

Neurologically intact controls were recruited from relatives and friends of the participants with TBI, from non-professional hospital staff, and from public advertising in local newspapers. Control participants were eligible if they had no history of TBI with loss of consciousness. They were also excluded for the same other conditions as were the participants with TBI (i.e., bipolar disorder, schizophrenia, or any condition affecting the CNS, including long-term alcohol or stimulant abuse resulting in neurologic sequelae). Controls were not matched 1:1 to patients, but periodic analyses of patient group characteristics allowed us to target the recruitment of controls so that the samples would be comparable on socio-demographic variables such as age, gender, race/ ethnicity, and education.

Each patient and control participant was asked to nominate a significant other (SO), a relative or friend who knew them well and could provide ratings of their behaviour for comparison to self-ratings.

Measures

Demographic variables were abstracted from the medical record or obtained by interview. Injury variables were abstracted from medical records and included cause of injury and Glasgow Coma Scale (GCS) score on presentation to the Emergency Department. Time to follow commands (TFC) was determined by the first date that the individual was able to follow simple motor commands accurately at least two times consecutively in a 24-hour period. Duration of post-traumatic amnesia (PTA), a sensitive index of the severity of neurologic injury, was calculated as the number of days between the TBI and the first of two occasions within 72 hours that the participant was fully oriented. Full orientation was defined as a score above 25 on the Orientation Log (Jackson, Novack, & Dowler, 1998), or documentation of consistent orientation for 72 hours in the acute medical record (i.e., prior to rehabilitation admission).

The 46-item FrSBe was administered to each participant and separately to the SO, using the appropriately worded forms that are provided with the test materials. Items on the scale are both positively and negatively worded and measure the frequency of each behaviour from “almost never” to “almost always.” Sample negatively worded items are, from the Apathy scale: “Have lost interest in things that used to be fun or important to me;” from the Disinhibition scale, “Do risky things just for the heck of it;” and from the Executive Dysfunction scale, “Mix up a sequence, get confused when doing several things in a row.” Raw scores for the 3 subscales and the Total score are converted to gender, age, and education corrected T scores with mean of 50 and SD of 10, where higher scores indicate more dysfunction. The FrSBe has shown good reliability and validityat the total score and subtest level in a variety of clinical groups and settings (Grace & Malloy, 2001),and compares favourably in these respects with other scales designed for a similar purpose (Malloy & Grace, 2005).The factor structure of the scale has been validated using informant ratings on two large samples of persons with neurologic diseases (Carvalho, Ready, Malloy, & Grace, 2013; Stout, Ready, Grace, Malloy, & Paulsen, 2003).

Procedure

This study complied with all institutional standards and federal regulations governing human research. Each participant, including SOs, provided informed consent. Depending on the parent study from which they were included, participants with TBI and their SOs were administered the FrSBe on either one occasion timed for within 2 months post TBI (n = 15), or on 3 occasions timed for 3, 6, and 12 months post injury (n = 36). Data from the 2 and 3 month testing occasions were combined to create a subacute sample with N = 51. The 26 controls and their SOs were tested only once. To establish consistency between in-person and telephone administrations, and to obviate literacy problems, the scale was administered orally with a research assistant marking each response. For the patients and their SOs, each item was recorded for before injury (Before) followed by the same item pertaining to “now” (After). Controls and their SOs provided only current ratings.

Data analysis

We used t-tests (for interval variables) and Chi-square tests (for categorical variables) to compare participants with TBI and controls with respect to demographic characteristics. Chi-square tests or MacNemar’s tests, as appropriate, were used to compare the proportions of each FrSBe scale, for each sample and rater type, that exceeded the recommended clinical cutoff of ≥ 65 (Grace & Malloy, 2001). Intraclass correlation (one-way single measure or ICC[1,1]) was used to assess the consistency of FrSBe Before ratings across the 3 longitudinal time points. Difference scores for the FrSBe scores were calculated by subtracting (After – Before) for the time comparisons, and (SO – Self) for the rater comparisons. Paired t-tests or t-tests for independent samples, as appropriate, were used to compare these difference scores. Alpha was set at .05 throughout, as these analyses were considered exploratory.

Results

Participant characteristics are displayed in Table 1. Patient and control samples were well balanced as to age, education, gender, and race/ ethnicity. Participants with TBI were more likely than controls to include spouses or parents, versus other relatives or friends, as collateral raters; this was no doubt due to the involvement of close relatives in the rehabilitation setting. However, the gender distribution was similar in the two groups (about three-quarters female). The severity indices confirm that the patient sample had moderate to severe TBI. As shown in the Table, testing was first performed at just under 3 months post TBI, on average. For the longitudinal sample, testing was done at a mean of 185 days (+/− 17) and 360 days (+/− 36) for the 6 and 12 month evaluations, respectively.

Table 1.

Participant characteristics

	TBI Sample		Control Sample
Demographic Variables	(n = 51)		(n = 26)
Age (M/ SD)	37.5	16.3	34.3	13.5
Gender (no. / % male)	35	69	17	68
Education (years; M / SD)	13.6	2.5	13.5	2.3
Race/ Ethnicity (no. / % white)	21	41.2	9	39.0
Relationship of collateral to patient (no./%):
Spouse/romantic partner	13	25.5	3	11.5
Parent	23	45.1	9	34.6
Other relative	11	21.6	8	30.8
Close friend	4	7.8	6	23.1
Gender of collateral respondent (no./% female)	40	78.4	19	73.1
Injury Variables
Mechanism of injury1 (no./%):
Vehicular incident	33	64.7	--	--
Fall	11	21.6
Intentional injury	6	11.8
GCS on admission to ED (M/SD)2	9.9	4.5	---	---
Time to follow commands, days (M/ SD)	7.4	11.3	---	---
Duration of post-traumatic amnesia, days (M/SD)	26.9	20.4	---	---
Interval between injury and testing, days (M/SD)	87.9	31.4	---	---

M = Mean; SD = Standard deviation; GCS = Glasgow Coma Scale; ED = Emergency Department

¹Mechanism was unknown for one case

²n = 35; 16 values were missing, primarily due to sedation/ intubation in the ED

Table 2 shows the proportions of Before and control T-scores that exceeded the cutoff for clinical abnormality (Grace & Malloy, 2001). None of the comparisons between the TBI and control samples reached statistical significance; that is, participants with TBI rated themselves as roughly comparable to controls prior to the TBI, and collateral ratings were also comparable across samples (all p values > .20). However, the collaterals of the patients with TBI rated nearly 7 times more patients as abnormal on the Apathy subscale prior to injury, compared to the proportion of abnormal self-ratings. This difference was highly significant and probably accounts for the difference also observed on the FrSBe Total score. Interestingly, the same pattern was seen in the control sample: collaterals rated proportionately many more participants as abnormal on the Apathy scale, although the difference did not achieve significance in this smaller sample (p = .11). Inspection of Table 2 reveals that in fact, all proportions of abnormal scores were higher for the collateral raters than the self-raters, in both TBI and control samples.

Table 2.

Proportions (%s) of Before-injury and control FrSBe scale T-scores in clinically abnormal range (≥65)

Sample	Apathy	Disinhibition	Executive Dysfunction	Total
TBI
Self-Ratings	3.9**	17.6	9.8	11.8*
Collateral Ratings	27.4**	21.6	21.6	31.4*
Control
Self-Ratings	7.7	7.7	15.4	15.4
Collateral Ratings	30.8	15.4	23.1	26.9

^*Proportions different at p < .02

^**Proportions different at p < .005

Note: In a normal distribution of T-scores with mean = 50 and standard deviation = 15, 6.7% of values would be ≥ 65.

The intraclass correlations for the Before ratings across the 3 time points are shown in Table 3. All of the collateral ratings achieved values in the range generally accepted to indicate good reliability, or in this case, consistency (≥ 0.75;(Koo & Li, 2016). In contrast, the self-ratings included two scales in the range of poor consistency (Apathy and Executive Dysfunction), one moderate (Total), and one good (Disinhibition). An analysis of trend did not reveal any systematic directional tendencies in the change of either self- or collateral-ratings over time (data not shown).

Table 3.

Intraclass correlations for self and collateral Before-injury ratings, calculated across 3 time points post injury

	Apathy	Disinhibition	Executive Dysfunction	Total
Self-Ratings	0.48	0.78	0.46	0.67
Collateral Ratings	0.84	0.89	0.75	0.88

The summary statistics for the FrSBe ratings from the subacute testing period and the control participants are displayed in Table 4, along with the results of significance tests. For the TBI sample, all of the collateral – self differences were significantly greater for the After injury ratings compared to the Before injury ratings. However, none of the control collateral – self differences were significantly discrepant from either the Before or After TBI difference scores (all p > .30). Interestingly, all of the mean difference scores were positive, indicating that even controls’ relatives tended to view participant behaviour in a more abnormal light than participants did. However, the large standard deviations indicate considerable variation in the difference scores.

Table 4.

FrSBe T scores and (collateral – self) difference scores for each subscale and Total score

FrsBe Scale	Control (N = 26)	TBI (N = 51)		Significance (Difference scores)
		Before Injury	After Injury
Apathy: Self	48.3 (11.3)	47.2 (9.4)	60.0 (17.5)	TBI A>B p=.02
Collateral	58.3 (15.3)	53.7 (14.8)	72.2 (20.3)
Difference	10.0 (17.7)	6.4 (15.1)	12.3 (18.9)
Disinhibition: Self	49.2 (12.7)	53.0 (14.7)	53.8 (17.0)	TBI A>B p<.001
Collateral	51.6 (13.6)	53.9 (14.6)	61.4 (18.8)
Difference	2.4 (18.0)	0.90 (15.2)	7.6 (17.0)
Executive Dysfunction: Self	48.8 (13.6)	50.0 (13.6)	56.2 (15.5)	TBI A>B p<.001
Collateral	54.5 (14.8)	54.5 (12.9)	67.0 (17.4)
Difference	5.7 (21.7)	4.5 (15.0)	10.8 (17.8)
Total: Self	48.7 (13.6)	50.1 (12.4)	57.5 (17.0)	TBI A>B p<.001
Collateral	55.4 (14.9)	54.5 (14.2)	69.3 (20.0)
Difference	6.7 (21.8)	4.4 (15.0)	11.8 (19.3)

Values in cells are means (standard deviations)

B = Before injury; A = After injury

Discussion

In this study we examined pre-injury ratings of executive function (FrSBe scores) in moderate to severe TBI from several perspectives. We hypothesized that the discrepancy between self-ratings and collateral ratings of persons with subacute, moderate to severe TBI would be larger than the comparable discrepancy calculated on pre-injury ratings. This hypothesis was confirmed for all FrSBe subscales as well as the Total score. Moreover, pre-injury discrepancy scores for persons with TBI did not differ from those of a demographically similar sample of healthy controls (nor did controls’ discrepancy scores differ from those for the patient sample post-injury, a somewhat surprising result). These findings suggest that both the persons with recent TBI and their collateral raters were able to use the frame of reference of “before the TBI”—a time when the patients would have resembled the control participants as to their executive behaviours—to complete the task. It was notable that all of the mean T-scores for the Before and control ratings (from both types of respondents) were within normal limits, i.e., 50 +/− 10 points, whereas all TBI collaterals’ means exceeded this range for the After ratings (Table 4). However, it was also observed that for both patient and control groups, all mean discrepancy scores were positive, meaning that there was a consistent tendency for collaterals to rate participants as more impaired than they rated themselves—whether before, after, or without TBI. This pattern was additionally borne out in the examination of proportions of participants who scored in the clinically abnormal range via self- or collateral ratings: all such proportions were substantially higher for collateral than for self-ratings, although the difference was statistically significant only for the Apathy and Total scales in the TBI sample.

Why do collateral raters tend to identify more dysexecutive behaviour than self-raters do? For patients with TBI, we routinely interpret this trend as impaired self-awareness of deficit. In the case of before-injury ratings, it is also possible that collaterals’ judgments of past behaviour is tempered by observation of current, injury-related problems (i.e., “he was always like that” response shift). However, this cannot explain why the same trend was seen in control participants. An interesting line of research has shown that in the general population, the worse the performance on social and intellectual tasks, the greater is the overestimation of one’s skill in these areas relative to peer assessments, the latter of which have been shown empirically to be more accurate(Ehrlinger, Johnson, Banner, Dunning, & Kruger, 2008). It is possible that self-evaluation of everyday executive function is a quite difficult “social and intellectual task,” one that is prone to over-estimation even under normal circumstances. In this scenario, impaired self-awareness of executive dysfunction after TBI may reflect an exacerbation of a normal tendency to err in one’s self-assessment of these complex behaviours. Structured training and feedback provided to healthy participants can improve not only complex intellectual operations but also the accuracy of self-assessment regarding performance on them (Dunning, Johnson, Ehrlinger, &Kruger, 2003), suggesting that impaired awareness of executive dysfunction following TBI might be improved directly by remediation of executive behaviours.

In a sub-sample of patients and collateral raters who were tested on three occasions during the first year post-injury, we observed greater consistency among the collateral raters compared to the patients. Although this conclusion is tentative, our findings suggest that a collateral rater may generally be the better source of pre-injury ratings as to executive or dysexecutive behaviours. Our longitudinal sample was too small to examine the characteristics of collateral raters that might be associated with greatest consistency, e.g., age or gender of the collateral, type or duration of the relationship, or time spent in proximity to the patient. Such factors would be fruitful targets for future research, as would the question of whether these results would hold true for other measures of premorbid function or behaviour.

As noted previously, the FrSBe captures meaningful variability in executive behaviour in uninjured individuals (Hoerold et al., 2008; Pluck et al., 2012; Spinella et al., 2004). Our finding that roughly one-third of collateral raters placed their relatives’ behaviour in the abnormal range in the absence of brain injury (both within the control sample and in the pre-injury ratings for the TBI sample) highlights the fact that clinically significant dysexecutive behaviour need not be the result of brain injury. Pre-injury ratings may thus be crucial for placing post-TBI behaviour in context, making the direction of the current study an important one for future research.

Several limitations of this study should be borne in mind when interpreting the results. Our sample was relatively small, particularly the subset of participants in the longitudinal component. Examination of a larger sample might uncover systematic time-related trends that could be related to response shift. In the current study, we did not have an opportunity to test the control sample (and their collateral raters) longitudinally; given our findings, it would be of interest to explore the temporal stability of self- and collateral ratings of everyday executive function in the general population. Another caveat for interpreting these results is the fact that the psychometric properties of the FrSBe instrument have not been as thoroughly studied in healthy samples compared to those with neurologic impairments. One study on a Spanish version of the scale, using Rasch analysis, proposed caution in interpreting subscale scores in healthy participants(Caracuel et al., 2012). Further research in this vein is warranted for the English version of the scale.

Finally, it is not possible in a study such as this to validate reports from either patients or collateral respondents. The fact that collateral reports were more reliable over time does not necessarily mean that they are more veridical. Further research is necessary to establish the best ways of validating the measurement of pre-injury status in TBI, a particularly pressing need in the complex domain of executive function.