Self-Reported Neurobehavioral Symptoms in Combat Veterans: An Examination of NSI with mBIAS Symptom Validity Scales and Potential Effects of Psychological Distress

Abstract

This study evaluated symptom validity scales from the Neurobehavioral Symptom Inventory (NSI) and mild Brain Injury Atypical Symptom Scale (mBIAS) in a sample of 338 combat veterans. Classification statistics were computed using the Structured Inventory of Malingered Symptomatology (SIMS) as the validity criterion. Symptom distress was assessed with the Patient Health Questionnaire-9 and PTSD Checklist-5. At SIMS > 14, the NSI total score resulted in the highest area under the curve (AUC; .91), followed by Validity-10 (AUC = .88) and mBIAS (AUC = .67). At SIMS > 23, both NSI total and Validity-10 AUCs decreased to .88; in contrast, mBIAS AUC increased to .75. The NSI total score and Validity-10 were interpreted to reflect symptom magnification, whereas the mBIAS may reflect symptom fabrication. There was a subsample with elevated PHQ-9 and PCL-5 scores who were significantly distressed but not deemed invalid on the NSI; however, there appears to be an upper threshold on the NSI total score (> 69) beyond which nobody produced an invalid score on the SIMS. A recommended approach is provided for using NSI-related validity measures.

Introduction

Traumatic Brain Injuries (TBI) are relatively common among military personnel. According to the Department of Defense, between 2000 and 2019 there have been nearly 414,000 service members (SM) diagnosed with TBI, with the vast majority of these injuries being mild in severity and occurring in the Garrison environment (Defense and Veterans Brain Injury Center, 2020). In response to the high number of TBI cases, the Department of Veterans Affairs (VA) has increased efforts to detect and manage TBI, typically involving inclusion of self-report symptoms questionnaires and screeners. In the Veterans Health Administration (VHA), the Neurobehavioral Symptom Inventory (NSI) is the selected standard for quantifying symptoms in the Polytrauma System of Care Comprehensive TBI Evaluations. Although these detection efforts are essential aspects of appropriate veteran health care, such screening processes have historically not provided adequate information on response validity.¹

A number of studies have demonstrated elevated rates of invalid symptom response styles in veterans (Ingram et al., 2019), highlighting the need to adequately sample symptom validity. Embedded symptom validity tests (SVTs) have been developed due to their efficiency and ease of administration. As the NSI is the primary TBI symptom measure in the VHA Polytrauma system and it is commonly used in the Department of Defense, the most relevant embedded SVTs to TBI evaluations are those developed for this instrument. The seminal work for this was completed by Vanderploeg and colleagues (2014), who developed and cross-validated several NSI validity indices in samples of veteran and active-duty SMs. Among these indices, the Validity-10 scale demonstrated the best psychometric properties. The relative utility of this scale, which represents a combination of unlikely and low frequency NSI items, has been supported by other researchers (e.g., Armistead-Jehle, et al., 2018; Lange, et al., 2013; 2015). In addition to the Validity-10, the NSI total score has been investigated as a symptom validity index. For example, Bodapati et al (2019) derived optimal cut scores for the NSI total score and Validity-10 in a sample of 183 veterans with a history of mild TBI (mTBI). The authors noted adequate specificity for both measures (> 94%), but less-than-optimal sensitivity (27 – 43%), when employing overreported and invalid scores on MMPI-2-RF scales as the criterion measure.

In order to balance the higher sensitivity associated with standalone SVTs and the efficiency and ease of use of embedded SVTs, Cooper and colleagues (2011) developed a brief self-report measure of post-concussive symptom over-reporting called the mild Brain Injury Atypical Symptom Scale (mBIAS; Cooper et al., 2011). The mBIAS consists of five rationally derived items that are notably uncommon following mTBI. These items were designed to be administered as part of the NSI or PTSD Checklist (PCL). Cooper and colleagues demonstrated that mBIAS items represented a unique factor among the NSI and PCL items and that the measure demonstrated adequate sensitivity and specificity above the defined cut score. Other studies have sought to validate the mBIAS in civilian and military samples (Armistead-Jehle, et al., 2018; Lange et al., 2013; Lippa et al., 2016), with findings suggesting somewhat limited sensitivity (.15 – .56) at specificity ≥ .90.

Most studies evaluating the criterion validity of the NSI validity scales and the mBIAS employed either the MMPI-2-RF or the PAI validity scales as criterion measures. The current study sought to extend the research on NSI-related SVTs by using a different criterion measure. To this end, the first aim of the study was to evaluate the diagnostic accuracy of several SVTs including the NSI Total, NSI Validity-10, and mBIAS in a veteran sample using the Structured Inventory of Malingered Symptomatology (SIMS) as a criterion. We predicted that, in line with prior studies, specificity of the NSI SVTs would be superior to the mBIAS, though sensitivities for all would be generally limited given the nature of embedded scales and the non-forensic sample context. In addition to evaluating against the total score, SIMS subscales will be compared to these SVTs in an effort to better understand the nature of the constructs underlying these scales. As the NSI was not originally developed to function as an SVT, there is a risk of false positive errors, especially when using the total score as an embedded SVT. Thus, the second aim of this study was to evaluate the NSI in individuals with elevated psychological distress to determine if a high total NSI score was more strongly associated with psychological distress than an invalid response style. We predicted that the NSI can be significantly elevated and valid, but that there will be a ceiling beyond which invalid response is likely.

Method

Participants

Data for this study were drawn from a study on the effects of primary blast exposure that enrolled participants between February 2016 and March 2019. Participants were identified either from the patient population of a Mid-Atlantic VA Medical Center or from re-contact lists of prior research studies. Inclusion criteria were: deployment to a combat zone after September 11, 2001, combat exposure based on any positive response on the Deployment Risk and Resiliency Inventory, second edition, English-speaking, able to comply with instructions to complete study tasks, and able to provide informed consent. Exclusion criteria were history of moderate or severe TBI, any penetrating head injury, non-deployment related TBI with loss of consciousness, presence of a neurologic disorder, severe mental illness (e.g., psychotic disorder, current mania), possible neurodegenerative disorder, or active substance use disorder. Participants were screened by telephone, with final eligibility determined during the first interview visit. Results from a second visit for neuroimaging were not evaluated here.

All study procedures were approved by the facility Institutional Review Board. All participants completed informed consent prior to initiating study activities. Participants were reimbursed for time and travel. Of 803 individuals screened, 417 met initial eligibility criteria, 342 completed the assessment visit, and four were considered ineligible after enrollment, leaving a final sample of 338. Of those who were ineligible, the most common reason related to having a TBI with loss of consciousness outside of deployment or moderate/severe TBI (40%). Other common reasons included refusal to screen or participate (29%) and inability to complete neuroimaging (17%); and presence of severe psychiatric or substance use disorders (< 10%). Participants were combat exposed veterans who served since 09/11/2001, and 80% had a history of at least one TBI at any point in their life; not all participants were blast-exposed, and there was a control subsample. Although 85% of the sample was receiving service-connected disability at the time of the testing visit, this was a research-only context, and participants were informed during the consent process that data could not be used outside of the research project and were not available for clinical or disability purposes.

Measures

Neurobehavioral Symptom Inventory (NSI).

The NSI (King et al., 2012) is a 22-item self-report questionnaire of assessing the severity of somatic, cognitive, and affective symptoms over the prior two weeks. Total scores range from 0 to 88, with higher scores indicating more severe symptom burden. The NSI is included as a standard instrument in the VA Comprehensive TBI Evaluations within the Polytrauma System of Care (Belanger et al., 2009), and is widely utilized in the VHA setting. The total score has also been evaluated as an SVT, with the ≥ 57 and ≥ 67 identified as liberal and conservative cutoff scores (Ashendorf, 2019). Three subscales have been identified in the NSI: somato-sensory, affective, and cognitive (Vanderploeg et al., 2014). Two of the embedded validity measures that have been previously identified are the Validity-10, which includes 10 low frequency items (Vanderploeg et al., 2014), and the Total score of all 22 items (Ashendorf, 2019; Bodapati et al, 2019)

Mild Brain Injury Atypical Symptoms (mBIAS).

The mBIAS contains five rationally-derived items determined to be uncommonly endorsed following concussion (Cooper et al., 2011). Each item is rated from 1 to 5, resulting in a total score range of 5 to 25 with higher score indicating more symptoms.

Structured Inventory of Malingered Symptomatology (SIMS).

The symptom validity criterion was the Total score from the SIMS (Widows & Smith, 2005). Two cutoff scores were used across analyses: the liberal cutoff score indicated in the manual, and a more conservative cutoff score (Wisdom, Callahan, & Shaw, 2010). The SIMS has been studied in several different populations, with a number of different cutoff scores evaluated (van Impelen et al., 2014). It was selected as the SVT for the study given its practicality (length, stand-alone, simple to score), adequate psychometric properties reported in the manual, common use in the VA, and its coverage of a variety of symptom types in a single scale (intelligence, psychosis, neurological, etc.). Moreover, the SIMS was selected because the Total score has a higher correlation to the Structured Interview of Reported Symptoms (SIRS; which many consider the gold-standard SVT) relative to other measures (MMPI-2 F, F-K, Fp, and M-FAST) in a veteran sample (Freeman et al, 2008).

The Patient Health Questionnaire-9 (PHQ-9).

The PHQ-9 (Kroenke et al., 2001) is a 9-item questionnaire assessing frequency of depressive symptoms over the prior two weeks. Items reflect the 9 inclusion symptoms of a major depressive episode, with each item rated from 0 to 3, lending to a total score range of 0 to 27; higher scores reflect more depressive symptom frequency. To identify individuals defined as “distressed” for aim 2, a cutoff score of > 9 was used as the threshold for clinically significant depressive symptom burden.

PTSD Checklist-5 (PCL-5).

The PCL-5 (Blevins et al., 2015) is a 20-item self-report questionnaire measuring how much respondents have been bothered by the 20 DSM-5 PTSD symptoms over the prior month. Items are rated from a scale of 0 to 4, with total score range from 0 to 80; higher scores reflect more symptom burden, with a cutoff of 30 – 33 showing the highest diagnostic efficiency at predicting the presence of PTSD based on the CAPS-5 (Bovin et al., 2016). Bovin and colleagues indicate scores from 31 – 33 are equally optimal to use as a cutoff. To identify the distressed group, those with a total score of > 31 were defined as reporting clinically significant PTSD symptom burden. In the current sample, this score falls in the range identified as optimal and includes all participants with scores 1 SD below the mean PCL-5 for those with a current PTSD based on the Clinician Administered PTSD Scale-5.

Analyses

All analyses were run using SAS Enterprise Guide 7.1. To evaluate diagnostic accuracy for aim 1, pass/fail chi square analyses were run against the mBIAS and NSI SVTs and two SIMS cutoffs. Bayesian analyses were used to calculate AUC and diagnostic accuracy statistics for the mBIAS and the two NSI SVTs compared to the SIMS at the two cutoffs identified a priori. Finally, correlations were run between NSI scores including subscales, NSI SVTs, mBIAS, and SIMS total and subtest scores. Aim 2 focused on assessing invalid response in relation to those who are distressed, but who produce a valid SIMS profile. Mutually exclusive groups were created: those who were invalid on the SIMS, those with valid SIMS and high emotional distress as measured by elevating either the PHQ-9 or the PCL-5, and those with valid scores but without emotional distress. NSI SVT and mBIAS scores were not normally distributed; thus, scores were compared across groups using Kruskal-Wallis tests in place of ANOVA.

Results

Descriptive statistics for the full sample are presented in Table 1. Participants were an average of 42 years old, mostly male (86%) and White (51%). Nearly half (46%) produced invalid scores on the criterion SVT at the manual or screening cutoff score; however, only 16% had invalid scores at the more conservative or diagnostic cutoff score. Both scores were used in additional analyses.

Table 1.

Descriptive Statistics for Total Sample (N = 338)

Variable	M or n	SD or %	Min - max
Age	41.57	10.00	23 – 71
Years Education	14.99	2.16	9 – 22
Male	292	86.39%
Race/Ethnicity
White	119	51.07%
Black	101	43.35%
Hispanic	21	6.21%
Asian	1	0.43%
SC	288	85.46%	0 – 100
Any TBI ever	269	79.60%
Multiple lifetime TBI	166	49.11%	2 – 38
Any deployment TBI	170	50.30%
Multiple deployment TBI	70	20.71%	2 – 14
Moderate/severe TBI	26	7.69%
Current PTSD, CAPS-5	126	37.28%
Lifetime PTSD, CAPS-5	225	66.57%
PCL-5	33.42	20.03	0 – 78
PHQ-9	12.14	7.12	0 – 30
NSI	26.86	17.64	0 – 78
m-BIAS	5.56	1.24	5 – 15
Validity-10	8.48	6.73	0 – 30
Cognitive subtest	5.73	4.26	0 – 16
SIMS	14.85	9.28	0 – 59
FSIQ	99.43	13.33	65 – 142
TOPF Simple	99.62	9.16	78 – 122
Invalid SIMS > 14	154	45.6%
Invalid SIMS > 23	54	15.98%
Invalid mBIAS ≥ 8	22	6.5%
Invalid Validity-10 ≥ 13	90	26.6%

Note. SC = participant was receiving service-connected VA disability for any reason; TBI = traumatic brain injury; PTSD = posttraumatic stress disorder; CAPS-5 = Clinician Administered PTSD Scale, 5th edition; PCL-5 = PTSD Checklist-5; PHQ-9 = Patient Health Questionnaire-9; NSI = Neurobehavioral Symptom Inventory; mBIAS = Mild Brain Injury Atypical Symptoms; SIMS = Structured Inventory of Malingered Symptomatology; FSIQ = Full Scale Intelligence Quotient; TOPF = Test of Premorbid Functioning. CAPS-5 n = 297.

mBIAS cutoff per Cooper et al. (2011); Validity-10 cutoff per Lange (2015); SIMS cutoffs per manual and Wisdom (2010)

We began by comparing the mBIAS and NSI SVTs using established cutoff scores to the SIMS, with chi square outcomes shown in Table 2. Of those with invalid scores on the SIMS at the > 14 and > 23 cutoffs, 15% and 31% also produced invalid scores on the NSI Total, respectively. For the Validity-10, 54% and 78% of those with invalid scores on the SIMS at the >14 and >23 cutoffs also produced invalid scores on the Validity-10, respectively. Of those with invalid scores on the SIMS at the >14 and >23 cutoffs, 13% and 20% also produced invalid scores on the mBIAS, respectively. The Validity-10 had the highest concordance rates of the three indices, though a trend was seen where at the higher SIMS cutoff, all three indices had increased agreement for identifying responders with invalid scores. Table 3 shows results from logistic regression; all were significant and the NSI Total score resulted in the highest AUC at both SIMS cutoff scores. AUCs decreased for the NSI Total and Validity-10 when increasing the SIMS cutoff; in contrast, the mBIAS AUC improved at the higher SIMS > 23 cutoff score. Diagnostic accuracy data are also shown in Table 3. Finally, correlations among the mBIAS and NSI SVTs and SIMS Total and subscales, PCL-5, and PHQ-9 scores are shown in Table 4. The mBIAS was less associated with the SIMS Total score than the NSI Total and the Validity-10. When examining which SIMS subtests correlated with the three SVT scores, the Neurological subscale was highest among all three. Further, for the two symptoms measures (PCL-5 and PHQ-9), the highest correlations were with the NSI Affective subscale, and the lowest were with the mBIAS.

Table 2.

Pass Fail Concordance Rates using Established Cutoff Scores

Variable	SIMS > 14		Χ2	p	Φ	SIMS > 23		Χ2	p	ϕ
	Pass	Fail				Pass	Fail
NSI Total			29.49	< .001	0..30			61.71	< .001	0.43
Pass	184 (54%)	131 (39%)				278 (82%)	37 (11%)
Fail ≥ 57	0 (0%)	23 (7%)				6 (6%)	17 (5%)
Validity-10			107.67	< .001	0.56			88.07	< .001	0.50
Pass	177 (52%)	71 (21%)				236 (70%)	12 (4%)
Fail ≥ 13	7 (2%)	83 (25%)				48 (14%)	42 (12%)
mBIAS			19.51	< .001	0.24			20.29	< .001	0.25
Pass	182 (54%)	134 (40%)				273 (81%)	43 (13%)
Fail ≥ 8	2 (1%)	20 (6%)				11 (3%)	11 (3%)

Note. SIMS = Structured Inventory of Malingered Symptomatology; NSI = Neurobehavioral Symptom Inventory; mBIAS = Mild Brain Injury Atypical Symptoms.

Table 3.

Logistic Regression and Diagnostic Accuracy Predicting SIMS Total Score

SIMS Cutoff	Index	Model Results			Cutoff	Sen.	Sp.	PLR	NLR	15%		30%
		AUC	Wald χ2	p						PPV	NPV	PPV	NPV
> 14	NSI Total	.905	92.67	< .001	≥ 36	.662	.913	7.34	0.14	.56	.94	.76	.87
					≥ 35	.675	.908
					≥ 34	.695	.897
	Validity-10	.882	90.57	< .001	≥ 14	.610	.940	7.54	0.13	.57	.94	.76	.86
					≥ 13	.656	.913
					≥ 12	.708	.859
	mBIAS	.672	28.02	< .001	≥ 8	.130	.989	6.50	0.15	.53	.87	.74	.74
					≥ 7	.208	.968
					≥ 6	.468	.859
> 23	NSI Total	.880	56.87	< .001	≥ 49	.463	.912	5.06	0.20	.47	.91	.68	.80
					≥ 48	.481	.905
					≥ 47	.574	.898
	Validity-10	.875	57.88	< .001	≥ 20	.370	.937	5.22	0.19	.48	.90	.69	.79
					≥ 19	.444	.915
					≥ 18	.537	.898
	mBIAS	.752	30.79	< .001	≥ 8	.204	.962	5.25	0.19	.48	.89	.69	.77
					≥ 7	.354	.933
					≥ 6	.685	.785

Note. SIMS = Structured Inventory of Malingered Symptomatology; NSI = Neurobehavioral Symptom Inventory; mBIAS = Mild Brain Injury Atypical Symptoms; Sen. = Sensitivity; Sp. = Specificity.

Table 4.

Correlations of NSI SVTs and m-BIAS to SIMS and Questionnaires

NSI	mBIAS	SIMS						PCL-5	PHQ-9
		Total	Psychosis	Neurologic	Amnestic	Low IQ	Affective
Total	.48	.78	.46	.72	.67	.28	.66	.86	.81
Somatic	.55	.74	.46	.74	.62	.29	.57	.74	.68
Affective	.37	.69	.40	.61	.58	.23	.65	.89	.83
Cognitive	.38	.71	.39	.62	.68	.26	.60	.74	.71
Validity-10	.54	.76	.46	.73	.66	.29	.59	.75	.69
mBIAS		.57	.48	.54	.45	.31	.34	.37	.32

Note. SIMS = Structured Inventory of Malingered Symptomatology; NSI = Neurobehavioral Symptom Inventory; mBIAS = Mild Brain Injury Atypical Symptoms. All correlations significant at p < .001. Highest correlation for each column is in bold.

For the second aim, we hypothesized that scores on the NSI can be significantly elevated but still valid. This is important to clarify, because if only invalid-scoring individuals significantly elevate the NSI then it would indicate the NSI is not appropriate for assessing severe symptoms and that the measure functions more as an SVT only at high elevations. To explore this, the sample was first divided into pass and fail groups by SIMS at the conservative cutoff and descriptive statistics run for the NSI total score (Table 5; Validity-10 and mBIAS included for comparison). For the NSI total score, the SIMS fail group clearly had higher means and range; however, in the pass group, NSI scores reach up to 69, and given the mean and SD, there is evidence that individuals can endorse a high number of symptoms but still remain valid based on the external criterion of the SIMS. Nevertheless, there is a threshold, and nobody scoring in the highest range of the NSI (70 – 88) was valid based on the SIMS. Next, the sample was separated into groups using SIMS pass/fail status and symptom measures (PHQ-9 and PCL-5) external to the NSI. Kruskal-Wallis tests were used to compare groups (Table 6). Results were significant for all NSI variables and all follow-up contrasts. There is a group that is significantly distressed (different from non-distressed), but also significantly different from the invalid group. Here again, there seems to be a threshold of symptoms burden above which one is invalid.

Table 5.

Means and Ranges by SIMS Pass/Fail

SIMS Status	Variable	Minimum	Maximum	M	SD
Pass n = 284	NSI Total	0	69	22.89	15.39
	Validity-10	0	26	6.95	5.75
	mBIAS	5	10	5.34	0.81
Fail n = 54	NSI Total	19	78	47.76	13.58
	Validity-10	5	30	16.48	5.81
	mBIAS	5	15	6.70	2.18

Note. SIMS = Structured Inventory of Malingered Symptomatology; NSI = Neurobehavioral Symptom Inventory; mBIAS = Mild Brain Injury Atypical Symptoms.

Table 6.

Kruskal-Wallis Tests Comparing Valid not Distressed, Valid Distressed, and Invalid Groups on NSI SVTs and mBIAS

Score	Group	M	SD	χ2	p	Ε2
NSI Total				252.31	< .001	0.61
	Valid	11.98	8.44
	Distressed	33.64	12.93
	Invalid	47.76	13.58
mBIAS				154.91	< .001	0.46
	Valid	3.79	3.50
	Distressed	10.10	5.81
	Invalid	16.48	5.81
Validity-10				64.42	< .001	.19
	Valid	5.14	0.37
	Distressed	5.54	1.04
	Invalid	6.70	2.18

Note. NSI = Neurobehavioral Symptom Inventory; mBIAS = Mild Brain Injury Atypical Symptoms.

Valid n = 141; Distressed n = 143; Invalid n = 54.

All post hoc comparisons were significant at p < .01

Discussion

The first aim of this paper was to validate the NSI SVTs and mBIAS using the SIMS as a validity criterion. Prior studies have predominantly used the MMPI-2/RF (Armistead-Jehle et al., 2018; Ashendorf, 2019; Bodapati et al., 2019; Lange et al., 2015; Lippa et al., 2016) or the PAI (Vanderploeg et al., 2014) validity scales as an external criterion. The SIMS is a stand-alone SVT with five subscales created to sample overreporting of neurological, affective, psychotic, intellectual, and amnestic symptoms. Using the SIMS to further validate the NSI/mBIAS extends prior research and further informs clinical decisions regarding cut scores. In this research sample of Iraq and Afghanistan combat veterans, all three scales (NSI Total, Validity-10, mBIAS) were significant predictors of invalid SIMS scores; both the NSI Total and the Validity-10 reached acceptable AUC levels (> .80), with the best model resulting from the NSI Total score at the SIMS cutoff of 14.

Examining individual cutoff scores in this sample, the highest sensitivity (while maintaining specificity ≥ .90) across both SIMS cutoff scores was the NSI total score. Prior studies have recommended using two different cutoffs depending on context (screening vs. diagnostic) or certainty (possible vs. probable exaggeration) (Armistead-Jehle et al., 2018; Bodapati et al., 2019; Lippa et al., 2016). Given two criterion cutoff scores were used, a similar conceptualization could be taken, with lower cutoff scores representing screening or possible exaggeration cutoffs and higher cut scores representing increased certainty in symptom invalidity. For the NSI total score, optimal (highest sensitivity while maintaining specificity of .90) cutoffs were ≥ 35 and ≥ 48. The total score with a cutoff of > 58 was initially presented with a sensitivity of .79 and .59 in development and clinical samples (Vanderploeg et al., 2014), respectively. The total score was evaluated in only two additional studies in the VA (see Table 7), with recommended cutoffs ranging from ≥ 51 to ≥ 67 (Ashendorf, 2019; Bodapati et al., 2019). Both of these prior studies used similar samples to ours (VA OEF/OIF sample, mild TBI), though both prior studies used the MMPI-2-RF as the criterion. Our more liberal cutoff score of ≥ 35 outperformed prior results with sensitivity of .68; however, the more conservative cutoff had poorer sensitivity of .48. The differing validity criterion and evaluation context could explain differing outcomes, though like prior studies, the NSI total score appears useful for assessing overreporting. The Validity-10 in our sample behaved similar to the NSI total score, but with slightly lower sensitivities at cutoff scores of ≥ 13 (sen. = .66) and ≥ 19 (sen. = .44). This replicates the results from Lange and colleagues (2015) who also recommended a cutoff of ≥ 13, with sensitivity at .63 in their sample.

Table 7.

Diagnostic Accuracy of Prior Studies Using the NSI SVTs and mBIAS

	Cooper, 2011	Vanderploeg, 2013	Lange, 2015	Lippa, 2016	Armistead-Jehle, 2018	Bodapati, 2019	Ashendorf, 2020
N	403	206 (cross-validation)	63	117	359	183	82
Sample Criterion	DoD, TBI > 1.5 SD on NSI and PCL-M	DoD, TBI PAI NIM	DoD, TBI MMPI-2-RF (overreporting scales)	VA, mixed MMPI-2 (F, FBS)	DoD, TBI MMPI-2-RF (overreporting scales)	VA, TBI MMPI-2-RF (overreporting scales)	VA, TBI MMPI-2-RF (overreporting scales)
NSI Total		> 58 (.59/.84)				≥ 57 (.43/.97) ≥ 67 (.33/.95)	53 (.51/1.0) 51 (.57/.91)
						≥ 51 (.59/.91) ≥ 63 (.52/.91)
Validity-10		> 22 (.61/.85)	≥ 13 (.63/.97)		≥ 7 (.89/.48) ≥ 33 (.01/1.0)	≥ 22 (.41/.96) ≥ 27 (.27/.95)	17 (.59/.91) 19 (.49/.97)
			≥ 11 (.71/.92)		≥ 17 (.42/.92)	≥ 21 (.47/.92) ≥ 24 (.52/.91)
mBIAS	≥ 8 (.94/.92)		≥ 8 (.17/1.00)	≥ 11 (.31–.57/.89–.94)	≥ 10 (.15/1.0)		Fs ≥ 9 (.32/.90)
			≥ 7 (.21/.92)	≥ 16 (.00–.11/.98–1.0)	≥ 8 (.26/.92)		RBS ≥ 9 (.33/.90)

Note. NSI = Neurobehavioral Symptom Inventory; mBIAS = Mild Brain Injury Atypical Symptoms; DoD = Department of Defense; VA = Veterans Affairs; TBI = traumatic brain injury; PCL-M = PTSD Checklist-Military; MMPI = Minnesota Multiphasic Personality Inventory; RF = Restructured Form; PAI = Personality Assessment Inventory.

Cutoffs are shown as score (sensitivity/specificity). Shaded cells are non-author recommended cutoff but reflecting highest sensitivity at specificity ≥ .90. For Bodapati (2019) the 2 scores are for possible versus probable exaggeration. For Ashendorf (2020) other than mBIAS, scores presented are for predicting invalid on any MMPI-2-RF validity scale. All samples were clinical contexts (not forensic).

For the NSI total and Validity-10 scores, sensitivity decreased when the SIMS cutoff score was raised to the conservative cutoff of > 23. In contrast, the mBIAS sensitivity increased at the higher cutoff, suggesting the mBIAS might be more sensitive to those presenting as more severe or as having global pathology. The best mBIAS cutoff score regardless of SIMS cutoff used was ≥ 7, which is equivalent to endorsing two items as mild or one item as moderate. This low threshold also suggests that the symptoms of the mBIAS are relatively pathognomonic or, at the very least, tapping a different construct than the NSI embedded scores. Further support of this point are the scales’ different diagnostic accuracy, contrasting behaviors at different SIMS cutoff scores, and (on the surface) differing item content. More specifically, the NSI total score and Validity-10 are comprised of “real” symptoms; in other words, the items reflect plausible complaints, but the mechanism of invalidity signal detection is the severity of total symptom burden endorsed. Thus, the two scales are conceptually analogous to symptom exaggeration specifically, as cutoffs reflect a threshold of symptom burden that would be rare or unlikely. In contrast, the mBIAS items are somewhat odd and bizarre and reflect symptoms that are unlikely to be experienced in general; in other words, the symptoms are not likely “real” and the scale thus reflects a measure of symptom fabrication. This would explain why there is such a low threshold of mBIAS item endorsement before invalidating the SIMS.

This dichotomy of fabricated versus magnified symptoms has been noted elsewhere. For example, the DSM-5 description of malingering includes “intentional production of false or grossly exaggerated” symptoms (Diagnostic and statistical manual of mental disorders: DSM-5, 5th ed, 2013; p. 726). The Structured Interview of Reported Symptoms (Rogers, Gillis, Dickens, & Bagby, 1991) includes several scales to identify unlikely symptoms (e.g., improbable symptoms) as well as amplified symptoms (e.g., symptom severity). Similarly, the PAI Negative Impression scale includes both “exaggerated or distorted” items and “bizarre and unlikely symptoms,” (Morey, 2007; p. 29). It may thus be helpful to avoid comparison between the NSI SVTs and the mBIAS, as they appear to be sampling different symptom validity constructs, which is also supported by our correlation results. Future research could further analyze this hypothesis.

As the NSI Total and Validity-10 do reflect real symptoms, but with a threshold of severity that is unlikely, there is a risk that those indices could be misclassifying examinees with actual and severe symptom burden as exaggerating. Thus, the second aim of this study was to assess neurobehavioral symptom burden (NSI total score) and invalidity to see if a valid but high distress group could be identified. Kruskal-Wallis tests and all post hoc comparisons were all significant, and distinct valid-not-distressed, valid distressed, and invalid groups were identified. Of note, anyone with NSI total scores over 69 was invalid. Thus, we recommend using both the NSI Total score and the mBIAS when evaluating symptom validity, as the two scores appear to measure somewhat different symptom validity constructs. The recommended mBIAS cutoff score is ≥ 7. Given the above results, three possible cutoff scores seem reasonable for the NSI Total score: ≥ 35 indicating possible exaggeration, ≥ 48 indicating probable exaggeration, and ≥70 suggesting complete invalidity (given nobody in the valid subsample scored that high).

There are several limitations to this study. First, sample characteristics may limit generalizability to active duty settings, veterans of other cohorts, or civilians. Other characteristics may also limit generalizability, including that this sample was mostly male, White or Black, relatively well-educated, with a mean age of 41, and all deployed to a combat zone. Further, the criterion measure (the SIMS) was invalid at the manual cutoff score for nearly half our sample, though the forensic cutoff was much closer to base rate expectations (15% fail rate in our research sample). As such, the SIMS may function differently with this population relative to the SIMS normative sample. Dedicated studies using more established criterion measures (e.g., SIRS/2) would further this research. Despite these limitations, the sample was well powered and characterized, and the study adds an additional veteran study to existing research. This is also the first study to use the SIMS as the primary SVT criterion, which adds to cumulating research on the NSI and mBIAS. Lange and colleagues (2013) also used the SIMS in a simulation study, but only as part of the procedure for characterizing the invalid groups. Future studies might explore the underlying factors of the NSI SVTs and mBIAS to determine if they do load on different factors.

Public Significance Statement:

This study extends prior research on symptom validity indices associated with the Neurobehavioral Symptom Inventory. A recommended interpretive approach is presented involving multiple indices.