The NEO-FFI in Multiple Sclerosis: Internal Consistency, Factorial Validity and Correspondence Between Self and Informant Reports

Abstract

Personality assessment is a potentially important component of clinical and empirical work with neurological patients because (1) individual differences in personality may be associated with different neurological outcomes and (2) central nervous system changes may give rise to alteration in personality. In order for personality assessment to be useful to clinicians and researchers, the tests must be reliable and valid, as self-report measures require certain baseline levels of comprehension and insight, both of which can be compromised by cerebral disease. In this study, we examined the psychometric properties of the widely used NEO Five Factor Inventory (NEO-FFI) in a group of 419 patients with multiple sclerosis (MS). Our objective was to determine if the NEO-FFI is reliable and valid in this population. Results showed adequate estimates of internal consistency, factorial validity and self-informant correlation that support its use with MS patients. Implications, limitations of the current study and directions for future research are discussed.

Multiple Sclerosis (MS) is an inflammatory disease of the central nervous system. The pathological hallmarks of MS are concentrated areas of demyelination within the cerebral and spinal chord white matter (Brownell & Hughes, 1962; Miller et al., 1996), but atrophy of the cerebral gray matter and its clinical effects are also now widely appreciated (Benedict, Bruce et al., 2006; Pirko et al., 2007; Tekok-Kilic et al., 2007). The clinical course of MS is most often characterized by recurrent attacks of neurological symptoms followed by remission. This is known as the relapsing-remitting (RR) course. Roughly 1/3 of MS patients develop progressive neurological disability with fewer relapses (Jacobs et al., 1999). Because MS is associated with atrophy or lesions in multiple regions of the central nervous system, the clinical presentation is heterogeneous, involving sensory loss, paraesthesias, weakness, fatigue, ataxia, depression and cognitive impairment (Arnett et al., 1999; Benedict, Carone, & Bakshi, 2004; Jacobs et al., 1999; Sandroni, Walker, & Starr, 1992).

Personality assessment is a new area of investigation in MS research, although subtle changes in personality were noted by clinicians as early as the late 19^th century (Brown & Davis, 1922; Charcot, 1877). Interest in measuring personality traits in MS is growing because personality is associated with prognosis in other medical samples as indicated by health behaviors (Bogg & Roberts, 2004), immune parameters (Miller, Cohen, Rabin, Skoner, & Doyle, 1999), and cumulative illness burden (Chapman, Duberstein, & Lyness, 2007). In addition, personality traits themselves may be affected by neurological changes due to disease progression (Benedict, Priore, Miller, Munschauer, & Jacobs, 2001; Benedict, Shapiro et al., 2001), rendering personality assessment a critically important barometer of the disease’s impact on psychological and behavioral function. Although personality changes are thought to be associated with Alzheimer’s disease (Bozzola, Gorelick, & Freels, 1992; Chatterjee, Strauss, Smyth, & Whitehouse, 1992; Dawson et al., 2000; Rubin et al., 1987; Siegler et al., 1994, Siegler et al, 1991; Welleford et al., 1995; Williams, Briggs & Coleman, 1995), traumatic brain injury (Kurtz, Putnam, & Stone, 1998; Prigatano, Altman, & O’Brien, 1990), and MS (Christodoulou et al., 1999; Johnson, DeLuca & Natelson, 1996; Lima et al., 2007; Merkelbach et al., 2003; Pepper, Krupp, Friedberg, Doscher, & Coyle, 1993), the psychometric properties of the personality measures used in these studies have not been subject to rigorous empirical scrutiny in a large, well-characterized neurologic population.

One of the most popular, empirically supported, and clinically and scientifically useful personality taxonomies is the Five Factor Model (FFM; Goldberg, 1993; McCrae & Costa, 1997). Meta-analyses have found average estimates of internal consistency to range from .73 to .78 for many FFM personality inventories (Viswesvaran & Ones 2000), A recent review on factor replicability found five factor rotations in 26 of 28 personality inventories to range from .70 to .99, underscoring the apparently robust nature of the five-factor structure (O’Conner, 2002). The concurrent validity of FFM inventories is also found in investigations of agreement between self and informant reports, which have been found to range from .30 to .60 in samples of psychiatric patients (Bagby et al., 1998; Kurtz & Putnam, 2006; Ready & Clark, 2002; Yang et al., 1999), estimates similar to that found in samples of neurological patients (Benedict et al., 2001a; Carone, Benedict, Munschauer, Fishman, & Weinstock-Guttman, 2005; Kurtz & Putnam, 2006; Welleford et al., 1995). Self-informant agreement on personality ratings in patient samples appears to be slightly lower than that found in presumably healthy controls (Costa & McCrae, 1992).

The FFM personality measure that has received the most empirical attention is the NEO Personality Inventory (NEO-PI), which now has a revised version (NEO-PI-R) as well as an abbreviated version (NEO Five Factor Inventory (NEO-FFI, Costa & McCrae, 1992). The NEO-FFI has been validated and used extensively in cross-cultural studies of healthy adolescents and adults (McCrae et al., 2000), as well as in samples of medical, psychiatric and substance abuse patients (Bagby et al., 2007; Booth, Shinka, Brown, Mortimer & Borenstein, 2004; Borman et al., 2006; Carter et al., 2001; Chapman, Duberstein, Sörensen, & Lyness, 2006; Chapman, Duberstein, & Lyness, 2007; Gleeson, Rawlings, Jackson & McGorry, 2005; Hopwood et al., 2007; Piedmont & Ciarrocchi, 1999; Weiss & Costa, 2005). Preliminary use of the NEO-FFI or FFM scales in MS has revealed associations between personality and chronic fatigue and depression, and MS patients have been shown to have personality profiles that are significantly different from those of healthy controls. Johnson, DeLuca and Natelson (1996) found MS patients to report higher mean levels of Neuroticism than healthy controls, but lower Neuroticism than patients with Major Depression. Our own work has found associations between cognitive impairment in MS and higher levels of informant rated Neuroticism and lower levels of Extraversion, Agreeableness and Conscientiousness, as well as lower agreement with self-reported personality traits (Benedict et al., 2001). More recently, we reported that female MS patients with a secondary progressive course rated themselves as higher in Extraversion and Openness than did their spouses/domestic partners, and this subgroup of MS patients also showed poor rater agreement with their informants on all traits except Agreeableness (Benedict et al., 2009). These studies however employed small samples and the reliability and validity of the NEO-FFI has not been investigated in a large, representative MS sample.

Given the evidence that self-informant agreement on NEO-FFI ratings in samples of psychiatric and neurologic patients appears to be slightly lower than that found in nonpatient samples (Costa & McCrae, 1992), it is important to investigate whether the correspondence between self-reported personality and ratings of personality by others might be compromised in MS. Disease status could potentially confuse external raters by altering perceptions of the patient personality characteristics (Funder & Colvin, 1997). Furthermore, personality traits could become less unique with progressive neurological decline, threatening the construct validity of the FFM. On the other hand, if the psychometric properties of self-reported NEO-FFI scores in MS are similar to those of the normative sample (Costa & McCrae, 1992) with respect to internal consistency, correspondence with informant reports, and factor structure, then the instrument has the important distinction of being relatively impervious to psychometric threats posed by MS. In turn, evidence of sound psychometric properties would underscore the credibility of previous findings on personality in MS and provide future researchers assurance that the NEO-FFI is a relatively robust personality assessment tool that can be employed with confidence in this and similar medical populations.

In the current study, we posed the general question of whether or not the FFM, as measured by the NEO-FFI, has adequate construct validity in MS. We hypothesized that the internal consistency estimates would be relatively low. We also hypothesized that the factor loadings of NEO-FFI items on the intended five personality factors may be lower in MS patients than has been evidenced in the neurologically intact normative sample. We investigated these questions statistically with estimates of internal consistency, tests of factorial validity, and self-informant correlations.

Methods

Participants and Procedures

Participants were 419 MS patients who were receiving medical care from an MS specialty clinic in western NY state. Neurologists specializing in the care of MS patients used commonly accepted diagnostic criteria to ascertain clinically definite MS status (Polman et al., 2005). Exclusion criteria were determined by structured interview and included current or past neurological or medical disorder other than MS that could affect psychological or cognitive status, such as prior substance dependence or current abuse, current major depression or psychotic disorder, treatment for psychiatric illness prior to the onset of MS, and MS relapse or corticosteroid pulse within six weeks of participation. Major depression was assessed via a site-specific semi-structured interview based on the DSM-IV (American Psychiatric Association, 2000). This interview mirrored the DSM-IV diagnostic criteria although accounting for the influence MS may have on the neurovegetative symptoms of depression such as insomnia and fatigue. Subjects meeting the criteria of a current major depression episode were excluded because major depressive episodes can cause transient poor performance on neuropsychological testing that is difficult to distinguish from MS-related impairment. In addition, subjects rated depressive symptoms on the Beck Depression Inventory - Fast Screen (BDI-FS; Beck, Steer & Brown, 2000; validated in MS in Benedict, Fishman, McClellan, Bakshi & Weinstock-Guttman, 2003) and revealed that 63.8% of subjects reported minimal symptoms of depression, 19.5% reported mild symptoms, 12.2% reported moderate symptoms, and 4.6% reported severe symptoms. Thus although subjects that met the criteria for a current major depressive episode were excluded based on clinical interview, the sample still contained a range of depressive symptoms consistent with MS diagnosis. Of the 419 patients, 22.8% were paid volunteers for research, 48.5% were referred for baseline testing or regularly scheduled routine monitoring of cognitive/personality function, and 28.7% were referred on a non-routine basis, such as when a patient, family member, employer or medical provider suspects a recent decline in cognitive status. All research participants provided written informed consent. The IRB approved the retrospective analysis of clinical data.

The sample composition was characterized by demographics and disease features that are consistent with population studies of MS (Jacobs et al., 1999), as follows: Mean (± SD) disease duration 11.1±8.2 years, age 46.8±9.2 years, female (76.6%), Caucasian (93.1%) and right-handed (86.9%). Mean (± SD) education was 14.3 ± 2.3 years. For 147 patients, significant others were available to complete the informant-report NEO-FFI (described below). Among informants, 70.5% were the patient’s spouse or domestic partner, 11.5% were the parent, 13.9% were another family member, and 4.1% were a close friend. The mean (± SD) number of weekly face-to-face contacts was 6.4 ± 1.7. Ninety-six informants (86%) were in direct contact with the patient seven days per week, and the mean (± SD) duration of the relationship between patient and informant was 27.9 ± 13.5 years. Disability Status Scale (EDSS) ratings within 6 months of testing were available for 184 patients. The median EDSS was 4.0 and the range was 0 to 8.0. This median rating corresponds to mild/moderate physical disability. We used the Minimal Assessment of Cognitive Function in MS (MACFIMS) to characterize the sample cognitively. This is a consensus battery approach recently validated in large prospective MS samples (Benedict, Cookfair et al., 2006). The specific cognitive tests are as follows: Controlled Oral Word Association Test (COWAT; Benton, Sivan, Hamsher, Varney & Spreen, 1994), Judgment of Line Orientation (JLO; Benton, Sivan, Hamsher, Varney & Spreen, 1994), California Verbal Learning Test, second edition (CVLT-2;(Delis, Kramer, Kaplan, & Ober, 2000), Brief Visuospatial Memory Test-Revised (BVMT-R; (Benedict, 1997), Paced Auditory Serial Addition Test (PASAT; (Gronwall, 1977), Symbol Digit Modalities Test (SDMT; (Smith, 1982), and the Delis-Kaplan Executive Function System Sorting Test (DKEFS Sort and DKEFS Description; Delis, Kaplan & Kramer, 2001). The mean z scores as compared to previously published norms (see Benedict et al., 2006) are reported in Table 1. It can be seen that this sample was, in general, mild to moderately cognitively impaired.

Table 1.

Sample mean (SD) Z-scores of cognitive test

Cognitive Test	Mean (SD) Z-score
COWAT	−0.43 (0.89)
JLO	−1.01 (1.76)
CVLT2 Learning	−0.74 (1.12)
CVLT2 Delay	−0.86 (1.31)
BVMT-R Learning	−1.41 (1.46)
BVMT-R Delay	−1.49 (1.79)
PASAT	−0.68 (1.12)
SDMT	−1.61 (1.47)
DKEFS Sorts	−0.76 (1.36)
DKEFS Description	−0.78 (1.25)

Measures

The NEO-FFI (Costa & McCrae, 1992) is a 60-item questionnaire with separate forms for both self-report and informant-report. There are 12-items each for Neuroticism, Extraversion, Openness, Agreeableness and Conscientiousness. We followed published instructions to administer the paper-pencil version. Each test item asked the respondent to rate the extent to which he or she concurs with a statement on a five-point Likert scale ranging from strongly disagree to strongly agree. We used the gender-specific norms reported in the manual to derive T-scores for each domain for self and informant.

Results

Internal Consistency

Internal consistency was assessed using Cronbach’s alpha for each of the five factors. These internal consistency estimates were compared to those reported in the NEO-FFI manual (Costa & McCrae, 1992) using Feldt’s W (1987). We chose the sample described in the manual for comparison in order to maintain consistency with other analyses treating the manual as the “gold standard” or common benchmark. It bears emphasis that although our sample was demographically similar to population studies of MS, our sample was not identical to the sample described in the manual: our sample had a higher proportion of Caucasian and female subjects, was slightly more educated, and included a narrower age range. In addition, our sample had a higher proportion of spouse/domestic partner informants than did the manual. However, a literature review did not yield a more preferable sample for our needs. We also used Feldt’s W to compare alphas within the sample to explore possible variation in internal consistency by recruitment strategy and to compare alphas obtained in self report with those obtained by informants. Cronbach’s alpha estimates of domain scores for both self-report and informant-report (Table 2) suggested that domain scores achieved acceptable levels of internal consistency for all scales except informant-report Openness where Cronbach’s alpha was 0.60. Comparisons with the Cronbach’s alphas reported for the manual normative sample revealed that none of the internal consistency estimates in the present sample differed from those among the normative sample except for the informant-report Openness scale (W = 1.67, p < .01), which was significantly lower than the value reported in the manual (.60 vs. .76). Comparison of alphas between self-report and informant-report revealed that internal consistency was lower for informant-report than for self-report Openness (W = 1.38, p < .01), and higher for informant-report Agreeableness (W = 2.08, p < .001) and Conscientiousness (W = 5.63, p < .001) than self-reports of these two traits.

Table 2.

NEO-FFI Mean T-scores, Cronbach’s Alpha, Pearson correlation, and Factor Congruence Coefficients

Factor	Self Report T-score2 (SD)	Informant Report T-score2 (SD)	Self- Report1 Alpha	Informant- Report2 Alpha	Pearson Corr.2, 3	Factor Congruence Coefficient1
Neuroticism	50.82 (11.29)	54.47 (11.31)***	.87	.87	.46	.86
Extraversion	49.10 (10.55)	45.12 (11.69)***	.80	.82	.39	.82
Openness to Experience	47.27 (8.66)	45.19 (9.51)*	.71	.60	.34	.82
Agreeableness	51.86 (11.61)	50.08 (12.07)	.73	.87	.46	.84
Conscientiousness	46.54 (12.79)	45.70 (12.09)	.84	.91	.33	.87

Notes: *p < .05,

^*** p < .001. SD = Standard Deviation.

¹N = 419;

²N = 147.

³All Pearson correlation coefficients are significant at the p < .001 level.

Internal consistency reliability differed by referral source for only one scale: informant reported Neuroticism was less internally consistent in observers of research volunteers (alpha = .77), than in the observers of routine clinical care (alpha = .91). Comparison of internal consistency coefficients between cognitively intact and cognitively impaired subjects (Table 3) revealed a statistically significant lower alpha for Extraversion in impaired versus cognitively intact subjects (.85 vs. .77, W = 1.53, p < .01). No other statistically significant differences in internal consistency coefficients were observed between these groups.

Table 3.

Comparison of Cronbach’s Alphas by Cognitive Impairment

Factor	Self-Report Alpha
	Intact1	Impaired2
Neuroticism	.89	.86
Extraversion	.85	.77**
Openness to Experience	.71	.71
Agreeableness	.73	.73
Conscientiousness	.86	.82

Notes: **p < .01.

¹N = 149;

²N = 270. Alphas compared using Feldt’s W (1987).

Correspondence of Self and Informant Reports

We examined correspondence between self and informant report using Pearson correlation coefficients. We computed r values in the overall sample for whom informant data were available and compared self-informant correlations in our sample with self-informant correlations reported in the manual¹. We also examined whether differences existed in reports of mean (T-score) levels of personality between patients and their informants using used a paired samples t-tests. Pearson correlations between self and informant ratings of 0.46, 0.39, 0.34, 0.46 and 0.33 were observed for Neuroticism, Extraversion, Openness, Agreeableness and Conscientiousness, respectively, for the whole sample of MS subjects (Table 2, all p < .001), indicating agreement between self and informant reports roughly comparable to that observed in the normative sample as a whole (Costa & McCrae, 1992). Although correspondence was statistically significant for all domains for the whole sample, a paired samples t-test revealed some differences in reported mean levels of Neuroticism, Extraversion, and Openness between self ratings and informant ratings (Table 2). Statistical comparison to the normative sample informant correlations was possible for spouse/domestic partner informants only. Comparison of Fisher Z transformations yielded no significant differences of spouse/domestic partner and self rating agreement between the present sample and the manual sample.

Factorial Validity

To examine the factor structure of the NEO-FFI in MS patients, we employed procrustes rotation to maximal congruence, using a target matrix representing the theoretical factor pattern of the NEO-FFI. A central difficulty is that even if the scale has a well-defined factor structure, confirmatory factor analysis (CFA) models do not fit scales with large numbers of items because there are too many trivial sources of item covariance (Floyd & Widaman, 1995). Procrustes rotation is preferred to CFA when assessing the factor pattern of many subscale or facets scales as are available in the NEO-FFI or its parent test the NEO Personality Inventory Revised (McCrae, Zonderman, Bond, Costa, & Paunonen, 1996), owing partly to the complexity of specifying CFA models with large numbers of factors and indicators (see also Barrett, 2007; Raykov, 1998). In the case of the NEO-FFI, the feasibility of item-level CFA is even more problematic, as the number of indicators would be doubled (60 items) and the number of estimated parameters for even a basic model more than twice this again. If the model was additionally parameterized to account for small source of item covariance beyond factor loadings, the number of estimated parameters would grow very large in relation to the sample size, resulting in imprecise estimates and a model too complex to easily interpret. Targeted rotation proceeded in three steps. First, we specified a target matrix of 0s and 1s representing the theoretical simple structure of the NEO-FFI, i.e., a factor pattern in which each item loads completely on the factor corresponding to the personality domain it is intended to represent and not at all on other domains (ten Berge, 1986). Then, we extracted five factors and performed a varimax rotation. We then rotated the resulting varimax pattern matrix to the target matrix of theoretically expected loadings (cf. Barrett, Petrides, Eysenck, & Eysenck, 1998), analogous to the validimax rotation recommended for use with the NEO Personality Inventory (McCrae et al., 1996). This procedure produced congruence coefficients for the overall factor solution, reflecting the comparability of the rotated solution to the “ideal” target model, as well as congruence coefficients for each factor and the loading of each item, reflecting the degree of similarity to the factor and pattern that would be obtained under the perfect NEO-FFI simple structure. Solutions with congruence of 0.85 and above are typically considered indicative of fit to the target factor pattern (Barrett, 1986; ten Berge, 1986). We also conducted Procrustes rotation to the factor structure reported by McCrae and Costa (2004, p. 591–592) in a combined high school and Baltimore Longitudinal Study of Aging (BLSA) sample.

Targeted rotation of the NEO-FFI produced an overall solution congruence of 0.85. Individual factor congruence coefficients were 0.86 for Neuroticism, 0.82 for Extraversion, 0.82 for Openness, 0.84 for Agreeableness and 0.87 for Conscientiousness. Inspection of individual items revealed that although no items on Neuroticism, Extraversion, Agreeableness, or Conscientiousness showed extremely low congruence coefficients (< 0.5), two Openness items showed extremely low congruence to target loadings: item 8, “Once I find the right way to do something, I stick to it” (congruence 0.39, due to a stronger loading on Conscientiousness), and item 48 “I have little interest in speculating on the nature of the universe or the human condition” (congruence 0.35, due to low loadings on all factors). Item-to-total correlations for item 8 were also less for Openness (r = 0.26, p < .001) than for Conscientiousness (r = 0.30, p < .001). Removing these two items improved both the Openness factor and overall solution congruence to 0.86. Thus, the targeted rotation indicated adequate fit between the observed and theoretical factor structure, with two Openness items exhibiting only trivial similarity to expected loadings. Table 5 presents the rotated pattern matrix. Rotation to the factor structure reported by McCrae and Costa (2004) also resulted in an overall solution congruence of 0.85, with factor congruences of 0.74 for Neuroticism, 0.91 for Extraversion, 0.89 for Openness, 0.84 for Agreeableness, and 0.89 for Conscientiousness. To some extent, this structure may be more realistic because it captures the cross-loadings that occur in real data.

Table 5.

Procrustes-Rotated NEO-FFI Factor Loadings

Item Number	Intended Factor	Neuroticism	Extraversion	Openness	Agree- ableness	Conscient- iousness
1	N	.26	−.10	.07	−.01	.08
2	E	−.11	.51	.01	.14	−.02
3	O	.15	−.02	.28	−.09	−.24
4	A	.12	.16	.04	.33	.12
5	C	.01	.00	−.07	.00	.55
6	N	.57	−.20	−.10	.00	−.13
7	E	−.15	.44	.14	−.03	.06
9	A	−.30	.00	.09	.38	.09
10	C	−.19	.05	−.06	.03	.60
11	N	.62	−.13	.01	.01	−.12
12	E	−.19	.40	.08	.17	.02
13	O	.02	.05	.62	.02	.04
14	A	−.17	.05	−.11	.49	.12
15	C	.00	−.02	.16	−.10	.31
16	N	.66	−.25	.02	.01	−.05
17	E	−.02	.51	.07	.31	.08
18	O	−.09	.12	.27	.10	.07
19	A	.15	.09	−.09	.24	−.03
20	C	.06	.18	−.05	.18	.44
21	N	.58	−.16	.03	−.10	−.08
22	E	.00	.53	.07	−.07	.07
23	O	−.02	.04	.56	.07	−.01
24	A	−.33	.14	.15	.53	.01
25	C	−.25	.14	.14	−.07	.57
26	N	.66	−.21	−.10	−.12	−.15
27	E	−.18	.30	.11	.30	−.01
28	O	−.02	.12	.28	.00	.03
29	A	−.29	.01	.21	.48	−.04
30	C	−.25	−.03	.01	.09	.56
31	N	.58	−.16	.04	−.01	.00
32	E	−.15	.38	.05	−.16	.31
33	O	.03	.07	.30	.14	.09
34	A	−.03	.32	−.12	.28	.13
35	C	−.10	.27	.08	.08	.52
36	N	.52	−.08	−.12	−.38	.01
37	E	−.23	.62	−.02	.11	.14
38	O	−.09	−.05	.22	−.08	−.01
39	A	−.02	.24	−.18	.62	.07
40	C	−.05	.30	−.04	.19	.53
41	N	.58	−.18	−.08	−.07	−.25
42	E	−.33	.48	.06	.27	.11
43	O	.01	.16	.57	.03	.00
44	A	−.17	.00	.04	.51	−.18
45	C	−.21	.13	−.13	.10	.54
46	N	.65	−.25	.04	.03	−.11
47	E	−.06	.33	.05	−.03	.19
49	A	.07	.23	.02	.35	.22
50	C	−.16	.28	.00	−.05	.60
51	N	.53	−.05	.02	−.09	−.36
52	E	−.20	.43	.01	.00	.33
53	O	−.07	.14	.55	−.04	.12
54	A	.04	−.08	.02	.46	−.12
55	C	−.28	−.07	.01	−.02	.62
56	N	.50	−.17	.04	−.21	−.28
57	E	−.28	.24	.07	.12	.15
58	O	−.08	.06	.64	−.21	.05
59	A	.01	−.03	−.11	.36	.10
60	C	−.01	.19	.09	.04	.55

Note. N = 419. Congruence for overall solution = .86; congruence for individual factors = 0.86 for Neuroticism, 0.82 for Extraversion, 0.82 for Openness, 0.84 for Agreeableness and 0.87 for Conscientiousness. Items 8 and 48 (both Openness) removed due to minimal loadings on Openness. N = Neuroticism, E = Extraversion, O = Openness, A = Agreeableness, C = Conscientiousness. Item loadings on intended factor are bolded.

Influence of Demographics and Disease Features

Finally, we examined whether the above analyses would be significantly affected by the influence of demographic features (age, gender, education), depression, or method of recruitment (research subject, routine clinical patient, or clinical referral). The factorial validity analyses were not conducive to this approach due to sample size limitations and similarly we were unable to compare subgroups of self-informant agreement because only a sub-sample of subjects had informant ratings. Method of recruitment was examined as a potential moderator of psychometric properties because recent findings suggest that MS patients recruited through either research, routine clinical care, or clinical referral channels may show different response patterns on neuropsychological tests (Duquin, Parmenter & Benedict, 2008). Demographic subgroups were defined as follows: men vs. women; high school or less vs. greater than high school education; and persons below the sample average age of 47 vs. those 47 and older. Depression subgroups were defined as persons scoring at or above the BDI-FS cutpoint of 3 versus those below. The significance of differences in Cronbach’s alphas across strata for each demographic or clinical characteristic were determined via Feldt’s W (1987). Throughout, due to the large number of comparisons, we used a conservative p value of .01 to determine significance. Analyses of internal consistency are presented in Table 4. Few factors appeared to be associated with these psychometric properties of the NEO-FFI. The Openness scale showed lower internal consistency among persons with high school or less education, as did Extraversion. Younger persons showed greater internal consistency than older persons on the Neuroticism scale, but lower internal consistency on Openness.

Table 4.

NEO-FFI Internal Consistency Across Subgroups Defined by Demographic Variables, Depression and Recruitment

Patient Subgroup	Internal Consistency (Cronbach’s Alpha)
	N	E	O	A	C
Gender
Men    (n’s = 98)	.85	.76	.71	.73	.82
Women    (n’s = 321)	.88	.81	.71	.73	.84
Education
≤ HS Education    (n = 134)	.86	.73**	.51***	.73	.84
> HS Education    (n = 283)	.88	.82**	.74***	.73	.84
Age
< 47 years old    (n = 190)	.90***	.82	.61***	.75	.84
≥ 47 years old    (n = 224a)	.84***	.78	.76***	.71	.83
Depression
BDI Fast Screen < 3    (n = 208)	.80	.79	.71	.76	.83
BDI Fast Screen ≥ 3    (n = 187)	.81	.74	.71	.70	.82
Recruitment Method
Research    (n = 93)	.90	.83	.70	.74	.87
Routine Clinical    (n = 198)	.86	.78	.71	.73	.81
Clinical Referral    (n = 117)	.85	.79	.72	.75	.85

Notes. N = Neuroticism, E = Extraversion, O = Openness, A = Agreeableness, C = Conscientiousness domains of NEO-FFI. Age groups based on above/below mean sample age of 47. Cognitive impairment determined by z-score of −1.5 or greater on 2 or more tests.

^**= difference between values for a given patient characteristic significant at p < .01 and

^***= difference between values for a given patient characteristic significant at p < .001 by Feldt W test (1987).

Discussion

These results from a large sample of MS patients support the internal consistency and factorial validity of the NEO-FFI, and to a lesser extent, supports the inter-rater reliability of the NEO-FFI across self- and informant-report forms.

Beginning with the internal consistency estimates, we report values that are roughly equivalent to those reported in the NEO-FFI manual (Costa & McCrae, 1992) with the exception of informant ratings of Openness. This alpha was also lower than that for self-report Openness obtained in our sample. In addition, informants in our sample were more internally consistent in ratings of Agreeableness and Conscientiousness than were patients who rated themselves. A similar trend is evident in the normative sample, with alphas of .73 versus .86 for self- versus informant-reported Agreeableness and .84 versus .90 for Conscientiousness. Exploring possible contributions of recruitment strategy on reliability coefficients yielded no significant differences in self-report alphas observed in subjects who were recruited through research venues, routine clinical visits or clinical referrals. Comparison of reliability coefficients between impaired and cognitively intact patients yielded equivalent estimates of internal consistency with the exception of Extraversion, where impaired patients were slightly less internally consistent (0.77 versus 0.85). Exploratory analyses of alternate classifications of cognitive impairment revealed that alphas were surprisingly robust to the most severe levels of cognitive impairment in our sample and self-informant correlations were nearly as robust but did show signs of decline in the very lowest performing subjects.

These internal consistency data, from both a patient and an informant sample, are important for clinicians and researchers interested in using this personality measure with MS patients. The findings suggest that the neurological deficits and symptom fluctuations commonly seen in this population do not disrupt scale homogeneity. Moreover, it is significant that the cognitive deficits and mood fluctuations that may be associated with MS do not undermine the internal consistency of informant report scales either. Internal consistency reliability did not vary with recruitment method, providing further support for the internal-consistency of the NEO-FFI across clinical and research venues.

Procrustes rotation suggested the expected five-factor structure emerges in MS patient self-report, albeit two Openness items appeared relatively unrelated to this scale in this sample. Overall, our impression of these data are that the Procrustes rotation supports the factorial validity of the NEO-FFI, and in turn the FFM in MS. This suggests that the dimensions of personality underlying the NEO-FFI are not meaningfully distorted by disease-related fluctuations or symptoms that may be associated with MS. The minor caveat of note here is that two Openness items showed low congruence to target loadings. Item 8, “Once I find the right way to do something, I stick to it,” loaded more on Conscientiousness than on Openness. Item 8 has also been observed to load more on Conscientiousness than on Openness in Greek (Panayiotou, Kokkinos, & Spanoudis, 2004) and Austrian (Murray, Rawlings, Allen & Trinder, 2003) as well as Swiss and Spanish (Aluja, Garcia, Rossier & Garcia, 2005) samples of normal adults, suggesting that the item is more indicative of Conscientiousness in other populations. Similarly, item 48, “I have little interest in speculating on the nature of the universe or the human condition,” also showed lower than expected congruence to the target loading. Although the precise reasons for the failure of these two Openness items to load adequately are uncertain, and no prior studies have specifically observed problems with item 48, the results are not unprecedented. Openness has generally shown relatively weaker estimates of internal consistency as well as factor loadings (Aluja, Garcia, Rossier & Garcia, 2005). Findings also suggest that in our sample, the instrument conformed equally well to the factor structure reported by McCrae and Costa (2004) in a combined high school and BLSA sample. Thus, factorial validity appears to be preserved in this sample whether one uses a theoretical or empirical standard of comparison.

Internal consistency estimates for most NEO-FFI domains tended not be significantly associated with demographic or clinical factors. The exception was Openness, which tended to be less internally consistent in younger and less educated persons; Extraversion was also more internally consistent in younger than older, and in cognitively intact vs. impaired subjects. What this means is that these traits may be measured less precisely, or with greater error, in these MS patient subgroups. Clinicians should consider such patients’ true standing on these traits to fall within a potentially wide range around their observed score, and researchers should be aware that increased measure error in these subgroups may attenuate associations with outcomes in regression models.

Analysis of self-other agreement indicated agreement between self and informant reports roughly comparable to that observed in the normative sample as a whole (Costa & McCrae, 1992). Statistical comparison to the normative sample informant correlations was possible for spouse/domestic partner informants only, and this yielded no significant differences of spouse/domestic partner and self rating agreement between the present sample and the manual sample. Clinicians however may wish to discuss rating discrepancies with patients and corroborative raters in an effort to reconcile discrepancies. Both pieces of information may provide more accurate information than either source of ratings alone.

Our study has its limitations. The sample size of MS informant-reported NEO-FFI, although the largest in the published literature, did not permit full investigation of its factor structure or sub-group analyses of self-information agreement. A relatively lower Cronbach’s alpha among MS informants on Openness may have attenuated cross-observer correlations and comparability with informant ratings of normative sample participants on this factor. In addition, direct comparison of internal consistency, self-informant correlations and factor structure in MS patients versus those found in national norms for non-diseased patients are complicated by numerous extraneous variables such as demographic differences. Also, informant data reported in the manual were weighted in order to balance out peer and spouse ratings, giving more weight to the latter, whereas our informants included a higher proportion of spouse/domestic partners and were un-weighted. However, these differences would be expected to produce discrepancies between our findings and those of the normative sample, rather than the large degree of similarity we found, and spouse agreement in our sample did not differ statistically from spouse agreement in the manual sample. Last, it should be noted that we chose to investigate the NEO-FFI, rather than the parent NEO-PI-R, and we found that the psychometric properties of this abbreviated form held up remarkably well, and we believe it is a useful tool in contexts in which testing time or fatigue is a concern.

In conclusion, we report on the NEO-FFI validity, and in turn the FFM validity, in the largest MS sample to date and find that the results support the use of the NEO-FFI with MS patients, with some minor caveats. Clinicians wishing to assess potential personality changes, dispositional risks for poorer prognosis, or general personality functioning in the context of MS will find the NEO-FFI a valid and reliable tool for personality assessment in this population.