Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Jun 1.
Published in final edited form as: Cortex. 2011 Mar 10;48(6):674–682. doi: 10.1016/j.cortex.2011.03.001

Neuropsychological correlates of dominance, warmth, and extraversion in neurodegenerative disease

Marc Sollberger 1,2,3,4, Christine M Stanley 1,2, Robin Ketelle 1,2, Victoria Beckman 1,2, Matthew Growdon 1,2, Jung Jang 1,2, John Neuhaus 5, Joel H Kramer 1,2, Bruce L Miller 1,2, Katherine P Rankin 1,2
PMCID: PMC3132224  NIHMSID: NIHMS278447  PMID: 21470601

Abstract

Introduction

Changes in personality differ qualitatively and quantitatively between patients with different neurodegenerative diseases, likely due to divergent patterns of regional neurodegeneration. Regional damage to circuits underlying various cognitive and emotional functions have been associated with interpersonal traits like dominance, extraversion, and warmth in patients with neurodegenerative diseases, suggesting that personality may in part be mediated by these more basic neuropsychological functions. In this study, we hypothesized that different combinations of cognitive, neuropsychiatric, and emotional measures would predict different interpersonal traits in patients with neurodegenerative diseases.

Methods

A battery of cognitive, neuropsychiatric, and emotional measures was administered to 286 patients with various neurodegenerative diseases such as Alzheimer’s disease, behavioral variant frontotemporal dementia, semantic dementia, and progressive supranuclear palsy, and informants described patients’ dominance, extraversion, and warmth using the Interpersonal Adjective Scales (IAS) personality questionnaire. Regression modeling was performed to identify which neuropsychological factors uniquely predicted current personality, controlling for age, gender, and premorbid personality.

Results

Social dominance covaried with patients’ capacity for cognitive control and verbal fluency. Conversely, warmth did not rely on these executive or verbal skills, but covaried primarily with patients’ capacity for emotional responsiveness. Extraversion, representing a blend of dominance and warmth, demonstrated an intermediate degree of relationship to both executive/verbal and emotional functions.

Conclusions

These findings suggest that different personality traits are partly subserved by specific cognitive and emotional functions in neurodegenerative disease patients. While this study was performed in the context of brain damage, the results raise the question of whether individual differences in these neuropsychological abilities may also underlie variability in normal personality.

Keywords: personality, neurodegenerative disease, cognition, emotion

1. Introduction

Personality change is a common and often early symptom in neurodegenerative disease (Duchek et al., 2007; Sollberger et al., in press). Specific patterns of personality change can be directly associated with the severity and the type of the brain disease (Sollberger et al., in press), providing strong evidence that personality arises in part from a neurologic basis. Indeed, disease-specific personality changes have recently been associated with degenerative lesions to specific brain structures (Sollberger et al., 2009). These patients also show focal patterns of cognitive impairments as also neuropsychiatric and emotional abnormalities, raising the question of whether the personality change observed to occur in these patients can be better understood as changes to more fundamental neuropsychological functions like working memory, verbal fluency, or emotional reactivity.

One study published in thirty-eight patients with Alzheimer’s disease (AD) used the Five-Factor Model of Personality (McCrae and John, 1992) to examine correlations between personality traits and global cognitive functioning (Chatterjee et al., 1992). Low extraversion correlated with high Mini-Mental State Examination (MMSE) scores, but no other correlations were found between MMSE and the traits of neuroticism, openness, conscientiousness, or agreeableness. Apart from a probable lack of power and restriction of range of personality scores within a single diagnostic group, the use of a crude overall measure for cognitive functioning like the MMSE may lack the precision required to delineate the complex cognitive characteristics underlying a personality trait. Moreover, personality traits likely involve more emotional and social as well as basic cognitive functions (Sollberger et al., 2009).

In this study, we hypothesized that different combinations of cognitive, neuropsychiatric, and emotional measures would predict the different personality traits of the Interpersonal Adjective Scales (IAS) in patients with neurodegenerative diseases. Based on the circumplex model of personality underlying the IAS (Fig. 1) (Wiggins, 1995), and evidence for how this model corresponds with specific neuroanatomic substrates (Sollberger et al., 2009), we proposed three hypotheses. 1) Individual variability in dominance, the personality trait characterized by a person’s tendency to be assertive and highly active towards others, would be partly explained by neuropsychiatric functioning like apathy and depression and cognitive functioning like executive and verbal skills, but would not be significantly related to emotional functioning. 2) Conversely, individual variability in warmth, the personality trait characterized by a person’s tendency to derive reward from positive affiliations with others, would be partly explained by neuropsychiatric and emotional functioning, while standard cognitive functions would be unrelated. 3) Extraversion, the personality trait representing a blend of dominance and warmth, would be partly explained by individual variation in cognitive, neuropsychiatric, and emotional functioning. To examine these hypotheses, we examined how measures of cognitive, neuropsychiatric, and emotional functioning interrelated to predict three aspects of personality (dominance, warmth, and extraversion) in a large cohort of patients with various neurodegenerative diseases.

Figure 1.

Figure 1

Open in a new tab

The Interpersonal Circumplex of the Interpersonal Adjective Scales: Eight interpersonal traits, derived from two orthogonal dimensions, agency and affiliation, are evenly distributed around the circumference (adapted from Wiggins, 1995).

2. Methods

2.1. Subjects

A total of 286 patients diagnosed with one of eight neurodegenerative diseases were recruited into the study. 66 patients were diagnosed with behavioral variant frontotemporal dementia (bvFTD) (Neary et al., 1998), 49 with semantic dementia (SemD) (Neary et al., 1998), 10 with progressive nonfluent aphasia (PNFA) (Neary et al., 1998), 91 with Alzheimer’s disease (AD) (McKhann et al., 1984), 14 with amnestic mild cognitive impairment (aMCI) (Petersen et al., 1999), 10 with AD plus dementia with Lewy bodies (DLB) (McKeith et al., 2005), 26 with a corticobasal syndrome (CBS) (Boxer et al., 2006), and 20 with progressive supranuclear palsy (PSP) (Boxer et al., 2006; Litvan et al., 1996). Ten bvFTD patients and one PNFA patient had coexisting motor neuron disease. We included patients from different diagnostic groups into the study specifically to increase the variance of both interpersonal trait scores and cognitive, neuropsychiatric, and emotional measures. Combining patients across diagnostic groups allowed the use of parametric statistical methods, increasing the study’s statistical power to detect correlations between interpersonal trait scores and their probable underlying neuropsychological correlates.

2.2. Interpersonal Adjective Scales (IAS)

The IAS is a well-validated self- or other-report questionnaire based on the circumplex model of personality, which aims to measure individual differences in interpersonal traits (Wiggins, 1995) (Fig. 1). The instrument has been detailed elsewhere (Rankin et al., 2005; Sollberger et al., 2009).

Loss of self-awareness is common in neurodegenerative diseases (Rankin et al., 2005). Thus, we assumed that not all patients in this study were capable of producing a valid self-assessment and used informant ratings to measure the interpersonal traits of all subjects. We considered informant ratings to be valid estimates of the subjects interpersonal traits because behaviors described by the IAS all are observable, not only by the subject, but by people who frequently interact with them. Spouses, relatives or close friends who knew the patients a minimum of 5 years were asked to fill out the personality questionnaire twice, first describing the patient’s current interpersonal characteristics, and then describing how the patient was before the onset of disease (premorbid interpersonal characteristics). Though self and informant ratings of personality show unique variances in healthy adults, there is evidence for a high convergent validity between the two rating types (Connolly et al., 2007). In patients with dementia, collecting data from caregivers and others who know the patient well is an effective and reliable method for assessing personality traits (Siegler et al., 1994; Strauss et al., 1993), and informant ratings using the IAS in particular have excellent internal and temporal reliability (Kurtz et al., 1999).

Based on recent neuroimaging findings for IAS interpersonal traits (Sollberger et al., 2009), we derived three interpersonal composite scores from the IAS and labeled them as Dominance (derived from an arithmetic combination of dominance and submissiveness scores), Extraversion (extraversion & introversion scores), and Warmth (warmth & coldness scores). We did not choose to use the scores from the fourth composite measuring ingenuousness and arrogance scores because this factor has shown poor diagnostic differentiation and anatomic specificity in previous studies with neurodegenerative samples (Sollberger et al., 2009).

2.3. Neuropsychological Measures

All subjects underwent neuropsychological testing with a comprehensive brief battery that has been described in detail elsewhere (Rosen et al., 2002). To measure the desired cognitive constructs more precisely, we calculated summary scores based on carefully selected combinations of neuropsychological tests. Scores designed to best represent each cognitive construct were generated a) by extracting variance associated with a test from another, related test, to derive a more “pure” measure of the desired construct, b) by creating composite scores across multiple tests with a shared component, to maximize the shared element across all of the tests, or c) by a combination of both methods. Measures which were entered into composite scores were weighted according to their mean scores in a sample of 150 neurologically healthy older adults. Pure visuospatial functioning was not included as a cognitive construct due to the lack of existing literature or face validity for any relationship with social behavior. Similarly, of the 12 neuropsychiatric syndromes measured by the Neuropsychiatric Inventory (NPI) (Cummings, 1997), only the five syndromes which appeared to be relevant to social interactions were selected (e.g., the scales for behaviors such as aberrant sleep, eating, and motor behavior were excluded).

Language Domain

1. Semantic Knowledge Score

This is a measure of individuals’ semantic knowledge about objects in their environment. The score is the total words spontaneously named within 20 seconds, or with the benefit of semantic cues, on the Boston Naming Test (BNT) (Kaplan et al., 1983).

2. Word Access Score

A measure of individuals’ ability to rapidly access and verbalize words on demand. The score is the standardized residual remaining in a verbal fluency score (a weighted composite of lexical and categorical verbal fluency scores) after variance associated with the 4-line figural (non-verbal) fluency test was extracted.

Memory Domain

3. Verbal Memory Score

A measure of individuals’ ability to learn, encode, and freely retrieve verbal information. The score is the number of elements recalled after a 10-minute delay on the California Verbal Learning Test, Mental Status Version (CVLT-MS) (Delis et al., 1987).

4. Visuospatial Memory Score

A measure of individuals’ ability to learn, encode, and freely retrieve non-verbal information. The score is the number of elements recalled after a 10-minute delay on a complex figure recall task (Benson complex figure) (Rosen et al., 2002).

Executive Functioning Domain

5. Verbal Working Memory Score

A measure of the individual’s capacity to temporarily hold and manipulate auditory information. This score is a composite of the number of digits backwards an individual can obtain on the Digits Span Test (Wechsler, 1981) and their Correct Total Trials 1–4 of the CVLT-MS (Delis et al., 1987).

6. Generation Score

A measure of the fluency with which an individual can generate cognitive material, whether verbal or non-verbal. This score is a weighted composite of lexical (# D-words/minute), categorical (# animals/minute), and 4-line figural fluency tests [# designs/minute; subtest of the Delis Kaplan Executive Functioning Scale Design Fluency Test (Delis et al., 2001)].

7. Set-Shifting Score

A measure of the individual’s ability to rapidly shift back and forth between cognitive sets. This score is the number of lines per minute (speed) of performance on the modified Trails B Test, which uses days of the week rather than letters of the alphabet (Reitan, 1958).

8. Cognitive Control Score

A measure of the individual’s ability to rapidly and accurately apply a superordinate cognitive rule to suppress an easier, automatic response. This score is the standardized residual remaining after variance associated with Word Access Score (below) was extracted from the Stroop Interference score (Stroop, 1935).

Neuropsychiatric Domain

9 – 13. Depression, Anxiety, Euphoria, Disinhibition, and Apathy Scores

These scores represent typical neuropsychiatric symptoms seen in neurodegenerative disease patients. The scores were derived from frequency by severity products of each score from the NPI (Cummings, 1997).

Emotional Domain

14. Emotional Responsiveness Score

A measure of the tendency of the individual to respond on an emotional level to the social and emotional behavior of others in their day-today life. The score is the standardized residual of Empathic Concern, a subscale of the Interpersonal Reactivity Index (IRI) (Davis, 1983), after variance associated with Perspective Taking (another subscale of the IRI) was extracted. We used IRI scores based on informant-reports of patients’ real-life empathic responsiveness.

2.4. Data Analyses

General linear models were used to compare groups for age, education, MMSE, Clinical Dementia Rating Scale (CDR), Dominance, Extraversion, and Warmth scores. Fisher’s exact test was used to assess for group differences in gender distribution. Inferential statistics were performed in the following order:

1. Partial Correlation Analyses

To reduce the number of predictor variables for the multiple linear regression analyses, partial correlations between each of the three personality traits and potential predictor variables were performed, adjusting for age, gender, and premorbid personality for the respective trait. We set a partial correlation coefficient (pr) either < −.20 or > .20 as the lower limit for inclusion in the next stage of analysis, which according to Cohen (Cohen, 1988) represents an effect size of small-to-medium magnitude.

2. Multiple Linear Regressions (Main Effect Analyses)

To determine the unique contribution of each predictor variable that passed the criterion level of association (pr < −.20 or > .20) in the partial correlation analyses, we performed multiple linear regression analyses using the modified Allen-Cady backward selection technique specified in Vittinghoff (Vittinghoff et al., 2004). Age, gender, and premorbid Dominance, Extraversion, or Warmth score were forced into the model as confounding covariates. At each step, the predictor with the largest p-value was excluded, and the multiple regression analysis was then rerun with the remaining predictors, until all predictors in the model satisfied the inclusion threshold (final model). A very permissive inclusion threshold of p < .20 was set up to ensure that predictors which remained in the model showed at least a modest independent relationship to Dominance, Extraversion, or Warmth score.

3. Multiple Linear Regressions including Diagnostic Groups as Confounding Covariates (Error Check)

Main effect analyses do not rule out the possibility that significant findings hold true only in one diagnostic group and do not represent a generalisable relationship between an interpersonal trait and another neuropsychological measure. To control for the effect of group membership, we parameterized each diagnosis (0 = no, 1 = yes) and included all 8 diagnostic groups as confounding covariates (using 7 dummy variables to represent the 8 groups) into the final model of each interpersonal trait. The results of this conservative error check must be considered in light of the main effects results, however, because controlling for group membership in this manner has a high likelihood of weakening or rejecting real relationships, i.e., it is susceptible to inflated Type 2 error.

3. Results

An omnibus analysis of variance using a general linear model with an alpha level of p < .05 showed significant differences in age, but not in gender and education, between diagnostic groups (Table 1). Specifically, bvFTD patients were significantly younger than AD and PSP patients (p<.05 based on a post-hoc Tukey-Kramer test). There were also significant differences in MMSE and CDR scores between diagnostic groups. In particular, aMCI patients showed significantly higher MMSE scores and lower CDR scores than bvFTD and SemD patients (p<.05). All three interpersonal trait scores, i.e., Dominance, Extraversion, and Warmth, were significantly different between diagnostic groups. In particular, interpersonal trait scores of the aMCI group were significantly higher than trait scores of most other diagnostic groups. Leaving out the aMCI group, Dominance scores were quite equally decreased across groups. In contrast, decreases of Warmth scores were disease-specific, showing the lowest scores in the bvFTD group, followed by the SemD and CBS groups.

Table 1.

Characteristics of patient sample classified by diagnostic group. F-statistic and p-values are for overall diagnostic group differences. Values are listed as Mean (Standard Deviation).

Variables bvFTD SemD PNFA AD aMCI AD&DLB CBS PSP F-statistic (df) p-value
n = 66 n = 49 n = 10 n = 91 n = 14 n = 10 n = 26 n = 20
Age 61.1 (8.1) 64.6 (7.9) 61.1 (8.7) 67.3 (11.0) 68.9 (9.5) 65.6 (9.6) 64.3 (8.2) 70.1 (8.6) 3.93(7, 278) <.001
Education 16.0 (2.5) 16.1 (3.0) 15.5 (1.7) 15.3 (3.4) 17.7 (2.2) 15.1 (3.7) 15.0 (2.3) 15.6 (3.3) 1.57(7, 259) .14
Gender (M/F) 42/24 30/19 4/6 43/48 9/5 7/3 12/14 12/8 χ2(7, N=286) = 8.27 .31
MMSE 22.3 (8.0) 19.1 (8.9) 22.3 (8.8) 20.1 (6.0) 28.6 (1.8) 24.0 (2.5) 22.6 (6.5) 25.5 (3.4) 4.78(7, 261) <.001
CDR 1.3 (.7) .9 (.6) .7 (.5) 1.0 (.5) .4 (.2) .8 (.3) .8 (.8) .8 (.4) 5.30(7, 241) <.001
Dominance 30.8 (12.9) 37.4 (11.0) 33.6 (13.3) 33.2 (11.8) 44.5 (9.4) 30.4 (13.4) 39.5 (13.7) 29.8 (13.3) 4.68(11, 257) <.001
Extraversion 27.1 (17.0) 34.5 (19.6) 35.7 (20.5) 38.3 (13.8) 51.9 (13.0) 30.7 (12.7) 31.6 (13.2) 21.5 (19.4) 6.71(11, 257) <.001
Warmth 32.6 (14.0) 37.7 (16.9) 45.1 (15.3) 48.1 (11.3) 54.6 (5.8) 48.0 (15.5) 38.6 (14.3) 45.1 (15.0) 11.04(11, 257) <.001
Open in a new tab

Abbreviations: bvFTD = Behavioral Variant Frontotemporal Dementia; SemD = Semantic Dementia; PNFA = Progressive Nonfluent Aphasia; AD = Alzheimer’s Disease; aMCI = Amnestic Mild Cognitive Impairment; DLB = Dementia with Lewy Bodies; CBS = Corticobasal Syndrome; PSP = Progressive Supranuclear Palsy. MMSE, Mini-Mental State Examination (0–30); CDR, Clinical Dementia Rating (0–3).

3.1. Partial Correlations

Dominance score correlated at or above the established statistical threshold (pr < −.20 or > .20) with one language variable (Word Access), all four executive functioning variables (Generation Score, Set Shifting, Cognitive Control, and Verbal Working Memory), and Apathy (Table 2). Extraversion score correlated at or above the same threshold with one language variable (Word Access), two executive functioning variables (Generation Score and Cognitive Control), and two neuropsychiatric variables (Disinhibition and Apathy) (Table 2). Warmth score correlated with three of the five neuropsychiatric variables (Euphoria, Disinhibition, and Apathy) and with Emotional Responsiveness (Table 2).

Table 2.

Partial correlations between interpersonal trait scores and selected cognitive, neuropsychiatric, and emotional measures, adjusted for the respective premorbid interpersonal trait score, age, and gender. Values are listed as correlation coefficients (pr).

Variables Dominance Extraversion Warmth
Language Domain
 Semantic Knowledge (n = 252) −.06 −.03 .05
 Word Access (n = 191) .29 .25 .17
Memory Domain
 Verbal Memory (n = 234) .11 −.01 −.03
 Visuospatial Memory (n = 229) .20 .08 .01
Executive Functioning Domain
 Verbal Working Memory (n = 214) .21 .05 −.09
 Generation Score (n = 190) .36 .32 .20
 Set Shifting (n = 214) .26 .19 .05
 Cognitive Control (n = 155) .35 .21 .04
Neuropsychiatric and Emotional Domains
 Depression (n = 209) −.02 −.10 .03
 Anxiety (n = 210) −.05 −.11 −.09
 Euphoria (n = 214) .11 .02 −.26
 Disinhibition (n = 213) .04 −.22 −.41
 Apathy (n = 206) −.34 −.53 −.47
 Emotional Responsiveness (n = 244) −.03 .16 .38
Open in a new tab

Variables: Semantic Knowledge: A measure of individuals’ semantic knowledge about objects in their environment. Word Access: A measure of individuals’ ability to rapidly access and verbalize words on demand. Verbal Memory: A measure of individuals’ ability to learn, encode, and freely retrieve verbal information. Visuospatial Memory: A measure of individuals’ ability to learn, encode, and freely retrieve non-verbal information. Verbal Working Memory: A measure of the individual’s capacity to temporarily hold and manipulate auditory information. Generation Score: A measure of the fluency with which an individual can generate cognitive material, whether verbal or non-verbal. Set-Shifting: A measure of the individual’s ability to rapidly shift back and forth between cognitive sets. Cognitive Control: A measure of the individual’s ability to rapidly and accurately apply a superordinate cognitive rule to suppress an easier, automatic response. Depression, Anxiety, Euphoria, Disinhibition, and Apathy Scores: These scores were derived from frequency by severity products of each score from the NPI. Emotional Responsiveness Score: A measure of the tendency of the individual to respond on an emotional level to the social and emotional behavior of others in their day-to-day life.

3.2. Multiple Linear Regressions

Dominance

Main Effect Analysis: A modified backward selection procedure was performed with the 6 predictors showing a pr < −.20 or > .20 with Dominance score (i.e., Word Access, Verbal Working Memory, Generation Score, Set Shifting, Cognitive Control, and Apathy), forcing premorbid Dominance score, age, and gender into the model. Verbal Working Memory and Set Shifting scores were removed from the model, because they showed no discernable unique relationship (p < .20) with Dominance score, leaving Apathy, Cognitive Control, Word Access, and Generation scores in the final model (Table 3). The final model explained 61% (R2adj.) of the total variance of Dominance score, representing a significant increase in explained variance [F (4, 107) = 17.62, p < .001, R2-change: 24%] compared to the covariates-only model (i.e., premorbid Dominance score, age, and gender). Apathy (standardized regression coefficient (β) = −.37), Cognitive Control (β = .34), and Word Access (β = .29) scores were significant independent predictors of Dominance in the final model (p < .05), whereas Generation Score (β = −.17) was not (p = .11). When this procedure was rerun after removal of two potentially influential outlier datapoints, the same four predictors remained in the model, explaining the total variance of Dominance score slightly better (R2-change: 26% vs. 24%) than the model with these outliers included. Analysis of Influence of Diagnostic Group Membership (Error Check): When adding diagnostic groups as confounding covariates to the final model, Apathy (β = −.47), Cognitive Control (β = .25), and Word Access (β = .29) scores remained significant predictors of Dominance score (p < .05) (Table 3). The considerable increase in the β of Apathy score when controlling for diagnostic groups suggested some suppression was operating in the model. That change of β was primarily driven by the bvFTD group, as shown by an increase of β from −.37 (main effect analysis) to −.45 when only controlling for bvFTD group membership (i.e., a bvFTD group and a group with all 7 other diagnoses together as confounding covariates). The suppression was driven to a lesser extent by the AD group (increase of β to −.41) and the AD&DLB group (increase of β to −.39). In contrast, the β of Cognitive Control score did not increase, but decreased considerably when controlling for diagnostic groups (from .34 to .25). The change of β was primarily driven by the AD group, as shown by a decrease of β from .34 to .30 when only controlling for AD group membership (i.e., an AD group and a group with all 7 other diagnoses together). For comparison, the β of Cognitive Control stayed around .33 and .35 when controlling for any of the other diagnostic groups separately. The decrease in the β of Cognitive Control score suggests disease-specific associations between Dominance and Cognitive Control in AD patients. The final model explained 63% (R2adj.) of the total variance of Dominance score, representing a significant increase in explained variance [F (4, 100) = 14.24, p < .001, R2-change: 19%] compared to the covariates-only model (i.e., diagnostic groups, premorbid Dominance score, age, and gender). The final model explained the total variance of Dominance score similarly well (R2-change: 18% vs. 19%) after removal of one potentially influential outlier datapoint.

Table 3.

Multiple linear regressions for variables predicting Dominance score adjusting for premorbid Dominance score, age, and gender (Main Effect Analysis) and, in addition, adjusting for diagnostic groups (Analysis of Influence of Diagnostic Group Membership). (n = 115).

Variables B SE t-value β sr2
Main Effect Analysis
Apathy −1.10 .18 −5.98* −.37 .12
Cognitive Control 4.19 .94 4.45* .34 .07
Word Access 3.85 1.17 3.28* .29 .04
Generation Score −.13 .08 −1.64 −.17 .01
Analysis of Influence of Diagnostic Group Membership
Apathy −1.38 .23 −6.14* −.47 .12
Cognitive Control 2.96 1.03 2.88* .25 .03
Word Access 3.95 1.24 3.16* .29 .03
Generation Score −.16 .09 −1.83 −.20 .01
Open in a new tab
*

p < .05. Abbreviations: B = regression coefficient; SE = standard error of regression coefficient; β = standardized regression coefficient; sr2 = squared semi-partial correlation coefficient.

Extraversion

Main Effect Analysis: A modified backward selection procedure was performed with the 5 predictors showing a pr < −.20 or > .20 with Extraversion score (i.e., Word Access, Generation Score, Cognitive Control, Disinhibition, and Apathy), forcing premorbid Extraversion score, age, and gender into the model. Generation and Disinhibition scores were removed from the model, because they showed no discernable unique relationship (p < .20) with Extraversion score, leaving Apathy, Cognitive Control, and Word Access scores in the final model (Table 4). The final model explained 57% (R2adj.) of the total variance of Extraversion score, representing a significant increase in explained variance [F (3, 108) = 25.34, p < .001, R2-change: 29%] compared to the covariates-only model (i.e., premorbid Extraversion score, age, and gender). All three predictors [Apathy (β = −.48), Cognitive Control (β = .17), and Word Access (β = .13)] were significant predictors of the total variance of Extraversion score (p < .05). When the procedure was rerun after removal of two potentially influential outlier datapoints, the same 3 predictors remained in the model, explaining the total variance of Extraversion score better (R2-change: 33% vs. 29%) than the model with these outliers included. Analysis of Influence of Diagnostic Group Membership (Error Check): After adding diagnostic groups to the final model as confounding covariates, only Apathy (β = −.47) remained a significant predictor of Extraversion score, whereas Cognitive Control (β = .14) and Word Access (β = .12) scores were no longer significant (p < .05) (Table 4). The decrease in the β of Cognitive Control score from .17 (main effect analysis) to .13 when controlling for diagnostic groups was primarily driven by controlling for SemD group membership (β decreased to .14), whereas the decrease in the β of Word Access score from .13 (main effect analysis) to .12 when controlling for diagnostic groups was mainly driven by controlling for the PNFA group (β decreased to .11) and, to a lesser extent, by controlling for the PSP group (β decreased to .12) and the aMCI group (β decreased to .12). The final model explained 57% (R2adj.) of the total variance of Extraversion score, representing a significant increase in explained variance [F (3, 101) = 15.14, p < .001, R2-change: 17%] compared to the covariates-only model (i.e., diagnostic groups, premorbid Extraversion score, age, and gender). When running the multiple regression analysis after removal of two potentially influential outlier datapoints, the model explained the total variance of Extraversion score better (R2-change: 20% vs. 17%) than with the two outliers included and Word Access score remained a significant predictor at p < .05 (β = .14).

Table 4.

Multiple linear regressions for variables predicting Extraversion score adjusting for premorbid Extraversion score, age, and gender (Main Effect Analysis) and, in addition, adjusting for diagnostic groups (Analysis of Influence of Diagnostic Group Membership). (n = 115).

Variables B SE t-value β sr2
Main Effect Analysis
Apathy −2.00 .26 −7.70* −.48 .22
Cognitive Control 3.00 1.01 2.82* .17 .03
Word Access 2.44 1.16 2.10* .13 .02
Analysis of Influence of Diagnostic Group Membership
Apathy −1.97 .34 −5.87* −.47 .13
Cognitive Control 2.45 1.38 1.78 .14 .01
Word Access 2.28 1.32 1.72 .12 .01
Open in a new tab
*

p < .05. For abbreviations, see Table 3.

Warmth

Main Effect Analysis: A modified backward selection procedure was performed with the 4 variables showing a pr < −.20 or > .20 with Warmth score (i.e., Euphoria, Disinhibition, Apathy, and Emotional Responsiveness), forcing premorbid Warmth score, age, and gender into the model. Emotional Responsiveness, Apathy, and Disinhibition scores remained in the final model (p > .20), whereas Euphoria score did not (Table 5). The final model explained 56% (R2adj.) of the total variance of Warmth score, representing a significant increase in explained variance [F (3, 176) = 29.88, p < .001, R2-change: 22%] compared to the covariates-only model (i.e., premorbid Warmth score, age, and gender). All three predictors [Emotional Responsiveness (β = .26), Apathy (β = −.24), and Disinhibition (β = −.15)] were significant independent predictors of the total variance of Warmth score (p < .05). When the procedure was rerun after removal of two potentially influential outlier datapoints, the same three predictors remained in the model, explaining the total variance of Warmth score as well (R2-change: 23% vs. 22%) as the model with these outliers included. Analysis of Influence of Diagnostic Group Membership (Error Check): After adding diagnostic groups to the final model as confounding covariates, Emotional Responsiveness (β = .23) and Apathy (β = −.17) scores remained significant predictors of Warmth score, whereas Disinhibition score (β = −.10) was no longer significant (p < .05) (Table 5). The decrease in the β of all three predictors after controlling for diagnostic groups was driven by controlling for bvFTD group membership, highlighting the strong and specific relations between bvFTD and changes in warmth, emotional (i.e., Emotional Responsiveness), and neuropsychiatric (i.e., Apathy and Disinhibition) measures (see also Table 1). The final model explained 58% (R2adj.) of the total variance of Warmth score, representing a significant increase in explained variance [F (3, 163) = 11.49, p < .001, R2-change: 8%] compared to the covariates-only model (i.e., diagnostic groups, premorbid Warmth score, age, and gender). In this model, there were no potentially influential outlier datapoints.

Table 5.

Multiple linear regressions for variables predicting Warmth score adjusting for premorbid Warmth score, age, and gender (Main Effect Analysis) and, in addition, adjusting for diagnostic groups (Analysis of Influence of Diagnostic Group Membership). (n = 183).

Variables B SE t-value β sr2
Main Effect Analysis
Emotional Responsiveness 4.06 .83 4.91* .26 .06
Apathy −.89 .22 −4.00* −.24 .04
Disinhibition −.65 .26 −2.52* −.15 .02
Analysis of Influence of Diagnostic Group Membership
Emotional Responsiveness 3.56 .85 4.21* .23 .04
Apathy −.63 .23 −2.64* −.17 .02
Disinhibition −.41 .27 −1.52 −.10 .01
Open in a new tab
*

p < .05. For abbreviations, see Table 3.

4. Discussion

This study suggests that there are meaningful relationships between personality traits and specific cognitive and emotional functions in neurodegenerative disease patients. The capacity to engage in socially dominant behavior covaries with, and may be primarily mediated by, patients’ executive and verbal skills, particularly their capacity for cognitive control and verbal fluency. The capacity to be interpersonally warm, conversely, does not rely on these executive or verbal skills, but covaries primarily with patients’ capacity for emotional responsiveness. Extraversion, a personality trait representing a blend of dominance and warmth, demonstrates an intermediate degree of relationship to these executive/verbal and emotional functions and appears to rely on both. In addition, we show that apathy inversely relates to all three personality traits (dominance, warmth, and extraversion), and partly accounts for the decrease in prosocial behaviors observed in neurodegenerative disease patients, above and beyond their cognitive deficits. While this study was performed in patients with neurodegenerative diseases, a sample demonstrating wide variability in both personality and neuropsychological functioning, these results raise the question of whether individual differences in these cognitive and emotional functions may underlie variability in normal personality as well.

Associations between social dominance and measures of language and executive functioning were expected, since these cognitive skills would naturally aid one’s ability to interact with the environment to accomplish personal goals. These findings draw support from results of a recent structural neuroimaging study, which investigated neural substrates of interpersonal traits in patients with neurodegenerative diseases (Sollberger et al., 2009). In that study social dominance related to parts of the left lateral frontopolar and dorsolateral prefrontal cortices, which are both involved in cognitive control (Koechlin et al., 2003; Tanji and Hoshi, 2008). For the purposes of this study, capacity for cognitive control was derived from a cognitive measure testing patients’ ability to maintain and apply an abstract rule that superseded an automatic response (i.e., the Stroop interference effect (Stroop, 1935), after removing the effect of any motor speech deficits in our patients). As a personality construct, dominance describes one’s ability to control a social setting via negotiation and effective assertiveness, without resorting to aggressive impulses (Wiggins, 1995). Individuals in our study with lower levels of cognitive control were described as lacking dominance, potentially because they may not have had the executive resources to appropriately suppress their emotional impulses in favor of enacting complex negotiations involving abstract rules to achieve long-term goals. Among executive functions, it was this ability, not verbal working memory, set shifting, or the ability to generate cognitive material, that significantly predicted dominance. Similarly, the specific measure of language functioning that directly related to dominance was “word access”, a score derived from patients’ performance on verbal fluency measures, after accounting for their ability to generate cognitive material regardless of cognitive domain (i.e., verbal vs. nonverbal). Thus, whether or not they were capable of exerting cognitive control, patients who produced less verbal material on demand were less likely to successfully negotiate for what they want in a social situation, thus were rated as less dominant. Other language deficits, such as the inability to find the right word, or even frank loss of semantic knowledge about the world (both as measured by a confrontation naming test), did not appear to significantly impact how dominant a patient was perceived to be, perhaps implying that social dominance is more about style than substance.. However, the summary score used to represent the construct of “word access” in our study was a composite of verbal fluency measures that included a semantic fluency task, which suggests that this relationship between social dominance and semantic knowledge is nuanced, and requires further investigation.

Cognitive control and word access also underpinned extraversion, but to a lesser degree than dominance, and these skills did not show a significant relationship to interpersonal warmth. Since extraversion requires less interpersonal agency than dominance, and warmth requires none (Wiggins, 1995), this continuum of relationship across the three personality traits supports the hypothesis that verbal fluency and executive functioning specifically facilitate interpersonal agency, but not interpersonal connectedness. In contrast, a measure describing patients’ degree of real-life emotional responsiveness was the primary factor underlying warm personality, as hypothesized. This variable was constructed by taking the empathic concern subscale of the other-report IRI empathy questionnaire (Davis, 1983), and removing the effects of the perspective taking subscale, in order to remove any shared elements representing cognitive empathy and leave a more pure measure of real-life emotional responsiveness. The association between warm personality and emotional responsiveness is supported by structural neuroimaging findings showing that warm personality monotonically decreases with atrophy to right-sided posterior regions of the orbitofrontal cortex, right insula, right ventromedial prefrontal cortex, and right anterior temporal regions in neurodegenerative disease patients (Sollberger et al., 2009). Studies of normal individuals and non-neurodegenerative patients also show these regions to be key in emotion processing (Bar-On et al., 2003; Liberzon and Martis, 2006; Olson et al., 2007; Phan et al., 2002).

One might hypothesize that any diminishment of prosocial behaviors like dominance or warmth in neurodegenerative disease patients might solely be explained by generalized apathy, but our data suggests this is not the case. Overall apathy did make an independent contribution to the prediction of dominance, extraversion, and warmth; however, cognitive and emotional factors continued to significantly predict personality when the effects of apathy were factored out of the models. Also, generalized apathy made a more substantial contribution to the prediction of decreased dominance and extraversion than to loss of warmth. This difference reflects both the nature of the personality constructs, as well as their known neural substrates in neurodegenerative diseases: dominance and extraversion, both traits involving interpersonal agency (Wiggins, 1995), are more strongly associated with approaching, external-focused interpersonal behavior than warmth (Sollberger et al., 2009), thus may have been more susceptible to the effects of generalized apathy. It is also important to note that apathy was the only neuropsychiatric symptom that showed a relationship to these three interpersonal traits, e.g., more disinhibited people were not viewed as more extraverted, nor were depressed or anxious people viewed as less dominant or warm. This suggests that the behaviors being measured by this personality test are distinct constructs from those measured by the neuropsychiatric inventory.

Another study, which examined the relationship between personality traits and cognition (Chatterjee et al., 1992), correlated scores from the Five-Factor Model of Personality (McCrae and John, 1992) with the MMSE, and found no relationship in a sample of thirty-eight AD patients, apart from a counterintuitive inverse relationship between MMSE and extraversion, i. e., lower MMSE scores correlated with higher Extraversion scores. These results suggested that MMSE, a crude measure of overall cognitive functioning, may not be a specific enough test to demonstrate a relationship to personality traits. To explore this assumption, we performed post-hoc partial correlation analyses between MMSE and each of the three interpersonal traits, adjusting for the respective premorbid interpersonal trait, age, and gender. Indeed, as expected, the strength of relationship between MMSE score and any of the three interpersonal trait scores did not surpass our liberal threshold of pr < −.20 or > .20 (4% shared variance).

The combination of various neuropsychological predictors explained between 8% (warmth) and 20% (extraversion) of the variance in each personality trait, after accounting for diagnostic group membership, premorbid trait score, age, and gender. This fairly low percentage of explained variance suggests that many additional factors likely influence the expression of personality traits in patients with neurodegenerative diseases. However, the percentage of explained variance decreased much more for warmth after adjusting for group membership (from 23% to 8%) than for dominance (26% to 18%) or extraversion (33% to 20%). This finding is most likely due to the fact that warmth was considerably decreased in only three of the eight diagnostic groups (bvFTD, SemD, and CBS; see Table 1), thus was more disease-specific than dominance, which was more equally decreased across groups.

This study is limited by the fact that informants rated patients’ premorbid interpersonal traits retrospectively, though we saw no systematic bias in personality rating by diagnostic group as would be revealed in significant premorbid group differences. In healthy adults, informant ratings may introduce an information bias, even though there is evidence for a high convergent validity between self and informant ratings of personality traits in healthy adults (Connolly et al., 2007). In this study we could not use self ratings, because patients with neurodegenerative diseases often show loss of self-awareness (Rankin et al., 2005) and are therefore not capable of producing a valid self-assessment. Also, while this study was cross-sectional and thus was designed only to demonstrate correlations, longitudinal studies of patients as they develop both cognitive and personality changes would be ideal for identifying the degree to which cognitive, neuropsychiatric, and emotional changes exert an independent or combined causative influence on personality traits. Lastly, while the analysis has greater power to examine these relationships in a sample of subjects with both cognitive and personality changes, using a clinically heterogeneous sample like ours has its own limitations. In particular, the exact nature of personality-neuropsychological relationships may differ across diagnostic groups. Yet this study was not designed to enable us to look at that issue directly. As with any study of this type, the range of the data is restricted within diagnostic groups, both due to small group sizes and to homogeneity of clinical symptoms within diagnostic groups (e.g., all AD patients perform in the impaired range on memory testing, etc.), making within-group analyses inadvisable. However, the fact that these relationships remain largely intact even after correction by removing the effect of diagnostic group membership suggests that these personality-neuropsychological relationships may actually be generalizable beyond these patients with neurodegenerative diseases.

In conclusion, this study suggests that different personality traits are subserved by specific set of cognitive and emotional functions in neurodegenerative disease patients. While the capacity to engage in socially dominant behavior relies primarily on patients’ cognitive control and verbal fluency, the ability to be interpersonally warm relies primarily on patients’ capacity for emotional responsiveness. While this study was performed in the context of brain damage, the results raise the question of whether individual differences in these cognitive and emotional functions may also underlie variability in normal personality.

Acknowledgments

This research was supported in part by the National Institute on Aging (NIA) [5-K23-AG021606-02 and 1R01AG029577-01 to K.P.R., PPG P01-AG1972403 and AG19724-01A1 to B.L.M.]; the State of California, Alzheimer’s Disease Research Center of California (ARCC) [01-154-20]; the Larry L. Hillblom Foundation, Inc., [2002/2J to K.P.R.]; UCSF [GCRC-M01-RR00079]; the Swiss National Science Foundation (SNSF) [PBBEB-113383]; the Scientific Society Basle, and the Velux Foundation.

References

  1. Bar-On R, Tranel D, Denburg NL, Bechara A. Exploring the neurological substrate of emotional and social intelligence. Brain. 2003;126 (8):1790–800. doi: 10.1093/brain/awg177. [DOI] [PubMed] [Google Scholar]
  2. Boxer AL, Geschwind MD, Belfor N, Gorno-Tempini ML, Schauer GF, Miller BL, Weiner MW, Rosen HJ. Patterns of brain atrophy that differentiate corticobasal degeneration syndrome from progressive supranuclear palsy. Archives of Neurology. 2006;63 (1):81–6. doi: 10.1001/archneur.63.1.81. [DOI] [PubMed] [Google Scholar]
  3. Chatterjee A, Strauss ME, Smyth KA, Whitehouse PJ. Personality changes in Alzheimer’s disease. Archives of Neurology. 1992;49 (5):486–91. doi: 10.1001/archneur.1992.00530290070014. [DOI] [PubMed] [Google Scholar]
  4. Cohen J. Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum; 1988. [Google Scholar]
  5. Connolly JJ, Kavanagh EJ, Viswesvaran C. The convergent validity between self and observer ratings of personality: A meta-analytic review. International Journal of Selection and Assessment. 2007;15 (1):110–117. [Google Scholar]
  6. Cummings JL. The Neuropsychiatric Inventory: assessing psychopathology in dementia patients. Neurology. 1997;48:S10–6. doi: 10.1212/wnl.48.5_suppl_6.10s. [DOI] [PubMed] [Google Scholar]
  7. Davis M. Measuring individual differences in empathy; evidence for a multidimensional approach. Journal of Personality and Social Psychology. 1983;12:314–25. [Google Scholar]
  8. Delis D, Kaplan FB, Kramer JH. The Delis-Kaplan Executive Function System. Odessa, FL: Psychological Corporation; 2001. [Google Scholar]
  9. Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test: Adult Version. San Antonio, TX: The Psychological Corporation; 1987. [Google Scholar]
  10. Duchek JM, Balota DA, Storandt M, Larsen R. The power of personality in discriminating between healthy aging and early-stage Alzheimer’s disease. Journal of Gerontology Series B Psychological Sciences Social Sciences. 2007;62 (6):P353–61. doi: 10.1093/geronb/62.6.p353. [DOI] [PubMed] [Google Scholar]
  11. Kaplan EF, Googlass H, Weintraub S. The Boston Naming Test. Philadelphia: Lea & Febinger; 1983. [Google Scholar]
  12. Koechlin E, Ody C, Kouneiher F. The architecture of cognitive control in the human prefrontal cortex. Science. 2003;302 (5648):1181–5. doi: 10.1126/science.1088545. [DOI] [PubMed] [Google Scholar]
  13. Kurtz JE, Lee PA, Sherker JL. Internal and temporal reliability estimates for informant ratings of personality using the NEO-PI-R and IAS. Assessment. 1999;6:103–113. doi: 10.1177/107319119900600201. [DOI] [PubMed] [Google Scholar]
  14. Liberzon I, Martis B. Neuroimaging studies of emotional responses in PTSD. Annals of the New York Academy of Sciences. 2006;1071:87–109. doi: 10.1196/annals.1364.009. [DOI] [PubMed] [Google Scholar]
  15. Litvan I, Agid Y, Calne D, Campbell G, Dubois B, Duvoisin RC, Goetz CG, Golbe LI, Grafman J, Growdon JH, Hallett M, Jankovic J, Quinn NP, Tolosa E, Zee DS. Clinical research criteria for the diagnosis of progressive supranuclear palsy (Steele-Richardson-Olszewski syndrome): report of the NINDS-SPSP international workshop. Neurology. 1996;47 (1):1–9. doi: 10.1212/wnl.47.1.1. [DOI] [PubMed] [Google Scholar]
  16. McCrae RR, John OP. An introduction to the five-factor model and its applications. Journal of Personality Special Issue: The Five-Factor Model: Issues and Applications. 1992;60:175–215. doi: 10.1111/j.1467-6494.1992.tb00970.x. [DOI] [PubMed] [Google Scholar]
  17. McKeith IG, Dickson DW, Lowe J, Emre M, O’Brien JT, Feldman H, Cummings J, Duda JE, Lippa C, Perry EK, Aarsland D, Arai H, Ballard CG, Boeve B, Burn DJ, Costa D, Del Ser T, Dubois B, Galasko D, Gauthier S, Goetz CG, Gomez-Tortosa E, Halliday G, Hansen LA, Hardy J, Iwatsubo T, Kalaria RN, Kaufer D, Kenny RA, Korczyn A, Kosaka K, Lee VM, Lees A, Litvan I, Londos E, Lopez OL, Minoshima S, Mizuno Y, Molina JA, Mukaetova-Ladinska EB, Pasquier F, Perry RH, Schulz JB, Trojanowski JQ, Yamada M. Diagnosis and management of dementia with Lewy bodies: third report of the DLB Consortium. Neurology. 2005;65 (12):1863–72. doi: 10.1212/01.wnl.0000187889.17253.b1. [DOI] [PubMed] [Google Scholar]
  18. McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984;34 (7):939–44. doi: 10.1212/wnl.34.7.939. [DOI] [PubMed] [Google Scholar]
  19. Neary D, Snowden JS, Gustafson L, Passant U, Stuss D, Black S, Freedman M, Kertesz A, Robert PH, Albert M, Boone K, Miller BL, Cummings J, Benson DF. Frontotemporal lobar degeneration: a consensus on clinical diagnostic criteria. Neurology. 1998;51 (6):1546–54. doi: 10.1212/wnl.51.6.1546. [DOI] [PubMed] [Google Scholar]
  20. Olson IR, Plotzker A, Ezzyat Y. The Enigmatic temporal pole: a review of findings on social and emotional processing. Brain. 2007;130 (7):1718–31. doi: 10.1093/brain/awm052. [DOI] [PubMed] [Google Scholar]
  21. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Archives of Neurology. 1999;56 (3):303–8. doi: 10.1001/archneur.56.3.303. [DOI] [PubMed] [Google Scholar]
  22. Phan KL, Wager T, Taylor SF, Liberzon I. Functional neuroanatomy of emotion: a meta-analysis of emotion activation studies in PET and fMRI. Neuroimage. 2002;16 (2):331–48. doi: 10.1006/nimg.2002.1087. [DOI] [PubMed] [Google Scholar]
  23. Rankin KP, Baldwin E, Pace-Savitsky C, Kramer JH, Miller BL. Self awareness and personality change in dementia. Journal of Neurology, Neurosurgery & Psychiatry. 2005;76 (5):632–9. doi: 10.1136/jnnp.2004.042879. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Reitan RM. Validity of the Trailmaking Test as an indication of organic brain damage. Perceptual & Motor Skills. 1958:271–76. [Google Scholar]
  25. Rosen HJ, Gorno-Tempini ML, Goldman WP, Perry RJ, Schuff N, Weiner M, Feiwell R, Kramer JH, Miller BL. Patterns of brain atrophy in frontotemporal dementia and semantic dementia. Neurology. 2002;58 (2):198–208. doi: 10.1212/wnl.58.2.198. [DOI] [PubMed] [Google Scholar]
  26. Siegler IC, Dawson DV, Welsh KA. Caregiver ratings of personality change in Alzheimer’s disease patients: a replication. Psychology and Aging. 1994;9 (3):464–6. doi: 10.1037//0882-7974.9.3.464. [DOI] [PubMed] [Google Scholar]
  27. Sollberger M, Neuhaus J, Ketelle R, Growdon M, Jung J, Miller BL, Rankin KP. Interpersonal traits change as a function of disease type and severity in neurodegenerative disease. Journal of Neurology, Neurosurgery & Psychiatry. doi: 10.1136/jnnp.2010.205047. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Sollberger M, Stanley CM, Wilson SM, Gyurak A, Beckman V, Growdon M, Jang J, Weiner MW, Miller BL, Rankin KP. Neural basis of interpersonal traits in neurodegenerative diseases. Neuropsychologia. 2009;47 (13):2812–27. doi: 10.1016/j.neuropsychologia.2009.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Strauss ME, Pasupathi M, Chatterjee A. Concordance between observers in descriptions of personality change in Alzheimer’s disease. Psychology and Aging. 1993;8 (4):475–80. doi: 10.1037//0882-7974.8.4.475. [DOI] [PubMed] [Google Scholar]
  30. Stroop JR. Studies of interference in serial verbal reactions. Journal of Experimental Psychology. 1935;18 (6):643–662. [Google Scholar]
  31. Tanji J, Hoshi E. Role of the lateral prefrontal cortex in executive behavioral control. Physiological Reviews. 2008;88 (1):37–57. doi: 10.1152/physrev.00014.2007. [DOI] [PubMed] [Google Scholar]
  32. Vittinghoff E, Glidden DV, Shiboski SCEMC. Regression Methods in Biostatistics: Linear, Logistic, Survival, and Repeated Measures Models. New York: Springer; 2004. [Google Scholar]
  33. Wechsler D. Manual for the Wechsler Adult Intelligence Scale - Revised. New York: Psychological Corporation; 1981. [Google Scholar]
  34. Wiggins JS. Interpersonal Adjective Scales: Professional Manual. Odessa FL: Psychological Assessment Resources, Inc; 1995. [Google Scholar]

RESOURCES