Social Markers of Mild Cognitive Impairment: Proportion of Word Counts in Free Conversational Speech

Abstract

Background: Detecting early signs of Alzheimer’s disease (AD) and mild cognitive impairment (MCI) during the pre-symptomatic phase is becoming increasingly important for cost-effective clinical trials and also for deriving maximum benefit from currently available treatment strategies. However, distinguishing early signs of MCI from normal cognitive aging is difficult. Biomarkers have been extensively examined as early indicators of the pathological process for AD, but assessing these biomarkers is expensive and challenging to apply widely among pre-symptomatic community dwelling older adults. Here we propose assessment of social markers, which could provide an alternative or complementary and ecologically valid strategy for identifying the pre-symptomatic phase leading to MCI and AD. Methods: The data came from a larger randomized controlled clinical trial (RCT), where we examined whether daily conversational interactions using remote video telecommunications software could improve cognitive functions of older adult participants. We assessed the proportion of words generated by participants out of total words produced by both participants and staff interviewers using transcribed conversations during the intervention trial as an indicator of how two people (participants and interviewers) interact with each other in one-on-one conversations. We examined whether the proportion differed between those with intact cognition and MCI, using first, generalized estimating equations with the proportion as outcome, and second, logistic regression models with cognitive status as outcome in order to estimate the area under ROC curve (ROC AUC). Results: Compared to those with normal cognitive function, MCI participants generated a greater proportion of words out of the total number of words during the timed conversation sessions (p=0.01). This difference remained after controlling for participant age, gender, interviewer and time of assessment (p=0.03). The logistic regression models showed the ROC AUC of identifying MCI (vs. normals) was 0.71 (95% Confidence Interval: 0.54 – 0.89) when average proportion of word counts spoken by subjects was included univariately into the model. Conclusion: An ecologically valid social marker such as the proportion of spoken words produced during spontaneous conversations may be sensitive to transitions from normal cognition to MCI.

INTRODUCTION

High value is given to detecting early signs indicating the transition from normal cognitive aging to Mild Cognitive Impairment (MCI) when early intervention and treatment against Alzheimer’s disease (AD) could be most effective. However, it is difficult to distinguish early signs of MCI from normal cognitive aging. Although biomarkers such as CSF beta-amyloid, tau and neuroimaging markers have been extensively examined as early indicators of the pathological process for AD, assessing these biomarkers is expensive and challenging to apply widely among pre-symptomatic older adults. Social behavioral markers (which we propose to call “social markers”, a subset of behavioral biomarkers) offer a cost-effective alternative or complementary tool for detection of the transition from normal cognition to MCI in community dwelling older adults. One area of research that has been receiving a lot of attention is assessment of speech characteristics. Recent rapid technological advancement in the area of social interactions research could potentially serve to facilitate the identification of easily measured aspects of conversation that could reflect early cognitive changes in at risk older adults. This includes social network analysis at the macro level (e.g., who contacts whom and how this network pattern or structural network size changes as individuals transit from normal cognition to MCI) and analysis of spoken language at the micro level [1-4].

The present study is a part of a larger randomized controlled clinical trial (RCT) that assessed whether frequent conversations via webcam and Internet-enabled personal computers could improve cognitive function in older persons with either normal cognition or MCI (ClinicalTirals.gov registration number: NCT01571427). The study protocol and the results has been described in detail elsewhere [5, 6]. Briefly, in the larger intervention trial, social interaction sessions were conducted using semi-structured conversations with trained interviewers for 30~35 minutes a day, 5 days a week for 6 weeks (i.e., 30 sessions) among the intervention group. The intervention group also completed a weekly Internet survey that assessed social engagement activities during the previous week, while the control group was contacted by phone to complete the same weekly survey. This weekly telephone call was the only contact the control group had through the trial. Our primary outcome was to assess changes in cognitive functions measured by neuropsychological tests and our secondary outcome was to assess changes in psychological well-being. The trial showed that adherence to the protocol among the intervention group was high (89%; range, 77%-100%). At the post-trial assessment, the normal cognition group assigned to the intervention improved on tests of language-based executive function (e.g., verbal fluency category) compared to normal controls [5].

During the trial, it came to our attention that interviewers had more difficulty changing conversation topics or ending conversation sessions with some participants. Given that one of our aims of the trial was to standardize the interviews across interviewers, we decided to further investigate the observed variability in conversational flow in this follow-up study. Our hypothesis was that MCI individuals would have an impaired ability to identify social cues required for smooth interactions and would have difficulty taking turns in conversation (e.g., keep talking until being interrupted by interviewers). Previous research indicates that executive functioning underlies many everyday activities that are difficult for individuals with MCI [7]. MCI older adults may not self-monitor conversational content well as compared to cognitively intact older adults. This could result in a larger proportion of words produced by participants (as opposed to the interviewers) during timed conversational sessions among the MCI older adults. Analyses of proportion of words would be a potentially useful and ecologically valid social marker that could be incorporated with other daily markers of everyday cognitive activity such as daily computer usage [8], walking speed and its variability [9], medication adherence [10, 11] time out of house [12] and others associated with early cognitive change that can be monitored over time by using passive and unobtrusive in-home sensing technologies [13]. In the present study, we compared the proportion of total word counts produced by participants among MCI and cognitively intact participants using the recorded conversations from the larger social engagement RCT noted above [5].

METHODS

Participants

Eighty-three older adults aged 70 years and older were enrolled and randomized into the intervention and control groups; mean age, 80.5 years and 76% female [5]. The original prevention study’s inclusion and exclusion criteria are listed in Table 1. Out of forty-one participants assigned to the intervention group, 33 consented to allow their daily conversational intervention sessions to be transcribed for speech characteristics analyses (n=21 cognitively intact defined as Clinical Dementia Rating (CDR) [14]=0; n=12, MCI defined as CDR=0.5). Additionally, eight subjects (n=6 cognitively intact, n=2 MCI) recruited during a pilot-testing study who went through the same intervention protocol also consented and were included in this study, generating a total of 41 subjects reported here. As described above, the intervention group engaged in 30 to 35 minute semi-structured conversation sessions daily except weekends for six weeks with trained interviewers using Internet connected personal computers with a webcam.

Table 1.

Inclusion and exclusion criteria used in the trial.

Inclusion Criteria: 1. Age 70 or older 2. CDR=0 or 0.5 3. Sufficient vision and hearing to engage in conversation by PC system. 4. Sufficient English language skills to complete all testing. 5. General health status that will not interfere with ability to complete longitudinal study. Conditions that will likely lead to this problem are listed below in the Study Exclusions list. Exclusion Criteria: 1. Plan to start: taking new classes, traveling which requires more than two nights of stay away, or having significant social events such as a family wedding or a family reunion, during the scheduled prevention trial. 2. Diseases associated with dementia such as AD, ischemic vascular dementia, normal pressure hydrocephalus, or Parkinson’s disease. 3. Significant disease of the central nervous system such as brain tumor, seizure disorder, subdural hematoma, cranial arteritis. 4. Current (within the last 2 years) alcohol or substance abuse 5. Current major depression, schizophrenia or other major psychiatric disorder 6. Unstable or significantly symptomatic cardiovascular disease such as coronary artery disease with frequent angina, or congestive heart failure with shortness of breath at rest. 7. Active systemic cancer within 5 years of study entry. 8. Illness that requires > 1 visit per month to a clinician. 9. Progressive vision loss (Age-related macular degeneration already beginning to significantly degrade vision). 10. Need for oxygen supplementation for adequate function. 11. Medications:    a. Frequent use of high doses of analgesics.    b. Sedative medications except for those used occasionally for sleep (use limited to no more than twice per week).    c. APPLICABLE TO CDR = 0.5 group only: Subjects on unstable dosing of Cholinesterase inhibitors (need to be stable dosing for 2 months).

Conversation Session Format

In order to take full advantage of a synthetic conversational format, we placed an emphasis on spontaneous responses rather than structured answers (i.e., the participants had to organize their thoughts). We used unstructured conversations such as talking about participants’ “childhood memories”, “hobbies”, “siblings and parents”, and “movies/books” [5] in addition to conversations generated by showing a daily picture stimuli (e.g., what is happening in this photo and where do you think it was taken?) and story generations (e.g., daily alphabetical conversation: tell me a story about apples, alligators, or alumni). These daily prompts were used uniformly for each session. We randomly selected one recorded session each from the baseline, the 3rd and 6th week of the trial, i.e., three sessions per participant. A single transcriber manually transcribed the conversations to extract word counts spoken by interviewers versus participants. This measure was implemented as one of the strategies to standardize the quality of interviews across interviewers, i.e. that one interviewer did not speak more than the others. Three interviewers were used in this trial. Each interviewer was assigned to talk with a specific participant during the 6 week trial. We assigned MCI and normal participants equally to each interviewer although interviewers were blind to the cognitive status (MCI vs. normal) of participants.

Neuropsychological Tests

In the original prevention study, pre- and post- trial changes in neuropsychological test scores were primary outcomes. In the current study, we examined the correlation between proportion of words spoken by participants and cognitive functions using the neuropsychological test scores at baseline. The cognitive tests (and cognitive domains they tap) used are as follows:(1) Immediate Memory: the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) Word List Learning [15]; (2) Delayed Memory: CERAD Word List Delayed Recall [15]; (3) Language: composite of verbal fluency for letters (F, A and S) [16]; (4) Psychomotor Speed: Trail Making A [17]; (5) Executive function: Trail Making B [17] and verbal fluency for category animals [16]; (6) Selective Attention/inhibition: Stroop test [16]; and (7) Pre-morbid and general intelligence: Wide Range Achievement Test-Revised (WRAT-R) [18].

Statistical Analyses

The proportion of words spoken by participants (out of total word counts in each trial session) was assessed. Student’s t-test and Wilcoxon rank sum test were used to examine the differences in baseline characteristics and the relative proportion of words spoken between MCI and cognitively intact participants. Chi-square and Fisher’s exact tests were used for categorical variables. We first assessed the spearman’s correlation coefficients between the proportion of words spoken by participants and neuropsychological tests and demographic variables. Next, the relationship between cognitive status (MCI vs. intact) and the proportion of words spoken by participants was analyzed using a multivariate generalized estimating equation (GEE) model in order to account for multiple observations per subject. In the GEE model, participant age, gender, years of education, interviewer (creating 2 dummy variables distinguishing 3 interviewers) and assessment time point (creating 2 dummy variables indicating 3^rd and 6^th week of assessment, respectively) were included in the model. The assessment time point was included to examine if the interview order (baseline, week 3, week 6) showed any systematic differences. Second, we generated a logistic regression model using cognitive status as the outcome and proportion of word counts averaged over the three sessions as the predictor. We examined the Receiver Operating Characteristics Area under the Curve (ROC AUC) of distinguishing MCI subjects from those with intact cognition byproportion of word counts spoken by participants. Goodness of fits in GEE models and logistic regression models were examined through visual inspection of residuals and Hosmer-Lemeshow test, respectively. SAS 9.4 (Cary, NC, USA) was used for the analyses.

RESULTS

Baseline characteristics of the 41 participants are listed in Table 2. Mean Mini-Mental State Examination (MMSE) [19] score at baseline was 26.9 (SD 2.1) among those with MCI; and 28.7 (SD 1.3) among those with normal cognition (p < .01). Category fluency test (semantic fluency, p=0.08) and letter fluency test (phonetic fluency, p=0.31) and stroop test scores (executive function/inhibition, p=0.07) were not significantly different between those with intact cognition and MCI, but other tests were. As for correlations, higher average proportion of words spoken by participants was correlated with older age (p=0.04) and fewer years of education (p=0.04). Among neuropsychological tests, MMSE (global cognition, p=0.02), word-list acquisition (learning, p=0.02) and stroop scores (executive function/inhibition, p=0.02) were negatively correlated with proportion of words spoken by participants. The average proportion of words spoken by participants across the 3 assessment time points was different between the groups (p=0.01) with 68.3% (SD 9.3) among those with MCI and 60.0% (SD 9.3) among cognitively intact.

Table 2.

Baseline characteristics and proportion of words spoken among MCI and cognitively intact participants.

Variable	Intact [N=27]	MCI [N=14]	p-value	Correlation# with Average % of Words Spoken by Participant Across 3 Sessions [N=41]	p-value
Age (years)	78.9 (5.5)	83.4 (8.8)	0.10	0.31	0.04*
Gender (% Women)	63%	86%	0.17	Men: 65.77 (11.30) Women: 61.63 (9.40)	0.23
Years of Education	16.6 (2.4)	14.0 (2.6)	0.003**	-0.31	0.04*
Mini-Mental State Exam	28.7 (1.3)	26.9 (2.1)	0.008**	-0.36	0.02*
Wide Range Achievement Test-Revised (WRAT-R)	76.9 (10.5)	65.9 (11.3)	0.004**	-0.28	0.08
Category Fluency (Animals)	20.0 (5.6)	16.8 (4.7)	0.08	-0.02	0.92
Letter Fluency Total (start with F/A/S)	38.5 (12.6)	33.9 (15.3)	0.31	-0.15	0.34
Word-List Acquisition	19.9 (3.6)	16.1 (5.9)	0.04*	-0.35	0.02*
Word-List Delayed Recall	5.1 (2.3)	3.6 (2.2)	0.05*	-0.11	0.48
Trail Making Test A (time in seconds)	39.7 (14.4)	54.4 (20.9)	0.03*	0.25	0.12
Trail Making Test B (time in seconds)	104.8 (41.7)	145.4 (61.2)	0.02*	0.25	0.12
Stroop Test	32.7 (9.5)	26.8 (9.6)	0.07	-0.36	0.02*
Average % of Words Spoken by Participant across 3 Sessions	60.0 (9.3)	68.3 (9.3)	0.01*	N/A

*: p<0.05, **: p<0.01. #: Spearman’s rank order correlation. For categorical variables (gender), proportion of the word counts (SD) and its difference by t-test was provided.

GEE analysis showed that those with MCI have about 6% higher proportion of word counts in a conversational session compared to those with intact cognition (Table 3). Older age (p=0.03) was also associated with higher proportion of word counts. There was no significant difference among assessment time points, although MCI subjects had a tendency of showing lower proportion of word counts at the 3^rd and 6^th week compared with baseline (i.e., more resembled to those observed among the normal, not in Table). Subjects interviewed by interviewer 2 talked significantly more (p<0.001), compared to those interviewed by interviewer 1, suggesting that Interviewer 2 could be more effective in getting the participants to talk, although we trained each interviewer extensively to standardize interviewer skills [5].

Table 3.

Generalized estimation equation (GEE) results with outcome being proportion of word counts spoken by subjects out of total word counts spoken by subjects and interviewers (n=41, total number of observation used=123)

Covariates	Coefficient	SE	p-Value
MCI (vs. cognitively intact)	5.86	2.64	0.03*
Age (years)	0.32	0.15	0.03*
Female (vs. Male)	-4.69	2.98	0.12
Education (in years)	-0.46	0.34	0.18
Interviewer 2 (vs. 1)	10.73	3.08	<0.001**
Interviewer 3 (vs. 1)	4.80	2.90	0.09
3rd week assessment (vs. baseline)	0.78	1.06	0.45
6th week assessment (vs. baseline)	-0.75	0.62	0.23

*p<0.05; **p<0.01.

The logistic regression models showed the ROC AUC of identifying MCI (vs. normals) was 0.71 (95% Confidence Interval: 0.54 – 0.89, Odds Ratio for the proportion of word counts = 1.12, p=0.02) when average proportion of word counts spoken by subjects was included univariately into the model. Adding age and education in the model further improved the ROC AUC to 0.84 (95% Confidence Interval: 0.69 – 0.99).

DISCUSSION

Holding conversations involves a synthesis of social and cognitive functions; a person is required to actively construct a representation of what another person is thinking or feeling, what that person believes, what they desire, and what their perspective is on topics and ideas [20, 21]. All of this “social cognition” requires remembering what was said a few seconds or minutes before, planning and organizing one’s next thoughts, keeping track of the conversation, iterating and updating the interaction, as well asself-monitoring in order to take turns, not interrupt, understand another’s feelings and inhibit inappropriate behaviors. Thus, at various times, attention, executive function, inhibition, abstract reasoning, memory and language abilities are simultaneously engaged. We compared proportion of word counts generated by participants out of total word counts, rather than comparing the word counts per se because our interest was in how two people (participants and interviewers) interact with each other in one-on-one conversations and in order to control for total duration of conversational sessions. We found that MCI participants spoke a greater proportion of words during the timed remote video telecommunication conversational sessions than those with normal cognitive function. Past research has examined the complexity, content and acoustic features of spoken language as an indicator of aging or cognitive status by using answers to neuropsychological tests or to specific tasks (e.g., ask subjects to reada paragraph, speech provided in press conferences) [2, 22, 23] To our knowledge, this is the first study attempting to examine interactions between two peoplein a one-on-one conversational setting using the proportion of word counts of MCI in comparison with normal participants. Using the Alzheimer's Disease Neuroimaging Initiative (ADNI) data, discriminatory ability (indicated by the area under the cure (AUC) of the receiver operator curve (ROC)) of biomarkers together with age and gender information in distinguishing MCI from the normal has been reported to range from 0.68 (hippocampal volume, age and gender in the model) to 0.77 (CSF Aβ42, CSF tau, sex and age in the model) [24].Others have found somewhat higher AUC values ranging from 0.70 to 0.96 for fluid biomarkers which may relate to differing methods or characteristics of the samples studied [25-27]. The ROC AUCs in our study using word counts (0.71 with the proportion of word counts alone and 0.84 with the word counts, age and education information) were comparable to many of the currently used biomarkers in the field. Thus we highlight that using the quantitative behavioral biomarker of word counts is particularly promising as an approach to identify populations at risk for decline or to track response to therapies, especially considering that the data collection can be done at home, and does not require invasive or expensive biomarker assessments.

Neuropsychological criteria used to define MCI [28] includes objective evidence of impairment on at least two tests within four or more cognitive domains, with scores falling at least 1 standard deviation (SD) or more below age-stratified normative data. The language domain is commonly measured by tests such as the Boston Naming Test [29], letter fluency(phonemic fluency) and category fluency tests (semantic fluency) [16]. In Alzheimer's disease (AD), category fluency has generally been found to be disproportionately impaired, whereas letter fluency ability is less impaired [30, 31], although not all studies agree with this finding [32]. It is hypothesized that the disproportionate impairment in semantic fluency, as opposed to phonemic fluency, could occur because the former relies more on temporal-lobe semantic stores, the area which is affected by AD. Our finding that participants with MCI (the prodromal stage of AD) spoke a higher proportion of word counts in free conversations may be reflecting their subtle decline in semantic fluency abilities. Patients with dementia are known to have high incidence of circumlocutions (i.e., the use of many words where fewer would do) and semantic jargon in their spontaneous speech [33]. We found a somewhat larger difference between MCI and normal groups on a semantic fluency test (p=0.08) than on a phonemic fluency test (p=0.31), although neither difference was statistically significant, likely due to small sample size. Possibly MCI participants may tend to struggle to find the right words and therefore may be more likely to need to substitute words in the conversation to convey their thoughts, especially in the early stage of MCI when phonemic fluency is still preserved, leading to increased proportion of word counts in timed conversations. It is noteworthy that in our study the differences insemantic and phonemic test scores between the MCI participants and those with intact cognition were not significant, while the proportion of words spoken by participants showed a significant difference between the two groups. The latter measure may be more sensitive to cognitive decline than traditional neuropsychological tests and may identify early stage MCI before impairment becomes symptomatic through neuropsychological tests.

There are other possible underlying mechanisms that may explain the disproportionate proportion of words contributed to a conversation by MCI participants. Increased word counts among MCI individuals may be due to subtle difficulties with the executive and self-monitoring aspects of conversation. Also MCI participants could have reduced passage-of-time estimation abilities relative to those with normal cognitive function. Thus those with MCI would not be able to anticipate and prepare for the end of the sessions. However, there is some evidence that time estimation is not affected in MCI, although it is associated with aging [34]. There is some evidence that MCI participants may acquire deficits in social cognition such as the ability to make inferences about the cognitive state, emotions and intentions of other people [35, 36]. Misreading of social cues and intent during social exchanges could result in more discursive conversation among MCI participants.

Epidemiological evidence suggests that tracking changes in linguistic features may be a promising tool in identifying early signs of AD. The Nun Study found a link between linguistic density (e.g., complexity, vivacity, fluency) in early life (most were in their 20s) and the risk of developing Alzheimer’s disease later on, by examining autobiographical essays written by the nuns upon joining the Sisterhood [37]. Other studies also suggest that subtle language impairment occurs long before an MCI diagnosis or at a very early stage of MCI. For example, Oulhaji et al., [38] showed that using the CAMCOG (the Cambridge Cognitive Examination), verbal expression and learning scores at baseline, conducted about 20 years beforehand, were the most significant predictors of incident MCI. Amieva et al., [39] showed that low category fluency test scores can be seen nine years before the clinical diagnosis of Alzheimer's disease. Recent MRI studies have found associations between the size and complexity of real-world social networks and the density of grey matter [3] and amygdala volume [40], providing some support for a link between biomarkers (such as brain structure) and social network size, an indirect measure of how well subjects communicate with others. Research is increasingly focused on identifying AD at the MCI stage, or even earlier, in order to prompt interventions and to access clinical trials with disease-modifying drugs [41]. Clearly, higher-order functional activities have been shown to distinguish those at risk of developing MCI from those remaining cognitively normal [42-44]. Studies utilizing continuous, unobtrusively monitored in-home activity data in a community setting also showed that these ecologically valid activity measures could identify early changes among those who convert to MCI [13]. We have been following seniors at their home over 4 years recording their in-home activities unobtrusively, including decline in computer use at home [8], changes in in-home walking speed and variability [9], and time out of house as an indicator of the amount of social interactions. Current information analysis science can provide objective assessments of the nature of social interactions through the examination of speech characteristics, including sentence complexity, verbal fluency, vocabulary, and acoustic features including affect which was not possible a decade ago [1, 2]. It will take some time to realize the assessment of speech characteristics including developing automatized algorithms to protect privacy, developing a reliable way of recording conversations, storing or transmitting the recorded data for analysis. However, the assessment protocol could be implemented as one of the in-home monitored activities to enhance the prediction of those at-risk of cognitive impairment in communities in the future. The results presented in this report, together with the past observational studies mentioned above, suggest that examining the way older adults interact with each other in conversation could facilitate the development of powerful tools for identifying early signs of MCI in community dwelling older adults. Further studies are required to validate our findings and to create practical applications of assessment protocols.

There were some limitations to this study. The sample size was relatively small. Participants were volunteers interested in participating in a behavioral intervention trial and this might limit the generalize ability of the results. The conversation session format used in this study was a remote telecommunications video chat between participants at their home and trained interviewers at the study site. Video chat conversational format differs in some ways from face to face conversation. However, we paid special attention to creating a user-friendly environment, including touch-screen monitors, which eliminated mouse use when receiving calls, a large monitor that allowed eye-to-eye contact during in-person conversations in order to retain attention during the session, and pop-up pictures on the screen to evoke conversations without any effort by the participants. Additionally, previous studies have shown that remote telehealth communication formats can be effective for a variety of assessments and interventions with older adults [45].

Conclusion

An ecologically valid social marker such as the proportion of spoken words produced during free conversation may be sensitive to transitions from normal cognition to MCI and may be able to detect the transitions before any changes in conventional cognitive tests. It could complement biomarker studies in selecting or identifying at-risk participants during their pre-symptomatic phase. Examinations of interactions among subjects in free conversational speech could provide an additional area of research useful for the early identification of MCI.

CONFLICT OF INTEREST

The authors confirm that this article content has no 
conflict of interest.