Detection of Dementia Signals from Longitudinal Clinical Visits Using One-Class Classification

Abstract

Dementia is one of the major health challenges in aging populations, with 50 million people diagnosed worldwide. However, dementia is often underdiagnosed or delayed resulting in missed opportunities for appropriate care plans. Identifying early signs of dementia is essential for better life quality of aging populations. Monitoring early signs of individual health changes could help clinicians diagnose dementia in its early stages with more effective treatment plans. However, rare data for dementia cases compared to the normal (i.e., imbalance class distribution) make it challenging to develop robust supervised learning models. In order to alleviate this issue, we investigated one-class classification (OCC) techniques, which use only majority class (i.e., normal cases) in model development to detect dementia signals from older adult clinical visits. The OCC models identify abnormality of older adults’ longitudinal health conditions to predict incident dementia. The predictive performance of the OCC was compared with a recent streaming clustering-based technique and demonstrated higher predictive power. Our analysis showed that OCC has a promising potential to increase power in predicting dementia.

I. Introduction

Dementia is one of the major health challenges in the aging population, with around 50 million people living with dementia worldwide [1]. Alzheimer’s disease is the most common cause of dementia which affects more than 6 million people in the United States currently, and it is expected that number will be doubled by 2060 [2]. These increasing numbers have led to an enormous burden for the health system, patients, and their families and resulted in an annual estimate of 18.5 billion hours of unpaid care at a value of $234 billion [3]. Despite the tremendous effect of dementia on the older population, more than 40% of patients with cognitive impairment (i.e., mild cognitive impairment [MCI] and dementia) are underdiagnosed [4]. Delays in diagnosing dementia in its early stages lead to adverse clinical outcomes. Furthermore, clinical diagnoses usually occur after the disease has progressed, and the chances for maximizing the quality of life are no longer achievable. [5–7]. Therefore, early detection of dementia is an essential step to help clinicians develop more effective treatment plans and lead to better treatment outcomes [8].

One crucial aspect of electronic health record (EHR) data is that it contains detailed records of longitudinal disease progression. Since dementia is a slowly progressing disease, monitoring the changes in patients’ clinical assessment during follow-up visits may reveal essential information to predict dementia. However, previous studies that used only the diagnosis codes as the model input resulted in models that may suffer from generalization issues in a real-world EHR [9–11]. Trajectory analysis has a potential to detect signals of dementia more effectively than the clinical diagnosis [12, 13] and a static model [14]. Our preliminary analysis has revealed distinct temporal patterns of activities of daily living between cognitively impaired and cognitively unimpaired older adults several years prior to the clinical diagnosis [15, 16].

Recent developments in machine learning can enhance the analysis of EHR data [17–21]. However, systematic analysis of patient data related to dementia in routinely collected EHRs, and how their temporal patterns are associated with the development of dementia, are not well known. Although dementia prediction and progression are challenging, we believe outlier detection models such as one-class classification (OCC) could be effective in capturing abnormal changes in patient data to predict early stages of dementia. This study systematically investigated the application of EHR data analysis and OCC techniques for outlier detection and predicting incidents of dementia.

Existing machine learning algorithms focused on dementia prediction are usually a 2-class problem. One drawback of these methods in the context of dementia detection within EHR environments is the imbalanced and heterogeneous data issues. There are very few dementia cases compared to the normal ones, which makes it an obstacle towards developing a robust classifier that can be generalized to detect dementia cases. OCC models are developed to solve issues of limited data availability of one class where the training is done using the dominant class, and samples from the minority class are detected as outliers or novelty events. In [22], OCC was employed to detect fall incidents in elderly monitoring data. OCC was also used to monitor sensor data of dementia patients in real-time to look for activities that were not performed by the patients in the past [23]. On the other hand, OCC has not been studied in the context of dementia prediction with EHR data. Therefore, in this paper, we study multiple OCC models to detect early signs of dementia using longitudinal health data. In [24], a streaming clustering temporal model was developed to detect early signs of dementia and MCI. The model detected the change of state to dementia by monitoring the outlier data samples in the streaming data. Therefore, in this paper, we study the ability of OCC to detect changes in the health visits (outliers) and compare the performance to [24].

II. Materials and Methods

This section first describes the dataset used in our model and the patient representation from the health visits. Then we explain the OCC algorithms used in the analysis. The experimental setup is depicted in the last section. This study includes two main components: OCC model development to detect incident dementia and the effect of feature selection on the models’ performance. We considered outlier detection of a patient health visit as a sign of change in the cognitive status, dementia.

A. Data

This study was approved by the Mayo Clinic Institutional Review Board and the Olmsted Medical Center Institutional Review Boards. The data from the Mayo Clinic Study of Aging (MCSA) [25] were used in the study. The MCSA is a prospective population-based cohort study of cognitive aging with comprehensive periodic cognitive assessments repeated every 15 months since 2004 for ages 70–89 and expanded in 2012 down to age 50. Eligible persons from the Olmsted County, Minnesota population, were randomly selected in an age- and sex-stratified manner and evaluated comprehensively in person by three independent evaluators. A study coordinator assessed sociodemographic characteristics, and asked question on memory, neuropsychiatric symptoms and activities of daily living. A physician reviewed the medical history, administered the Short Test of Mental status and did a neurologic examination and a psychometrist administered nine test to assess four cognitive domains: (1) memory (Auditory Verbal Learning Test (AVLT) delayed response [26], Wechsler Memory Scale-Revised (WMS-R) [27], Logical Memory II and Visual Reproduction II); (2) language (Boston Naming Test [28], Category Fluency)[29]; (3) attention/executive (Trail-Making Test B [29, 30], Wechsler Adult Intelligence Scale-Revised (WAIS-R), Digit Symbol); and (4) visuospatial (WAIS-R Picture Completion and Block Design [31]). A consensus committee that consisted of the study coordinator, the physician and a neuropsychologist, reviewed the data for each participant and used previously published criteria to diagnose the participants with MCI [32] or dementia [33]. In addition to cognitive assessment, other data elements are abstracted from the medical records (e.g., comorbid conditions, vascular risk factors). The MCSA cohort comprises 6,185 unique patients with 26,807 visits (4.3 average visits per patient). Among these, 3,070 patients were female (49.6%), and 729 patients (11.6%) finally progressed to dementia.

Table 1 summarizes the variables collected from the MCSA and used to predict the incidents of dementia. It summarizes the 58 input variables used in the model into six different categories, including patient demographics, physical characteristics, neuropsychiatric characteristics, social characteristics, functional status, and neuropsychological characteristics; all characteristics were considered time-dependent variables.

Table 1.

Input Variables

Patient Demographics

Age, Sex, Race, Ethnicity

Physical characteristics

BMI, Smoking status, Alcohol problem

Sleep apnea, Hypertension, Dyslipidemia, Atrial fibrillation, Angina, Congestive heart failure, Coronary artery disease, Myocardial infarction, Coronary artery bypass graft, Diabetes, ESS score

Neuropsychiatric characteristics

Delusions, Hallucinations, Agitation, Depression, Anxiety, Euphoria, Apathy, Disinhibition, Irritability/lability, Motor behavior, Nighttime behavior, Appetite/eating change, BDI-II grand total, BDI depression (Total >=13), BAI total (0–63)

Social characteristics

Education, Occupation, Marital status, Personal care (as extracted from the clinical dementia rating scale)

Functional status

FAQ Total Score (0–30), ECog-12

Neuropsychological test scores

For four cognitive domains: (1) memory (AVLT delayed response, WMS-R Logical Memory II and Visual Reproduction II); (2) language (Boston Naming Test, Category Fluency); (3) attention/executive (Trail-Making Test B, WAIS-R Digit Symbol) and (4) visuospatial (WAIS-R Picture Completion and Block Design).

Abbreviations: ESSscore: Epworth Sleepiness Scale score (0–24), BDI: Beck Depression Inventory scores, BAI: Beck, Anxiety Inventory, Functional Activities Questionnaire (FAQ): 10-item questionnaire on instrumental activity of daily living, Everyday Cognition (ECog-12): scales to measure multiple cognitively relevant everyday abilities, covering six domains. BMI: Body Mass Index.

AVLT: Auditory Verbal Learning Test, WMS-R : Wechsler Memory Scale-Revised, WAIS-R: Wechsler Adult Intelligence Scale-Revised.

B. Patient Representation

The patients’ visits were represented in a temporal mode and used them in our OCC model. To account for the temporal information, we extracted the features in Table 1 to represent each visit for each patient. Body mass index (BMI) is also split into 4 groups (Underweight: BMI < 18.5 kg/m2, normal: 18.5 ⩽ BMI < 25 kg/m2, overweight: 25 ⩽ BMI < 30 kg/m2, and obese: BMI ⩾ 30 kg/m2). Age is divided into four categories (Age ⩽ 64, 65 ⩽ Age ⩽ 74, 75 ⩽ Age ⩽ 84, Age ⩾ 85). The Epworth Sleepiness Scale (ESS) (ESS < 10, ESS ⩾ 10) and the years of education (Education ⩽ 12, Education > 12) are also divided into two categories. For normal participants, we used all the visits of each one. However, for patients with dementia, we only used all the visits before the dementia diagnosis visit. For example, if a patient has 8 visits in our cohort, and the dementia diagnosis was at visit 6. In that case, we use data from the first 5 visits in our model to predict dementia. Each participant is categorized as dementia or without dementia. The resulting dataset and labels are used in all the experiments in this paper.

C. One-Class Classification Models

OCC is a commonly used technique for outlier detection and novelty detection. We believe that detecting early signs of dementia falls under the umbrella of novelty detection. This is mainly because, in the beginning, the patient’s health is considered normal until the patient falls under the dementia group due to a change in the cognitive status. Therefore, OCC could be used to detect the new patterns (novelty) and generate an early sign of dementia alert. OCC learns to observe examples from one class compared to multiple-class classifiers. The main idea is to train the classifier to distinguish the target class (non-dementia) from the other class (dementia) that was not included during the training [34]. This is essential for many applications such as intrusion, disease prediction, and fault detection.

The first model is a single Gaussian target distribution (SGTD) OCC [34]. The normal class (non-dementia visits) is modeled as a Gaussian distribution in this technique. The Mahalanobis distance is used as the metric to avoid numerical instabilities, as in equation (1).

$$f\left(x\right)={\left(x-\mu \right)}^T{\sum }^{-1}\left(x-\mu \right)$$ (1)

The classifier is defined as:

$$h\left(x\right)=\{\begin{array}{lll}\mathit{\text{target}}\hfill & if\hfill & f\left(x\right)\le \theta \hfill \\ \mathit{\text{outlier}}\hfill & if\hfill & f\left(x\right)>\theta \hfill \end{array}$$ (2)

where μ and ∑ are the mean and covariance matrix for the gaussian distribution, and the threshold θ is based on the target error and set by the user.

The second model is a mixture of Gaussian target distribution (MGTD) OCC [34]. This method is similar to the previous one, except it represents the normal class using K Gaussian distributions for more robust performance. The model is described in equation (3).

$$f\left(x\right)=\sum _{i=1}^K{p}_i\left(-{\left(x-\mu \right)}^T{\sum }^{-1}\left(x-\mu \right)\right)$$ (3)

The classifier is defined as in equation (2). The parameters P_i, μ_i, and Σ_i are optimized using the Expectation Maximization (EM) algorithm. If a large value of K is selected, the number of free parameters can grow significantly, especially for the covariance matrices. Therefore, the covariance matrices can be constrained. If the model is trained with samples only from the target class, then the K Gaussians are used to represent that class only. However, if the training dataset contains samples from the outlier class, a mixture of Gaussians is built for both classes (K_traget and K_outliers). If objects from both classes are included in the training data, one extra outlier cluster is introduced with a wide covariance matrix to avoid closed decision boundary around the target class.

The third model is autoencoder target distribution (AETD) OCC [34]. In this technique, an autoencoder was trained to reconstruct the non-dementia class. The difference between the input and output pattern (equation (5)) is used as a characterization of the target class.

$$f\left(x\right)={\left(x-\mathit{\text{Reconstructed}}\left(x\right)\right)}^2$$ (5)

The classifier then becomes as in equation (2).

The last OCC model is the k-means target distribution (kmeansTD) OCC [34]. The standard k-means algorithm is used to cluster the data into K clusters such that the average distance to a cluster center c_i is minimized. The target class is represented using equation (6) and the classifier becomes similar to the one in equation (2).

$$f\left(x\right)=\underset{i}{\text{min}}{\left(x-c_i\right)}^2$$ (6)

D. Experimental Methods

To evaluate the effectiveness of the proposed method, we first compared the performance of the OCC models described in section (II-C) using 58 features from Table 1. OCC was developed with 70% of the total visits from the participants without dementia in the cohort. The remaining visits from those without dementia and the visits from the dementia patients are used in the testing phase. The visits are labeled as either from a participant who progressed to dementia or one who did not. The model is trained with visits from participants without dementia only. When testing the model, the visits of each participant are fed to the model sequentially according to the visit date. If one of the participant’s visits is flagged as an outlier (novelty detection), we consider it a change in the participant’s health status and treat it as a sign of dementia. We assessed the ability of the model to detect early signs of dementia using precision, recall, and F1-score. F1-score is an effective metric for imbalanced data focusing on the minority class (dementia) by combining precision and recall.

We also examined the importance of the features on the model performance. We applied the out-of-bag (OOB) predictor importance ranking technique [35, 36] to investigate the effect of the selected features on the model performance. OOB measures the influence of a specific feature on the model at predicting the response. If a predictor is important in the prediction, then permuting its values should affect the model error. On the other hand, if a predictor is not significant, the permutation of its values should have little to no effect on the model error. The OOB predictor importance process is described in Figure 1. We selected the number of normal patients equal to the dementia ones to avoid data imbalance issues. We then used the same features in the previous section and fed the data to the OOB model.

Figure 1.

Out-of-bag permutated predictor importance algorithm

Then, the model performance was compared to a streaming clustering-based approach (called MUSC) recently developed on the same dataset [24]. Missing values were imputed using the mean of each feature.

III. Experimental Results

A. Results using All Features

Table 2 shows the results using all 58 features. We notice that all OCC models outperformed the streaming clustering approach (MUSC). This might be because MUSC is unsupervised approach. However, OCC models are trained to detect the visits of participants without dementia and learn how to distinguish them from the dementia visits (outliers).

Table 2.

OCC performance on the MCSA data using all 58 features

Model	Precision	Recall	F1-Score
MUSC	0.6200	0.6800	0.6486
SGTD	0.8738	0.6129	0.7204
MGTD	0.7440	0.8530	0.7950
AETD	0.7940	0.9230	0.8535
kmeansTD	0.8484	0.7219	0.7800
Model	Precision	Recall	F1-Score

The deep learning-based OCC (AETD) performed the best with the highest F1-score. The second best one was the MGTD, possibly because it represents the normal patients with multiple Gaussians. MGTD outperformed the SGTD in F1-score possibly due to building a more robust model to represent the participants without dementia, leading to better detection of outliers (dementia patients). The kmeansTD performed similar to MGTD. We believe this is because it represents the normal patients with multiple clusters and better detects dementia visits from others.

B. Results using Selected Features

We performed feature selection to examine the importance of individual features as well as to decrease the model complexity and computation (Figure 2). We compared the performance of the OCC models with all features versus top 20-features. The top 20 highly ranked features are as follow: 65% from neuropsychological test scores, 10% from social characteristics, 10% from neuropsychiatric characteristics, 10% social characteristics, and 5% from physical characteristics. The majority of the highly ranked characteristics are mainly from the neuropsychological features, which is expected as they measure cognitive performance and contribute to the evidence of cognitive impairment for the dementia criteria. We observed a decrease in the performance when the number of features is reduced. However, the decrease is not significant if we account for the model complexity reduction. AETD performed the best in both cases, as in Tables 2 and 3, considering its capability to build a better boundary to distinguish dementia visits from the normal ones.

Figure 2.

OOB feature importance

Table 3.

Results on the 20 top-ranked features

Model	Precision	Recall	F1-Score
SGTD	0.8826	0.5562	0.6824
MGTD	0.7100	0.8040	0.7541
AETD	0.7624	0.8898	0.8212
KmeansTD	0.8429	0.6454	0.7310

IV. Discussion

Dementia is a critical health issue in the aging population and results in life-changing outcomes. However, there is a lack of automated methods to detect dementia in its early stages, especially using longitudinal patient health data with limited dementia cases. In this paper, we have explored an OCC technique to predict dementia using only normal participants’ longitudinal health data in model development.

Our analyses demonstrated the capability of OCC models to capture abnormal signals (i.e., outliers) in patients’ longitudinal health visits, predicting dementia cases. The OCC models deal with the issue of limited data samples for dementia cases (i.e., imbalanced class distribution of predominant normal vs. minor dementia cases) by training the classifier with examples from normal cases only. Therefore, our models could be well applicable to a real scenario in health care settings to help the clinical staff in dementia prediction.

We also investigated feature importance and its effect on the model performance examining the potential for a less complex model and lower computational cost. Neuropsychological features were mostly ranked at the top as they measure cognitive performance reflecting cognitive impairment in dementia.

Our models used variables in the MCSA, which may not be fully available in other institutions. However, the OCC models explored in this study could be fully utilized to facilitate dementia prediction. The OCC models can also be extended using variables from routine EHR data for board applicability. Since these dementia OCC models were developed by a single institution, they warrant the external validity to assess its generalizability.

V. Conclusion

The OCC models using longitudinal visit data demonstrated a promising potential to detect abnormal signs before incident dementia. The predictive performance of the OCC models was higher than a streaming clustering approach. Unlike most existing dementia prediction techniques, our OCC model is based on using visits from normal patients during the training process, alleviating the issue of limited data samples for dementia patients. Our approach could facilitate early alert for dementia, which addresses a significant current delay in dementia diagnosis.