Implementation of the telephone montreal cognitive assessment in a telemedicine based pre-admission testing clinic during COVID-19

Abstract

Introduction

Postoperative delirium (POD) affects 10-70% of patients 60 years or older and has been linked to increasing length of hospitalization, mortality, and morbidity. Pre-existing cognitive impairment is a predictor of POD. COVID-19 restricted use of in-person cognitive screens. The Telephone Montreal Cognitive Assessment (T-MoCA) can screen for cognitive dysfunction remotely. We evaluated the feasibility of administering T-MoCA in a multiethnic population during pre-operative testing televisits.

Methods

Patients scheduled for surgery between July 2020 and August 2020 were asked to participate in the T-MoCA at the end of their preadmission testing (PAT) televisit. A retrospective chart review was conducted to collect patient comorbidities and demographics. Patients were stratified by negative (T-MoCA≥19) or positive (T-MoCA<19) for mild cognitive impairment (MCI) and compared using 2-tailed χ²-tests. Univariate logistic regression was used to identify associations between patient characteristics and positive T-MoCA result.

Results

Fifty out of 65 (77%) patients who consented to the T-MoCA completed the test. The average time to complete the assessment was 10.5 mins. Twenty two (44%) had a negative score and 28 (56%) had a positive score. Patients who had a positive T-MoCA were older (70.04±7.61 yrs) compared to those with a negative T-MoCA (67.68±4.69 yrs, p=0.007), although the distribution of patients above and below age 65 was not different (p=0.243). The two groups did not vary by gender, race/ethnicity, obesity, surgery type, or medical co-morbidities. When we examined our population for predictors of a positive T-MoCA, we found a trend toward men being less likely to score positive on T-MoCA (OR=0.33, 95% CI: 0.10-1.10, p=0.07) compared to women; and that patients with Hispanic race/ethnicity were more likely to test positive on the T-MoCA (OR=4.13, 95% CI: 0.84-20.28, p=0.08) compared to Non-Hispanic Whites.

Conclusions

Implementation of the T-MoCA in a telemedicine-based PAT setting is feasible. In our cohort, most people who consented to the assessment completed it, and more than half scored positively, which may have important implications on the surgical plan and post-operative recovery. There may be limitations in using T-MoCA in certain populations, such as non-English preferred language, hearing difficulties, lack of focus, and use of external aids, which would need to be explored in a larger sample size.

Introduction

Delirium is an acute change in cognitive function and attention that is characterized by confusion, reduced memory, and altered mental status. Postoperative delirium (POD) is delirium that occurs after surgery and is one of the most common post-operative complications. Symptoms can appear within hours or weeks after surgery. It can affect 10-70% of patients 60 years or older, and has been linked to adverse healthcare outcomes such as increased length of hospitalization, cost of care, mortality, and morbidity.^{1 , 2} POD represents a major obstacle to the delivery of surgical care to at-risk populations, such as the elderly and those with pre-existing cognitive impairment, since POD complicates post-operative management and has been shown to irreversibly worsen pre-existing cognitive issues³.

Studies have suggested that the best strategies to prevent POD may lie in the preoperative setting⁴. Pre-existing cognitive impairment, preoperative depressive symptoms, and a history of prior stroke/transient ischemic attack, have all been demonstrated to be risk factors for the development of POD⁵. Therefore, evaluating mental status prior to surgery is a method to risk-stratify the likelihood of POD. There are a number of validated cognitive function assessment tools, such as the Mini-Mental State Examination (MMSE).⁶ However, the COVID-19 pandemic forced many hospitals to switch to telemedicine to decrease viral transmission.⁷ In order to determine which patients are at risk of POD, it has become necessary to use a telemedicine-based cognitive assessment.

One such test is the Telephone Montreal Cognitive Assessment (T-MoCA), a rapid screening instrument for MCI that is administered over the telephone⁸. The T-MoCA assesses a number of different cognitive domains, including attention and concentration, memory, language, conceptual thinking, calculations, and orientation. T-MoCA is an adapted version of the original MoCA and contains the same items as the original MoCA, except those requiring visual abilities have been removed. While the MoCA and it's scoring has been validated by a number of different studies⁹, the cutoff score for determining MCI for the T-MoCA has not been widely validated at this point. Telephone Interview for Cognitive Status (TICS) is the most widely validated telephone-based screening tool for MCI.¹⁰ However, studies have shown that the TICS has difficulties differentiating MCI from normal cognition.¹¹ In this aspect, the T-MoCA is hypothesized to perform better, as the MoCA has been shown to outperform the MMSE for detecting MCI.¹¹ The objectives of this study were to determine the feasibility of administering the test over the phone in a multiethnic population and determine potential variables affecting T-MoCA score.

Methods

Study Population and Study Design

We conducted an institutional review board (IRB)-approved cross-sectional study of patients presenting for pre-admission testing (PAT) prior to surgery at an academic health center serving a racially diverse, urban population. This study was conducted using STROBE guidelines. Due to the COVID-19 pandemic, patients undergoing surgery between July 2020 to August 2020, were scheduled for a phone-based telemedicine PAT appointment at least one week prior to their surgery. Following the PAT appointment, patients were asked by the anesthesiologist if they wished to participate in this study. Patients who verbally consented to participate were contacted by a single investigator. Written consent was not obtained as the investigators only communicated with patients over the phone. The investigator then asked for the patients’ ethnicity, primary language, gender and education level before administering the telephone-based Montreal Cognitive Assessment Test (T-MoCA) 22 v 8.1. Professional hospital-based phone interpreters were used to conduct the T-MoCA in non-English speaking patients (Language Line Solutions, Monterey, CA).

Medical record review of those who consented to participate was performed to collect patient demographics (e.g. age, body mass index (BMI)), intended surgery, patient comorbidities (e.g. presence of neurocognitive diagnosis, defined as any disorders of the central nervous system, such as cerebral aneurysm, multiple sclerosis, or intracranial mass; depression, anxiety, hypertension, diabetes mellitus type 1 or 2, coronary artery disease, stroke) and lifestyle characteristics (current smoking status and alcohol use). Data was stored using REDCap electronic data capture tools hosted at the Albert Einstein College of Medicine.¹²

Cognitive Assessment

The T-MoCA is a validated telephone-based screening tool for the detection of mild cognitive impairment.^{11 , 13} It assesses attention, concentration, memory, language, conceptual thinking, calculations, and orientation. We used the T-MoCA 22 v8.1 to conduct the cognitive assessment. The original MoCA had 30 test elements; in the T-MoCA 22, tasks that require visual elements were removed. The total score a patient can achieve on the T-MoCA is 22 points, with a score of 18 or below considered a positive screen for mild cognitive dysfunction. Scoring details are provided in Table S1. A single investigator performed the T-MoCA assessment; this investigator was supervised by an experienced investigator in T-MoCA but did not receive MoCA certification from the manufacturer of MoCA (mocatest.org).

Power Analysis

An analysis of the in-person MoCA done in a pragmatic, validation study of 150 patients found a standardized effect size (Cohen's d) of 1.4 for mild cognitive impairment vs no dementia.^{14 , 15} Given that the T-MoCA likely has a lower effect size compared to the in-person MoCA, but since no effect sizes have been estimated for the T-MoCA in the prior literature, we decided to use a more conservative estimate of 0.9 for the standardized effect size. A power analysis using the two-tailed Student's t-test, Sidak corrected for 1 comparison, with an alpha of 0.05 and a power of 0.8 was performed. From the analysis it was found that 21 samples from each group would be required. Statistical Analysis

We stratified patients by whether they had a negative T-MoCA result (score of 19 or greater) or positive T-MoCA result (score of 18 or less). We compared continuous variables (reported as mean with standard deviation (SD)) between the two groups using Student's t-test, and categorical variables (reported as number with column percentage) using the χ²-test. All statistical tests were 2-sided, with a significance threshold of p ≤ 0.05. We then estimated the odds ratio (OR) of a positive T-MoCA result using univariate logistic regression models. All analysis was conducted in SPSS v27 (IBM, Armonk, NY).

Results

During the study period, 187 patients were called and 133 answered the phone; 65 patients consented to the T-MoCA and 50 of those patients successfully completed the assessment (Figure 1 ). Among these, 28 (56%) had a positive T-MoCA result (Table 1 ). Our study population had an average age of 69 ± 6.53 years, with 72% (n=36) being 65 years or older. Thirty six percent were (n=18) female. Our population was diverse, with non-Hispanic Black (NHB) patients and Hispanic patients making up 32% (n=16) and 24% (n=12), respectively. Patients without English as their first language made up 12% (n=6) of our population. Four of these patients could speak conversational English, while two patients required a translator. Overall, the prevalence of obesity (defined as BMI ≥ 30) was 50% (n=25). The most common type of surgical procedures that our population underwent were urologic (28%, n=14), followed by gynecologic (14%, n=7), general surgery (12%, n=6), and orthopedic (12%, n=6) based.

Fig. 1.

Study population flowchart.

Table 1.

Associations of patient demographic and clinical features, among all patients and stratified by T-MoCA result (negative vs positive).

	All patients
N=50	Negative T-MoCA (≥19)
N=22	Positive T-MoCA (<19)
N=28	p*
Age, mean, SD	69, 6.53	67.68, 4.69	70.04, 7.61	0.007
	n (%)	n (%)	n (%)
Age				0.243
< 65	14 (28)	8 (36.4)	6 (21.4)
≥ 65	36 (72)	14 (63.4)	22 (78.6)
Gender				0.068
Female	32 (64)	11 (50.0)	21 (75.0)
Male	18 (36)	11 (50.0)	7 (25.0)
Race/Ethnicity				0.120
Non-Hispanic White	19 (38)	11 (50.0)	8 (38.0)
Non-Hispanic Black	16 (32)	8 (36.4)	8 (32.0)
Hispanic	12 (24)	3 (13.6)	9 (24.0)
Other	3 (6)	0 (0.0)	3 (6.0)
Primary Language				0.150
English	44 (88)	21 (95.5)	23 (82.1)
Other**	6 (12)	1 (4.5)	5 (17.9)
Surgery Type				0.314
Bariatric	1 (2)	0 (0.0)	1 (3.6)
Cardiology	3 (6)	1 (4.5)	2 (7.1)
Cardiothoracic	4 (8)	4 (18.2)	0 (0.0)
General	6 (12)	3 (13.6)	3 (10.7)
Gynecology	7 (14)	1 (4.5)	6 (21.5)
Neuro	3 (6)	1 (4.5)	2 (7.1)
Orthopedics	6 (12)	3 (13.6)	3 (10.7)
Plastic	1 (2)	1 (4.5)	0 (0.0)
Urology	14 (28)	7 (31.8)	7 (25.0)
Vascular	5 (10)	1 (4.5)	4 (14.3)
BMI				0.569
< 30	25 (50)	12 (54.5)	13 (46.4)
≥ 30	25 (50)	10 (45.5)	15 (53.6)
Comorbidities
Presence of Neurocognitive Diagnosis***	8 (16)	3 (13.6)	5 (17.9)	0.686
Depression	6 (12)	4 (18.2)	2 (7.1)	0.233
Anxiety	7 (14)	5 (22.7)	2 (7.1)	0.115
Ambulatory	13 (26)	6 (27.3)	7 (25.0)	0.856
Hypertension	38 (76)	15 (68.2)	23 (82.1)	0.251
Diabetes Mellitus	17 (34)	5 (22.6)	12 (42.9)	0.136
Coronary Artery Disease	11 (22)	6 (27.3)	5 (17.9)	0.425
Stroke	4 (8)	3 (13.6)	1 (3.6)	0.193
O2-Dependent Chronic Obstructive Pulmonary Disease	7 (14)	4 (18.2)	3 (10.7)	0.45
Heart Failure	5 (10)	3 (13.6)	2 (7.1)	0.447
Smoker	27 (54)	14 (63.6)	13 (46.4)	0.226
Alcohol Use	11 (22)	6 (27.3)	5 (17.9)	0.425

*P-value results to t-test for continuous normally distributed variables (i.e. age, BMI) and Chi-square test for categorical variables.

**Other languages include Spanish, Portuguese, and Russian

***Neurocognitive disorders include disorders that damage the central nervous system (cerebral aneurysm, multiple sclerosis, migraines, intracranial mass)

When looking at feasibility, the average time to complete the T-MoCA was 10.5 mins and ranged anywhere from 7 to 15mins. Fifty out of 65 (76.9%) of patients who started the assessment successfully completed the assessment. Potential implementation issues identified by the telephone screener included difficulty hearing (7.7%), lack of focus (6.2%), use of external aids (3.1%), and use of a translator (3.1%; Table 2 ). Overall, these problems were minor and did not affect a large portion of the participants (13.8%).

Table 2.

Implementation issues identified by examiner.

Issue	Number Affected (%)
Difficulty Hearing	5 (7.7%)
Lack of Focus*	4 (6.2%)
Use of External Aid**	2 (3.1%)
Use of a Translator	2 (3.1%)
Total Participants Affected***	9 (13.8%)

*Lack of focus were due to walking outside, driving, taking care of kids, or drinking while interviewing

**Outside help includes using another person or writing on a pen and paper to help answer questions

***Multiple issues may have affected some participants

When we stratified our data by negative vs. positive result on the T-MoCA score, we found that patients with a positive T-MoCA result were older (70.04 ± 7.61 yrs vs 67.68 ± 4.69 yrs, p<0.007). No other variables were significantly associated with a positive T-MoCA result. Further exploration using univariate analysis to evaluate the direction and magnitude of associations with a positive T-MoCA result revealed a trend among men being less likely to score a positive T-MoCA result (OR=0.33, 95% CI: 0.10-1.10, p=0.07) and patients of Hispanic race/ethnicity being more likely to score a positive T-MoCA result (OR=4.13, 95% CI: 0.84-20.28, p=0.08)(Table 3 ).

Table 3.

Univariate analysis of association between patient demographic variables and positive T-MoCA score.

Characteristic	OR	95% CI	p
Age
< 65	1		ref
≥ 65	2.10	0.60-7.33	0.25
Sex
Female	1		ref
Male	0.33	0.10-1.10	0.07
Ethnicity
Non-Hispanic White	1		ref
Non-Hispanic Black	1.38	0.361-5.240	0.64
Hispanic	4.13	0.84-20.28	0.08
Primary Language
English	1		Ref
Non-English	4.57	0.49-42.33	0.18
BMI
< 30	1		ref
≥ 30	1.39	0.45-4.25	0.57
Smoking	0.50	0.16-1.55	0.23

Discussion

POD is one of the most common post-operative complications and is associated with longer hospitalization, increased mortality, and increased morbidity. The T-MoCA shows promise as a screening tool for POD when in person screens are not available. Our study shows implementation of the T-MoCA in a telemedicine-based PAT setting is feasible as demonstrated by the high completion rate of those who consented to participate in our study. On average, administration of the T-MoCA took 10.5 minutes, which is comparable to the 10 minutes it takes to complete the TICS¹⁰, making it similarly accessible for both physicians and patients. More than half of the patients were Non-Hispanic Black or Hispanic demonstrating the feasibility of administering T-MoCA in a diverse population. This is in line with the study done by Katz et al., which validated the T-MoCA in the same population¹¹. Furthermore, the majority of patients who participated were over the age of 65, which is an important population to screen and risk stratify for POD prior to elective surgery.

Some limitations associated with telephone use were identified in administering the T-MoCA. It was more difficult to hear participants over the phone than in person. This became an issue as verbal clarity was critical to scoring well on the T-MoCA, as it includes multiple subtests where the participant was to repeat what the examiner said exactly. There were also instances where it was unclear if a participant was using an outside resource, such as a pen or paper or another person, to help them remember words or statements; this may be mitigated by administration of this test via a video visit in the future. Other concerns of administering the T-MoCA was lack of focus or motivation. Some participants were driving, drinking, or outside walking during the assessment. This would make it difficult for them to devote their full attention to the exam, which may have affected their results. Others gave up very quickly on the assessment as they simply did not want to take an optional test over the telephone. Taking the assessment in person in a clinical setting might provide more incentive to complete it. Scoring the T-MoCA was difficult when using a translator. The investigator could not communicate directly with the participant so scoring was based on what the translator conveyed. Use of a translator also prolonged the amount of time needed to complete the assessment. This may have frustrated participants and neither of the 2 participants who required a translator completed the test.

In terms of identifying predictors of a positive T-MoCA score, age was the only significant variable associated with a positive T-MoCA score. There were borderline associations between female gender and Hispanic ethnicity and a positive T-MoCA score. Only one other study has investigated the T-MoCA score based on patient characteristics. Katz el al. found that older age and being Hispanic was associated with a worse score on the T-MoCA¹¹, which is consistent with our study. However, they also found those of Black race and male gender scored lower on the T-MoCA. Our findings on age and race are also consistent with studies investigating the MoCA in general. Multiple studies have confirmed that older age is negatively associated with scoring well on the MoCA^{16 , 17}. The Hispanic population has been shown to have lower MoCA scores in various studies as well^{18 , 19}, with one study even suggesting the optimal cutoff score for the Hispanic and Black populations should be lower than for Non-Hispanic Whites¹⁹.

Our study has several strengths and limitations. Strengths of our study include its sampling of a racially and ethnically diverse patient population undergoing a wide range of surgical procedures. Additionally, the T-MoCA was implemented under real-world conditions of these patients’ clinical care, compared to the research-setting, and therefore provides evidence that the T-MoCA can be successfully implemented for other PAT practices. Also, we examined a patient population in which a significant number of patients did not speak English as a primary language. Limitations of our study include its small sample size, which likely prevented our analysis from detecting statistically significant associations between patient demographic characteristics and a positive T-MoCA score. About half of patients declined participation in the study, and we could not collect demographic information on these patients due to informed consent obligations. Therefore, we cannot determine if patients who refused to participate have differences in their demographics compared to included patients. However, our participation rate is similar to other post-telemedicine appointment studies.²⁰ We suspect that patients would be more amenable to participation if the T-MoCA was a routine component of the PAT evaluation. We were also not able to follow patients for the development of POD, and cannot report whether the T-MoCA is effective in risk stratifying these patients. Additionally, certain patient demographic information (age, BMI, surgery type, and co-morbidities) was collected via retrospective review of medical records and were not confirmed at time of interview. Also, the investigator who carried out T-MoCA assessment did not receive certification from the MoCA manufacturer, which may alter our results. However, this investigator was trained and supervised by an investigator with experience in T-MoCA. Future research is needed to enroll larger numbers of patients in order to determine whether the T-MoCA can accurately predict the risk of post-operative delirium, or other post-operative neurologic disorders.

Conclusion

Virtual cognitive screens have become necessary to risk stratify patients for POD as the COVID-19 pandemic restricts use of in-person cognitive screens. Our study has shown that the T-MoCA is a feasible and easily accessible exam for both physicians and patients. Most of its limitations are minor and are associated with telephone use. Older age is associated with a worse T-MoCA score and there may be limitations in administering the T-MoCA in Hispanic and female participants which would need to be explored in a larger sample size.

CRediT author statement

Nick Yu: Conceptualization, Methodology, Formal Analysis, Investigation, Data Curation, Writing- Original Draft, Visualization. Denzel Zhu: Conceptualization, Methodology, Formal Analysis, Data Curation, Resources, Writing – Review & Editing. Kara Watts: Conceptualization, Methodology, Writing – Review & Editing, Supervision. Nitya Abraham: Conceptualization, Methodology, Writing – Review & Editing, Supervision, Project Administration, Funding Acquisition. Curtis Choice: Conceptualization, Methodology, Writing – Review & Editing, Supervision, Project Administration

Declaration of Competing Interest

The authors have no conflicts of interest, financial or otherwise, to declare.