Comparison of Three Cognitive Exams in Cardiac Arrest Survivors

Abstract

Background

Cognitive deficits may detract from quality of life after cardiac arrest. Their pattern and prevalence are not well documented. We used the Computer Assessment of Mild Cognitive Impairment (CAMCI), the Montreal Cognitive Assessment (MOCA) and the 41 Cent Test (41CT) to assess cognitive impairment in cardiac arrest survivors and examine the exams’ diagnostic accuracy. We hypothesized that the scores of these exams would indicate the presence of cognitive impairment in arrest survivors, that the overall scores on the three study assessments would correlate with one another, and that the 41CT, MOCA, and executive function element of the CAMCI would vary independently from other non-executive CAMCI components, reflecting executive function impairment after cardiac arrest.

Methods

Four researchers administered the CAMCI, MOCA, and/or the 41CT to cardiac arrest survivors after discharge from the intensive care unit between 2010 and 2015. Physicians screened patients with the Mini-Mental State Exam to determine when this cognitive testing was feasible, generally when the patient was able to score 20-25 points on the MMSE. We performed pairwise correlations between the different subscales' and tests' scores.

Results

One hundred and fourteen participants completed the CAMCI, of which 38 (33.3%) participants additionally completed the MOCA and 41CT. The median (IQR) percentile score for CAMCI for all 114 participants was 33.5 (18.3, 49.8), which corresponds to moderately low risk of impairment. The median (IQR) for the MOCA was 22.0 (19, 24.8) out of a possible 30, which is considered indicative of abnormal cognitive function, and for the 41CT was 6 (5, 7) out of a possible 7 points when all 38 participants were included. MOCA correlated strongly with the overall CAMCI score (r = 0.71); the CAMCI correlated moderately strongly with the 41CT (r =0.62) and the MOCA and 41CT were moderately strongly correlated with each other (r = 0.56). When all 114 CAMCI scores were considered, the Executive Accuracy subscale was strongly correlated with the overall CAMCI score (r= 0.81).

Conclusion

The CAMCI detects cognitive impairment after cardiac arrest. The MOCA correlates strongly with the overall CAMCI and the executive function subscale of the CAMCI. The 41CT as appears less effective than the MOCA in detecting cognitive deficits.

Background

Cardiac arrest affects approximately 350,000 people yearly in the United States, and survival is an estimated 8%.¹ Survivors exhibit cognitive decline or impairment^{2, 3} that ranges from mild to severe, including memory loss^2–6 decreases in psychomotor function,^{5, 7} executive function,⁵ and visuospatial function.⁵ These impairments affect up to 88% of long-term arrest survivors and can detract from health-related quality of life for many years.⁸

Different examinations assessed impairment after cardiac arrest, including the Mini-Mental State Examination (MMSE), Cerebral Performance Categories (CPC), and the Modified Rankin Scale (mRS).^{3, 9} However, these exams may be inadequate. The MMSE has shown ceiling effects when used in patients without dementia, limiting its usefulness to detect mild cognitive impairment in non-demented patients.^{10, 11} The MMSE was insensitive when used in cardiac arrest patient populations,¹² and requires that the patient exhibit large and noticeable deficits to decrease his or her MMSE score.¹³ Additionally, in the cardiac arrest patient population, many patients who scored well the MMSE still needed assistance with basic activities of daily living, making it a less-than-ideal examination to use in isolation.¹⁴ The CPC and mRS are five- and six-point scales used to evaluate functional and global status. They do not directly assess cognitive function. Few levels separate survival with good neurological outcome from coma and death; these exams are too coarse to detect more subtle cognitive impairments that can impact the quality of life for post-arrest patients. Moreover, patients with a satisfactory health-related quality of life score have been found to score poorly on the CPC, indicating cognitive impairment despite a high functional status.¹⁵ No exams designed specifically to assess cognitive impairment in cardiac arrest survivors currently exist.

The Post-Cardiac Arrest Service (PCAS) at the University of Pittsburgh routinely administers three cognitive exams to survivors of cardiac arrest before they are discharged from the hospital: the 41 Cent Test (41CT), the Montreal Cognitive Assessment (MOCA), and the Computer Assessment of Mild Cognitive Impairment (CAMCI). We assessed these bedside exams to determine if they were more sensitive than and potentially superior to the MMSE when applied to cardiac arrest survivors. Both exams assess mild cognitive impairment and were shown to be more sensitive than the MMSE.^{10, 16–18} We have previously reported on the CAMCI in the acute care setting.¹⁹ The MOCA has been utilized for a small number of out-of-hospital cardiac arrest survivors as a very long-term follow-up cognition measure.⁸ The 41CT, developed at the University of Pittsburgh, is a simple screening exam involving mental manipulation of coins that can be given orally. However, this test has not been rigorously studied and its sensitivity compared to the other exams is unknown.

The objective of this study was to examine the utility of the CAMCI, MOCA, and 41CT for identifying cognitive impairment in cardiac arrest survivors who were assessed by physicians with the MMSE. We also determined the correlations between the CAMCI, MOCA, and 41CT. We hypothesized that the scores of these exams would indicate cognitive impairment in arrest survivors. We further hypothesized that the overall scores on the three study assessments would correlate with one another; strong associations would provide evidence of convergent validity. Lastly, we hypothesized that the 41CT, MOCA, and executive function subscore of the CAMCI would reflect executive function impairment after cardiac arrest, which has been well documented in cardiac arrest survivors.²⁰

Methods

The University of Pittsburgh Institutional Review Board approved the study. All subjects were treated by the PCAS at UPMC Presbyterian and Montefiore hospitals and received standardized post-cardiac arrest care including that has been reported previously.²¹ Four researchers administered the CAMCI, MOCA, and/or the 41CT to cardiac arrest survivors no sooner than 24 hours after discharge from the intensive care unit (ICU) between April 2010 and January 2015. These researchers were employed as specialists with the University of Pittsburgh Department of Emergency Medicine and received training on exam administration. Testing was administered by one researcher, who was present for all three exams and who could not be blinded for results of the other exams in the exam set. A physician must have deemed the patient ready for additional cognitive testing using a “low bar MMSE” before the patient attempted any of the study exams. The low bar MMSE requires that the patient must be awake, alert, and oriented to self, time and place as well as able to understand basic logic. This would equate to a minimum score of 20-25 out of a possible 30 points on the MMSE.

Researchers began to conduct the CAMCI independently in April 2010. The MOCA and 41CT were administered in October 2012, at which point all participants were given the 41CT, MOCA, and CAMCI in that order during one session. The three-exam session was conducted within 24-72 hours after the low bar MMSE was given. Participants completed the study exams once and were allowed to cease participation at any time during the examination period. The researcher interacted with the patient only as necessary; visitors were asked to step out of the room during testing to ensure a quiet environment. Participants were included if they completed the CAMCI with or without the MOCA or 41CT by January 2015.

Study Exams

The MMSE was used as a screening tool in this study. It has been described elsewhere.²² A physician may have administered the MMSE to a post-cardiac arrest patient multiple times until the patient achieved a satisfactory score, defined for this study as greater than or equal to 20 out of 30 points, before enrollment. We chose this score as it would allow patients with mild to no cognitive impairment to be included, as including only patients with an “unimpaired” MMSE score of 27 out of 30 would exclude most, if not all, of our participants. Patients were excluded from this study if they did not achieve the prerequisite MMSE score before hospital discharge.

The 41CT is a novel six-question exam utilizing American coinage to assess cognitive processing ability. The participant manipulates a penny, nickel, dime and quarter mentally and does calculations with the currency. The exam takes approximately five minutes to complete and is scored by the exam proctor, with each question being worth one point. The patient is given the option to use physical coins; the ability to answer the questions without the physical coins is worth one additional point, making the exam worth a total of seven points. The exam can be given using little to no equipment. Using the same cutoff score as the MOCA (86.7%), a passing score would be 6 out of 7 points.

The MOCA (Version 7.2, Alternative) is a 30-point exam that is administered in paper and pencil format that was recommended for use in the cardiac arrest patient population by the European Resuscitation Council.²³ This test has been described elsewhere in the literature.¹⁸ The MOCA takes approximately ten minutes to complete and is scored by the exam proctor; scores greater than or equal to 26 out of 30 (86.7%) points are considered normal. Scores of less than or equal to 25 out of 30 are considered abnormal.¹⁸

The CAMCI utilizes a portable, touch-screen tablet computer with a digitized pen. This exam has been described elsewhere in the literature.²⁴ The exam determines the patient’s risk of mild cognitive impairment and the results are ranked on eleven cognition subscales. The CAMCI was designed to progress at the patient’s own pace, and generally requires at least thirty minutes to complete; the exam is automatically scored upon the patient’s completion of the exam. Scores are adjusted based on the exam taker’s level of education and familiarity with both computer and ATM usage. It does not adjust for age or sex. The risk of mild cognitive impairment is categorized as low, moderate, or high risk and the scores are presented in percentile format.¹⁰

We describe the variation in exam scores using median and interquartile range (IQR). We investigated the Pearson correlation coefficients for overall CAMCI score, the eleven subscales contained in the CAMCI, the MOCA score and the 41CT score. A correlation coefficient greater than or equal to 0.70 or less than or equal to -0.70 was considered very strong, while a correlation between 0.50 - 0.69 and -0.69 - -0.50 was considered moderately strong. Coefficients between -0.49 - 0.49 were considered not correlated. We used the Student’s t-test to compare means where appropriate and McNemar’s test to compare pass/fail rates between the three exams with an alpha of 0.05.

Results

Between 2010-2015, 1,081 patients were treated by the PCAS; 650 (60.1%) did not survive to undergo testing. Of the 431 survivors, 95 (22.0%) could not follow commands, leaving 336 (88.0%) who could be assessed for study eligibility. Of these 336 patients, 207 (61.6%) were excluded due to patient availability or MMSE scores, leaving 129 patients who were offered testing; 3 refused. The CAMCI was completed by 114 patients. Thirty-eight (33.3%) of those participants also completed the MOCA and 41CT. The remaining 12 patients completed the MOCA and the 41CT but were unable to complete the CAMCI.[Figure 1] In sum, 50 sets of 41CT and MOCA exam scores were recorded and 38 of these also included the CAMCI. Background information on the subjects is in the Supplemental Table.

Figure 1.

Inclusion Criteria Scheme

The median (IQR) percentile score for the overall CAMCI for all 114 patients was 33.5 (18.3, 49.8). The median (IQR) percentile overall CAMCI score for the multiple-exam cohort was 30.0 (15.8, 48.5) while the median (IQR) score for the MOCA was 22 (19, 24.8) out of a possible 30 and for the 41CT was 6 (5, 7) out of a possible 7 points. The average CAMCI score was not significantly different between participants who completed the CAMCI by itself and those who completed the MOCA and 41CT as well (p = 0.458). The median (IQR) percentile scores for each of the CAMCI subscales for all 114 participants are presented in Table 1. There was no correlation between age, sex, or education and overall CAMCI score (r = -0.28, -0.08, and 0.06, respectively), and there was no significant difference in CAMCI performance between participants with a post-high school education and those with a high school education/GED or less (p = 0.73).

Table 1.

Median (IQR)CAMCI Subscale Scores

Subscale	Percentile Score, median (IQR)	Risk of Mild Cognitive Impairment
Attention Accuracy	69.0 (4, 71)	Low
Attention Speed	27.5 (6, 63)	Moderate
Executive Accuracy	30.0 (13, 54)	Moderately Low
Executive Speed	15.5 (1, 43)	Moderate
Processing Speed	25.5 (3, 49)	Moderate
Verbal Memory Accuracy	28.0 (9, 55)	Moderate
Verbal Memory Speed	31.0 (10, 52)	Moderately Low
Nonverbal Memory Accuracy	27.0 (6, 54)	Moderate
Nonverbal Memory Speed	19.0 (1, 60)	Moderate
Functional Memory Accuracy	60.0 (15, 86)	Low
Incidental Memory Accuracy	46.0 (12, 78)	Low

The overall CAMCI score was highly correlated with the Executive Accuracy CAMCI subscale (r = 0.81, 95% CI 0.751-0.963). Overall CAMCI score showed a moderately strong correlation Verbal and Nonverbal Memory Accuracy scores (r = 0.54, and 0.51, 95% CI 0.086- 0.8, and 0.435-0.901, respectively). Processing Speed had a moderately strong correlation with both Attention Speed (r = 0.67, 95% CI 0.253-0.854) and Executive Speed (r = 0.69, 95% CI 0.552-0.927). Nonverbal Memory Speed was strongly correlated with Processing Speed (r = 0.83, 95% CI 0.032-0.779). The remaining subscales were not correlated with each other or with the overall CAMCI score.

When only the scores of the 38 participants who completed all three exams were considered, the CAMCI correlated very strongly with the overall MOCA score (r = 0.71, 95% CI 0.620-0.940) and moderately strongly with the 41CT (r = 0.62, 95% CI 0.118-0.811). [Table 2] The MOCA and 41 CT were moderately correlated with each other (r = 0.56, 95% CI 0.354- 0.882) and both were moderately correlated with Executive Accuracy (r = 0.65 and 0.51, 95% CI 0.447-0.904 and -0.062-0.740, respectively). The MOCA and 41 CT were moderately correlated with other CAMCI subscales.[Table 2] Score distributions are shown in Figure 2. The exact McNemar’s significance probability when comparing the pass/fail rates for CAMCI and MOCA was 0.001, for CAMCI and 41CT was 0.109 and for MOCA and 41CT was 0.000.

Table 2.

Correlation Coefficients of CAMCI, MOCA and 41 Cent Test Scores: very strongly correlated variables are indicated in grey*

	CAM CI	MOC A	41C T	AttA cc	AttSpe ed	ExecA cc	Exec Spee d	Proc Spee d	VMA cc	VM Spee d	NVMA cc	NV M Spee d	FMA cc	IMA cc
CAMCI	1
MOCA	0.71	1
41CT	0.62	0.56	1
AttAcc	0.73	0.30	0.32	1
AttSpeed	0.18	0.20	0.03	−0.01	1
ExecAcc	0.83	0.65	0.51	0.63	0.11	1
ExecSpeed	0.59	0.43	0.35	0.38	0.30	0.57	1
ProcSpeed	0.59	0.58	0.42	0.28	0.56	0.52	0.78	1
VMAcc	0.42	0.51	0.21	0.08	0.12	0.27	0.21	0.37	1
VMSpeed	0.39	0.53	0.39	0.14	0.30	0.32	0.51	0.64	0.20	1
NVMAcc	0.74	0.48	0.48	0.68	0.00	0.46	0.38	0.39	0.20	0.41	1
NVMSpeed	0.42	0.51	0.33	0.19	0.47	0.41	0.61	0.87	0.34	0.58	0.29	1
FMAcc	0.48	0.52	0.43	0.26	0.18	0.28	0.25	0.37	0.33	0.43	0.29	0.16	1
IMAcc	0.42	0.29	0.18	0.28	0.05	0.24	0.18	0.09	0.18	0.25	0.37	− 0.05	0.29	1

Figure 2.

Paired Score Distributions: pass/fail cutoff scores are indicated with a black cross

Discussion

We demonstrate a cohort of patients with indicators of impairment or risk of impairment despite the their ability to pass a modified MMSE. The mean CAMCI score indicated “Moderate-Low Risk” of impairment, with significant variation in exam scores, ranging from 0 to the 78^th percentile. The average MOCA and 41CT scores were in the “Abnormal” range. Scores on the 41CT, MOCA and CAMCI were positively associated with each other. CAMCI and MOCA scores were strongly correlated with the Executive Accuracy subscale of the CAMCI. Executive function may be a domain of cognition that is particularly affected by cardiac arrest and one that physicians and caregivers can focus on during the patient’s rehabilitation phase.

Each exam utilized in this study has its own inherent benefits and drawbacks. The CAMCI’s computerized nature requires exam proctor supervision. Results are scored automatically, yielding a detailed report pinpointing areas of impairment. However, the length of time required by the CAMCI limits its utility in the clinical setting, and the scoring system relies heavily on the physical abilities of the test-taker. Participants with impaired hearing, vision, or fine motor skills had more difficulty than test-takers without such impairments simply due to the physical demands of the exam’s format.

The MOCA and 41CT, while not as precise as the CAMCI, have unique benefits: they are shorter, less intensive and require inexpensive equipment, making them more practical to administer than the CAMCI. Patients with reduced physical capabilities are able to complete the exams, as they do not place emphasis on timing or physical manipulation of the exam. The MOCA breaks down its tasks into general domains of cognition, which can help identify areas of impairment. Both the MOCA and 41CT require exam proctor training. As both MOCA and 41CT were correlated with the CAMCI score, they may be more rapid screening tools than the CAMCI.

Figure 3 depicts a hierarchical testing model incorporating the three exams progressing from the shortest to longest exam. A participant’s performance on the shorter exams can determine whether further testing is needed. Sequential administration may be useful for clinicians screening patients for cognitive impairments. Participants who scored poorly on the 41CT scored poorly on all further exams; those who passed had varying future success. Testing may be halted after a patient passes both the 41CT and MOCA, rendering the CAMCI unnecessary, as the patient is likely to pass that exam as well. Testing may also be halted after a patient fails the 41CT, as the individual is unlikely to succeed on the other exams and may suffer from widespread cognitive deficits. The CAMCI may not be an efficient screening tool, but it can be employed to identify impairment in specific areas to tailor rehabilitation efforts.

Figure 3.

Pass/Fail Dichotomization with Testing Recommendations

Limitations of the study include a data acquisition period of five years and four different researchers, each of whom may have had different styles of conducting the examinations. We did not record which researcher conducted each exam to determine this variability. However, each received identical training in exam administration, making this unlikely. Moreover, researchers often administered the study exams in pairs during the training period, reducing variability. One reason for the discrepancy in the number of participants who completed all three exams was that the CAMCI being administered in isolation for two years before the MOCA and 41CT exams were added. Additionally, the exam regimen was conducted while the patients were still in the hospital, and several factors may have affected their test performance: pain, sleep disturbances, psychological stress, and post-ICU syndrome.

Several patient factors were not controlled for: time from ICU discharge to exam administration, patient response to the ICU environment, the participants’ level of cognitive function prior to their arrest event, and differences in resuscitative care that could have altered the duration of hypoxia/anoxia. Each of these factors could have impacted the participants’ ability to complete these cognitive exams. As this study was conducted while the participants were still hospitalized, there is little to no data in existing literature to compare our data to, limiting our ability to generalize the results.

There is likely selection bias in this cohort. All participants had to have achieved a satisfactory score on the low bar MMSE to be enrolled in this study. Therefore, patients who: survived but remained comatose, had difficulty remaining alert for extended periods of time, and awake patients who scored too poorly on the MMSE were excluded from additional cognitive testing. Excluded patients likely have more severe cognitive deficits than the tested cohort. We believe that MMSE testing should still be utilized as a screening tool prior to the administration of exams such as the MOCA, as patients who cannot satisfactorily complete the MMSE are likely to experience unnecessary stress while completing more in-depth and intense exams. Additionally, if a patient achieved a satisfactory score on the MMSE but struggled or became upset during either the 41CT or the MOCA, CAMCI testing was withheld. Finally, independent of their cognitive status, patients were excluded if their vision, hearing, or motor skills prevented them from taking the exams without assistance beyond vision and hearing aids; these physical deficits may or may not have existed prior to the arrest event. Thus, our cohort represents a subset of survivors with high cognitive, functional and psychological or emotional status.

Future studies should evaluate the effect ICU admission on cognitive function, as the deficits seen may represent both anoxic brain injury and the effect of ICU admission, as well as how post-arrest cognitive impairments will impact the day-to-day lives of cardiac arrest survivors and their family and caregivers. In-hospital cognitive exams are important, as they provide early identification of patients who are at risk for experiencing cognitive dysfunction. Performing exams such as the 41CT, MOCA, or CAMCI prior to hospital discharge would allow patients to be given focused support and recommendations for cognitive rehabilitation before they are discharged home and have fewer resources available to them.

Conclusion

CAMCI, MOCA, and 41CT testing detected mild cognitive impairment in post-arrest patients who scored satisfactorily on the low-bar MMSE. The three exam scores were associated with each other, ranging from moderately strong to very strong correlations. The Executive Accuracy score was correlated with the overall CAMCI, MOCA, and 41CT scores, indicating impairment in executive function following cardiac arrest. The 41CT and MOCA may be useful screening tools prior to the administration of longer and more intense exams such as the CAMCI.

Abbreviations

AttAcc

Attention Accuracy

NVMAcc

Non-Verbal Memory Accuracy

AttSpeed

Attention speed

NVMSpeed

Non-Verbal Memory Speed

ExecAcc

Executive Accuracy

FMAcc

Functional Memory Accuracy

ExecSpeed

Executive Speed

IMAcc

Incidental Memory Accuracy

ProcSpeed

Processing Speed

VMSpeed

Verbal Memory Speed

VMAcc

Verbal Memory Accuracy