Discrepancies between Perceived and Measured Cognition in Kidney Transplant Recipients: Implications for Clinical Management

Abstract

Background

Cognitive impairment is common in kidney transplant recipients and affects quality of life, graft survival, morbidity and mortality. Failure to identify patients with cognitive impairment can withhold intervention. This study determines whether measured cognition with standard screening tools offers any advantage over perceived cognition in screening transplant patients for cognitive impairment.

Methods

Cognition was assessed in 157 kidney transplant recipients using the Montreal Cognitive Assessment (MoCA) (measured cognition). In addition, transplant physicians and nurse coordinators were asked to rate transplant recipients’ level of cognition after routine clinical interactions (perceived cognition). Physicians and nurses were blind to MoCA scores. Perceived cognition scores were compared to MoCA scores.

Results

Perceived cognition scores weakly correlated with MOCA scores (γ= 0.24, p= 0.001 for physicians and γ=0.33, p <0.0001 for nurses). Physician scores moderately correlated with nurses scores (κ= 0.44, p<0.0001). Clinical perception had a low accuracy for identifying patients with cognitive impairment (sensitivity 66% for physicians, 65% for nurses), and those without cognitive impairment (specificity 67% for physicians, 76% for nurses).

Conclusion

Clinical perception is inaccurate at detecting cognitive impairment in kidney transplant recipients. Objective tests should be considered to screen kidney transplant recipients for cognitive impairment.

Introduction

Cognitive impairment is common in kidney transplant (KT) recipients [1] and negatively impacts allograft survival and mortality [2,3]. Compared with prevalence in age-matched controls, cognitive impairment is increased in ESRD and affects up to 87% of patients on dialysis [4]. Further, although typically considered a problem of the old, younger KT recipients also have cognitive impairment [1,5]. Fortunately, cognitive and psychomotor function have been shown to improve after kidney transplantation [5–9].

Despite data demonstrating prevalent cognitive impairment in the kidney disease population and a potential risk of worse patient outcomes, routine screening for cognitive impairment in KT recipients is generally not performed. Several tools for screening for cognitive impairment are available [10], however, their use in nephrology and transplant clinics is limited, leading to under-recognition of cognitive impairment [4]. Despite validated cognitive screening tools, transplant centers frequently depend on subjective clinical assessment of patient cognition. Failure to recognize cognitive impairment is likely to delay intervention [11,12], and risk patient safety [13].

The care of a KT recipient is a complex and laborious combination of clinic appointments, laboratory tests, polypharmacy, and medication and dietary adherence. Medical non-adherence is fairly common in KT, and is a major cause for rejection and graft loss [14,15]. Cognitive impairment can impair understanding of instructions and the ability to follow them, ultimately leading to medical non-adherence [16,17]. Recognition of cognitive impairment can help tailor the complexity of terms used while delivering education and instructions to patients, provide additional support from transplant coordinators and pharmacists to help with medical adherence, and help in timely identification, involvement, and counseling of caregivers and surrogate decision makers [18,19]. Identification of cognitive impairment can help with prognostication and management of patients [20]. Large studies have shown that patients with unrecognized cognitive impairment are less likely to be evaluated for reversible causes [21].

Studies in other populations have indicated that the identification and documentation of cognitive impairment by physicians, unaided by an objective screening tool remains poor [22]. It remains unclear if the subjective assessment of cognition by physicians or nurses is as reliable as an objective assessment in KT recipients. In this study, we evaluated the accuracy of transplant physicians and nurses’ perception of KT recipients’ cognition compared to a standard screening tool.

Methods

Study Population

Adult KT recipients presenting for outpatient post-transplant care at University of Kansas Medical Center between June 29, 2015 and January 1, 2017 were screened for cognitive impairment by using the Montreal Cognitive Assessment (MoCA) [23] as part of their clinical care. After the clinic visit, one of the 6 physicians and 1 nurse practitioner (referred to as “physicians” hereafter), and 9 nurse coordinators (referred to as “nurses” hereafter), who assessed the patient that day, scored the patients’ cognition. The physicians and nurses were blinded to the MoCA results. Data collection was initiated as a quality improvement project to measure cognition in KT recipients and was declared Institutional Review Board exempt. Informed consent was therefore not obtained. Data were collected on paper case report forms and then entered into electronic case report forms using REDCap (Research Electronic Data Capture), a web-based, electronic data capture tool hosted on a secure, password protected, HIPAA-compliant server [24]. To minimize the impact of high dose steroids and operative procedures on cognition, all cognitive assessments were performed at least one month after KT. As part of a standardized protocol, all patients were maintained on a mycophenolic acid compound and a calcineurin inhibitor with or without prednisone.

Adult English speaking KT recipients were included. Patients were excluded if they had i) any hearing or visual impairment that would prohibit them from participating in MoCA; ii) if they were unable to read, write, speak or understand English; iii) if they had stoke, concussion, or traumatic brain injury within two months of the evaluation; v) if they already had a diagnosis of dementia; or iv) if they had uncontrolled psychosis or active seizure disorder. Also, patients with psychiatric diseases or on psychotropic medications were not selected for MoCA.

Standardized Screening for cognitive impairment

Cognitive function was assessed with the Montreal Cognitive Assessment (MoCA) (supplementary data 1) [23]. The MoCA is a validated, clinic-based tool that samples from various domains of cognition and is sensitive in detecting mild cognitive impairment. MoCA consists of a single page test with a maximum score of 30 (range: 0–30). The MoCA takes less than 10 minutes to complete and assesses seven domains of cognition: visuospatial/executive, naming, memory (delayed recall), attention, language, abstraction and orientation. The original English version 7.1 was used (http://www.mocatest.org/paper-tests/moca-test-full/). We chose MoCA over the more commonly used mini-mental state exam (MMSE) for detecting cognitive impairment, as the latter is more focused on memory rather than domains such as executive function, that are more commonly affected in kidney disease [25]. Compared to MMSE, MoCA has a better ability to detect vascular dementia with a higher area under the curve [26].

The MoCA was administered by medical assistants who underwent an hour of training that included detailed review of the online instructions on MoCA along with practice sessions on mock patients. MoCA was administered in a private room to minimize distraction and assure confidentiality.

Perceived cognition as assessed by transplant physicians and nurses

During a clinic visit, patients were evaluated by a physician and a nurse. All physicians and nurses independently rated the patients’ cognitive status on a 10-point Likert scale (1= severe dementia, 10= no cognitive impairment) after the completion of clinic visit. These physicians and nurses were blinded to the MoCA results as well as to scores of other physicians and nurses. The interpretations of the scores from 1–10 (1–2 indicated dementia; 3–5, moderate cognitive impairment; 6–8, mild cognitive impairment, and 9–10, no cognitive impairment) and the definitions of dementia, moderate cognitive impairment, mid cognitive impairment and no cognitive impairment were provided to the physicians and nurses (supplementary data 2) on the scoring sheet where they were asked to assess patients by circling a number between 1–10 on the Likert scale. Since the aim for the project was to assess the ability of physicians and nurses to identify patients with cognitive impairment based on their current experience and knowledge, the physicians and nurses were not provided any additional education on detection of cognitive impairment.

Statistical Analysis

Demographic and clinical variables that can potentially influence cognitive function such as age, gender, ethnicity, level of education, body mass index (BMI), blood pressure, history of smoking, diabetes, cause of ESRD, serum hemoglobin, estimated glomerular filtration rate (eGFR), time on dialysis prior to transplant, and time after transplant were abstracted through review of the patient’s electronic medical record. Descriptive statistics (mean and standard deviation for continuous measures, and frequencies and relative frequencies for categorical measures) were calculated for these variables.

The MoCA scores were compared with the physicians’ and nurses' perceived cognition scores using gamma statistics because the measurement scales were different (0–30 for MoCA and 1–10 for perceived cognition). Gamma statistics provide an assessment of not only association, but also the direction of that association by assessing directional concordance in a structure free manner by testing for monotonic relationships. The interpretation of gamma statistics is similar to that of a nonparametric correlation coefficient. They are both scaled from -1 (for negative associations) to 1 (for positive associations). The physician scores were compared with the nurses’ scores using gamma and weighted kappa statistics. Kappa and weighted kappa measure agreement (both assessments providing the same rating on an observation) beyond what would be expected if the scoring were performed at random. While values can theoretically range from -1 to 1, typically values greater than zero are observed, indicating some level of agreement beyond that expected by chance along. Interpretation of the κ and γ values was performed according to Landis and Koch. Values < 0.20 are considered as poor, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial and 0.81–1.00 as very good[27]. Correlation coefficients were calculated for each domain of cognition assessed by MoCA, namely, visuospatial/executive function, naming, memory (delayed recall), attention, language, abstraction and orientation.

The performance characteristics of perceived cognitive scores by physicians and nurses were also calculated. The MoCA scores were dichotomized into two categories: cognitive impairment (score <26) and no cognitive impairment (score ≥ 26) based on published data [23]. The perceived scores were dichotomized into two categories: cognitive impairment (score of <8) and no cognitive impairment (score of 9 or 10). The sensitivity, specificity, positive predictive value and negative predictive value of physicians’ and nurses’ perception for determination of cognitive impairment were calculated. For all analyses, a P<0.05 was considered statistically significant. Analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC, 2002–2012) and R version 3.3.1 (R Core Team 2016).

Results

All but four patients who were approached completed the MoCA assessment. Two of the four patients did not offer a specific reason for not completing the MoCA, one patient had forgotten his eye glasses on the day of the assessment, and one patient was getting late for another clinic appointment. Thus, a total of 157 patients completed MoCA. Of these, 57% were men, 74% were white, mean age was 55 ± 14.8 years, mean time post-transplant was 5.7 ± 5.1 years, 37% had a bachelor’s degree or higher, and 28% had some college education. Thirty percent of the patients had a MoCA score of <26. Physicians and nurses perceived 43% and 36% of the patients to be cognitively impaired respectively. Table 1 lists baseline demographic and clinical data of patients based on their measured and perceived cognition. Ninety-five percent of the patients had their cognition assessed both by the physicians and nurses. Five patients had the cognitive assessment only by a physician and three patients had cognitive assessment only by a nurse. The mean MoCA score was 26.6 ± 2.9, and the mean perceived score by a physician was 9 ± 1.4 and by a nurse was 8.6 ± 1.6.

Table 1.

Patient Characteristics.

Patient Characteristics (n=157)	Measured Cognition by MoCA		Perceived Cognition by Physicians		Perceived Cognition by Nurses
	Cognitive impairment (n=48)	No cognitive impairment (n=109)	Cognitive impairment (n=65)	No cognitive impairment (n=84)	Cognitive impairment (n=55)	No cognitive impairment (n=97)
Age, mean ± SD (years)	58.5 ± 14.7	53.35 ± 14.7	60.1 ± 13.9	50.9 ± 14.5	60.1 ± 13.0	51.9 ± 15.2
Gender, n (%)
female	20 (41.7)	48 (44)	24 (36.9)	40 (47.6)	18 (32.7)	48 (49.5)
Race, n (%)
Caucasian	35 (72.9)	81 (74.3)	46 (70.8)	65 (77.3)	41 (74.5)	72 (74.2)
African American	11 (22.9)	19 (17.4)	15 (23.1)	12 (14.3)	13 (23.6)	15 (15.5)
Other race	2 (4.2)	9 (8.3)	4 (6.2)	7 (8.3)	1 (0.0)	10 (10.3)
Education level n (%)
≥ Bachelor’s Degree	15 (31.3)	43 (39.4)	19 (29.2)	36 (42.8)	18 (32.7)	38 (39.2)
Some College	12 (25)	31 (28.4)	17 (26.2)	25 (29.8)	9 (16.4)	33 (34)
BMI, n (%)
≥ 30 (kg/m2)	18 (37.5)	48 (44)	30 (46.2)	34 (40.5)	24 (43.6)	40 (41.2)
Blood Pressure, mean ± SD (mmHg)
Systolic	140.8 ± 18.9	138.7 ± 15.6	143.5 ± 19.5	136.1 ± 13.9	142.9 ± 18.9	137.5 ± 15.2
Diastolic	75.3 ± 14.3	76.2 ± 12.5	73.9 ± 14.4	77.5 ± 11.7	74.3 ± 14.3	76.8 ± 12.4
Serum hemoglobin, mean ± SD (g/dl)	12.8 ± 1.7	13.1 ± 1.8	12.8 ± 1.9	13.2 ± 1.7	12.8 ± 1.8	13.1 ± 1.8
eGFR, mean ± SD (ml/min/1.73 m2)	50.4 ± 14.5	50.32 ± 13.3	50.1 ± 14.2	50.6 ± 13.3	50.8 ± 13.1	50.0 ± 14.2
Time since KT, mean ± SD (years)	7.0 ± 4.9	5.0 ± 5.1	6.0 ± 5.0	5.0 ± 4.2	5.9 ± 4.7	5.5 ± 5.4
Time on dialysis, mean ± SD (years)	2.3 ± 3.4	2.8 ± 3.3	3.4 ± 4.8	2.1 ± 1.1	3.05 ± 3.19	2.32 ± 3.48
H/o smoking, n (%)	26 (54.2)	42 (38.5)	32 (49.2)	32 (38.1)	29 (52.7)	36 (37.1)
H/o diabetes, n (%)	25 (52.1)	29 (26.6)	27 (41.5)	24 (28.6)	23 (41.8)	30 (30.9)

MoCA, Montreal cognitive assessment; BMI, body mass index; KT, kidney transplantation; H/o, history of.

There was a fair correlation between the measured cognition scores and perceived cognition scores for physicians γ= 0.24, CI (0.10, 0.39), p= 0.001 and nurses γ= 0.33, CI (0.19, 0.46), p <0.0001 (Table 2). The variability in perceived cognitive impairment by physicians and nurses, when compared to MoCA scores, did not follow a specific direction (Figure 1). Next, we compared the perceived score with individual domain scores in the MOCA and found domain specific differences in correlation. Perceived cognition had the highest correlation with the naming domain for physicians (γ=0.56, CI (0.28, 0.85), p= 0.0001) and nurses (γ=0.72, CI (0.55, 0.89), p<0.001) (Table 2). Conversely, the correlation of perceived cognition with attention, abstraction or orientation domains was not as robust.

Table 2.

Correlation of MoCA scores with the transplant physicians’ and nurses’ perceived scores. A. Correlation of MoCA score (0–30) with the transplant physician and nurses’ perceived scores (1–10), and B. Correlation of MoCA scores of different domains with the transplant physician and nurses’ perceived scores (1–10).

A. Correlation of MoCA score (0–30) with the physician and nurses’ perceived scores (1–10)
	Correlation with physician scores		Correlation with nurse scores
	Correlation Coefficient (γ)	Confidence Interval (CI)	Correlation Coefficient (γ)	Confidence Interval (CI)
MoCA score (0–30)	0.24**	0.10 – 0.39	0.35X	0.19 – 0.46

B. Correlation of MoCA scores of different domains with the physician and nurses’ perceived scores (1–10)
	Correlation Coefficient (γ)	Confidence Interval (CI)	Correlation Coefficient (γ)	Confidence Interval (CI)
Visuospatial/executive (0–5)	0.26**	0.08 – 0.44	0.37X	0.21 – 0.53
Naming (0–3)	0.56X	0.28 – 0.85	0.72X	0.55 – 0.89
Attention (0–2)	0.05	−0.16 – 0.25	0.04	−0.19 – 0.27
Language (0–1)	0.33X	0.15 – 0.51	0.34X	0.18 – 0.51
Abstraction (0–2)	0.14	−0.21 – 0.50	0.31	−0.03 – 0.65
Memory (0–5)	0.28**	0.11 – 0.45	0.31X	0.14 – 0.48
Orientation (0–6)	0.14	−0.30 – 0.59	0.05	−0.48 – 0.59

^*p values <0.05;

^**p value <0.01; and

^Xp value <0.001.

Figure 1.

Boxplot of MoCA scores and A) physician and B) nurse perceived scores. Perceived scores and MoCA scores are overlaid on a scatterplot of jittered data.

Next, we compared the accuracy of perceived scores by the physicians and the nurses for predicting cognitive impairment as defined by MoCA score of <26. The agreement between the physicians and the nurses was moderate with κ = 0.437, CI (0.347, 0.527), p<0.0001) (Table 2). The sensitivity, specificity, positive predictive value and negative predictive values for perceived cognition are shown in Table 3. The perceived cognitive scores had a good negative predictive value for both physicians (82%) and nurses (84%), but poor sensitivity, specificity and positive predictive value.

Table 3.

Sensitivity, specificity, positive predictive value and negative predictive value of Physician Perceived Cognition score and Nurse Perceived Cognition score in recognizing cognitive impairment.

	Physician Perceived Cognition score	Nurse Perceived Cognition score
Sensitivity	65.9 %	65.2 %
Specificity	67.2 %	76.4 %
Positive predictive value	46.9 %	54.5 %
Negative predictive value	81.8 %	83.5 %

Discussion

We found that the correlation between perceived cognition of KT recipients by physicians and nurses and measured cognition was not very good. In addition, there was a wide variability in the perceived cognition scores for each value of the MoCA score. Some domains of cognition such as naming correlated better than other domains with the perceived cognition. There was a moderate agreement between the cognitive assessment by the transplant physicians and nurses.

Our data suggest that physicians and nurses are unable to reliably identify KT recipients with cognitive impairment. This lack of reliability was bidirectional: many patients with normal MOCA scores were perceived to have cognitive impairment and other patients with abnormal MOCA scores were perceived to have normal cognition (Figure 1). We also analyzed the performance characteristics by education, gender and race (supplementary data 3) and found that the sensitivity for perceived cognition by both physicians and nurses was lower for younger patients. It is possible that physicians and nurses were influenced in their ratings based historical interactions with the patients. Conversely, one would expect that repeated interactions helped the physicians and nurses to predict cognitive impairment more accurately. A plausible explanation for poor accuracy in identifying patients with cognitive impairment may be that the physicians and nurses are distracted by medical conditions of the patients and therefore focus less on cognition. Similar results, indicating inadequate identification of cognitive impairment in other high risk patient populations [21,22,28] indicate a need for objective assessments of cognition rather than relying on physician or nurse perception. Alternatively, physicians and nurses may lack education on detection of cognitive impairment, and education and awareness on cognitive impairment may improve the accuracy of perceived cognition, as it does in other populations [29].

We analyzed the physician and nurses scores separately as they may have different kinds of interactions with patients. This also helps us identify variability in perceived cognition among different health care providers. Despite a poor correlation between measured and perceived cognition, there was a moderate agreement between physicians and nurses perceived cognition scores. This may indicate that patient factors rather than provider factors may play an important role in perception of cognition. For instance, post hoc analyses suggest that the sensitivity of health care providers in predicting cognitive impairment was much lower in younger than older patients; 50 % vs. 78% (supplementary data 3). Although not powered, these observations suggest that both physicians and nurses perceive younger patients to be less likely to have cognitive impairment, simply because cognitive impairment is not suspected in younger patients. Further, the domain of naming correlated better with measured cognition for both physicians and nurses (Table 2). It is possible that naming as a domains is easier to evaluate during routine clinic visits and thereby correlate better with measured cognition. However, the domains of memory and executive function, are likely better determinants of medical adherence, but perhaps more difficult to assess during routine clinic visits, leading to a false estimation of global cognition by physicians and nurses.

To the best of our knowledge, this is the first study to assess whether perceived cognition by physicians and nurses correlates with measured cognition using a standard test in KT recipients. Our study is relevant because other studies have indicated a high prevalence of cognitive impairment in KT recipients [1] and association of poor cognition with inferior graft outcomes [2], perhaps through affecting medical adherence [14,15]. Non-adherence in patients may not be by patient’s choice, but instead may be due to cognitive impairment. Cognitive impairment can further complicate the complex post-transplant management. Identification of patients with cognitive impairment is an important first step in management of cognitive impairment. Failure to accurately identify patients with cognitive impairment can deprive them of additional support that could otherwise be made available. This support may include repeated counseling, simplified verbal and written instructions, alarms or reminders, and perhaps more frequent monitoring. Cognitive impairment may also help identify patients with frailty who may benefit from additional support [30].

Our study does have some limitations. We did not perform a detailed neuropsychological assessment to follow screening by MoCA. We used MoCA because it is a clinic-based screening tool to assess cognition. Previous publications compared two screening tools, Mini Mental State Examination (MMSE) and MoCA and found that the MoCA was more sensitive than MMSE for both mild cognitive impairment (sensitivity 90% vs. 18%) and for mild Alzheimer’s dementia (sensitivity 100% vs.78%) with comparable specificity (87% vs. 100%) [23]. MoCA is a practical test and can be used during routine clinic visits where detailed neuropsychological assessments are not readily available. We agree that further neuropsychological assessments should be performed in patients with abnormal MoCA scores. The Likert scale used for measuring perceived cognition needs psychometric validation. As a single center study, with a majority of cohort being white and with a high level of education, the generalizability of our results may be limited, but our data suggests that more studies are needed to investigate cognitive impairment in KT recipients.

In conclusion, our study found that physicians and nurses do not accurately screen patients with cognitive impairment. Because cognitive impairment affects medical adherence and post-transplant outcomes, objective tests are needed for an accurate assessment of cognition. Objective screening can appropriately identify patients with cognitive impairment so that appropriate interventions can be instituted.

Abbreviations

KT

Kidney Transplant

MoCA

Montreal Cognitive Assessment

MMSE

Mini-mental state exam