Advances in psychometric tests for screening minimal hepatic encephalopathy: From paper-and-pencil to computer-aided assessment

Abstract

Minimal hepatic encephalopathy (MHE) is a major neuropsychiatric complication of liver cirrhosis and portosystemic shunting. Although MHE produces a spectrum of cognitive impairments in the domains of short-term attention, working memory, and executive function, it generally does not present with obvious clinical manifestation on conventional assessments. Paper-and-pencil psychometric tests, such as the psychometric hepatic encephalopathy score and the repeatable battery for the assessment of neuropsychological status, are recommended to diagnose MHE. However, these tests are neither rapid nor convenient to use in practice. To facilitate repeated testing in clinic and follow-up, computer-aided psychometric tests, such as the scan test, Cognitive Drug Research assessment battery, inhibitory control test, EncephalApp Stroop App, and critical flicker frequency, have been used to screen for MHE among patients with liver cirrhosis. The aim of this review was to describe the progression from the utility of paper-and-pencil to computer-aided psychometric tests for MHE screening in clinical practice.

INTRODUCTION

Hepatic encephalopathy (HE) refers to cerebral dysfunction resulting from decompensated liver disease and/or portosystemic shunting. HE is classified as either overt or covert based on the severity of its clinical manifestations (1). Patients with overt HE (OHE) present varying degrees of neuropsychiatric symptoms, such as asterixis, dyspraxia, stupor, and even coma. In contrast, patients with covert HE, including minimal HE (MHE) and West Haven Grade I HE, exhibit no such obvious neurological or mental symptoms. However, MHE is regarded as the preclinical stage of OHE and includes a spectrum of cognitive deficits in attention span, psychomotor speed, and working memory (2). The prevalence of MHE varies between 30% and 74% among patients with liver cirrhosis based on evaluations using various diagnostic techniques in different populations (3–7). Moreover, MHE impairs daily function, affects quality of life, and carries high risk of progression to OHE (3–5).

Although the early diagnosis of MHE is critical for subsequent treatment and follow-up, the diagnosis of MHE is complicated by the lack of standard and reliable diagnostic tests that are suitable for clinical practice (8). Neurophysiological and psychometric tests are recommended to diagnose MHE based on the recent guideline by the American Association for the Study of Liver Diseases (AASLD) and the European Association for the Study of the Liver (1,9). The psychometric hepatic encephalopathy score (PHES) and repeatable battery for the assessment of neuropsychological status (RBANS) are the two most extensively used paper-and-pencil psychometric tests for MHE diagnosis (Table 1) (6,10–18). While the PHES and RBANS have been validated in clinical studies, they often require a significant amount of time to administer and, therefore, cannot be rapidly and conveniently performed to diagnose MHE in clinical practice (19). A survey by the AASLD indicated that although most hepatologists believe that MHE should be diagnosed based on test results, only a small number of patients with cirrhosis are, in fact, routinely tested for MHE (20). Therefore, simple and rapid tests that can be administered by medical personnel would increase the frequency of MHE testing during both initial clinical assessment and follow-up.

Table 1.

Diagnostic and screening methods of MHE

Tests	Tested domains	Time required (min)	Influence factor	Advantages	Disadvantages
Paper-and-pencil test PHES	Psychomotor speed, visual perception, visuospatial orientation, visuomotor ability, and attention	20	Age, education, and culture	Gold standard for MHE diagnosis validated internationally	Learning effects
RBANS	Psychomotor speed, anterograde memory, and working memory	30	Age, education, and culture	Rigorous population-based standardization values	Required further validation in randomized controlled trials
Computer-aided test Scan test	Working memory, vigilance, and attention	15	Age and education	Simple administration	Learning effects
CDR assessment battery	Reaction time, memory, and recognition	15	Age, education, and culture	Appreciable test-retest reliability	Learning effects
ICT	Response inhibition, working memory, vigilance, and attention	15	Age, education, and culture	Simple administration, higher sensitivity/specificity, and appreciable test-retest reliability	Not suitable for non-English-speaking patients
EncephalApp Stroop App	Psychomotor speed and cognitive alertness	10	Age, education, and culture	Rapid and simple administration, and good test-retest reliability	Should be familiar with iPhone/iPad
CFF	Visual discrimination and general arousal	10	Age	Simple and easily performed and no learning effects	Not suitable for red-green blindness and visual impairment

PHES: psychometric hepatic encephalopathy score; RBANS: repeatable battery for the assessment of neuropsychological status; CDR: Cognitive Drug Research; ICT: inhibitory control test; CFF: critical flicker frequency; MHE: minimal hepatic encephalopathy

Several recent studies have indicated that psychometric tests can be computerized for simplification and administered in the clinic within a few minutes (5,21–23). Several computer-aided psychometric tests are available for screening MHE in patients with cirrhosis, including the scan test, Cognitive Drug Research (CDR) assessment battery, inhibitory control test (ICT), EncephalApp Stroop App, and critical flicker frequency (CFF) (Table 1, 2) (5,21–23). The instructions for psychometric testing do not differ between paper-and-pencil tests and computer-aided tests, and testing results are reviewed by medical professionals to reach achieve a substantial conclusion. A preferable strategy for MHE diagnosis is to initially screen patients with cirrhosis who may exhibit MHE using rapid, convenient and highly sensitive computer-aided psychometric tests and then conduct the PHES for further validation (24). In this review, we summarize the advances from paper-and-pencil to computer-aided psychometric tests for MHE screening.

Table 2.

Clinical trials of computer-aided psychometric tests for screening MHE

Computer-aided psychometric tests	Study	Nationality	No. of patients with cirrhosis	Reference standard	Sensitivity (%)	Specificity (%)	AUC
Scan test	Amodio et al. (42)	Italy	32	EEG	36	90	-
CDR assessment battery	Mardini et al. (22)	United Kingdom	89	PHES	86.4	81	0.917
ICT	Bajaj et al. (5)	United States	50	SPT	90	90	0.958
	Bajaj et al. (47)	United States	136	SPT	88	77	0.902
	Gupta et al. (29)	India	200	PHES	92.6	78.5	0.855
	Taneja et al. (54)	India	102	PHES	78	65.6	0.735
	Sharma et al. (51)	India	50	PHES	88.5	56	0.695
	Amodio et al. (53)	Italy	75	PHES	82.5	72.6	0.810
EncephalApp Stroop App	Bajaj et al. (23)	United States	125	SPT	78	90	0.890
	Bajaj et al. (59)	United States	167	SPT	89.1	82.1	0.910
	Allampati et al. (60)	United States	437	PHES	72	88	0.800
CFF	Romero et al. (66)	Spain	114	PHES	77	73	0.754
	Dhiman et al. (65)	India	100	PHES	35	92	-
	Kircheis et al. (64)	Germany	148	PHES	97	100	-

MHE: minimal hepatic encephalopathy; CDR: Cognitive Drug Research system; ICT: inhibitory control test; EEG: electroencephalography; SPT: standard psychometric tests; PHES: psychometric hepatic encephalopathy score; CFF: critical flicker frequency; AUC: area under the receiving operative curve

Paper-and-pencil psychometric tests

Psychometric hepatic encephalopathy score

Among different paper-and-pencil psychometric tests, the PHES is currently recommended internationally as the gold standard for the diagnosis of MHE (1,25,26). It was specifically designed for the diagnosis of MHE and is composed of the following five tests: number connect test (NCT)-A, NCT-B, serial dotting test, line tracing test, and digit symbol test. It evaluates motor speed, motor accuracy, concentration, attention, visual perception, visual construction, and memory, which are related to most of the neuropsychological impairments of MHE (27). It has been validated internationally in Germany, Italy, Spain, India, Korea and China, and local population-based normative values have been established to reduce the bias induced by cultural differences (Table 3) (6,11–13,28,29). It has been proven to be of diagnostic as well as prognostic use. It can predict both occurrence of OHE and survival in patients with MHE, as well as identify patients with cirrhosis who are at risk of falling within 1 year after the testing (14,28,30,31).

Table 3.

Validation of the PHES for the diagnosis of MHE

Study	Nationality	No. of patients with cirrhosis	Cut-off value	MHE (%)
Wunsch et al. (10)	Poland	50	≤-5	22
Li et al. (11)	China	53	≤-4	49.1
Ampuero et al. (12)	Spain	112	≤-4	25.9
Riggio et al. (28)	Italy	79	≤-4	57
Dhiman et al. (14)	India	100	≤-5	48
Seo et al. (6)	Korea	160	≤-5	25.6
Duarte-Rojo et al. (19)	Mexico	84	≤-4	15
Coskun et al. (15)	Turkey	60	≤-4	31.6
Badea et al. (16)	Romania	106	≤-3	34.7
Kircheis et al. (13)	Germany	448	≤-4	54.1

MHE: minimal hepatic encephalopathy; PHES: psychometric hepatic encephalopathy score

Univariate analysis has identified significant age and education effects on PHES scores (32). These effects, however, are controlled to a certain extent by the large normative data set obtained from healthy volunteers, minimizing the chance for error. In addition, some of the subtests showed significant learning effects, which are balanced in the composite score of the whole battery. A previous study demonstrated that learning effects after the first application of the PHES wash out upon repeated testing at 6 months (33). Recent findings of neurological deficits among patients with cirrhosis classified as not having MHE by the PHES should be noted, indicating the need for more sensitive tests (34). Therefore, although the PHES has been very useful to homogenize the assessment of neurological changes in patients with cirrhosis, it may be not be sufficiently sensitive to detect early neurological changes that may be relevant to the clinical course of patients with cirrhosis (35,36).

Repeatable battery for the assessment of neuropsychological status

The RBANS has been used as an alternative test to the PHES (Pearson Education, Inc., San Antonio, TX, USA) and has been recommended for the diagnosis of MHE by the International Society for the Study of Hepatic Encephalopathy and Nitrogen Metabolism (25). It assesses anterograde memory, working memory, cognitive processing speed, language, and visuospatial function, which are cognitive domains not influenced in HE (27,37). It has a rigorous population-based standardization and normative values in the United States and has been extensively used in the screening of a variety of cognitive disorders, including stroke, schizophrenia, and Alzheimer’s disease (38–40). Thus far, it has been used for the assessment of cognitive deficits among liver transplant candidates in a few studies. Meyer et al. (17) reported that the RBANS is a relatively quick and reliable method for evaluating cognitive impairments among consecutive outpatients being assessed prior to liver transplantation. Moreover, the RBANS scores strongly correlated with the severity of end-stage liver disease and independently predicted the prognosis in liver transplant candidates (17,18). However, these studies were aimed to assess cognitive dysfunction in patients with liver failure and were not designed to evaluate the sensitivity and specificity of the RBANS for diagnosing MHE. Moreover, the RBANS would require a cultural-matched database of the local population when it is used to screen patients with cirrhosis outside the United States. Therefore, the diagnostic value of the RBANS for MHE required further validation.

Computer-aided psychometric tests

Scan test

The scan test is a computerized digit recognition test based on the Sternberg paradigm (41). It is performed by randomly displaying a series of 72 sorted pairs of numbers for 3 s on a computer screen. Patients are instructed to press the appropriate number on a keyboard if they identify a common digit in the sequence of numbers presented (Figure 1). The mean reaction times and the percentage of errors are recorded, and the results are evaluated using the reaction times weighted by the number of errors (21).

Figure 1.

The ideograph of the scan test. Patients are asked to push the digit on the keyboard depending on whether they identify a common digit in the pairs of numbers on the computer screen

It can assess impairments in sustained attention, altered short-term memory, and reduced motor speed that are related to the clinical manifestations of MHE (41). Furthermore, the Sternberg paradigm, which is the prototype of the scan test, correlates with the score on the number connection test and quantitative electroencephalography (EEG) parameters in patients with cirrhosis without OHE (41). Therefore, the scan test has been used to identify MHE in patients with liver cirrhosis and has been validated in Italy as a reliable psychometric test (21,42). Amodio et al. (21) found that performance scores for the scan test are significantly lower among patients with cirrhosis, and that this test is at least as effective as the number connection test for psychometric assessment of patients with cirrhosis. Moreover, they reported an association between the lower scan test scores and an increased risk of death in 1 year of follow-up, indicating a prognostic value of the scan test for the survival of patients with cirrhosis (21).

Although an initial study found that the reaction times on the scan test are higher among patients with alcoholic cirrhosis than those with non-alcoholic cirrhosis, further studies have suggested that these results are related to the severity of liver cirrhosis and are independent of its etiology (21,42). The scan test, however, was not found to correlate with EEG alterations in patients with cirrhosis, suggesting the possibility that test results might be influenced by age and educational background (42). Therefore, age and education should be considered before administering the scan test. Moreover, this test needs to be validated in other populations after adjusting for age and education.

CDR assessment battery

The CDR assessment battery is a computerized battery of cognitive tests designed by the Cognitive Drug Research Ltd. (Goring-on-Thames, UK). It measures reaction time, memory, and recognition. The task stimuli are presented on a laptop, and subjects provide the correct response using the “YES” and “NO” buttons on a two-button response box, which records both accuracy and reaction time (22).

It is widely used for the evaluation of cognitive impairment in patients with dementia and hepatitis C (43,44). Moreover, it is currently available in the United Kingdom for screening MHE. Mardini et al. (22) reported that the sensitivity and specificity of the CDR assessment battery for screening MHE are 86.4% and 81%, respectively, in patients with cirrhosis, using the PHES used as the gold standard for comparison. In addition, a highly significant correlation between the PHES subtests and the CDR composite scores was identified (22). The CDR assessment battery is considered as a good tool for diagnosing MHE with a reliable association to the PHES and high sensitivity in populations with cirrhosis. The PHES did not reliably evaluate cognitive improvement when patients with cirrhosis were retested after treatment (45). However, the CDR system has been used extensively in clinical trials and has shown appreciable test-retest reliability (46). Similarly, the CDR system may be used to evaluate the cognitive improvement following treatment in patients with cirrhosis with MHE.

Although the CDR assessment battery is simple and convenient to administer, it requires a practice session before evaluation. The learning effect induced by the practice session may influence the reliability and validity of the CDR assessment battery for diagnosing MHE. Moreover, reference data for the CDR assessment battery were collected from a large sample of matched controls in the United Kingdom; however, using the CDR assessment battery to screen populations outside Britain would require an age-, education-, and cultural-matched database of the local population (46).

Inhibitory control test

The ICT can be performed using a laptop and is analyzed using an automatic computerized system that greatly improves the convenience and flexibility of using this test in the clinical setting (47). It consists of a continuous stream of letters presented on a computer screen every 500 ms, with targets (alternating X and Y) and lures (non-target X and Y). Subjects are asked to press a button if, in a random series of letters shown, an X is followed by a Y. The subjects also tend to hit the button for the lures but must inhibit this response (Figure 2) (48). The lure and target response rates and the lure and target reaction times are automatically calculated at the end of the test. A lower lure response, higher target response, and shorter lure and target reaction times indicate good psychometric performance. A modified version of the ICT is available for download (www.hecme.tv).

Figure 2.

The ideograph of the inhibitory control test. A continuous sequence of letters is displayed on the computer screen every 500 msec. The subject is instructed to respond only if an X is preceded by a Y, or a Y is preceded by an X, but responses must be inhibited if an X is followed by an X, or a Y is followed by a Y

It has been used to evaluate cognitive deficits in inhibition, attention span, vigilance, and working memory in patients with schizophrenia and traumatic brain injury (49,50). In addition, it has been validated for MHE screening in the United States and is available in Germany, Italy, and India (47,51–53). Bajaj et al. (5) found that the ICT is simple to administer and has higher sensitivity/specificity and test-retest reliability for screening MHE in the United States than the standard psychometric test (SPT) battery (47). Similarly, in an Indian population, the ICT was reported to have a sensitivity of 92.6% and a specificity of 78.5% for detecting MHE, using the PHES as the reference standard, with excellent test-retest reliability (54). Moreover, these studies have reported that the ICT correlates with the underlying liver disease severity as assessed using the Child-Pugh classification and the Model for End-Stage Liver Disease score, predicting the development of OHE and survival, as well as the PHES (5,54). Moreover, the ICT results improve after lactulose therapy and worsen after a transjugular intrahepatic portosystemic shunt (TIPS), indicating that the ICT can be used to evaluate psychometric alterations that are induced by disease-related therapies (47). Patients with MHE exhibit cognitive impairments that are associated with an increased risk of motor vehicle accidents and traffic violations (55). The ICT score was associated with a significantly higher rate of motor vehicle accidents in a study with a 1-year follow-up and is the most cost-effective MHE diagnostic strategy for preventing motor vehicle accidents (55,56). These results suggest that the ICT could potentially emerge as an alternative to the PHES for screening MHE. Previous studies have demonstrated that the PHES and SPT are influenced by age, whereas the results of the ICT are not affected by age in patients with cirrhosis as well as controls, which will broaden and simplify the applicability of the ICT in the clinical setting (47,57). In addition, there was no difference in psychometric performance on the ICT and SPT between patients with alcoholic and non-alcoholic cirrhosis (47). This finding further increases the applicability of the ICT across different etiologies of chronic liver disease.

In contrast, Taneja et al. (58) found that the ICT is not as useful as the PHES in diagnosing MHE in patients with cirrhosis and that the ICT results do not correlate with the severity of liver cirrhosis or predict survival as reliably as the PHES (58). This finding may be explained by the possibility that the ICT and PHES detect a different spectra of the cognitive deficits observed in patients with MHE. Future studies involving larger sample sizes and longer follow-up are needed to determine the ability of the ICT to predict survival in patients with cirrhosis with MHE. Moreover, in the study of an Italian population by Amodio et al. (53), the ICT was not useful for screening MHE, unless adjusted by target accuracy. This result contradicts the conclusions presented in the previous study by Bajaj et al. (5,47). A possible explanation for this discrepancy is that the familiarity with computers and games differs between Italians and Americans. Another reason for this inconsistent conclusion is that the ICT results were calculated using different parameters and methods across studies, leading to difference in the cut-off values for normal and abnormal results across different populations. Only 12% of the patients with cirrhosis preferred the ICT over the CFF and PHES when patients with cirrhosis were given a questionnaire regarding which tests they would like to repeat to psychometrically evaluate their condition during follow-up (51). A lower preference for repeated ICT evaluation might be due to the fact that some patients are unfamiliar with using a computer. This finding suggests that the results of the ICT are influenced by educational background, as with most psychometric tests. To control for this effect, the ICT scores need to be adjusted to different age ranges and educational backgrounds of the study population. Further, more studies are needed to validate the ICT norms in populations of different age ranges and educational backgrounds. In addition, the ICT involves recognizing English letters; therefore, it should be modified before administration to non-English-speaking patients.

EncephalApp Stroop App

With the popularity of smartphones, blood glucose and blood pressure can be monitored using relevant applications. In 2013, Bajaj et al. (23) developed an application, the EncephalApp Stroop App, for screening MHE that is operated by the iOS system on the iPhone and iPad. The core of this innovative application is the Stroop test, which evaluates psychomotor speed and cognitive alertness by measuring the time required to correctly identify a series of symbols and printed words with different colors (Figure 3). The EncephalApp Stroop App has been translated into several languages, including English, German, and Chinese, and this application and its operational instructions can be easily downloaded (www.encephalapp.com).

Figure 3. a, b.

The ideograph of the EncephalApp Stroop App. (a) Subjects should correctly press the color corresponding to the that of the presented symbols. (b) Subjects should correctly press the color corresponding to the that of the presented word

Bajaj et al. (23) showed that patients with cirrhosis with MHE are significantly slower in performing the Stroop test than the control group, with the EncephalApp Stroop App showing a 78% sensitivity and 90% specificity in a validation cohort of patients with cirrhosis, using the PHES as the gold standard for reference. In addition, they found that the results of the EncephalApp Stroop App are specifically correlated to driving, reaction time, working memory, and psychomotor speed, highlighting and validating its clinical benefits (59). Further, their study showed that the application has good test-retest reliability and is validated for patients with TIPS and hypernatremia (59). In a multicenter study that compared the EncephalApp Stroop App to the PHES and ICT, the EncephalApp Stroop App had good sensitivity (70%–80%) for MHE screening and was predictive of the progression to OHE using the adjusted United States-based population norms (60). These studies show that the EncephalApp Stroop App is easy to operate, quick to explain to patients, and simple to score and evaluate. This App may be an alternative tool to rapidly screen outpatients who require further neurophysiological evaluation for the diagnosis of MHE.

For other psychometric tests, age and educational background can influence the results of the EncephalApp Stroop App. Epidemiological surveys in various countries show that the majority of clinical patients with cirrhosis are not well-educated or old (3,4,6,7). However, Bajaj et al. (23) reported that patients with cirrhosis have a relatively higher educational background, which may limit the generalizability of this application. Moreover, aging-related cognitive impairment, such as Alzheimer’s disease, Parkinson’s disease, or cerebrovascular disease, may affect the reliability of the application. Additionally, linguistic differences, ethnic origin, and familiarity with a smartphone are likely to have confounding effects that should be considered before administering the application (61). Furthermore, the screen size and luminance of diagnostic equipment, such as the iPhone and iPad, vary, which may affect the test results of patients with cirrhosis with visual impairment. Therefore, further studies are required to validate the utility of the EncephalApp Stroop App for screening MHE, controlling for possible confounding factors, including educational background, age, culture, and diagnostic equipment.

Critical flicker frequency

The CFF was designed originally to evaluate visual acuity and to screen for optic nerve lesions for ophthalmological examination (62). Light pulses are presented to a subject in decreasing frequency from 60 to 25 Hz, and the subject has to press a button as soon as the impression of fused light switches to flickering light (63). After a training session, this test is repeated 8–10 times to calculate the mean and standard deviation of flicker frequencies as CFF results.

Critical flicker frequency has the advantages of being performed with a portable device and not being dependent on language, verbal fluency, and numeracy. It has been performed in Germany, Spain, and India (64–66). Kircheis et al. (64) reported that a CFF cut-off of 39 Hz shows a sensitivity of 55% and a specificity of 100% for MHE diagnosis (64). Similarly, Romero-Gómez et al. (66) showed that a CFF threshold of 38 Hz identifies patients with MHE with a sensitivity of 72.4% and a specificity of 77.2% compared with the PHES. Moreover, a meta-analysis of nine studies indicated that CFF has a high summary specificity (79%) and moderate sensitivity (61%) for diagnosing MHE (67). However, Goldbecker et al. (68) found that CFF shows a sensitivity of only 40% in patients with cirrhosis with Grade I HE. The reason of the difference is the influence of age and the etiology of cirrhosis on CFF results. Several previous studies have shown that CFF values decrease by 0.6–0.7 Hz/life decade, and patients with alcoholic cirrhosis have significantly lower CFF than those with cirrhosis due to other etiologies (64,65,68). Thus, a fixed cut-off may induce biased CFF results in elderly patients and patients with alcoholic cirrhosis.

Critical flicker frequency is easily understood by patients with cirrhosis and shows no learning effects in the studies; thus, it is appropriate for follow-up (69). It predicts the first episode of OHE and predicts mortality risk in patients with cirrhosis, with lower CFF values predicting the development of OHE following TIPS (66,70,71). Labenz et al. (72) found that the CFF results are associated with impaired health-related quality of life and sleep quality in patients with cirrhosis (72). However, Lauridsen et al. (73) noted that 11% of the included patients cannot complete the CFF tests because they are unable to pay close attention to the light diode. Furthermore, Romero-Gomez et al. (66) found that nine patients and three controls cannot perform the CFF due to visual impairment. Therefore, the CFF assessment requires intact binocular vision, absence of red-green blindness, and standardization of operating procedures (74).

Despite the limitations, CFF is a simple and easy to perform test for diagnosing MHE in patients with cirrhosis or portosystemic shunting. It is suggested as an adjunct to conventional PHES and can possibly become a replacement screening test if further studies show an improvement in sensitivity and specificity.

In conclusion, computer-aided psychometric tests provide rapid, effective and simple alternatives to paper-and-pencil tests to screen for MHE in clinical practice, reliably identifying patients to undergo further psychometric and neurophysiological evaluation. The advances from paper-and-pencil to computer-aided versions of psychometric tests may not only help hepatologists to screen patients for MHE but, ultimately, also support early treatment of MHE, improving the quality of life and reducing the risk of progression to OHE. Nonetheless, recent studies evaluating the clinical application of computer-aided psychometric tests are inadequate for meta-analysis. Therefore, further studies in different populations are required to achieve a convincing conclusion.