Exploring current smartphone-based cognitive assessments in schizophrenia and bipolar disorder

Abstract

Schizophrenia and bipolar disorder are associated with cognitive deficits that contribute significantly to disability. However, traditional in-lab cognitive assessments are time-consuming and not optimized for remote administration. Recent advancements in smartphone technology enable momentary cognitive assessments in a real-world context. This brief report reviews recent research in momentary cognitive assessments in individuals with schizophrenia and bipolar disorder through reviewing mobile platforms and cognitive assessments studied. A total of 14 experimental articles were reviewed, focusing on cognitive domains including visual working memory, processing speed, executive function, verbal fluency, verbal memory, social cognition, and typing patterns. The review highlights the feasibility of remote cognitive assessment with smartphones, and provides a layout of domains studied in this context, but illustrates a low volume of current research, the need for additional studies, and the potential for innovations like digital phenotyping.

1. Introduction

Schizophrenia and bipolar disorder are persistent diseases that affect 1 % of the global population, and 4.4 % of the U.S. adult population respectively (Bipolar Disorder, no date; Schizophrenia, 2022). Both disorders are also characterized by cognitive dysfunction, particularly in verbal memory, and executive functioning (Krishnadas et al., 2014; Martínez-Arán et al., 2011) that are associated with significant disability, including in vocational performance and interpersonal relationships (Harvey et al., 2012; What Are Bipolar Disorders?, no date). Early identification of cognitive impairment is critical, as it is often the earliest sign of illness and the best predictor of functional outcomes (Kelly et al., 2019; Mucci et al., 2021). While there are numerous well-established measurements used to assess cognition in schizophrenia and bipolar disorder (Fray et al., 1996; Keefe et al., 2004; Keefe et al., 2014; Zheng et al., 2021), they may take up to 90 min to administer, often require specialized training, and must be performed in a lab/hospital setting (Nuechterlein and Green, 2002; Smieszek and Russwinkel, 2013).

Smartphone based cognitive assessments may offer a remote, accessible, and low-cost approach that has the added advantage of capturing related environmental data on when and where the assessment was taken (e.g. at home after a night of poor sleep). These in-vivo assessments could thus provide a more comprehensive assessment of cognition in context and provide complementary data to traditional assessments. (Berrouiguet et al., 2018; Chen et al., 2019; Guimond et al., 2019; Taylor et al., 2016; Training & Certification, no date). Smartphone app-based assessments are poised for high uptake among patients. Studies have consistently shown that a significant percentage of individuals with psychosis own smartphones and express a desire to utilize them to improve their health (Beard et al., 2019; Ben-Zeev et al., 2013; Jonathan et al., 2019; Gitlow et al., 2017).

This brief report aims to examine the present utilization of smartphones in evaluating momentary cognition among individuals with schizophrenia and bipolar disorder. Specifically, we aim to note measurement tools used, the different platforms these measurements are collected on, and how studies are utilizing any related smartphone data to provide environmental or social context to cognitive scores (Fig. 1).

Fig. 1.

The figure depicts the various cognitive tests (represented as circles) available in each app (displayed in the top row) and their corresponding cognitive domains (represented as squares next to the circles).

2. Methods

2.1. Search strategy

Relevant publications, published from 2000 through 2023, using a smartphone application were identified using the PubMed Database. The following keywords were used: ((Bipolar[Title/Abstract]) OR (serious mental illness[Title/Abstract]) OR (psychosis[Title/Abstract]) OR (Schizophreni*[Title/Abstract])) AND ((cogni*[Title/Abstract]) OR(neuropsych*[Title/Abstract])) AND ((smartphone) OR (digital) OR (app) OR (remote*) OR (mobile*)). A total of 493 articles were retrieved and screened using the following inclusion and exclusion criteria. Selected articles were then retrieved for full-text screening and data extraction.

2.2. Inclusion criteria

Studies were included if (1) at least 25 % of participants had a diagnosis of schizophrenia or bipolar disorder, (2) measured cognition, (3) used a smartphone application to assess cognition.

2.3. Exclusion criterion

Studies were excluded if (1) the focus was not on serious mental illness, (2) they used services like SMS or phone calls to conduct EMAs, (3) they were grey literature (e.g. abstracts), (4) they were research proposals/ study protocols.

2.4. Data extraction

The following predetermined variables were extracted from the selected articles: (1) Bibliographical data (authors name, year of publication), (2) Sample size (3) Sample description (4) Length of the study, (5) Smartphone application used, (6) Task completed during the study, (6) Cognitive domain assessed in the study, (8) Current measures used to validate the task, and (9) use of related digital phenotyping / sensor data in conjunction with cognition.

3. Results

Table 1, directly below, lists the 14 experimental articles selected for review. None of the studies reported results using any digital phenotyping data / related sensors (eg geolocation, accelerometer, screen state) to provide context to any of the smartphone cognitive scores.

Table 1.

The table lists the 14 experimental articles selected for review. It contains bibliographical data (authors name, year of publication), smartphone application used, cognitive domain assessed in the study, task completed during the study, current measure used to validate, sample size, sample age range, length of the study.

Authors	App Name	Cognitive domain	Task	Validated against	Sample size	Sample age	Length	Passive Sensing
	NeuroUX	Visual Working Memory Processing Speed Visual Working Memory (primary); Processing Speed (secondary) Visual Working Memory	Memory Matrix Matching Pair Odd One Out CopyKat.	The NIH Toolbox Cognition Battery	BD: n = 46 HC: n = 20	18–65	14 days	No
Depp et al., 2021	NeuroUX	Social Cognition/ Emotion Recognition	Mobile Ecological Test of Emotion Recognition (METER)	Bell-Lysaker Emotion Recognition Task (BLERT)	SZ: n = 34 SZA: n = 35 BD: n = 15 MDD: n = 2	18–65	11 days	No
Dupuy et al., 2018	Unspecified app	Executive Function/ Inhibitory Control Verbal Fluency	Mobile Color Word Inference Letter Word Generation	Stroop Test, Letter Word Generation	SZ: n = 22 HC: n = 27	19–52	7 days	No
Dupuy et al., 2022	Unspecified app	Executive Function/ Inhibitory Control Verbal Fluency	Mobile Color Word Interference Letter Word Generation	Stroop Test Letter Word Generation	SZ: n = 33 HC: n = 42	23–43	7 days	No
Hays et al., 2020	mindLAMP	Processing Speed Executive Function/ Cognitive Flexibility	Jewels Trail A Jewels Trail B	No Validation	SZ: n = 42 HC: n = 43	18–52	90 days	No
Henson et al., 2021	mindLAMP	Processing Speed Executive Function/ Cognitive Flexibility	Jewels Trail A, Jewels Trail B	No Validation	SZ n = 54 HC n = 34	22–54	90 days	No
Liu et al., 2019	mindLAMP	Processing Speed Executive Function/ Cognitive Flexibility	Jewels Trail A, Jewels Trail B	No Validation	SZ: n = 18 HC: n = 17	21–31	84 days	No
Moore et al., 2022	NeuroUX	Processing Speed Visual Working Memory Visual Working Memory (primary); Processing Speed (secondary) Verbal Memory Processing Speed Executive Function/ Response Inhibition Visual Working Memory	Matching Pair Memory Matrix Odd One Out Mobile Variable Difficulty List Memory Test (VLMT) Quick Tap 1 Quick Tap 2 CopyKat	The NIH Toolbox Cognition Battery	BD: n = 45 HC: n = 21	18–65	14 days	No
Parrish et al., 2021	NeuroUX	Verbal Memory	Mobile Variable Difficulty List Memory Test (VLMT)	Hopkins Verbal Learning Test	SZ/SZA: n = 98 BD: n = 70	18–65	30 days	No
Parrish et al., 2022	NeuroUX	Social Cognition/ Emotion Recognition	Mobile Ecological Test of Emotion Recognition (METER)	Bell-Lysaker Emotion Recognition Task (BLERT)	BD:n = 31 SZ:n = 44 SZA: n = 57 MDD: n = 29	18–65	10 days	No
Ross et al., 2021	BiAffect	Executive Function/ Cognitive Flexibility Typing Patterns	digital Trail Making Test Part B (dTMT-B) Keystroke Pattern	Trails Making Test B No Validation	BD: n = 11 HC: n = 8	36–58	56 days	No
Shvetz et al., 2021	mindLAMP	Processing Speed, Executive Function/ Cognitive Flexibility	Jewels Trail A, Jewels Trail B	Trails Making test A Trails Making Test B	SZ: n = 32	18–65	Up to 90 days	No
Tao et al., 2023	ReMind	Visual Working Memory Verbal Memory	Visual Patterns Test Letter-Number Sequencing Test	Visual Patterns Test Letter-Number Sequencing Test	SZ: n = 110	18–55	365 days	No
Zulueta et al., 2021	BiAffect	Typing Patterns	Keystroke Pattern	No Validation	BD: n = 227 HC: n = 117	18–88	84 days	No

For the sample size, SZ = Schizophrenia, SZA = Schizoaffective Disorders, BD = Bipolar Disorder, HC = Healthy Controls.

3.1. Visual working memory

Visual working memory was measured using a total of four tests administered on two different apps, across three different studies out of the fourteen reviewed. The tests assessed participants' ability to remember square locations, along with performance on a digital version of the n-back task, in which participants must remember an item from a listed sequence n items ago, and a digital version of the Visual Patterns Test, in which participants must recall the layout of differently filled square grids.

3.2. Processing speed

Processing speed was measured using a total of four tests administered on two different apps. This domain was investigated in six out of the fourteen articles. The assessments consist of a selecting matching color pairs on a grid, selection of a non-matching symbol from a group of six, tapping of a target image on a blank square as quickly as possible, and digital versions of the Trail Making Test Part A, in which participants must tap a set of numbered ‘Jewels’ placed about the screen in numeric order.

3.3. Executive function

Executive functioning was measured using a total of four tests administered across four different apps, one of which was unspecified. The domain was investigated in eight out of the fourteen studies, focusing on two subdomains: response inhibition and cognitive flexibility. The response inhibition task, Color Word Inference, was based on the Stroop test. Cognitive flexibility was assessed using smartphone versions of the Trail Making Test Part B, in which participants must tap a set of numbered icons about the screen in their numeric order, sometimes alternating between two ordered sets.

3.4. Verbal fluency

Verbal Fluency was measured using one test, administered on one unspecified app, in two out of the fourteen studies. It is measured using the Letter Word Generation Task. Participants are given a letter, such as the letter ‘E,’ and are asked to name as many words as they can think of that start with that letter.

3.5. Verbal memory

Verbal Memory has been measured in two assessments, across two apps, in three out of the fourteen studies. In the first assessment, the mobile Variable Difficulty List Memory Test (VLMT), participants are presented with a list of six-twelve words and given 30 s to learn them, after which they must identify from other random words the originals shown In the Letter Number Sequencing test, participants are given a random sequence of letter-number combinations. They are instructed to speak the numbers in ascending order and the letters in alphabetical order.

3.6. Social cognition

Social cognition has been measured using one assessment, administered via one app, across two out of the fourteen studies, and only targeting the subdomain of emotion recognition. Social cognition has been exclusively examined using a web-based platform rather than a native app. The assessment used is the Mobile Ecological Test of Emotion Recognition (METER). During the task, participants were presented with 10 faces, each displaying one of five emotions: happiness, sadness, anger, fear, or no emotion. Participants were asked to identify the emotion displayed from the set of options.

3.7. Keystroke patterns

Keystroke patterns have been measured using one measurement, administered on one app, in two out of the fourteen studies. Keystroke patterns are measured with the ‘BiAffect keyboard’. The assessment examined variations in keystroke patterns throughout the day, with participants using their phones as they typically would.

4. Discussion

The review revealed that there are already many apps for assessing cognition in schizophrenia and bipolar disorder and each appears feasible in terms of patient acceptability and engagement.

Published studies did not consistently provide data on the psychometric quality (i.e., test-retest reliability, discriminant validity) of cognitive scores obtained with these app-based assessments. The absence of estimates of reliability and validity is a gap at this stage of smartphone assessment development. Further, it is unclear if the psychometric models (e.g., classical test theory) used to adjudge the reliability of self-report personality measures, clinical rating scales, and classic cognitive measures are adaptable to determining the reliability of assessments of momentary cognition.

Although passive sensors have shown utility in assessing mild cognitive impairment, their full potential remains untapped (Berrouiguet et al., 2018; Chen et al., 2019), and no reviewed studies utilized them (see Table 1). This is critical as while cognition is considered very stable, and the choice of long retest intervals such as 4 weeks in the MATRICS battery's validation study (Nuechterlein et al., 2008) are in keeping with this assumption, smartphone-assessed cognition may capture performance that could be less stable than initially assumed and multidetermined by interactive environmental factors. Further, our findings indicate a predominant focus of previous research on visual working memory and executive function, with limited literature available on verbal fluency and social cognition.

In conclusion, the use of smartphones for momentary cognitive assessment in individuals with schizophrenia and bipolar disorder holds promise. However, to fully capitalize on these tools, there must be a renewed emphasis on domains of cognition beyond executive functioning and visual working memory, along with consistent efforts to validate the new instruments used. Additionally, maximizing the potential of passive sensors is crucial as findings around the current limitations of app-based cognitive assessment highlight room for innovation and enhancing current forms of enhancing measurement.

CRediT authorship contribution statement

Erlend Lane: Writing – review & editing, Visualization, Validation, Investigation. Devayani Joshi: Writing – review & editing, Writing – original draft, Methodology, Investigation. Synthia Guimond: Writing – review & editing, Writing – original draft, Methodology, Investigation. Raeanne Moore: Writing – review & editing, Writing – original draft, Methodology, Investigation. Anthony O. Ahmed: Writing – review & editing, Writing – original draft, Methodology, Investigation. Olusola Ajilore: Writing – review & editing, Writing – original draft, Methodology, Investigation. John Torous: Writing – review & editing, Writing – original draft, Supervision, Project administration, Methodology, Investigation.

Declaration of competing interest

Raeanne Moore and Olusola Ajilore are founders of Keywise AI. None of the other authors declare any competing interests.