IC3 protocol: a longitudinal observational study of cognition after stroke using novel digital health technology

Abstract

Introduction

Stroke is a major cause of death and disability worldwide, frequently resulting in persistent cognitive deficits among survivors. These deficits negatively impact recovery and therapy engagement, and their treatment is consistently rated as high priority by stakeholders and clinicians. Although clinical guidelines endorse cognitive screening for poststroke management, there is currently no gold-standard approach for identifying cognitive deficits after stroke, and clinical stroke services lack the capacity for long-term cognitive monitoring and care. Currently, available assessment tools are either not stroke-specific, not in-depth or lack scalability, leading to heterogeneity in patient assessments.

Methods and analysis

To address these challenges, a cost-effective, scalable and comprehensive screening tool is needed to provide a stroke-specific assessment of cognition. The current study presents such a novel digital tool, the Imperial Comprehensive Cognitive Assessment in Cerebrovascular Disease (IC3), designed to detect both domain-general and domain-specific cognitive deficits in patients after stroke with minimal input from a health professional. To ensure its reliability, we will use multiple validation approaches, and aim to recruit a large normative sample of age-matched, gender-matched and education-matched UK-based controls. Moreover, the IC3 assessment will be integrated within a larger prospective observational longitudinal clinical trial, where poststroke cognition will be examined in tandem with brain imaging and blood biomarkers to identify novel multimodal biomarkers of recovery after stroke. This study will enable deeper cognitive phenotyping of patients at a large scale, while identifying those with highest risk of progressive cognitive decline, as well as those with greatest potential for recovery.

Ethics and dissemination

This study has been approved by South West—Frenchay Research Ethics Committee (IRAS 299333) and authorised by the UK’s Health Research Authority. Results from the study will be disseminated at conferences and within peer-reviewed journals.

Trial registration number

NCT05885295. Stage: Pre-results.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• A major methodological strength of the study involves the development of the first digital screening technology that allows remote monitoring of stroke-specific cognitive deficits in a manner that is cost-effective and suitable for large-scale population-based investigations.

• Through its digital and conventional assessments, the current study provides a scalable yet deep cognitive phenotyping of both domain-specific and domain-general impairments, longitudinally mapping poststroke recovery trajectory.

• A particular strength of the study is the comprehensive collection of multimodal data (behaviour, brain imaging and blood) that allows the development of novel quantifiable biomarkers of poststroke recovery.

• A limitation of the digital assessment is that it is designed to target stroke survivors with mild to moderate cognitive impairment, thus likely will have limited applicability in patients with severe deficits.

• While the use of technology to administer unsupervised cognitive assessments at a large scale facilitates improved diagnostic and monitoring capabilities, it may also lead to systematic exclusion of those with limited access to technology.

Introduction

Stroke is a leading cause of death and disability globally, with frequent cognitive sequalae affecting three-quarters of survivors.¹ The spectrum of cognitive deficits includes background ‘vascular cognitive impairment’ and more domain-specific deficits, such as aphasia which affects one-third of patients.² Collectively, these impairments negatively impact poststroke recovery and engagement with therapy.³ Furthermore, there is a high failure rate of poststroke cognitive rehabilitation trials, due to our poor understanding of individual patient-level mechanisms of poststroke cognitive recovery and the ensuing inadequate stratification of both treatment responders and non-responders in the same clinical trials. It is, therefore, important to understand the factors that affect cognitive recovery poststroke, a topic increasingly recognised by funders and stakeholders as a high priority research area.^4–6

Although clinical guidelines have been recently updated to recommend cognitive screening for poststroke management, there is currently no gold-standard approach for identifying cognitive deficits after stroke.⁷ Clinicians and researchers often select cognitive tests depending on preference, familiarity and availability, leaving little prospect for generalisability of their findings. Moreover, clinical stroke services do not have the capacity to provide long-term cognitive monitoring and care for stroke survivors who have unpredictable recovery trajectories.^{3 8} This challenge is further compounded by the lack of cost-effective cognitive tools suitable for long-term monitoring, and the reliance of such monitoring on resource-stretched healthcare professionals.

Availability of a cost-effective, reliable and comprehensive screening tool that provides a stroke-specific deep phenotyping of cognition would be a game-changer. In the clinical setting, it would improve the detection rate and monitoring of cognitive deficits poststroke, and in the research setting, it would facilitate much needed large-scale population-based mechanistic studies aimed at understanding the mechanisms of cognitive recovery. Here, we present such a novel digital adaptive tool: The Imperial Comprehensive Cognitive Assessment in Cerebrovascular Disease (IC3). IC3 is a digital assessment designed to require minimal input from a clinician in detecting both domain-general and domain-specific cognitive deficits in patients after stroke. To ensure its reliability, several validation studies will be conducted against established clinical screening tools, and normative samples will be computed using representative age-matched, gender-matched and education-matched UK-based controls.

In line with recent efforts in identifying biomarkers of poststroke recovery,⁹ the IC3 tool will be integrated within a larger prospective longitudinal clinical study, where poststroke cognition will be examined in tandem with brain imaging and novel blood biomarkers. This study is entitled ‘understanding factors affecting cognitive function in cerebrovascular disease’ and will hereafter be referred to as the ‘main study’.

Methods and analysis

Aims

1. Develop an adaptive, scalable, self-administered, digital comprehensive cognitive screening tool to detect cognitive impairments in patients with cerebrovascular disease. The IC3 will:

   1. Detect both domain-general (eg, executive function) and domain-specific (eg aphasia, apraxia, neglect) deficits poststroke.¹⁰

   2. Allow monitoring to occur at scale, in a cost-efficient manner, as test administration can occur independent of trained professionals.

   3. Minimise the effects of neglect or aphasia on cognitive assessments.

   4. Have high test–retest reliability for repeated-testing longitudinally.

2. Validate the IC3 against commonly used cognitive screening tools.

3. Use IC3 in a cohort of stroke survivors to map the cognitive recovery trajectory over the course of the first year after stroke.

4. Identify novel biomarkers of cognitive outcome (at 1 year) and recovery trajectory through the main study using:

   1. Demographics, and comorbid physical, neuropsychological and socioeconomic factors.

   2. Structural and functional MRI brain imaging, to assess metrics of cerebrovascular disease load, stroke lesion topology and brain network dynamics.

   3. Blood biomarkers of Alzheimer’s disease (phosphorylated tau forms and amyloid β42/40 ratio), neuroaxonal injury (neurofilament light, NFL and brain-derived tau) and astrocytic activation (glial fibrillary acidic protein, GFAP). The study will characterise the time course of these biomarkers over the first year after stroke and relate these to MRI measures of axonal injury and brain atrophy, as well as cognitive and functional outcomes.

IC3 assessment design and development

Cognitive tests

The IC3 assessment covers 22 short tasks, spanning a wide range of cognitive domains (table 1), followed by several clinically validated questionnaires, designed to be completed in under 60–70 min. IC3 has in-built automated break reminders every 20 min with the additional option to take an unscheduled break at any time. IC3 is available via a web-browser on any modern device (smartphone, tablet, computer/laptop) by clicking on a link created by the study team. The IC3 test is implemented via Cognitron, a state-of-the art platform for remote neuropsychological testing rapidly being adopted by large scale population studies both in the UK and internationally.¹¹

Table 1.

Tasks tested in the IC3 and their associated cognitive domain

Cognitive domain			Task name
Primary	Secondary	Tertiary
Attention	Spatial ability	Space	Pear cancellation (visual-spatial ability)
		Object
	Auditory sustained attention		Auditory attention task
	Visual sustained attention	Simple upper-limb movement	Simple reaction time
Executive	Task switching and attention		Non-verbal trail making (task switching)
	Reasoning	Mental shifting and flexibility	Rule finding
			Odd one out (pattern recognition)
		Spatial planning	Blocks
Language	Speech production	Phonology and semantics	Naming
		Phonology	Repetition
		Phonology, fluency and semantics	Spoken picture description
		Reading	Reading
	Semantics	Verbal semantics	Semantic judgement
	Comprehension (word-picture or word-sentence matching)	Auditory and orthographic processing	Language comprehension
Memory	Long term		Orientation
	Short term	Verbal working memory	Digit span
		Visuo-spatial working memory	Spatial span
		Visuospatial memory	Paired sssociates learning
	Episodic	Visuospatial	Task recall
		Verbal	Target word recall from auditory attention test
No skills			Graded calculation
Praxis	Ideomotor action semantics		Gesture recognition
Motor ability	Left-right upper-limb movement	Attention	Choice reaction time (click on left or right side of the screen)
	Multi-directional upper-limb movement		Motor control (click on moving target)

IC3, Imperial Comprehensive Cognitive Assessment in Cerebrovascular Disease.

The digital nature of the IC3 affords scalability in cognitive monitoring by being usable in both the clinical setting and home environment in the absence of a trained clinician. As well as reducing healthcare costs, this feature promotes accessibility of the assessment for physically disabled patients in whom attendance to healthcare or research setting is difficult. Compared with pen-and-paper tests, the IC3 test administration is standardised, more detailed response metrics per individual tests are provided (eg, accuracy, reaction time and trial-by-trial variability) and real-time patient scores are calculated automatically against a large age-matched, education-matched and sex-matched control sample. The IC3 specifications are summarised in figure 1.

Figure 1.

Infographic outlining the main specifications of the IC3 digital cognitive testing. IC3, Imperial Comprehensive Cognitive Assessment in Cerebrovascular Disease.

Demographic and neuropsychiatric questionnaires

A number of health questionnaires and modified versions of relevant clinically validated questionnaires are completed by the patient or carers, including Apathy Evaluation Scale,⁷ Fatigue Severity Scale,¹² Geriatric Depression Scale,¹³ Instrumental Activities of Daily Living (see table 2). On completion of the cognitive tests, the participants also fill in modified versions of relevant clinically validated questionnaires, including Modified short version of Apathy Evaluation Scale,⁷ Modified short version of Fatigue Severity Scale,¹² Short form of the Geriatric Depression Scale,¹³ Instrumental Activities of Daily Living.¹⁴

Table 2.

Questionnaire items included in the IC3

	Range/categories
Age	0–100
Biological sex	Male/female
Years of education	Forced choice answers scaled to 0–21
Handedness	Likert scale ranging from 1 to 5, where 1 is right handed, 3 is ambidextrous and 5 is left handed
Hand used for making responses	Left/right
Hand impairments	Yes/no
English proficiency	Native, excellent but non-native, moderate level, low level
Ethnicity	White, Asian, black, Arab, mixed, other
Occupational status	Unemployed, long-term sick, retired, student, self-employed, employed
Annual income	£0–£100 000+
Existence of stroke risk factors	Diabetes, high-blood pressure, high cholesterol, heart disease, kidney disease, alcohol dependency, over-weight, smoker, ex-smoker
Existence of medical conditions (followed by more probing questions if an option is selected)	Stroke, other neurological problems, depression, anxiety
If participant indicates a history of stroke
How long ago was the stroke?	Date/time
How many strokes	0–10
Type of stroke	Ischaemic, haemorrhagic, Transient Ischaemic Attack, subarachnoid haemorrhage, I don’t know
Problems caused by the stroke	Language, memory, concentration, movement of the right arm/leg, movement of the left arm/leg, vision, other
Movement	Difficulties in standing, sitting and in sit-to-stand

IC3, Imperial Comprehensive Cognitive Assessment in Cerebrovascular Disease.

At the end of the assessment, the participant will be given a graphical, easy-to-understand diagram that highlights their performance against normative data (figure 2). The normative data will be based on single time point data collected remotely from 5000 healthy controls. For tasks with normally distributed data, mild, moderate and severe impairment will be classed using suggested cut-offs of −1.5 to –2.0 and −2.5 SD below the mean, respectively. For tasks with ceiling effects, 90th, 95th and 99th percentiles are the suggested cut-offs.

Figure 2.

Example of cognitive summary report. Patient’s performance (in purple) is outlined against a normative group (in grey), corrected for device, age, gender and education. Each interval spacing represents 0.5 SD units (SD) such that this participant’s language comprehension performance was 1.5 SD below the mean normative performance. On the periphery of the spider diagram, the percentile score is shown for each task in purple, alongside a graded level of impairment.

Main IC3 study design

This prospective observational longitudinal study aims to recruit 300 patients with acute stroke and obtain repeated measures of cognition and psychosocial status at baseline, 3 months, 6 months and 12 months postictus. IC3 is currently a single-site study at Imperial College Healthcare NHS Trust, England, and commenced recruitment in February 2022. In addition to the cognitive monitoring, all patients will be invited to take part in MRI brain imaging and blood biomarker substudies (figure 3; see substudies for more details).

Figure 3.

(A) Overview of the main IC3 study and data collection. (B) Patient recruitment progress between February 2022 and March 2023. Cognitive assessments will be performed using the IC3 online assessment. Additional substudies will include blood biomarkers and MRI brain imaging. The 1-year follow-up is expected to take place between 12 months and 24 months poststroke.C3, Imperial Comprehensive Cognitive Assessment in Cerebrovascular Disease.

IC3 validation substudy

A minimum of 100 patients will undergo clinically validated pen-and-paper cognitive screens used routinely in clinical practice including the Montreal Cognitive Assessment (MOCA),¹⁵ star cancellation task¹⁶ and/or the Oxford Cognitive Screen (OCS). Sensitivity, specificity analyses, as well as convergence and divergence analyses will be performed. Test–retest reliability analyses will estimate the impact of learning effects on performance, and compare remote administration with in-person delivery of the assessment.

MRI substudy

Clinically acquired brain imaging at the time of the acute stroke (including fluid-attenuated inversion recovery (FLAIR) and diffusion-weighted imaging sequences) will be obtained. All 300 patients will be invited to undergo additional brain MRI imaging at 3 and 12 months using a Siemens Verio 3T scanner at the Imperial College Clinical Imaging Facility. MRI measures will be used as prognostic indicators for cognitive trajectory. MRI data will be collected from additional 50 age-matched control participants for comparison. The following sequences will be acquired at each session over 60–90 min (breaks included):

1. Structural MRI: T1 (1 mm³ resolution for volumetric analyses of lesion volume and brain atrophy); susceptibility weighted imaging (0.8×0.8×3.0 mm³ for identification of microbleeds); T2 FLAIR (1 mm³ for detecting white matter hyperintensity volume (WMHV)); and neurite orientation dispersion and density imaging (NODDI, an advanced multishell diffusion MRI with 90 directions and 9 b₀ images) to assess axonal integrity and tissue microstructure. Lesions will be defined using established standards.¹⁷

2. Functional MRI: Resting-state and breath-hold paradigm will measure dynamic regional brain activity and vascular reactivity, respectively.¹⁸

Blood biomarkers substudy

Blood samples will be obtained at baseline (0–15 days poststroke), and at 3, 6 and 12 months from 200 patients. Two EDTA (plasma) and two Serum-Seperating Tube samples will be collected. Sample processing involves centrifugation at 2000 g for 10 min at 4°C, followed by secure storage at −80°C. Additional single PAXgene sample will be collected at one time point for DNA storage for future analysis of apolipoprotein E genotype and polygenic risk score measures. The patient’s identity is kept confidential. The primary plasma biomarker of interest will be NFL.¹⁹ Additional markers including GFAP (for astrocytic activation²⁰), brain-derived τ (a novel marker for neuronal injury²¹), phosphorylated τ forms and Aβ42/40 ratio (both biomarkers for Alzheimer’s pathologies²²) will be quantified. Testing will be performed at University College London via a Quanterix Simoa analyser to provide ultrasensitive measurement of concentrations.

Patient and public involvement statement

The IC3 assessment was developed and refined during a 6 months iterative process of design modifications based on patient and user feedback. We conducted feedback sessions with stroke survivors part-way through the development of our assessment to ensure that (1) our cognitive assessments are intuitive, (2) the instructions are comprehensible, (3) patients can complete the assessment with minimum supervision and (4) the burden of assessments are acceptable. Based on these sessions, we made several improvements to the design of the assessment, primarily focusing on improving the user interface, user experience and the instructions. We also involved multiple age-matched members of the public, without a diagnosis of stroke, to ensure that our design changes were effective and to identify any issues with performing the assessment remotely. Finally, we disseminated the study design through local public outreach events (eg, educational events for older adults, health advocacy charities) and incorporated the received feedback in the end product.

Study population

Participant identification, recruitment

Online supplemental table 1 outlines the patient inclusion and exclusion criteria. Patients will be recruited from the Imperial College Healthcare NHS Trust by study personnel and the clinical team. Our goal is to recruit patients as early after stroke as is practical, and ideally within 15 days to facilitate acute blood biomarker assessment. Control participants will be aged >18 and without history of neurological or severe psychiatric illness.

Outcome measures

The primary outcome measure for cognition will be the IC3-derived accuracy measures for each primary cognitive domain listed in table 1 at 1-year post ictus. Secondary outcome measures will be (a) MOCA score,¹⁵ (b) functional measures including National Institute of Health Stroke Scale,²³ Modified Rankin Scale,²⁴ Instrumental Activities of Daily Living¹⁴ and (c) psychological measures via the Geriatric Depression Scale¹³ and Apathy Evaluation Scale.⁷

Data analysis plan

Cognition

Dimensionality reduction techniques, such as principal component analysis, graph-based methods and hierarchical clustering will examine the inter-relationship between cognitive domains and will be used to derive latent variables. These will be used to relate to blood and imaging biomarkers. Such cognitive profiling will also form the basis for future real-time optimisation of the IC3 task presentation based on patients’ performance.

To assess the trajectories of functional, clinical and psychological outcome measures, we will employ linear mixed effect models. These models will include cognitive measures as the main predictors, together with standard covariates such as age, sex, lesion and initial cognitive deficit.

Blood biomarkers

NFL, GFAP, amyloid and tau biomarkers will be analysed to describe their dynamics after acute stroke and to compare differences between groups. Additionally, longitudinal changes in these biomarkers will be assessed within patients using non-parametric tests, where appropriate. The relationship between blood biomarker levels and continuous outcome measures at 1 year (eg, cognitive scores) will be investigated using mixed-effects linear regression. For binarised outcome measures (eg, favourable or unfavourable MRS scores), logistic regression will be employed.

Brain imaging

The following analyses will be performed to derive imaging predictors of cognition and relate to blood biomarker levels: (1) WMHV evaluated using FSL Brain Intensity Abnormality Classification Algorithm package (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/BIANCA); (2) stroke lesion topology manually delineated on FLAIR images; (3) NODDI measures of axonal injury and tissue microstructure (fractional anisotropy, neural density and orientation dispersion index) analysed in NODDI MATLAB Toolbox; (4) Cortical thickness measured using Freesurfer; (5) vascular reactivity analysis¹⁸ and (6) Resting-state fMRI analyses of the effect of stroke on functional brain networks. These measures will be compared between groups. Longitudinally, each patient will act as their own ‘control’ in linear regressions accounting for age, sex, initial stroke severity, lesion volume and time. Finally, structural equation modelling will be used to determine the relationship between demographics, comorbidities, blood and brain imaging biomarkers and cognitive and functional outcome measures.

Sample size

The primary blood biomarker predictor is NFL. Based on preliminary reported group differences,¹⁹ we expect 140 patients at baseline, and 60 patients at 3-months poststroke (when peak NFL is expected to occur), will be able to detect group differences in NFL levels between patients and controls (Wilcoxon Mann Whitney test, Cohen’s d=0.38 and 0.58, respectively, alpha=0.05, power=0.8, one tailed, allocation ratio 2:1). A much smaller sample size of 56 will be enough to detect correlations between NFL and time of sampling of the acute blood: (r²=0.10,¹⁹ alpha=0.05, power=0.8, one tailed).

The primary imaging outcome measure is WMHV. A sample size of 80 will be sufficiently powered to detect correlation between NFL and WMHV based on preliminary findings²⁵ (r=0.32, alpha 0.05, power 0.95). A sample size of N=70 will be sufficiently powered to detect the relationship between peak plasma NFL levels and binary functional outcomes (based on similar effects observed after acute brain injury (R²=0.28,²⁶ α=0.05, power=0.9). Expecting a 20%–25% lost to follow-up rate (due to patient factors and technical factors), the stated sample size of 200 in blood biomarker substudy is sufficiently powered to detect all the above-mentioned effects.

A secondary blood biomarker predictor is raised phosphorylated tau. Based on the mean and SD reported for longitudinal change in phosphorylated tau (N=374) in population of elderly cognitively normal population,²⁷ a sample size of N=65, will have 95% power in detecting any true change in mean over 1-year period (α=0.05, effect size 0.41). Thus, the stated sample size is sufficiently powered to detect longitudinal change in phosphorylated tau.

Discussion

Cognitive impairments are highly prevalent after stroke, having detrimental implications to recovery.¹ Identifying, monitoring and managing these deficits is consistently voted the highest priority by stakeholders.⁵ Given the heterogeneity of stroke and its comorbid conditions (such as ageing, neurodegeneration and background small vessel disease), there is a need for large-scale longitudinal studies that enable accurate predictions of recovery and pave the way for personalised therapy allocation. Previous longitudinal studies of poststroke outcomes have failed to identify reliable biomarkers of recovery.⁸ On the one hand, this is due to a narrow focus on only limited contributors to recovery (eg, lesion topology), despite the fact that stroke recovery is known to be inherently multimodal.⁹ On the other hand, this is due to using brief, shallow cognitive assessments in small-sized samples. Our main longitudinal study addresses this gap by coupling emerging blood and brain imaging biomarkers with a novel scalable digital technology (IC3) that allows in-depth remote monitoring of cognition, thus providing a multimodal account of recovery.

Moreover, current stand-alone cognitive screening tools commonly used in routine clinical care are either not stroke-specific, or not comprehensive. Commonly used tools, such as MOCA,¹⁵ Mini Mental State Examination²⁸ and Addenbrooke’s Cognitive Examination-Revised,²⁹ are tailored to detect deficits in neurodegenerative dementias and are not suitable for patients with stroke, who often have domain-specific as well as domain-general deficits.¹⁰ Additionally, increasingly used cognitive screens designed specifically for stroke, such as the OCS, OCS-Plus and the Cognitive Assessment Scale for Stroke Patients are not in-depth enough to allow a deep cognitive phenotyping and likely to miss the milder end of the severity spectrum. Assessments that are both stroke-specific and comprehensive, such as Birmingham Cognitive Screen, are too resource-intensive for routine clinical practice or longitudinal research studies into poststroke cognition. The IC3 assessment tool developed and applied in this study addresses these shortcomings, building the foundation for routine detailed monitoring of cognition and for scalable large-scale population-based studies of poststroke cognitive impairment.

The blood biomarker substudies are expected to yield novel findings concerning the role of emerging biomarkers in neurodegenerative dementias, specifically amyloid and tau entities, as well as the role of biomarkers of neuroaxonal injury, such as NFL and GFAP, in determining cognitive outcomes poststroke. The study will investigate the longitudinal dynamics of these biomarkers in the postacute stroke period. The integration of both brain imaging biomarkers (eg, white matter integrity, tissue microstructure, vascular reactivity, small vessel disease load) and blood biomarkers is expected to result in cutting-edge and easily applicable predictive models for poststroke cognition. The IC3 study is expected to make a significant contribution to a growing number of well-designed biomarker studies aimed at poststroke cognition (such as R4VAD³⁰ and Discovery³¹) which have the potential to significantly advance the treatment of a disease that affects millions of people globally every year. These studies will enable more precise targeting of cognitive rehabilitation to individuals who are most at risk of progressive decline and those who have the highest potential for recovery.

Data management

All data collected will be pseudoanonymised and securely stored on university servers. Blood samples will be stored at the Imperial College Clinical Research Facility and Imperial College Healthcare NHS Trust. Blood biomarkers will be processed and analysed following Good Laboratory Practice. Data management procedures will be compliant with both Imperial College London guidelines and General Data Protection Regulation regulations.

Ethics and dissemination

Ethical considerations

This study has been approved by South West—Frenchay Research Ethics Committee (IRAS 299333) and authorised by the UK’s Health Research Authority. All participants satisfying the inclusion and exclusion criteria will be approached and consented. If a patient is unable to provide fully informed consent, for example, due to severe aphasia, we will assent with permission from a consultee. Patients who have been assented will be reconsented if they are subsequently able to provide full informed consent. Our study poses minimal risk to participants and will not affect their privacy, be invasive or restrictive. The consent procedure will be carried out in strict compliance with national legislation and General Data Protection Regulation.

Dissemination

The findings from this research will be disseminated through presentations at scientific conferences, as well as through publication in peer-reviewed journals. Additionally, the data will be made available to authorised researchers to facilitate additional analysis, contributing to the development of multimodal biomarkers of recovery in stroke.

Ethics statements

Patient consent for publication

Not applicable.