A new Patient‐Reported Outcome Scale to assess autoimmune Encephalitis: PROSE

Abstract

Objective

Autoimmune encephalitis (AIE) has long‐lasting effects. Generic clinical assessment‐scales insufficiently capture disease‐specific sequelae. Patient‐reported outcome measures (PROMs) assess outcomes important to patients, ensuring relevance in the clinic and trials. We developed a disease‐specific patient‐reported outcome scale for encephalitis (PROSE).

Methods

Existing PROM‐items (n = 235) were evaluated for content‐validity and supplemented with focus‐group‐derived items. Patients with a history of anti‐NMDAR, anti‐LGI1, or anti‐CASPR2 encephalitis completed the preliminary item pool (n = 166). To refine and reduce the item‐pool and create the PROSE, we conducted item‐, factor‐, and Rasch‐analyses. Test–retest reliability was assessed over 2 weeks and convergent‐validity against established measures.

Results

The analysis included data from 202 patients with anti‐LGI1 (n = 97), anti‐NMDAR (n = 85), and anti‐CASPR2 (n = 20) encephalitis, median 3.5 years from diagnosis (range = 0.05–14; n = 41 <1 year). Prospective, repeated assessments (n = 52) were available in 22 patients diagnosed <24 months prior. Analyses supported the viability of a 40‐item unidimensional interval scale for patient‐perceived disease‐severity: the PROSE. The person‐separation (Cronbach's alpha = 0.92, p < 0.001) and test–retest reliability (intra‐class correlation‐coefficient = 0.93, p < 0.001, n = 23) were high. PROSE covers cognition, behavior, emotional and physical wellbeing, daily activities, and participation. PROSE scores were normally distributed and informative across subtypes and demographic groups, correlated with time‐since‐diagnosis (R = −0.32, p = 0.00018) and captured improvement over time prospectively (n = 22). They showed reasonable‐to‐good agreement with self‐reported physical and emotional wellbeing (R = 0.57 and 0.55, p < 0.001) and quality of life (R = 0.57, p < 0.001) on Visual Analogue Scales, to existing PROMs (ρ = 0.42 to ρ = 0.81) and functional scores (mRS ρ = 0.44, p < 0.001, CASE ρ = 0.40, p < 0.001), with strongest correlations to aggregate constructs.

Interpretation

PROSE is a psychometrically robust, disease‐specific patient‐reported outcome assessment‐tool for encephalitis. ANN NEUROL 2026;100:511–524

Autoimmune encephalitis (AIE) is a group of severe neurological diseases, affecting people of all ages. ¹ Despite its severity, most patients show remarkable recovery following immune therapy. ² Historically, a favorable outcome has been reported for most patients with AIE. However, more recent studies have uncovered long‐term cognitive deficits, self‐reported psychosocial and functional sequelae, fatigue, and a decreased quality of life in the majority of patients, ³ , ⁴ , ⁵ , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ even in those with a “favorable” outcome using generic clinical assessment scales. The most commonly reported cognitive difficulties across antibody types include deficits in memory, attention, processing speed, and executive function ⁴ , ⁶ , ¹⁴ ; psychosocial sequelae often related to anxiety, depressive symptoms, or difficulties with social interaction. ⁶ , ⁷ , ¹⁵ Persistent cognitive and psychosocial difficulties may have serious ramifications for the ability of patients to reintegrate into work, school, and social activities. For example, a third of the patients with anti‐N‐methyl‐D‐aspartate receptor (anti‐NMDAR) encephalitis do not return to school or work, whereas another 15 to 20% return at a lower level or with adjustments. ⁶ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰

Cohort studies and trials in AIE frequently use generic clinical assessment scales, most notably the modified Rankin Scale (mRS), to evaluate outcomes. Although widely adopted in the field of AIE, the mRS was originally designed as an outcome assessment scale for stroke and focuses primarily on physical symptoms and functional impairments affecting independent daily living. ²¹ Despite modifications or adapted scoring matrices, ²² the cognitive, behavioral, and psychosocial consequences of encephalitis are largely overlooked. The Clinical Assessment Scale for Autoimmune Encephalitis (CASE), introduced in 2019 as a disease‐specific assessment scale for encephalitis, is more informative than the mRS in the acute phase, ²³ however, remains largely unable to capture longer‐term cognitive and psychosocial impairments. ²⁴ The lack of sensitivity to subtle sequelae (ceiling effect) reduces the power of clinical trials. ¹⁵ Standardized cognitive assessments and the use of patient‐reported outcome measures (PROMs) to capture the psychosocial and more nuanced physical (ie, fatiguability) sequelae of encephalitis, as well as their impact on daily activities, participation, and quality of life can fill this knowledge gap. PROMs enable the patient's perspective to be directly reflected, complementing clinical assessments by providing insights that cannot be measured by health care professionals alone. Developing PROMs for populations with cognitive deficits and psychiatric manifestations may present methodological challenges; these can be addressed through careful item construction, patient involvement, and psychometric evaluation. ²⁵ A systematic review of literature was unable to identify a PROM that was already developed or validated for use in patients with encephalitis. Instead, a heterogeneous compilation of generic patient‐reported outcome assessments had been applied. ¹⁵ Over the past years, experts in the field of AIE have expressed the need for more sensitive, disease‐specific outcome measures for AIE. ⁴ , ⁷ , ¹⁵ , ²⁶ , ²⁷ , ²⁸ This study aimed to develop a new psychometrically robust, disease‐specific Patient‐Reported Outcome Scale for Encephalitis—“PROSE”—to assess the most relevant outcomes for this population.

Methods

We developed PROSE using a structured methodology incorporating both qualitative and quantitative approaches (Fig 1, Supporting Information S1). The final scale underwent a series of validation studies and reliability testing in a large cohort of patients with AIE.

FIGURE 1.

(A) Flowchart of the development process of the patient‐reported outcome scale for encephalitis (PROSE). (B) Flowchart describing item reduction during Rasch analysis.

Generation of the Item Pool ‐ Content Validity of Existing PROMs

We assessed the content validity of PROMs previously applied in encephalitis research ¹⁵ by assigning a validity index to all individual items/questions included within each of the scales. Three clinical investigators (authors J.B., Y.S.C., and J.M.dV.) scored each individual item (n = 235) of PROMs (n = 14) sourced from encephalitis literature, ¹⁵ evaluating its level of relevance to the autoimmune encephalitis population (Supplementary Table S1). Scores ranged from zero (“completely irrelevant”) to three (“definitely relevant”). Evaluations of the investigators were summed, resulting in a “content validity index”. Items with an index of 6 or higher were automatically included in the preliminary item pool (see Fig 1). Items with an index below 3 were excluded. Those with a moderate index rating (3–5) were discussed with a fourth investigator (author M.J.T.), to reach consensus for their inclusion. PROMs with a proportion of >50% “relevant” items were included in their complete form in the preliminary item pool, and were also used for external validation testing. The preliminary item pool included 148 items from different existing PROMs.

Generation of the Item Pool ‐ Patient Involvement

To ensure that no important topics were overlooked by the previously applied PROMs (content saturation), and to confirm content validity of the selected item pool, 2 focus group discussions were conducted. We invited 13 patients who had lived experience of different types of AIE; anti‐NMDAR encephalitis (n = 7), anti‐Leucine‐Rich Glioma‐Inactivated Protein 1 (anti‐LGI1, n = 4), and anti‐Glutamic Acid Decarboxylase 65‐kilodalton isoform (anti‐GAD65, n = 2). Participants were representative of different age groups, genders, educational levels, socio‐economic and ethnic background (Supplementary Table S2). A qualitative approach was used, where participants were encouraged to talk about their perceived outcomes and determining factors, with minimal interference by the investigators. Drawing from audio recordings and verbatim transcripts, open coding by 2 investigators identified relevant themes relating to outcomes, which were verified with experts in autoimmune encephalitis and neuropsychology until consensus was reached. The resulting themes were cross‐referenced against the list of existing PROM items. Where gaps were identified, new items (n = 18) were found to cover important themes, sourced from PROMs used in related fields of research (n = 11) or newly developed when no suitable existing items were available (n = 7; Supplementary Table S3). Details regarding the focus group design, recruitment strategy, and qualitative analysis are provided in the supplement.

Phrasing and Scoring

Introductory sentences, recall periods, and response categories of items originating from existing questionnaires were sometimes rephrased for uniformity. Original scoring formats were primarily retained. Where necessary, response categories were re‐coded so that items were anchored at 0 (eg, transformed from 1‐2‐3‐4 to 0‐1‐2‐3) or reversed in scoring to ensure uniformity across positive and negatively‐worded question styles (see Supplementary Tables S1 and S4).

Patient Selection and Data Collection

The Erasmus University Medical Centre (Erasmus MC) is the national referral center for autoimmune encephalitis in the Netherlands, and provides all antibody testing. This ensures that we are aware of all patients diagnosed with autoimmune encephalitis in the Netherlands. All patients who fulfilled the diagnostic criteria ¹ for anti‐NMDAR, anti‐LGI1, or anti‐Contactin‐Associated Protein‐like 2 (anti‐CASPR2) encephalitis were contacted for participation, provided they were (1) able to self‐report – requiring a certain level of consciousness (Glasgow Coma Score ≥ 12) and fluency in the Dutch language, and (2) aged at least 16 years at the moment of the study. Patients diagnosed between December 2020 and July 2023 were prospectively evaluated at structured assessment times; all other patients diagnosed with any of these types of autoimmune encephalitis in the Netherlands since the discovery of the disease were contacted for a single cross‐sectional assessment.

Data were collected via a digital platform integrating self‐reported (online) and cognitive (visit) assessments, supplemented with clinical data of the disease episode and demographic information. Participants completed the 166‐item preliminary item pool and four 0 to 100 Visual Analogue Scales (VAS) on self‐assessed physical functioning, emotional wellbeing, daily functioning, and quality of life from home (online or on paper). The online version did not allow items to be skipped (preventing missing data; participants were able to leave a comment) and completing all items took, on average, 45 to 90 minutes. Patients could receive help from caregivers in reading and comprehending questions, and in marking responses, but only the patient was permitted to determine their answers. Participants were invited to subsequently visit our outpatient clinic to provide feedback on the items, and to determine concurrent functional status (mRS and CASE) and cognitive ability (through a standardized 60–90‐minute cognitive test battery; Supplementary Fig S1). ⁶ For those unable to travel, our research team travelled to a local clinic to still include these patients and minimize selection bias. ⁶

Item Analysis

The preliminary pool of 166 items was assessed for highly intercorrelated items (r > 0.70), indicative of redundancy, and for items with very low intercorrelations (r < 0.30), which could undermine the internal consistency of the scale.

Factor Analysis

Cronbach's coefficient alpha was used to assess internal consistency, and items were subsequently tested using the Kaiser‐Meyer‐Olkin value^e1 > 0.80 and the Bartlett's Test of Sphericity^e2, p < 0.05, for suitability to undergo Principal Components Analysis, which was undertaken to determine the number and characteristics of factors.

Rasch Modeling

Rasch analysis allowed for the outcome assessment to provide data on the interval level, to enable calculations using parametric tests (Supporting Information S1). This was achieved by creating a model aimed to calculate the probability of a person's response, given their responses to other items in the scale (Partial Credit Model), through analyses of local dependency, unidimensionality (Principal Components Analysis of the residuals), item misfit, and disordered thresholds. The item pool was assessed for local dependency and overlapping content to remove items that were too closely related. For unidimensionality to be confirmed, <5% of the participants were required to have significantly different scores on the identified subsets. Misfitting items were sequentially rescored or removed to improve model fit. Item fit was evaluated using a combination of standardized fit residuals and item‐trait interaction statistics. Given the relatively small sample size, standardized statistics were interpreted with caution, and greater weight was placed on non‐standardized fit indices when determining item misfit.

Differential item functioning (DIF) indicated items with potential biases. An examination of disordered thresholds was undertaken, and item scoring formats were adjusted if necessary.

Overall fit statistics with an acceptable range of 1.5 (item‐person and item‐trait) were evaluated.

Refinements of the item pool and scoring aligned both items and patients along a common interval‐level scale, where those with less ability may only achieve the best score for the easiest items, and only the most able patients can achieve the best scores for the most difficult items.

Reliability and Smallest Detectable Change

Internal consistency and reliability of the scale was assessed using the Person Separation Index. Test–retest reliability was evaluated in patients who completed 2 assessments with an interval of 2 weeks. Only patients over 3 years post‐diagnosis were included to minimize the risk of symptom improvement within the 2‐week interval. ⁶ We calculated intra‐class correlation coefficients for the individual items and composite scores as a measure of reliability.

To estimate the amount of change that would be needed to exceed measurement error, we calculated the smallest detectable change (SDC) from the test–retest data as 1.96 times the standard deviation (SD) of the paired differences between test and retest PROSE scores.

Criterion and Construct Validity

Scores on the reduced questionnaire were correlated with (1) self‐reported assessments of physical functioning, emotional wellbeing, daily functioning, and quality of life on VAS, (2) PROMs that had been included in their complete form; (3) physician‐scored functional outcomes (mRS and CASE), and (4) cognitive assessments. Correlations between variables of at least interval level (including the PROSE) were calculated as Pearson's correlation coefficients (r); correlations among ordinal‐level outcomes (mRS and existing PROM scores) were assessed with Spearman's rho. When the directionality differed between outcome measures (higher scores represented either better or poorer outcomes), correlations were reported as absolute values to consistently reflect the strength of association between conceptually aligned outcomes.

Longitudinal Responsiveness

If feasible, newly diagnosed patients were assessed more than once within the first 2 years after diagnosis. Assessments were scheduled at shorter intervals (weeks) during the early recovery phase to capture rapid changes, and at longer intervals (months) thereafter when recovery becomes more gradual, to limit the burden of testing. PROSE scores were plotted over time to evaluate longitudinal responsiveness.

Ethics Approval

The study was approved by the Institutional Review Board of the Erasmus University Medical Centre (NL77821.078.21). Written informed consent was obtained from all participants or their legal representatives.

Statistical Programs

Psychometric evaluations incorporating Classical Test Theory (item and factor analysis) and Item Response Theory (Rasch modeling) were conducted in SPSS version 28.0^e4 and RUMM2030^e5. Group comparisons and correlations to clinical data, cognitive data, or self‐reported data were performed in R software (version 4.3.2; R Core‐Team 2023).^e6

Results

Self‐reported data on a 166‐item preliminary item pool were collected in 202 patients with anti‐NMDAR (n = 85; average age = 36 years, 78% female patients), anti‐LGI1 (n = 97; average age = 67 years, 35% female patients), and anti‐CASPR2 (n = 20; average age = 67 years, 5% female patients) encephalitis (Table 1a, b). Median timing of the assessments was 3.5 years after diagnosis (interquartile range [IQR] = 1.5–6, range = 0.05–14 years, 41 assessments were <1 year from diagnosis). The median mRS score at the time of the assessment was 2 (IQR = 1–2, range = 0–4). Fifty‐nine of 144 patients (44%) who were employed or enrolled in an educational program before the encephalitis, had returned to their original employment/education, and an additional 21 (15%) returned to a lower‐level job/education or with adaptations.

TABLE 1a.

Demographics and Self‐Reported Outcomes of the PROSE Development Cohort

	Anti‐NMDAR, N = 85	Anti‐LGI1, N = 97	Anti‐CASPR2, N = 20	Overall, N = 202	p‐value
Patient characteristics
Gender, nfemale/N (%)	66/85 (78)	34/97 (35)	1/20 (5)	101/202 (50)	<0.001
Age a , mean (SD, range)	36 (16, 16–74)	67 (10, 35–87)	67 (7, 47–80)	54 (20, 16–87)	<0.001
Educational level pre‐illness, n/N (%)
Low	14/84 (17)	29/93 (31)	4/18 (22)	47/195 (24%)	0.020
Middle	29/84 (35)	31/93 (33)	5/18 (28)	65/195 (33%)
High	41/84 (49)	33/93 (35)	9/18 (50)	83/195 (43%)
mRS at diagnosis, median (IQR, range)	3 (2–3, 1–5)	2 (2–3, 1–4)	2 (2–3, 1–3)	3 (2–3, 1–5)	<0.001
Self‐reported outcomes
Timing (years), median (IQR, range) b	5 (2–7, 0–14)	3 (1–6, 0–13)	3 (1–5, 0–10)	4 (1–6, 0–14)	0.021
PROSE, mean (SD, range)	28 (7, 6–41)	27 (7, 10–43)	28 (8, 10–38)	28 (7, 6–43)	0.60
EQ‐5D‐5L, mean (SD, range)	0.81 (0.19, 0.15–1.00)	0.83 (0.16, 0.33–1.00)	0.75 (0.17, 0.39–1.00)	0.81 (0.18, 0.15–1.00)	0.076
WHO‐5 wellbeing index, median (IQR)	66 (48, 76)	72 (52, 80)	62 (46, 80)	68 (48, 80)	0.34
SF‐36‐v2, median (IQR)
General health	65 (50–80)	60 (50–75)	48 (40–73)	60 (50–75)	0.16
Physical functioning	95 (80–100)	90 (60–95)	70 (50–85)	90 (65–100)	<0.001
Role physical problems	69 (44–94)	69 (44–88)	41 (25–72)	63 (44–88)	0.056
Pain	88 (67–100)	90 (67–100)	67 (56–100)	90 (67–100)	0.066
Emotional wellbeing	75 (60–90)	75 (60–90)	75 (53–90)	75 (60–90)	0.98
Role emotional problems	75 (50–100)	75 (50–100)	67 (42–75)	75 (50–100)	0.50
Social functioning	75 (50–100)	75 (50–100)	75 (69–94)	75 (50–100)	0.69
Vitality	56 (44–69)	63 (50–75)	50 (38–66)	63 (44–75)	0.066
WHO‐DAS‐II, median (IQR)
Mobility	0 (0–13)	6 (0–25)	25 (9–38)	6 (0–25)	<0.001
Cognition	15 (5–30)	25 (5–35)	28 (3–38)	18 (5–35)	0.51
Self‐care	0 (0–10)	0 (0–10)	0 (0–10)	0 (0–10)	0.72
Daily activities	10 (0–30)	10 (0–30)	20 (10–50)	10 (0–30)	0.69
Social interaction	17 (8–33)	17 (8–33)	17 (8–25)	17 (8–33)	0.95
Participation	25 (13–42)	21 (8–38)	23 (15–46)	23 (8–42)	0.79
HADS, median (IQR)
Depression subscale	3.0 (1.0–6.0)	2.0 (1.0–6.0)	4.5 (1.5–8.0)	3.0 (1.0–6.0)	0.32
Anxiety subscale	5.0 (2.0–7.0)	3.0 (1.0–6.0)	4.0 (1.5–9.5)	4.0 (1.0–7.0)	0.22
BDI, median (IQR)	7.0 (4.0–12.0)	6.0 (3.0–11.0)	8.5 (4.5–15.5)	7.0 (3.0–12.0)	0.13
FSS, median (IQR)	3.7 (2.4–5.0)	3.3 (2.1–4.8)	4.9 (3.2–6.1)	3.6 (2.2–4.9)	0.029

^a At participation.

^b Time of the questionnaires after diagnosis and treatment.

Abbreviations: BDI = Beck Depression Inventory (0–63, higher scores = poorer outcome); EQ‐5D‐5L = EuroQol‐5 Dimensions‐5 Levels (0–1, higher scores = better outcome); FSS = Fatigue Severity Scale (1–7, higher = poorer); HADS = Hospital Anxiety and Depression Scale (0–21, higher scores = poorer outcome); mRS = modified Rankin Scale; PROSE = Patient Reported Outcome Scale for Encephalitis (0–100, higher scores = poorer outcomes); SF‐36 = 36‐Item Short Form Health Survey (0–100, higher scores = better outcome); WHO‐5 = World Health Organization‐Five Well‐being Index (0–100, higher scores = better outcome); WHO‐DAS = World Health Organization Disability Assessment Schedule (0–100, higher scores = poorer outcome).

TABLE 1b.

Self‐Reported, Functional and Cognitive Outcomes of the PROSE Development Cohort

Visual analogue scales a
Physical wellbeing, mean (SD, range)	74 (15, 4–98)	73 (17, 20–100)	68 (17, 40–94)	73 (16, 4–100)	0.34
Emotional wellbeing, mean (SD, range)	78 (18, 10–100)	78 (20, 5–100)	81 (18, 41–100)	78 (19, 5–100)	0.75
Daily functioning, mean (SD, range)	78 (17, 3–100)	76 (19, 3–100)	72 (15, 50–97)	76 (18, 3–100)	0.21
Quality of life, mean (SD, range)	79 (18, 25–100)	79 (20, 20–100)	80 (16, 50–100)	79 (19, 20–100)	0.99
Functional outcomes b
mRS, median (IQR, range)	1 (1–2, 0–4)	2 (1–2, 0–3)	2 (1–2, 1–4)	1 (1–2, 0–4)	0.054
CASE, median (IQR, range)	1 (1–2, 0–7)	1 (1–3, 0–5)	1 (1–2, 0–5)	1 (1–2, 0–7)	0.61
Employment/education, n/N (%) c					<0.001
Did not return to school/work	15/65 (23)	32/56 (57)	6/12 (50)	53/133 (40)
Returned with adaptations	12/65 (18)	5/56 (9)	4/12 (33)	21/133 (16)
Returned to pre‐illness level	38/65 (58)	19/56 (34)	2/12 (17)	59/133 (44)
Cognitive outcomes b	(N = 64)	(N = 74)	(N = 18)	(N = 156)
Memory, mean z‐score (SD, range)	−0.92 (0.96, −3.28–0.94)	−0.77 (0.67, −1.96–1.02)	−0.49 (0.40, −1.24–0.03)	−0.80 (0.79, −3.28–1.02)	0.089
Executive functioning, mean z‐score (SD, range)	−0.69 (0.83, −3.25–0.97)	−0.32 (0.60, −2.01–0.74)	−0.03 (0.64, −1.26–0.91)	−0.43 (0.74, −3.25–0.97)	0.001

^a Self‐reported on a 0–100 scale, from worst to best possible outcome.

^b At the time of data collection.

^c Of patients attending school/work pre‐illness.

Concomitant cognitive assessments were available for 156 participants (average time between self‐report and cognitive assessment 11 days, SD = 39). Prospective, repeated assessments (n = 52) were performed in 22 patients.

Item Pool Generation

The 166‐item pool was generated from 148 items extracted from existing PROMs (comprising 8 complete PROMs and 9 individual items) and 18 items added based on themes identified through the focus groups (see Supplementary Fig S1A and Supplementary Table S3), retrieved from other existing PROMs (not previously applied in AIE; 11 items) or newly developed (7 items). Notably, 9 of these 18 patient‐derived items (50%) were retained in the final 40‐item PROSE (after item, factor, and Rasch analysis), demonstrating substantial patient contribution and robust psychometric properties.

Item Analysis

A total of 13 items had weak correlations (R < 0.30) with >5% of other items in the pool, and 26 items were redundant (intercorrelations R > 0.70; Supporting Information S1). These items were removed, with the initial exception of items with high content validity based on the focus groups (n = 4, see Fig 1).

Factor Analysis

Factor analysis was conducted for 132 items. Principal components analysis revealed the presence of 10 components (Supplementary Fig S2). The component matrix revealed that most (120/132) items loaded strongly (>0.3) onto the first component. After exploring solutions with multiple factors (Supporting Information S1), we decided to promote a unidimensional scale, removing items that did not load onto the primary component (12 items), resulting in a total of 120 items included for Rasch modeling.

Rasch Analysis

Rasch analysis resulted in the removal of 77 items with local dependencies and overlapping content and 3 misfitting items, and in the adjustment of some scoring formats (see Fig 1B and Supporting Information S1). This led to a unidimensional 40‐item scale (PROSE) with good overall model fit (chi‐square probability = 0.013, nonsignificant p‐value with Bonferroni correction). The Person Separation Index (0.92) indicated good internal consistency.

Update to the Scoring

The Rasch‐derived 40‐item PROSE resulted in a 0‐to‐128 point ordinal scale, which was transformed to a 0‐to‐100 point interval scale (ranging from 0 = unaffected through to 100 = severely affected) using location values accounting for item difficulty and patient ability (Supporting Information S2 and Supplementary Table S5).

Test–Retest Reliability and Smallest Detectable Change

The PROSE showed excellent test–retest reliability (Intra‐Class Correlation Coefficient = 0.95, p < 0.001) in a subset of patients (n = 23) who had multiple assessments. The mean difference between test and retest scores was −0.57 points, with an SD of paired differences of 2.41. The SDC was 4.7 points, indicating that a change of approximately 5 points is required to exceed measurement error with 95% confidence interval.

PROSE in AIE (Criterion and Construct Validity)

PROSE scores were equally distributed in all 3 AIE subtypes (Fig 2A and Supplementary Fig S3), with a mean of 28 (SD = 7, range = 6–43 points). PROSE scores were similar across sex, age categories, and educational levels (Fig 2B–D).

FIGURE 2.

Distribution of patient‐reported outcome scale for encephalitis (PROSE) scores in the development cohort. Distribution of PROSE scores in the development cohort (n = 202). PROSE scores were normally distributed (A), and not influenced by antibody type (A), educational level. (B), age (C) or gender (D). [Color figure can be viewed at www.annalsofneurology.org]

Plotting PROSE scores of the cross‐sectional cohort against time of assessment revealed higher average scores (reflecting a poorer outcome) in the first year after diagnosis (40 patients assessed in the first year; mean PROSE score 31, SD = 7), regressing toward the group average (mean among patients assessed >1 year = 27, SD = 7) in the first 3 to 5 years after diagnosis (correlation of PROSE scores to time of assessment in the first 5 years R = −0.32, p = 0.00018; R = −0.53, p = 0.00017 for patients with anti‐NMDAR encephalitis, R = −0.20, p = 0.10 for anti‐LGI1; and R = −0.32, p = 0.25 for anti‐CASPR2; Fig 3A). Twenty‐two individuals were prospectively assessed; 1 had 4 assessments, 4 had 3, and the remaining patients had 2 assessments within the study period. In these patients, we could confirm the ability to measure improvement over time, especially within the first 18 months after diagnosis (Fig 3B; Supporting Information S1).

FIGURE 3.

Patient‐reported outcome scale for encephalitis (PROSE) scores were highest in the first years after diagnosis and it was possible to measure improvement over time. Average PROSE scores (reflecting a poorer outcome) were higher in the first year after diagnosis (mean = 31, SD = 7), regressing toward the group average (mean of patients assessed >1 year = 27, SD = 7) in 3 to 5 years after diagnosis (correlation of PROSE scores to time of assessment in the first 5 years R = −0.32, p = 0.00018; R = −0.53, p = 0.00017 for patients with anti‐NMDAR encephalitis, R = −0.20, p = 0.10 for anti‐LGI1, and R = −0.32, p = 0.25 for anti‐CASPR2). In prospectively assessed individuals (n = 22), we could confirm the ability to measure improvement over time within individual patients, especially within the first 18 months after diagnosis. [Color figure can be viewed at www.annalsofneurology.org]

The PROSE targets 6 constructs/aspects of outcome. PROSE scores correlated significantly with scores on all 8 included existing scales, but the strongest correlations (>0.6) were with the subscales directed at cognition, social interaction and participation, and emotional wellbeing (see Table 2 and Supplementary Fig S4).

Table 2.

Correlation of the Patient‐Reported Outcome Scale for Encephalitis (PROSE) Score to Existing (Sub)Scales

Existing subscale	Correlation coefficient	p‐value
EQ‐5D‐5L	0.67*	< 0.001
WHO‐5 wellbeing index	0.81*	< 0.001
SF‐36‐v2
Physical component	0.44	< 0.001
General health	0.50	< 0.001
Physical functioning	0.44	< 0.001
Limitations due to physical health	0.63*	< 0.001
Pain	0.37	< 0.001
Mental component	0.79*	< 0.001
Emotional wellbeing	0.78*	< 0.001
Limitations due to emotional problems	0.62*	< 0.001
Social functioning	0.74*	< 0.001
Vitality	0.72*	< 0.001
WHO‐DAS‐II
Mobility	0.42	< 0.001
Cognition	0.77*	< 0.001
Self‐care	0.57	< 0.001
Daily activities	0.65*	< 0.001
Social interaction	0.63*	< 0.001
Participation	0.74*	< 0.001
HADS
Depression subscale	0.80*	< 0.001
Anxiety subscale	0.70*	< 0.001
BDI‐I	0.80*	< 0.001
FSS	0.63*	< 0.001
Visual analogue scales
Liverpool outcome score	0.57	< 0.001
VAS physical functioning	0.57	< 0.001
VAS emotional wellbeing	0.55	< 0.001
VAS daily functioning	0.56	< 0.001
VAS quality of life	0.62*	< 0.001

*The PROSE correlated significantly to all scales but the best correlations (>0.6, indicated by *) were typically with the subscales directed at cognition, social interaction and participation and emotional wellbeing. A Spearman coefficient was calculated for all ordinal scales, a Pearson coefficient for interval or continuous scales (EQ‐5D‐5L and VAS).

BDI = Beck Depression Inventory (0–63, higher scores = poorer outcome); EQ‐5D‐5L = EuroQol‐5 Dimensions‐5 Levels (scores 0–1, higher scores = better outcome); FSS = Fatigue Severity Scale (0–63, higher scores = poorer outcome); HADS = Hospital Anxiety and Depression Scale (0–21, higher scores = poorer outcome); PROSE = Patient Reported Outcome Scale for Encephalitis (scores range from 0 to 100, higher scores correspond to a poorer outcome); SF‐36 = 36‐Item Short Form Health Survey (0–100, higher scores = better outcome); VAS = Visual Analogue Scales; WHO‐5 = World Health Organization‐Five Well‐being Index (0–100, higher scores = better outcome); WHO‐DAS = World Health Organization Disability Assessment Schedule (0–100, higher scores = poorer outcome).

PROSE scores showed medium‐sized correlations to self‐reported global assessments of physical and emotional wellbeing (R = 0.57 and 0.55, respectively, p < 0.001; Fig 4A, B). Participants with the best possible self‐reported physical or emotional wellbeing, respectively, (a score of 100 on the VAS) still had a range of outcomes on the PROSE suggesting that PROSE reflected the combination of physical and emotional wellbeing (Fig 4C). Adding self‐perceived ability in daily functioning and quality of life (also VAS scores; Supplementary Fig S6) provided the best correlation of 0.68 (p < 0.001; Fig 4D).

FIGURE 4.

Patient‐reported outcome scale for encephalitis (PROSE) scores correlate best to composite scores of physical and emotional wellbeing and functioning. PROSE scores correlate well to self‐reported physical (A) and emotional wellbeing (B) on Visual Analogue Scales, better to a composite score of self‐reported physical and emotional wellbeing (C) and best to a score also including self‐reported ability in daily activities and quality of life (D). Self‐reported ability in daily activities and quality of life are provided in Supplementary Figure S6.

There was a moderate correlation of PROSE scores with physician‐scored functional outcomes (mRS, ρ = 0.45, p < 0.001, and CASE, ρ = 0.41, p < 0.001; Fig 5), and PROSE scores in the group of patients with no reported problems on the mRS (mRS = 0, n = 23) were significantly lower (reflecting better outcomes) than in patients with an mRS of 1 and up (n = 178; mean PROSE score = 19.9, SD = 7.1, vs mean = 28.7, SD = 6.5; Mann‐Whitney U = 4.93, p < 0.001). PROSE scores did not significantly correlate with cognitive assessments (Supplementary Fig S5).

FIGURE 5.

The patient‐reported outcome scale for encephalitis (PROSE) shows a moderate correlation to the modified Rankin Scale (mRS) and clinical assessment scale for autoimmune encephalitis (CASE) at the time of the survey. [Color figure can be viewed at www.annalsofneurology.org]

Discussion

In this study, we developed a psychometrically robust, disease‐specific PROSE, and provided evidence of its usefulness in a large cohort of 3 subtypes of AIE. This is very relevant, as this allows us to better capture long‐term outcomes of encephalitis on an interval scale, over time, both in regular care and research.

The PROSE is shown to be applicable in 3 of the most prevalent subtypes of AIE (anti‐NMDAR, anti‐LGI1, and anti‐CASPR2 encephalitis), ²⁹ in patients of a variety of ages (between 16 and > 80) and educational backgrounds. For patients able to self‐report, the PROSE is a single, brief (10–20 minutes to complete), and all‐round assessment, that is reliable and can be assessed repeatedly.

The co‐design process of the PROSE (ie, the focus groups) resulted in the inclusion of topics important to the patients. Notably, 20% (8/40) of final items were directly included based on patient involvement, whereas other discussed topics were already reflected by items included based on the systematic review. ¹⁵ This includes the multifaceted cognitive and behavioral sequelae of autoimmune encephalitis, allowing the PROSE to avoid ceiling effects that hamper other widely used functional scores (such as mRS and CASE) and remain suitable beyond the subacute phase, when functional scores fall short. This is reflected by the greater level of detail/variance in PROSE scores among patients who score at the lower end (better functional outcomes) of the mRS and CASE scales. Our cross‐sectional and longitudinal analyses reveal the most change over time within the initial recovery phase, however, improvement persists up to 3 to 5 years. This suggests the PROSE is a relevant tool from the subacute to the long‐term follow‐up of AIE, when physician‐scored scales (mRS and CASE) have become less useful. The PROSE is also far less time‐consuming than full neuropsychological evaluations, and less vulnerable to learning effects allowing for repeated testing, while still being able to capture relevant differences over time.

The PROSE addresses 6 constructs, and this is reflected in correlations with established scales. An important outcome of AIE is the impact of cognitive deficits in daily life, which is clearly captured with the PROSE. The finding that self‐perceived wellbeing, functioning, and quality of life show limited correlation to (objective) cognitive outcomes is in line with previous research. ⁶ , ¹⁹ This suggests that factors beyond cognitive recovery play a major role in regaining functioning and quality of life.

While addressing different constructs of outcome, we deliberately pursued a unidimensional scale, to enable transformation of scores to an interval‐level metric, allowing its use as a continuous outcome assessment in research/clinical trials, including statistical comparison between cohorts and treatment strategies. Rasch modeling was chosen for this purpose, as it aligns item difficulty and person ability on a common metric. This methodological approach distinguishes the PROSE from other patient‐reported outcome scores in neurological diseases, including the LANTERN tool suggested for anti‐LGI1 encephalitis. ³⁰ Although the LANTERN also promotes patient‐centered outcome assessment and shares similar content domains (memory, attention and executive function, behavior, mental wellbeing, and vocational activities) – independently selected through patient involvement, validating their relevance to the autoimmune encephalitis population – the PROSE differs in this methodology and in its scope. The PROSE was developed across multiple subtypes of encephalitis, and – informed by systematic review of previously applied items – addresses additional domains (ie, language abilities, learning, social interaction, and intimacy) that may be particularly relevant to different subtypes of autoimmune encephalitis and their respective population characteristics (ie, the younger patients with anti‐NMDAR encephalitis).

A limitation of the PROSE—as for any patient‐reported assessment—is the restricted applicability in the hyperacute phase (first weeks) and most severely affected patients (floor effect). To be included in the study, patients had to be able to self‐report—requiring a certain level of consciousness, language ability, and disease insight. This led to exclusion of the most severe cases from early assessments, although we attempted to include these patients in a later stage of recovery. The PROSE should therefore not be considered a replacement for clinician‐rated assessments (ie, mRS and CASE) in the acute phase of the disease, but rather as a tool to capture more detail, especially in the post‐acute and longer term, when patients have recovered sufficiently to self‐report and when subtle persistent cognitive, behavioral, and psychosocial sequelae may no longer be adequately reflected by clinician‐rated scales. In the earliest phase of encephalitis, clinician‐rated functional assessments, such as the CASE, together with (bedside) cognitive assessments, remain essential to capture acute disease severity. In addition, everyday functional and psychosocial consequences might be captured through proxy‐report for patients who are unable to self‐report at that stage. We have started collecting proxy reports to evaluate patient‐proxy agreement in a future project; detailed analyses of these data fall beyond the scope of the present study. A clinician‐scored version of the PROSE could also be explored, although proxy report may be more suitable as several PROSE items address consequences that are best observed in everyday contexts, such as household organization or social participation.

A related consideration regarding the interpretation of any patient‐reported outcome assessment is the extent to which biological effects are reflected, as opposed to factors like disease‐insight and/or coping style. Future comparison of patient‐ and proxy‐reported PROSE scores may show whether patients and proxies report similar outcomes, and where their perspectives differ.

As we intended to fill a gap in outcome assessment of AIE, there is currently no gold standard for validation. However, the development of the PROSE—grounded in content validity and patient involvement—and its correlations with existing clinical measures of different constructs and recovery over time, strengthen confidence in its relevance.

The PROSE proved relevant in 3 common subtypes of AIE. These subtypes have similarities but also distinct differences in clinical phenotype, ⁴ , ¹¹ , ³¹ advocating the applicability of the PROSE across a clinically heterogeneous spectrum of AIE. As the range of potential sequelae of these 3 distinct subtypes largely overlap with other forms of autoimmune encephalitis, we hypothesize that the PROSE will also be applicable in other cohorts, although this requires validation. In particular, future work may assess its performance in clinically more distinct entities such as anti‐IgLON5 disease, which has a more prominent impact on movement and sleep, besides cognition and behavior; these aspects are also part of the more common types of autoimmune encephalitis and are addressed in the PROSE.

The PROSE was developed in a national cohort of the Netherlands and officially translated into English; Rasch‐driven modifications were performed in the development dataset. As patient‐reported outcome scales are often developed within a specific language (and its associated cultural context), cross‐cultural representation in the PROSE development cohort was limited. In the process of content appraisal and item reduction, when clinically and psychometrically comparable items were encountered, for example, items showing local dependency, preference was given to items less dependent on specific cultural practices to support future generalizability. Future research will extend PROSE's validation in independent cohorts, other languages and cultures, and in pediatric populations. In addition to confirming our reported findings in a similar manner, we will focus on prospective data collection in these cohorts, and combine data to reach relevant numbers for analysis of longitudinal responsiveness. External validation studies will also explore the minimal clinically important difference (MCID) for the PROSE, for example, using anchor‐based approaches linking score change to patient‐ or clinician‐reported global change. The SDC reported in this manuscript provides an estimate of measurement precision, based on a small subgroup, reflecting the expected amount of change that will be needed to exceed measurement error with 95% confidence interval. This should not be interpreted as an MCID, which will require anchor‐based longitudinal validation.

Conclusions

We developed a disease‐specific PROSE, through rigorous psychometric development and validation methodology, capturing long‐term outcomes of encephalitis on an interval scale.

We envision the PROSE to be incorporated as part of standardized outcome assessments of AIE, in addition to objective cognitive and functional clinical assessments. ⁴ Standardized, sensitive outcome assessments will enable comparison of diagnostic and therapeutic strategies promoting clinical‐trial readiness, as well as multicenter and international collaborations paramount in rare diseases such as AIE.

Author Contributions

J.B., M.R.M., E.vdB., and M.J.T. contributed to the conception and design of the study. J.B., Y.S.C., I.K., S.H.C.O., C.J.R., A.A.G.T., L.P.K., J.C.P.V., C.N.K., B.T., F.I.vW., J.K., R.W.vS., A.E.M.B., M.A.A.M.dB., A.vS., and M.R.M. contributed to the acquisition and analysis of data. J.B., M.R.M., and M.J.T. contributed to drafting the text and preparing the figures.

Potential Conflicts of Interest

J.B. and M.J.T., on behalf of the Erasmus University Medical Center, have intellectual property rights on the PROSE. The remaining authors have nothing to report.

Supporting information

Supplementary Data S1. Supporting information.

Supplementary Data S2. Supporting information.

Supplementary Table S1. Scoring of the content validity of all items from previously applied PROMs in encephalitis studies.

Supplementary Table S2. Characteristics of focus group participants.

Supplementary Table S3. Items added to the preliminary item pool, targeting unaddressed themes identified through the focus groups.

Supplementary Table S4. The Patient‐reported Outcome Scale for Encephalitis (PROSE), including information on the source of the item, adaptations to the original for items originating from existing questionnaires, and constructs targeted by the items.

Supplementary Table S5. Conversion table for PROSE scores.

Supplementary Figure S1. Cognitive testing battery.

Supplementary Figure S2. Scree plot of the factor analysis to show the 10 components.

Supplementary Figure S3. Histogram and Q‐Q plot of PROSE scores, demonstrating the scores approximated normality.

Supplementary Figure S4. The correlation of PROSE scores to scores on included complete existing PROMs ranged from 0.37 to 0.81.

Supplementary Figure S5. There was a limited correlation of PROSE scores to cognitive assessments.

Supplementary Figure S6. PROSE scores correlate to self‐reported quality of life (A) and ability in daily activities (B) on visual analogue scales.

Supplementary Figure S7. Item‐person map showing the spread of the 40‐item PROSE across the spectrum of disease severity.

Supplementary Figure S8. Item‐threshold maps of all 40 items included in the PROSE.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.