Study protocol: retrospectively mining multisite clinical data to presymptomatically predict seizure onset for individual patients with Sturge-Weber

Abstract

Introduction

Secondary analysis of hospital-hosted clinical data can save time and cost compared with prospective clinical trials for neuroimaging biomarker development. We present such a study for Sturge-Weber syndrome (SWS), a rare neurovascular disorder that affects 1 in 20 000–50 000 newborns. Children with SWS are at risk for developing neurocognitive deficit by school age. A critical period for early intervention is before 2 years of age, but early diagnostic and prognostic biomarkers are lacking. We aim to retrospectively mine clinical data for SWS at two national centres to develop presymptomatic biomarkers.

Methods and analysis

We will retrospectively collect clinical, MRI and neurocognitive outcome data for patients with SWS who underwent brain MRI before 2 years of age at two national SWS care centres. Expert review of clinical records and MRI quality control will be used to refine the cohort. The merged multisite data will be used to develop algorithms for abnormality detection, lesion-symptom mapping to identify neural substrate and machine learning to predict individual outcomes (presence or absence of seizures) by 2 years of age. Presymptomatic treatment in 0–2 years and before seizure onset may delay or prevent the onset of seizures by 2 years of age, and thereby improve neurocognitive outcomes. The proposed work, if successful, will be one of the largest and most comprehensive multisite databases for the presymptomatic phase of this rare disease.

Ethics and dissemination

This study involves human participants and was approved by Boston Children’s Hospital Institutional Review Board: IRB-P00014482 and IRB-P00025916 Johns Hopkins School of Medicine Institutional Review Board: NA_00043846. Participants gave informed consent to participate in the study before taking part. The Institutional Review Boards at Kennedy Krieger Institute and Boston Children’s Hospital approval have been obtained at each site to retrospectively study this data. Results will be disseminated by presentations, publication and sharing of algorithms generated.

Strengths and limitations of this study.

• We will retrospectively gather multisite neuroimaging and comprehensive data from >100 subjects with Sturge-Weber syndrome, a rare disease, compared with prior single-site studies with typically only a dozen or so patients.

• This will be the first rigorous multisite neuroimaging biomarker study in the presymptomatic phase of Sturge-Weber syndrome, compared with most studies focusing on the postsymptomatic phase.

• We will develop and use statistically rigorous lesion-symptom mapping to predict neurocognitive outcomes in Sturge-Weber syndrome and rigorously test whether combining clinical and MRI information outperforms brain MRI alone in outcome prediction.

• Heterogenous brain MRI protocols (scanner, vendor, scanning site, age at MRI, etc) will be used; however, this heterogeneity increases the generalisability of the results.

Introduction

About 1/1000 infants are born with port-wine birthmarks (PWB) on the face or neck. Of those infants, 5% develop a vascular neurocutaneous disorder called Sturge-Weber syndrome (SWS).¹ This makes SWS patients 1 in 20 000 to 1 in 50 000 newborns, and, therefore, a rare disease.^{2 3} SWS is usually caused by an activating somatic mutation in the gene GNAQ, encoding the Gα_q subunit in the affected tissue.¹ Diagnosis is based on the classic PWB at birth and vascular findings in brain MRI that are routinely acquired during infancy. The disease is not commonly fatal but is incurable.^{4 5} Neurocognitive impairment by school age (6–10 years of age) is a primary issue in SWS.

A landmark event in the progression of SWS is the presence or absence of seizures by 2 years of age (figure 1). About 15% of patients with SWS do not develop seizures by 2 years and they often enjoy good school-age neurocognitive outcomes^{6 7} Because of this, a critical time window to treat or intervene in patients with SWS is during infancy up to 2 years of age, the so-called presymptomatic phase. In this phase, very early treatment with antiepilepsy drugs (AEDs, such as levetiracetam, with or without low-dose aspirin)^8–10 may delay or even avoid seizure symptoms by 2 years of age and thus improve school-age outcomes.^{6 7} The evidence of the early onset of seizures resulting in worse neurologic outcome⁶ has led to a demand for early diagnosis and aggressive intervention for those at increased risk of developing seizures.

Figure 1.

Need for the early prediction of risk to develop epilepsy/seizure symptoms by 2 years of age among patients with SWS in the ‘pre-symptomatic phase’. SWS, Sturge-Weber syndrome.

However, two questions remain open. For one, how to accurately identify who is at risk to develop the seizure/epilepsy symptoms by 2 years of age? These at-risk patients should be the ideal target patients to be included in future trials for testing AEDs in the critical ‘presymptomatic’ time window. For the other, what are the underlying neuroanatomic mechanisms that drive infants with SWS to develop epilepsy/seizure symptoms before 2 years of age? The neuromechanisms should be the targets to design the next-generation AEDs or other presymptomatic treatment strategies.^{4 11}

Infant brain MRI is suggested by recent expert consensus (2018⁴; 2019)⁵ in the field to address these two open questions. This is because infant brain MRI is routinely acquired during the clinical care of patients with SWS and is non-invasive. Besides, MRI can probe 3D neuroanatomy, with both diagnostic value (commonly seen presymptomatic brain abnormalities including leptomeningeal, parenchymal and choroidal capillary venous malformations with congenitally abnormal and ectatic vessels)¹² and prognostic value (abnormal neuronal excitation,¹³ transient diffusion abnormalities^{14 15} and bilateral hemispheric injuries¹ as seen in MRI may suggest an adverse outcome). However, these findings were based on single-site data, small sample sizes (single-digit or only a dozen or so). Moreover, expert interpretations of MRIs are largely qualitative or descriptive and may not be replicated across sites. This issue is further complicated by the limited clinical radiology expertise on this rare disease, especially in less developed regions or countries.^{4 16}

To address these issues, this study aims to collect a relatively large multisite cohort for this rare disease and develop machine learning biomarkers. Existing MRI studies for SWS usually had ≤15^{5 17–19} or dozens11 16 20 21 of patients with SWS in single sites. We aim to merge existing clinical data from Boston Children’s Hospital (BCH) and Kennedy Krieger Institute (KKI) to achieve an sample size at least five times bigger. Existing MRI studies for SWS mostly used expert reviews. We will leverage MRI analytics and machine learning algorithms, which have had success in characterising subtle yet complex injury patterns^{22 23} and in predicting outcomes for stroke,²⁴ tumour,²⁵ autism²⁶ and other neurologic diseases,²⁷ but have not yet been used in SWS. Our hypothesis is that machine learning-driven MRI and clinical biomarkers can predict the onset or absence of seizure symptoms by 2 years at a sensitivity and specificity higher than random guess (ie, 50%). Setting the random guess as the bar is because the lack of such a presymptomatic biomarker for SWS.

Methods and analysis

Overview

This retrospective study was approved by institutional review boards at BCH and KKI. Written consent was waived for the retrospective analysis of existing data that are anonymised. Figure 2 outlines the major components of our study. We plan to start the study in April or December of 2022 (pending funding from National Institute of Health (NIH)). The study period will be for ~2 years.

Figure 2.

Outline of our study protocol.

Part 1. Retrospective data acquisition

Find candidate patients

We plan to collect data for all patients who (a) were born between 2009 and 2020, (b) were under 2 years of age at the first MRI and (c) had SWS brain involvement confirmed by contrast-enhanced MRI before 2 years of age.

Patients will be primarily identified by clinical registries, in the neurology departments from patients cared by Dr. Pinto at BCH and Dr. Comi at KKI.

A secondary source will be the hospital-wide database. We will search clinical database and picture achieving and communication system (PACS) imaging systems by the ICD codes for SWS (table 1). Candidates missed in the clinical registry will be further vetted by their clinical records to confirm eligibility to be included in our database.

Table 1.

ICD-9/10 codes for SWS as a secondary source to query candidate patients beyond the clinical registry as the primary source

ICD-9	Meaning	ICD-10	Meaning
759.6	Other hamartoses, not elsewhere classified	Q85.8	Other phakomatoses, not elsewhere classified
757.32	Angiomatosis	Q85.02	Other congenital malformations
759.6	Sturge (-Weber) (-Dimitri) (encephalocutaneous angiomatosis)

SWS, Sturge-Weber syndrome.

Query and fetch clinical data

We will review patients’ electronic health records (EHRs) and gathered the following clinical variables.

• Demographics: sex, the extent of brain involvement, the extent of PWB (bilateral, unilateral in the left or right side), age at first MRI, year of MRI, age at electroencephalogram (EEG) and clinical result, year of birth and family history of epilepsy

• Treatment: whether the patient was treated presymptomatically (ie, before the first onset of seizures).

• Outcome by 2 years: the primary neurological outcome will be the presence or absence of seizure onset by 2 years, and the age at seizure onset if present. The secondary outcome is the SWS neuroscore (including seizure score). The neuroscore is a scale that measures seizure frequency, hemiparesis, visual field cut and cognitive function (points range from 0 to 15, with lower scores indicating better neurologic status.^28–31 Patients treated presymptomatically were assigned a seizure score of 0 at the time of treatment initiation. Subjects with a seizure score of 0 at 2 years of age are patients who have achieved 2 years of age without seizure onset.

Gather brain MRI data and quality control

We will use patients’ medical record numbers to search and download their brain MRIs in the clinical PACS. The Computational Health Research Integration System (ChRIS) platform offers this function,³² similar to other retrospective paediatric brain MRI projects at BCH.^{27 33} KKI also hosts the i2b2³⁴ platform for this function.

MRI sequences to be downloaded include: T1-weighted (T1w; spin echo vs 3D gradient echo), T2-weighted (T2w) spin echo, susceptibility-weighted imaging (SWI) including the phase and magnitude images, or T2*-weighted gradient echo, fluid-attenuated inversion recovery (FLAIR), diffusion-weighted imaging or diffusion tensor imaging (DTI) and postgadolinium-contrast T1w (spin echo or 3D gradient-echo T1 contrast (T1c) images).

For quality control, MRIs will be visually reviewed by imaging experts (Dr. Ou for BCH data and Dr. Lin for KKI data). MRIs with severe motion, artefacts or part of the brain missing will be discarded.

For the kept MRIs, we will record scanner vendors, scanner models, field of strength, in addition to the available sequences among those listed above.

Manage data

We will use REDCap³⁵ to manually enter and maintain clinical variables from EHR. REDCap is Health Insurance Portability and Accountability Act (HIPAA)-complicant, secure and user-friendly and has been used in similar multisite database by this team.²⁷

Part 2. Identify presymptomatic injury patterns for seizure outcome (population analysis)

MRI preprocessing

Each patient’s T1w brain MRI will undergo our extensively validated brain MRI analysis pipeline. Steps include N4 bias correction,³⁶ field of view normalisation,³⁷ multiatlas skull stripping,³⁸ atlas-based regional segmentation³⁹ as adapted and validated in infants 0–2 years of age.^{38 40 41} The computed brain mask in T1w MRI will automatically skull strip other MRI sequences when rigidly aligned with the T1w MRI.² To compare across patients, their T1w MRIs will be non-rigidly aligned into our atlas space³³ by our validated⁴² and public deformable image registration algorithm.⁴³ DTIs will be used to derive the fractional anisotropy (FA) and apparent diffusion coefficient (ADC) maps, after motion correction and tensor reconstruction using Functional Magnetic Resonance Imaging of the Brain Software Library (FSL),⁴⁴ by factoring out the effects of scanners and sites, while controlling for other covariates such as sex, age, race, etc. We will use the validated⁴⁵ ComBat tool to harmonise MRI metrics extracted from multisite data.⁴⁶

Abnormality detection

Figure 3 shows different types of abnormalities associated with SWS that manifest in multiparametric brain MRI:

Figure 3.

Typical abnormalities (different rows) found in the brain MRI of patient with SWS, in multiple MRI sequences (different columns). Different figure panels are from different patients. Orange arrows point out the abnormal regions. ADC, apparent diffusion coefficient; FLAIR, fluid-attenuated inversion recovery; SWI, susceptibility-weighted imaging; SWS, Sturge-Weber syndrome.

• atrophy, infarction and focal cortical dysplasia or polymicrogyria are often observed in T1w 11 14 21 30 47–49 and T2w ^50–52, FLAIR ^50–52 and SWI⁵³;

• abnormal veins (leptomeningeal venous malformation, enlarged deep medullary and ependymal veins) are often observed in T1w with Gadolinium-contrast enhancement (T1-Gad ^{12 53–55}), and SWI ^{54 56};

• calcification is best visualised on SWI⁵³);

• abnormal diffusion, accelerated myelination or ischaemic injury are often observed in ADC map,^57–59 a parameter map from diffusion MRI sequences; and

• damages in the white matter microstructures or disruptions in fibre tracts are often observed inFA map^60–62 of the DTI.

To automate the abnormality detection, we will extend our recently-developed algorithm that quantifies normal and detect abnormalities as outliers to normal.

The first major component in this algorithm is to quantify normal through constructing age-specific normal brain atlases. The atlases will quantify the normal range of signals (mean and SD) at each voxel in a normal control cohort. We recently developed age-specific normal ADC atlases, based on ADC maps from 201 normal participants who were scanned during 0–6 years of age, divided into 10 age groups (first 2 weeks, every quarter in the first year, then every year till 6 years).³³ The atlases in each age group quantified voxel-wise normal range of ADC values, and characterised regional and hemispheric differences in early-childhood brain development patterns.⁴⁰ The atlases are unbiased to any subject’s anatomy,^{63 64} based on our extensively-validated⁴² group-wise deformable image registration algorithm.⁴³ We plan to build such age-specific atlases for each MRI sequence (T1w, T2w, FLAIR, SWI, ADC), based on our recently-gathered >200 multi-parametric brain MRIs on normal participants. These normal brain MRIs are from published data acquired from Massachusetts General Hospital and BCH (see references for more details on demographics and imaging protocols)^{33 40} and they are distinctive from the SWS patient data we plan to merge from KKI and BCH.

The second step is to detect abnormalities. This will be done by converting the patient’s MRI signal in each sequence into a Z score map. The Z map will quantify voxel-wise deviation (eg, how many SD away) from the normal mean intensity at the corresponding voxel in the age-matched atlas of the same sequence. We recently used this strategy and successfully identified pre-symptomatic ischaemia lesions in ADC maps of 7 SWS patients.⁵⁹ Another study used a similar strategy on T1w and T2w images (after histogram normalisation) and identified white matter abnormalities in preterm infant’s brain MRIs.⁶⁵ We will do this for each MRI sequence, generating Z_T1, Z_T2, Z_FLAIR, Z_SWI and Z_ADC maps, and we will call abnormalities when Z values are lower than −2.

Accuracy for abnormality detection will be quantified by calculating the Dice overlap, sensitivity and specificity compared with expert-annotated abnormality regions. However, expert annotations of exact abnormality boundary in patients with SWS will be subjected to intrareader/inter-reader variability. Therefore, an alternative approach is to indirectly evaluate the accuracy of abnormality detection in the context of pattern analysis and individual outcome prediction (later sections).

Mapping brain injury patterns to outcome

We will focus on the group differences between those who had MRI and no symptom till 2 years (group A) and those whose seizure occurred after MRI and before 2 years (group B). The postsymptomatic patients (first MRI occurred after seizures but before 2 years of age) will form a group C that serves as a validation cohort.

The key question is the ‘neural substrate’ underlying outcome—namely, patterns of brain injuries in the presymptomatic MRI that underpin the subsequent development of seizure by 2 years? This maps to group analysis between group A and group B.

We will use the lesion-symptom mapping (LSM) framework^{66 67} to identify the neural substrate of the outcomes (seizure onset or not) by 2 years. To reduce false positives, we will control for patient sex, age at MRI and whole-brain abnormality volume as covariates.^{68 69} Three versions of LSM models will be used and compared. Voxel-wise LSM (V-LSM) compares whether the presence or absence of abnormality at each voxel is statistically associated with the binary outcomes across patients.⁶⁸ A limitation of V-LSM is that it overlooks the potential interaction among voxels. To address this, multivariate LSM (M-LSM) jointly considers voxels and voxel clusters and uses multivariate machine learning (support vector machine (SVM)) to find a subset of voxels that jointly associate with outcomes.^{70 71} A potential limitation of M-LSM is that the autoselected voxel clusters may not have anatomic explanations. To address this, the latest generation of LSM, known as connectome-based LSM or C-LSM,^72–74 builds on findings in NIH’s human connectome project^75–77 and NIH’s Brain Research Through Advancing Innovative Neurotechnologies (BRAIN) initiative⁷⁷ and quantifies which fibre bundles or functional brain circuits, if injured, are associated with outcomes.⁷² The assumption is that patients may differ in exact anatomic locations that are injured, but their injuries may intersect with the same fibre bundles or brain circuit, which will explain why they experience similar outcomes.

LSM successfully identified neural substrates of survival and language outcomes after adult brain tumor⁷⁸; LSM identified neural substrates of language,^79–81 motor,⁸² word comprehension,⁸³ voice recognition,⁸⁴ attention,⁸⁵ computational ability⁸² and somatosensory outcomes⁸⁶ after adult stroke lesions, etc. These studies used one LSM model, not three, and mostly V-LSM, the basic LSM model. We plan to use all three LSM models, with different assumptions, for the first time in SWS, based on the identified abnormality regions from the previous task (abnormality detection).

We will use two methods to validate the identified neural substrate for the outcome. One indirect validation will be by comparison with group C (postsymptomatic cohort). We hypothesise that the injury patterns that differentiate between group A and group B are the early and subtle stage of abnormalities that will also manifest, to a more severe extent, in group C. The other validation is a direct quantification of the accuracy in individual predictions, as described next.

Part 3. Outcome prediction (individual precision)

The previous part 2 focuses on the group contrast. However, patients in the same outcome group are different in their clinical and MRI characteristics. The key question in this part 3 is: how to use the presymptomatic brain MRI to predict which patient is at risk to develop seizures after MRI and by 2 years? This is individual prediction.

Traditional machine learning models

SVM^{87 88} or random forest (RF),^{89 90} represent each patient by a feature vector. One set of features will come from the presence or absence (1/0) of abnormality in the key neuroanatomic regions as identified by V-LSM, M-LSM and C-LSM neural substrate pattern analysis. This set of features assumes that the spatial patterns of abnormalities can infer outcome. Our second set of features will be imaging appearance features, as listed in table 2. We will concatenate features in neural substrates (spatial features) and imaging appearance features into a long feature vector to represent a patient. We will use our iterative forward inclusion and backward elimination (iFIBE) feature selection algorithm to automatically identify the subset of features that optimally predict outcomes.^{43 91–94} Here, optimality is defined as the sensitivity, specificity and area under the receiver operating characteristic curve (AUC) in k-fold (we will choose k=5) cross validations.

Table 2.

MRI appearance features characterising abnormalities for outcome prediction.

Categories	Details of features
I. Anatomy of abnormalities	I.1. Mass centre in standard atlas space; I.2. Percentage of the whole-brain volume and the volume of each of the 61 auto-segmented brain structures being injured39 41; I.3. Ratios of volumetric injury in the same brain structures between the left and right hemisphere; I.4. Percentage and distribution of abnormalities in 28 major fibre tracts as defined in the JHU atlas.145
II. Geometry of abnormalities	II.1. Volume of abnormal regions; II.2. Maximum diameter along different orthogonal directions, maximum surface of abnormal regions, geometric compactness, spherecity, surface-to-volume ratio in the abnormal regions.
III. Heterogeneity of abnormalities	III.1. Histogram analysis (0, 25, 50, 75 and 100-percentile) of T1w, T2w, T1-Gad, FLAIR, ADC, SWI, ZT1w, ZT2w, ZFLAIR, ZADC, ZSWI signal values within the abnormal regions; III.2. Skewness (asymmetry), kurtosis (flatness), and randomness (entropy, SD) of T1w, T2w, T1-Gad, FLAIR, ADC, SWI, ZT1w, ZT2w, ZFLAIR, ZADC, ZSWI signal values within abnormal regions;
IV. Texture of abnormalities	IV.1. Gray-level co-occurrence matrix features and gray-level run-length matrix of T1w, T2w, T1-Gad, FLAIR, ADC, SWI, ZT1w, ZT2w, ZT1-Gad, ZFLAIR, ZADC, ZSWI signal values within abnormal regions; IV.2. fractal analysis, Minkowski functionals, wavelet transform and Laplacian transforms of Gaussian-filtered images for the abnormal regions.

ADC, apparent diffusion coefficient; FLAIR, fluid-attenuated inversion recovery; SWI, susceptibility-weighted imaging.

Not all features are equally useful. Some features may be redundant. To select the best feature combinations, we developed and implemented into software an iFIBE feature selection tool.^{43 91} Usually, iFIBE automatically selects 5–70 features out of hundreds to thousands of features. It automatically decides how many features to select, based on tasks. It starts from the single most predictive feature (by AUC in cross-validation) and iteratively adds features that increase the AUC than adding any other feature. When no more features can be added that further improves AUC, the algorithm removes features from the selected subset, one at a time, such that the removed feature will increase the AUC of the remaining feature subset more than removing any other feature. The iFIBE algorithm iterates between adding and removing features, until no more features can be added or removed to increase the AUC. Using our iFIBE feature selector^{43 91} with Gaussian-kernel SVM,^{87 88} we selected 12 texture features out of ~200 features for deformable brain MRI registration^{42 43 95}; we selected 7 from >3 00 000 features for histology-based tumour staging (AUC=0.96) and grading (AUC=0.83),⁹¹ from 83 patients; we selected 53 features among >2000 brain MRI features for classifying bipolar disorder, major depression and normal controls (AUC=0.61),⁹² from 113 subjects; and, we selected 69 from >2000 brain MRI features for diagnosing schizophrenia (AUC=0.75),⁹³ from 124 subjects.

These MRI features and the SVM/RF prediction models are commonly used radiomics features and models for predicting brain disease progression and outcomes. The use of SVM and RF is because of their wide use in radiomics to predict brain disease progression and outcomes, and also because they are the baseline model on which we built the iFIBE feature selection algorithm (ie, our iFIBE feature selection is at the best performance when coupled with SVM and RF). Using some of these MRI features in prediction models has led to 0.6–1.0 sensitivity/0.76–0.99 specificity classifying original or metastatic brain tumor,^{96 97} 0.77–0.93 sensitivity/0.76–0.92 specificity predicting good/poor outcomes after acute brain haemorrhage,^{98 99} 0.91–0.94 sensitivity/0.91–1.0 specificity grading brain tumor¹⁰⁰ 0.86 sensitivity/0.92 specificity predicting psychomotor outcome in preterm neonates,¹⁰¹ and 0.71–0.82 sensitivity/0.68–0.92 specificity predicting multiple sclerosis worsening or not¹⁰² or predicting relapsing-remitting multiple sclerosis (MS) patients.¹⁰³

Other clinical features include patient sex, the extent of PWB (left/right/bilateral), age at first MRI, the extent of brain involvement and EEG report (normal/not). Our hypothesis is that combining these clinical variables with MRI features will improve the sensitivity and specificity of outcome prediction compared with using MRI features alone. Our feature selection algorithm will be used to select the subset of clinical and MRI features that together best predicts the outcome.

Deep learning

Deep learning convolutional neural network (CNN) takes the whole image or image patches as input for outcome prediction. We will use the transfer learning strategy,¹⁰⁴ using the pretrained 2D CNN models that were trained in the ImageNet database (>14 million natural images).^105–107 The big data-powered pretrained CNN model will be refined in our SWS cohort, on 2D slices that contain the abnormal regions. This transfer learning strategy is widely used to enable and stabilise deep learning in small sample studies. Successful applications included image-based classification and outcome prediction in 2D chest X-ray images,^{104 108} 2D retina images,^109–112 2D histological images^113–118 and recently in 3D brain MRI for tumour classification.¹¹ Prediction results on various slices of the same patient can be combined through majority voting into a final prediction.¹¹⁶ This plan avoids training a 3D CNN directly on our data, where the sample size may lead to overfitting. The accuracy of deep learning outcome prediction will also be quantified by sensitivity, specificity and AUC in fivefold cross-validations and be compared with those metrics by the traditional machine learning.

The transfer learning strategy has been commonly used to deal with small sample problems. For 2D images, we can initialise with the benchmark artificial intelligence (AI) model from imageNet, and refine the model by samples in a specific task, for each specific AI study.¹⁰⁴ The benchmark AI model can initialise studies (1) across sample size: trained in >14 million images in ImageNet, it can initialise other AI studies with only hundreds to thousands of samples^{104 114 119}; (2) across tasks: trained to recognise natural objects (eg, cats, dogs, balloons, etc), it can initialise clinical AI tasks, such as disease diagnosis,104 110 111 113 115 120 severity evaluation,^{109 117 118} treatment evaluation³ and prediction of prognosis108 112 114 121 and (3) across imaging modalities: trained in camera-acquired natural images, it can initialise studies involving chest X-ray images,^{104 108} retina images^{110 111} and histological images.^{113 114 119} In these Nature, Cell, Lancet, JAMA, PNAS studies, initialisation with the benchmark model and refinement with samples in each specific AI task has led superior accuracies than training from scratch in specific tasks.^{104 122} In 3D medical images, we just trained a 3D benchmark AI model from 16 705 brain MRIs across the lifespan for predicting the continuously valued age.^{123 124} This offers one of the first benchmark AI models that can be transferred to small sample size studies such as in this SWS case. We will test the prediction accuracy in SWS with and without transferring the benchmark model.

Integrate MRI and non-MRI information

We hypothesise that such integration can more accurately predict outcomes than using MRI alone. In traditional machine learning models, clinical features can be concatenated into the feature vector for each patient. The remaining steps (classification, feature selection and cross-validation) stay the same. In deep learning models, clinical features can be added into each feature layer in the CNN architecture,¹²⁵ and the rest is the same. In both cases, the learning process will automatically parse the interaction between MRI and clinical information and find the best combination. We will compare the sensitivity, specificity and AUC between the traditional and deep learning models after combining MRI and clinical information.

Expected sample size and power analysis

From clinical registries, we have identified 98 patients (62 from KKI and 36 from BCH) with brain MRIs before 2 years of age, of which 63 (42 from KKI and 21 from BCH) had the first MRI before the onset of seizure. We expect a more thorough search of clinical registries and ICD-based hospital-wide search can bring us to the expected 80 sample size for infants who had the first MRI before the first onset of seizure. This sample size will be >5 times bigger than most existing presymptomatic studies for SWS.

In the single-variate analysis, with power=0.8 and alpha=0.05, correlating a predictive variable and the neuroscore outcome variable will need 194, 85, 47, 29 subjects to call significance if the correlation is 0.2, 0.3, 0.4, 0.5. Between patient groups presenting and not presenting seizure symptoms by 2 years of age, we need 393, 99 or 63 patients to call significant group differences if a feature has a shared SD between two groups and the difference in this feature between two groups is 20%, 40%, or 50% of the mean of one group. With our expected sample size (BCH+KKI), we will be able to call a predictive variable significantly associated with outcome if the correlation is greater than 0.41, and the group difference is more than 50% of the mean in one group.

In multivariate machine learning, our expected sample size of 80 will allow our algorithm to select up to eight MRI and clinical variables for outcome prediction, according to the ‘rule of 10’ in multivariate ML prediction.^126–128 Our algorithm typically selects 5–10 variables from hundreds of variables extracted from MRI and clinical variabels.11 14 25 48 49 Thus, overfitting is not a big concern in this multisite MRI data for SWS.

Patient and public involvement

Patient advocacy groups will be involved in the distribution of the summary of the findings after publication of results. The development of the research question was informed by the unmet needs determined during the Sturge-Weber Foundation Clinical Care Network conference in 2018. Patients or the public were not involved in the design, or conduct, or reporting or dissemination plans of our research.

Discussion and dessemination

Secondary use of hospital-hosted data can be time-efficient and cost-efficient. This efficiency is further amplified in rare diseases. A typical clinical trial often requires significant funding and years to collect data from single digits or dozens of patients with SWS. Our study protocol, on the other than, plans to retrospectively collect presymptomatic data for ~80 patients with SWS who had neuroimaging under 2 years of age and were cared in BCH and KKI, two large nationwide centres for SWS care, including presymptomatic SWS care.

The importance of identifying reliable biomarkers for early diagnosis of SWS relies on the potential for presymptomatic intervention impacting neurological outcomes of SWS. The first goal of this study is to further elucidate the pathogenesis of early epileptogenesis and brain injury in SWS by quantifying and comparing neuroimaging patterns, which reflect the integrity and reorganisation of neural networks. Comparing brain MRIs in patients with SWS with or without seizures after the first MRI will identify injury patterns that may aid early diagnosis of SWS brain involvement, and guide treatment to improve the prognosis of those at high risk for seizures before 2 years of age and subsequent cognitive and neurologic impairment.

SWS is a developmental disorder and the epileptogenesis mechanism during early ages will likely differ from the mechanisms in the mature brain. Prior studies have repeatedly associated early age of seizure onset in SWS with worse neurologic outcome, more severe epilepsy and worse cognitive outcome.^{6 48 129} Thus, delaying or preventing seizure onset in SWS has become a main treatment goal driving recent efforts to enhance early diagnosis of brain involvement.⁹ Standard imaging techniques and assessments result in only about a 25% sensitivity in early imaging¹³⁰ resulting in the loss of opportunity to intervene early in many children and slowing efforts to determine the most effective presymptomatic treatment. The proposed quantitative and objective study may improve early diagnosis. More infants with SWS brain involvement may benefit from presymptomatic studies, anticipatory guidance and close monitoring for neurologic and developmental impairments. Zallman et al showed that infants with PWBs and evidence of SWS brain involvement on imaging were more likely to receive appropriate early care, compared with those who were not diagnosed before the onset of seizures.¹³⁰

SWS is a spectrum disorder with a well-identified pathological process allowing precise diagnosis, and it is an excellent model for LSM to quantify neuroanatomic basis of outcomes. The brain involvement ranging in extent from single lobe (usually occipital) to extensive bilateral brain involvement of both hemispheres.¹³¹ As a result, the most important factor determining the neurologic and cognitive outcome, other than the age of seizure onset, is the extent of brain involvement. The greater extent of SWS brain involvement is correlated with earlier age of seizure onset.^131–133 Early age of seizure onset, particularly less than a year of age, has been associated with a significant impact on intelligence quotient (IQ),⁴⁸ worse hemiparesis¹²⁹ and lower cognitive function quality of life.²⁸ Therefore, the full extent of SWS brain involvement, as determined by later MRI imaging with contrast and appropriate imaging, after a year of age, is needed; early imaging, even when abnormal, can underestimate the extent of brain involvement. The extent of brain involvement (on imaging after a year of age) will be an important clinical factor in the AI analysis.

Adding clinical variables to neuroimaging is important. Males are associated with worse cognitive outcomes, and with a greater risk of suicidality in SWS.^{134 135} Whether gender impacts the early evolution of SWS as manifested in neuroimaging remains to be studied. A family history of seizures or epilepsy is associated with an early age of seizure onset in SWS.¹³⁶ This suggests that genetic factors, beyond the underlying somatic mutation in GNAQ, have a role in determining the timing of seizure onset in SWS.¹ The extent of skin involvement with PBW may also be a prognostic factor. A greater extent of PWB (bilateral vs unilateral/none) is associated with worse neurologic status.¹³⁷ Bilateral skin involvement is associated with bilateral brain involvement. Therefore, the extent of skin involvement is to some extent a proxy for the extent of brain involvement that impacts the neurologic outcome. However, it has also been shown that patients with no PWB present later and have a better outcome than patients with a similar extent of brain involvement.^{138 139} This observation suggests that more extensive SWS (in terms of more structures (brain, skin and/or eye) involved is in some way different with regards to the evolution of brain injury, from less extensive SWS (brain only) involvement. Gender, family history of epilepsy and the extent of skin involvement are, therefore, also important predictive clinical factors to include in the AI neuroimaging modelling and analysis.

During early stage SWS, the extent of neuroimaging alteration is similar to that of other small diffuse lesions; our sample size is similar to other published datasets (including chronic stroke lesions^{140 141} and multiple sclerosis).^142–144 Identificatication of biomarkers is a complex process but has proved to result in rapid bench-to-bedside translation in many other pathological conditions. The identification of reliable presymptomatic biomarkers will ultimately be used to predict response to therapies and influence ‘go’ or ‘no go’ decisions before advancing clinical trials for SWS.

This protocol describes a multicentre effort to validate emerging techniques in a rare potentially devastating disease. The centres involved have the expertise and the combined number of cases, indicating the strong feasibility of this study. The results will benefit future prospective trials.

Ethics and dissemination

The Institutional Review Boards at Kennedy Krieger Institute and Boston Children’s Hospital approval have been obtained at each site to retrospectively study this data. Results will be disseminated by presentations, publication and sharing of algorithms generated.

Ethics statements

Patient consent for publication

Not applicable.