Case-control study developing Scottish Epilepsy Deaths Study Score to predict epilepsy-related death

Abstract

This study aimed to develop a risk prediction model for epilepsy-related death in adults.

In this age- and sex-matched case-control study, we compared adults (aged ≥16 years) who had epilepsy-related death between 2009 and 2016 to living adults with epilepsy in Scotland. Cases were identified from validated administrative national datasets linked to mortality records. ICD-10 cause-of-death coding was used to define epilepsy-related death. Controls were recruited from a research database and epilepsy clinics. Clinical data from medical records were abstracted and used to undertake univariable and multivariable conditional logistic regression to develop a risk prediction model consisting of four variables chosen a priori. A weighted sum of the factors present was taken to create a risk index—the Scottish Epilepsy Deaths Study Score. Odds ratios were estimated with 95% confidence intervals (CIs).

Here, 224 deceased cases (mean age 48 years, 114 male) and 224 matched living controls were compared. In univariable analysis, predictors of epilepsy-related death were recent epilepsy-related accident and emergency attendance (odds ratio 5.1, 95% CI 3.2–8.3), living in deprived areas (odds ratio 2.5, 95% CI 1.6–4.0), developmental epilepsy (odds ratio 3.1, 95% CI 1.7–5.7), raised Charlson Comorbidity Index score (odds ratio 2.5, 95% CI 1.2–5.2), alcohol abuse (odds ratio 4.4, 95% CI 2.2–9.2), absent recent neurology review (odds ratio 3.8, 95% CI 2.4–6.1) and generalized epilepsy (odds ratio 1.9, 95% CI 1.2–3.0). Scottish Epilepsy Deaths Study Score model variables were derived from the first four listed before, with Charlson Comorbidity Index ≥2 given 1 point, living in the two most deprived areas given 2 points, having an inherited or congenital aetiology or risk factor for developing epilepsy given 2 points and recent epilepsy-related accident and emergency attendance given 3 points. Compared to having a Scottish Epilepsy Deaths Study Score of 0, those with a Scottish Epilepsy Deaths Study Score of 1 remained low risk, with odds ratio 1.6 (95% CI 0.5–4.8). Those with a Scottish Epilepsy Deaths Study Score of 2–3 had moderate risk, with odds ratio 2.8 (95% CI 1.3–6.2). Those with a Scottish Epilepsy Deaths Study Score of 4–5 and 6–8 were high risk, with odds ratio 14.4 (95% CI 5.9–35.2) and 24.0 (95% CI 8.1–71.2), respectively.

The Scottish Epilepsy Deaths Study Score may be a helpful tool for identifying adults at high risk of epilepsy-related death and requires external validation.

Mbizvo et al. develop the SEDS Score: a risk prediction model to identify adults at high risk of epilepsy-related death. Individuals with a SEDS Score of 1 remain at low risk; those with scores of 2–3 have moderately increased risk; while scores of 4–5 and 6–8 reflect high risk, with 14- and 24-times increased risk, respectively.

Introduction

Epilepsy is common, affecting 70 million people globally.¹ In the UK, there are 600 000 people with epilepsy, of which 60 000 live in Scotland.² The 5.4 million population of Scotland is roughly 8% that of the UK. People with epilepsy are at significantly increased risk of premature death.³ Some of those deaths may be entirely unrelated to their epilepsy. However, a substantial proportion relate to the epilepsy itself, its treatment or comorbidities associated with epilepsy.³ Together, these are termed ‘epilepsy-related deaths’, and they are likely to contribute a significant burden of total years of potential life lost.^3,4 A recent study of 1921 epilepsy-related deaths identified across Scotland between 2009 and 2016 showed that these have remained high over time, with age-specific mortality ratios significantly increased in young adults aged 16–54 years, peaking at 5.3 [95% confidence interval (CI) 1.8–8.8] in those aged 16–24 years.⁵ Sudden unexpected death in epilepsy (SUDEP) constituted 30% of the 553 young adult epilepsy-related deaths, with several other non-SUDEP fatal mechanisms identified including aspiration pneumonia, cardiac arrest, antiseizure medication (ASM) or narcotic poisoning, drowning and alcohol dependence. A substantial number of these young adult epilepsy-related deaths were potentially avoidable,⁵ as has also been suggested in other countries.^6–8 This highlights the importance of trying to identify those individuals who are at highest risk so they can be targeted with earlier and more aggressive care. Few studies have attempted to do this through risk prediction modelling (RPM), particularly focusing on epilepsy-related deaths specifically.^9–14 To help address this, we have undertaken a Scottish multicentre case-control study using routine clinical information gathered from medical records. The study aims to develop a pragmatic RPM to identify adults at increased risk of epilepsy-related death. We will describe multiple potential risk factors for epilepsy-related death to help build the model.

Materials and methods

Study design summary

We carried out an age- and sex-matched case-control study comparing adults who had epilepsy-related death to living adults with epilepsy. Data pertaining to potential risk factors for epilepsy-related death were collected from the medical records of cases and controls to allow multivariable analysis and development of an RPM. The study was designed for model development and internal validation,¹⁵ and follows standard reporting guidelines.¹⁶

Data sources and participants

Case ascertainment

Cases were screened for inclusion from the dataset of a previous study investigating the diagnostic accuracy of four administrative healthcare datasets used to identify deceased adults with epilepsy in Scotland.¹⁷ The study linked national mortality records, hospital admissions, ASM prescriptions and regional primary care attendances to identify adults (aged ≥16 years) who died between 1 January 2009 and 1 January 2016. Coded indicators of epilepsy were compared to confirmed epilepsy diagnoses made by a senior epileptologist upon reviewing medical records (see ‘Confirming epilepsy diagnosis’ section). The study identified 614 confirmed epilepsy cases who had epilepsy-related death during the 2009–16 study period. Epilepsy-related death was defined as death occurring with at least one International Classification of Diseases (ICD)-10-coded cause listed as G40–41 (epilepsy–status epilepticus) and/or R56.8 (seizure) in any position within the sequence of events that lead to death in the mortality records, or with cause-of-death free texts pertaining to seizures (alongside at least one community-prescribed ASM), epilepsy or SUDEP.^5,17 Such a definition is consistent with national guidance that the presence of an epilepsy code/indicator anywhere in the cause-of-death record indicates that epilepsy was thought by the certifying doctor to be either part of the sequence of events leading to death or else a factor contributing to the death.^18,19 Additional case-ascertainment benefit was drawn from the routine cause-of-death quality assurance processes in place to improve the accuracy of Scottish death certification and coding,^20,21 the robustness of which was highlighted recently.²² These quality assurance processes helped reduce the likelihood of there being an underestimation in the number of epilepsy-related deaths captured by the study,^5,17 which can sometimes happen in studies using death certification.

Control ascertainment

Living control participants were screened for inclusion from a nationwide Scottish Health Research Register (SHARE),^23,24 supplemented by controls recruited directly from National Health Service (NHS) Lothian epilepsy clinics. Written consent was obtained from both groups.

The SHARE register consists of nearly 300 000 research volunteers in Scotland, aged ≥11 years.^23,24 The SHARE research staff connected us with SHARE registrants aged ≥16 years with at least one G40–41-ICD-10-coded activity in their health records and consenting to be contacted by us. We reviewed the medical records of those consenting to participate.

Clinic controls were recruited in a consecutive manner directly from NHS Lothian epilepsy clinics. We reviewed the medical records of those consenting to participate.

Eligible cases and controls required sufficiently complete medical records to allow confirmation of a diagnosis of epilepsy and abstraction of clinical data. Controls were matched 1:1 by age (±10 years) and sex to cases of epilepsy-related death after recruitment.

Confirming epilepsy diagnosis

Access to medical records was made possible through the support of a national network of clinical colleagues, as described in greater detail previously.^5,17 For each candidate case and control, the medical records were reviewed by an experienced consultant epileptologist (S.E.D.), who used the medical records to confirm the presence or absence of epilepsy.²⁵ This was based on corroborative evidence such as the presence of two or more unprovoked seizures or clear documentation of a diagnosis of epilepsy from a neurologist.^26,27 The medical records were extensive, and included electronic and paper records from general and/or specialist inpatients, emergency care, outpatients (including neurology clinics), hospital discharge summaries, referral letters from general practitioners, radiology scans, neurophysiology reports and medication lists.

Data abstraction, outcome and predictors

The outcome for model prediction fell into binary categories: 1 = deceased (i.e. epilepsy-related death) and 0 = alive (living control).

Abstraction of data from medical records was undertaken by S.E.D. during several hospital visits made between 11 October 2017 and 24 February 2019 for cases, and 25 February 2019 and 15 January 2020 for controls. Data on multiple potential risk factors were collected using a data abstraction tool developed a priori using systematic review evidence (Supplementary material, section 1).³ Candidate predictor variable selection was refined by consensus among the research team and informed by consultation with external epilepsy specialists, patient and charity groups, and parliamentary policymakers.^28–30Table 1 lists the predictor variables that were ultimately selected as candidates for model development. The Scottish Index of Multiple Deprivation (SIMD) looks at the extent to which an area is deprived across seven domains: income, employment, education, health, access to services, crime and housing. This is ranked from 1 = most deprived to 5 = least deprived areas of residence.³² For each control participant, we inserted their home postcode into the Scottish Government’s publicly available SIMD postcode lookup file to reveal their SIMD quintile.³² The Charlson Comorbidity Index (CCI) is a well-validated prognostic index of comorbid conditions that alter 10-year survival in patients with multiple comorbidities.^33–35 The comorbidities captured include cardiovascular, cerebrovascular, respiratory, neurodegenerative and neoplastic. The CCI is scored from 0 to 37 points, with higher scores associated with lower survival. We extracted each participant’s current comorbidities from the medical records and used this to calculate their CCI score, which is easily done using an online calculator (www.mdcalc.com/calc/3917/charlson-comorbidity-index-cci).³⁶ We grouped the CCI scores into categories 0–1 (associated with a 96–98% estimated 10-year survival), 2–3 (77–90% estimated 10-year survival) and ≥4 (≤53% estimated 10-year survival),^33–35 as grouped elsewhere.³¹ However, we were unable to include age within our CCI calculations as cases and controls were already 1:1 matched by age in our study, meaning our CCI scores took a maximum of 33 points. CCI scores were used as they would allow us to capture the comorbidities most relevant to mortality and increase applicability of our methods in other regions, as CCI is widely used.^31–38

Table 1.

Variables considered for analysis (n = 224 deceased cases and 224 alive controls)

Variable	Coding	Missing data burden		Chosen for multivariable model
		Cases	Controls
SIMD quintilea	Group 1 = 2 most deprived quintiles combined Group 2 = middle deprived quintile Group 3 = 2 least deprived quintiles combined	0%	3%	Yes
A&E attendance or hospital admission for epilepsy/seizures in preceding 12 months	Yes No Unable to tell (missing data)	4%	1%	Yes
Aetiology and risk factors for developing epilepsy	No risk factors for epilepsy Inherited/congenital/genetic/acquired early lifeb Acquired later lifec Unable to tell (missing data)	13%	12%	Yes
CCId	0–1 2–3 ≥ 4 Unable to tell (missing data)	4%	13%	Yes
Alcohol abuse in preceding 24 months	Yes No Unable to tell (missing data)	11%	4%	No
Seizure type	Focal seizures ± secondary generalization Generalized seizures	20%	9%	No
Mental health problems in preceding 24 monthse	Yes No Unable to tell (missing data)	16%	14%	No
Epilepsy surgery ever	Yes No Unable to tell (missing data)	7%	1%	No
Seen by neurologist in preceding 12 months	Yes No Unable to tell (missing data)	1%	1%	No
Number of ASMs when last seen	Not on an ASM 1 ASM 2 ASMs ≥3 ASM Unable to tell (missing data)	0%	0%	No
Seizure frequency when last seen	<1 seizure per year ≥1 seizure per year Unable to tell (missing data)	40%	11%	No
Seizure timing	Mainly daytime Manly nocturnal Mixture of both daytime and nocturnal Unable to tell (missing data)	70%	56%	No
Seizure alarm in place	Yes No Unable to tell (missing data)	60%	36%	No
Status epilepticus in preceding 24 months	Yes No Unable to tell (missing data)	13%	1%	No

A&E = accident and emergency.

^aSocioeconomic deprivation markers splitting the population into quintiles ordered 1 (most deprived) to 5 (least deprived) areas of residence.

^bInherited/congenital/genetic aetiologies or risk factors acquired at <5 years of age: Febrile convulsions, first degree relative with epilepsy, congenital abnormality/malformation (e.g. cerebral palsy, metabolic infancy syndrome, birth hypoxia), genetic syndrome, attention deficit hyperactivity disorder, autism spectrum disorder, developmental/intellectual delay, premature birth, birth/perinatal difficulties, hydrocephalus, neonatal seizures, or the following acquired at <5 years of age: meningitis/encephalitis, severe head injury, brain tumour, cerebrovascular disease, limbic encephalitis, brain surgery, CNS demyelinating disease, abnormal brain imaging, other hypoxic brain injury, neurodegenerative disease.

^cRisk factors acquired at ≥ 5 years: History of meningitis/encephalitis, severe head injury, brain tumour, cerebrovascular disease, limbic encephalitis, brain surgery, CNS demyelinating disease, abnormal brain imaging, other hypoxic brain injury, neurodegenerative disease.

^dCCI: Predicts 10-year survival in patients with multiple comorbidities, accessible here: www.mdcalc.com/calc/3917/charlson-comorbidity-index-cci. Patients normally have a score of 0–37, although we did not include age within our CCI calculations as cases and controls were already 1:1 matched by age in our study. We chose the initial grouping based on recent literature.³¹

^eIncludes psychosis, depression, anxiety, contact with psychiatric/psychological/learning disability services, admission under psychiatry, self-harm/suicide attempts.

Statistical analysis

All analyses were performed using IBM SPSS Statistics for Macintosh, v.25.0 (Armonk, NY, USA: IBM Corporation). Where relevant, categories with fewer than five events/participants were combined or reported as ‘<5’ to avoid patient identification.³⁹

Sample size

We estimated that we could capture a convenience sample size of 200–250 controls, matched to a mirrored number of age- and sex-matched cases taken from the 614 deceased adults who had epilepsy-related death, for whom medical records had already been reviewed during the diagnostic accuracy study.¹⁷ This number of controls was based on our estimated capacity and the time taken to fully recruit living controls and abstract their clinical data from medical records. Using a recommended rule of 50 events per variable for logistic regression to estimate coefficients with adequate precision,^40,41 such a sample size would allow for a multivariable RPM size of four predictor variables. Using more variables than the sample data could support would increase the risk of overfitting (achieving overly optimistic predictive accuracy and hence resulting in failure to replicate the results elsewhere).⁴²

Missing data

An a priori threshold was set to exclude from further analysis any variables with ≥30% of their values missing either within cases or within controls, based on previous literature.⁴³ We also excluded any variables with >10% difference in the missing data burden between cases compared to controls.⁴⁴ These exclusions allowed us to prioritize creating a pragmatic RPM consisting of variables likely to be found complete in medical records, and that were uniformly represented between cases and controls. For variables with <30% of their values missing, an overall summary of the missing data patterns was given and a logistic regression model for multiple imputation was used to replace the missing values. The missing values were imputed using fully conditional specification techniques based on the iterative Markov chain Monte Carlo method, with 20 imputed datasets gathered and imputed results pooled.^45,46 The missing data handling methods used were based on an assumption that data were missing at random. This was a plausible assumption given that a large number of clinically relevant variables were tested, meaning the probability of missingness would probably depend on only the observed data already captured within the dataset.⁴⁷

Model development

Where possible, we reported both the original and imputed datasets for the models developed. Primary analysis was of the original data (complete case analysis), and the imputed data were used for sensitivity analysis.

Univariable analysis was performed using conditional logistic regression for included variables to provide an odds ratio (OR) and 95% CI.

Correlation was examined in a pairwise fashion between the variables, using contingency tables and chi-square tests with a two-sided 5% significance. Where correlation was significant (P < 0.05 for both complete case and imputed data), the Lambda statistic was used to assess the strength of association, interpreted as 0.01–0.09 = weak, 0.10–0.29 = moderate, 0.30–0.99 = strong and 1.00 = perfect association.⁴⁸ Correlation was assessed for both cases and controls separately, with the highest Lambda statistic between both groups reported (Table 2).

Table 2.

Correlation results for variable pairs

Variable	Variable	Lambda correlation
Seizure type	Aetiology and risk factors for developing epilepsy	0.31	Strong association
Seen by neurologist	Number of ASMs	0.28	Moderate association
Seen by neurologist	CCI	0.20	Moderate association
Aetiology and risk factors for developing epilepsy	CCI	0.19	Moderate association
Alcohol abuse	Aetiology and risk factors for developing epilepsy	0.15	Moderate association
A&E attendance or hospital admission	Mental health problems	0.12	Moderate association
SIMD quintile	Aetiology and risk factors for developing epilepsy	0.07	Weak association
SIMD quintile	Seen by neurologist	0.06	Weak association
Alcohol abuse	Mental health problems	0.05	Weak association

For model selection, two consultant epileptologists (S.E.D. and R.F.M.C.) independently made a clinically driven decision to select four variables for inclusion in the final RPM,⁴² while blinded to the univariable results and considering variable correlations and missing data burdens (see Supplementary material, section 2 for justification of each choice made). Any disagreements in variable selection were resolved by consensus. This variable selection approach was undertaken because it is often advised that variable selection should be focused more on clinical knowledge and previous literature than statistical selection methods alone.^42,49 The rationale is that it is better to select variables based on a wider body of clinical knowledge than to try to depend on statistical significance of results from a lesser quantity of information, particularly in a moderate-sized sample such as ours.^50,51 A data-driven, sample-based selection would be more sensitive to random variation in the data points due to sampling variability and more likely to generate false signals. Furthermore, there are several problems with the popular statistical selection methods themselves, including stepwise variable selection, backward elimination and forward selection, as summarized in detail elsewhere.⁵⁰

For multivariable analysis, multivariable conditional logistic regression was performed to provide ORs for the four variables selected for RPM, adjusted for one another, alongside 95% CIs. Any variables (or levels within variables) retaining significance within the univariable analysis were taken forward into the multivariable model, where they were compared against all remaining cases as a reference for nominal variables. For ordinal variables, any lower-ranking levels that were positive were combined with higher-ranking levels to create a positivity threshold. We assessed the positive variables within the multivariable model and assigned a weight (number of points) to each, based on the observed magnitudes of ORs in the complete case analysis, but also taking imputed data trends into account.⁵² A sum of these points was then taken to give a total score, which we termed a Scottish Epilepsy Deaths Study Score (SEDS Score). An internal validation process was performed for each model (i.e. quantifying statistical performance of the model using the data on which the model was developed).¹⁵ This was done by performing 1000 bootstrap analysis samples to provide more robust P-values, which were then checked for agreement against the original model’s P-values for each variable. Where there was poor agreement, the bootstrap result was prioritized.

We used Nagelkerke’s R² to estimate what proportion of variance in the outcome was explained by the predictor variables in the model, and the Hosmer–Lemeshow goodness-of-fit test to determine whether the model adequately described these outcome data (where P ≥ 0.05 indicates a good fit). The Hosmer–Lemeshow contingency table was used to assess model calibration by illustrating agreement between subgroups of predicted probabilities and their observed frequencies.^15,53 Discrimination describes how well the model differentiates between patients who experienced the outcome (deceased cases) and those who did not (living controls).¹⁵ This would normally be measured using an area under the receiver operating characteristic curve.⁵³ However, receiver operating characteristic statistics can be misleading for binary or categorical predictors and, as such, they are cautioned against in this scenario.^54,55 Therefore, we approximated discrimination primarily through indicating the model’s ability to capture all of the deceased cases by estimating its sensitivity, and all of the living outcomes by estimating its specificity. Positive predictive value, negative predictive value and model accuracy [sensitivity × prevalence + specificity × (1 − prevalence)] were reported for reference only given that these estimates depend on prevalence of the outcome, which was fixed at 50% in this study because of the 1:1 matched case-control design: 95% CIs were included.

Approvals

This study was approved by South East Scotland Research Ethics Committee 2 (15/SS/0165, IRAS 181131), and the Scottish Public Benefit and Privacy Panel for Health and Social Care.

Data availability

Approved researchers can access linked datasets via application to the electronic Data Research and Innovation Service (eDRIS) of the Scottish Information Services Division at www.isdscotland.org/Products-and-Services/eDRIS//. Aggregate case-control data are available on reasonable request (www.muirmaxwellcentre.com/contact-us/). For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.

Results

Baseline characteristics of cases and controls

The study cohort comprised 224 deceased cases (114 male, 110 female) and 224 living controls (114 males, 110 female), spread across multiple NHS Health Board regions in Scotland (Fig. 1).^{5 ,17} Mean age at death for cases was 48 years (±17 SD, range 16–90). This was no different to the mean age at assessment of 48 years (± 16 SD, range 19–87) for controls, demonstrating that the age-matching limits of ± 10 years did not result in any major differences in age between the cases and controls that were actually studied. A total of 124 control participants were recruited from SHARE Scotland,²³ with the remaining 100 recruited from NHS Lothian epilepsy clinics.

Figure 1.

Study flow diagram. F25 = primary care diagnostic read codes for epilepsy; G40–41 = ICD-10 codes for epilepsy and status epilepticus; R56.8 = ICD-10 code for seizures; CHI = Scottish Community Health Index patient identification number.

Missing data

Several predictor variables were excluded from further analysis owing to their missing data burden (Table 1 and Supplementary material, section 2). The remaining variables were taken forward into multiple imputation and univariable analysis.

Correlation

A strong association was seen only between the type of seizures (focal or general) and the risk factors for developing epilepsy (inherited/congenital or acquired) (Table 2. The remaining pairs demonstrated moderate or weak association, with the greatest association among these being between having been seen by a neurologist in the preceding year and the number of ASMs prescribed when last seen.

Univariable analysis

The odds of epilepsy-related death occurring were significantly increased in those who lived in the two most deprived SIMD quintiles, those with a history of alcohol abuse in the preceding 2 years, those with generalized epilepsy, those who had experienced at least one accident and emergency (A&E) attendance or hospital admission in the preceding year related to seizures or epilepsy, those with evidence of mental health problems in the preceding 2 years, those who had not been seen by a neurologist in the preceding year, those with a CCI of 2–3 and those with an inherited/congenital/genetic aetiology or risk factor acquired in early life (<5 years of age) for developing epilepsy (Table 3). ORs and CIs for CCI 2–3 and ≥4 almost entirely overlapped, indicating a similar level of association beyond CCI ≥2 in this dataset. The odds of epilepsy-related death occurring were significantly reduced in those who had undergone previous epilepsy surgery in the complete case analysis. However, only a small number of cases and controls had had epilepsy surgery (<10% of the sample, overall), and the difference associated with this variable was lost when imputed data were included. The number of ASMs taken when last seen were not significant individual predictors. Very few cases and controls (<3% of the sample) were not on an ASM.

Table 3.

Univariable results

Predictor variable	Grouping	Study status		Total	Complete case data analysis OR (95% CI)	P-value	Multiple imputation data OR (95% CI)	P-value
		Alive (control)	Deceased (case)
SIMD Quintile	Two least deprived	85	50	135	Reference		Reference
	Middle deprivation	42	43	85	1.8 (1.0–3.2)	0.06	1.8 (1.0–3.2)	0.05
	Two most deprived	91	131	222	2.5 (1.6–4.0)	0.00	2.5 (1.6–4.0)	0.00
Total		218	224	442
Alcohol abuse	No	202	157	359	Reference		Reference
	Yes	13	42	55	4.4 (2.2–9.2)	0.000	3.7 (1.8–7.9)	0.001
Total		215	199	414
Seizure type	Focal epilepsy	153	119	272	Reference		Reference
	Generalized epilepsy	71	105	176	1.9 (1.2–3.0)	0.006	1.8 (1.1–2.8)	0.012
Total		224	224	448
A&E attendance or hospital admission	No	181	92	273	Reference		Reference
	Yes	41	123	164	5.1 (3.2–8.3)	0.000	5.1 (3.2–8.2)	0.000
Total		222	215	437
Mental health problems	No	101	80	181	Reference		Reference
	Yes	91	109	200	1.6 (1.0–2.6)	0.034	1.6 (1.1–2.4)	0.027
Total		192	189	381
Epilepsy surgery	No	193	203	396	Reference		Reference
	Yes	29	6	35	0.2 (0.1–0.5)	0.001	0.4 (0.1–1.4)	0.159
Total		222	209	431
Seen by neurologist (entire dataset)	Yes	180	115	295	Reference		Reference
	No	42	106	148	3.8 (2.4–6.1)	0.000	3.9 (2.5–6.3)	0.000
Total		222	221	443
Seen by neurologist (SHARE-only dataset)	No	26	61	87	3.4 (1.8–6.1)	0.000
	Yes	96	61	157	Reference		Reference
Total		122	122	244
Aetiology and risk factors for developing epilepsy	None	92	55	147	Reference		Reference
	Inherited/congenital/genetic/acquired early life	41	73	114	3.1 (1.7–5.7)	0.000	3.3 (1.9–5.8)	0.000
	Acquired later life	64	67	131	1.7 (0.9–3.2)	0.088	1.9 (1.1–3.4)	0.030
Total		197	195	392
CCI score	0–1	143	127	270	Reference		Reference
	2–3	33	61	94	2.5 (1.2– 5.2)	0.010	3.3 (1.7–6.6)	0.000
	≥4	19	27	46	2.0 (0.7–5.2)	0.200	3.2 (1.2–8.3)	0.018
Total		195	215	410
Number of AEDs	Not on an ASM	3	8	11	Reference		Reference
	1 ASM	101	96	197	0.4 (0.1–1.3)	0.127	0.3 (0.1–1.3)	0.336
	2 ASMs	67	75	142	0.4 (0.1–1.6)	0.201	0.4 (0.1–1.5)	0.397
	≥3 ASMs	53	44	97	0.3 (0.1–1.2)	0.098	0.3 (0.1–1.2)	0.289
Total		224	223	447

Bold font within the table body corresponds to statistically significant results.

In an exploratory post hoc analysis, the odds of epilepsy-related death occurring remained significantly increased in those who had not been seen by a neurologist in the preceding year even with analysis restricted to using only the SHARE control participants. This indicates that this variable was significant even when excluding the sample of control participants recruited directly from epilepsy clinics.

Multivariable analysis

The four variables ultimately selected for multivariable analysis are listed here (see Table 1 for expanded definitions of each variable): (i) SIMD quintile; (ii) A&E attendance or hospital admission for epilepsy/seizures; (iii) aetiology and risk factors for developing epilepsy; and (iv) CCI.

Table 4 summarizes results of the multivariable model derived from these four variables. The SEDS Score will be calculated by taking a sum of these factors, when present, weighted as⁵²:

Table 4.

Multivariable modelling

Predictor variable	Grouping	Complete case data analysis OR (95% CI)	P-value	Bootstrap P-value	Multiple imputation data OR (95% CI)	P-value	SEDS Score weighting
SIMD Quintile	Two least deprived and two middle deprived	Reference			Reference
	Two most deprived	2.2 (1.2–3.8)	0.009	0.005	2.2 (1.3–3.8)	0.003	2
A&E attendance or hospital admission	No	Reference			Reference
	Yes	4.2 (2.3–7.7)	0.000	0.001	4.7 (2.7–8.0)	0.000	3
Aetiology and risk factors for developing epilepsy	No risk factors and acquired later life risk factors	Reference			Reference
	Inherited/congenital/genetic/acquired early life	2.8 (1.4–5.6)	0.003	0.005	3.5 (1.8–6.5)	0.000	2
CCI score	0–1	Reference			Reference
	≥2	1.6 (0.6–4.3)	0.389	0.413	2.6 (1.1–6.1)	0.034	1

Bold font within the table body corresponds to statistically significant results.

• (i)One point for a CCI of ≥2: A CCI score ≥2 was given the lowest weighting of 1 as it had an OR of only 1.6 in complete case analysis (which was not significant), but an OR of 2.6 in multiple imputation analysis (which was significant). Our clinical interpretation of this, supported by prior literature,^33–35 would suggest higher CCI scores would normally be associated with increased mortality; suggesting some of the missing data within our CCI variable could have contributed to the lower complete case result. However, we applied a cautious weighting of only 1 as we did not wish to inflate the weight for this variable given the complete case result, which was the basis for primary analysis.
• (ii)Two points for having inherited/congenital/genetic aetiology or early-life acquired risk factors for developing epilepsy: assigned this weight because the complete case OR for this variable was higher than for CCI ≥2, was in a similar range to OR for the two most deprived SIMD quintiles and was lower than OR for A&E attendance or hospital admission.
• (iii)Two points for living in the two most deprived SIMD quintiles: assigned this weight because complete case OR for this variable was similar to OR for inherited/congenital/genetic aetiology or early-life acquired risk factors for developing epilepsy.
• (iv)Three points for at least one seizure- or epilepsy-related A&E attendance and/or hospital admission in the preceding year: assigned this weight because this variable had the highest complete case OR.

Therefore, each individual could have a SEDS Score of 0–8.

Model diagnostics showed evidence of internal validity for this multivariable model, with the P-values from 1000 bootstrap samples matching the significance patterns of the complete case analysis. The model was a good fit for the data (P > 0.05 for the Hosmer–Lemeshow goodness-of-fit test both using complete case and imputed data) and 30% of the variance in the outcome was explained by the predictor variables included in this model (R² = 0.303). The model demonstrated good calibration, with the predicted estimates mirroring the actual estimates closely throughout the Hosmer–Lemeshow contingency table (Table 5). In terms of discrimination, the model demonstrated a sensitivity of 72% (95% CI 65–78%) and specificity of 72% (95% CI 65–79%) in the complete case analysis, indicating that most cases and controls were identified. There was little difference in these discrimination figures when imputed values were included. Positive predictive value, negative predictive value and model accuracy were 74% (95% CI 69–79%), 70% (95% CI 64–75%) and 72% (95% CI 67–77%), respectively.

Table 5.

Contingency table for the Hosmer and Lemeshow test

Multivariable model				SEDS Score model
Study status = alive (control)		Study status = deceased (case)		Study status = alive (control)		Study status = deceased (case)
Observed	Predicted	Observed	Predicted	Observed	Predicted	Observed	Predicted
35	35	6	6	59	59	12	12
33	35	17	15	27	27	10	10
21	19	8	10	102	102	73	73
18	17	9	10	29	29	81	81
21	19	19	21	7	7	48	48
10	13	23	20	–	–	–	–
9	9	23	23	–	–	–	–
8	9	28	27	–	–	–	–
<5	4	28	27	–	–	–	–
5	3	21	23	–	–	–	–

The SEDS Scores were analysed in a multivariable model (SEDS Score model), using a score of 0 points as the reference category. This was compared to SEDS Scores in groups of 1, 2–3, 4–5 and 6–8. The results of this model are summarized in Table 6. The odds of epilepsy-related death occurring were not significantly increased in those with a SEDS Score of 1. The odds were then significantly increased in the remaining SEDS Score groups. The suggestion of a dose-response in increasing odds was seen, with OR increasing from 2.8 to 14.4, then 24.0 for SEDS Scores of 2–3, 4–5 and 6–8, respectively. However, there were also increasingly fewer cases and controls as SEDS Score increased, which acted to widen the CIs and thereby reduce confidence about the precise magnitude of odds as they increased. There was little difference between the complete case and imputed data analysis for SEDS Score modelling.

Table 6.

SEDS Score model results

SEDS Score	Alive (control)	Deceased (case)	Complete case data analysis OR (95% CI)	P-value	Bootstrap P-value	Multiple imputation data OR (95% CI)	P-value
0	59	12	Reference			Reference
1	27	10	1.6 (0.5–4.8)	0.443	0.456	1.7 (0.5–6.2)	0.412
2–3	102	73	2.8 (1.3–6.2)	0.009	0.009	3.3 (1.3–8.5)	0.012
4–5	29	81	14.4 (5.9–35.2)	0.000	0.001	14.7 (5.4–39.9)	0.000
6–8	7	48	24.0 (8.1–71.2)	0.000	0.001	29.5 (9.1–96.3)	0.000

Bold font within the table body corresponds to statistically significant results.

Model diagnostics showed evidence of internal model validity for the SEDS Score, with the P-values from 1000 bootstrap samples matching the significance patterns of the complete case analysis. The model was also a good fit for the data (P > 0.05 for the Hosmer–Lemeshow goodness-of-fit test both using complete case and imputed data): 28% of the variance in the outcome was explained by the predictor variables included in this model (R² = 0.283). The model also demonstrated good calibration, with the predicted estimates mirroring the actual estimates closely throughout the Hosmer–Lemeshow contingency table (Table 5). In terms of discrimination, the model demonstrated a sensitivity of 58% (95% CI 51–64%) and specificity of 84% (95% CI 78–88%) using complete case data. There was little difference in these discrimination figures when imputed values were included. Positive predictive value, negative predictive value and model accuracy were 78% (95% CI 72–83%), 66% (95% CI 63–70%) and 71% (95% 66–75%), respectively.

Discussion

Summary of findings

In the first reported case-control study investigating epilepsy-related death in Scotland, data from the medical records of 224 deceased cases and living controls from multiple regional centres were analysed to identify several potential risk factors for epilepsy-related death and to develop a pragmatic RPM for identifying those at the highest risk.

In terms of risk factors, univariable analysis revealed that eight variables were significantly associated with increased risk of epilepsy-related death. These were having a recent A&E attendance or hospital admission for seizures or epilepsy, a recent history of alcohol abuse, absent recent neurology review, the presence of inherited/congenital/genetic aetiology or early-life acquired risk factors for developing epilepsy, living in the two most deprived Scottish quintile areas, generalized epilepsy, a recent history of mental health problems and having a raised comorbidity burden (CCI ≥2). ORs ranged between 1.6 and 5.1 between these variables. Multivariable modelling was undertaken for the positive factors within variables CCI (≥2 assigned 1 point), SIMD quintile (two most deprived assigned 2 points); aetiology or risk factors for developing epilepsy (inherited/congenital/genetic aetiology or early-life acquired risk factors assigned 2 points) and A&E attendance or hospital admission for epilepsy/seizures (assigned 3 points). Modelling of the composite SEDS Scores derived from these points revealed three levels of risk for epilepsy-related death occurring: low risk in those with a SEDS Score of 1 (ORs 1.6–1.7), moderate risk for a SEDS Score of 2–3 (ORs 2.8–3.3) and high risk for a SEDS Score of ≥4 (ORs 14.4–14.7 for SEDS Scores 4–5 and 24.0–29.5 for SEDS Scores 6–8). Wide CIs were noted surrounding the high-risk groups, owing to increasingly fewer events as factors were combined. This serves to justify proceeding to a larger study in a different country to externally validate the model and narrow CIs further in the process.⁵⁶ The Supplementary material, section 3 illustrates the potential avenue to clinical implementation of the SEDS Score as an example risk scoring card for frontline clinicians to use (also available as an interactive prototype at: seds-tool.github.io/seds).

Interpretation

Aside from a history of inherited/congenital/genetic aetiology or early-life acquired risk factors and risk factors for developing epilepsy, the remaining three predictors used to generate the SEDS Score were potentially modifiable and serve as a target for future risk prevention strategies clinically or from a wider epidemiological perspective. Seizures are the most common neurological cause of unscheduled hospital admission in the UK.⁵⁷ While such admissions are a sign of poorer epilepsy control, they are often clinically unnecessary and typically lead to little benefit for epilepsy management.^58,59 As such, there are several studies proposing methods to reduce seizure-related admissions in people with epilepsy.^59–61 This may be achieved through, for example, intensifying scheduled specialist and community resources for people with epilepsy, or developing alternative care pathways.^61,62 Our study findings support this need as we show that seizure-related hospital admissions were the strongest predictor for subsequent epilepsy-related death. SIMD is modifiable from a population perspective, including from government incentives targeted towards high-risk groups.²⁸ Comorbidities must managed using approaches such as multidisciplinary team clinics to help reduce mortality.⁶³

While it is well established in literature that alcohol abuse, congenital or developmental epilepsies, social deprivation, generalized epilepsy and mental health problems are risk factors for all-cause mortality in people with epilepsy,^3,7,64 we show that they can be further streamlined into predicting epilepsy-related mortality specifically. Furthermore, we test the role of recent A&E attendance or hospital admission for seizures and epilepsy on predicting epilepsy-related mortality, and the role of absent neurology review within a year on this outcome. There is little study of these two variables in epilepsy mortality literature.³ Our data suggest the odds of epilepsy-related death occurring may be up to five times increased in those attending A&E or admitted to hospital for seizures/epilepsy within a year, and up to three times increased in those not reviewed by a neurologist within a year. The SUDEP and Seizure Safety Checklist is a free evidence-based tool supporting clinicians in discussing risk with people with epilepsy, with over 1000 clinician subscribers in the UK currently.¹¹ It lists risk factors linked to epilepsy mortality including (but not restricted to) SUDEP. However, it was not developed using regression modelling of the proposed risk factors against one another, but rather as expert consensus on risk factors identified in literature.¹¹ These risk factors include, for example, active seizures, generalized tonic-clonic seizures, status epilepticus, nocturnal seizures, poor medicine adherence, alcohol or substance misuse and depression.¹¹ Following on from the findings of our study, it may be reasonable to consider including A&E attendance or hospital admission within the preceding year for seizures/epilepsy, or including absent neurology review within a year in future iterations of the SUDEP and Seizure Safety Checklist.

RPMs should be both internally and externally validated before being adopted into clinical practice.^15,56 The preferred approach for internal validation is bootstrapping, which we have undertaken.^15,56 The preferred approach for external validation uses independent data from a different location than the development data to reassesses model performance.⁵⁶ This is the logical next phase of research for the SEDS Score, and could be undertaken using the national Secure Anonymized Information Linkage databank in Wales, for example.^65,66 Prognostic models can also be assessed for their impact in a clinical setting.⁵⁶ For example, mortality outcomes could be compared between patients managed by clinicians with and without access to the SEDS Score for triaging.⁵⁶ Such an approach was taken in the STarTBack trial for back pain and the results showed a significantly larger reduction in disability as well as cost savings in the group receiving prediction-model-based care compared with controls.⁵⁶

Our study develops an RPM using standard observational methods for this purpose.^15,56 It does not test interventions to alter the SEDS Score variables or overall score, nor the effect this would have on mortality. This would require a clinical trial. Therefore, we are unable to make recommendations to clinicians about how to manage their patients effectively once a high SEDS Score is identified. Such management would need to be individualized on the basis of patient circumstances, local policy and available treatments. However, by identifying the high-risk individuals, our RPM would support clinicians in triaging which patients need to be prioritized. For example, in a non-specialist setting, where such patients often first present,^57,58 a high SEDS Score could prompt re-discussion about seizure safety, signposting online resources for support (such as www.epilepsy.org.uk/info) and rapid-access epilepsy clinic referral (rather than reserving such clinics for first-fit patients alone). The potential impact of such approaches is well recognized. For example, the asthma deaths review demonstrated that many deaths were likely to be preventable simply through better proactive clinical management, such as arranging for patients to be seen promptly after a hospital admission and through following clinical guidelines.⁶⁷ In a specialist setting, a high SEDS Score could prompt referral for epilepsy surgery earlier than would have otherwise been considered, discussion in a multidisciplinary team meeting, further medication reviews or implementation of seizure alarms and nocturnal supervision.

Our previous study corroborated a heavy burden of seizure-related A&E attendances and hospital admissions in the period leading up to epilepsy-related death, with a substantial number of these patients lacking any recent neurology review or follow-up.⁵ Such A&E attendances and hospital admissions therefore represent missed opportunities for creating effective pathways to specialist care, a problem also highlighted in England.⁵⁸ As such, it would be important for the SEDS Score to be available to non-specialist doctors including general practitioners, general physicians and emergency care doctors, and for it to be designed in such a way that these non-specialists could use the tool. This includes basing it on variables likely to have little missing data in these non-specialist medical records. Support from local and national charities and organizations would help with disseminating the model to non-specialist areas, as is being done by SUDEP Action for the SUDEP and Seizure Safety Checklist.¹¹ We would also apply to have the SEDS Score included in free online clinical scoring tool databases, such as MDCalc (www.mdcalc.com), meaning it would be easily accessible globally.³⁶

The methods used in our study can be translated to any country with unique patient identifiers and linked administrative data or electronic health records. For example, all patients using NHS services in England benefit from a unique patient identifier (NHS number). A study was recently published evaluating a nationwide cohort of >54 million people in England using NHS number-linked electronic health records and administrative data.⁶⁸ Studies from outside of the UK have used a similar approach, including in the USA,⁶⁹ Canada,⁷⁰ Sweden⁷¹ and South Africa.⁷² Our methodology is focused on developing a generalizable RPM, selecting variables that are clinically and statistically appropriate to include, with little overall missing data. Consequently, the model identifies four risk factors that should be readily available within patient medical records and could even be obtained from administrative data, facilitating clinical implementation or research in other regions in future. Each variable has been successfully studied in other global regions using electronic health records or administrative data.^3,57,73–76

Study limitations

A limited sample size allowed development of only a four-variable RPM. This is, however, a similar model size to the three-item SUDEP-3 inventory—an RPM for SUDEP that was developed from a seven-item inventory (SUDEP-7).^9,10,12 While a smaller sample allowed us to investigate each medical record in greater detail, it meant we were unable match cases and controls by centre, instead pooling them all into one Scottish centre. However, both the cases and controls were mainly recruited from Scotland’s Central Belt region (Glasgow, Ayrshire, Falkirk, Edinburgh, Lothian and Fife; Fig. 1).

Controls were drawn from those signing up to SHARE and those attending epilepsy clinics, yet no such selection process applied to cases. This may have created a selection bias, especially towards controls being generally healthier than cases. This is unlikely to have influenced the conclusions we drew from SEDS Scoring as the OR figures for this were markedly high, and would therefore be unlikely to have been entirely attributable to differences in baseline characteristics of the groups. Furthermore, SHARE provided evidence their study population is representative of the general Scottish population in terms SIMD and ethnicity (Supplementary material, section 4) with percentage difference between the two being no more than 2% in each SIMD decile: 4% and 7% of the general population and SHARE population, respectively, are from ethnic minority backgrounds.

Although our study attempted to extract data on several other potentially modifiable risk factors for epilepsy-related death including seizure frequency, timing, seizure alarm use and a history of status epilepticus, large amounts of missing data in the medical records precluded analysis of these. We were strict to exclude from further analysis any variables with ≥30% missing data because including variables with large amounts of missing data would not only substantially reduce the model’s predictive power,⁴³ but also it would curtail its clinical utility. This is because end users might also struggle to find these variables within their medical records. The resultant predictive tool should, therefore, be more widely applicable in a range of clinical environments. Not all variables within the model would need to be modifiable for the model to be effective at identifying high-risk groups and allowing triage to effective management that might be independent of the specific risk factors identified. For example, within the widely used A—Age, B—Blood pressure, C—Clinical transient ischaemic attack features, D—Duration of symptoms and D—Diabetes (ABCD2) score to identify those at high risk of stroke following transient ischaemic attack,⁷⁷ only diabetes and blood pressure are modifiable. However, the tool is effective at helping prevent stroke because it identifies those at high risk effectively and therefore allows earlier implementation of stroke prevention strategies beyond simply controlling blood pressure and diabetes, including lipid profiling, cardiac monitoring, carotid imaging and considering dual antiplatelets.⁷⁸

Our model was a good fit for the data statistically, as demonstrated by a P-value >0.05 on the Hosmer–Lemeshow goodness-of-fit test. The absolute figures for discrimination through sensitivity and specificity estimates were moderate, ∼70%. This highlights some of the complexities of regression model interpretation, which measures predictive ability in several different ways. Discrimination is related to correctly separating individuals with and without disease. Therefore, it is more important in diagnostic accuracy studies, unlike ours. Calibration measures the agreement between observed and predicted risk, and is therefore more important in prognostic studies such as ours.⁷⁹ Calibration is demonstrated highly in our model by the predicted estimates mirroring the actual estimates closely throughout the Hosmer–Lemeshow contingency table (Table 5). This is reassuring for the clinical conclusions drawn from this model.

Finally, the study was designed to identify those at increased risk of epilepsy-related deaths as a group, but made no attempt to risk stratify according to the type of epilepsy-related death specifically, e.g. SUDEP, drowning, accidents or suicide. A future study could attempt to further stratify by type of epilepsy-related death following on from our demonstration of increased risks for the overall group.

Conclusion

We propose the SEDS Score as a potential tool for identifying adults at increased risk of epilepsy-related death based on their recent attendance history to hospital for seizures or epilepsy, area of deprivation, epilepsy aetiology profile and their serious comorbidity burden. This 4-point score needs external validation in another healthcare system.^15,56 The SEDS Score has the potential to be applied clinically to help streamline frontline care for people with epilepsy and to inform clinical discussions with patients about risk. We describe additional risk factors to consider as part of the overall analysis of epilepsy-related mortality risk in people with epilepsy, including recent review by a neurologist, alcohol abuse and mental health problems. There is the potential for a larger study to try to incorporate these into future iterations of the SEDS Score and for them to be included in other tools such as in the SUDEP and Seizure Safety Checklist.¹¹ The missing data patterns in our study for several other potentially modifiable risk factors for epilepsy-related indicate the need for better clinical documentation within medical records, perhaps with use of seizure history collection proformas in non-specialist areas. There is also a need for more granular administrative healthcare data sources to allow such variables to be studied at scale.

Funding

This work was charitably supported by Epilepsy Research UK (R44007) and the Juliet Bergqvist Memorial Fund. The funding sources played no role in the study design, data collection, data analysis, interpretation of results or the preparation and decision to submit the article for publication. The researchers remain independent from the funders.

Competing interests

The authors report no competing interests.

Supplementary material

Supplementary material is available at Brain online.