Visual detection of seizures in mice using supervised machine learning

Summary

Seizures are caused by abnormal synchronous brain activity. The resulting changes in muscle tone, such as twitching, stiffness, or jerking, are used in visual scoring systems such as the Racine scale to quantify seizure intensity. However, visual inspection is time consuming, low throughput, and partially subjective, and there is a need for scalable and rigorous quantitative approaches. We used supervised machine learning approaches to develop automated classifiers to predict seizure severity directly from non-invasive video data. Using the pentylenetetrazole (PTZ)-induced seizure model in mice, we trained video-only classifiers to predict ictal events and combined these events to predict composite seizure intensity for a recording session, as well as time-localized seizure intensity scores. Our results show that seizure events and overall intensity can be rigorously quantified directly from overhead video of mice in a standard open field using supervised approaches. These results enable high-throughput, non-invasive, and standardized seizure scoring for neurogenetics and therapeutic discovery.

Graphical abstract

Highlights

• •A supervised ML approach to detect seizure behaviors in open-field mouse videos
• •Trained behavioral classifiers to successfully detect specific seizure behaviors
• •Prediction of session-level maximum Racine score using a composite seizure scale
• •Prediction of time-localized seizure intensity in time-series data

Motivation

Epilepsy, a complex neurological disorder characterized by seizures, often co-occurs with other neurodevelopmental, psychiatric, and cognitive disorders. While electroencephalogram (EEG) is the gold standard for diagnosis, it is invasive, is expensive, and has limited throughput. Using machine vision, we overcome these barriers and develop a high-throughput, objective, sensitive, and economical alternative using video data. We accurately detect individual seizure behaviors using supervised machine learning and use them to construct a seizure intensity scale. Our methods can greatly enhance mechanistic and preclinical studies of epilepsy in the laboratory mouse.

Sabnis et al. use non-invasive video recordings of mice to develop a method using supervised machine learning to automatically quantify seizure severity. This approach provides a scalable and objective tool for seizure assessment, augmenting manual scoring and offering a more precise method for drug discovery and disease modeling in preclinical research.

Introduction

Epilepsy is a broad collection of neurological disorders defined by the occurrence of spontaneous, unprovoked seizures, which are often comorbid with other neurological disorders, including autism spectrum disorder, anxiety, and intellectual disability.^1,2 Seizures are caused by abnormally synchronous brain activity that can result in changes in muscle tone, such as twitching, stiffness, limpness, or rhythmic jerking, and loss of consciousness.¹ Genetics has a substantial influence on epilepsy, including both common variants that modify risk for common epilepsies³ and monogenic etiologies that cause severe developmental and epileptic encephalopathies.^4,5,6,7,8 With these findings, the potential for genetically informed, precision therapy is beginning to be realized.^9,10 However, despite high heritability and the many known monogenic genetic etiologies, there is large unmet medical need in the treatment of patients; nearly one-third of patients are refractory to all existing medications, and there are no known disease-modifying therapies.¹¹ Thus, mouse models of epilepsy are a critical experimental resource, given our ability to control mouse genetics and the high homology between mouse and human neural circuits underlying seizures. However, maximal use of mice for quantitative genetics and preclinical modeling requires rigorous, quantitative, high-throughput seizure phenotyping.

Clinically, seizures are diagnosed using an electroencephalogram (EEG), which measures the electrical activity in the brain and can detect abnormal synchronous activity as large rhythmic patterns in brain activity. However, the use of EEG in preclinical studies has significant limitations. In particular, EEG is invasive, expensive, and low throughput, requiring surgery to implant a monitoring device. However, convulsive seizures in animal models have behavioral manifestations that correlate with underlying brain activity, and therefore with seizure severity, although these correlates can depend on species and etiology.^12,13,14 Thus, there are several options for scoring seizure severity that differ in their precision and quantitative rigor. The simplest approach is to observe seizure events, either in real time or on video, and score them according to a standard rubric, such as the Racine scale.^12,13 This is the lowest-cost option but is not high throughput enough for long-term monitoring and is partially subjective. In contrast, invasive EEG monitoring yields direct measurements of brain activity and has been the gold standard for seizure studies, but these systems always require invasive long-term implantation of electrodes into or near the brain, which carries a non-trivial risk of harm to the animal via infection and can alter motor behavior and circadian rhythms.^15,16,17 Furthermore, depending on the recording system (e.g., tethered vs. wireless telemetry), the equipment itself may limit the other types of behavioral screening that can be performed in those animals. This can be a significant limitation for studies that seek to correlate seizure activity with other behavioral changes to study, for example, the well-established relationship between epilepsy and autism.¹ Low-profile, wireless systems impose the fewest restrictions on other behavioral assays (see Lundt et al. and references therein¹⁵) but are also the most expensive, which also limits throughput for most investigators. An intermediate alternative is to use a minimally invasive or non-invasive surrogate signals, such as accelerometry,¹⁸ to allow much higher throughput and lower costs, along with rigorous signals for reproducible seizure detection. In all the aforementioned approaches, seizures are scored by an expert analyst, perhaps after a selection of putative seizure video clips by auxiliary signal processing. However, to date, there have been no approaches to directly score seizure intensity in animal models from video data directly.

Video is critical in human epilepsy diagnosis, particularly video encephalography, as it can sometimes identify motor seizures with clear characteristics.¹⁹ Recent advancements in computer vision (CV) have shown increasing promise as efficient, low-cost tools for video seizure detection and classification in humans.²⁰ Automated analysis of video-recorded seizures in humans is needed to support detecting, identifying, and understanding their temporal evolution.²¹ CV methods have been applied to detect and classify various types of human seizure semiology,²⁰ including major motor seizures,²² myoclonic jerks,²³ clonic and tonic-clonic seizures,²⁴ hyper-motor seizures,²⁵ focal impaired awareness seizures,²⁶ and even differentiating epileptic seizures from psychogenic non-epileptic seizures.²⁷ Techniques used for human video analysis include hand-designed features like optical flow or motion strength; traditional machine learning (ML) classifiers; and, increasingly, deep learning models like convolutional neural networks (CNNs), long short-term memory networks (LSTMs), 3D CNNs, and transformers.²¹

Similar to humans, seizures in animal models have behavioral manifestations that correlate with underlying brain activity. Recent advancements in animal behavior annotation have resulted from applications of new methods from the statistical learning and computer science fields for biological tasks. These combine machine learning and CV (machine vision) and are often referred to as “artificial intelligence.”^28,29,30,31 We have successfully applied these methods for tracking,³² action detection,³³ frailty,^34,35 nociception,^36,37 sleep,³⁸ and gait and posture analysis.³⁹ Furthermore, we have built a shared open platform for end-to-end behavior annotation.⁴⁰ Despite the advances in the human field in seizure detection, the methods are not necessarily readily adopted for seizure detection in animal models. The repertoire of human movement patterns and facial expressions is extensive, complex, and heterogeneous compared to those studied in animal models.¹⁹ Detecting lower intensity seizures in rodents using video alone remains challenging because their motion is similar to everyday activities like walking or grooming. Given these limitations and advances in machine vision approaches, we explored whether a video-only CV approach could be a solution for non-invasive seizure scoring in mice. There has been an increasing interest in leveraging machine vision-based analyses to identify seizures through videos in animal models. More specifically, EPIDetect⁴¹ integrates action recognition and object detection networks to detect convulsive seizures in chronically epileptic mice from home cage videos. Mullen et al.⁴² propose a multi-modal machine learning system combining video (using a transformer-based model like VideoMAE) with ECoG and Piezo data for seizure detection in rats. Diaz-Arce et al.⁴³ describe a semi-automated method using motion and spectral analysis of video for convulsive seizure detection in mice. A common thread underlying these works is that they all focus on facilitating binary classification: distinguishing between extreme (high intensity) and non-seizures. Specifically, EPIDetect⁴¹ and Diaz-Arce et al.⁴³ focus on detecting convulsive seizures. Convulsive seizures include a tonic phase, marked by muscle stiffness, followed by the clonic phase, characterized by rhythmic jerking movements. Convulsive seizures are readily observable and considered one of the most severe forms of seizures, surpassing the earlier stages of the Racine scale (score > 4). Mullen et al.⁴² focus on classifying status epilepticus, which is said to have occurred when rats experience continuous tonic-clonic seizures or intermittent class 4 or 5 seizures without recovery of normal behavior. We take a different approach by focusing on directly scoring seizure intensity on the ordinal Racine scale in animal models from video data using supervised machine learning (ordinal multiclass classification), which is important for evaluating the efficacy of anti-seizure drugs in preclinical studies. Another important difference is that these methods use raw video files to detect seizures in individual frames. In contrast, our approach first involves extracting pose key points in each frame using deep neural networks as intermediate representations.³⁹ Our pose-based approach offers the flexibility to use the identified poses to train classifiers for multiple behaviors, and we have used this approach to train classifiers for seizure-related behaviors. Gschwind et al. have recently used an unsupervised approach to detect behavior anomalies due to seizures.⁴⁴ Their method, MoSeq,⁴⁵ automatically classifies video into discrete sequences of “syllables,” each of which is a short behavioral pattern (∼300 milliseconds). Interestingly, while patterns in the MoSeq syllables did correlate with the severity of seizure events, the classification performance was modest (F₁ = 0.39 ± 0.13), which they attributed to several factors, including that MoSeq syllables correspond to very short behaviors lasting only a few hundred milliseconds, while human scoring is based on highly specific behaviors lasting much longer.⁴⁴ Thus, the most robust changes observed using unsupervised methods are interictal rather than ictal events. These findings, combined with our prior work utilizing supervised and heuristic classification of rodent behaviors, such as grooming,³³ and combining multiple behaviors to score higher-level constructs such as frailty^34,35 and nociception³⁷ strongly suggest that the behavioral manifestations of frank seizures should be robustly detectable from video data. This is a significant gap because long-term monitoring of large cohorts may be critical for future preclinical studies of epilepsy.

In this study, we used a supervised learning approach to detect seizures in mouse behavioral videos. The community standard seizure severity score for preclinical models is the Racine scale,^12,13,14 which is an ordinal scale from one to seven denoting progressively more pronounced behavioral manifestations of seizures, from whisker trembling (Racine score = 1) to a tonic-clonic seizure followed by tonic extension and possibly respiratory arrest or death (Racine score = 7).¹³ To enrich for seizure events, we induced seizures using the convulsant pentylenetetrazole (PTZ), a gamma-aminobutyric acid receptor antagonist, which has been commonly reported in the literature.^13,46,47 We hypothesized that we could train robust, automated classifiers to predict seizure severity using video data alone. Overall, we show that supervised methods can accurately detect specific seizure events and that these events can be combined to create a seizure intensity score.

Results

The design of our experiment is illustrated in Figure 1A. To create robust training data with many seizures, we challenged our mice using PTZ at varying doses. We monitored mice in an open field during PTZ-induced seizures that were scored according to the Racine scale by an expert observer.¹³ This created our human-annotated training data for seizure intensity. In order to automate the seizure detection, we used the following steps. First, we extracted pose-estimation and body-segmentation time series from raw behavioral videos using existing deep neural network models.^32,39,40 Second, from the extracted time series, we built classifiers for six characteristic behavioral seizure manifestations: freezing (i.e., behavioral arrest, score of 1), Straub tail (score of 4), leg splaying (score of 4), circling (score of 5), and side seizure and wild jumping (score of 6). Third, and finally, we summarized the classifier outputs as a set of features (e.g., “time spent in side seizure”) that were used in an ordinal regression model to predict seizure severity. To detect seizure intensity, we carried out two analyses. In a first analysis, we predicted the seizure severity over the course of an entire session. In a second analysis, we predicted time-localized seizure intensity.

Figure 1.

Experimental design, manual Racine scores, and behavioral features for seizure classification

(A) Visual representation of the experimental workflow.

(B) A bar plot summarizing the number of animals per strain, sex, and PTZ dose.

(C) A violin plot summarizing the human-annotated data. Racine scores range from −1 to 7.

(D) A histogram distribution of the scores (ground truth) assigned by the tester. The colors indicate the Racine intensity group from Table 1.

(E) Pearson correlations of features’ summary statistics with seizure (see Table 1).

(F) A correlation matrix showing the estimated bi-variate correlations across features. See also Table 1, Figure S1, and Table S1.

Manual seizure scoring

We used both male and female C57BL/6J (B6J) and C57BL/6NJ (B6NJ) mice with 4 doses of PTZ—0 mg/kg (vehicle), 40 mg/kg, 60 mg/kg, and 80 mg/kg (Figure 1B). These doses, strains, and methods were based on a previously published paper.¹³ Mice were allowed to acclimate for at least 15 min in an open field arena before being injected with PTZ and returned to the open field (Figure 1A). Mice were observed by a single trained observer for a maximum of 20 min and given a Racine score. JAX Institutional Animal Care and Use Committee protocols dictate that any mouse observed to have tonic extension or maintained a Racine score greater than 3 (neck jerks) for 1 minute must be immediately euthanized. A mouse is also removed from the open field and euthanized following the first signs of tonic extension or extreme seizure activity. Following video capture, data were processed using machine learning and CV methods.

Racine scale scores (1 through 7) were assigned to behaviors during the session, with a score of −1 assigned to baseline, non-seizure activity by the observer. When mice were presented with multiple Racine scores, the highest score was used as an overall score (all Racine scores are in Table S1). We asked two independent scorers to independently assign Racine scores to each video to improve and robustify our training data labels. We blinded the scorers to each other’s evaluations and used the average of the two scores as the final score for each video. We weighted the disagreements proportionally based on how far apart the ratings were and found that the agreement between the two raters was substantial, κ = 0.72 (Cohen’s kappa⁴⁸), and greater than would be expected by chance (z = 8.83, p < 1e-16) (Figure S1C). The distribution of the average Racine scores across strains and aggregate distribution showed that, across both strains of mice, the Racine score increased as a function of dose (Figures 1C and 1D, Figure S1B). We binned the Racine scores into 4 categories of seizure intensity for downstream modeling (no, low, medium, and high, Table 1; Figure 1D). We observed all modalities of seizures (Table S1), and, after taking the maximum Racine score for each animal, we expected that it would be difficult to distinguish the medium group from the rest due to the uneven distribution of Racine scores (Figure 1D) and thus seizure levels. This manual Racine scoring analysis indicated that the data were of high quality and suitable for modeling ictal events.

Table 1.

Racine scale scores and behaviors

Score	Description	Seizure intensity
−1	normal baseline	no
0	whisker trembling	low
1	sudden behavioral arrest	low
2	facial jerking	low
3	neck jerks	medium
4	clonic seizure (sitting)	medium
5	clonic, tonic-clonic seizure (lying on belly)	high
6	clonic, tonic-clonic seizure (lying on side) and wild jumping	high
7	tonic extension, possibly leading to respiratory arrest and death (laying down with forelimbs and hindlimbs outstretched)	high

Seizure prediction

To predict seizure events, we developed classifiers for six seizure-associated behaviors: Straub tail, leg splaying, side seizure, wild jumping, freezing, and circling (described in Table 2).^13,49,50,51 Examples of these behaviors can be seen in Video S1. Straub tail behavior, related to Table 2 on Ictal behavior descriptions and classifier accuracy, Video S2. Leg splaying behavior, related to Table 2 on Ictal behavior descriptions and classifier accuracy, Video S3. Side seizure behavior, related to Table 2 on Ictal behavior descriptions and classifier accuracy, Video S4. Wild jumping behavior, related to Table 2 on Ictal behavior descriptions and classifier accuracy. These behaviors were chosen based on whether the behavior was easy to distinguish on video and whether there were enough clear instances of that behavior to train a classifier. We used JAX Animal Behavior System (JABS)⁴⁰ to classify Straub tail, leg splaying, side seizure, and wild jumping events. The JABS active learning module uses pose-based features extracted from the videos to classify behaviors.^40,52 We used our HRNet-based pose-estimation neural network³⁹ to estimate the location of twelve key points in the videos and computed the number of per-frame and per-window features. Our JABS framework then computes informative features like the distance between key points, linear and angular velocity between key points, and several others, which we use as input for training these classifiers using either an XGBoost or a random forest model. We also incorporate temporal information from the videos by computing window features that include information from w (window size) frames on each side of the current frame. Next, we trained multiple classifiers and evaluated how the classifier’s performance varied with the features’ choice of three window sizes (w = 3, 10, and 30). Our synthetic experiments with three different window sizes showed that the classifiers’ F1 scores improved marginally with decreasing window sizes for leg splaying and Straub tail classifiers but improved substantially for wild jumping and side seizure classifiers. Overall, all seizure-associated behavior classifiers had good performance based on F₁ (Table 2). To classify freezing and circling, we define two heuristic classifiers (Table 2). Next, we extracted summary statistics for each behavior, including total behavior time, proportion of behavior time in trial, number of behavioral bouts, and average length of behavioral bouts similar to our previous applications³⁷ (fully described in Table S3). Combined with previously used open-field features, we used 38 total features.

Table 2.

Ictal behavior descriptions and classifier accuracy

Type	Behavior	Description	F1
JABS	Straub tail	tail stiffens, rises quickly, may jerk down	0.87
JABS	leg splaying	hind legs straighten, visible, wide stance	0.92
JABS	side seizure	nouse falls to side, shakes/seizes	0.81
JABS	wild jumping	mouse runs and jumps quickly around arena	0.86
Heuristic	freezing	no movement for at least 3 s	–
Heuristic	circling	tight circular locomotion, followed by another within 6 s	–

Next we determined if these features are useful for seizure intensity classification. As a preliminary test for association to seizure intensity, we correlated these summary statistics with seizure groups and PTZ dose for all test groups, including sex and strain stratification (all mice, all females, all males, all B6J, all B6J females, all B6J males, all B6NJ, all B6NJ females, and all B6NJ males [Figures 1E and S1A]). We found that certain behaviors strongly correlate with PTZ dose and seizure intensity groups. For example, the amount of Straub tail behavior had consistently high positive correlation with dose across all strains and sexes. Straub tail and side seizure behavior also showed positive correlations with dose, but with a less pronounced effect for B6NJ. Certain seizure behaviors such as wild jumping and circling were rare in our dataset and did not show a strong correlation with dose or seizure intensity (Figure 1E, pink and yellow), mainly because there were very few instances of these in our data. Freezing behavior is quite frequent; however, it occurs in all three PTZ doses and then does not show a strong correlation with dose or seizure intensity (Figure 1E, navy blue). We also calculated the proportion of time across all total behaviors relative to the time in the arena for each mouse, which showed a significant positive correlation with the dose for all groups except B6NJ males (Figure 1E). Thus, the correlation heatmap for behavior summary statistics with highest Racine score is very similar to the one for PTZ dose (Figure S1A). In contrast to seizure-specific behaviors, routine open-field behaviors, such as activity and anxiety measures, correlate less strongly with seizure intensity or the dose of PTZ (Figure 1E, dark green).

The correlation matrix among all features shows multicollinearity, with high correlations among features derived from a single behavior (e.g., leg splaying) and substantial correlation between different behaviors (Figure 1F). Thus, these behaviors co-vary across animals in structured ways, as expected for highly stereotyped seizures. This can cause model parameters estimated from the data to be unreliable and sometimes even unidentifiable. We used regularization to mitigate issues such as overfitting, instability of parameter estimates, and poor generalization in our models.^53,54

Visual detection of seizures

We used the seizure behavior classifiers to visualize individual seizure events over time with ethograms (Figure 2). We plotted all seizure behaviors for each mouse at each dose (Figure 2A) and for each behavior at each dose (Figure 2B). Since mice were removed if they had severe seizures or a period of mild seizures, we mark the end of each animal’s test (black points). We observe that, as the dose increased, mice were removed from the arena in less time, which indicates earlier onset of severe seizures. Most of the animals in the 80 mg/kg dose were in the arena less than 4 min before being euthanized due to severe seizures. We also observe that mild events such as freezing and Straub tail are observed in the 40 mg/kg dose. At 60 and 80 mg/kg, we observe all classes of seizures, including wild jumping, side seizures, circling, and leg splaying. Overall, these results visually show the onset, frequency, and euthanasia of each animal over time. It also demonstrates that mild and severe seizure events are captured by distinct behaviors—freezing and Straub tail behaviors vs. leg splaying, circling, side seizures, and wild jumping, respectively. In a separate experiment, we added a subthreshold PTZ dose (20 mg/kg) group (n = 24) to test if we can resolve seizures below the threshold for detection by an observer. Our analysis established that our features capture differences that distinguish animals given a 20 mg/kg dosage from animals in the 0 and 40 mg/kg groups (Figure S1D). This validates our approach of scoring individual seizure events using supervised machine learning.

Figure 2.

Ethograms of seizure behaviors

Time 0 represents time when mice are placed in open field arena after injection with vehicle (0 mg) or increasing dose of PTZ.

(A) Each dot represents the occurrence of a behavior (all colors except black) for a mouse (each row on y axis) at a certain time point (x axis) during the duration of the experiment before the mouse is removed (black dots).

(B) Same as (A) except all behaviors performed by every animal are arranged by behavior rather than animal.

An interpretable composite seizure scale

Next, we sought to create a composite scale using the 38 features. Composite scales can be a powerful method to condense multiple features into one metric that can be applied toward multiple tasks. They are frequently used in biological applications⁵⁵ and can be constructed using various regression modeling approaches. A composite seizure scale offers several advantages: (1) easy to construct once the model is fit, (2) it is highly interpretable, and (3) it can easily be implemented with new data. We have applied it previously for frailty indexing and pain scoring.^34,35,37

We first established that our features capture differences that distinguish animals given different doses and belonging to different seizure groups based on their Racine scores (Figures 3A and 3B). Linear discriminant analysis indicated that both dose and seizure intensity could be distinguished. We find that the medium- and high-intensity Racine groups and the 60 and 80 mg/kg PTZ dose groups are challenging to distinguish (Figures 3A and 3B, green and purple). This indicates that even the 60 mg/kg dose is likely saturating our dose-response curve.

Figure 3.

Composite seizure scale for seizure intensity

(A) We used linear discriminant analysis (LDA) to quantitatively distinguish across animals (A) belonging to different Racine groups, (B) administered different dosages.

(C) We have plotted the features’ contributions to the seizure intensity scale. The vertical bars represent the standard errors of the mean (SEM) across cross-validation folds.

(D) We created a univariate seizure scale by training ordinal regression models to create a latent continuous scale with segments corresponding to different seizure intensities. The resulting distribution of mice along the scale shows an ideal separation of seizure intensity in each segment. See also Figure S1.

We trained a supervised latent variable model to construct a composite scale that utilizes a data-driven approach to predict seizure intensity. Specifically, we used an ordinal regression model to estimate the probability of our features classifying each animal as belonging to no, low, medium, or high seizure groups. We took several steps to prevent the models from overfitting the data and overemphasizing patterns that are not reproducible, thereby preventing models from memorizing. First, we adopted the elastic net penalty⁵⁴ for regularization to address multicollinearity (Figure 1F), perform feature selection, and improve prediction accuracy. Second, we used a cross-validation (CV) approach to build and validate our predictive model. We used 10-fold nested CV to tune the elastic net regularization hyperparameter (inner fold) and get an unbiased estimate of the tuned model’s out-of-sample misclassification rate (outer fold). Averaging over the folds gave us a misclassification error of 0.28 ± 0.07 (see Table S2 for comparison with other methods). For comparison, a classifier that guesses seizure intensity would have a misclassification error of 0.75. Third, we performed a synthetic experiment that added noise from two known distributions to understand the interplay between noise and the downstream impact of using features reconstructed from noisy PCs on seizure class classification. More specifically, we added noise from two continuous distributions, i.e., a Gaussian distribution $\mathcal{N}\left(0,{\sigma }^2\right)$ and a uniform distribution $\mathcal{U}\left(-\sigma ,\sigma \right)$, to test the effect of increasing σ on seizure classification. As before, we used 10-fold nested CV to obtain a misclassification error across each fold. Next, we used 10-fold CV to determine the misclassification error across each fold. We have plotted the mean CV error across all folds (Figure S1E). The performance of the classification model (y axis) with no error (x axis) indicates the model’s baseline performance. The misclassification error (y axis) increases as the error is taken from distributions with increasing variability (x axis), albeit with a not substantial error, and is much lower than the misclassification error due to random guessing (dashed line, Figure S1E). Fourth, to further assess the stability of the feature contributions, we computed the feature coefficients for models trained on datasets with 1-fold left out. We then averaged these coefficients across all folds. We plotted them along with the standard errors across folds and found that the model’s feature coefficients are stable across folds, indicating that the model does not overfit the data. These coefficients have a natural interpretation; a positive (or a negative) coefficient shifts probability toward a higher (or a lower) seizure group (Figure 3C). Indeed, we see positive coefficients estimated for both Straub tail (number of bouts and average bout length) and leg splaying (number of bouts); even side seizure has an estimated positive coefficient. We used the feature weights estimated from the model to construct a composite seizure scale by taking a weighted sum of the features using the estimated model parameters as weights—the scale distinguished animals with no and low seizures from medium and high seizures. However, distinguishing animals with no from low seizures and medium seizures from high seizures was more challenging (Figure 3D). Based on these analyses, we concluded that our model is robust and generalizes reasonably well to unseen data. Further, we also assessed the model’s performance when we did not categorize the Racine scores into four groups (No, Low, Medium, and High). Increasing the number of classes from four to six introduces a greater risk of class imbalance, increases the complexity of the decision space, and typically reduces class separability, all contributing to a higher misclassification error rate. Indeed, the misclassification error rate is 0.28 for four groups vs. 0.60 for six groups (Figure S1F).

Temporal detection of seizure intensity

Our results mentioned earlier quantify seizure severity by considering a complete seizure event. However, seizures are caused by a sudden and uncontrolled burst of synchronous electrical activity in the brain and generally occur spontaneously, evolve in time, and, in many rodent models, can be rare events.^1,56,57 Thus, beyond scoring complete events, which can last several minutes, it is desirable to detect the temporal variation in seizure intensity as a function of time, both to identify seizure events in potentially long recordings and to track the seizure evolution. Thus, we sought to build a temporally resolved seizure intensity scale that operated on a minute of data that could potentially be used for time-localized seizure detection (Figure 4A). Using the PTZ dataset, we trained an ordinal linear mixed model⁵⁸ to predict the overall seizure intensity based on 1-min bins of data from each mouse. Our underlying assumption was that characteristic 1-min bins would correlate with the evolution of seizure intensity.

Figure 4.

Temporal analysis of seizure intensity

(A) Seizure intensity score is assigned on one-minute video segments. An ideal seizure detector will identify rare time bins when seizures occur (e.g., red frame grab). We use a weighted sum score of features at each 1-min interval for this analysis.

(B) Instantaneous seizure intensity estimates across 20 1-min intervals were estimated using supervised hierarchical regression models. The thin lines correspond to individual mice seizure profiles while the bold lines are the maximum seizure intensity for each seizure group. Also see Figure S1.

(C) Validation of the instantaneous seizure intensity model using leave-one-animal cross-validation (LOOCV) where we estimated the maximum seizure intensity of held-out mice (y axis) and compared it to their true Racine groups (x axis).

(D) We select representative animals from each group and overlay annotated behaviors on top of predicted behavior profiles.

(E) Temporal analysis correlates with intense seizure events. We count the number of bouts (color) of annotated behaviors (y axis) in each seizure group (x axis) for the animals in (D). See also Figure S1.

We undertook quantitative and qualitative approaches to validate our model on previously unseen data. We removed one animal to perform leave-one-out cross-validation and validated the model’s performance on the unseen animal. In particular, we regressed the seizure intensity on the features by treating the repeatedly measured features across each minute as a random effect and estimated the model parameters from the data. We then computed the predicted seizure intensity score for the left-out animal as a function of time, attempting to approximate a behaviorist labeling the videos and assigning a Racine score at each 1-min bin (Figure 4B; Figure S1G). We validated the model by calculating the predicted temporal profile’s mean, median, and maximum measures. As with the composite scale, we found that the three summary measures captured group differences between the seizure intensity groups (Figures 4C and S1H). In other words, the average temporal profiles recapitulated the overall group differences. As before, the predicted temporal profiles did not distinguish as strongly between medium and high seizure groups as they did for the other groups.

The maximum measure identifies the minute bin at which we predicted each left-out animal to have the maximum score. We picked one animal from each seizure intensity group with the highest predicted score and overlaid the behavior classifier predictions (Figure 4D). We found that the predicted temporal profiles coincided with the co-occurrence of seizure-specific behavioral bouts (Figure 4E), demonstrating a qualitative agreement between the seizure intensity profiles and major seizure events.

Discussion

In this study, we used supervised machine learning to develop a new model for automated seizure scoring using open-field video monitoring. Specifically, we perform three levels of analysis. We are able to detect specific seizure behaviors. We use these and other features to predict the session-level maximum Racine score. Finally, we carry out time-localized seizure intensity scoring to detect seizure intensity in time-series data. Our approach is non-invasive, reproducible, and scalable.

The virtues of purely video-based monitoring are appreciated by the epilepsy field as an alternative to cumbersome gold standard EEG recordings. Moreover, outside of epilepsy, it is appreciated that seizures are a significant class of adverse events leading to high attrition in drug development outside of seizure disorders, and automated video-based seizure scoring is recognized as a potential solution.^59,60 This approach has been applied toward detection of seizures in humans^{22,61,62,63,64,65,66,67} and is recognized as a critical frontier for human phenotyping.¹⁹ Despite the advances in the human field in seizure detection, challenges exist in human and rodent domains, such as distinguishing target behavior from normal activity and handling environmental variability (lighting, occlusion, and multiple subjects). The nature of these variabilities differs between clinical epilepsy monitoring units, home settings, and laboratory rodent cages. These challenges highlight the need for new methods and datasets specifically tailored to animal models’ unique behavioral characteristics and recording environments. At the same time, visual detection of seizures in rodents has seen an increasing interest in using machine-vision-based approaches. There are several methods, which either employ a multi-modal approach combining data from various sources like ECoG, Piezo sensors, and video⁴² or focus on general motion analysis and detection of convulsive seizures using action recognition and object detection⁴¹ or rely on semi-automated software for seizure detection.⁴³ However, they all focus on binary classification seizure vs. no seizure, whereas we focus on classifying seizure intensity on an ordinal scale, i.e., classifying seizures at three varying levels of intensity: non-seizure (Racine score < 0), low (0 ≤ Racine score ≤ 2), medium (3 ≤ Racine score ≤ 4), and high (≥5) (Table 1). In addition, we expand this to predict the Racine scores on the original scale, further increasing the number of classes from four to more than 4. This experiment introduces a greater risk of class imbalance, increases the complexity of the decision space, and typically reduces class separability, all contributing to a higher misclassification error rate. However, it also helps detect seizure intensities at a finer level, which is important for evaluating the efficacy of anti-seizure drugs in preclinical studies. Ryait et al.⁶⁸ trained a deep neural network that can score behavioral impairments in rodent stroke models. They showed how a data-driven approach, i.e., processing the video frames through a CNN for feature extraction and then analyzing them temporally via a recurrent neural network, helped them predict a movement deficit score that achieved a high correlation with expert human scoring of motor deficits in stroke rats performing single-pellet reaching task. In our previous work,³³ we took a similar data-driven approach and developed a general-purpose CNN for dynamic action detection specifically applied to mouse grooming. Our network achieved human observer-level performance by identifying grooming behavior across 62 mouse strains with high visual diversity. However, we are taking a slightly different approach here. It involves first extracting an intermediate representation by identifying pose key points in each frame using a deep neural network,³⁹ which serves as input for JABS-based machine learning classifiers. Gschwind et al.⁴⁴ recently reported that unsupervised segmentation of sub-second behavioral motifs was sufficient to discriminate between epileptic and non-epileptic animals.⁴⁴ Their approach utilizing MoSeq⁴⁵ uses interictal behaviors to carry out this task. While MoSeq was able to detect behavioral differences between epileptic and non-epileptic animals, the detected behavioral motifs did not neatly classify into seizure and non-seizure motifs. In contrast, by using a supervised learning approach, we were able to directly classify a robust set of behaviors associated with seizures (Table 2) and integrate them into seizure intensity scores.

It is important to note that our approach is not a surrogate for Racine scoring per se, in that we do not classify all Racine score features. In particular, some of the semiological features in the Racine scale, such as facial twitches and whisker trembling, are extremely subtle and are not easily detected by overhead video. We focus on those features that involve relatively large changes detectable at the level of posture or locomotion. This approach dramatically simplifies the video recording setup to be in line with those for widely adopted behavioral tasks and, in particular, does not require special equipment such as a depth camera. Furthermore, a computational ethology⁶⁹ approach has the potential to enable detection of multiple behaviors simultaneously. Future developments can combine seizure detection with other behaviors, such as homeostatic behaviors, social behaviors, and more, to holistically phenotype an animal. Furthermore, unsupervised and supervised methods could be used in a complementary assay to provide rich partitioning of ictal and interictal behaviors. New version of MoSeq enables behavior segmentation using keypoint data.⁷⁰ Other unsupervised methods such as MotionMapper, VAME, B-SOiD, and others could be explored.^71,72,73

Our predictive models were particularly strong at separating out the “No” and “Low” seizure groups from the “Medium” and “High” groups, while it was harder to distinguish the “Medium” and “High” groups from each other. We attribute this largely to the performance of the base classifiers (Table 2), where the F1 scores for both “Side seizure” and “Wild jumping,” which discriminate between the “Medium” and “High” groups on the Racine scale, were lower than the F1 scores for “Straub tail” and “Leg splaying.” Thus, noise from the base classifier outputs potentially contributed to the overlap between the “Medium” and “High” groups in the feature space. However, we also note that the number of “Medium” scores was substantially smaller than the other three groups and was composed exclusively of individuals with overall Racine scores of 4. Thus, discrimination of “Medium” from “High,” in this case, required detecting the difference between a sitting tonic-clonic seizure (Racine score = 4) and a tonic-clonic seizure on the belly (Racine score = 5) (Table 1). Thus, increasing the number of individuals in the “Medium” class may improve future models, although we stress that titrating the dosage to achieve “Medium” and not “High” seizure responses is a significant experimental challenge. More generally, we argue that seizure severity is truly a continuum and that, while binning this continuum into discrete categories can aid in model training, we should always expect edge cases that are difficult to classify. Nevertheless, our model’s performance significantly captures the overall variability in seizure severity.

A significant hurdle is making our trained model accessible to labs without extensive computational expertise. To address this, beyond the technical details in the STAR Methods, we have developed an integrated mouse phenotyping platform. This complete hardware and software solution handles tracking, pose estimation, feature extraction, and automated behavior analysis, as described here.⁴⁰ While we have designed our platform for a specific open-field setup, researchers with their data collection systems can still leverage our trained model by ensuring they generate the same input features.

Together, our results demonstrate that the main semiological features of mouse behavioral seizures are robustly detectable from video data alone. Our models provide reproducible quantitative scores that we expect to be valuable for downstream applications, such as drug screening studies, quantitative genetics, and correlating seizure phenotypes with behavioral comorbidities.

Limitations of the study

• (1)We focus on features that involve relatively large changes detectable at the level of posture or locomotion. In particular, some of the semiological features in the Racine scale, such as facial twitches and whisker trembling, are extremely subtle and are not easily detected by overhead video.
• (2)A direct adoption of our classifiers requires the new videos to be visually similar to our videos. The video specifications are described in detail in the JABS manuscript.⁴⁰ If the video differs substantially, researchers can train new seizure-behavioral classifiers by training new models using our raw video data, but the burden of validating the new classifiers lies with the user.

Resource availability

Lead contact

Requests for further information and resources should be directed to and will be fulfilled by the lead contact, Vivek Kumar (vivek.kumar@jax.org).

Materials availability

This study did not generate new unique reagents.

Data and code availability

• •The video data will be shared through the EMBER data sharing platform and made available at https://doi.org/10.60533/ember-2025-k2n5.
• •The feature data and code are available at https://github.com/KumarLabJax/ptz-seizure-supervised.
• •Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

Acknowledgments

We thank Kumar Lab members for helpful advice and comments. We thank Erin Day for assistance in carrying out the experiments. This work was funded by The Jackson Laboratory Directors Innovation Fund, National Institutes of Health, National Institute on Drug Abuse DA051235 and DA048634 (V.K.), and National Institute on Aging AG078530 (V.K.).

Author contributions

G.S.S., L.H., A.M., M.S., and V.K. designed the experiments and analyzed the data. G.S.S., L.H., and M.M. carried out statistical modeling analysis. All authors wrote and edited the paper.

Declaration of interests

The authors declare no competing interests.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Chemicals, peptides, and recombinant proteins
Pentylenetetrazole CAS 54-95-5	Sigma P6500	CAS 54-95-5
Deposited data
Video data	This paper	https://doi.org/10.60533/ember-2025-k2n5
Experimental models: Organisms/strains
C57BL/6J	JAX	JAX 000664
C57BL/6NJ	JAX	JAX 005304
Software and algorithms
Python		https://www.python.org/
R		https://www.r-project.org/
JAX Animal Behavior System (JABS)		https://github.com/KumarLabJax/JABS-data-pipeline
Feature data and code	This paper	https://github.com/KumarLabJax/ptz-seizure-supervised https://doi.org/10.5281/zenodo.17427421
Other
Video hardware		https://github.com/KumarLabJax/JABS-data-pipeline/

Experimental model and study participant details

Animals and PTZ protocol

In this study, we tested both male and female C57BL/6J (B6J, JAX 000664) and C57BL/6NJ (B6NJ, JAX 005304) mice with 4 doses of PTZ (pentylenetetrazole, CAS 54-95-5, Sigma, cat# P6500; 0 mg/kg (vehicle), 40 mg/kg, 60 mg/kg, and 80 mg/kg) injected intraperitoneally. All procedures were approved by JAX IACUC (Animal Use Summary #14010 and the Routine Procedure #PCP20-12 - these are available upon request). Animals were either housed at JAX’s Research Animal Facility (RAF) B6 room or Center for Biometric Analysis (CBA) Annex 2 room. They were allowed to habituate to the room for at least 1 week prior to testing. They received 5K52 - Lab Diet 6% sterilized grain and acidified water. Boxes were checked daily and changed biweekly. Mice were weighed prior to testing. Fresh PTZ solutions were formulated in 0.9% saline at appropriate doses and administered by intraperitoneal injections.We collected a total of 85 videos (47 B6J, 38 B6NJ) (Figure 1B). All doses had at least 5 males and 5 females, with the exception of B6NJ at 0 mg/kg (Figure 1B).

Open field assay

The open field behavioral assays were conducted as previously described using a top-down camera view^32,74 using our JAX Animal Behavior System (JABS).⁴⁰ Mice were allowed to habituate for at least 15 min in the open field arena before being injected with one of the listed doses of PTZ and returned to the open field. Following our IACUC-approved protocol, mice were directly observed by a tester for a maximum of 20 min and given a Racine score.¹³ A mouse was removed from the open field and euthanized following the first signs of tonic extension or extreme seizure activity. A mouse was also euthanized if it maintained a Racine score greater than 3 (neck jerks) for a minute. Additionally, in the control group, if a mouse behaved normally for 5 min (i.e., no behaviors associated with seizures were observed), the animal was removed from the arena to save time from manual observation. Using this protocol, we collected a total of 85 videos (47 B6, 38 B6NJ) (Figure 1B). All doses had at least 5 males and 5 females, with the exception of B6NJ at 0 mg/kg dose (Figure 1B). A direct observer of the animals did the scoring.

Method details

Video recording, segmentation, and tracking

The JABS open field arena, video capture methods, and tracking and segmentation networks are as detailed previously.^32,40 Briefly, we used a neural network trained to produce a segmentation mask of the mouse to produce an ellipse fit of the mouse at each frame, as well as a mouse track.³² We estimated the 12-point two-dimensional pose using a deep convolutional neural network.^39,75 The points captured are the nose, left ear, right ear, the base of the neck, left forepaw, right forepaw, mid-spine, left rear-paw, right rear-paw, the base of the tail, mid-tail, and the tip of the tail. Each point at each frame has an x-coordinate, a y-coordinate, and a confidence score. We use a minimum confidence score of 0.3 to determine the points we include in our analyses. JABS is a versatile open source system, which we have used for gait and posture measures,³⁹ grooming,³³ biological age,³⁴ and pain states.³⁷

JABS classifiers

Briefly, our data acquisition uses custom-designed standardized data acquisition hardware and software that provides a controlled environment, optimized video storage, and live monitoring capabilities. This is leveraged by the JABS annotation and classification software which is a python based GUI utility for behavior annotation and training classifiers using the annotated data. Full details can be found in Beane et al. (2022).⁴⁰ One can then use the trained classifiers to predict whether behavior happens or not in the unlabeled frames. For this study, we trained behavioral classifiers for behaviors such as straub tail, side seizure, leg splaying, and wild jumping and reported the F₁ scores (Table 2). We calculated the F₁ score as the harmonic mean of precision and recall to provide a single measure that combines both. F₁ scores range from 0 to 1, where a higher F₁ score signifies a better balance between precision and recall, indicating a more accurate behavior classifier. Based on previous experience, we use bout wise accuracy metrics with an IoU of 50%.⁴⁰

Open field measures and feature engineering

Open field measures were derived from ellipse tracking of mice as described previously.³² Tracking was used to produce locomotor activity and anxiety features. Freezing behavior was heuristically derived by taking the average speed of the nose, base of head, and base of tail points at each frame, and finding periods of at least 3 s where the average speed of the mouse was less than 0.01 cm/s. For detecting tight circling events, we begin by observing changes in angles the mouse is facing. Once these changes in direction exceed 360° in either direction, we segment out that as a circle event. We then calculate the distance traveled during that event. If the mouse has traveled more than 6 cm, then the mouse was walking around the arena. Distances less than 6cm indicate a tight circling event. To adjust for the different durations that animals were observed during experiments with varying PTZ doses, we standardized each feature by the time (in minutes) the animal was taken out.

Quantification and statistical analysis

Linear discriminant analysis (LDA)

We centered the features to have zero mean. To address multicollinearity across features and avoid overfitting with LDA,⁷⁶ we performed principal component analysis (PCA) to obtain projections of features that are mutually uncorrelated and ordered by variance. We used all principal components (PCs) in the subsequent LDA algorithm to avoid the risk of throwing away critical dimensions. We projected the data to 2D LDA space for visualization.

Ordinal linear models for predicting seizure intensity

We used a supervised latent variable model (ordinal regression) to regress a continuous latent variable underlying the seizure groups (highest Racine scores) (see Table S1) onto behavioral features. This produced a univariate seizure scale divided into discontinuous segments based on the threshold parameters estimated in a data-driven manner by the model. More specifically, we used cumulative link models, a type of ordinal regression, most commonly used to analyze ordinal data.⁷⁷ The mouse’s seizure group, one of the four ordinal categories (High, Medium, Low and No) is regressed onto the mouse’s measured behavioral features. The regression coefficients and thresholding parameters for the continuous latent scale are inferred from the data. We used the ordinalNet package⁷⁸ to fit the regularized ordinal regression model.

To estimate the seizure intensity for a mouse at each time point of the open-field assay, we fit a hierarchical version of the cumulative link models, namely, cumulative linear mixed models (CLMM). More specifically, we leveraged the repeated measures for each animal in each time bin by treating them as random effects and treated the features tail jerk, leg splaying, side seizure, wild jumping, freezing, and circling as fixed effects. We used leave-out-one-animal cross validation (LOOCV) to validate our ordinal mixed-effects model. We used the estimated model coefficients for both fixed and random effects to estimate the instantaneous per minute seizure intensity for the left-out animal. We used the ordinal package⁷⁹ to fit the cumulative linear mixed model.

Published: November 26, 2025