Feasibility of forecasting self-injurious behavior among autistic youth using wearable sensors and machine learning models

Abstract

Self-injurious behavior (SIB) is a substantial clinical challenge for many individuals on the autism spectrum, and support strategies are often only reactive. Forecasting of SIB could enable timely support, especially using wearable sensors, but its feasibility is not well understood. We evaluated SIB forecasting using a previously collected dataset (n = 9) comprising motion and physiological data. We compared the performance of four machine learning models—Random Forest, AdaBoost.M2, Long Short-Term Memory (LSTM), and a Double-Stacked LSTM—across five forecast horizons (3s to 120s) and three feature sets: Motion-Only (from accelerometers), Physiological-Only (e.g., heart rate, skin conductance), and Combined. Performance was measured with a range of metrics, using Leave-One-Subject-Out cross-validation. We found a significant main effect of forecast horizon on the Area Under the Precision-Recall Curve; performance rose from near-chance at short horizons to having median scores above chance at one minute or longer. While aggregated results showed no significant differences between models or feature sets, subject-level analysis suggested predictive feasibility and that the optimal model configuration were highly person-specific. Our findings demonstrate that forecasting using wearable sensor data is feasible, but the substantial performance variability highlights a critical need for person-specific approaches to enable the development of clinically useful, proactive support systems.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-026-50079-z.

Introduction

A substantial clinical challenge for many individuals on the autism spectrum is self-injurious behavior (SIB), a heterogeneous set of self-inflicted actions that result in physical injury, with common forms including head banging, self-biting, hair pulling, and forceful contact with hard surfaces^1–4. While prevalence estimates vary, SIB is thought to occur in approximately 25–50% of autistic individuals across their lifespan^2,5,6. This behavior can negatively impact the quality of life for both the individuals and their families/caregivers^2,7,8; lead to restricted educational, vocational, and community opportunities^2,9,10; and is associated with a higher likelihood of psychiatric hospitalization⁹.

Since the functions that maintain SIB are complex and vary considerably between individuals^3,11, effective intervention often requires an individualized approach. Underlying this approach is a functional behavioral analysis (FBA), which is used to form a data-driven hypothesis about why SIB occurs^3,12. This hypothesis, in turn, guides the design of function-based interventions. However, the observational methods central to an FBA present substantial practical challenges: they are labor-intensive, place heavy demands on caregivers and clinicians, and are prone to inconsistent or incomplete data logging^13,14. These limitations underscore the need for more efficient and objective methods to acquire the high-fidelity behavioral data required to inform and evaluate effective care.

Emerging technologies, including wearable sensors and computer vision, have been applied to monitor internal states such as stress that are often precursors to challenging behaviors (for a review, see Cano et al.¹⁵). However, a focus has been on the detection and/or clarification of such behavior¹⁶. For example, SIB can be detected with wearable sensors, with reported performance (e.g., accuracy) varying from ~ 70% to nearly 100%, contingent on factors such as SIB topography, experimental settings (e.g., laboratory vs. clinical environments), and machine learning approaches e.g.^17–22. Similarly, computer vision approaches have achieved detection accuracies between 77% and 98%, though these approaches have typically been tested on a limited set of SIB topographies, such as arm flapping and head banging e.g.^23–26. While this body of work supports the potential for automated behavioral logging, these detection systems are fundamentally reactive, identifying behavior only as it occurs (or soon after).

To move beyond this reactive paradigm, a small number of studies have explored forecasting an SIB event using wearable sensors. Goodwin et al.²⁷ showed the feasibility of predicting aggression one minute in advance, achieving an Area Under the Curve (AUC) of 0.71 for population models and 0.84 for subject-specific models. More recent work predicted the imminent onset of motor repetitive behaviors by identifying “transition behaviors” in data immediately preceding an event, achieving an AUC of 0.88²⁸. The clinical potential of such forecasting is considerable, offering a crucial window for proactive support while prompting focused data collection for functional analyses. However, SIB forecasting remains a nascent field with critical unanswered questions. In particular, there is a need to understand the trade-offs between predictive lead time (or forecast horizon) and model performance and to clarify the relative contributions of different sensor modalities.

Our objective in this exploratory study was to investigate SIB forecasting, using an existing dataset comprising multimodal wearable sensor data from autistic children obtained during normal behaviors and SIB events¹⁸. We developed and compared several machine learning models to answer the following three research questions: (1) How does predictive performance change across a spectrum of forecast horizons? (2) What are the relative contributions of motion and physiological data streams to predictive performance? and (3) How do different machine learning architectures, namely ensemble methods and recurrent neural networks, compare in this forecasting task? Our findings were intended to provide a quantitative analysis of the trade-offs between forecast horizon and predictive accuracy, and the relative merits of different sensor modalities and modeling approach, to inform the future development of clinically useful, proactive support systems.

Methods

Dataset description

We utilized a dataset collected as described by Cantin-Garside et al.¹⁸. These data were collected from 11 autistic children (aged 5–14 years) with a frequent history of SIB (> 3 times/hour). The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Virginia Tech Institutional Review Board. Informed consent for study participation was obtained from a parent and/or legal guardian for all child participants. Children provided assent (> 7 years of age and of sufficient developmental level) before any data collection. All experimental methods were performed in accordance with relevant guidelines and regulations.

For the present analysis, we included data from the nine participants who wore a consistent sensor configuration that comprised both an Empatica E4 wristband (Empatica, Inc., Milan, Italy) and a waist-worn ActiGraph accelerometer (ActiGraph GT9X Link; ActiGraph LLC, Pensacola, FL). Data from these two sensor locations formed the basis for all forecasting analyses reported here. The Empatica E4 provided a suite of signals at different sampling frequencies: 3-axis accelerometer data (32 Hz), blood volume pulse (BVP; 64 Hz), electrodermal activity (EDA; 4 Hz), and skin temperature (1 Hz), along with event-based inter-beat interval (IBI) data. The ActiGraph devices collected 3-axis motion data at 60 Hz. All data collection sessions were video-recorded. These recordings were used to generate a binary ‘SIB’ vs. ‘non-SIB’ annotation, which served as the ground-truth target for this study. Our operational definition of labeled target behaviors included both SIB and aggressive behaviors (e.g., head banging, self-biting, punching) as well as stereotyped/repetitive behaviors (e.g., arm flapping, spinning), the latter of which were included because they are conceptually related to SIB and may represent a precursor to SIB^29,30.

Data preprocessing and feature extraction

The raw multi-modal sensor data were processed through a four-stage pipeline, involving synchronization, valid analysis period segmentation, feature extraction, and forecasting label construction. First, all data from the ActiGraph and Empatica E4 were aligned to a common Coordinated Universal Time time base. Signals from the E4 were upsampled to 60 Hz using linear interpolation.

Second, the synchronized time-series data were segmented using a sliding window approach. A 10-second window was chosen pragmatically, to capture short-term physiological and behavioral dynamics while maintaining computational efficiency. This window was advanced in 1-second increments (90% overlap). To ensure that our models were trained to forecast the onset of SIB from a non-SIB state, we used a “clean window” approach. In this approach, any 10-second window containing a ground-truth SIB label was discarded, leaving only SIB-free windows for processing.

Third, a set of features was extracted from each 10-second window. Prior to feature extraction, accelerometer signals were low-pass filtered (bidirectional; 4th Butterworth; 20 Hz cutoff)¹⁸, and the BVP signal was band-pass filtered (bidirectional; 4th Butterworth; 0.5–5 Hz) to isolate the primary cardiac component³¹. Spurious values in the raw IBI series were removed based on physiologically plausible heart rates³². Twenty features were extracted, which were designed to be computationally efficient for future real-time implementation and are categorized as follows:

• Motion features (6 features): From the vector magnitude of both the E4 and ActiGraph accelerometers, we extracted three orientation-independent features from each sensor: the mean, standard deviation, and the mean magnitude of the jerk vector (the derivative of acceleration).

• EDA features (4 features): To characterize sympathetic arousal, the EDA signal was decomposed into its tonic and phasic components³³. The tonic component was represented by the signal’s mean, while phasic activity was captured by the standard deviation, range, and mean of the first derivative (i.e., jerkiness).

• Cardiac and heart rate variability (HRV) features (7 features): The mean and standard deviation of the BVP signal amplitude were extracted. From the heart rate data (R-R intervals), we calculated the mean, standard deviation, and range. Two time-domain HRV features reflecting parasympathetic activity were computed from the R-R intervals: the Root Mean Square of Successive Differences (RMSSD) and the percentage of successive RR intervals differing by more than 50 ms (pNN50)³⁴.

• Skin Temperature Features (3 features): To capture the onset of slow thermal regulation changes, the mean, standard deviation, and the linear slope of the skin temperature signal were calculated.

The feature extraction process yielded a feature matrix where each row represents a 10-second physiological and behavioral state.

Fourth, each clean data window was assigned a binary forecasting label. For this task, the 10-second data window serves as the observation window. As illustrated in Fig. 1, the label was determined by whether an SIB event occurs within a future prediction window, which is separated from the observation window by a latency period. An observation window was assigned a positive label (‘1’, pre-SIB) if an SIB onset occurred within its corresponding prediction window. Otherwise, it was assigned a negative label (‘0’, non-precursor). This approach ensured that all features were derived from non-SIB periods, creating a true forecasting problem.

Fig. 1.

Schematic of the forecasting label assignment. An observation window is labeled based on the occurrence of an SIB event (red star) within a future forecast horizon, which comprises a latency period and a prediction window. (a) A positive label (‘pre-SIB’) is assigned if an SIB event occurs within the prediction window. (b) Otherwise, a negative label (‘non-precursor’) is assigned.

Forecasting problem formulation

The goal of the forecasting task was to predict an upcoming SIB event using features derived from a 10-second observation window. To investigate the trade-offs in predictive lead time, we defined five distinct forecast horizons (H1–H5), each defined by a unique combination of a latency period and prediction window (Table 1).

Table 1.

Forecast horizons and examples of potential responses. (See Fig. 1 for definitions of the latency period and prediction window.)

Horizon	Latency Period (s)	Prediction Window (s)	Example Response Type (Immediate → Proactive)
H1	3	7	Redirecting immediately (e.g., verbal prompt)
H2	15	15	Offering an alternative (e.g., preferred task)
H3	30	30	Changing the environment pre-emptively (e.g., calming routine)
H4	60	30	Heightening observation & support
H5	60	60	Executing a planned proactive strategy

Machine learning models

We evaluated two fundamentally different classes of machine learning models: ensemble methods and recurrent neural networks. The former treats each 10-second observation window as an independent data point and was selected for its computational efficiency. The latter was chosen to explicitly model temporal dependencies across sequences of consecutive observation windows. This comparison allowed us to assess both static and sequential modeling approaches. All models were implemented in MATLAB (R2024b, The MathWorks, Inc., Natick, MA) using the Statistics and Machine Learning Toolbox and the Deep Learning Toolbox.

Ensemble models

We compared two decision tree-based ensemble models, Random Forest (RF) and AdaBoost.M2, since these models are well-suited for tabular data and often require less training time and hyperparameter tuning than deep learning approaches^35,36. Their selection was further supported by prior work showing that RF has high performance in classifying stereotypical motor movements from similar wearable sensor feature sets³⁷. RF utilizes a “bagging” method to reduce variance and overfitting³⁸, while AdaBoost.M2 utilizes a “boosting” method to iteratively reduce bias, though this may result in higher sensitivity to noisy data and outliers^39,40.

To address the class imbalance inherent in the dataset—which can bias models towards the majority (non-precursor) class—we used the Synthetic Minority Over-sampling Technique (SMOTE)⁴¹. SMOTE was applied only to training data during model fitting, creating a more balanced dataset for the model to learn from without artificially influencing the evaluation on the held-out test data. To ensure reproducibility, the random seed was fixed prior to network initialization and training. For hyperparameter optimization, we utilized the MATLAB automated fitcensemble function with the ‘OptimizeHyperparameters’, ‘auto’ option (i.e., a search budget of 30 iteration). This function employs a Bayesian optimization scheme with an internal (nested) 3-fold cross-validation (CV) within the training data to efficiently identify a high-performing set of model parameters.

Recurrent neural network models

We evaluated Long Short-Term Memory (LSTM) networks to explicitly model temporal dependencies. Unlike the ensemble methods, LSTMs require sequential input. Thus, the feature matrix was transformed into overlapping sequences of five vectors each. This sequence length was selected to balance providing enough temporal context for the model against the computational costs of training on longer sequences. Given the 1-second increment between consecutive vectors, each five-vector sequence spanned a total of 14 s of observation time (a 10-second window plus four 1-second increments).

We evaluated two LSTM architectures to explore the trade-off between model complexity and performance: a single-stack model with one LSTM layer and a double-stacked model with two consecutive LSTM layers. The double-stacked architecture was included to determine if a deeper model, capable of learning more hierarchical temporal representations, would yield improved performance. In both architectures, a dropout layer (rate = 0.3) was applied after each LSTM layer, and the network terminated with a fully connected layer and a sigmoid activation function for binary classification. To ensure reproducibility, the random seed was fixed prior to network initialization and training. Hyperparameter optimization was conducted for both architectures using a Bayesian approach over 35 iterations with a nested 3-fold CV within the training data. Networks were trained for a maximum of 40 epochs. This process tuned the initial learning rate (range: 5e-5 to 2e-3), L2 regularization strength (1e-5 to 1e-2), and mini-batch size (32 to 128). The number of hidden units was also tuned: for the single-stack LSTM, a range of 32 to 128 was used; for the double-stacked LSTM, the units for the first (32 to 128) and second (16 to 64) layers were optimized independently. To address class imbalance, we implemented a cost-sensitive learning approach by applying class weights to the cross-entropy loss function⁴².

Experimental design and evaluation

Our experiment was designed to investigate the effects of three factors on predictive performance: Model (RF, AdaBoost.M2, Single-Stack LSTM, Double-Stacked LSTM), Forecast Horizon (H1–H5), and input Feature Set (Motion: 6 features, Physiological: 14 features, Combined: 20 features).

Cross validation

We employed Leave-One-Subject-Out (LOSO) cross-validation to estimate how well the models would generalize to a new individual. Training a model on data from all participants but one and testing on the held-out individual provides a realistic, albeit challenging, estimate of real-world performance⁴³. This entire LOSO cross-validation procedure was performed independently for each combination of model, forecast horizon, and feature set levels.

Model performance was evaluated using several binary classification performance metrics, with a focus on those metrics robust to the class imbalance in the dataset. Our primary evaluation was based on two threshold-independent metrics. We used the Area Under the Receiver Operating Characteristic Curve (AUC-ROC) as an aggregate measure of overall discriminative performance across all classification thresholds⁴⁴. To assess the reliability of positive prediction⁴⁵, we used the Area Under the Precision-Recall Curve (AUC-PR), which we interpreted as a measure of predictive reliability. Note that while the chance-level performance for AUC-ROC is always 0.5, the chance level for AUC-PR is data-dependent and corresponds to the overall event rate (0.12) in the test data across all cross-validation folds.

To supplement these primary measures, we also calculated several threshold-dependent metrics to provide a more detailed picture of performance⁴⁶: Accuracy, the overall proportion of correct classifications; Precision, the proportion of positive predictions that were truly positive; Recall (Sensitivity), the proportion of actual positive cases correctly identified; Specificity, the proportion of actual negative cases correctly identified; and the F1-Score, the harmonic mean of Precision and Recall. In addition, we report the Matthews Correlation Coefficient (MCC). MCC produces a high score only if the classifier performs well across all four confusion matrix categories, making it a balanced and robust measure for imbalanced datasets⁴⁷.

Statistical analysis

To determine if the observed differences in model performance were statistically significant, we performed separate, three-way analysis of variance (ANOVA) on the AUC-ROC and AUC-PR scores. The models were fit using Restricted Maximum Likelihood (REML) estimation, and an inspection of the model residuals confirmed that the assumptions for parametric testing were met. This analysis tested for the main and interaction effects of Model, Forecast Horizon, and Feature Set. A significant main or interaction effect was followed by targeted post-hoc pairwise comparisons using paired t-tests. P-values were adjusted for multiple comparisons using the Bonferroni method, and a corrected p-value of < 0.05 was considered statistically significant. All statistical analyses were conducted using the Statistics and Machine Learning Toolbox in MATLAB (R2024b, The MathWorks, Inc., Natick, MA).

Results

Effects of forecast horizon

A summary of ANOVA results on AUC-ROC and AUC-PR scores is presented in Table 2. There were no significant main or interaction effects on AUC-ROC scores, and AUC-PR scores were significantly affected only by the Forecast Horizon main effect (F(4, 480) = 9.88, p < 0.001). Post-hoc comparisons indicated that predictive performance improved with each increase in forecast horizon from H1 to H4 (all pairwise p < 0.001), and with no significant difference found between H4 and H5.

Table 2.

Summary of three-way ANOVA results for primary performance metrics.

	Model (M) (df = 3)	Forecast Horizon (FH) (df = 4)	Feature Set (FS) (df = 2)	M×FH (df = 12)	M×FS (df = 6)	FH×FS (df = 8)	M×FH×FS (df = 24)
AUC-ROC	0.382 (0.766)	0.545 (0.703)	0.176 (0.838)	0.139 (0.999)	0.954 (0.456)	0.365 (0.939)	0.244 (0.999)
AUC-PR	0.001 (1.000)	9.878 (< 0.001)	0.152 (0.859)	0.031 (1.000)	0.093 (0.997)	0.392 (0.925)	0.063 (1.000)

Note that each entry value is the F-statistic (p value), that denominator degrees of freedom (df) was 480 for all tests, and that significant effects (p < 0.05) are highlighted in bold.

Effect of model across forecast horizons

Although there was no significant effect of Model, Fig. 2 is presented to visualize the descriptive performance trends and the high variability. The figure shows that the AUC-ROC and AUC-PR scores for each of the four models across the five forecast horizons, using the Combined feature set. AUC-ROC scores had wide and largely overlapping distributions, with most models performing above the 0.5 chance level (Fig. 2a). In contrast, the AUC-PR scores indicate better performance at longer horizons, with the median scores becoming well above the calculated chance level of 0.12 (Fig. 2b). The corresponding performance distributions for the Motion-Only and Physiological-Only feature sets are provided in Appendix A, which show that AUC-PR improved with longer forecast horizons across all three feature sets.

Fig. 2.

Forecasting performance by model and horizon for the ‘Combined’ feature set. The distributions show (a) AUC-ROC and (b) AUC-PR scores from LOSO cross-validation. Dashed horizontal lines represent the chance level for each panel: 0.5 for AUC-ROC (a) and 0.12 for AUC-PR (b).

Effect of feature set across forecast horizons

Similarly, while Feature Set has no significant effect, Fig. 3 (AUC-ROC) and 4 (AUC-PR) are presented to illustrate the contributions of different feature sets for the RF and LSTM models, respectively. The performance distributions for the Motion-Only, Physiological-Only, and Combined feature sets were highly overlapping across all conditions. Descriptively, however, the Physiological-Only and Combined sets appeared to yield higher median performance than the Motion-Only set at the longest forecast horizons, particularly for the LSTM model. The corresponding plots for the AdaBoost.M2 and Double-Stacked LSTM models are provided in Appendix B and show this same non-significant outcome. To investigate this further, subsequent analyses were performed to characterize inter-subject variability (Table C1 in Appendix C) and statistically evaluate the contribution of physiological data (Table C2). These analyses indicated that while group-level effects were non-significant, models performed better for specific participants, and including physiological data enabled substantial performance improvements.

Fig. 3.

AUC-ROC performance by feature set for (a) the RF model and (b) the LSTM model. Distributions of scores are shown across the five forecast horizons. The dashed horizontal lines indicate the chance level.

Summary of classification performance

Aggregated confusion matrices for each of the five forecast horizons provide a more practical summary of classification performance (Table 3). These matrices summarize the total counts of correct and incorrect classifications across all models and participant folds. In addition, a more detailed breakdown of performance is provided in the appendices. Appendix D contains the confusion matrices for each individual model at each horizon. Appendix E provides a comprehensive summary of all calculated secondary performance metrics (e.g., Accuracy, F1-Score, MCC) for every experimental condition.

Table 3.

Confusion matrices by forecast horizon.

Each matrix shows the total classification counts (True/False Positives and Negatives) for a given forecast horizon. Values are aggregated across all models and all participant folds from the cross-validation.

Discussion

We evaluated SIB forecasting by examining how predictive performance is affected by the forecast horizon, machine learning model, and feature set. While the forecast horizon had a significant effect on performance, the type of model and the feature set did not. Specifically, forecasting performance, quantified by AUC-PR, generally improved as the forecast horizon increased (Figs. 2b and 4). Conversely, no statistically significant differences were observed in either AUC-PR or AUC-ROC with respect to the machine learning model or feature set used (Figs. 2a and 3). We attribute this lack of significance to the high inter-subject variability, given that our LOSO cross-validation demonstrated that the performance of a given model varied greatly when tested on different individuals (i.e., the one left out, or unseen data). This performance variability suggests that physiological and behavioral precursors to SIB are likely individualized, making it difficult for a single, generalized model to be effective across individuals.

Fig. 4.

AUC-PR performance by feature set for (a) the RF model and (b) the LSTM model. Distributions of scores are shown across the five forecast horizons. The dashed horizontal lines indicate the calculated chance level.

The significant main effect of forecast horizon on AUC-PR suggests a shift in the nature of the predictive task as lead time increases. At short horizons (e.g., 3–10 s), forecasting appears to rely on identifying immediate behavioral precursors. The poor performance in these conditions may indicate that such precursors are highly variable and difficult to distinguish from benign movements (Figs. 1b and 3; Appendices B and D). This difficulty may also be exacerbated by our clean window approach, since this approach likely led to the exclusion of behaviors immediately preceding the SIB event. Short-term forecasting can thus be considered a different and more challenging task than the detection of an SIB event once it has already begun, a task for which prior work has reported accuracies in the 70–100% range ^{e.g., 17–22}. In contrast, forecasting at longer horizons (e.g., 60–120 s) may depend more on identifying sustained patterns in physiological signals that reflect physiological dysregulation. Since these shifts likely precede the motor action, such patterns seem to remain detectable even when the SIB event window is excluded. This interpretation is consistent with theories of autonomic nervous system function, which indicate that prolonged sympathetic activation or parasympathetic withdrawal can create a state of vulnerability that precedes challenging behaviors^48,49. Indeed, Goodwin and colleagues^27,50 demonstrated that physiological arousal could forecast aggression in minimally-verbal autistic youth one minute before aggression onset. Collectively, these points suggest that physiological states offer a more robust signal for forecasting than the more variable, immediate behavioral precursors.

Although not statistically significant, trends in model performance may provide some insight into the nature of SIB precursors. Ensemble methods (RF and Adaboost.M2) had relatively consistent performance across all horizons (Fig. 2), likely due to effectiveness in modeling complex, non-linear relationships within a static feature set³⁹. Conversely, LSTM models exhibited a trend of improved relative performance at longer horizons. Given that LSTMs are designed to capture temporal dependencies in sequential data^51,52, this trend may reflect the capacity to model the temporal evolution of physiological dysregulation that signals a more distant SIB event. However, the wide variance in performance observed in our LOSO cross-validation emphasizes that precursors to SIB are highly idiosyncratic. This variability is evident in the subject-level results (Table C1 in Appendix C) and the analysis of physiological feature contributions (Table C2). This finding is consistent with this individual-level heterogeneity, a well-documented characteristic of the autism spectrum^53,54, which potentially accounts for the lack of statistical significance we found between models and feature sets. For instance, Antezana et al.⁵⁵ reported that factors such as emotion dysregulation, repetitive motor movements, and co-occurring medical conditions were differentially associated with specific forms of SIB (e.g., self-hitting vs. skin-picking). Further, Goodwin et al.⁵⁰ found that person-dependent models outperformed a population-level model (e.g., 30% difference between highest and lowest prediction performance). Taken together, this evidence suggests that the performance of any generalized, person-independent forecasting model will likely be constrained, highlighting the need for person-specific approaches.

Our findings should be considered in light of several limitations. First, the small sample size (n = 9) and the clinical data collection setting may limit the generalizability of our findings to more naturalistic environments. Second, the temporal context provided to our sequential models was constrained; the 14-second input sequence used by the LSTMs may be insufficient to fully capture the slow-building physiological processes that can signal dysregulation over longer periods. Future studies should therefore explore using longer input sequences, collecting data form larger and diverse samples, and integrating contextual data from other modalities such as computer vision. For example, Das et al.⁵⁶ used a vision-based model to forecast challenging behaviors at a three-minute horizon in a classroom setting; they achieved only a modest F1-score, highlighting the inherent difficulty of long-range forecasting. A multimodal approach that fuses the contextual richness of computer vision with the individual-specific precision of wearable biosensors might therefore be a promising approach for future research. Third, our data windowing and forecast horizons might have introduced performance biases. The clean window approach excluded immediate behavioral cues, potentially disadvantaging short-term models, whereas the variable prediction window lengths (Table 1) could increase baseline event probabilities for longer horizons, theoretically inflating those metrics. Relatedly, using SMOTE to address class imbalance in ensemble models carries a risk of distorting temporal patterns within the physiological data. However, the comparable performance of our LSTM models (which used cost-sensitive learning) suggests that this potential distortion was not the primary driver of our results. Lastly, a fourth limitation relates to the practical implementation of these models, a challenge highlighted by our secondary performance metrics (Appendix E). While AUC-PR results demonstrate that a predictive signal exists, the low F1 and MCC scores indicate that at a standard decision threshold, model performance is modest, likely due to the severe class imbalance inherent in SIB data. To provide further context on this trade-off between sensitivity and false alarms, we analyzed the Alert Ratio across horizons (Appendix F). We found that the ratio of false alarms to true positives improved as the forecast horizon was extended. At the shortest horizon, the deep learning models generated a large number of false alarms relative to true positives. In contrast, this ratio at the longest horizon improved to 4.4:1 for the Double-Stacked LSTM, which is a modest improvement over the RF (4.6:1). Overall, these results suggest that long-horizon forecasting could serve as an attention-cueing aid within a human-in-the-loop workflow, but also emphasize a continuing need to develop person-specific decision thresholds that balance the cost of false alarms against the benefit of timely alerts.

Conclusions

Our study provides an evaluation of machine learning models for SIB forecasting using both motion and physiological wearable sensor data. We found that forecasting is potentially feasible, with longer lead times of up to one minute providing a stronger predictive signal. This effect appears to be driven by physiological data, which became more informative at longer forecast horizons, likely reflecting the detection of slow-moving changes in physiological states. These results support the potential for SIB forecasting systems to identify periods of elevated risk, which could potentially prompt caregivers to heighten their observation and adopt a proactive stance. However, our findings also highlight a substantial challenge: model performance was largely affected by the high degree of inter-subject variability in SIB precursors. This performance variability highlights the important need for future research to focus on the development of person-specific forecasting approaches to realize their clinical potential.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

S.K.: Designed the modeling experiments, performed the data analysis, and wrote the original draft of the manuscript. K.D.C-G.: Conceptualized and conducted the original data collection study and contributed to the interpretation and editing of the current manuscript. M.A.N.: Conceptualized and supervised the original data collection and contributed to the interpretation and editing of the manuscript. All authors have read and approved the final version.

Data availability

The dataset used for this study are not publicly available due to ethical restrictions, as they contain sensitive information and identifiable behaviors from minors. Data may be made available from the corresponding author upon reasonable request and with appropriate institutional approvals.

Declarations

Competing interests

The authors declare no competing interests.