Early Sound-Based Meal Onset Detection Using Hidden Markov Models (HMMs)

Abstract

Objective: Meal detection is essential for automated insulin delivery systems in patients with Type 1 Diabetes (T1D), as meals are the primary drivers of rapid postprandial glucose excursions. Current closed-loop systems depend on manual meal announcements, which are often inaccurate or omitted, reducing safety and autonomy. To address this, we developed an automatic meal detector leveraging physiological sound sequences from the body. Methods and procedures: The dataset comprised 50 meal recordings from ten participants (aged 24–30 years) using a throat-mounted microphone. Twenty recordings were collected in quiet conditions, while 30 included deliberate, realistic disturbances. Audio signals were preprocessed, segmented, and transformed into 46 features. Linear Discriminant Analysis (LDA) was implemented as a benchmark, while continuous Hidden Markov Model (HMM)s were applied to capture temporal dependencies and estimate meal onset probability. Results: Performance was evaluated using both an 80:20 train-test split and Leave-One-Subject-Out Cross-Validation (LOSO CV). Explainable Artificial Intelligence (XAI) techniques were employed to interpret model behavior and identify informative features of meal events. LDA achieved 90% accuracy, 78% sensitivity, and a detection delay of 12.3 s, while HMM reached 91% for both accuracy and sensitivity, with an average detection time of 11.7 s. Both models performed well under quiet conditions but were sensitive to background disturbances. Conclusion: These findings demonstrate the feasibility of detecting meal onset from physiological sounds. XAI analysis revealed that HMM effectively captures sequential transitions associated with chewing and swallowing, offering interpretable insights beyond non-temporal methods. Thus, HMM shows potential as a rapid, context-aware meal classifier for closed-loop insulin therapy, thanks to its effective modeling of temporal structure. Clinical Impact: This pilot study uses an innovative throat-mounted microphone to detect early meal intake by capturing chewing and swallowing sounds, offering an alternative to traditional measures such as continuous glucose monitoring (CGM).

I. Introduction

Diabetes mellitus is a chronic disorder of impaired insulin production or use. In T1D, pancreatic malfunction disrupts glucose homeostasis. Consequently, the blood glucose concentration is outside the desired or nominal values. Despite therapies, technology, and educational programs, improving daily life and preventing complications remains an active research area [1], [2]. Postprandial glucose excursions are one of the main challenges for closed-loop Artificial Pancreas (AP) systems [3], which rely on self-reported meal announcements. Without them, hyperglycemia often occurs [4], leaving APs as a hybrid rather than a fully automated system.

Commercial APs primarily rely on BGL information from continuous glucose monitors to respond to meals, which may result in delayed meal detection and subsequent hyperglycemia. More broadly, automated meal-detection methods that do not require meal announcement have been developed using CGM signals, insulin-delivery data, physiological measurements, and wearable sensors [5]. According to a recent systematic review of these approaches [6], considerable variation exists in both the methods employed and their reported performance, making direct comparisons difficult. CGM-based approaches are particularly attractive because CGM is already an integral component of automated insulin-delivery systems. However, these approaches generally rely on the physiological response to meal intake, which may introduce a delay between meal consumption and detection [7]. Ideally, meal onset should be detected as soon as the person starts eating, before glucose enters the bloodstream and causes a glucose excursion. One way to achieve earlier detection is to consider the sequence of actions that occur during food ingestion (e.g., biting, chewing, swallowing). These actions are not easily detectable in themselves, but several easily observable quantities (such as relevant body sounds) exist that are correlated with them, motivating sequential probabilistic modeling.

HMMs have exactly this capability. They represent one of the techniques currently yielding the best performance when used with limited [8] and naturally sequenced training data [9], [10], such as in the case of meal detection for glucose control. HMMs have been widely used to detect the pronounced words said by eyes-and-hands-busy users, so they have been a recurring theme in speech recognition and biology. The first thorough and comprehensible introduction to these models was provided in 1989 by Rabiner [9]; since then, they have been used extensively and with great success in speech recognition and generation, as well as for sequential analyses in biology and other fields. Examples include speech recognition systems and methods for utilizing spectral information [11] and a method for recognizing spoken English digits [12]. In a 2005 study [13], HMMs were used to discriminate eating and drinking gestures from other usual movements by using wearable sensing technology to assist in monitoring eating habits. Dong and Biswas [14] developed a wearable sensor system for estimating time and duration of a meal by combining swallowing signatures of breathing, measured with respiratory inductance plethysmography, with hand acceleration. Eating behaviors were also estimated with the HMM-based meal intake monitoring system designed by Costa et al. [15], suitable for remote monitoring of the elderly at home due to its non-intrusive characteristics.

Currently, available closed-loop systems for automated insulin delivery are hybrid in nature and still require the person to count carbohydrates and manually announce meals to the system. In this work, we conducted a pilot study using acoustic data to exploit the ability of HMMs to describe the probabilistic relationship between a sequence of observations (non-verbal body sounds) and a sequence of hidden states (biting, chewing, swallowing, and digestion), to build an early detector of the onset of a meal, which can later be integrated into the insulin pump controller. The early meal detector would help manage missed or under-administered insulin doses, even in non-closed-loop systems, as it can function simply as a reminder of a missed insulin dose, thereby improving time within the therapeutic range for diabetes management. However, a meal-onset detection alone is not sufficient to eliminate the need for carbohydrates counting or to fully automate mealtime insulin dosing. Rather, it represents an important first step toward a more fully automated and personalized approach to mealtime insulin delivery.

II. Theoretical Background

A. List of Symbols

λ

Complete parameter set of a HMM.

Probability of a transition to hidden state given the hidden state , i.e., .

A

State-transition probability matrix associated with an HMM; .

Multivariate probability density function of the observations given the hidden state at time t, i.e., .

B

Set of emission probability density functions; .

Number of features that constitute an observation.

The z ’th sound signal feature.

Vector of audio feature values corresponding to time segment t, i.e., .

O

Matrix of observations, .

Number of observations.

Probability that the Markov chain will start in state .

π

Vector of initial probabilities for each hidden state; .

i’th hidden state.

S

Vector of the finite, discrete set of hidden states.

Number of hidden states.

Actual hidden state of the HMM at time t, i.e. if the HMM is in hidden state at time t, then .

x

Discrete stochastic process.

B. Definitions

1) Emission

Sequence of observation likelihoods, each expressing the probability of an observation being generated from a state . Emission probability matrix is the probability that an observation O is emitted when the HMM is in hidden state .

2) Feature

Numerical representation of a particular property of a short segment of the audio signal.

3) Hidden state

States that are only (indirectly) observable via the system’s emissions. They are the sequence that the model seeks to recover from the observable data.

4) Initial probability

Probability distribution of the initial hidden state; , is the probability that the Markov chain will start in state i.

5) Observation

Observable data emitted by the system.

6) Transition

The event of an HMM changing from one hidden state to another (or possibly the same) hidden state.

C. Hidden Markov Models

A HMM is a statistical model in which observable signals depend probabilistically on internal, unobserved (‘hidden’) states [16], as illustrated in Figure 1. In this work, ‘Observations’ come from audio-derived , with features forming the ‘feature vector’ observed at time t. The hidden states , with being the i’th Hidden state and being the number of hidden states, represent meal-related actions such as ‘chewing’ and ‘swallowing’. Although these states are not directly observable, variations in the recorded sound features provide information about the underlying eating behavior. In this study, we use a first-order continuous HMM. Let x be a discrete stochastic process whose value at time t denotes the hidden state. Under the first-order Markov assumption, the probability of the current state depends only on the previous state [17]. In mathematical terms:

Our model can now be represented by the four-tuple

FIGURE 1.

Structure for an HMM for early meal detection. Transition among hidden states are represented as , every single observation at time t is a vector of sound features , and the probability of having a feature given a hidden state is denoted by .

In this model, is the set of hidden states. The vector gives the Initial probability distribution, where is the chance of starting in state . Transition matrix defines the probability of moving from state to (cf. arrows connecting the different hidden states in Figure 1). Emission probability distribution functions describe the likelihood of observing given hidden state . Finally, denotes the sequence of continuous observations, with the number of observations.

This study aimed to detect meal onset using two identically structured but separately trained HMMs (suggested by the two layers in Figure 1): a ‘meal model’ capturing eating dynamics including the sequence of actions that occur during a meal, and a ‘no-meal model’ accounting for non-intake signals (e.g., breathing, activity, heartbeat, noise). By decoding observations into likely state transitions and comparing model likelihoods, the algorithm determines which model better explains the observed data, and consequently, whether a meal has started.

Training an HMM involves determining , , , and in Equation 2. In supervised learning, labeled data provide state transitions and emission distributions (e.g., histograms or Gaussian mixtures). In unsupervised learning, all parameters, including the number of hidden states, are inferred directly from the data, which is the traditional approach. Conventional HMMs assume regularly sampled data at discrete intervals, whereas continuous-time HMMs allow state transitions and observations at arbitrary (continuous) times, making them suitable for irregularly sampled data like clinical measurements. HMMs are categorized as discrete, with finite observation symbols and distributions suited to categorical tasks (e.g., POS tagging, phoneme recognition), or continuous, with real-valued vectors modeled by probability densities (often Gaussian mixtures) [18]. Continuous HMMs better capture variability in domains like speech or gesture recognition but demand greater computation and careful parameter estimation.

D. Characteristics of the Sounds

A wide variety of possible sounds is produced by the process of eating diverse types of foods. Chewing yields repetitive, irregular bursts with broadband spectra ( and 2 kHz), typically occurring in sequences that reflect the rhythmic motion of the jaw and food fragmentation [19] and varying by food texture and chewing force [20], [21]. Crispy foods emphasize , while crunchy foods ~1-2 kHz [22]. The temporal duration of chewing bursts can vary significantly, often lasting 200-600 ms, with pauses. Swallowing is more transient, marked by brief low-frequency components (400 Hz and 1 kHz) and a characteristic burst-silence pattern [23]. These signatures, defined by duration, spectrum, and amplitude envelope, enable acoustic detection and classification.

III. Data and Experimental Setup

A. Device

Sounds were recorded with a Knowles FG-23329 microphone mounted in a custom-designed stethoscope (SINTEF, Trondheim, Norway; PLA-printed cup with metal disc, 38 mm outer/12 mm inner diameter, 8 mm height, and an internal height of approximately 3 mm). The stethoscope fed into a Zoom F2 Field Recorder, as shown in Figure 2a, storing audio as 44.1 Hz, 32 bits WAV files. Positioned at the throat as shown in Figure 2b, and secured with double-sided tape, it captured chewing and swallowing sounds due to its proximity to the esophagus.

FIGURE 2.

Microphone equipment used for sound data collection.

B. Experimental Setup

This pilot study leverages the natural temporal sequences of the data to build a robust model by incorporating pre- and post-meal events with randomized timing and disturbances as illustrated in Figure 3. Meal recordings followed a structured protocol: After informed consent and collection of participant details, participants fasted (3h) and avoided fluids (1h) before sessions. Sessions were conducted quietly with Zoom F2 Field Recorders, movement limited to scheduled activities (Figure 3), randomized events performed, and annotations. Raw data were stored anonymously in JSON for analysis. Comfort was ensured through careful microphone placement and attachment testing.

FIGURE 3.

Example of a recording timeline. The green sections represent different sources of deliberate disturbances, and the red section represents the meal intake.

C. Dataset

The dataset comprised 50 meal recordings from 10 participants (aged 24–30, gender-balanced), each contributing 1–11 sessions (20 without and 30 with deliberate disturbances, cf. Figure 3), as reported in Table 1. Meals included mixed foods. The throat-mounted microphone captured acoustic signals including chewing and swallowing sounds, muscle activity artifacts, and respiratory sounds. The recorded time-series are rich in low-frequency energy due to the direct contact. Ethical approval (no. 84374 by Regional Committees for Medical and Healthcare Research (REK)) was obtained for this pilot study. Procedures followed the Declaration of Helsinki [24], data were anonymized and securely stored with a universally unique identifier under APT’s protocol.

TABLE 1. Distribution of subject genders and recordings.

Subject ID	6g8b	5XNp	7SEJ	3CPi	cL6E	Lv22	eGAR	jvir	osYS	TGnJ
Gender	M	F	M	M	F	F	M	F	F	M
# Recordings	11	9	11	1	5	1	3	3	4	2

IV. Methodology

To detect meal onset in real time, the collected time-series data were modeled as a sequence of labels. Each recording was segmented, and each segment was classified into two classes, as appropriate: No Meal (resting or non-eating activities) labeled as 0 and Meal (from lifting food to the final swallow) labeled as 1. To capture temporal dynamics and hidden behavioral states such as chewing and swallowing, two HMMs (one per class) were employed as shown in Figure 1. The dataset was first divided into training (80%) and testing (20%) subsets, with the test set held out throughout the model-development process.

A. Development Setup

The machine learning pipeline, including signal processing, model training, and testing, was implemented in Python. Jupyter Notebooks supported exploratory analysis and visualization, while Visual Studio Code [25] provided an integrated environment for Python and notebook development. Most programming was done locally, but signal processing ran on a Services for Sensitive Data (TSD) virtual machine due to data sensitivity. Processed features were then securely transferred for further analysis. Git enabled version control and experimental branching [26], and a Python virtual environment ensured reproducibility and dependency isolation [27], facilitating deployment on the TSD platform. For the rest of this manuscript, any references in verbatim relate to functions in this environment.

B. Preprocessing

Raw audio data were preprocessed through cleaning, structuring, and downsampling, as shown in Figure 4. Downsampling was performed using librosa (v0.10.2) [28], selected for its comprehensive suite of audio analysis functionalities, with the load function reading the recorded WAV files and applying high-quality sinc interpolation and anti-aliasing filters [29]. A sampling rate of 22 050 Hz was selected to balance spectral detail and computational efficiency, capturing frequencies up to 11 025 Hz, consistent with acoustic signal characteristics discussed in Section II-D.

FIGURE 4.

Flowchart of the preprocessing pipeline. Raw audio is downsampled, notch-filtered, segmented, and labeled. Green blocks indicate cleaning steps, while blue blocks show how the data is structured for later use.

To reduce powerline interference, a notch filter targeting 50 Hz was applied using scipy.signal.iirnotch with a Q-factor of 30 [30]. This frequency overlaps with key physiological signals—heart (20–150 Hz) and respiratory sounds (60–700 Hz) relevant to meal intake detection [31], [32]. Preserving these bands ensures signal integrity and analysis accuracy. Cleaned signals were segmented into 1 s windows with 50% overlap, resulting in a sequence of 70 overlapping segments to capture temporal dynamics and isolate events like chewing or swallowing, as shown in Figure 5.

FIGURE 5.

Representation of the segmentation process. Each segment combines new signal data with an overlapping portion of the previous one, continuing until the full signal is covered.

To support supervised learning, each signal segment was labeled as ‘meal’ or ‘non-meal’ based on metadata in accompanying JSON files specifying meal intervals. Labels were computed manually using the sampling rate, segment duration, and overlap, with metadata parsed via the json package [33]. In addition to the meal/non-meal labels, for 50 % of the recordings we had another process of manually attaching ‘chewing’ and ‘swallowing’ labels to segments of the recorded data. This process was performed through a combination of listening to the audio and inspecting the spectrogram. These labels were not used in the supervised learning, but in the interpretation of the HMM results, as described in subsection IV-G and used in Figures 14, 15, and 29.

FIGURE 14.

Most probable state transition for the HMM for meal data with annotated chewing and swallowing. In the plot, chewing is marked green and swallowing is red, following the manual labeling scheme described in subsection IV-B.

FIGURE 15.

Heatmap of the correspondence between the manually annotated label (following the manual labeling scheme described in subsection IV-B) and the most probable hidden state.

C. Feature Extraction

To enable HMM modeling, features capturing temporal and spectral properties were extracted from each signal segment to reflect meal-related activity. Segment features were stored in arrays per recording, then combined into a larger array representing the full dataset, as shown in Equation 3.

Time-domain features such as zero-crossing rate and Root-Mean-Square Error (RMSE) were computed, while frequency-domain analysis involved converting segments into Mel spectrograms using librosa [28]. To do this, an FFT window size of and a hop length of 512 were set. To reduce complexity, key spectral features, including centroid, bandwidth, contrast, roll-off, flatness, and 39 Mel Frequency Cepstral Coefficient (MFCC), were extracted to compactly represent signal characteristics [34], [35]. A total of 39 MFCCs were computed per time step using librosa.feature.mfcc, providing a compact representation of the signal’s timbral properties, while the remaining spectral features were extracted via librosa.feature functions. Figure 6 compares spectrograms from two 30 s windows extracted from the meal and non-meal portions of a single recording. The upper panel represents a period without movement (Figure 6a), whereas the lower panel corresponds to a meal-related activity (Figure 6b). Meal-related recordings showed higher energy and denser frequency content in spectrograms, especially between 64 Hz–2048 Hz, indicating chewing and swallowing (particularly in the 256–2018 Hz band). Features were averaged per segment, normalized using StandardScaler from sklearn.preprocessing to ensure scale consistency, and cached for reuse and to reduce computation. To illustrate how the MFCC features relate to the underlying spectrograms, Supplementary Material A includes three visualizations showing the successive derivation steps for both types of sound (Figure 16), which reveal that chewing and swallowing are also characterized by higher energy content and denser frequency patterns.

FIGURE 6.

Spectrograms illustrating a non-eating portion of the recording (top) and a meal-related portion that includes associated activities (bottom).

D. Feature Selection

Irrelevant variables can hinder model performance by adding noise [36]. To mitigate this, feature selection methods were performed exclusively on the training dataset and were used to reduce dimensionality, improve accuracy, and lower computational cost [37]. The test dataset was kept fixed throughout model development and was not used during feature selection, ensuring a consistent and unbiased evaluation of the final model. The two feature selection methods described below identified the most informative variables, and the resulting subsets were assessed using 5-fold cross-validation.

1). Feature Ranking Metrics

The following two ranking metrics were calculated and subsequently used for the final feature selection. Each metric was used to rank the features from 1 (most relevant) to 46 (least relevant).

Mutual Information (MI) quantifies correspondence between each feature and the target [38]. Features were ranked by relative MI values. While MI is fast and effective, it evaluates features individually, ignoring interactions that may boost performance, and cannot adapt to model-specific dynamics [39].

Sequential Feature Selection (SFS) is a greedy algorithm that incrementally adjusts the feature set to improve model performance [40]. This study used forward selection, beginning with an empty set and adding the feature that yielded the largest gain until no further improvement occurred [41]. The estimator with fixed hyperparameters guided selection, with k_features set to one less than the total number of features to obtain a full ranking. The subsets_ attribute captured the selection path, allowing earlier-chosen features to be ranked as more important, while the feature never selected ranked lowest. SFS was applied with the groups parameter, though it is computationally expensive and may overlook interactions the estimator cannot model.

2). Final Feature Selection and Evaluation

The feature rankings were used to evaluate model performance across varying subset sizes. Starting from the top-ranked feature, subsets were extracted from the training data incrementally and assessed using 5-fold cross-validation via Mlxtend (n_jobs=1), with entire recordings assigned to single folds GroupKFold to preserve sequence [42], treating each recording as a separate group. For each subset, models were trained and validated, and performance was measured by accuracy and F1 macro scores averaged across subjects, folds, and runs using np.mean. Results were plotted in matplotlib [43] with standard deviation.

E. Training

In this study, the LDA model and HMM were constructed due to their ability to support sequential classification. LDA was used as a benchmark, thanks to its simplicity and multiclass efficiency, to highlight the temporal dependencies modeled by HMMs.

1). Classification Pipeline

The selected features were used for model development and hyperparameter tuning using 5-fold cross-validation, with macro F1-score as the primary performance metric. As mentioned before, the test set of 20% was fixed and was not used for feature selection or hyperparameter optimization, thereby preventing information leakage and ensuring an independent evaluation of the final model. In the 5-fold cross-validation (sklearn.model_selection.KFold [44]), each recording was kept contiguous within a single fold, such that all segments from a given recording were assigned to either the training or validation set, and each recording was used once for validation. As illustrated in Figure 7, the optimized model was subsequently retrained using the complete training set and evaluated on the previously unseen test set to assess its performance on held-out data.

FIGURE 7.

Each step of the procedure for training the HMM, beginning with raw data and ending with the evaluation of the model. No data from the test set was used in feature selection, hyperparameter tuning, or model training. Inspired by [45].

To further assess generalization to unseen participants, LOSO CV was performed as an alternative to the 80:20 train-test split. In each iteration, data from one participant were held out for testing, while data from all remaining participants were used for training. This subject-independent evaluation better reflects the expected deployment scenario in which the model is applied to a previously unseen individual. LOSO CV was implemented using sklearn.model_selection.GroupKFold, with participant identity used as the grouping variable. Results were analyzed in Jupyter using pickle files containing labels, predictions, and metadata. Accuracy, recall, F1-score, and confusion matrices were computed using scikit-learn 1.5.2 [44], while specificity was calculated manually. Detection time was defined as the difference between the actual and predicted meal start times.

Macro F1-score was chosen as the main metric due to the dataset’s class imbalance (80% non-meal). Unlike accuracy, which can favor the majority class, macro F1 equally weighs precision and recall across classes, as shown in Equation 4, offering a fairer evaluation of meal detection performance [46].

Model performance was visualized using matplotlib 3.9.2 [43], featuring Receiver Operating Characteristics (ROC) curves and temporal plots comparing predictions to ground truth. These, along with metrics, provided a thorough assessment of reliability, accuracy, and timing.

2). Hidden Markov Model

The meal subtypes were merged into a single ‘meal’ class to reduce variability, with standardized noise included in the training. To ensure consistent accuracy evaluation, the predictions and labels were post-processed and mapped to a binary representation, as aforementioned, with 0 representing non-meal and 1 representing meal. This step guaranteed that errors in distinguishing among different non-meal activities did not affect the evaluation of the primary binary meal-detection task. As a result, the final sequential prediction results reflect the model’s ability to distinguish meals from all types of non-meal activities present, providing a consistent basis for comparison. The model used GMMHMM from HMMlearn 0.3.3 [47], adapted for scikit-learn, with separate models per class, with decisions based on their differences, allowing threshold-based performance evaluation. Training preserved sequence structure, while predictions used log-likelihood scoring on subsequences, yielding probabilistic outputs via posterior probabilities. Subsequence generation involves combining features from a fixed number of consecutive segments. Using a sliding window, each new subsequence drops the first segment and appends the next, maintaining temporal continuity. Hyperparameters were optimized using sklearn.model_selection.GridSearchCV [48] with GroupKFold [42] to preserve temporal continuity across recordings, guided by macro F1-score. A custom HMMWrapper, built on BaseEstimator and ClassifierMixin, ensured scikit-learn compatibility and enforced a two-class constraint. Hyperparameter search tested [1, 3, 5, 7, 9] hidden states and Gaussian components for meal and non-meal models, using a step of 2 to reduce cost. Parallel evaluation n_jobs saved results to a pandas CSV. The final model was retrained with optimal settings.

F. Testing

As mentioned in Section IV-E1, 20% of the recordings were used for testing. After the training and validation phases, the resulting models were evaluated using a test set that had not been utilized in any of the training steps.

G. Explainable Artificial Intelligence

Eating typically follows a sequence—chewing, swallowing, and digestion—though liquid meals may skip the chewing step. To assess whether the HMM captured key acoustic events at meal onset and improve interpretability, XAI techniques were applied across three levels. Pre-modeling analysis focused on feature selection and performance metrics [45]. In-model methods, inspired by DPVis [49], visualized hidden state transitions linked to events such as chewing and swallowing. Post-model visualization helped interpret output-event mappings. Together, this XAI strategy clarified data relevance, model behavior, and prediction outcomes. To apply XAI methods, some recordings were decoded using the model’s decode function [47] to extract the most likely hidden state sequences. Others were manually labeled for chewing and swallowing, following the method described in subsection IV-B. These sequences were plotted against annotations for visual comparison (Figures 14, 15, and 29). To assess state-event alignment, the decoded states were analyzed for consistency with chewing or swallowing, and a heatmap was generated to compare annotated labels with predicted states.

V. Results

A. Feature Selection

Results for feature rank metrics are shown in Table 2 with color coding to aid interpretability. MI and SFS ranks diverge notably for features such as spectral centroid, spectral bandwidth, and spectral attenuation. Some features align across methods: MFCC1 and MFCC12 rank high in both, while MFCC3, 24, 26, 28, 29, 31, and 38 consistently rank low.

TABLE 2. Feature-importance ranking for HMM feature selection. Dark green marks the most important features, while higher ranks in dark purple indicate lower importance. Corresponding LDA rankings are provided in the Supplementary Material A, Table 5.

Feature	MI	SFS	Feature	MI	SFS
Zero-Crossing Rate	4	28	MFCC16	22	4
Root-Mean-Square Energy	1	7	MFCC17	37	16
Spectral Centroid	3	39	MFCC18	11	27
Spectral Bandwidth	2	44	MFCC19	33	10
Spectral Contrast	20	2	MFCC20	15	43
Spectral Rolloff	6	46	MFCC21	25	12
Spectral Flatness	7	24	MFCC22	18	30
MFCC0	28	9	MFCC23	45	23
MFCC1	5	8	MFCC24	23	45
MFCC2	24	22	MFCC25	42	18
MFCC3	10	33	MFCC26	32	37
MFCC4	27	42	MFCC27	34	5
MFCC5	13	1	MFCC28	30	41
MFCC6	19	26	MFCC29	44	32
MFCC7	29	14	MFCC30	26	19
MFCC8	31	3	MFCC31	46	31
MFCC9	38	34	MFCC32	16	38
MFCC10	43	29	MFCC33	36	6
MFCC11	35	21	MFCC34	17	20
MFCC12	9	13	MFCC35	41	17
MFCC13	14	36	MFCC36	12	25
MFCC14	8	15	MFCC37	40	11
MFCC15	39	35	MFCC38	21	40

B. Model Performance of HMM with Ranked Features

Preserving temporal structure is crucial for HMMs to learn realistic transition dynamics. Figure 8 shows how performance changes with feature subset size, highlighting trends in accuracy and macro F1-score. The meal model used 5 states and 3 Gaussians, while the non-meal model used 7 states and 6 Gaussians.

FIGURE 8.

HMM performance using top-n ranked features across selection methods. Curves show average accuracy and macro F1-score over 3 runs. Shaded areas indicate standard deviation.

Accuracy improves up to 25 features, after which MI plateaus and SFS slightly declines. For macro F1-score, MI rises steadily, while SFS peaks around 14 features and drops after 23. Overall, SFS yields the best performance, with MI close in accuracy. Compared to LDA (see Figure 17 reported Supplementary Material A), both metrics show similar trends: most gains occur with fewer features, with HMM being more variable and consistently outperforms. HMM reaches 0.90 accuracy and 0.83 F1-score, versus 0.89 and 0.73 for LDA. Selecting 14–23 features via SFS balances performance and model complexity. The analysis identified key features that boosted the model’s performance: SFS favored spectral contrast, and several MFCCs (7, 8, 16, 19, 23, 37), while MI highlighted spectral bandwidth, centroid, MFCC1, and MFCC12. Both models struggled with single features but improved notably at 12 (LDA) and 14 (HMM) features. Using all features reduced accuracy, prompting subset testing. The best results came with 23 features, though sequential predictions, shown in Figure 9, resembled those from the 14-feature model (Figure 20), salads and complex meals were well detected, soups yielded false positives, and bread-based meals remained challenging. Results for different feature configurations are detailed in Supplementary Material A.

FIGURE 9.

Sequential segment predictions across five recordings (out of 9 test recordings) using 23 features selected by SFS for the HMM.

C. Hyperparameter Tuning

Hyperparameter tuning was assessed using heatmaps for the top 14 and 23 SFS-selected features. Performance peaked in the lower-right regions, with 23 features giving slightly better results. The non-meal model improved with more states and mixtures, while the meal model performed best with fewer, consistent across both feature sets. Macro F1-scores for all pairs of number of hidden states ( ) and number of Gaussian mixtures ( ) using 23 features are shown in Figure 10, the strongest configuration after accounting for shrinkage and solver effects [50], [51], [52]. The sequential predictions for the five validation recordings using the tuned 23-features model are shown in Figure 24 and the 14-feature heatmap appears in Figure 27 (Supplementary Material A). Heatmaps show overall trends but can miss globally optimal settings. Table 3 thus lists the top three hyperparameter sets, revealing a wide variation: meal models use 1–7 states, non-meal models 3–9. The best 23-feature score (0.847) exceeds the top 14-feature score (0.831), and all top 23-feature results outperform the 14-feature ones, underscoring the value of richer feature sets.

FIGURE 10.

Macro F1-score heatmaps for HMM with 23 features by using the SFS method. Light/yellow color indicates a higher F1-score. Left: shrinkage and solver effects; right: solver tolerance impact.

TABLE 3. Best hyperparameter combinations and scores when using 14 and 23 features for meal and non-meal models.

	Meal	No meal	Meal	No meal	F1-score
14 feat	1	3	7	9	0.831
	3	7	7	7	0.831
	7	7	9	5	0.830
23 feat	5	9	7	5	0.847
	3	9	7	5	0.838
	7	9	3	5	0.833

D. Separation of Classes

As noted in subsection V-C, noisy non-meal activities caused classification confusion even after hyperparameter tuning. To address this, a multiclass approach was tested by adding a noise class during training and prediction. However, results in Figure 25 in Supplementary Material A show no performance improvement compared to previous analyses.

E. Probability Analysis

Meal classification used a default threshold, labeling meals when their probability exceeded that of non-meals. To assess robustness, a probability analysis varied this threshold across the same five recordings using log-probability differences from HMMs with 23 SFS-selected features. As shown in Figure 11, clear meal peaks appeared in 35jaLn and bYUN6g, while 3BcKJP stayed below zero. Hvoq3y showed spikes during meals and physical activity, while T6PCJU had elevated values from speech and minor noise events. Adjusting the threshold couldn’t reliably separate noise without harming overall meal detection. The HMM was evaluated via 10-fold cross-validation across thresholds from -600 to 600 in steps of 20 (Figure 12a). Segments were labeled as meals when their log-probability exceeded non-meal scores by the threshold. Accuracy peaked (~0.91) near 50, while macro F1-score was highest at 0 and dropped sharply beyond, though accuracy remained stable over a wider range. The ROC curve in Figure 12b shows average sensitivity and (1-specificity) across thresholds. The point closest to the top-left corner occurs at -60, with a True Positive Rate (TPR) of ~0.90 and False Positive Rate (FPR) of ~0.10. However, the highest macro F1-score was at threshold 0, which slightly lowers both TPR and FPR.

FIGURE 11.

Log-probability differences across five recordings using HMM with 23 SFS-selected features. The black line shows meal vs. non-meal likelihood, dotted line is the threshold for detection; green and red areas mark true meal and non-meal periods. Recording names appear in each plot.

FIGURE 12.

(a) Shows HMM performance across probability thresholds for meal prediction, with each curve giving the average over all training folds. (b) shows the ROC curve (TPR vs. FPR) across HMM probability thresholds. The grey dashed line marks random performance, the red dot the best TPR/FPR trade-off, and the green dot the highest macro F1-score.

F. Train-Test Data Split

After tuning, the HMM was evaluated on nine unseen recordings from the 20% test split, using five states and seven mixtures for meals and nine states with five mixtures for non-meals. Predictions used 70 overlapping segments and 23 SFS-selected features. As shown in Table 4, the model achieved an average detection time of 11.7 s, slightly faster than LDA’s 12.3 s. The HMM outperformed LDA, with macro F1-score rising from 0.839 to 0.879, mainly due to higher sensitivity (0.907 vs 0.780). Specificity dipped slightly (0.918 vs 0.930), while precision improved to 0.764. Overall accuracy reached ~91%, indicating better meal detection with a small increase in false positives.

TABLE 4. Test results for tuned HMM and LDA model.

Test	HMM	LDA
Accuracy	0.913	0.895
Sensitivity	0.907	0.780
Precision	0.764	0.744
Specificity	0.918	0.930
Macro F1-score	0.879	0.839
Detection time (s)	11.7	12.3

Figure 13 shows sequential predictions for 3 of the test recordings (results for more recordings are reported in Figure 26 in Supplementary Material A). All meal events were detected with minor false positives, showing occasional confusion with non-meal activities.

FIGURE 13.

Sequential HMM predictions, after hyperparameter tuning, across three test recordings using 23 SFS-selected features. The black line shows binary meal labels; green and red areas indicate true meal and non-meal periods.

G. Leave-One-Subject-out Cross-Validation

To investigate cross-subject generalisability, a separate LOSO CV analysis was subsequently performed. In each iteration, all recordings from one subject were held out and the model was evaluated on that subject, after the remaining subjects were used for training. This analysis was conducted as a complementary assessment of subject-wise generalisation and did not replace the predefined 80:20 train-test evaluation. Because feature selection and hyperparameter tuning were performed prior to this analysis, the LOSO CV results should be interpreted as an assessment of cross-subject robustness rather than as a fully independent estimate of generalisation performance [53]. Results after applying LOSO CV are reported in Figure 28 in the Supplementary Material A, showing no substantial differences: one recording exhibited improved predictions, another showed an increase in false positives, and the third remained unchanged.

H. Explainable Artificial Intelligence

XAI was applied to the optimized HMM (5 hidden states, 7 Gaussian mixtures) using 23 SFS-selected features. Figure 14 shows the most likely state transitions across three annotated meal recordings (Figure 29 in Supplementary Material A shows more recordings). Clear chewing–swallowing alternation appears in meals like salad and oats (e.g., 37Bf8z, shifting between states 0 and 3). State 1 often reflects swallowing, though its role varies. Soup recordings (e.g., 3A9kCr) show repeated transitions from states 3–4 to 1. Some oats recordings show oscillatory patterns (3av2QO), while others cluster around states 1 and 3 (KThDxB). Recording 3BcKJp is dominated by state 4 with minimal switching, whereas Q4g6f8 shows rapid transitions consistent with frequent chewing.

A heatmap of label counts per hidden state (Figure 15) shows that background dominates states 0 and 2, while chewing and swallowing align most with states 1 and 3. These states often capture chewing and swallowing, though no state corresponds exclusively to a single behavior.

VI. Discussion

Meal classification works well with statistical models like HMM. Performance plateaus after roughly 20 features, adding more yields minimal benefit while risking overfitting, as seen in the F1-scores of HMM and the accuracy of LDA in Figure 8. Feature selection therefore boosts reliability by emphasizing informative inputs. HMM curves fluctuate more than LDA due to fewer runs (3 vs. 10), but overall trends remain stable, indicating robust results despite the higher computational cost [54]. Regarding the results in the feature rankings across the selection methods presented in Table 2, the differences are expected because each method evaluates feature relevance according to a different criterion. MI assesses the dependence between each individual feature and the target variable [55], whereas SFS evaluates the incremental contribution of a feature to model performance given the features already selected [56]. Among the methods tested in Table 2, SFS delivered the best accuracy and macro F1-score, though it was the most computationally expensive. MI was the fastest but prone to selecting redundant features [57]. Therefore, SFS was chosen for the final feature selection, with the robustness potentially improved by combining the rankings between the methods and the runs [58].

The hyperparameter adjustment of HMM showed substantial variability, with no consistently better performance as can be seen in Figure 24 in Supplementary Material A. Since HMMs are typically trained using an iterative optimization method such as Baum-Welch, which may converge to different local optima depending on parameter initialization, rather than converging to a global optimum, training the model under similar conditions can result in variations in the estimated model parameters and, consequently, predictive performance [59]. This variability can be mitigated by using multiple random initializations, selecting the best-performing model based on validation data, and reporting performance across repeated training runs. This makes identifying a global optimum difficult, particularly for small datasets where the parameters may not generalize [60]. Variability also affects interpretability: the best model with 14 features used only one hidden state, failing to separate chewing from swallowing, while models with three or more states captured clearer meal phases. Models with high performance often required multiple Gaussian mixtures per state, indicating intra-state variability from individual differences, noise, or microphone placement, and showing that a single Gaussian is insufficient (Figure 10). While greater complexity can really boost the expressiveness of a model, it also comes with the risk of overfitting and higher computational costs. On the other hand, models that are too simple might overlook important transitions. When deploying models in real-world situations, it’s all about finding the right balance between richness, efficiency, and generalizability to ensure the best performance.

Noise-related activities were included to mimic real-world situations. However, they can be tricky to tell apart from real eating moments, since talking, gesturing, and general movement often resemble eating signals and can lead to false positives (see Figure 25 in the Supplementary Material A). Even with an accuracy above 90%, such errors are dangerous in closed-loop insulin delivery, where misclassification could trigger insulin without food and increase the risk of hypoglycemia. Precise detection of meal onset is more critical than avoiding missed meals, underscoring the need to strengthen model robustness.

Threshold tuning improved the balance between sensitivity and specificity, showing that LDA can nearly match HMM performance (accuracy ~0.90, F1-score 0.78 for LDA vs. 0.92 and 0.85 for HMM in Figure 12a). However, recordings varied: Some meal and non-meal probabilities were clearly separated, while others showed elevated values during speech or movement, which increased false positives. Raising thresholds reduces these errors but also lowers true positives, reflecting the sensitivity–specificity trade-off seen in the ROC curve (Figure 12b) [61]. This indicates that thresholding alone is insufficient, and additional safeguards, such as context-aware sensing or temporal pattern analysis, are needed for robustness.

Test-set results showed similar overall performance for LDA and HMM (Table 4), with the HMM achieving higher sensitivity and macro F1 but producing more false positives and requiring longer computation. Both models struggled most with noise activities like talking or movement, which were harder to distinguish from meals than silence. False positives often appeared as short spikes; more frequent in HMM (Figure 13), while LDA outputs were smoothed by averaging, indicating that post-processing could help, though with trade-offs in sensitivity, precision, and timeliness. One recording performed notably worse, likely due to microphone issues, highlighting sensitivity to small variations and the need for larger, more diverse datasets to improve robustness. Applying LOSO CV provided insight into generalizability, with performance closely matching a randomized 80:20 split. As shown in Figure 28, this consistency indicates that the model learns patterns beyond subject-specific traits and can generalize to unseen users.

Regarding the XAI analysis, HMM state transitions aligned well with chewing and swallowing, capturing meal phases useful for estimating meal size and duration. As seen in Figures 14 and 15, the model represented transitions more effectively than single actions, highlighting the strength of sequential methods. Some recordings (e.g., 3BcKJp) showed misalignment, possibly due to annotation or equipment issues. XAI also suggested that partial labels for chewing and swallowing [62] could improve accuracy and interpretability. Overall, performance reflected a variability in data quality, labeling, and microphone conditions, emphasizing the importance of robust annotation, sensor reliability, and balanced model design for real-world deployment.

VII. Concluding Remarks and Future Directions

This pilot study underscores both the promise and the challenge of detecting meal onset for improving insulin delivery in artificial pancreas systems using sequential models. Although enhanced probability analysis narrows the gap between LDA and HMM, high-confidence errors during noise events remain a major obstacle. Achieving reliable performance in real-world situations will require models that better separate true eating from everyday activities and associated sources of noise. For approaches prone to false positives, post-processing can reduce errors, but it introduces trade-offs among robustness, sensitivity, and timeliness. Future work should explore sequential modeling using a recurrent neural network architecture with long short-term memory to provide an alternative approach. Additionally, it should reinforce feature selection by repeating ranking procedures to identify stable predictors and by using dimensionality reduction methods such as principal component analysis to handle correlations [63]. Eliminating false positives during non-meal activities is crucial for safe real-world deployment, especially in automated insulin delivery. Improved feature engineering, more robust sequential models, and partial supervision may further enhance discrimination between meals and similar activities while increasing interpretability and reliability. Finally, the dataset, which includes only 50 recordings from 10 subjects without diabetes, represents a significant limitation. Developing a more robust and broadly applicable mode, one with stronger confidence, will require a larger and more diverse dataset.

To ensure the model is relevant and reliable for its intended application, the follow-up study should incorporate recordings collected under varied and realistic conditions, as well as data from participants with T1D. Alternatively, in addition to expanding the dataset, complementary approaches should be considered, such as incorporating more robust, wearable systems that are less dependent on acoustic conditions. For example, a system combining breathing and hand movement signals with a microphone placed on the throat could improve robustness against ambient noise and sensor artifacts while maintaining the specificity of acoustic signals related to swallowing and chewing for meal detection. Such multimodal sensing could complement, rather than replace, existing CGM-based meal-detection strategies by providing an independent indication of meal onset while CGM continues to provide information on the resulting glucose dynamics. In a future automated insulin-delivery system, the detected meal onset could be transmitted to a smartphone, wearable processor, or insulin-pump controller and integrated with existing CGM-based control algorithms. This could enable earlier detection of unannounced meals and potentially provide a prompt for a missed meal bolus or serve as an input to future automated insulin-delivery algorithms. However, extensive validation under free-living conditions will be required before such a system can be considered for clinical use.

Data and Code Availability

The raw data is sensitive and thus confidential (according to the ethical approval), but the calculated features and the code can be shared on request.

Declaration of Competing Interest

The authors have no competing interests to report.

Funding Statement

This work was supported in part by the Research Council of Norway under Project 294828 and in part by the Centre for Digital Life Norway.