A comparative analysis of unsupervised machine‐learning methods in PSG‐related phenotyping

Summary

Obstructive sleep apnea is a heterogeneous sleep disorder with varying phenotypes. Several studies have already performed cluster analyses to discover various obstructive sleep apnea phenotypic clusters. However, the selection of the clustering method might affect the outputs. Consequently, it is unclear whether similar obstructive sleep apnea clusters can be reproduced using different clustering methods. In this study, we applied four well‐known clustering methods: Agglomerative Hierarchical Clustering; K‐means; Fuzzy C‐means; and Gaussian Mixture Model to a population of 865 suspected obstructive sleep apnea patients. By creating five clusters with each method, we examined the effect of clustering methods on forming obstructive sleep apnea clusters and the differences in their physiological characteristics. We utilized a visualization technique to indicate the cluster formations, Cohen's kappa statistics to find the similarity and agreement between clustering methods, and performance evaluation to compare the clustering performance. As a result, two out of five clusters were distinctly different with all four methods, while three other clusters exhibited overlapping features across all methods. In terms of agreement, Fuzzy C‐means and K‐means had the strongest (κ = 0.87), and Agglomerative hierarchical clustering and Gaussian Mixture Model had the weakest agreement (κ = 0.51) between each other. The K‐means showed the best clustering performance, followed by the Fuzzy C‐means in most evaluation criteria. Moreover, Fuzzy C‐means showed the greatest potential in handling overlapping clusters compared with other methods. In conclusion, we revealed a direct impact of clustering method selection on the formation and physiological characteristics of obstructive sleep apnea clusters. In addition, we highlighted the capability of soft clustering methods, particularly Fuzzy C‐means, in the application of obstructive sleep apnea phenotyping.

1. INTRODUCTION

Obstructive sleep apnea (OSA) is recognized as a multifactorial and heterogeneous sleep disorder with a high prevalence (de Araujo Dantas et al., ²⁰²³). At present, the diagnosis and assessment of OSA severity predominantly rely on a single polysomnographic (PSG) index, the apnea–hypopnea index (AHI). However, the AHI only reflects the frequency of respiratory events, thus ignoring a vast amount of physiological information available on PSGs, as well as leaving out essential information on metabolic, cardiovascular and neural physiology that characterize the heterogeneity of OSA (Malhotra et al., 2021). Also, the first‐line treatment for OSA has centered around continuous positive airway pressure (CPAP; Semelka et al., ²⁰¹⁶), which heavily relies on the AHI, and thus it might not be the optimal choice for all patients with OSA (Randerath et al., 2021; Sutherland et al., ²⁰¹⁸). To address these shortcomings, several studies have employed cluster analyses to discover distinct groups of patients with similar characteristics (i.e. OSA phenotypes), aiming to fully capture the heterogeneity of OSA reflected in PSGs (Zinchuk et al., 2017; Zinchuk & Yaggi, ²⁰²⁰). Such methods have also shown great potential in personalized OSA management and treatment (Bonsignore et al., 2017; de Chazal et al., ²⁰²⁰; McNicholas & Korkalainen, ²⁰²³; Randerath et al., ²⁰²²; Verbraecken et al., ²⁰²²).

Unsupervised machine‐learning methods have played a key role in the main core of cluster analysis, enabling the analysis of complex and multidimensional health data, like PSG, without requiring prior label knowledge (Pitafi et al., 2023). In addition, unsupervised machine‐learning methods can enable the integration of the PSG‐based variables with anthropometrics and demographic characteristics, symptoms, comorbidity profiles, as well as questionnaire data (Zinchuk et al., 2017). Therefore, these methods are well suited for discovering various OSA phenotypes.

However, the choice of clustering methods is a recognized challenge of cluster analysis in various research fields (Jaeger & Banks, 2023). This has also been quite controversial in the field of OSA phenotyping as previous studies have reported that the clustering findings heavily depend on the chosen clustering method (Kim et al., 2020; Zhang et al., ²⁰²²; Zhu et al., ²⁰²³); clustering outputs might change by utilizing different clustering methods and settings. In this regard, follow‐up analyses and the development of targeted treatment strategies might be significantly biased, leading to incorrect OSA treatment and unsatisfied patient outcomes.

In this comparative study, we aimed to investigate how the clustering method selection affects the cluster formations and physiological outcomes by utilizing a large number of individuals with suspected OSA. We applied assorted unsupervised machine‐learning methods and compared the clustering outputs across all methods. In addition, we evaluated the agreements and similarities between the clustering methods and examined the clustering performance of applied methods through internal cluster validation indices (Ran et al., 2022).

2. MATERIALS AND METHODS

2.1. Clinical PSG recordings

We conducted a retrospective study for 912 consecutive individuals referred for PSG at the Princess Alexandra Hospital, Brisbane, Australia due to OSA suspicion. PSGs were recorded between 2015 and 2017 using the Compumedics Grael 4 K PSG system (Abbotsford, Australia). All PSGs were conducted, adjusted and analysed by medical experts following the newest guidelines available at the time of analysis, set by the American Academy of Sleep Medicine (AASM; Troester et al., 2023). As OSA has been linked with a wide range of physiological outcomes, particularly neurocognitive impairment (Tam et al., 2014), we used a psychomotor vigilance task (PVT) to compare the vigilance levels of individuals within clusters. In our study, all individuals completed a PVT in the evening before the PSG. In PVT, a total of 120 visual stimuli were presented on a touchpad screen, and individuals were instructed to react by pressing a screen with their thumb or index finger as fast as possible. If the reaction time exceeded 500 ms, it was counted as a lapse. The Metro South Human Research Ethics Committee (Brisbane, Australia) accepted the retrospective data collection and research use of the data (LNR/2019/QMS/54313).

For further analysis, individuals with incomplete PSG recording or demographic information (n = 40) or a total sleep time (TST) of less than 1 hr (n = 7) were excluded. After exclusion, a total of 865 individuals with suspected OSA were included in the final analysis.

2.2. Data preprocessing

We selected 42 clustering variables after careful review and examination. We used only continuous variables, and thus no categorical variables like sex were included. The selected variables included nine domains: heart rate variability (HRV); sleep architecture; breathing disturbance; blood oxygen saturation (SpO₂)‐based variables; snoring; periodic limb movement (PLM); arousals; PVT outcomes; and a domain consisting of questionnaire, demographic and anthropometric data (Table 1).

TABLE 1.

Domains of selected variables.

Variable category	Variables
Questionnaire, demographic and anthropometric data	ESS (points)
	BMI (kg m−2)
	Age (years)
Sleep architecture	WASO (min)
	TST (min)
	Sleep efficiency (%)
	Percentage of stage N1 (% of TST)
	Percentage of stage N2 (% of TST)
	Percentage of stage N3 (% of TST)
	Percentage of REM sleep (% of TST)
Breathing disturbance	AHI (events per hr)
	Hypopnea index (events per hr)
	Apnea index (events per hr)
	Central apnea index (events per hr)
	REM AHI (events per hr)
	NREM AHI (events per hr)
	Supine AHI (events per hr)
	Non‐supine AHI (events per hr)
	Mean apnea duration (s)
	Mean hypopnea duration (s)
HRV	Average RR (s)
	pRR50 (%)
	RMSSD (s)
	SDRR (s)
	LF/HF ratio
SpO2‐based variables	Oxygen desaturation index (events per hr)
	Desaturation severity (%‐unit)
	Desaturation duration (%‐unit)
	Total transient drop severity (%‐unit)
	Recovery severity (%‐unit)
	Recovery duration (%‐unit)
	Obstruction severity (s%)
	Percentage of time spent below 90% oxygen saturation (% of TST)
	Median duration ratio
	Median depth ratio
	Median area ratio
	Median slope ratio
Snoring	Snoring time (% of TST)
Arousals	Arousal index (events per hr)
PLM	PLM index (events per hr)
PVT outcomes	Median reaction time (ms)
	Number of lapses

AHI, apnea–hypopnea index; average RR, average of RR intervals; BMI, body mass index; ESS, Epworth Sleepiness Scale; HRV, heart rate variability; LF/HF, low‐frequency to high‐frequency ratio; NREM, non‐rapid eye movement; PLM, periodic limb movement; pRR50, percentage of RR intervals differing by more than 50 ms; PVT, psychomotor vigilance task; REM, rapid eye movement; RMSSD, root mean square of successive differences; SDRR, standard deviation of RR intervals; SpO₂, blood oxygen saturation; TST, total sleep time; WASO, wake after sleep onset.

In addition to conventional SpO₂ variables (i.e. oxygen desaturation index and percentage of time spent below 90% oxygen saturation [T90]), we calculated several more comprehensive SpO₂ variables (Figure 1) by using the ABOSA software (Karhu et al., 2022). The included variables primarily focus on the depth, area and duration of saturation and resaturation oxygen events. They have also been shown to more accurately characterize the hypoxic load and indicate the severity of OSA compared with conventional variables (Kulkas et al., 2013). These variables included desaturation severity (the sum of desaturation event areas divided by TST), desaturation duration (the sum of desaturation event durations divided by TST), recovery severity (the sum of recovery event areas divided by TST), recovery duration (the sum of recovery event durations divided by TST) and total transient drop severity (the sum of desaturation and recovery areas from the 100% reference). Moreover, we calculated median ratio parameters between desaturation and recovery event depths, areas, durations and slopes. In addition, we calculated the obstruction severity (Kulkas et al., 2013) with the following formula:

$$\text{Obstruction Severity}\left(s\%\right)=\frac{\sum _{n=1}^L\left({\text{HypDur}}_n\times {\text{DesatArea}}_n\right)+\sum _{n=1}^{\mathrm{L}}\left({\text{ApDur}}_n\times {\text{DesatArea}}_n\right)}{\mathrm{TST}}$$ (1)

where $L$ is the number of desaturation events, $\text{DesatArea}\left(s\%\right)$ is a desaturation area of an individual event (Figure 1), and $\text{HypDur}\left(s\right)$ and $\text{ApDur}\left(s\right)$ are the durations of individual hypopnea and apnea events, respectively. In Equation (1), each hypopnea and apnea event were linked to a desaturation event.

FIGURE 1.

The visualization of SpO₂‐based variables.

Also, for conventional overnight (long‐term) time‐ and frequency domain HRV metrics, we detected the R peaks from electrocardiographic signals and corrected the peaks using Kubios HRV Premium 3.4.1 (Kubios Oy, Kuopio, Finland) software with default settings (Tarvainen et al., 2014). The average of RR intervals (average RR) and standard deviation of RR intervals (SDRR) were calculated to reflect the overall HRV. The percentage of RR intervals differing by more than 50 ms (pRR50) and root mean square of successive differences (RMSSD) were calculated to reflect the parasympathetic activity. In addition, we calculated the power spectral densities (PSDs) from each 5‐min segment of the detrended RR series with Welch's method (eight sections, 50% overlap, Hamming window). From the PSDs, the low‐frequency to high‐frequency band power ratio (LF/HF) ratio was calculated to illustrate the sympathovagal balance (Shaffer & Ginsberg, 2017).

Before clustering, the selected variables were normalized by utilizing the Z‐score technique (Schaffer & Green, 1996). The normality of the cluster distributions was examined using the Kolmogorov–Smirnov test. Also, the difference between clustering variables across all clusters in each clustering method was examined using a one‐way analysis of variance or Kruskal–Wallis as appropriate. Moreover, all the variables were pairwise compared using a post‐hoc Dunn's test with a Bonferroni correction for multiple comparisons. The limit for statistical significance was set at p < 0.05.

2.3. Unsupervised machine‐learning methods

We selected a diverse range of unsupervised machine‐learning methods utilized in the medical research and application of OSA phenotyping, i.e. Agglomerative Hierarchical Clustering (AHC; Johnson, ¹⁹⁶⁷), K‐means (MacQueen, 1967), Gaussian Mixture Model (GMM; Cai et al., ²⁰¹⁹) and Fuzzy C‐means (FCM; Bezdek et al., ¹⁹⁸⁴). These methods encompass both hard and soft clustering methods (Pitafi et al., 2023). The hard clustering algorithms (i.e. AHC and K‐means) assign each data point to a single cluster, allowing straightforward categorization where distinct and non‐overlapping clusters are presented. On the other hand, soft clustering methods (i.e. FCM and GMM) can assign each data point to multiple groups with different membership probabilities (Pitafi et al., 2023).

We employed diverse model selection criteria to obtain the optimal number of clusters. However, the recommended number of clusters differed across the clustering methods. Also, these selection methods do not ensure that the recommended number of clusters implies the natural clusters in the data (Alashwal et al., 2019). We determined the number of clusters to be five for this study based on a consensus from multiple methods, the clinical interpretability of the clusters, and the necessity for a balanced and fair comparison across methods.

2.3.1. Agglomerative Hierarchical Clustering

The AHC starts by creating sub‐clusters that include only one observation each, and continues by combining sub‐clusters from bottom to top until there is only one large cluster. The sub‐clusters are merged through different similarity measures (e.g. based on distance, the nearest neighbour, and the partition's quality evaluation after merging; Govender & Sivakumar, ²⁰²⁰). In this study, the five clusters were made using Ward's method as the similarity measure to merge sub‐clusters, and Euclidean distance was used as the similarity metric. The Lance–Williams updating formula and the corresponding coefficients were employed for updating similarity measures. The stats (version 3.6.2) package in R was used to implement the AHC method (Flynt & Dean, 2016).

2.3.2. K‐means

The K‐means assigns observations to a single cluster with the smallest distance to the centroid of a cluster (Pitafi et al., 2023). In this study, the Euclidean distance was chosen for computing the distance between cluster centroids and observations. Also, to ensure that the algorithm has fully converged, the maximum number of iterations was set to 2000. The initial centroids were randomly chosen. However, the number of starts was set at 25; this means that the algorithm was executed 25 times with different random initial centroids, and the final cluster was returned by the one with the lowest cluster sum of squares. The implementation was done by the stats (version 3.6.2) package in R (Flynt & Dean, 2016).

2.3.3. Fuzzy C‐means

The FCM generalizes K‐means by assigning each observation to all clusters based on different membership probabilities. It iteratively updates the cluster centroids vector and membership matrix until the convergence of the objective function is achieved or the maximum iteration is reached (Askari, 2021). We used Euclidian distance as the distance metric in the objective function. The initial centroid vectors and membership matrix were chosen based on the K‐means++ initialization (Bahmani et al., 2012) and random sampling function (Cebeci, n.d.). Similar to K‐means, the number of starts and the maximum number of iterations were set to 25 and 2000, respectively. To avoid leading to suboptimal clusters, we set the conservative threshold of 1e‐9 in the process of converging the objective function. The fuzzy membership degree was set to 1.1. The “ppclust” (version 1.1.0) package in R was used for the implementation of the FCM (Cebeci et al., 2019).

2.3.4. Gaussian Mixture Model

The GMM is a soft parametric clustering technique that uses multivariate Gaussian distributions to group data points according to their maximum likelihood under the distributions (Torkkola, 2003). In our implementation, the expectation–maximization (EM) algorithm was used to maximize the complete likelihood of the objective function. The performance of different EM models was measured using the Bayesian information criterion (Jung & Wickrama, 2008). The “mclust” (version 6.0.1) package in R was used to implement the GMM method (Jung & Wickrama, 2008).

2.4. Comparison of clustering results

We utilized the t‐distributed stochastic neighbour embedding (t‐SNE) technique to visually compare the clustering methods and understand how clusters are formed. This method effectively visualizes patterns between data by converting high dimensions to lower dimensions, while maintaining local structures and emphasizing dissimilarities (Van der Maaten & Hinton, 2008).

We also evaluated the agreement between clustering methods, and measured their similarity and dissimilarity. We utilized Cohen's kappa statistics to assess the pairwise agreement between clustering methods. For this purpose, we adjusted the same labels for two corresponding clusters derived from two different clustering methods if they exhibited approximately similar physiological characteristics.

Furthermore, we evaluated clustering performance using validation indices. There are two ways to assess the clustering performance: external and internal validation methods. External validation methods focus on comparing the clustering results with true membership labels. In contrast, internal validation methods emphasize internal properties such as the separateness and compactness of the clusters. In this study, due to the lack of access to true membership labels, we only compared clustering methods using internal validation methods. For this purpose, we utilized the Calinski–Harabasz index (CHI), Davies–Bouldin index (DBI), Dunn index (DI), and Silhouette coefficient (SC; Ran et al., ²⁰²²).

3. RESULTS

3.1. Physiological characteristics of individuals

The demographics and PSG characteristics of the study population are presented in Table 2. The heatmap of various clustering variables and the summary of cluster characteristics are presented in Figures 2 and 3, respectively. In addition, the detailed parameter values and statistical analyses, as well as the distribution of OSA severity in each cluster, are reported in Tables S1–S4 and Figure S1, respectively (Supplementary Data S1).

TABLE 2.

Demographic, clinical and PSG characteristics of the applied dataset are reported as medians (interquartile range) or counts (percentage).

Characteristic	All (n = 865)	No OSA (AHI < 5) (n = 135)	Mild OSA (5 ≤ AHI < 15) (n = 245)	Moderate OSA (15 ≤ AHI < 30) (n = 206)	Severe OSA (AHI ≥ 30) (n = 279)
Female (%)	392 (45.3)	89 (65.9)	133 (54.3)	87 (42.2)	83 (29.7)
Age (years)	56.0 (45.0–66.0)	45.0 (32.0–59.0)	55.0 (45.0–64.5)	56.0 (48.0–66.0)	59.0 (48.0–69.0)
BMI (kg m−2)	34.0 (29.0–40.0)	29.4 (25.2–35.2)	33.6 (28.4–38.9)	34.0 (30.4–39.8)	36.5 (31.9–43.4)
TST (min)	309.0 (254.0–360.0)	344.0 (281.0–382.0)	324.0 (265.0–374.0)	309.0 (260.0–356.0)	284.0 (222.0–332.0)
Sleep efficiency (%)	72.0 (60.0–83.0)	79.0 (67.0–87.0)	76.0 (63.0–85.0)	70.0 (60.0–81.0)	66.0 (54.0–76.0)
WASO (min)	96.0 (57.0–142.0)	64.5 (42.5–106.0)	77.5 (46.0–119.0)	102.0 (61.0–137.1)	124.5 (86.5–170.5)
Percentage of stage N1 (% of TST)	11.0 (7.0–19.0)	6.0 (3.0–9.0)	9.0 (5.0–13.0)	11.0 (7.0–17.0)	20.0 (12.0–32.0)
Percentage of stage N2 (% of TST)	48.0 (41.0–56.0)	50.0 (42.0–57.0)	50.0 (43.0–57.0)	49.0 (42.0–57.0)	46.0 (38.0–54.0)
Percentage of stage N3 (% of TST)	18.0 (10.0–27.0)	22.0 (14.0–31.0)	21.0 (13.0–30.0)	19.0 (11.0–28.0)	13.0 (5.0–22.0)
Percentage of stage REM (% of TST)	17.0 (12.0–22.0)	18.0 (14.0–25.0)	19.0 (14.0–23.0)	17.0 (12.0–22.0)	14.0 (10.0–20.0)
AHI (events per hr)	18.0 (8.0–38.0)	2.5 (1.3–3.6)	9.6 (7.3–12.1)	21.2 (17.6–25.5)	51.5 (39.0–73.1)
Oxygen desaturation index (events per hr)	13.0 (3.0–32.0)	1.2 (0.4–2.5)	4.9 (2.1–9.3)	14.2 (9.1–21.2)	44.4 (30.5–65.7)

AHI, apnea–hypopnea index; BMI, body mass index; OSA, obstructive sleep apnea; REM, rapid eye movement; TST, total sleep time; WASO, wake after sleep onset.

FIGURE 2.

Illustration of a subset of clustering variables with heatmap. The absolute values for all clustering variables with all clustering methods are presented in Tables S1–S4. AHI, apnea–hypopnea index; average RR, average of RR intervals; BMI, body mass index; ESS, Epworth Sleepiness Scale; LF/HF ratio, low‐frequency to high‐frequency ratio; PLM, periodic limb movement; T90, percentage of time spent below 90% oxygen saturation.

FIGURE 3.

Typical characteristics of individuals in different clusters identified with the four compared methods. AHI, apnea–hypopnea index; average RR, average of RR intervals; BMI, body mass index; EDS, excessive daytime sleepiness; ESS, Epworth Sleepiness Scale; LF/HF ratio, low‐frequency to high‐frequency ratio; N, number of individuals within clusters; OSA, obstructive sleep apnea; PLM, periodic limb movement; pRR50, percentage of RR intervals differing by more than 50 ms; REM, rapid eye movement; RMSSD, root mean square of successive differences; SDRR, standard deviation of RR intervals, All values are based on Median.

To summarize the findings, the following results were consistent between all clusters.

Cluster 1 was the least populated cluster, comprising 2.19% (n = 19) of the population. Individuals within Cluster 1 were young, non‐obese and did not snore (Tables S1–S4). They had the lowest AHI (i.e. no‐OSA) and no hypoxic load during sleep. They also indicated a higher level of vigilance compared with other clusters. However, this result did not reach statistical significance (p > 0.05). In terms of the duration of respiratory events, these clusters had the shortest apnea duration.

Cluster 2 was the most densely populated group. This cluster included individuals with mild OSA who most often reported excessive daytime sleepiness (EDS; median ESS ≥ 11; Tables S1–S4). The individuals in this cluster had good sleep quality, as evidenced by the greatest sleep efficiency, longest TST and shortest wake after sleep onset (WASO). In addition, the individuals in this cluster generally had low hypoxic load.

Cluster 3 corresponded to individuals who had the highest number of PLM events during the night (Tables S1–S4). Also, individuals in this cluster had a significantly higher number of respiratory events and more severe hypoxic load in comparison with individuals within Cluster 2 (p < 0.05). However, they experienced fewer central apnea compared with Cluster 2.

Cluster 4 comprised individuals with severe OSA, having longer breathing disturbances, higher hypoxic load, and more snoring compared with individuals in Clusters 2 and 3 (Tables S1–S4). Furthermore, although individuals in Cluster 4 had a significantly higher frequency of arousals compared with those in Cluster 3, they had significantly better sleep quality in most clustering methods (i.e. the case of AHC, K‐means and FCM).

Cluster 5 corresponded to individuals with extremely severe OSA (median AHIs ≥ 73.5; Figure S1) and hypoxic load. They were obese, snorers, and had the lowest number of PLM events (Tables S1–S4). They exhibited poor sleep quality, characterized by a high percentage of N1 sleep, and very low percentages of N3 and REM sleep. Individuals within this cluster also experienced the highest frequency of arousal during sleep (p < 0.05). In terms of HRV parameters, individuals in Cluster 5 tended to have the highest values of the LF/HF ratio with the shortest average of RR intervals.

In addition to similar physiological features that were consistently discovered between clustering methods, several physiological characteristics differed across the clustering methods. For instance, the individuals within Cluster 4 discovered by AHC exhibited low sympathetic activity and generally good HRV, while other clustering methods did not find similar characteristics in Cluster 4. In terms of the duration of respiratory events, inconsistencies were found between clustering methods; in most of the cases, Cluster 1 had the shortest, and Cluster 5 had the longest apnea durations, whereas the hypopnea durations differed across clustering methods.

3.2. Agreements between the clustering methods

The highest agreement in the clustering of the individuals into the same clusters was achieved between FCM and K‐means with Cohen's kappa reaching an excellent value of κ = 0.87 (Table 3). Additionally, a good agreement was obtained between GMM and K‐means (κ = 0.70). On the other hand, AHC demonstrated moderate agreement with other clustering methods. The weakest agreement was observed between the GMM and AHC methods with κ = 0.51.

TABLE 3.

Cohen's kappa values for pairwise comparisons of clustering methods.

Clustering methods (number of clusters = 5)	Cohen's kappa values among 865 suspected OSA individuals
	AHC	K‐means	FCM (m = 1.1)	GMM
AHC	1	–	–	–
K‐means	0.56	1	–	–
FCM (m = 1.1)	0.60	0.87	1	–
GMM	0.51	0.70	0.71	1

AHC, Agglomerative Hierarchical Clustering; FCM, Fuzzy c‐means; GMM, Gaussian Mixture Model; OSA, obstructive sleep apnea.

3.3. Internal validation of the clustering methods

In the internal validation of the clustering methods, K‐means outperformed both soft clustering methods (i.e. FCM and GMM) as well as AHC with a SC score of 0.090, a DBI score of 2.386, and a CHI score of 102.037 (Table 4). According to the DI, AHC had the best performance with a score of 0.097. When comparing soft clustering methods, FCM consistently outperformed GMM in all criteria (SC = 0.081 versus 0.060, DBI = 2.510 versus 2.838, CHI = 100.989 versus 92.226, and DI = 0.093 versus 0.071). In summary, although AHC had the best performance based on the DI, K‐means demonstrated the best performance followed by FCM in the other three remaining criteria.

TABLE 4.

Performance evaluation of clustering methods.

Clustering methods (number of clusters = 5)	Cluster validation metrics (index scores) among 865 suspected OSA individuals
	CHI (↑)	DBI (↓)	DI (↑)	SC (↑)
AHC	88.36	2.673	0.097	0.062
K‐means	102.037	2.386	0.092	0.090
FCM (m = 1.1)	100.989	2.510	0.093	0.081
GMM	92.226	2.838	0.071	0.060

Note: Bold indicates the best index values.

(↑) and (↓) indicate whether higher or lower values will result in improved performance, respectively.

AHC, Agglomerative Hierarchical Clustering; CHI, Calinski–Harabasz index; DBI, Davies–Bouldin index; DI, Dunn index; FCM, Fuzzy c‐means; GMM, Gaussian Mixture Model; OSA, obstructive sleep apnea; SC, Silhouette coefficient.

3.4. Visualization of the clusters

To visually represent the cluster formations, we reduced 42 dimensions of input data into two‐dimensional embeddings using t‐SNE and visualized the cluster formations using the output labels of each method (Figure 4). Additionally, in Figure 4(e), we utilized traditional OSA severities as labels to represent the diversity of OSA severity among individuals within each cluster across all methods.

FIGURE 4.

Visualization of cluster formations using clustering labels and conventional OSA severity categorization. AHC, agglomerative hierarchical clustering; AHI, apnea‐hypopnea index; FCM, Fuzzy c‐means; GMM, Gaussian Mixture Model; OSA, obstructive sleep apnea.

As seen in Figure 4, Cluster 1 was completely separated from the other clusters with all clustering methods. In addition, individuals in Cluster 5 were identified highly similarly across all clustering methods. However, the formation of Clusters 2–4 varied among different clustering methods. That is, individuals within these clusters showed inconsistencies in clustering outputs and physiological characteristics (Figures 2 and 3). According to membership probabilities and cluster formations, Cluster 3 was surrounded by Clusters 2 and 4, and had complex mixed boundaries with them. Thus, we summarized the membership probability of individuals within Cluster 3 categorized by FCM and GMM (Table 5). For instance, those individuals who were categorized to Cluster 3 by FCM had probabilities of 0.224 and 0.112 to belong to Clusters 2 and 4, respectively. As another example, individuals with no OSA within Cluster 3 showed a membership probability of 0.334 for Cluster 2 and a membership probability of 0.053 for Cluster 1. GMM tended to produce high posterior probabilities.

TABLE 5.

Example of membership probabilities for individuals belonging to Cluster 3.

Clusters	Soft clustering methods	Median membership probabilities of individuals within cluster 3
		Individuals with AHI ≤ 5	Individuals with 5 ≤ AHI < 15	Individuals with 15 ≤ AHI < 30	Individuals with 30 ≤ AHI	ALL individuals
Cluster 1	FCM	0.005	0.005	0.001	0.002	0.003
	GMM	0.005	0.000	0.000	0.000	0.001
Cluster 2	FCM	0.334	0.267	0.170	0.117	0.224
	GMM	0.060	0.040	0.020	0.000	0.020
Cluster 3	FCM	0.575	0.605	0.580	0.520	0.582
	GMM	0.930	0.950	0.970	0.920	0.940
Cluster 4	FCM	0.045	0.065	0.210	0.310	0.112
	GMM	0.000	0.040	0.005	0.063	0.027
Cluster 5	FCM	0.000	0.000	0.000	0.000	0.002
	GMM	0.000	0.000	0.000	0.000	0.000

AHI, apnea–hypopnea index; FCM, Fuzzy C‐means; GMM, Gaussian Mixture Model.

4. DISCUSSION

In this study, we applied four clustering methods to examine how different clustering methods affect cluster formation and physiological outcomes in a highly obese (BMI ≥ 30 kg m⁻², 72.37%) population of 865 individuals suspected of having OSA. The comparison revealed that clearly separated observations were clustered highly similarly across all methods (Figure 4). However, with overlapping clusters, the agreement between methods decreased. This may suggest that, regardless of the selected set of variables, the agreement between clustering methods is likely to decrease when dealing with overlapping clusters within the data. In such scenarios, the choice of clustering method can affect the cluster formations and physiological characteristics (Figures 2 and 3; Tables S1–S4).

In our study, Cohen's kappa values indicated the weakest agreement between AHC and the other three clustering methods (i.e. the case of K‐means, FCM and GMM; Table 3). AHC is a subgroup of hierarchical clustering algorithms, while other methods utilized in this study, like K‐means, are partitional clustering algorithms (Pitafi et al., 2023). Therefore, this weak agreement is likely due to the different clustering mechanisms and structures. It should be noted that AHC and K‐means have been primarily utilized in the latest applications of OSA phenotyping (Bazoukis et al., 2023). This indicates that clustering outputs can be highly sensitive to the methods derived by different clustering mechanisms. Thus, it would be beneficial to validate the reproducibility of the discovered clusters by using methods with varying mechanisms. On the other hand, K‐means and FCM were in excellent agreement, suggesting that the formed clusters are highly similar (Table 3). This discrepancy in agreements between different clustering methods is something to be aware of when comparing the results between different publications. However, it has to be stated that our study does not provide an answer to which of the methods is the best from the clinical point of view. Further studies are therefore required to link different clusters and clustering methods to OSA‐related symptoms, co‐morbidities and other longitudinal outcomes.

We also evaluated the performances of clustering methods using internal evaluation indices. K‐means demonstrated the best performance in discovering well‐separated and cohesive groups of individuals. FCM also demonstrated satisfactory performance, outperforming GMM in all performance indices and surpassing AHC in most of the indices (Table 4). FCM also showed great potential to handle overlapping clusters by providing membership probabilities for all individuals (Table 5). As FCM exhibited more flexibility compared with other methods, FCM has great potential to be considered in the current application of OSA phenotyping. However, as stated above, this has to be confirmed by linking the FCM‐defined clusters to OSA‐related adverse health consequences.

We incorporated new features into common OSA phenotyping variables, including detailed SpO₂‐based variables, HRV parameters and PVT outcomes. The objective of the utilization of such parameters was to provide more detailed information and better characterization of hypoxic load, autonomic nervous system and individual level of vigilance, allowing us to elucidate new phenotypic expressions of OSA. For instance, the included SpO₂ variables, such as desaturation and recovery severities and duration indexes, demonstrated that individuals in clusters with severe OSA characteristics experienced more frequent and longer desaturations, coupled with shorter and less effective recovery durations.

In this study, we identified a broad spectrum of phenotypic characteristics that were in line with previous literature. Cluster 1 primarily included the mixture of individuals with no OSA (i.e. AHI < 5, n = 15, 79.0%) and a few individuals with a higher frequency of respiratory events (i.e. 5 ≤ AHI < 15, n = 2, 10.5%; and 15 ≤ AHI < 30, n = 2, 10.5%; Figure 4). One common characteristic among these individuals was the absence of hypoxemia, despite experiencing few hypopneas but basically no apneas (Tables S1–S4). In addition, several physiological characteristics of individuals within Cluster 1 are highly similar to those described in “Cluster 1” by Eun‐Yeol et al. (Ma et al., 2021), where the individuals were young and non‐obese, with good sleep quality and few respiratory and desaturation events.

Individuals in Cluster 2 had the best sleep quality, marked by the greatest sleep efficiency, longest TST and shortest WASO (Tables S1–S4). However, these individuals reported the worst daytime sleepiness based on the Epworth Sleepiness Scale (ESS) scores. Monitoring and managing EDS is essential as it increases the risk of motor vehicle and occupational accidents as well as decreases the overall quality of life (Garbarino et al., 2016; Lal et al., ²⁰²¹; Tregear et al., ²⁰⁰⁹). Furthermore, several previous studies related to OSA phenotyping have demonstrated daytime sleepiness to be one of the clustering characteristics (Bailly et al., 2021; Gasa et al., ²⁰²³; Ida et al., ²⁰²²). However, the main characteristics of EDS clusters differ between studies. Our findings in Cluster 2 are the most closely in line with “Cluster 2” described by Gasa et al. (2023), where individuals had mild OSA, good sleep quality, yet reported symptoms of daytime sleepiness.

Cluster 3 comprised individuals with traditional OSA severities ranging from no‐OSA to severe (Figure S1), and the highest number of PLM events (Tables S1–S4). Furthermore, Cluster 3, located between Clusters 2 and 4 with heavily mixed boundaries (Figure 4) showing high uncertainty and a wide range of membership probabilities (Table 5). This indicates that individuals in Cluster 3 can be easily mixed with adjacent clusters. Moreover, it is possible that the night‐to‐night variability, in terms of the AHI, plays an important role in Cluster 3 individuals as the majority of them have mild‐to‐moderate OSA and are more susceptible to such variation compared with those with severe OSA (Stöberl et al., 2017). Therefore, by considering all these aspects, it is likely that the identification and correct classification of Cluster 3 individuals is the most complex and challenging task among the found clusters. Based on these findings, it is possible to further speculate that planning, tailoring and managing effective treatment for these individuals would be a challenge.

Cluster 4 contained individuals with severe OSA. Although this cluster was more extreme than Cluster 3 regarding most of the physiological domains, they had better sleep quality in most cases. The individuals in Cluster 5 also exhibited severe OSA, but on more extreme scales as the PSG parameters were significantly worse compared with Cluster 4 (Tables S1–S4). In our study, the main characteristics of Clusters 4 and 5 are highly similar to “Cluster 3, moderate to severe OSA with hypopnea” and “Cluster 4, severe OSA with hypoxaemia” described by Kim et al. (2020), respectively. In Kim's study, “Cluster 4” was marked by very severe OSA, severe obesity, the highest AHI and the most oxygen desaturation events, while “Cluster 3” included older individuals with more PLM events and longer apnea–hypopnea durations compared with “Cluster 4”.

In this study, the utilization of soft clustering methods enabled the identification of specific subgroups of clusters that presented multiple phenotypic characteristics (Table 5). For instance, we showed that individuals in Cluster 3 had various membership probabilities to multiple clusters, and this information can further assist in clinical decision‐making. For example, there may be a necessity to closely monitor the worsening of symptoms and comorbidities in individuals within Cluster 3 who showed a considerable probability of being members of Cluster 4, where OSA was more severe in general.

Our study is not without limitations. In this study, we only utilized continuous variables derived from PSG and PVT, while other possible phenotypic variables such as sex, ethnicity, dentofacial characteristics, symptoms and comorbidities were not considered (Eckert et al., 2013; Ferreira‐Santos & Rodrigues, ²⁰²³; Subramanian et al., ²⁰¹¹; Ye et al., ²⁰⁰⁹). Including these variables in PSG‐based phenotyping could further help to understand the pathophysiology of OSA, and lead to the identification of optimal treatments and improvements in prognostics (Zinchuk & Yaggi, 2020). However, this selection was done with a purpose: being able to identify OSA phenotypes based on continuous parameters such as age, BMI, ESS, PSG metrics and PVT measurements could help in clinical decision‐making without having information on OSA‐related comorbidities. That is, if certain PSG phenotypes (i.e. OSA clusters) are related to high risks of, for example, cardiovascular consequences, this information can be used for planning further clinical considerations or treatments. We also set the number of clusters to five for this study to have clinically interpretable clusters and to fairly compare the outputs of different clustering methods. Considering a fixed number of clusters for each method might present a limitation in this study as different clustering algorithms operate using unique mechanisms and might include varying numbers of clusters based on different model selection criteria.

In conclusion, we compared the physiological outcomes of five clusters across assorted unsupervised machine‐learning methods where Cluster 1 consisted of individuals with no OSA, Cluster 2 consisted of sleepy individuals with mild OSA and good sleep quality, Cluster 3 consisted of a mix of different OSA severities and a high number of PLM events, Cluster 4 consisted of individuals with traditional severe OSA, and Cluster 5 consisted of individuals with extremely severe OSA and poor sleep quality. However, some characteristics like the duration of respiratory events and HRV parameters differed across the clustering methods. The results of this study indeed highlight the effect of clustering methods on cluster formations and physiological outcomes. Also, our findings showed that FCM demonstrated satisfactory performance in terms of identifying distinct OSA clusters and handling complex and mixed structures within the data. By utilizing fuzzy clustering methods, we can identify subgroups of individuals within clusters who might indicate multiple phenotypic characteristics. This can significantly reduce the discrepancies between clustering methods, and enhance the robustness and reliability of the clustering findings.

AUTHOR CONTRIBUTIONS

Mohammadreza Ghorvei: Conceptualization; investigation; funding acquisition; writing – original draft; writing – review and editing; visualization; validation; software; formal analysis; resources; data curation; methodology. Tuomas Karhu: Writing – review and editing; methodology; visualization; software; data curation; resources; investigation. Salla Hietakoste: Resources; data curation; software; formal analysis; methodology; writing – review and editing; investigation. Daniela Ferreira‐Santos: Writing – review and editing; project administration; supervision. Harald Hrubos‐Strøm: Writing – review and editing. Anna Sigridur Islind: Writing – review and editing. Luka Biedebach: Writing – review and editing. Sami Nikkonen: Conceptualization; investigation; funding acquisition; writing – review and editing; validation; visualization; methodology; software; formal analysis; project administration; resources; supervision; data curation. Timo Leppänen: Data curation; supervision; resources; project administration; formal analysis; software; methodology; validation; visualization; writing – review and editing; funding acquisition; investigation; conceptualization. Matias Rusanen: Conceptualization; investigation; funding acquisition; writing – review and editing; visualization; validation; methodology; project administration; formal analysis; software; data curation; supervision; resources.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflict of interest relevant to this study.

Supporting information

DATA S1. Supporting Information.

Ghorvei, M. , Karhu, T. , Hietakoste, S. , Ferreira‐Santos, D. , Hrubos‐Strøm, H. , Islind, A. S. , Biedebach, L. , Nikkonen, S. , Leppänen, T. , & Rusanen, M. (2025). A comparative analysis of unsupervised machine‐learning methods in PSG‐related phenotyping. Journal of Sleep Research, 34(3), e14349. 10.1111/jsr.14349

DATA AVAILABILITY STATEMENT

Data sharing is not applicable to this article as no new data were created or analyzed in this study.