Profiling the sleep architecture of ageing adults using a seven‐state continuous‐time Markov model

Summary

Sleep is a complex biological process regulated by networks of neurons and environmental factors. As one falls asleep, neurotransmitters from sleep–wake regulating neurones work in synergy to control the switching of different sleep states throughout the night. As sleep disorders or underlying neuropathology can manifest as irregular switching, analysing these patterns is crucial in sleep medicine and neuroscience. While hypnograms represent the switching of sleep states well, current analyses of hypnograms often rely on oversimplified temporal descriptive statistics (TDS, e.g., total time spent in a sleep state), which miss the opportunity to study the sleep state switching by overlooking the complex structures of hypnograms. In this paper, we propose analysing sleep hypnograms using a seven‐state continuous‐time Markov model (CTMM). This proposed model leverages the CTMM to depict the time‐varying sleep‐state transitions, and probes three types of insomnia by distinguishing three types of wake states. Fitting the proposed model to data from 2056 ageing adults in the Multi‐Ethnic Study of Atherosclerosis (MESA) Sleep study, we profiled sleep architectures in this population and identified the various associations between the sleep state transitions and demographic factors and subjective sleep questions. Ageing, sex, and race all show distinctive patterns of sleep state transitions. Furthermore, we also found that the perception of insomnia and restless sleep are significantly associated with critical transitions in the sleep architecture. By incorporating three wake states in a continuous‐time Markov model, our proposed method reveals interesting insights into the relationships between objective hypnogram data and subjective sleep quality assessments.

1. INTRODUCTION

Sleep is a crucial part of human health, responsible for memory consolidation and the repair of daily stresses on major body systems (Eugene & Masiak, 2015). Given that about a third of human experience is spent asleep, understanding what factors influence sleep quality is useful for maintaining long‐term health (Wells & Vaughn, 2012). Notably, poor sleep quality can manifest in sleep disorders such as insomnia, narcolepsy (Maski et al., 2022), idiopathic hypersomnia (Billiard & Sonka, 2016), and sleep apnea (Cartwright, 2001), which in turn impact health outcomes such as cardiovascular health and cognition (Mirjat et al., 2020).

The ‘gold standard’ of sleep quality measurement is polysomnography (PSG). The PSG method tracks brain activity, physiological activity, and physical activity via a set of real‐time sensors (e.g., electroencephalogram [EEG], electrocardiogram [ECG], electro‐oculogram [EOG], electromyogram [EMG], and photoplethysmography [PPG]) during one's sleep, and integrates them into discrete sleep stages at 30‐s or 1‐min intervals, resulting in a hypnogram. Hypnograms have been most often analysed using temporal descriptive statistics (TDS). These metrics are duration‐related information and their derivatives for different sleep stages, such as sleep onset latency (SOL), total time spent in wake after sleep onset (WASO), total sleep time (TST), total time in bed (TIB), and the ratio between the TST and TIB (i.e., sleep efficiency [SE]). As descriptive statistics, these metrics are easy to compute and interpret, but they oversimplify sleep architecture and can be subjected to outliers and noise in the data. Furthermore, these simple metrics fail to capture the sleep state transitions, which are regulated by complex neuronal circuitry and have significant biological links to neurotransmitters (Saper et al., 2010). As one falls asleep, neurotransmitters from sleep–wake regulating neurones work in synergy to control the switching of different sleep states throughout the night. Ignoring these transitions in a sleep study may overlook important insights into how sleep affects health.

Some research groups applied survival models (Swihart et al., 2008; Swihart et al., 2012; Yassouridis et al., 1999) and discrete‐time Markov models (Kemp & Kamphuisen, 1986; Kneib & Hennerfeind, 2008; Yang & Hursch, 1973), to analyse sleep hypnograms. However, these models did not gain popularity in the sleep hypnogram analysis, due to computation burden and difficulty in parameter interpretation (Swihart et al., 2015). Moreover, these models have only considered a three‐state (Swihart et al., 2008) (Kneib & Hennerfeind, 2008) or a five‐state (Swihart et al., 2015) sleep architecture, in which ‘wake’ is considered as one state. In reality, wake states occur in three distinctive times during sleep: wake before sleep onset (WBSO), short periods of WASO, and final wake after sleep (WASF). Furthermore, transitions of these three states are linked to three different types of insomnia (Hohagen et al., 1994). Taking a long time to transition to WBSO is linked to sleep onset insomnia, transitioning too early to WASF is linked to insomnia with early‐morning awakening, and spending too much time in WASO is linked to sleep maintenance insomnia, or sleep instability. Although the recently‐revised diagnostic (American Academy of Sleep Medicine, 2023) or treatment criteria (Riemann et al., 2023) no longer distinguish the subtypes of insomnia, these differentiations are still assessed by self‐report to identify insomnia (American Academy of Sleep Medicine, 2023). Moreover, for researchers interested in understanding how a particular form of wakefulness is linked to the subjective perception of insomnia, combining them into a single wake state will miss the opportunity (Bjorøy et al., 2020).

Given the issues in the existing hypnogram analyses, in this paper, we propose a seven‐state, continuous‐time Markov model (CTMM) to profile sleep patterns of ageing adults in the Multi‐Ethnic Study of Atherosclerosis (MESA) Sleep study. By leveraging the CTMM to depict the time‐varying sleep‐state switching and distinguishing three types of wake states to probe different types of insomnia (Figure 1), we profile the relationships between transition intensities of sleep states and demographic factors as well as subjective sleep measures of different aspects of sleep.

FIGURE 1.

A seven‐state hypnogram of one subject in the Multi‐Ethnic Study of Atherosclerosis (MESA) Sleep study. The x‐axis represents the time elapsed since the lights‐out event, and the y‐axis represents the state of the hypnogram. In this framework, wake before sleep onset (WBSO) and wake after sleep onset (WASO) are both sleep states. The duration of WBSO is the sleep onset latency (SOL), and the total time spent in all WASO states is the commonly referred WASO. WASF is the final wake state, marking the end of the sleep process. Our parameters of interest are the population‐level quantities of transitions throughout the sleep process.

2. METHODS

2.1. Data collection and study sample

A total of 2237 people aged 45–84 years were enrolled in the MESA Sleep study between 2010 and 2012. Institutional Review Board approval was obtained at each participating centre and written informed consent was obtained from all participants (Chen et al., 2015). During the study, participants completed an unattended overnight PSG at home, for which raw data exist for 2056 people. A detailed description of the in‐home PSG setup can be found in the original publication of MESA Sleep study (Chen et al., 2015) and the documentation in the data repository on National Sleep Research Resource's website with registered access: https://sleepdata.org/datasets/mesa. Briefly, the PSG system was Compumedics Somte System (Compumedics Ltd., Abbotsford, Australia), consisting of 15‐channels. These channels include EEG (three leads, 256 Hz), bilateral EOG (two leads, 256 Hz), a chin EMG (one lead, 256 Hz), bipolar ECG (256 Hz), thoracic and abdominal respiratory inductance, PPG (256 Hz), airflow measured by thermocouple and nasal pressure cannula, finger pulse oximetry, and bilateral limb movements.

In addition to the sensor setup, trained staff members completed signal calibrations and checked impedance. Nocturnal recordings were transmitted to the centralised reading centre at Brigham and Women's Hospital and data were scored by trained technicians using current guidelines. These PSG data were manually examined and scored by sleep experts at the Sleep Reading Center at the Brigham and Women's Hospital, using the American Association of Sleep Medicine (AASM) criteria (Moser et al., 2009). In addition, they manually rated the quality of the data by scoring the percentage of usable signal for each channel between lights‐out time and lights‐on time on a Likert scale from 1 to 5. Then, they scored the data on a scale from 3 to 7, where a 3 indicated fair data collection and a 7 indicated outstanding data collection. The MESA study team expected that at least 85% of the PSG data would rate at a 5 or above and retraining or supervision was implemented for staff who did not maintain at least an average rating of a 5 over a month on their obtained PSG data. Of the 2056 people who successfully completed a PSG, 1646 (80.01%) of them had a quality rating of at least a 5.

2.2. Sleep questionnaires

Before completing the overnight PSG, the participants answered various sleep questionnaires about their sleep quality, including the Women's Health Initiative Insomnia Rating Scale (WHIIRS) (Shahid et al., 2012), the Epworth Sleepiness Scale (ESS) (Johns, 1991), and Morning–Eveningness scale (Horne & Ostberg, 1976). We were interested in the WHIIRS questions that can be directly linked to objective sleep measures, such as ‘In the last 4 weeks, how often did you (1) have trouble falling asleep? (2) wake several times during the night? (3) wake earlier than planned? (4) have trouble getting back to sleep after waking too early?’ The response to these four questions was on a five‐point Likert scale (1 = ‘No, not in the past 4 weeks’, 2 = ‘Yes, less than once a week’, 3 = ‘Yes, 1 or 2 times a week’, 4 = ‘Yes, 3 or 4 times a week’, 5 = ‘Yes, 5 or more times a week’). The last question (5) ‘In the last 4 weeks, how often did you have a typical night's sleep?’ is also on a Likert scale (1 = ‘Very sound or restful’, 2 = ‘Sound and restful’, 3 = ‘Average quality’, 4 = ‘Restless’, 5 = ‘Very restless’). Based on the scale of this question, we considered this question as a measure of ‘how restless was your sleep?’.

2.3. Temporal descriptive statistics (TDS)

In addition to the sleep questionnaire answers, the MESA Sleep Dataset includes several TDS sleep metrics extracted from sleep hypnogram data, such as SOL—duration between lights off (an indicator of intent to sleep) and falling asleep, total time spent in the WASO state (from here on, we refer to this metric TDS WASO to distinguish it from the WASO state), TST—duration between falling asleep and waking up minus the total time of WASO, total TIB—duration between time going to bed and time getting out of bed, and SE—the ratio between TST and TIB. These metrics were computed based on the PSG data. We calculated the Spearman correlations between these TDS and the five sleep questionnaire variables in WHIIRS.

2.4. Modelling strategy

2.4.1. Pre‐processing

A total of 2056 participants successfully completed their PSG study. Among them, 88 participants had unknown or ambiguous sleep states, resulting in a total 1968 participants who entered the analysis. Every PSG used in the study recorded the time that the lights were turned off in the room the participant slept in, as well as whenever the participants were in or out of bed. All hypnogram data were aligned to this ‘lights‐out’ time. After aligning each hypnogram, the WBSO and WASF were separated from the rest of the hypnogram by examining when the first and last WASO times were recorded in the data. Due to data loss, some patients’ end sleep state was not WASF. To ensure the homogeneity of hypnogram data, we excluded 247 patients who did not reach the WASF state in the hypnogram data. This left 1721 hypnograms for our analysis. In all, 1300 of the 1721 hypnograms (80.12%) had a quality score of ≥5 as defined in Section 2.1.

2.4.2. A seven‐state Markov model

We consider a seven‐state, CTMM to analyse the sleep architecture, with states WBSO, WASO, N1, N2, N3, rapid‐eye movement (REM), and WASF. This type of model captures a dynamic process where the participants transit from one state to another in time. The primary set of parameters is the transition intensities at a given time t, denoted as q _rs(t), which is the instantaneous probability of transitioning from state r to s. For a time‐homogeneous Markov model, q _rs is defined as:

$$q_{\mathrm{rs}}\left(t\right)=\underset{\mathit{\delta t}\to 0}{\lim }\frac{P\left(Y_{t+\mathit{\delta t}}=s|Y_t=r\right)}{\mathit{\delta t}},$$ (1)

where Y _t is the sleep stage at each time t. Gathering the transition intensities for all pairs of states forms a generator matrix Q, with the constraint that the diagonal elements $q_{\mathrm{rr}}=-{\sum }_{r\ne s}{q}_{\mathrm{rs}}$. The transition probability matrix P(t) can thus be derived using the Kolmogorov differential equations given that the initial condition, P(0), is an identity matrix (Cox and Miller, 2017):

$$P\left(t\right)=\exp \left(\mathit{Qt}\right),$$ (2)

which can be expanded using power series for matrix exponentiation:

$$P\left(t\right)=\sum _{k=0}^{\infty }\frac{{\left(\mathit{Qt}\right)}^k}{k!},$$ (3)

For our seven‐state sleep model, we define Q as:

$$\left[\begin{array}{ccccccc}0& 0& q_{\text{WBSO}\to \mathrm{N}1}& 0& 0& 0& 0\\ 0& 0& q_{\text{WASO}\to \mathrm{N}1}& 0& 0& q_{\text{WASO}\to \mathrm{REM}}& 0\\ 0& q_{\mathrm{N}1\to \text{WASO}}& 0& q_{\mathrm{N}1\to \mathrm{N}2}& 0& q_{\mathrm{N}1\to \mathrm{REM}}& q_{\mathrm{N}1\to \text{WASF}}\\ 0& q_{\mathrm{N}2\to \text{WASO}}& q_{\mathrm{N}2\to \mathrm{N}1}& 0& q_{\mathrm{N}2\to \mathrm{N}3}& q_{\mathrm{N}2\to \mathrm{REM}}& q_{\mathrm{N}2\to \text{WASF}}\\ 0& q_{\mathrm{N}3\to \text{WASO}}& 0& q_{\mathrm{N}3\to \mathrm{N}2}& 0& 0& 0\\ 0& q_{\mathrm{REM}\to \text{WASO}}& q_{\mathrm{REM}\to \mathrm{N}1}& 0& 0& 0& q_{\mathrm{REM}\to \text{WASF}}\\ 0& 0& 0& 0& 0& 0& 0\end{array}\right]$$

Here, intensity parameters that do not need to be estimated are labelled as 0, either because they are constrained by other parameters (e.g., the diagonal elements), or because instantaneous transitions between those two states rarely or never occur. For example, because WASF is an absorbing state (a person in WASF will not go back to sleep), the last row of Q contains only 0's. Similarly, once someone starts sleeping, they will never return to WBSO, so the first column contains only 0s. Other transitions occurred in only a small percentage of subjects (e.g., WASO→N3, N3 → WASF). While removing these rare transitions will not affect the parameter estimation of the more prevalent transitions, including them would bias the estimation of the more common population‐level intensities. Therefore, we purposefully chose not to include these rare transitions in the model. In total, we included 17 intensity parameters to represent the Markov chain, resulting in a sleep architecture as shown in Figure 2. These parameters can be estimated using the maximum likelihood function:

$$L\left(\theta |Y\right)=P\left[Y_T=y_T,\dots ,Y_t=y_t,\dots ,Y_2=y_2,Y_1=y_1\right]$$ (4)

FIGURE 2.

The sleep architecture defined by our choice of Q matrix. REM, rapid eye movement; WASF, final wake after sleep; WASO, wake after sleep onset; WBSO, wake before sleep onset.

Given the Markovian property, Equation (4) can be reduced to:

$$L\left(\theta |Y\right)=\prod _{t=2}^{T}P\left[Y_t=y_t|Y_{t-1}=y_{t-1}\right],$$ (5)

where vector θ captures the parameters of interest, the transition intensities from the Q matrix. $P\left[Y_t=y_t|Y_{t-1}=y_{t-1}\right]$ represent the transition probabilities from time t – 1 to time t. The analytical form of these transition probabilities can be retrieved from the transition probability matrix P, represented by elements in the Q matrix, given Equation (3).

Intuitively, the transition intensity, or the instantaneous probability q _rs captures the speed of a transition event r→s occurring, in other words, the rate at which transition probabilities change over time. This quantity is determined by two factors during estimation: the time stayed in r state before transitioning to s, and the number of transitioning events at a time point. As the time derivative of transition probability, transition intensities are to transition probabilities as speed is to distance. Just as instantaneous speeds may vary over time but result in the same distance, transition intensities can also vary over time but result in the same transition probability.

2.4.3. Time‐inhomogeneity and model assessment

It is known that the transitions between sleep states can be non‐homogeneous and time‐dependent (Luce, 1965; Yang & Hursch, 1973). This means the transition intensities may not be constant throughout the night. To address the time‐inhomogeneity issue in modelling, we added a time‐varying covariate—phase—to all models. This phase variable indexes sequentially 30‐min periods since the time of lights out. For model‐fitting diagnostics, we plotted the observed prevalence and expected prevalence for each state from the model containing only the phase covariate to check for relative agreement. In addition to examining prevalence curves, we checked for relative agreement between estimated time‐varying intensities curves and empirically estimated intensity curves (Titman & Sharples, 2008). These model diagnostic plots and the qualitative model evaluations are presented in Appendix A.

2.4.4. Covariate associations

We assessed the effects of age, sex, self‐reported race/ethnicity, job schedule, and the five sleep questionnaire answers outlined in Section 2.2 on the sleep hypnogram. For job schedule, we re‐organised the categories into three levels: those who did not work, those who worked day shift, and those who worked all other shifts. The effect of each covariate will be assessed by the transition intensity ratios (TRs). A TR _r→s >1 indicates the covariate is associated with an increased instantaneous probability of transitioning into a state s from state r at a time instant and similarly, a TR _r→s <1 indicates a reduced instantaneous probability. In the speed‐distance analogy, the interpretation of transition ratios is similar to ‘person A is traveling twice as fast as person B’. TR_r→s >1 indicates a shorter stay at state r before transitioning to state s and/or increased transition events. Increased TR _r↔s (in both transition directions) indicate shorter stays at both state r and state s and/or increased transition events between these two states. All models were adjusted for the phase covariate. All models were implemented using the ‘msm’ package (Version 1.7.1) (Jackson, 2011) in R (R Version 4.4.0).

3. RESULTS

Table 1 displays the summary statistics for covariates of interest, grouped by job schedule. In all, 14 participants were not included in Table 1 due to a missing schedule variable. The majority of the participants were retired, White, and female. In terms of sleep questions, most participants reported not having trouble falling asleep and not waking up earlier than planned in the past 4 weeks. However, most participants also reported waking up during the night at least five times per week and having trouble going back to sleep at least five times per week. Most participants also reported having ‘average’ typical night of sleep, and more participants reported having restful than restless sleep. While age is much higher in the retired group, sex, race/ethnicity, and answers to the sleep questions were balanced among the three job schedules.

TABLE 1.

Summary statistics by job schedule (N = 14 missing job schedule).

Variable	Job schedule			Overall (N = 1707)
	Retired/not working (N = 949)	Day shift (N = 532)	Other shifts (N = 226)
Age, years
Mean (standard deviation)	72.9 (8.56)	64.2 (7.16)	65.2 (7.37)	69.2 (9.02)
Median (minimum, maximum)	73.0 (54.0, 94.0)	62.0 (55.0, 89.0)	64.0 (54.0, 89.0)	68.0 (54.0, 94.0)
Sex, n (%)
Female	521 (54.9)	274 (51.5)	100 (44.2)	895 (52.4)
Male	428 (45.1)	258 (48.5)	126 (55.8)	812 (47.6)
Race/ethnicity, n (%)
White	353 (37.2)	211 (39.7)	94 (41.6)	658 (38.5)
Asian	89 (9.4)	68 (12.8)	24 (10.6)	181 (10.6)
Black	278 (29.3)	140 (26.3)	66 (29.2)	484 (28.4)
Hispanic	229 (24.1)	113 (21.2)	42 (18.6)	384 (22.5)
Question 1: Having trouble falling asleep, n (%)
Not in 4 weeks = 0	504 (53.1)	323 (60.7)	125 (55.3)	952 (55.8)
<1 per week = 1	97 (10.2)	49 (9.2)	16 (7.1)	162 (9.5)
1–2 per week = 2	161 (17.0)	85 (16.0)	52 (23.0)	298 (17.5)
3–4 per week = 3	100 (10.5)	36 (6.8)	19 (8.4)	155 (9.1)
≥5 per week = 4	87 (9.2)	39 (7.3)	14 (6.2)	140 (8.2)
Question 2: Wake up several times during the night, n (%)
Not in 4 weeks = 0	158 (16.6)	116 (21.8)	41 (18.1)	315 (18.5)
<1 per week = 1	42 (4.4)	38 (7.1)	14 (6.2)	94 (5.5)
1–2 per week = 2	177 (18.7)	100 (18.8)	45 (19.9)	322 (18.9)
3–4 per week = 3	155 (16.3)	86 (16.2)	36 (15.9)	277 (16.2)
≥5 per week = 4	417 (43.9)	192 (36.1)	90 (39.8)	699 (40.9)
Question 3: Wake up earlier than planned, n (%)
Not in 4 weeks = 0	476 (50.2)	230 (43.2)	98 (43.4)	804 (47.1)
<1 per week = 1	74 (7.8)	56 (10.5)	22 (9.7)	152 (8.9)
1–2 per week = 2	168 (17.7)	109 (20.5)	49 (21.7)	326 (19.1)
3–4 per week = 3	87 (9.2)	60 (11.3)	28 (12.4)	175 (10.3)
≥5 per week = 4	139 (14.6)	75 (14.1)	28 (12.4)	242 (14.2)
Missing	5 (0.5)	2 (0.4)	1 (0.4)	8 (0.5)
Question 4: Having trouble going back to sleep, n (%)
Not in 4 weeks = 0	213 (22.4)	136 (25.6)	46 (20.4)	395 (23.1)
<1 per week = 1	44 (4.6)	37 (7.0)	19 (8.4)	100 (5.9)
1–2 per week = 2	98 (10.3)	82 (15.4)	32 (14.2)	212 (12.4)
3–4 per week = 3	71 (7.5)	32 (6.0)	14 (6.2)	117 (6.9)
≥5 per week = 4	63 (6.6)	21 (3.9)	17 (7.5)	101 (5.9)
Missing	460 (48.5)	224 (42.1)	98 (43.4)	782 (45.8)
Question 5: Having a typical night's sleep, n (%)
Very restful = 0	89 (9.4)	51 (9.6)	15 (6.6)	155 (9.1)
Restful = 1	300 (31.6)	162 (30.5)	70 (31.0)	532 (31.2)
Average = 2	399 (42.0)	232 (43.6)	105 (46.5)	736 (43.1)
Restless = 3	143 (15.1)	71 (13.3)	30 (13.3)	244 (14.3)
Very restless = 4	18 (1.9)	15 (2.8)	6 (2.7)	39 (2.3)
Missing	0 (0)	1 (0.2)	0 (0)	1 (0.1)

Figure 3 displays the correlation matrix of various TDS metrics obtained from the MESA study and their correlations with WHIIRS items. As expected, the total score of the WHIIRS questions is highly correlated with each of the five items. In addition, the individual items showed significant correlations, suggesting good internal consistency of the WHIIRS questionnaire (Levine et al., 2005; Shahid et al., 2012). Among the TDS metrics, we observed negative correlations between SOL and sleep duration, time spent in WASO and sleep duration, and time spent in WASO and average SE. This is probably because longer time spent in bed or in WBSO and WASO reduces SE. However, only a few TDS sleep metrics were associated with the insomnia questions: WASO is positively correlated with the wake‐up frequency question, and SE is negatively correlated with the wake‐up frequency question and the typical night of sleep question. In other words, participants who perceived they were waking up more frequently at night and having more restless sleep tended to spend longer in bed, have longer WASO duration, and have lower SE. To our surprise, SOL, was not associated with the ‘Having trouble falling asleep’ question.

FIGURE 3.

Correlation matrix of sleep measures. Spearman correlations were used here, and only significant correlations were displayed (p < 0.05). The first six items are sleep questionnaire variables and the total sum score of them. The last five items are the temporal descriptive statistics (TDS) sleep metrics. SE, sleep efficiency; SOL, sleep onset latency; TIB, time in bed; TST, total sleep time; WASO, total time spent in the wake after sleep onset state.

Figures 4, 5, 6 show the transition intensity ratios (TRs) of demographic covariates and sleep questionnaire answers. The exact TRs and their 95% confidence intervals (CIs) are presented in Tables S1 and S2 in Appendix B. Our multistate model confirms that ageing has a negative impact on sleep quality. First, age is associated with faster transitions into WASO (e.g., TR_N1→WASO = 1.15 [95% CI 1.13, 1.17], and TR_N2→WASO = 1.08 [95% CI 1.06, 1.09]) and slower transitions out of WASO (e.g., TR_WASO→N1 = 0.89 [95% CI 0.88, 0.89] and TR_WASO→REM = 0.75 [95% CI 0.73, 0.78]). On average, for every decade increase in age after 45 years, transitions out of WASO decrease by ≥10%. This indicates that older people tend to have more episodes of WASO and spend more time in WASO. Second, transitions into WASF drastically slow down as people age: for every decade of increase in age, people transition out of sleep ≥30% slower (e.g., TR_N1→WASF = 0.7 [95% CI 0.62, 0.78], TR_N2→WASF = 0.57 [95% CI 0.51, 0.64], and TR_REM→WASF = 0.58 [95% CI 0.49, 0.68]), indicating it is slower to finally wake up for older participants than younger participants.

FIGURE 4.

Forest plot of transition intensity ratios for demographic variables. Age was modelled as a continuous variable, centred at the minimum of the cohort (45 years) with 10‐year increment. The reference level for the job schedule is retired/not working, and the reference group for sex is female. Blue indicates a transition from a lighter to a deeper stage. Red indicates a transition from a deeper to a lighter stage. Bars indicate 95% confidence intervals (CIs), and transition pairs with CI bars that do not cross 1.0 means the covariate has a significantly different transition intensity ratio when compared with the reference. REM, rapid eye movement; WASF, final wake after sleep; WASO, wake after sleep onset; WBSO, wake before sleep onset.

FIGURE 5.

Forest plot of transition intensity ratios for race/ethnicity (R/E). The reference level is White. Blue indicates a transition from a lighter to a deeper sleep stage. Red indicates a transition from a deeper to a lighter sleep stage. Bars indicate 95% confidence intervals (CIs), and transition pairs with CI bars that do not cross 1.0 indicate that the covariate has a significantly different transition intensity ratio when compared with the reference. REM, rapid eye movement; WASF, final wake after sleep; WASO, wake after sleep onset; WBSO, wake before sleep onset.

FIGURE 6.

Forest plot of transition intensity ratios for sleep questions. For all questions, an increasing value indicates a poorer outcome, with the best sleep quality levels as the reference. Blue indicates a transition from a lighter to a deeper sleep stage. Red indicates a transition from a deeper to a lighter sleep stage. Bars indicate 95% confidence intervals (CIs), and transition pairs with CI bars that do not cross 1.0 indicate that the covariate has a significantly different transition intensity ratio when compared with the reference. REM, rapid eye movement; WASF, final wake after sleep; WASO, wake after sleep onset; WBSO, wake before sleep onset.

Notably, transition intensities differed between men and women. Men fell asleep much faster than women (TR_WBSO→N1 = 1.22 [95% CI 1.11, 1.35]). Men transitioned faster out of light sleep and REM stages into WASO (e.g., TR_N1→WASO = 1.55 [95% CI 1.50, 1.59], TR_N2→WASO = 1.26 [95% CI 1.22, 1.30], and TR_REM→WASO = 1.92 [95% CI 1.85, 1.99]), but slower transitions from the deep sleep stage to WASO (TR_N3→WASO = 0.47 [95% CI 0.37, 0.59]). Another sex difference is that men transitioned faster into and out of REM than women, indicating that at any given time when men were in REM sleep, they spent less time in REM compared to women that were in REM sleep. Lastly, men also had faster transitions into the WASF state (TR_REM→WASF = 1.58 [95% CI 1.26, 1.99]), indicating it is quicker for men to finally wake up than women.

Transition intensities also differed significantly by race/ethnicity. Non‐White participants transitioned faster from N1 to WASO. For example, Asian participants transitioned 27% faster (TR_N1→WASO = 1.27 [95% CI 1.21, 1.34]) and Black participants transitioned 12% faster (TR_N1→WASO = 1.26 [95% CI 1.22, 1.31]). Non‐White participants transitioned even faster from N1 into REM than White participants, by as much as 85% among Asian participants (TR_N1→REM = 1.85 [95% CI 1.68, 2.03]), 37% among Black participants (TR_N1→REM = 1.37 [95% CI 1.27, 1.48]), and 33% among Hispanic participants (TR_N1→REM = 1.33 [95% CI 1.23, 1.44]), suggesting that they spent more time in REM states. Conversely, non‐White participants all transitioned slower out of N2 (by as much as 26% for Black participants compared to White participants), suggesting that they spent more time in N2 than White participants.

Compared with retired participants, those who had a day job fell asleep faster (TR_WBSO→N1 = 1.23 [95% CI 1.11, 1.37]), transitioned faster out of WASO (e.g., TR_WASO→N1 = 1.22 [95% CI 1.20, 1.25], TR_WASO→REM = 1.45 [95% CI 1.34, 1.56]), and transitioned slower into WASO (e.g., TR_N1→WASO = 0.81 [95% CI 0.78, 0.84]). Compared with retired participants, those who had shift jobs also had faster transitions out of WASO and slower transitions into WASO, but with smaller effect sizes than those with day job (TR_WASO→N1 = 1.12 [95% CI 1.09, 1.15] and TR_N1→WASO = 0.92 [95% CI 0.89, 0.96]). These findings indicate that participants who were not retired tended to have better sleep quality than those who were retired.

Models with sleep questionnaire items as covariates revealed significant associations in certain transitions that were not apparent by using the TDS metrics. For example, the ‘having trouble falling asleep’ question is significantly associated with a slower transition into the first sleep event (TR_WBSO→N1 = 0.94 [95% CI 0.88, 1.00]). Unlike the null finding in the correlation between SOL and this question, our multistate model shows that objective sleep measures indeed matched the participants’ perception of having trouble falling asleep.

For the question about ‘waking up often’, we expected transitions related to WASO to be associated with question answers. However, the only significant transitions were from N1 to WASO and WASO to N1, with small effect sizes (TR_N1→WASO = 1.02 [95% CI 1.00, 1.04] and TR_WASO→N1 = 0.99 [95% CI 0.98, 1.00]). For the question about ‘waking up earlier than planned’, we expected to observe that transitions related to WASF would be associated with question answers. However, none of the transitions into WASF were significant, although all TRs were >1 (TR_N1→WASF = 1.05, TR_N2→WASF = 1.08, and TR_REM→WASF = 1.02). On the other hand, participants who answered higher on the scale for the question ‘waking up earlier than planned’ showed associations with WASO transitions, although these also had small effects (e.g., TR_N1→WASO = 1.03 [95% CI 1.02, 1.05], TR_N2→WASO = 1.01 [95% CI 1.00, 1.03]).

Participants who answered higher on the scale of the ‘typical night of sleep’ question (i.e., their sleep is more restless than restful), had reduced transition intensity out of REM state (TR_REM→WASO = 0.94 [95% CI 0.91, 0.97], and TR_REM→N1 = 0.91 [95% CI 0.85, 0.97]). This suggests that participants who have less restful sleep transition out of REM sleep slower than those who believe they have more restful sleep. They also had increased transition intensities between N2 and N3 sleep (TR_N2→N3 = 1.06 [95% CI 1.04, 1.09], TR_N3→N2 = 1.03 [95% CI 1.01, 1.05]). This further suggests that participants who had less restful sleep had more fragmented deep sleep. The ‘trouble going back to sleep’ question, which asks about the frequency of having trouble getting back to sleep after waking too early, is naturally conditioned on the ‘waking up earlier than planned’ item. These two items had a Spearman correlation of 0.31 (Figure 3). This question is associated with increased transitions between WASO and N1, and increased transitions between N1 and N2 (TR_WASO→N1 = 1.06 [95% CI 1.05, 1.08] and TR_N1→N2 = 1.04 [95% CI 1.02, 1.05]), indicating more fragmented non‐REM sleep. In contrast to the ‘wake up earlier than planned’ question, this question is associated with slower transition from N1 to WASF (TR_N1→WASF = 0.85 [95% CI 0.73, 0.99]), which suggests participants who answered higher for this question spent more time in N1 before waking up. This contrast to the ‘wake up earlier than planned’ question suggests that even though waking up earlier than planned is a necessary condition for the ‘trouble going back to sleep’ question, how the participants perceive the final wake‐up state causes different answers to the questions.

4. DISCUSSION

Our analyses revealed distinct demographic patterns in sleep hypnograms and significant associations between subjective sleep perceptions and hypnogram data. Our analyses confirmed that ageing negatively impacts on sleep quality assessed by hypnograms. For example, older participants had significantly faster transitions to WASO and slower transitions to deeper sleep states (i.e., N3 and REM). This finding agrees with the meta‐analysis performed by Baglioni et al. (2014), who found reduced time spent in deep sleep in older people with primary insomnia. We found that although men fell asleep significantly faster than women, it is more challenging for men to go into the deep sleep stage from the lighter sleep stage or stay in the deep sleep stage. Moreover, men also had shorter overall sleep duration than women. We also found Race/Ethnicity patterns in sleep hypnograms. Compared to White participants, Black participants fell asleep faster than White participants whereas the Hispanic group fell asleep slower than White subjects. All three minority groups had significantly reduced transition intensity from N2 to N3 stage, indicating it is more challenging for the minorities to go to the deep sleep stage than the White participants.

In terms of sleep questionnaires, while our correlation analysis shows that participants’ perception of having trouble falling asleep is not correlated with the simple TDS metric of SOL, our model did capture this association by estimating the TR_WBSO→N1 (i.e., the TR from WBSO to the first sleep event). Focusing on only WBSO to N1 transition but a similar survival analysis, Nisha Aurora et al. (2011) also found a significant association between the ESS and TR. Second, while there is no consensus in the current literature on whether REM is associated with less restful sleep, we found that reduced transition intensity into and out of the REM state is significantly associated with more restless sleep. We also found that increased transition intensities into and out of deep sleep were similarly associated with more restless sleep. The association between reduced TR_REM→WASF and restless sleep also suggests that finally waking up from the REM state tends to associate with the perception of more restful sleep (Dale Purves et al., 2001).

Interestingly, the question about ‘waking up too often’ was correlated with TDS WASO, but it was not associated with any of the TRs into or out of WASO. While counter‐intuitive, this is expected because TDS WASO and the state WASO are two different measures. The former is the total duration of WASO regardless of the frequency of WASO, and the latter is a stage in the sleep architecture. Someone can have a higher TDS WASO with a low transition intensity into the state WASO, say, if they woke up once during sleep and took a long time before going back to sleep. Conversely, subjects who have trouble maintaining sleep can transit frequently to the WASO state and result in a high transition intensity, but not stay long enough in WASO each time and result in a low TDS WASO. Thus, even though the state WASO and the TDS WASO are related, they may not be correlated with the same set of sleep questions, as revealed in our results.

Our proposed approach has several advantages for modelling hypnogram data. First, by separating the three types of wake state (WBSO, WASO, and WASF), our seven‐state model allows probing into how each type of insomnia is connected with the perception of insomnia. Given that falling asleep and finally waking up versus WASO are regulated by different neurotransmitters (Saper et al., 2010), analysing them separately will also shed light on how different types of insomnia are related to a health outcome. For example, neurone groups in the arousal system have been suggested to link to WASO and sleep fragmentation (Nofzinger et al., 2006), whereas sleep–wake cycles are dually regulated mostly by the flip‐flop switches orexin neurones and neurone groups in the ventrolateral preoptic nucleus (Saper et al., 2010). Conversely, mixing these three types of wake states will lead to inaccurate estimation of WASO, due to highly varying SOL. Second, the key parameters of interest in our model—transition intensities—offer additional insight into changes to sleep architecture. Sleep research has considered changes to sleep architecture by looking at total time spent in sleep states for a long time (Baglioni et al., 2014). However, this is not the only way to investigate sleep patterns in sleep architecture. Our research shows that examining rates of transition between states also reveals differences in sleep architecture patterns. Third, the CTMMs we proposed offer important time‐varying transition information about sleep architecture. This transition information is estimated by transition intensities, which describes the rate at which transition probabilities between states change over time. Furthermore, the mathematical advantage of CTMMs provides more precise estimations of transition quantities that are not bound by the temporal resolution of the hypnograms, as well as flexibility in extrapolating additional parameters based on transition intensities. For example, using Equation (3), we can also estimate the transition probabilities that are commonly estimated in a discrete‐time transition model (Swihart et al., 2008), whereas transition intensities cannot be obtained from a discrete‐time transition model. Duration‐based estimates can also be extrapolated from CTMMs, which are not available in discrete‐time models. Lastly, leveraging the ‘msm’ package, we directly analysed intensive longitudinal data (Jackson, 2011) in the panel data format without any transformation (e.g., the survival data format), and the computation time was not increased for a seven‐state model in comparison with previous three‐state or five‐state models (Kemp & Kamphuisen, 1986). Furthermore, the direct applicability of the ‘msm’ package will also enable more researchers to employ advanced multistate models when studying sleep hypnogram data.

One limitation of our study is the reliability of the ‘lights‐out’ time in the MESA Sleep study. During pre‐processing, we found that not every participant had a lights‐out time on record, and some participants were already asleep before the recorded lights‐out time. Notably, some participants also have a spuriously long period between lights‐out and falling asleep. In addition, participants may have had a very short SOL, which was considered invalid in the MESA Sleep study. Thus in our analysis, only 1721 participants with reliable SOL reading out of the 2056 that participated in the PSG study were included. Future sleep studies should consider the participants’ habitual sleep onset time in the experimental plan (Shrivastava et al., 2014). On the other hand, as the PSG studies in MESA Sleep were conducted in the participants’ own homes, it is expected that the ‘laboratory effect’ of adapting to the experimental conditions before falling asleep was minimised, thus the conclusions about SOL are likely to be close to reality. Second, certain sleep aspects were not considered in the questionnaires in the MESA Sleep Study (e.g., WHIIRS only focused on insomnia and did not measure dreams). Future sleep questionnaires should incorporate questions about dreams, and more comprehensive sleep questionnaires such as the Pittsburgh Sleep Quality Index can be considered when studying how objective sleep measures are related to subjective perception. Another limitation in WHIIRS was three questions ‘wake up earlier than planned’, ‘wake up often’, and ‘have trouble going back to sleep after waking up early’ have similar phrasing, which potentially leads to participants’ misunderstanding of the questions. Nevertheless, our model showed that all three questions are associated with slightly elevated transition intensities into WASO and N1, suggesting an inclination to lighter sleep. Lastly, because the average age of people who did not work was 5–10 years older than those who did work, the differences in sleep patterns for different job schedules could be due to age. Future research should test the interaction effects of job schedules and age, especially when studying older participants.

In conclusion, poor sleep quality can manifest in various transition patterns between sleep stages. Our proposed model provides a novel way to analyse the time‐inhomogeneous, multi‐state hypnogram generated by the complex sleep state switching patterns. By overlooking the sleep state transitions, the TDS failed to reveal important associations of transition patterns in sleep architecture and sleep questionnaires. For future sleep studies that focus on insomnia, sleep perceptions, and how sleep quality is associated with particular health outcomes, we recommend considering the seven‐state CTMM, which provides not only insightful connections to subjective perceptions of sleep quality but also rich information about the complex sleep state switching patterns.

AUTHOR CONTRIBUTIONS

Jonathon Jacobs: Software; formal analysis; visualization; writing – original draft; writing – review and editing; investigation; methodology. Caitlin E. Martin: Writing – review and editing. Bernard Fuemmeler: Writing – review and editing. Shanshan Chen: Conceptualization; investigation; methodology; validation; visualization; writing – review and editing; software; formal analysis; project administration; supervision; writing – original draft.

FUNDING INFORMATION

This work was supported by the following National Institutes of Health Grants: R34DA058494 and K23DA053507.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

Supporting information

Data S1. Supporting Information.

Jacobs, J. , Martin, C. E. , Fuemmeler, B. , & Chen, S. (2025). Profiling the sleep architecture of ageing adults using a seven‐state continuous‐time Markov model. Journal of Sleep Research, 34(2), e14331. 10.1111/jsr.14331

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are openly available in MESA Sleep Study at https://sleepdata.org/datasets/mesa.