Screening for orthosomnia: a reliability generalization meta-analysis of the Anxiety and Preoccupation about Sleep Questionnaire (APSQ)

Abstract

This mini-meta-analysis evaluated the internal consistency of the Anxiety and Preoccupation about Sleep Questionnaire (APSQ) across existing studies to assess its potential as an orthosomnia (an obsessive preoccupation with achieving perfect sleep) screening tool. A systematic literature search identified four studies with 2,506 participants using English, Swedish, Turkish, and Arabic versions. Cronbach’s alpha ranged from 0.91 to 0.95 across studies. The APSQ demonstrated high overall internal consistency reliability (pooled Cronbach’s alpha of the entire ASPQ = 0.93, 95% CI 0.91–0.94), suggesting utility for screening orthosomnia symptoms. The pooled Cronbach’s alpha of the first and second factors of the ASPQ were: 0.91 (95% CI 0.89–0.93) and 0.87 (95% CI 0.84–0.89), respectively. APSQ demonstrated high overall internal consistency reliability; however, limited linguistic/cultural representation and significant heterogeneity across studies impact generalizability.

Introduction

Orthosomnia is an obsessive pursuit of optimal sleep fueled by an unhealthy fixation on data from sleep-tracking devices [1]. Individuals with orthosomnia become preoccupied with achieving perfect sleep metrics and quality scores, leading to counterproductive behaviors that can paradoxically impair their sleep [1]. The rise of consumer sleep-tracking devices has led to increasing concerns about orthosomnia in the general population [2]. The need for a reliable screening tool for orthosomnia arises from the potential detrimental effects of this condition on an individual’s daily life and sleep quality [3, 4]. Orthosomnia can lead to heightened anxiety, distress, and counterproductive behaviors that inadvertently disrupt normal sleep patterns [2]. This obsession with sleep perfection can manifest as excessive sleep monitoring, rigid sleep routines, and rumination over perceived sleep deficits, precipitating and ultimately perpetuating insomnia and daytime impairments [2]. The ability to reliably identify individuals exhibiting orthosomnia symptoms is crucial for early intervention and prevention of insomnia and related functional impairments [2, 4]. The potential for orthosomnia to interfere with normal daytime function and nighttime sleep quality underscores the importance of developing reliable and validated screening tools to identify at-risk individuals [2].

The Anxiety and Preoccupation about Sleep Questionnaire (APSQ) was initially created by Tang and Harvey as a tool to measure anxiety and worries related to sleep [5]. It was further validated in 2011 by Jansson-Fröjmark and colleagues [6]. The APSQ consists of 10 items that form two distinct factors [6]. The first factor is formed by six items that focus on concerns over the potential consequences of poor sleep quality [3, 6]. The remaining four items, forming the second factor, capture fears and perceived lack of control over one’s ability to sleep well [3, 6]. Evaluations of the APSQ have demonstrated its validity and reliability in assessing sleep-related anxiety [3, 6, 7]. Furthermore, the questionnaire has shown a notable association between heightened anxiety levels and diminished sleep quality [3, 6, 7]. However, the APSQ must demonstrate adequate reliability across studies before it can be considered a useful orthosomnia screening tool [2]. This mini-reliability generalization meta-analysis evaluates the internal consistency of the APSQ across existing studies and discusses its potential as an orthosomnia screening tool.

Methods

We employed a reliability generalization meta-analysis (REGEMA) to assess the reliability of the APSQ [8]. The REGEMA approach allows researchers to estimate the overall reliability of a specific instrument and to investigate potential sources of variability in reliability estimates across studies. A comprehensive literature search was conducted to identify relevant studies reporting reliability estimates of the APSQ. Several databases, including PsycINFO, PubMed/MEDLINE, and Scopus, were searched from inception until the first week of March 2024. The search terms used included “Anxiety and Preoccupation about Sleep Questionnaire,” “APSQ,” “psychometric properties,” “reliability,” and “validity.” Additional studies were identified through manual reference list searches of relevant articles and reviews. Studies were included in the meta-analysis if they met the following criteria: (1) reported reliability estimates of the APSQ using Cronbach’s alpha (α); (2) published in a peer-reviewed journal; (3) utilized the APSQ or its translation as a self-report instrument for validation/reliability testing purposes as the main tool; (4) provided sufficient data to calculate effect sizes. A search of CrossRef citations up to May 8th, 2024 identified 36 published works that have referenced the APSQ since its initial validation by Jansson-Fröjmark et al. in 2011 [6]. An examination of these 36 cited works revealed that only two publications explicitly reported on the internal consistency reliability estimates for the APSQ in their respective study samples [3, 7].

Two authors (HJ and KT) independently screened the identified studies for inclusion with disagreements resolved through discussion. These authors then extracted information including study characteristics (e.g., sample size, participant age, and clinical status), reliability estimates, and potential factors that might influence reliability (e.g., study design, setting, and administration method). Disagreements in extraction were resolved through discussion with a third author (AP).

The overall reliability of the APSQ was estimated using a random-effects model, which considers the variability in reliability coefficients across studies. Heterogeneity among studies was assessed using the Q statistic, tau, tau2, and the I² index [9]. Residual analysis and influence diagnostics were employed to identify potential outliers and influential studies within the model’s framework. Studentized residuals exceeding the critical value derived from a Bonferroni-corrected normal distribution quantile were flagged as potential outliers [10]. This critical value accounted for the number of studies included in the meta-analysis and a two-sided alpha of 0.05. In addition, studies with a Cook’s distance surpassing the median plus six times the interquartile range of the Cook’s distances were considered influential observations [10]. To assess publication bias, the rank correlation test, and the regression test, using the standard error of the observed outcomes as a predictor variable, were utilized to examine funnel plot asymmetry [10].

Results

A meta-analysis was conducted to examine the internal consistency reliability of APSQ. The analysis included four studies [3, 5–7] with a total sample size of 2506 participants. The Cronbach alpha coefficients for the individual studies ranged from 0.91 to 0.95. The fixed effect pooled internal consistency was 0.92 (95% CI 0.91–0.93). The random-effects pooled internal consistency was 0.93 (95% CI 0.91–0.94) (see Fig. 1), the 95% prediction interval was 0.88 to 0.97. Therefore, the average outcome differed significantly from zero (z = 79.4901, p < 0.0001). The Schmidt–Hunter weighted mean internal consistency was 0.92 (95% CI 0.91–0.93). However, there was significant heterogeneity between studies (I² = 84%, p = 0.002). According to the Q-test, the true outcomes appear to be heterogeneous (Q⁽²⁾ = 17.5983, p = 0.0002, tau² = 0.0004. The credibility interval for the weighted mean internal consistency ranged from 0.91 to 0.94, indicating variability in the true population internal consistency. An examination of the studentized residuals revealed that one study (Turkish) had a value larger than ± 2.4 and may be a potential outlier in the context of this model. According to the Cook’s distances, none of the studies could be considered to be overly influential. Neither the rank correlation nor the regression test indicated any funnel plot asymmetry (p = 1.00 and p = 0.81, respectively).

Fig. 1.

Meta-analysis of the internal consistency of the Anxiety and Preoccupation about Sleep Questionnaire (APSQ)

For the first factor of the APSQ correlation coefficient: 0.91 (95% CI 0.89–0.93). Heterogeneity detected (tau² = 0.0002, I² = 78.77%). No outliers or influential studies found. No funnel plot asymmetry. Publication bias tests: Fail-Safe N = 61,099 (p < 0.001), Begg and Mazumdar Rank Correlation = − 0.33 (p = 1.00), Egger’s Regression = − 0.76 (p = 0.45).

For the second factor of the APSQ correlation coefficient: 0.87 (95% CI 0.84–0.89). Heterogeneity detected (tau² = 0.0004, I² = 76.97%). One potential outlier identified. No influential studies found. No funnel plot asymmetry. Publication bias tests: Fail-Safe N = 28,293 (p < 0.001), Begg and Mazumdar Rank Correlation = −0.33 (p = 1.00), Egger’s Regression = 0.59 (p = 0.56).

Discussion

The current mini-meta-analysis found high Cronbach’s alpha values for the APSQ across the four studies, indicating good overall internal consistency reliability. This suggests that the APSQ has the potential to be a useful screening tool for assessing orthosomnia symptoms. However, only full psychometric studies using the Swedish [6], Turkish [7], and Arabic [3] versions of the APSQ were available. The limited number of studies providing this foundational psychometric information for the APSQ highlights a gap in the current literature and emphasizes the need for more systematic reporting of reliability metrics when this measure is utilized across different populations and research contexts.

The overall internal consistency reliability for the APSQ was high, with a random-effects pooled value of 0.93 (95% CI 0.91–0.94). However, when looking at the two factors separately, the first factor had a slightly higher reliability coefficient of 0.91 (95% CI 0.89–0.93) compared to the second factor at 0.87 (95% CI 0.84–0.89). This suggests that while the overall measure demonstrates excellent internal consistency, there may be some variability in the reliabilities of the subscales or factors that comprise the APSQ. It is important to examine not just the scales’ overall reliability, but also the reliabilities of the individual factors or subscales when evaluating the psychometric properties of a multidimensional measure.

While sleep is a universal human need, the way individuals perceive and report sleep-related issues can be heavily influenced by cultural factors [4]. Concepts of what constitutes healthy sleep, beliefs surrounding the causes and consequences of sleep disturbances, and the vocabulary used to describe sleep experiences may vary significantly across different linguistic and cultural contexts [3]. Given these important concerns, it is crucial to undertake cultural adaptation and validation studies when assessing sleep-related worries in diverse populations [3]. Failure to account for these cultural nuances can lead to inaccurate interpretations, missed diagnoses, and inappropriate or ineffective interventions [3]. By adapting assessment tools, diagnostic criteria, and treatment approaches to specific cultural contexts, healthcare providers can better understand and address the unique sleep concerns and needs of individuals from different backgrounds [3].

Cultural sensitivity and rigorous reliability are essential steps in promoting equitable and effective sleep healthcare for all. Translations of the APSQ into other languages are needed to fully evaluate its potential as a widely applicable orthosomnia screening tool. Future research should examine the reliability and validity of the APSQ in more diverse linguistic and cultural contexts. Increasing widespread use of consumer sleep trackers means orthosomnia could be a growing concern across different nations. Developing APSQ versions in additional languages will allow for the more culturally appropriate screening of orthosomnia symptoms globally and more accurate comparison of cross-cultural rates. With established reliability in multiple languages, the APSQ can become a standardized global tool for detecting problematic orthosomnia.

The present reliability generalization study focused specifically on examining the internal consistency of the APSQ across various samples and contexts. While internal consistency is a crucial aspect of reliability, it is important to acknowledge that a comprehensive psychometric evaluation of a measure should also investigate other properties, such as test–retest reliability, construct validity, and other forms of validity evidence. Unfortunately, due to the limited information available in the included studies, we were unable to conduct a broader psychometric analysis within the scope of this paper. Future research efforts could aim to gather and synthesize data on additional psychometric properties, thereby providing a more comprehensive understanding of the measure’s performance and facilitating its wider adoption by researchers in the field. Such efforts would contribute to the ongoing refinement and validation of the APSQ, potentially accelerating its recognition as an adequate and well-established instrument.

This mini-meta-analysis represents a crucial step in evaluating the reliability of APSQ and discussed its value as a potential screening tool for orthosomnia. The notable strength of this mini-review is that the systematic literature search and adherence to established guidelines for REGEMA contribute to the robustness of the findings. However, the limited number of studies included and the lack of diversity in the cultural contexts represented pose significant limitations. This restricted sample may not fully capture the variability in reliability estimates that could arise from diverse linguistic and cultural contexts. In addition, the significant heterogeneity observed between studies highlights the need for further investigation into potential sources of variability, such as differences in study populations, administration methods, or cultural factors.

Conclusion

This meta-analysis found high Cronbach alpha values for the questionnaire overall, suggesting good reliability of the APSQ. However, the significant between-study heterogeneity needs to be explored further to understand the contributors to the variability in reliability estimates. Future research should prioritize the development and psychometric evaluation of the APSQ in diverse cultural contexts, accounting for potential variations in sleep-related beliefs, attitudes, and expressions across different populations. By addressing these limitations, the APSQ has the potential to become a robust and widely applicable tool for detecting and monitoring orthosomnia symptoms, ultimately facilitating early identification and intervention and promoting better sleep health outcomes worldwide.

Author contributions

Conceptualization, HJ, KT; methodology, HJ, KT, AA, AP; software, HJ; formal analysis, HJ; writing—original draft preparation, HJ, KT, AP, AA, ASB, SRP, MVV, writing—review and editing HJ, KT, AP, AA, ASB, SRP, MVV; funding acquisition, not applicable.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Declarations

Conflict of interest

The authors declare no conflicts of interest.

Ethical approval

Not applicable. This was a systematic review and meta-analysis of published studies that were indexed in the public domain.

Consent for publication

Not applicable.

Informed consent

Not applicable. This was a systematic review and meta-analysis of published studies that were indexed in the public domain.