Measures of sleep in rheumatologic diseases: Sleep quality patient reported outcomes in rheumatologic diseases

Introduction

Sleep is a fundamental human need, and can be impacted by biological, psychosocial, and environmental influences. Suboptimal sleep is associated with a decrease in quality of life and an increase in mortality (1). Common subjective sleep complaints in individuals with rheumatic diseases include difficulty falling or staying asleep, non-restorative sleep, and increased daytime sleepiness or fatigue. Primary sleep disorders such as obstructive sleep apnea (OSA) or restless legs syndrome are more common in patients with rheumatic diseases in comparison to the general population (2–4). As disruption in sleep has been linked with an induction of pro-inflammatory cytokines (5, 6), the evaluation of sleep disturbances and primary sleep disorders are particularly relevant to the comprehensive care and management of patients with rheumatic disease (7).

While objective measurements such as polysomnography are the gold standard in diagnosing sleep disorders, expense and access to these specialized facilities limit routine use. Patient reported outcomes (PRO) measures are widely available, validated instruments that can be used for assessment of sleep function and screening for sleep disorders. PROs are subjective measurements, and it is important that the clinician and/or researcher understand the psychometric properties and appropriate applications of different PROs.

In the following sections, we discuss the properties of several sleep-related PROs and their use in rheumatology, updating and expanding upon the 2011 review by T.A. Omachi (8). We begin with reviewing two legacy instruments, the Pittsburgh Sleep Quality Index (9) and the Epworth Sleepiness Scale (10), which assess sleep quality and daytime somnolence, respectively. We then introduce the Patient Reported Outcome Measurement Information System (PROMIS) sleep disturbance (SD) and sleep related impairment (SRI) instruments, which globally assess sleep function (11). Finally, we discuss PROs evaluating specific sleep disorders including the Insomnia Severity Index and Athens Insomnia Scale for insomnia (12, 13), and the Berlin Questionnaire for obstructive sleep apnea (14).

PITTSBURGH SLEEP QUALITY INDEX (PSQI)

Description

Purpose:

To measure sleep quality, assess sleep disturbances, and discriminate between good and poor sleepers (9).

Domains:

The PSQI measures seven domains of sleep quality: subjective sleep quality, sleep latency (time to fall asleep initially), sleep duration, sleep efficiency (percent of time in bed spent asleep), sleep disturbances, use of sleep medications, and daytime dysfunction.

Number of items:

There are 19 items included in the scoring. There are five additional items to be completed by a bed partner, which may be useful clinically, but are not scored.

Response options/scale:

Items 1–4 are free entry items asking habitual bedtime, subjective sleep latency, waketime, and subjective hours of sleep per night. Items 5–19 are 4-point Likert scales assessing frequency of a range of sleep symptoms, rating of sleep quality overall, and a final question on “how much of a problem [it] has been…to keep up enough enthusiasm to get things done.”

Recall period for items:

Previous one month.

Cost to use:

The PSQI may be reprinted without charge only for non-commercial research and educational purposes.

How to obtain:

The questionnaire and scoring instructions are available in the original publication (9) and at the following website: https://www.sleep.pitt.edu/instruments/. Permission for use may be requested online via a link on the website.

Practical application

Method of administration:

The questionnaire is a self-administered written form.

Scoring:

From the 19 items, seven domain subscores are calculated, each on a scale of 0–3, based on scoring algorithms involving a combination of categorization of free text responses and arithmetic determinations, as well as averaging of Likert responses. The global PSQI score is the unweighted sum of the seven subscores.

Score interpretation:

The global score ranges from 0–21 points, with higher scores indicating worse sleep quality. In the original validation study, a score of ≥ 5 identified “poor sleepers” with a sensitivity of 89.6% and specificity of 86.5% (9). The validation population included outpatients diagnosed with “disorders of initiating and maintaining sleep” or “disorders of excessive somnolence,” along with inpatients and outpatients with depression enrolled in studies on sleep and aging, nocturnal penile tumescence, and sleep in depression (9). Subsequent studies of patients with primary insomnia suggest a cutoff of six may better discriminate “poor sleepers” from “good sleepers,” with a reported sensitivity of 93–99% and specificity reaching 100% (15, 16).

Respondent time to complete:

Respondent completion time ranges from 5–10 minutes (9).

Administrative burden:

Scoring of the PSQI, without a calculator, can be performed in approximately 5–10 minutes (9). No software or special equipment is needed, though they may improve efficiency, particularly due to need for calculations from free text entries.

Translations/adaptations:

The PSQI is available in 56 different languages (17). A “shortPSQI” consisting of 13 items from five of the seven components in the original PSQI has been validated in a college student population (18).

Psychometric properties

Floor and ceiling effects:

Limited data are available. When assessed in an Ethiopian cohort of community dwelling adults, no floor or ceiling effect was found (19).

Reliability:

Internal consistency:

Cronbach’s alpha for global PSQI scores ranges between 0.67 to 0.83, depending upon the population sampled (20). In a cohort of patients with rheumatoid arthritis (RA), Cronbach’s alpha was 0.73 (21).

Test-retest:

The intraclass correlation coefficient (ICC) has been reported to be 0.90 for two days between testing (15), and 0.86 for a period of two to six weeks between testing (15, 20).

Validity:

Content/face:

PSQI items were developed based on clinical judgement rather than focus groups of all stakeholders including poor sleepers. However, the PSQI items capture a wide range of issues related to sleep quality and content appears suitable to accomplish its aim to distinguish good from poor sleepers.

Criterion:

Based on a gold standard of clinical evaluation, the PSQI distinguished “good sleepers” from “poor sleepers” in the original validation article with reasonable sensitivity and specificity (see Score Interpretation above) (9). Later work has challenged the single-factor model in which the seven subcomponents are summed into a single PSQI score to discriminate good and poor sleep and suggests that a 3-factor model is statistically favored and may better capture the multidimensionality of sleep assessed by the instrument (22). The 3-factor latent groups proposed by this study were 1) sleep efficiency, 2) perceived sleep quality, and 3) daily disturbances. Several others have also proposed 2-and 3-factor models for the PSQI (23).

Construct:

Convergent construct validity has been shown with other subjective sleep measures including the insomnia severity index (Pearson r = 0.80) (24), sleep efficiency score from a sleep diary (Spearman r = −0.562) (25), sleep problems from the Symptom Experience Report (Pearson r = 0.65–0.77) (26), and sleep restlessness from the Centers for Epidemiological Studies Depression (CES-D) scale (Pearson r = 0.69–0.75) (26). Minimal to moderate convergent construct validity between PSQI and depression as assessed by the CES-D and Beckworth Depression Index-I and II (r = 0.50–0.63) and tension/anxiety as assessed by the Profile of Mood States (r = 0.36–0.62) has also been shown (26–28). There is poor correlation (Pearson r = 0.16) between PSQI and ESS (29). While a few PSQI subdomains have been shown to possess a statistically significant correlation with more objective polysomnographic components, the strength of correlation is very low (Pearson r = 0.20–0.37) (9, 15, 25). There is also poor correlation with actigraphy, with sleep latency and sleep duration showing the best correlation at Spearman r = 0.275 and r = 0.204, respectively (25). Divergent construct validity has been shown when examining correlations of unrelated constructs of nausea, vomiting, and taste changes (Pearson r ≤ 0.37) (26).

Responsiveness:

Unknown.

Minimally important differences:

The PSQI is largely used to discriminate poor versus good sleepers with a cutpoint of five. A minimally important difference has not been reported.

Generalizability:

The PSQI is a legacy instrument that has been used to study prevalence of poor sleep and associated factors in a wide range of patients with rheumatologic diagnoses, including SLE (30–32), RA (33–36), spondyloarthritides (37–39), systemic sclerosis (28, 33), Sjögren’s syndrome (40, 41), osteoarthritis (42–44), and fibromyalgia (45, 46). It may not be able to detect well-known disturbances of sleep which accompany aging (25).

Use in clinical trials:

It is widely used in clinical trials as a measure of baseline sleep quality and change in sleep quality with intervention (27, 47–51).

Critical appraisal

Strengths:

The PSQI is a widely used legacy instrument assessing a variety of sleep domains to evaluate the construct of sleep quality. It can be self-administered and completed in a relatively short period of time. The availability of the PSQI in 56 different languages promotes cross-cultural assessment and comparison of sleep quality. The PSQI and scoring materials are readily accessible, free, and require no specific training to use.

Caveats and cautions:

Studies have reported the percent of questions omitted by participants to range between 6–10% (9, 52), predominately affecting the free text items. Immediate review of completed surveys by the clinician/researcher may aid in reducing missingness. While the PSQI has been shown to have validity in distinguishing good and poor sleepers based on clinical diagnosis of sleep dysfunction and other subjective measures of sleep, it has not been shown to have a strong correlation with objective measurements of sleep such as actigraphy or polysomnography (25, 29). Some researchers postulate that the PSQI may partially reflect a “negative cognitive viewpoint and pessimistic thinking characteristic of depression” given the lack of association with objectively measured aspects of sleep (25). Given associations between the PSQI and depression, researchers should consider controlling for depression in any studies utilizing the PSQI.

Clinical usability:

As the PSQI is separated into seven components affecting sleep quality, the primary driver of poor sleep (e.g. problems with latency, duration, etc.) may be identified through evaluation of subset scores. This may be helpful in evaluating possible interventions to improve sleep. It is important to note that the PSQI global score or subscale scores are not diagnostic of specific sleep disorders including insomnia, narcolepsy, obstructive sleep apnea, restless leg syndrome, or parasomnias (53).

Research usability:

Improvements in PSQI global score or domain subsets have been used to demonstrate success of a sleep intervention in randomized controlled trials and to examine associations between patient characteristics, comorbid conditions, risk factors and/or behaviors and sleep scores in observational studies (48, 50, 54–56). Cluster-based analysis of scores on subdomains (e.g. sleep latency, sleep duration, etc.) may identify subgroups of patients with different patterns of sleep dysfunction, as shown in a cohort of patients with systemic lupus erythematosus (SLE), which can inform the design of future sleep interventions (31).

Summary / Recommendations

Patients with rheumatic disease more commonly report subjective sleep dysfunction on the PSQI when compared to a control healthy population, and those who are “poor sleepers” have higher levels of pain and worse quality of life (37). This underscores the importance of incorporating sleep assessment in overall management of disease. The PSQI is an instrument that is particularly useful for characterizing an individual as a good or poor sleeper and has been validated in multiple languages facilitating its use among multiple populations. However, it is not designed for assessment of specific sleep disorders.

EPWORTH SLEEPINESS SCALE (ESS)

Description

Purpose.

To measure daytime sleepiness (10).

Content.

The ESS is intended to measure somnolence (i.e., somnoficity, or sleep propensity, which is the net interaction between the waking and sleep drive) in common situations. Somnolence depends on the situation in which it is measured (57). Thus, the instrument asks subjects to rate how likely they would “doze off or fall asleep” in eight different situations (i.e. highly somnolent situations such as “sitting quietly after lunch,” versus lower somnolent scenarios such as “sitting and talking to someone”). The ESS asks respondents to presume how a situation “would have affected you,” even if they have not done a given activity recently.

Number of items.

There are eight questions.

Response options/scale.

4-point Likert scale (0 = would never doze, 1 = slight chance of dozing, 2 = moderate chance of dozing, and 3 = high chance of dozing).

Recall period for items.

“In recent times.” Further specificity is intentionally not provided.

Cost to Use.

Free for individual users and academic research. An annual license fee may be applicable if usage is “deemed commercial in nature.”

How to obtain.

The survey instrument is available in the original validating publication (10), and is also available at http://epworthsleepinessscale.com. Permission to use can be obtained from Murray W. Johns, PhD, who can be contacted through the above web site or at Epworth Sleep Centre, Melbourne, Victoria, Australia. E-mail: 10), and is also available at mjohns@optalert.com.

Practical application

Method of administration.

The questionnaire is a self-administered written form.

Scoring.

The eight Likert response items are summed to calculate a total score.

Score interpretation.

The score range is 0–24, with higher scores indicating greater daytime sleepiness. Scores of 11–24 represent excessive daytime sleepiness (EDS). This criteria for EDS was based on a mean of 4.6 ± 2.8 (SD) among 72 healthy Australian workers (58).

Respondent burden.

2–3 minutes.

Administration burden.

Time to score is < 1 minute.

Translations/adaptations.

The ESS has been translated and validated in multiple languages, including Chinese, French, German, Greek, Hindi, Italian, Japanese, Korean, Spanish, Thai, Turkish (59–69). Additional translations are available but may not be validated (70).

Psychometric Information

Floor and Ceiling effects.

The ESS total score showed marginal floor (1.7%) but no ceiling (0%) effects in subjects with Parkinson’s disease (71). Specific items in ESS may have floor effects (“watching TV,” “sitting and talking”) or ceiling (“lying down to rest”) effects in healthy controls (72).

Reliability.

Internal consistency:

Adequate, with Cronbach’s alpha ranging from 0.74 to 0.88 (72–74).

Test–retest reliability:

This was reported to be high in healthy subjects who completed the questionnaire separated in time by five months (Pearson r = 0.82, P < 0.001) (73). However, in subjects with suspected OSA, 23% of patients had an ESS score discrepancy ≥ 5, separated by an average of 71 days, (Pearson r = 0.73, P < 0.001) (75). In a similar study, an ESS score discrepancy ≥ 5 was present in 21% of patients with OSA, (Pearson r = 0.67, P <0.01) (76).

Validity.

Content/face:

The focus of the ESS is to assess sleep propensity in different situations (face validity). While the initial examination of content validity does not reveal any inappropriate situations, the somnolent situations used in ESS have been criticized due to missing or vague rationales. For example, two of the scenarios were obtained from a study in abstract form (77) that used a population-based survey asking respondents to rank defined daily situations based on their relative somnolence. The two scenarios ranked “most sleepy” (“sitting, inactive in a public place” and “as a passenger in a car”) were incorporated into the ESS. More concerning, the remaining six questions have no known justifications for their inclusion. This leads to obvious omissions: for example, the relationship between OSA and motor vehicle accidents is well-established. A situation such as “driving a car,” which is distinct from situation eight in the ESS “in a car, while stopped for a few minutes in the traffic” since the subject is not told whether they are the passenger or driver, can be considered as an omission (78). Other real-world variables, such as caffeine consumption and work performance, are also missing.

Construct validity:

Concurrent validity of the ESS has been assessed in comparison with the multiple sleep latency test (MSLT) and maintenance of wakefulness test (MWT). The ESS has shown statistically significant but weak correlations of Spearman r = −0.30 and −0.37, respectively (79, 80). Despite the weak correlation, ESS has shown higher sensitivity and specificity compared to the MSLT and MWT in a study of distinguishing EDS from narcolepsy and normal subjects (81). One explanation is that ESS estimates sleep propensity over eight different situations, whereas the MSLT or MWT can measure only one situation. ESS has shown weak correlation to polysomnography in some studies, however, the subjects were predominantly male (82). Studies involving higher proportions of women, showed that ESS is less associated with the presence of sleep disordered breathing (SDB) in women compared with men (83). Other studies have shown no correlation with polysomnography (29). ESS has not shown any correlation with actigraphy (84).

Responsiveness.

The responsiveness of ESS scores to treatment effects has been demonstrated by their reduction after continuous positive airway pressure (CPAP) treatment for OSA (pre-treatment ESS = 13.5, post-treatment = 8.0) (85), with a mean reduction between −3.8 (86) and −2.7 points (87). The responsiveness of ESS has been assessed in the treatment of patients with narcolepsy. Improvement in average ESS scores of −5.8 after treatment with pitolisant, compared to −6.9 points with modafinil, and −3.4 points with placebo over an eight-week treatment period has been demonstrated in narcolepsy (88). Solriamfetol has also shown improvement in average ESS scores of −6.4 points, compared to −1.6 points for placebo, over a 12-week treatment period in narcolepsy (89).

Minimally Important Differences.

MID has not been formally established. Proposed MID in the direction of improvement range somewhere between 2–3 points (90, 91).

Generalizability.

The ESS has been used frequently in the study of OSA. It has been applied to multiple chronic comorbidities (71, 92, 93). ESS is used alongside other sleep measures, such as polysomnography and PSQI, to specifically measure the component of excessive daytime sleepiness. ESS has been used to evaluate the prevalence of EDS and to screen for sleep disorder comorbidities, such as OSA.

In rheumatic diseases, ESS has been used in RA (94–97), SLE (98, 99), fibromyalgia (100), sarcoidosis (101), and ankylosing spondylitis (102). It is used in evaluating the associations between disease activity and sleep, as well as effects of rheumatologic treatments (98, 102).

The ESS may not be applicable to all subjects (103), generating wide scatter. For example, the Hindi version required the revision of situation eight as a lower proportion of users of the Hindi version use or drive a car (63). Additionally, ESS may not be accurate in older adults: nondemented older subjects showed an underestimated and substantial incomplete response rate (104). ESS may also underestimate severity in women, as sleepiness in women may manifest as insomnia, fatigue, or mood disturbance (83, 94, 105).

Use in clinical trials.

The ESS has been used in randomized clinical trials to assess response to OSA treatment, where it is used alongside other sleep measures, such as the PSQI, and polysomnography (85, 106). ESS is used as an outcome measure when assessing the treatment response of medications used for narcolepsy (88, 89, 107, 108).

Critical appraisal

Strengths:

The ESS is one of the most widely used subjective sleep PROs, allowing the instrument to be compared across many populations and comorbidities. The ease of administration allows the ESS to be administered without adding additional burden in the research setting. The ESS, by specifically addressing the daytime symptoms of sleepiness, is an orthogonal measure to other sleep instruments, such as the PSQI which address nocturnal sleep disturbances. Finally, test-retest reliability was excellent in healthy subjects.

Caveats and cautions:

A concern in content validity is insufficient rationale for inclusion of the items used in the ESS. Different populations/individuals have variable familiarity with situations queried in the ESS. Also, not all subjects perform equally including older adults and women (83, 104). Given the score discrepancy seen on test-retest reliability in OSA populations, longitudinal assessments of ESS should be interpreted carefully (76).

Clinical usability:

It is a feasible questionnaire to administer and score. It is one of the only instruments that focuses entirely on somnolence. However, as mentioned previously, while ESS has been used as a screening tool for OSA, it possesses low negative predictive value (24.1%) and should not be used as the sole tool for diagnosing OSA (109).

Research usability:

It may be an inadequate singular tool in assessing sleep disorders comprehensively or assessing risk of OSA. However, the ESS can be a useful adjunct specifically focusing on excessive daytime sleepiness (110).

Summary / Recommendations

The ESS is a widely used sleep PRO in assessing daytime symptoms associated with poor sleep. In rheumatology, it is often used alongside other measures, such as the PSQI or BQ, to assess the prevalence of sleep disorders or the association with rheumatologic disease activity in patients. Although the ESS has been used to assess OSA severity, and has demonstrated responsiveness to change, it has low negative predictive value for the diagnosis of OSA. The ESS may be underestimated in older and female subjects.

PATIENT-REPORTED OUTCOMES INFORMATION SYSTEM (PROMIS) SLEEP DISTURBANCE (-SD) AND SLEEP RELATED IMPAIRMENT (-SRI)

Description

Purpose:

The PROMIS-SD and PROMIS-SRI evaluate qualitative aspects of sleep and wake function, across a range of conditions. They do not include time-based items nor assess specific sleep disorders (103).

Content or domains:

The PROMIS-SD is a unidimensional assessment of self-reported “perceptions of sleep quality, sleep depth, and restoration associated with sleep” (111). The PROMIS-SRI is a unidimensional assessment of self-reported “perceptions of alertness, sleepiness, and tiredness during usual waking hours, and the perceived functional impairments during wakefulness associated with sleep problems or impaired alertness” (111).

Number of items:

The PROMIS-SD full item bank contains a total of 27 questions. PROMIS-SD fixed length short forms are available as a 4-, 6-, or 8-question survey. The PROMIS-SRI full item bank contains a total of 16 questions. PROMIS-SRI fixed length short form is available as a 4-or 8-question survey. The PROMIS-SD and PROMIS-SRI are available as a Computer Adaptive Test (CAT). The CAT administers a varying number of questions from the full item bank, typically 5 to 8 questions, until a desired level of precision is reached. In a systemic sclerosis clinic, the mean number of items administered for PROMIS-SD CAT was 6.3 and PROMIS-SRI CAT was 6.9 (112).

Response options/scale:

Response options are graded using a Likert scale ranging from 1–5. Of note, some questions are positively phrased, e.g. “I was satisfied with my sleep” (in contrast to the majority of questions which are negatively phrased, e.g. “I had difficulty falling asleep”), and thus the scores for the positively-phrased answer choices are reversed (111).

Recall period for items:

Past seven days.

Cost to use:

All English and Spanish static paper form PROMIS measures are publicly available for use in research, clinical practice, educational assessment, or other application without licensing or royalty fees. Commercial users must seek permission to use, reproduce, or distribute measures. Web-based administration of short forms or CATs can be provided at www.AssessmentCenter.net and includes automatic scoring. New studies are charged a fee of $5,000 per study per calendar year. An Assessment Center Application Programming Interface (AC API) is also available which integrates with existing patient portal systems, survey systems, and mobile applications (e.g. Epic, REDCap, etc.) and can administer and score CATs and short forms. An annual license fee is required, and an individualized quote can be obtained by contacting api@assessmentcenter.net. Lastly, the PROMIS iPad application can be obtained via the Apple iTunes store for $500 and contains the PROMIS-SD CAT version and short forms 8a and 8b. The PROMIS iPad application does not include the PROMIS-SRI CAT but does include short form 8a.

How to obtain:

Short form surveys can be obtained at http://www.healthmeasures.net/search-view-measures. Further instructions on how to obtain and administer surveys using web-based administration, AC API, and/or the PROMIS iPad application can be found at http://www.healthmeasures.net/explore-measurement-systems/promis/obtain-administer-measures.

Practical application

Method of administration:

Written instruments, online surveys, or as CATs, as detailed under “Cost to Use” section.

Scoring:

For each fixed length short form survey, the raw score is summed and rescaled on the PROMIS score conversion table to determine the standardized T-score and standard error. Instructions are provided on how to handle missing data. The CAT is designed to administer the minimal number of questions to estimate the T-score and achieve a standard error < 0.30 (corresponding to a reliability of > 0.90) (113).

Score interpretation:

The PROMIS T-scores were calibrated on a large sample with a population mean of 50 and a standard deviation of 10. A higher PROMIS T-score represents more of the concept being measured, i.e. a higher PROMIS-SD T-score indicates more sleep disturbance subjectively experienced by the patient. A PROMIS-SD T-score of 60 indicates that the subject’s perceived sleep disturbance is one standard deviation worse than the average US population (111).

Respondent time to complete:

The average time to complete the written short forms varies by length, but on average, is 1–2 minutes. Computerized adaptive testing allows administration of fewer overall items, typically 5–8 questions. In a scleroderma clinic utilizing CAT, the mean time to completion was 0.9 minutes for PROMIS-SD and 1.0 minute for PROMIS-SRI (112).

Administrative burden:

The time to score each short form survey is < 1 min. T-scores can be manually determined for each respondent using scoring instructions provided at http://www.healthmeasures.net/search-view-measures. For investigators collecting survey responses in an Excel file, the Excel file saved as a comma separated version file type can be uploaded for automatic scoring using the free HealthMeasures scoring system powered by the Assessment Center at the following link: https://www.assessmentcenter.net/ac_scoringservice. Instructions on using the scoring system is found in a YouTube video at the following link in the references (114). The administration methods requiring a fee (web-based, AC API, PROMIS iPad app) perform automatic scoring.

Translations/adaptations:

The PROMIS-SRI and PROMIS-SD item banks are available in PDF in Spanish. PROMIS-SD is available in Chinese, Dutch, French, German, Hebrew, Korean, Latvian, Portuguese. PROMIS-SRI is available in Chinese, Dutch, French, Hebrew, Portuguese. Languages other than Spanish are subject to a distribution fee and can be obtained by contacting translations@healthmeasures.net. The SD and SRI are also available in pediatric versions and parent proxy versions, for both short forms and CATs.

Psychometric Properties

Floor and ceiling effects:

No floor or ceiling effects were observed in a scleroderma or RA cohort (112, 115). Minimal floor and ceiling effects (<5%) were seen in an SLE cohort (116).

Reliability:

Internal Consistency:

In the original validation study, the internal consistency was high, with a Cronbach alpha of 0.96 for the combined PROMIS sleep disturbance and sleep related impairment item bank, while item-total correlations were smaller than 0.40 for 34% of the items, indicating that these items were not too strongly related to a single underlying dimension (11). In an observational study of RA patients, the Cronbach’s alpha was 0.987 for PROMIS-SD and 0.978 for PROMIS-SRI (115). In a multi-racial and multi-ethnic observational cohort of patients with SLE, Cronbach’s alpha for PROMIS-SD was 0.80 and PROMIS-SRI was 0.92 (116).

Test-retest:

The test-retest ICC, in patients with RA, was r = 0.831, p < 0.001 for PROMIS-SD, and r = 0.725, p < 0.001 for PROMIS-SRI measured two days apart (115).

Validity

Content/face:

The PROMIS item banks have been developed through a systematic process of literature reviews, expert consensus, qualitative research methods, classic test theory (CTT) methods, and item response theory (IRT) analyses (11). The initial literature search yielded a list of 82 questionnaires and 2,529 sleep items assessing the sleep-wake domain. Qualitative item review per an established protocol reduced items with redundant content (117). Focus groups of subjects with sleep disorders, comorbid sleep disorders and psychiatric disorders, and normal sleepers, resulted in additional themes identified and inclusion of additional items. Cognitive interviews were performed to evaluate whether questions were understandable, and selected questions revised for readability, with a final average reading level of 3^rd grade and maximum of 7^th grade. Pilot testing reduced the item bank further, and initial psychometric testing was conducted using 128 items. CTT and IRT analyses were then performed, further reducing the number of questions. Through this highly iterative process, the final questions used in the PROMIS-SD (n = 27) and PROMIS-SRI (n = 16) reflect a highly calibrated item set with high face/content validity (11).

Criterion:

Criterion validation was based on subject self-identification of sleep problems. Surveyed subjects for psychometric validation were asked, “Have you ever been told by a doctor or health professional that you have a sleep disorder? What type of sleep disorder (with 13 options)? Has your sleep disorder been treated? Did the treatment help you?” Given the nature of the sampling frame, the presence or absence of participants’ self-reported clinical diagnoses could not be verified (11).

Construct:

Convergent construct validity was assessed with the PSQI and ESS in the original development of the PROMIS-SD and -SRI instrument banks. For PROMIS-SD, the Pearson correlation for the PSQI global score was r = 0.85 and for ESS was r = 0.25 (11). This agreed with the hypothesis that PROMIS-SD and PSQI measure similar attributes and verified discriminative ability of the PROMIS-SD instrument for measuring sleep disturbance rather than sleep related impairment. For the PROMIS-SRI, the Pearson correlation for the PSQI was higher than expected at r = 0.70, and lower than expected for PROMIS-SRI and ESS at r = 0.45. However, as the PROMIS-SRI and ESS correlation was higher than the PROMIS-SD and ESS correlation, Buysse et al. concluded these results support the validity of the instruments in measuring two different sleep-wake constructs (11). A later study by Yu et al. evaluated construct validity of the PROMIS-SD and PROMIS-SRI 8-item short forms. The Pearson correlations between the 8-item short form and full bank was 0.95 for PROMIS-SD and 0.98 for PROMIS-SRI, and was equivalent or better than correlations between the CAT and full banks (103). The Pearson correlations between the PROMIS-SD and PROMIS-SRI 8-item short forms and PSQI was 0.83 and 0.68, respectively. The Pearson correlations between the PROMIS-SD and PROMIS-SRI 8-item short forms and ESS was 0.30 and 0.46 (103).

Responsiveness:

Responsiveness to change has been demonstrated in a prospective longitudinal study in RA patients. After 12 weeks of DMARD therapy, PROMIS-SD improved from a mean of 55.2 ± 8.5 to 50.9 ± 8.8 (p<0.001), and PROMIS-SRI improved from a mean of 55.2 ± 9.8 to 52.0 ± 10.5 (118).

Minimally important differences:

In a SLE cross-sectional study, distribution-based estimates of MID was approximately 4, and anchor-based estimates of MID was two points for worsening and one to two points for improvement (119). Other PROMIS measures not measuring sleep, had MID that varied from two to three points (120–122).

Generalizability:

The goal in the development of the PROMIS measures was to create universal, non-disease specific, patient reported outcomes measures. Therefore, diverse populations were recruited to develop and validate the instruments. In aggregate, the population used for psychometric testing and development of PROMIS-SD and PROMIS-SRI item banks included a total of 2,252 subjects, 44% women, 82% white, and 12% black (11). The sample was also enriched with 259 patients from sleep medicine, general medicine, and psychiatric clinics endorsing sleep symptoms. In the development cohort of PROMIS-SD and PROMIS-SRI short forms, 25% reported having one chronic health condition, 21% reported having two, and 33% reported having three or more. PROMIS-SD has been assessed in older adults (mean age 83.6 years) with good performance and predictive of depressive symptoms, stress, and quality of life at 12 months (123).

Psychometric properties of PROMIS sleep measures have been assessed in RA (115, 118), SLE (116, 119), scleroderma (112), cancer (124, 125), and elderly populations (123). Additionally, PROMIS sleep measures have been used in multiple sclerosis (126), inflammatory bowel disease (127), and hospitalized patients (128) to study the prevalence of sleep problems.

Use in clinical trials:

The effect of DMARD use on PROMIS measures, including SD and SRI, in RA patients has been evaluated (118). The PROMIS SD and SRI have been used in a clinical trial examining the effectiveness of melatonin and behavioral sleep-wake cycling in patients with delayed sleep-wake disorder to demonstrate effectiveness of therapy (129), and in patients with diabetes initiating positive airway pressure to demonstrate improvements in sleep (130). Few studies are available regarding the use of PROMIS-SD or PROMIS-SRI in randomized clinical trials (129).

Critical appraisal

Strengths:

PROMIS started as a National Institutes of Health (NIH)-funded consortium to develop universal core questionnaires that measure health-outcome domains in a variety of chronic diseases. PROMIS measures are analogous to “thermometers” that provide continuous, relative values for individuals instead of condition-specific measures that categorize based on a cut-off score (11). PROMIS item banks have been developed through a systematic process of literature reviews, expert consensus, qualitative research methods, classic test theory methods, and item response theory (IRT) analyses (11, 103), which leads to high content/face validity. IRT is a psychometric technique in which the probability of choosing each item category is modeled as a function of a latent trait of interest (103). Different items are better at discriminating people at different levels on the continuum of severity. With IRT, CAT avoids administering test items that do not further discriminate an individual’s measure of sleep, leading to fewer questions that need to be answered by participants in order to arrive at a precise T-score. Another important strength of PROMIS instruments is the use of a common T-score metric across multiple conditions experienced by patients, as well as across the general population, which may improve the ability to compare sleep problems across many different diseases. This is in contrast to disease specific legacy instruments (115).

Caveats/Cautions:

One caveat of the PROMIS sleep written instruments, leading to inadvertent incorrect answers, is the reverse-score items. For example, most questions are negatively phrased, e.g. “I felt tired,” with the highest Likert score “5” on the right-most column. When a positively phrased question “My sleep was refreshing,” with the lowest Likert score “1” on the right-most column, is intermittently administered, patients may be confused and inadvertently answer the question in reverse scale. In the authors’ experience, as well as in a cancer cohort, reverse-scored items as a method effect contribute to model misfit (125).

Additionally, the psychometric validations of the PROMIS sleep instruments used a population in which the majority of subjects had a higher educational background. Specifically, the percentage of education attainment was high school or less (14%), some college (38%), college degree (28%), and advanced degree (20%) (103). Therefore, the instrument may not be representative of patients with lower educational background. However, reading level of questions was assessed, and determined to range between 3rd and 7th grade.

While the PROMIS instruments were designed to be administered without regard to an underlying disease state, response shifts in populations with chronic diseases have been observed. For example, in RA patients with a CDAI score ≤1.0, the median T-scores of multiple PROMIS instruments were better than population norms (i.e. < 50), including a median PROMIS-SRI T-score of 44.5 (115). It is postulated that the potential response shifts occur as patients recalibrate their experiences and redefine a new baseline or expected norm. Their ideal T-score may simply be their baseline rather than an absence of symptoms (115). Further research in patients across spectrums of disease severity, duration, and disability is warranted to define the norms of specific chronic diseases.

Clinical Usability:

PROMIS-SD and PROMIS-SRI show high feasibility to integrate into clinical practice given minimal time to complete. The availability of CAT integration allows for patients to fill out multiple PROMIS instruments while maximizing precision and minimizing respondent burden. The AC API functionality allows for integration into the electronic medical record system such as EPIC. Integration into the electronic medical record system allows for point-of-contact review of results with the patient by the clinician, facilitating discussion of scores and possible interventions. The scale is easy to interpret and explain, with a T-score reflecting a population mean for the domain being assessed.

Research usability:

The minimal time to complete and automated scoring assistance aids in feasibility to incorporate into research. The more automated methods utilizing web-based administration, AC API integration into research databases such as REDCap, and iPad applications may be cost prohibitive for many. However, short forms, which have been validated, provide a free alternative. Additional studies are needed to define MID and responsiveness to change. While it is meant as a universal instrument, there are some indications that different populations may have different averages (115).

Summary / Recommendations

The PROMIS-SD and PROMIS-SRI item banks were developed through a rigorous systematic process with high face, content, and construct validity. The PROMIS instruments can be administered in written form or via computer adaptive testing in a manner that maximizes accuracy while minimizing respondent burden. Computer adaptive testing may allow multiple PROMIS domains to be assessed with high precision and minimal time.

The PROMIS instruments are calibrated with a common T-score and standard deviation across the general population including healthy subjects and patients with comorbidities, allowing the results to be compared across diseases. This non-disease-based instrument development approach allows for comparison of PROMIS-SD and PROMIS-SRI across multiple rheumatic diseases. The instruments are not meant to dichotomize patients into “good sleepers” or “poor sleepers,” as is the purpose of PSQI, or identify “excessive daytime sleepiness,” as is the purpose of ESS. Rather, an estimate of a patient’s global PROMIS-SD and PROMIS-SRI with regard to a reference population is provided and thus may pose some challenges in interpretation of results in research settings. Additionally, further work on understanding the MID in rheumatic diseases is needed prior to use as an outcome in clinical trials.

INSOMNIA SEVERITY INDEX (ISI)

Description

Purpose:

To measure insomnia symptoms and insomnia-related distress.

Domains:

The ISI corresponds with the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria for insomnia. The questions assess 1) sleep onset, 2) sleep maintenance, 3) early awakening, 4) satisfaction with sleep, 5) interference with daytime functioning, 6) noticeability of sleep problem to others, and 7) distress caused by the sleep problem.

Number of items:

There are seven items.

Response options/scale:

5-point Likert response, with zero corresponding to “not at all” or “very satisfied” and a score of four corresponding to “very much” or “very dissatisfied.”

Recall period for items:

Last two weeks.

Cost to use:

Free to use with permission. The author, Charles M. Morin, Ph.D., can be contacted at cmorin@psy.ulaval.ca.

How to obtain:

The ISI can be obtained from the original publication and original validation study (13, 131).

Practical application

Method of administration:

The questionnaire is a self-administered written form. There are three parallel versions of the ISI: 1) participant (ISI-P) version, 2) clinician (ISI-C) version, and 3) participant’s significant other (ISI-SO) version. The original validation focused on the participant’s self-administered version (13). Few studies have compared all three versions (132).

Scoring:

The seven response items are summed to calculate a total score.

Score interpretation:

The score range is 0–28 points. Higher scores indicate greater severity of insomnia. The authors suggest the following categories defined by a range of scores: no clinically significant insomnia (0–7), subthreshold insomnia (8–14), moderate clinical insomnia (15–21), severe clinical insomnia (22–28). These score guidelines, however, have not been validated (13).

A cutoff score of ten was optimal for detecting insomnia in a community-based sample, with a sensitivity of 86.1% and specificity of 87.7%, whereas a cutoff score of eleven was optimal for detecting insomnia in a clinical sample with a sensitivity of 97.2% and a specificity of 100% (24). In a cancer cohort, the cut-off score of eight was optimal for detecting insomnia (132). In a primary care clinic, a cutoff score of 14 was optimal for detecting insomnia (133).

Respondent time to complete:

Time to complete is < 5 minutes (13).

Administrative burden:

Time to score is < 1 minute (13).

Translations/adaptations:

The ISI is available in Arabic, Chinese, French, German, Hindi, Italian, Korean, and Spanish (134–142).

Psychometric properties

Floor and ceiling effects:

There were no floor or ceiling effects in a random sample of Ethiopian adults (143).

Reliability:

Internal consistency:

Cronbach’s alpha was 0.74 in the original validation study (13). A subsequent population-based study of 959 individuals showed high internal consistency with a Cronbach’s alpha of 0.90 and 0.91 for community and clinical samples, respectively (24). Item-total correlations ranged 0.50 to 0.85. Items about satisfaction and worry showed higher item-total correlations, while items about difficulty initiating sleep or early morning awakening showed lower correlations.

Test-retest reliability:

Pearson correlation was r = 0.83 after one month, r = 0.77 after two months, and r = 0.73 after three months in a cancer cohort (132).

Validity:

Content/Face:

The items in the ISI correspond to many of the criteria for insomnia in the DSM-IV, as well as the updated DSM-5, including difficulty initiating or maintaining sleep, dissatisfaction in sleep, early morning awakening, interference with daytime functioning, and distress related to the sleep disturbance (144). In the original validation study, a principal component analysis explored the ISI content validity. This analysis yielded three components that explained 72% of the total variance. Component I included items related to daytime interference, noticeability by others, and level of distress. Component II related to severity of sleep onset, maintenance, and early morning awakening difficulties. Component III included satisfaction with sleep and severity of initial insomnia (13).

Construct:

The ISI total score was positively correlated with the PSQI total score (Pearson correlation r = 0.80, p < 0.05) and total wake time (r = 0.59, p < 0.05) and negatively correlated with total sleep time (r = −0.54, p < 0.05) and sleep efficiency (r = −0.59, p < 0.05) on daily sleep diaries (24). The ISI total score was negatively correlated, albeit weakly, with sleep efficiency on polysomnography (Pearson correlation r = −0.16, p < 0.05).

Responsiveness:

The ISI showed a mean score reduction of −5.0 points in 1527 patients in an open-label trial of ramelteon over 12 weeks (145). In a randomized double-blind control trial, among patients receiving eszopiclone, the mean ISI decreased from 17.9 points to 8.3 points (improvement), whereas patients on placebo decreased from a mean of 17.8 points to 12.9 points at six months (p < 0.0001 for the difference between groups) (146). Cancer patients who were randomized to cognitive behavioral therapy demonstrated a mean ISI improvement from 16.2 points to 7.6 points, compared to a waiting-list control group of 16.1 to 13.7 (147).

Minimally important differences:

Data from a randomized double-blind clinical trial of eszopiclone for primary insomnia recommended a six-point score reduction (corresponding to 1.5 standard deviations) in the ISI as being the MID (148). A large population based study that also evaluated treatment response recommended a > 7 point score reduction to be considered moderately improved (24).

Generalizability:

The ISI has been validated in the general population (24, 133), patients with sleep disorders (24, 149), and cancer (132, 150). It has been used to assess sleep difficulties in RA (151), SLE (152), fibromyalgia (153), and osteoarthritis (154, 155).

Use in clinical trials:

The ISI has been used in randomized clinical trials involving sleep medications (146), cognitive behavioral therapy (147), and both (156). ISI has been able to discriminate between patients in the intervention group compared to control group.

Critical appraisal

Strengths:

The ISI is feasible to administer and has high face validity with the DSM-IV. The ISI has been widely used and tested to assess psychometric properties and to determine optimal cutoffs and MID, depending on the setting and population.

Caveats and cautions:

While the ISI corresponds to the DSM-IV criteria in regards to the description of sleep difficulties, the criteria for duration of symptoms is different. ISI pertains to the past two weeks, while the duration criterion is one month in the DSM-IV and three months in DSM-5. The ISI is limited in ability to diagnose insomnia compared to a structured clinical interview (157), as it was designed to measure insomnia severity.

Clinical and research usability:

The ISI has been validated in the general population and primary care cohorts as well as in patients with comorbid disease, thus it is able to span the spectrum of healthy subjects to patients with multiple comorbidities. The ISI can be easily integrated in the clinical setting as a screen for insomnia or sleep disorders. Validation studies have recommended varying cutoffs for different populations, thus it may be hard to establish the most agreed upon cutoff for a future investigation (13, 24, 133, 149). However, the ISI has also been used as a continuous variable in clinical trials, and has shown responsiveness to change appropriately (146–148, 156).

Summary / Recommendations

The ISI is an excellent tool for assessing insomnia symptoms, due to its short length, feasibility, and applicability to both healthy and patient populations. The ISI has been widely used in various populations to measure severity of insomnia, with responsiveness demonstrated in multiple populations including in clinical trials. For the diagnosis of insomnia, different optimal cutoffs have been determined in different populations, and have yet to be validated against updated diagnostic criteria.

ATHENS INSOMNIA SCALE (AIS)

Description

Purpose:

To simultaneously establish the diagnosis of insomnia and measure its severity (12).

Domains:

The first five questions pertain to sleep onset, awakenings during the night, early morning awakening, total sleep time, and sleep quality (criterion A of the International Classification of Diseases, 10th Revision (ICD-10) criteria for insomnia).

The last three questions refer to well-being, functional capacity, and sleepiness during the day (criteria C and D of the ICD-10 requirements for insomnia) (12, 158).

Number of items:

The standard questionnaire is eight items (AIS-8). A brief five item version (AIS-5) is also available and contains only the first five items that correspond to nocturnal sleep quality, but not daytime symptoms.

Response options/scale:

Each item is rated on a 4-point Likert scale, with zero corresponding to “no problem” or “normal,” and a score of three corresponding to “serious problem” or “did not sleep at all.” The cumulative score ranges from 0–24.

Recall period for items:

“At least three times per week during the last month,” which corresponds to criterion B of the ICD-10 for insomnia.

Cost to use:

Free to use.

How to obtain:

The AIS can be obtained from the original publication (12).

Practical application

Method of administration:

The questionnaire is a self-administered written form.

Scoring:

The eight responses are summed to calculate a total score, which ranges from 0–24.

Score interpretation:

A higher score indicates greater insomnia symptoms. For the 8-item scale, a cutoff score of ≥ 6 is recommended to identify individuals with a diagnosis of insomnia (159).

Respondent time to complete:

2–3 minutes.

Administrative burden:

Time to score is < 1 minute.

Translations/adaptations:

The AIS is available in Arabic, Chinese, Japanese, Greek, Korean, Polish, and Spanish (160–167).

Psychometric properties

Floor and ceiling effects:

Unknown.

Reliability:

Internal consistency:

Cronbach’s alpha was 0.87 (AIS-5) to 0.89 (AIS-8) in the original cohort (12).

Test-retest:

The Pearson correlation coefficient was 0.88 (AIS-5) to 0.89 (AIS-8) for questionnaires administered one week apart (12). For a longer period of test-retest reliability at a mean of 81 days in a chronic pain population, the ICC was 0.64 for AIS-8 (168).

Validity:

Content/Face:

AIS has high face validity as the items match ICD-10 criteria for nonorganic insomnia. Notably, the International Classification of Sleep Disorders, version 3 (ICSD-3) does not differentiate between organic, nonorganic, or other subtypes of insomnia (except acute or chronic) and has different diagnostic criteria compared to ICD-10 (169). The DSM-5 similarly removed the distinction between primary and secondary insomnia, and has different diagnostic criteria compared to ICD-10 (144). Since the ICSD-3 and DSM-5 are used more widely in sleep medicine, the face validity of AIS may be decreased compared to these updated diagnostic criteria.

Criterion:

Using a cut-off of ≥ 6, the AIS-8 distinguished individuals with insomnia from those without insomnia with a sensitivity of 93% and specificity of 85% in the original validation cohort (12). The original validation cohort consisted of 176 insomniacs and 123 noninsomniacs diagnosed according to non-structured clinical interview with diagnosis based on ICD-10 criteria. This cohort was comprised of 105 patients presenting to a primary health clinic with complaint of insomnia without an obvious cause, 144 psychiatric patients, and 50 non-patient controls.

Construct:

In the original validation cohort, strong convergent construct validity was shown with the Sleep Problems Scale, with Pearson correlation coefficients of 0.85 (AIS-5) and 0.90 (AIS-8) (12). Construct validity with the PSQI and Brief Fatigue Inventory was also observed in a Taiwanese cancer cohort (Pearson r = 0.82 and r = 0.56, respectively) (167). In this same cohort, AIS also weakly correlated with actigraphy parameters of sleep onset latency (Pearson r = 0.39), sleep efficiency (r = −0.54), wake after sleep onset (r = 0.28), and total sleep time (r = −0.30) (167). In a renal transplant population, AIS was not correlated with polysomnography parameters (170).

Responsiveness:

In RA, acupuncture therapy significantly improved AIS scores from 14.1 ± 1.5 to 6.2 ± 1.5, compared to RA controls receiving estazolam with improvement in AIS from 14.2 ± 1.3 to 8.2 ± 1.7 (171). Limited studies are available regarding change in AIS over time.

Minimally important differences:

Unknown.

Generalizability:

AIS has been used to evaluate insomnia in cancer (167), chronic pain (168, 172), kidney transplant (170), OSA (173), RA (95, 97, 171, 174), SLE (175), and Sjögren’s syndrome (176). In these studies, the role of AIS has varied from studying sleep disorders, insomnia, and fatigue in specific comorbidities alongside other measures, such as PSQI, ISI, Berlin Questionnaire, or actigraphy. AIS has also been used to determine the baseline prevalence of insomnia as a comorbidity.

Use in clinical trials:

One trial has studied the change in AIS in RA patients after acupuncture therapy, and several upcoming intervention trials in RA will assess AIS at baseline and follow up visits (171, 177–179).

Critical appraisal

Strengths:

The AIS is based on ICD-10 criteria for nonorganic insomnia. AIS was endorsed by the OMERACT 9 working group for RA (180). AIS was the highest-ranked sleep instrument among a list of 15 instruments in regard to the OMERACT filters of truth (validity and reliability) and feasibility. AIS was able to assess the top 4 sleep-related domains (sleep adequacy, sleep maintenance, sleep initiation, and daytime functioning) ranked by the OMERACT working group (180). The AIS is brief – 8 items or 5 items in the short version – and therefore has minimal respondent or administrative burden.

Caveats and cautions:

ICSD-3 and DSM-5 criteria for insomnia no longer distinguish between nonorganic or organic (or primary or secondary) insomnia, and have slightly different criteria and time frames for diagnosis of insomnia (e.g. DSM-5 criteria now uses a recall time frame of 3 months). Therefore, the insomnia diagnosis by AIS may not correspond to insomnia diagnosis by ICSD-3 or DSM-5 criteria.

In the original validating cohort, a cutoff ≥ 6 maximized sensitivity and specificity, but other studies have suggested higher cutoffs of ≥ 7 in a Taiwanese cancer cohort (sensitivity of 96% and specificity of 76%) or ≥ 8 in a Japanese chronic pain cohort (sensitivity of 78% and specificity of 70%) (167, 168). A cut-off score of ≥ 10 was chosen for the purpose of higher positive predictive value in a study among kidney transplant recipients (159, 170). Limitations in OSA have been suggested as patients with OSA have altered perceptions of sleep (173). Other sleep surveys, such as the Berlin Questionnaire may be more appropriate if intended to screen for comorbid OSA (173, 181).

Clinical and research usability:

In clinical and research settings, AIS can serve to estimate prevalence of insomnia as a baseline comorbidity. The minimally important difference has not been well established.

Summary / Recommendations

The AIS has high face and construct validity for screening for insomnia by ICD-10 criteria and has high feasibility for implementation both clinically and in research. The precise cut-off at which sensitivity and specificity is maximized may differ by population, and it is unknown how well AIS performs against more recent insomnia diagnostic criteria in ICSD-3 or DSM-5. Additional studies in specific rheumatic diseases determining best cut-offs are still needed to improve generalizability of the AIS. For measuring severity of insomnia, there are insufficient data, particularly in rheumatological diseases, to support use of AIS, and the responsiveness to change and MID are unknown.

BERLIN QUESTIONNAIRE (BQ)

Description

Purpose:

To screen for sleep apnea, categorizing individuals as high or low risk for OSA (14).

Domains:

The BQ consists of three categories related to the risk of having OSA: snoring, daytime sleepiness, and hypertension.

Number of items:

There are ten questions. Additional information asked includes height and weight (for BMI calculation), age, and sex.

Response options/scale:

Four questions have options: “Yes,” “No,” or “Don’t know.” Six questions ask about frequency ranging from “never or nearly never” to “nearly every day.”

Recall period for items:

Not specified.

Cost to use:

Free to use with permission from the American College of Physicians. permissions@acponline.org.

How to obtain:

The BQ can be obtained at the following web address: https://www.ncbi.nlm.nih.gov/books/NBK424168/bin/appb-fm1.pdf

Practical application

Method of administration:

The questionnaire is a self-administered written form.

Scoring:

There are three categories, and scoring differs for each category and their component items, which may be scored on a 0–1 or 0–2 scale. Category 1 is positive if the total score is 2 or more points for items 2, 3, 4, 5, or 6; category 2 is positive if the total score is 2 or more points for items 7, 8, or 9, and category 3 is positive if item 10 is present (hypertension) or BMI > 30 kg/m².

Score interpretation:

The respondent is considered high risk for OSA if two or three categories are positive and considered low risk if zero or one category is positive.

Respondent time to complete:

2–3 minutes.

Administrative burden:

Scoring requires an answer key, but time to score is < 1 minute.

Translations/adaptations:

The BQ is available in Arabic, Chinese, French, Greek, Korean, Malay, Persian, Portuguese, Spanish, and Thai (182–190).

Psychometric Properties

Floor and ceiling effects:

Unknown.

Reliability:

Internal consistency:

Cronbach’s alpha was adequate, ranging from 0.86 – 0.92, in the original validation study (14).

Test-retest reliability:

Cohen’s kappa was 0.92 after a time interval range of one to 27 days (median: eight days) in a pre-operative surgery cohort (191).

Validity:

Content/face:

The BQ was developed based on expert consensus, using questions that were selected from the literature to elicit factors or behaviors that consistently predicted sleep disordered breathing (14). Questions review snoring, fatigue/tiredness, and comorbidities (obesity, hypertension) associated with higher risk for OSA, and therefore the BQ has high face validity.

Criterion validity:

The criterion validation cohort was drawn from 100 patients seen for any indication from five primary care practice sites. A portable home monitor was used to detect a respiratory disturbance event defined as a) a decrease in nasal or oral airflow, alone or with chest wall movement that lasted for 10 seconds or more, or b) a decrease in SaO2 of 4% or more. From this information, a respiratory disturbance index (RDI) (measured as the number of respiratory events per hour in bed) was generated, and a higher mean RDI was seen in the BQ high risk group (21.1 ± 18.5) as compared to the BQ low risk group (4.7 ± 7.0). For RDI cutoff > 5, the sensitivity of BQ for identifying the high-risk group was 86% and specificity 77%. For RDI cutoff > 15, sensitivity was 54% and specificity was 97% (14).

Construct validity:

The BQ has moderate construct validity with polysomnography in studies including a pre-operative cohort, primary care cohort, and sleep clinic cohort (184, 191, 192). In these studies, the high-risk group as categorized by BQ had a sensitivity of 69–84% and specificity of 39–56% for detecting patients with an apnea-hypopnea index (AHI) > 5 (mild apnea) by polysomnography, with only marginal improvements in sensitivity to 77–87% and decrease in specificity to 33–51% for an AHI > 15 (moderate apnea).

Responsiveness:

Unknown.

Minimally important differences:

Unknown.

Generalizability:

The BQ has been widely used to assess the prevalence of OSA and to evaluate associations between OSA and other comorbidities (181, 193–195). In rheumatology, the BQ has been used in osteoarthritis (154), RA (2, 94, 196), and Behcet’s disease (197). In women, the BQ has been shown to have a higher sensitivity but lower specificity as compared to men (181).

Use in clinical trials:

In the Resistant Hypertension Optimal Treatment (ReHOT) randomized clinical trial comparing spironolactone versus clonidine treatment for resistant hypertension, the BQ was used singularly to estimate the baseline comorbidity of “high risk for sleep-apnea” (198). In a randomized control study evaluating the impact of bariatric surgery on OSA, the BQ was measured pre-operatively and 90 days post-operatively (193). There was significant improvement in the BQ, which was consistent with a significant change in the mean AHI before and after bariatric surgery, which may also provide some evidence of responsiveness.

Critical appraisal

Strengths:

The BQ is a feasible survey to administer with good sensitivity in the primary care and sleep clinic populations.

Caveats and cautions:

Despite the good sensitivity in identifying OSA, multiple studies have demonstrated that the BQ has low specificity (183, 192). Also, many of the studies evaluating its diagnostic performance involve sleep clinic cohorts, which have a higher proportion of sleep disorders and OSA compared to the general population (199). This may limit the generalizability of its diagnostic performance in the general population.

Clinical and research usability:

Its use in research has remained focused on screening for OSA, often in conjunction with other sleep measures. Some studies have used the BQ as a surrogate for baseline OSA risk (198), however, its low specificity may lead to an overrepresentation with false-positives. Other sleep instruments evaluating for OSA, such as STOP, or STOP-Bang, have been evaluated alongside the BQ, but have similarly high sensitivities with low specificities (192, 199). The ESS, on the other hand, performs with higher specificity and low sensitivity (see section on ESS) (109). In clinical settings, the BQ should be used only when the advantage of higher sensitivity outweighs the disadvantage of low specificity (200). The BQ does not replace objective diagnostic tools (i.e. in-home or laboratory polysomnography) for OSA.

Summary / Recommendations

The BQ is a short and feasible survey used as screening for OSA. In rheumatology, the BQ has been used as a surrogate for OSA or alongside other sleep PROs to assess sleep quality. Although the BQ has good sensitivity in screening for OSA, the low specificity limits its diagnostic performance. While some studies use the BQ as a surrogate for OSA risk, this may lead to overrepresentation with false positives. STOP and STOP-BANG are similar surveys used to screen for OSA, but similarly have high sensitivity and low specificity for detecting OSA compared to polysomnography.

Table 1:

Practical applications

Measure	Number of items	Content/Domains	Method of administration	Recall period	Response format	Range of scores	Score interpretation	Cross-cultural validation
PSQI	19	Sleep quality, sleep disturbances	Written	1 month	Free entry & Likert	0 to 21	Higher scores indicate worse sleep quality; Score ≥5 identifies “poor sleepers”	Available in 56 different languages
ESS	8	Daytime sleepiness	Written	Recent times	Likert	0 to 24	Higher scores indicate greater daytime sleepiness; Score ≥11 indicates EDS	Available in at least 11 different languages
PROMIS-SD & PROMIS-SRI	Variable (4 to 27)	Sleep quality, sleepiness, impaired alertness	Written, online, or CAT	7 days	Likert	0 to 100	Higher scores indicate worse sleep; mean population T-score = 50, and standard deviation of 10	Available in at least 5 different languages
ISI	7	Insomnia, insomnia-related distress	Written	2 weeks	Likert	0 to 28	Higher scores indicate greater insomnia severity; suggested but not validated guidelines‖	Available in 8 different languages
AIS	8	Insomnia, insomnia-related impairment	Written	1 month	Likert	0 to 24	Higher scores indicate greater insomnia severity; different cutoffs ranging 6–10 have been proposed for identifying insomnia	Available in 7 different languages
BQ	10	Snoring, daytime sleepiness, hypertension	Written	Not specified	Free entry & Likert	0 to 3	Positivity in 2 to 3 categories indicates high risk for OSA	Available in 10 different languages

Table 2:

Psychometrics

Measure	Floor, ceiling effects	Reliability	Validity	Responsiveness	Minimally important differences	Generalizability	Used in RCTs
PSQI	No	α = 0.67–0.83† Test-retest ICC = 0.90 after 2 days	Concurrent validity with ISI, sleep efficiency from sleep diary.	Responsive to change in clinical trials	Not reported	Used in multiple different populations; may not detect aging related sleep disturbances	Yes
ESS	Marginal floor, no ceiling	α = 0.74–0.88† Test-retest r = 0.82‡ after 5 months	Concurrent validity with MSLT and MWT	Responsiveness to change in clinical trials	Not formally established, Proposed MID 2–3	Used in multiple different populations; may underestimate in female and older individuals.	Yes
PROMIS-SD & PROMIS-SRI	No	α = 0.96† Test-retest ICC = 0.83 and 0.73, respectively, after 2 days	Concurrent validity with PSQI and ESS	Responsive to change in clinical trials	Not formally established, Proposed MID 2–4 in SLE	Used in multiple different populations	Yes
ISI	No	α = 0.74–0.91† Test-retest r = 0.83‡ after 1 month, r = 0.73‡ after 3 months	Concurrent validity with PSQI and sleep diary. Weak correlation with sleep efficiency on polysomnography	Responsive to change in clinical trials	Not formally established, Proposed MID 6–7	Used in multiple different populations	Yes
AIS	Unknown	α = 0.89† Test-retest r = 0.89‡ after 1 week Test-retest ICC = 0.64 after 81 days	Concurrent validity with PSQI, Sleep Problems Scale, Brief Fatigue Inventory, actigraphy	Limited studies available	Unknown	Used in multiple different populations	Limited studies available
BQ	Unknown	α = 0.86–0.92† Test-retest = 0.92§ after 8 days	Concurrent validity with polysomnography	Unknown	Unknown	Used in multiple different populations	Yes

^*PSQI = Pittsburgh Sleep Quality Index; ESS = Epworth Sleepiness Scale; EDS = excessive daytime sleepiness; MSLT = Multiple sleep latency test; MWT = maintenance of wakefulness test; MID = minimally important differences; PROMIS-SD & -SRI = Patient Reported Outcomes Information System -Sleep Disturbance and Sleep Related Impairment; CAT = computer adaptive test; ISI = Insomnia Severity Index; AIS = Athens Insomnia Scale; BQ = Berlin Questionnaire

^†Cronbach’s alpha

^‡Pearson correlation coefficient

^§Cohen’s kappa

^‖ISI guidelines: 0–7 no insomnia, 8–14 subthreshold insomnia, 15–21 moderate clinical insomnia, 22–28 severe clinical insomnia

Glossary

(PSQI)

Pittsburgh Sleep Quality Index

(ESS)

Epworth Sleepiness Scale

(PROMIS)

Patient Reported Outcomes Information System

(SD)

Sleep Disturbance

(SRI)

Sleep Related Impairment

(ISI)

Insomnia Severity Index

(AIS)

Athens Insomnia Scale

(BQ)

Berlin Questionnaire