Components of Pittsburgh Sleep Quality Index in Iranian adult population: an item response theory model

Abstract

Objective/background

There have been conducted few studies in Iran on the quality of sleep in the general population. This study aimed to use the item response theory (IRT) model to examine the accuracy of the seven components of the Pittsburgh Sleep Quality Index (PSQI) and to provide an appropriate cut-off point for population-based studies.

Methods

This study was performed using the data of the second phase of the Shahroud Eye Cohort Study (ShECS) in 2014. The sleep quality of 4710 participants was measured through PSQI. Using an IRT model, the seven components of the index are considered as indicators and sleep quality as the latent variable in the measurement model. This model supposed that there is only one hidden component to explain the respondent's behavior to a number of items.

Results

Results of analyzing different components of PSQI showed that component 6 (using sleep medication) and 7 (daytime dysfunction disorder) had the lowest values of discrimination parameter and component 4 (habitual sleep efficiency) and 1 (sleep quality) had the highest value of discrimination parameter. Persons with an expected sleep quality score of less than or equal to 6.5 will be defined as good sleep quality pattern.

Conclusions

Since discrimination values for components 6 and 7 are less than the values for other components, the use of the standardized latent scores is emphasized for assessing the quality of sleep in the population.

Highlights

• •For seven components of PSQI, difficulty and discrimination parameters were estimated using IRT.
• •Using an IRT model persons with an expected sleep quality score ≤ 6.5 will be defined as good sleep quality pattern.
• •Almost 42.9% and 0.2% had poor and very poor sleep quality.
• •Sleep quality, and habitual sleep efficiency components had the highest value of discrimination parameter.
• •Component 6 and 7 play the lowest role in assessing sleep quality in the general population.

1. Introduction

Sleep is the best form of rest and revitalization, and the quality of sleep and its related problems are among the factors which can impact health [1]. Extensive research has shown that sleep disorders are associated with various illnesses including depression and anxiety [1,2], physical problems [3], congestive heart failure [4,5], unwanted injuries [6], and decreased quality of life [7]. Since sleep quality is a multidimensional concept that includes satisfaction with sleep, sleep adequacy and sleep effect on daily functioning, self-administered questionnaires are useful for assessing sleep quality in the population of patients and the general population [8]. Sleep quality can also be related to the duration of sleep, sleep latency and frequent waking throughout the night [9]. Poor sleep quality can reduce cognitive functioning [10] and it also may be due to a cognitive appraisal disorder [11]. Objective sleep quality assessment using standard overnight polysomnography (PSG) or electroencephalographic spectral component analysis are often impractical for large population-based studies [12]. Subjective sleep quality assessing can help investigators to identify persons at high risk for insomnia because insomnia identity is highly related to sleep pattern and sleep dissatisfaction [11,13]. However, insomnia can occurs independently of poor sleep [11].

According to the National Sleep Foundation in 2014, about 12% of adults reported poor sleep quality and 23% reported it as only fair [14]. In a study conducted in Germany, about 36% of the population aged 18–80 years had poor sleep quality [12]. Studies using Pittsburgh Sleep Quality Index (PSQI) indicated that in Austria, 32% [15] and in Hong Kong, 39% [16], and in Iran, 37% [17] of the general population believed they had poor sleep quality.

There have been few studies in Iran on the quality of sleep in the general population. Most of the studies conducted in Iran have used Pittsburgh Sleep Quality Index (PSQI) with specific groups such as students [18] or different kinds of patients, such as patients with breast cancer [19], patients with chronic obstructive pulmonary disease [20], or diabetic patients [21]. Farrahi Moghadam and colleagues [22] conducted a study to determine the validity and reliability of the Persian version of PSQI in two groups of patients with psychological disorders and healthy people. In their study, comparing the sleep quality scores with the scores on the 12-item General Health Questionnaire (GHQ-12) and the clinical examination based on Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV), they concluded that the cutoff point of 6 and above (≥ 6) had the highest sensitivity (93.6%) and specificity (72.2%) for differentiation between insomniac and healthy people. The cut-off point presented in their study is more than the cut-off point in the PSQI manual (Total score of 5) [9]. This difference in cut-off point has also been highlighted in other studies. For example, a validation study of PSQI in Korea reported that the cut-off point of 8.5 had the most sensitivity and specificity [23]. The finding of a validation study showed PSQI is a single-factor scoring structure in an adolescent sample and further validation studies are required to determine an appropriate PSQI clinical cut-off score for adolescents [24]. In a systematic review on the dimensionality of PSQI, the authors concluded further validation studies needed for the various PSQI factor structures for standard sleep assessment in research [25].

The item response model (IRT) is one of the most appropriate and commonly used methods to examine the psychological characteristics of new and existing questionnaires and to revise items in a questionnaire. In this model, based on some related items, a latent variable is constructed and then, based on this latent variable, items are calibrated. The accuracy of estimating the model with this method depends on meeting the assumptions of the model [26].

Despite the frequent use of the Pittsburgh Sleep Quality Index (PSQI) in studies conducted in Iran, only one study with 133 healthy subjects and 125 patients with psychological problems examined psychological characteristics, validity and reliability of this questionnaire [22]. Moreover, most of these studies have been conducted in specific groups. This study aimed to use the item response theory (IRT) model to investigate the accuracy of the seven components of the PSQI and to provide an appropriate cut-off point for population-based studies.

2. Method

This study was performed using the data of the second phase of the Shahroud Eye Cohort Study (ShECS). Details of the methodology of that study have already been published [27]. In ShECS, first, in 2009, using the cluster sampling method, 300 clusters were randomly selected from 9 strata in Shahroud (northeast of Iran). Health care centers were considered as strata. From each cluster, at least 20 individuals aged 40–64 years were selected to participate in the study, and after being informed of the objectives of the study, all individuals were invited for a full ophthalmic examination. A total of 5190 patients were examined in the first phase of the study. In the second phase of the study, a total of 4737 participants from the first phase were re-examined and questioned in 2014 (response rate of 91.3%). The protocols of both phases of the study were reviewed and approved by the Ethics Committee of Shahroud University of Medical Sciences. Before interview and clinical examinations, a written informed consent was obtained from each participant.

In this study, sleep quality was measured through Pittsburgh Sleep Quality Index (PSQI) [9,22]. This self-report questionnaire has been designed for elderly people and has 19 items which form seven components including 1- Sleep quality, 2- Sleep latency, 3- Sleep duration, 4- Habitual sleep efficiency, 5- Sleep disturbances, 6- Use of sleep medications and 7- daytime dysfunction disorder over the recent one month. Each of these components is measured by one or more separate items on different scales and are scored based on the instruction given for the seven components, where each component receives a score between 0 and 3, and the overall score obtained from this index can range from 0 to 21. Based on the manual of the index, a score of 5 and above is defined as poor quality of sleep. The Cronbach's alpha coefficient for the components of the questionnaire was estimated to be 0.83, indicating the high internal consistency of the PSQI [9]. The Persian version of this index was validated with 123 patients with psychological problems and 133 healthy people. Internal consistency for components of the Persian version was acceptable (Cronbach's α = 0.77) [22].

3. Item response theory (IRT)

In this method of analysis, the relationship between a latent variable (ie sleep quality) with the responses provided to the items of the sleep quality questionnaire (components of the Pittsburgh Sleep Quality Index) is investigated. In IRT, the probability of a correct response to an item by an individual is specially assessed by the values of the latent variable (that is Theta) and the characteristics of the item. Theta value for each person is called the person location [28]. The two-parameter graded response model of IRT introduced by Samejima [29] was used in this study. The assumptions of this model are:

• 1There is only one hidden attribute to explain the respondent's behavior to some items. In other words, it is assumed that there is a unidimensional latent space. Theoretically, it is assumed that the PSQI is designed and validated to measure one dimension of sleep quality. The difference in responses to an item by people in two groups with the same latent variable can pose problems to this assumption. This is known as the differential item functioning (DIF) [26].
• 2.The response of a person to an item is independent of his response to other items when we condition the ability (latent variable) which is called conditional independence.

In this model, it is assumed that responses to polytomous items are a logistical function of the latent variable so that the probability of answering the higher levels of the item is associated with increasing the level of the latent variable. The value of the latent variable is represented on a z-score scale which ranges from −3 to 3 with a mean of zero and a standard deviation of 1. For each question or item, two parameters are estimated:

• 1Difficulty parameter of the item represented as β which is the item location over the latent variable or the ability axis (in the present study, the quality of sleep is the latent variable). Item difficulty represents the location of an item on the ability axis (or the latent variable) in which the probability of changing the response option or a correct response is fifty percent (θ = β). This means that the probability of a correct or incorrect response to the item is equal to the ability level. Easier items are more likely to be answered correctly at each level of the latent variable compared to the more difficult items. Values between −3 and 3 are acceptable for the item difficulty [30].
• 2Discrimination parameter: the ability of the item to differentiate between people with low and high abilities in terms of the latent variable. The discrimination parameter indicates the rate of change in the probability of success in the closest ability (latent variable) to the difficulty parameter. A high level of discrimination value of an item indicates a high correlation between the latent attribute and the probability of success in answering that item. Therefore, items with a higher discrimination value can better distinguish between the lower and higher values of the latent traits [31,32]. Based on Baker's model, the discrimination value (alpha parameter) between 0.65 and 1.35 are average and the discrimination values between 1.35 and 1.69 are high and discrimination values higher than 1.70 are interpreted as very high values. In the IRT model, the probability of an item being responded correctly based on the value of the latent variable is displayed on the item characteristics curves (ICC). The reliability or accuracy of the measurement in the IRT model is a continuous function in terms of the latent variable known as item information function (IIF) and test information function (TIF). Standard error of measurement in terms of the latent variable is estimated as the reciprocal value of the test information function square root. The test characteristic curve (TCV) indicates the quality of sleep scores that we can expect from individuals with different levels of latent quality of sleep [26].

The data were analyzed using STATA-14 software and the effect of cluster sampling was taken into account.

4. Results

The sleep quality data were acquired for 4710 people out of 4737 persons who had participated in the second phase of ShECS, and so these people were investigated. The mean age of participants was 55.9 (SD = 6.2) years. Among the participants, 2777 (59.0%) were women and the rest were men. The main characteristics of participants in the second phase of study are shown in Table 1. The average sleep quality score was 6.9 ± 4.0 (sleep quality score ranged from 0 to 20). The mean and standard deviation of each component of PSQI are shown in Table 2.

Table 1.

Demographic characteristics of participants of the study.

Variables	Frequency (%)
Sex
Male	1933 (41.0)
Female	2777 (59.0)
Education
Illiterate	526 (11.2)
Primary	1489 (31.6)
Guidance	712 (15.1)
High school	1455 (30.9)
Collage	522 (11.2)
Marital status
single	47 (1.0)
married	4218 (89.5)
Widow/divorced	448 (9.5)

Table 2.

Means with standard deviation (SD), difficulty (threshold) and discrimination parameters with standard errors (SE), for each of the 7 components of Pittsburg Sleep Quality Index.

component		Mean (SD)	Difficulty (SE)			Discrimination (SE)
			1	2	3
1	Subjective Sleep quality	0.84 (0.12)	−0.38 (0.03)	1.23 (0.04)	2.27 (0.08)	1.80 (0.09)
2	Sleep latency	1.33 (0.02)	−0.69 (0.03)	0.28 (0.03)	0.99 (0.04)	1.56 (0.06)
3	Sleep duration	1.45 (0.01)	−1.59 (0.05)	0.31 (0.03)	1.31 (0.06)	1.43 (0.07)
4	Habitual sleep efficiency	0.99 (0.02)	−0.10 (0.03)	0.69 (0.04)	1.22 (0.05)	1.94 (0.11)
5	Sleep disturbance	1.05 (0.01)	−2.46 (0.10)	1.99 (0.08)	5.14 (0.28)	1.16 (0.06)
6	Use of sleeping medications	0.41 (0.01)	2.43 (0.16)	2.92 (0.20)	3.40 (0.23)	0.70 (0.05)
7	Daytime dysfunction disorder	0.81 (0.01)	−0.20 (0.05)	2.02 (0.13)	4.48 (0.29)	0.67 (0.05)

The results of the IRT analysis of the seven components of PSQI and parameters of difficulty (thresholds) and discrimination parameters are also displayed in Table 2. Results of analyzing different components of PSQI showed that components 6 (using sleep medication) and 7 (daytime dysfunction disorder) had the lowest values of discrimination parameter and components 4 (habitual sleep efficiency) and 1 (sleep quality) had the highest value of discrimination parameter. In other words, component 4, followed by component 1, was best able to discriminate people with low sleep quality from healthy people, while components 6 and 7 had the lowest ability to make such discrimination.

The probability curves for choosing the Kth-category of response known as category characteristic curves (CCC) for components four and seven are displayed in Fig. 1(A) and (B) respectively. In this plot, the intersection of the two adjacent categories is the point of transition from one response category to the other category. Therefore, according to Fig. 1-A, people with a low score of sleep quality, ie, $\theta \le$0.2, are more likely to select the zero score for component four (habitual sleep efficiency). As for component seven (day time dysfunction disorder) of PSQI, the results of Fig. 1-B showed that individuals with a level below 0.7 for latent sleep quality were most likely to choose zero score. Individuals with a sleep quality level between 0.7 and 1.8 were most likely to respond category 1 and people with a sleep quality level between 1.9 and 2.9 were most likely to respond category 2 and people with a sleep quality level higher than 2.9 were most likely to respond category 3.

Fig. 1.

Category characteristic curves for components of habitual sleep efficiency (A) and daytime dysfunction disorder (B) in Pittsburgh Sleep Quality Index: The intersection of the two adjacent categories is the point of transition from one response category to the other category.

Fig. 2, Fig. 3, which are item information function (IIF), and test information function (TIF), represent the reliability or precision of each item or the overall test. The results showed that components 1 to 4 had the highest precision and discrimination estimates (Table 1), and provide more information for both the low and high sleep quality scores, and components 6 and 7 had high difficulty values and low discrimination estimates (Table 2) which is due to the nature of these variables and therefore they are expected to have higher discrimination values and provide more information. Using sleep medication and daytime dysfunction disorder is more common in people with sleep disturbances and we expect the difficulty of these items to be high and we expect people with low sleep quality scores (good sleep pattern) to provide little information on these two items. Also, the results of Fig. 2 showed that components three and five are more accurate in low values of latent sleep quality spectrum.

Fig. 2.

Item information curves for all seven components of Pittsburgh Sleep Quality Index: The reliability or precision of each item.

Fig. 3.

Test information curves for Pittsburgh Sleep Quality Index: The reliability or precision of the overall test.

Fig. 3 showed that PSQI provides the most information for the latent sleep quality spectrum of 1.5–2.5. Moreover, comparison of Fig. 2, Fig. 3 showed that components 1 to 5 of PSQI tend to provide accurate information for both low and high latent sleep quality scores, but components six and seven do not have good accuracy for all levels of latent sleep quality scores.

Fig. 4, which is the test characteristics curve (TCC), indicates the expected test scores in a responder with different levels of the latent sleeping quality. We expect that above-average individuals ($\theta =0)$ to have a test score of sleep quality of 6.55 on the scale defined for the PSQI out of a possible score of 21. We also expect people with latent sleep quality between −1.96 and + 1.96 to have an expected score between 1.7 and 14.2 on the scale defined for the PSQI. Therefore, it can be concluded that 95% of the participants had an expected score between these two scores. Accordingly, the score of sleep quality calculated for the participants can be divided into three groups.

• 1Good Sleep Quality Score: People with latent sleep quality (theta) less than zero. Therefore, they are defined with an expected sleep quality score of less than or equal to 6.5.
• 2Poor Sleep Quality: People with sleeping quality (theta) between zero and 1.96. Therefore, these individuals are defined with an expected sleep quality score ranging from 6.5 to 14.2.
• 3Very Poor Sleep Quality: People with sleeping quality (theta) greater than 1.96. Therefore, they are defined by an expected sleep quality score higher than 14.2.

Fig. 4.

Test characteristics curve of Pittsburgh Sleep Quality Index which indicates expected test scores in a responder with different levels of the latent sleeping quality.

According to the classification presented in this article, 2612 participants (55.1%) had good sleep quality, while 2033 people (42.9%) and 92 participants (0.2%) had poor and very poor sleep quality, respectively. Also, we categorized individuals on the Pittsburgh Index, in which the cutoff point of higher than 5 is considered as poor quality of sleep. According to this categorization, the prevalence of poor sleep quality was 66.6% (3136 people). The standardized z-score calculation for the scores of the sleep quality using the sum scores on observed seven components of the index, and its correlation with the standard scores of latent sleep quality by the IRT model showed that there was a highly significant correlation between the two scores (r = 0.96, p < 0.001). Changes in latent scores based on the standardized scores derived from the sum of the seven components of the sleep quality index were computed based on the PSQI and are displayed in Fig. 5.

Fig. 5.

Distribution of latent standardized scores in terms of the standardized total score of Pittsburgh Sleep Quality Index.

5. Discussion

The mean total score of sleep quality in the population under study was 6.9 ± 4.03 which is significantly greater than the mean scores obtained in other studies such as studies in Germany (age range of 18–80 years where mean = 5.0 ± 3.37) [12], Hong Kong (Mean = 5.3) [16] and Iran (age group older than 18 years where mean = 5.06, 0.95 CI: 4.9–5.1) [17]. This difference can be due to differences in the mean age of the studied population, and their environmental and occupational conditions, and their background diseases.

The total sleep quality score is generally calculated by summing the scores of the different components of the PSQI. This score can be expressed as either percentile or as a standardized z score. In this method, the weight of each component of the quality of sleep is considered equal and is called the raw calculation method. Unlike the total raw score calculation method, in the item response model (IRT), the seven components of the questionnaire are considered indicators and sleep quality as the latent variable in the measurement model. The scores obtained from this model are presented as standard z scores. Cho and et al. [33], examined the validity of the Cambridge Face Memory Test through the IRT method. The results showed that the standardized values using the raw method (total scores) and the IRT method (latent standard scores) were highly correlated. However, in the present study, despite the correlation between the two values, the standardized scores change greatly based on the standardized total scores (Fig. 5), and the raw scores have less value than latent values. This occurs more in less reliable and multi-dimensional scales [33]. Regarding the changes of standardized latent scores with standardized total scores (Fig. 5), some points are noteworthy. Standardized latent scores can classify individuals with a raw score in different groups, which is due to the fact that in the method of calculating raw scores, each component has equal shares in the calculation of the total score. The IRT method is based on the response pattern to the items. In this method, people with a correct response to items with higher coefficients of discrimination receive a higher score than those with a correct response to items with lower discrimination coefficients [33]. Since discrimination coefficients for components 6 and 7 are less than the coefficients for other components, the use of the standardized latent scores is emphasized over the total raw scores for assessing the quality of sleep [33]. Since IRT has not been used to study Pittsburgh Sleep Quality Index (PSQI) components in a population, it is not possible to compare the difficulty and discrimination indices of these components with other studies.

The main characteristics of the latent standard scores of sleep quality based on the IRT model compared to the standardized total score are that: 1- In the latent standardized method, the individual standard error is calculated but in the general method only the mean of the standard error for the total score is estimated. 2- The measurement scale in the latent standardized score is interval, but in the raw total score it is an ordinal scale. 3- Item characteristics of difficulty and discrimination are sample independent, that is they are invariant and remain unchanged when different people take the test. Therefore, the item characteristics coefficients in the present study (Table 2) can be used to calculate latent sleep quality in smaller studies without implementing the IRT model [33,34].

The results of PSQI analysis revealed that components including sleep quality, sleep latency, sleep duration, habitual sleep efficiency and Sleep disturbance have a good ability to discriminate poor sleepers. Some of the above components are indirectly answered by the participants and are the result of combining the answers to two or more items with a score of 0–3. Comparison of the mean scores obtained from each component of sleep quality (Table 1) showed that the mean score in component 6 (use of sleep medication) has the lowest mean score among these components, and in other studies, similar results have been reported too [12,17,[35], [36], [37]]. The discrimination value of component 6 (use of sleep medication) also has the lowest value. This component, due to the nature of sleep disturbances, requires treatment in individuals, and so in people without sleep disorders, the score is usually zero.

Different studies have reported different cut-off points for PSQI questionnaire. In the study done in Korean [23], 8.5 and in the study by Backhaus and et al. [13] and Farrahi Moghaddam [22], the cut-off point of 6 had the highest sensitivity and specificity. In the present study, the expected cut-off point was equal to 6.5, which is different from the one in the manual of PSQI (a score greater than 5).

According to the IRT standardized scores cut-off, 43% of the general population had poor sleep quality. In comparison with other studies in Germany (36%) [12], Austria, 32% [15] and in Hong Kong, 39% [16], our estimation is higher. This difference can be due to differences in the demographic, physical health and pshychological characteristics of studied populations. However, using this method of analysis which is accompanish with considering a higher cut-off, may be lead to a lower estimation. Mental well-being and physical complaints may lead to sleep problems, but poor sleep quality can also cause reduced mental and physical health [12]. In fact, using a cut-off point alone may not be appropriate to identify individuals with sleep complains in the population. Therefore, in addition to examining the scores of the seven components of the PQSI, the use of other evaluation tools (ie, Insomnia Severity Index, sleep dissatisfaction or apnea index) to assess sleep disorders is recommended [11]. Psychological interventions such as cognitive behavioral therapy can be suitable for individuals with poor sleep quality [38,39].

The present study is the first population-based study that uses item response theory (IRT) to examine the item characteristics of PSQI in a general population. In addition to the analysis method used, this study is the largest population survey on sleep quality in Iran, in which characteristics of components of the Pittsburg Sleep Quality Index (PQSI) have been investigated and can be the criterion for future studies in Iran. Also, the large size of the selected sample can be effective in distributing different types of responses to items with different degrees of difficulty and can increase the accuracy in estimating the study indices. The data of the present study are based on electronic records and are collected by daily monitoring and enjoy good accuracy and quality. Focusing on only the urban population is one of the limitations of the present study. Regarding the measurement of the difficulty and discrimination of the various components of this questionnaire, it can be concluded that the use of component 6 on the use of sleep medication and component 7 in the form of raw scoring with similar weights index, play the lowest role in assessing sleep quality in the general population, compared to the other component of PSQI. Althought this index fails to assess the clinical value of the measure, it is proper for population surveys. For screening of the sleep disorders in the population, considering the seven component scores of the PQSI in addition to using other sleep questionnaire such as Insomnia Severity Index, Apnea Index, sleep diaries and Epworth Sleepiness Scale is recommended.

Author contributions

AK, drafted the manuscript and contributed in preparation of the study protocol and conceptualized and conducted all statistical analyses and was the primary author of the article. MHE, HH and AF contributed in the conceptualization of the paper and the statistical analyses and critically revised the manuscript.

Conflict of interest

The following is the supplementary data to this article:

Multimedia component 1