Psychometric Analysis of the Pittsburgh Insomnia Rating Scale among University Population of Poor Sleepers in India

Abstract

Background:

Pittsburgh insomnia rating scale is a 65 item self administered open source questionnaire. The scale is widely used in clinical practice but its psychometric properties are not well established. Therefore keeping in mind this lacuna the current study was designed for university population of poor sleepers in India.

Aims:

The purpose of this study was to establish the Pittsburgh sleep Quality Index test- retest reliability, validity and internal consistency of Pittsburgh insomnia rating scale.

Materials and Methods:

Twenty five subjects were randomly chosen from the screened population of poor sleepers. Pittsburgh insomnia rating scale, Pittsburgh sleep quality index and Insomnia severity index were administered on test day. Retest was administered after one week.

Results:

Eight males and seventeen females with mean age 24 + 7.04 were recruited. The test retest reliability for Pittsburgh insomnia rating scale total score showed excellent reliability (ICC_2,1-0.93). The results also show that the total score is moderately correlated with Pittsburgh sleep Quality Index (r-0.31) and moderately correlated with Insomnia severity index (r-0.49). Internal consistency for the test was excellent (Cronbach's alpha- 0.930)

Conclusion:

The study findings suggest that Pittsburgh insomnia rating scale has excellent internal consistency, test-retest reliability and good validity for university population of poor sleepers in India. It is an important first line of assessment scale for screening of sleep problems.

Introduction

Almost everyone can recall more than one night of poor sleep in the night and its resultant effect on the next day or a full night's non-restorative sleep. Symptoms like these have become a daily experience because of the stress full lifestyle. Subjective assessment of sleep is a very important construct for a number of researchers and clinicians reporting questionnaires that can be evaluated using self reporting questionnaires. Whenever a scale is being used in a new population it must be validated and its reliability must be tested for the new population. This assessment though is subjective; many of its components are objective. Insomnia is very common complaint that one encounters in general practice. Depending on the severity, its prevalence ranges from 9 to 27%.[1,2] Its clinical features include inadequate quality or quantity of sleep which is further associated with negative impact on health and function.[3] It carries a huge burden on the economy in terms of cost of management, higher chances of depression and functional limitations.[4,5,6,7,8,9,10] Despite of all these problems, insomnia still goes highly untreated or managed by over-the-counter medications.[2] Early and correct diagnosis is a key to reducing the health costs. For this purpose easy, reliable and valid tools are required. The most important symptoms of insomnia can be listed as:

1. Difficulty in initiation of sleep

2. Difficulty in sleep maintenance

3. Falling back to sleep after nocturnal awakenings

4. Spontaneous early awakenings.

Hence, clinical assessment is considered to be the gold standard in diagnosing insomnia.[11,12] But this evaluation is both time and energy consuming. Hence, screening tools like questionnaires have an important place in the management. These tools not only help in diagnosis but also in reassessment of the patients post intervention. These values are easy for the patient to understand hence, they become an important tool in building the confidence of the patient. There are a few tools available like Insomnia severity index (ISI),[13] Pittsburgh sleep quality index (PSQI),[14] Athens insomnia scale,[15] Oviedo sleep questionnaire[16] etc. These tests have different recall periods, types and number of items, response scales and scoring and interpretation methods. But because they are subjective scales they measure patient's response to various questions that are closely related to the construct of insomnia. Each of these scales has their own advantages and disadvantages.[17,18,19,20]

The pittsburgh insomnia rating scale(PIRS)

PIRS is a widely used instrument in clinical and research practice. It is a scale with 65-items. It was designed to rate the severity of insomnia in clinical trials, clinical practice etc. Subjects score the items that have three broad sections. Initial is the subjective distress score (46 items), then subjective sleep parameters (10 items) and last is the quality-of-life (9 items). The items have to be scored according to the last week.[21] This scale is still under development, but preliminary data that was published in the form of an abstract of a poster indicated that the PIRS had good test-retest reliability as a measure of insomnia severity in the past week. It appeared to have good concurrent validity with the PSQI.[21] It does not have a ceiling or floor effect for measuring the severity of insomnia.[21] A number of studies have been conducted using PIRS as assessment tool[21,22,23,24,25,26,27] Some researchers have also suggested that it measures non restorative sleep.[28,29] There are 4 items which correspond to non-restorative sleep. But this scale was not developed for this as the primary function and the scoring of the scale does not lead to a specific domain of this either. Details of studies which have used PIRS have been mentioned in Table 1. Section A of the scale have a 10 cm line to mark the quality of sleep in the past week. This answer is not used in the scoring. Section B has 46 questions which have to be answered on the likert scale from 0-3 (0 = not at all bothered, 1 = slightly bothered, 2 = moderately bothered, 3 severely bothered). It is scored by adding all the answers. This is the distress score. Section C has 10 questions which have to be answered on the likert scale 0-3 with variable answers depending on the question. Score of this section is the addition of all and is termed as sleep parameters score. Section D has 9 questions which have to be answered on the likert scale from 0-3 (0 = excellent, 1 = good, 2 = fair, 3 = poor). Addition of all the answers gives the final score which is termed as Quality of life score. Section E is about comments which the patient wants to put in but it is not included in the scoring.. Final score is the grand total of all the three components. Minimum score is 0 (good) and maximum is 195 (bad).

Table 1.

Studies which have used the full version of PIRS

Insomnia severity index(ISI)

The ISI is a valid and reliable screening tool to assess and diagnose the severity of insomnia of the patients.[13] The scale is based on the Diagnostic and Statistical Manual of Mental Disorders (4th edition). It is available in several languages and is increasingly used as a metric of treatment response in clinical research. It consists of 7 questions concerning sleep onset, sleep maintenance, early awakening, level of satisfaction with sleep pattern, extent of interference with daily functioning, conspicuousness of impairment caused by sleep problem, and level of concern about current sleep problem. Each item is marked on a 5-point Likert scale (0 to 4). Total scores after evaluation range from 0 to 28; higher the score more severe is the insomnia. Scores 0 to 7 indicate no clinically significant insomnia, 8 to 14 sub-threshold insomnia, 15 to 21 clinically significant insomnia (moderate), and 22 to 28 clinically significant insomnia (severe). The cutoff score of 14 has optimal sensitivity (94%) and specificity (94%).[30] The ISI has been validated against both objective as well as subjective measures for reliability and validity. It has acceptable internal consistency and concurrent validity.[30,31]

Pittsburgh sleep quality index (PSQI)

The PSQI is one of the most commonly used scale in sleep research. It was originally designed by the authors for use in clinical populations as a simple and valid assessment of both sleep quality and disturbance that might affect sleep quality. According to the scale's authors, advantages of the PSQI include abilities to: (1) make out the patterns of sleep dysfunction over a stipulated period of one month by conducting an assessment of qualitative as well as quantitative data (2) calculate a simple, global score which is able to convey both the number and severity of sleep problems which can be used for research as well as clinical practice.[14] The PSQI consists of 19 items which produce a global sleep quality score and component scores. These components are sleep quality, sleep latency, sleep duration, habitual sleep efficiency, sleep disturbance, use of sleeping medications, and daytime dysfunction. These Items and component scores were designed based on the standard areas which are assessed by clinicians during routine examination of sleep problems. PSQI items use varying response categories that include recording usual bed time, usual wake time, number of actual hours slept, and number of minutes to fall asleep, as well as forced-choice Likert-type responses. A score > 5 suggests poor sleep quality. The psychometric properties of the PSQI have been documented in multiple studies[32] including one with a French-Canadian sample[33] Psychometric properties of the PSQI were supported using data collected from 52 healthy subjects and 96 individuals with sleep problems.[14]

The psychometric studies conducted on PIRS were not sufficient but the usage of this scale is increasing with time. Moreover; there have been no reported studies on the use of this scale on the Indian population. To fill up this knowledge gap this study was done with the aim:

1. To establish the test-retest reliability of PIRS

2. To establish the validity of PIRS

3. To establish the internal consistency of PIRS.

Materials and Methods

Study sample

The study was carried out at Jamia Millia Islamia, New Delhi, India and was approved by the institutional ethics committee prior to the commencement of the study. Subjects (n = 200) were chosen from the university population by convenient sampling to select poor sleepers for a larger study. All the subjects were aged between 18-40 years. All the subjects completed the PSQI scale in English which were scored according to the standard procedure being widely used.[14] The subjects scoring 5 or more were recognized as poor sleepers. Twenty five poor sleepers were randomly recruited from this by sealed envelope method. Exclusion criteria from the study were diagnosed depression, anxiety or any other psychiatric, surgical, medical or orthopaedic condition. Both the genders were included in the study.

Procedure

All the subjects gave their written informed consent to participate in the study. The subjects were administered PIRS, ISI and PSQI on day 1 independently. All the scales administered were in English. All questionnaires were self administered. PIRS was repeated after 1 week to examine the test -retest reliability. The scales were scored based on their respective procedures.[13,14,21]

Statistical Analysis

Sample size calculation

A power calculation was done using F test with a significance level F <0.05. The tests results revealed that 25 subjects with two observations per subject achieves 90% power to detect an interclass correlation of 0.90 under the alternative hypothesis when the interclass correlation under the null hypothesis is 0.70.[34]

Statistical analysis was carried out using SPSS version 21 for Microsoft windows. Mean and standard deviation for age and BMI was calculated for all the subjects. Frequency was calculated for the gender for all the subjects. A Kolmogorov-Smirnov test was conducted which indicated that the data were normally distributed hence parametric testing was done. Systematic error (the mean of differences of scores of the test and retest) was calculated by paired t-test with a significance level of 0.05. Scatter plot was given for the total PIRS score.

Test-retest reliability of PIRS was evaluated by intra-class correlation coefficient (ICC) at an alpha level which was fixed at 0.05. The aim was to generalize these findings of PIRS to similarly trained clinicians who use this scale therefore ICC model 2, 1 was used. Wier JP (2005) reported that ICCs values above 0.75 were considered to represent good, 0.40 to 0.75 moderate and <0.40 poor reliability.[35] ICC_2,1 was calculated for the PIRS total score as well as PIRS component scores. Standard error of measurement (SEM) was calculated (SEM = SD *√ 1-r_test-retest).[36] It is a measure of typical error that is associated with the measurement. It was calculated for the PIRS total score as well as PIRS component scores. Graphical representation of test retest reliability was done by Bland-Altman plots.[37] They were obtained for the PIRS total score as well as PIRS component scores by plotting the difference between the retest and test score versus the mean of the retest and initial test scores.[37] Internal consistency of the scale was calculated by Cronbach's alpha coefficient which was calculated for the PIRS total score Cronbach's alpha reliability coefficient normally ranges between 0 and 1. However, there is truly no lower limit to the coefficient. The closer its value is to 1.0 the greater is the internal consistency of the items in the scale George and Mallery provided the following rules of thumb for the interpretation of the value of the Cronbach's alpha coefficient “ >0. 9 – Excellent, >0. 8 – Good, >0. 7 – Acceptable> 0.6 – Questionable >0. 5 – Poor, and <0.5 – Unacceptable.[38] While increasing the value of alpha is partially dependent upon the number of items in the scale, it ought to be noted that this has decreasing returns.

The concurrent validity is the correlation in between two scales at the same point of time. It was calculated for PIRS by two tailed Pearson's coefficient. The evaluation was done by correlating (a) the total score of PIRS with the total score of ISI (b) the total score of PIRS with the total score of PSQI.

Results

Descriptive statistics

Twenty five subjects (8 males and 17 females) with mean age being 25.24 + 7.04 and BMI as 21.17 + 3.73 were recruited. All the subjects completed the retest for all the scales. There were no missing data at the baseline or the retest. An independent-samples t-test was conducted to compare test and retest values for Total PIRS score, distress score, sleep parameters score and quality of life scores. There was no significant difference in the all the scores other then sleep parameters score. The details have been given in Table 2 for Total PIRS score test (68.36 + 20.2) and retest (68.50 + 23.6) conditions. Specifically, our results suggest that there was no significant difference in the baseline data of the two groups

Table 2.

Comparison of test-retest mean scores of PIRS and its components

Test retest reliability

The retest reliability (ICC_{2, 1}) with 95% CI, SEM results for the same are given in Table 3. The ICC_2,1 for PIRS total score was 0.93 which indicates excellent reliability. Distress score ICC_2,1 was 0.90 which indicates excellent reliability. Sleep parameters score ICC_2,1 was 0.70 which indicates a good reliability. Quality of life score ICC_2,1 was 0.71 which indicates a good reliability. The Bland-Altman plot provided for PIRS total score is in Figure 1a, distress score in Figure 1b, sleep parameters score in Figure 1c and quality of life score in Figure 1d illustrated that all data points were within the 95% limits of agreement for all and only one out of 25 data points of sleep parameter lies outside.

Table 3.

Test- retest reliability of PIRS total and component scores

Figure 1.

(a) Bland-Altman plot of PIRS mean + SD. PIRS T1 is the test and PIRS T2 is retest value. (b) Bland-Altman plot of Distress score mean + SD.P1 T1 is the test and P1 T2 is retest value. (c) Bland-Altman plot of Sleep parameters mean + SD.P2T1 is the test and P2T2 is retest value. (d) Bland-Altman plot of Quality of life mean + SD.P3T1 is the test and P3T2 is retest value

Concurrent validity

Concurrent validity of PIRS (extent to which the scale correlates with other scale at the same time) was calculated by Pearson's coefficient by correlating the final score of PIRS with PSQI and total score of PIRS with ISI. There is a moderately positive correlation between PSQI and PIRS (r-0.31 P value-0.13). There was also an important indication of a moderately positive correlation between PIRS and ISI (r-0.49 P value-0.012).

Internal consistency

Cronbach's alpha coefficient was calculated for Total PIRS score (α-0.93) which indicates excellent internal consistency. Component score's Cronbach's alpha coefficient are- Distress score (α-0.82), Sleep parameters score (α-0.70) and Quality of life score (α-0.71). All have acceptable internal consistency. To further assess the homogeneity of the scale Pearson correlation between the scores was calculated [Table 4]. PIRS total score to component scores ranged from high to moderately correlated positive relationship(r’s-0.96 to 0.37) correlations between individual items and total score ranged from 0.07 to 0.96. Scatter plot was plotted for the total PIRS score [Figure 2].

Table 4.

Correlation matrix of Pearson's coefficient for PIRS and its component scores

Figure 2.

Scatter plot shows the relation between test-retest Pearson's Coefficient (r = 0.88) and found to be fairly associated. PIRST1 is the test value of the total PIRS score and PIRST2 is the re-test value of the total PIRS score

Discussion

This is preliminary study that reported the psychometric properties of PIRS among university population of poor sleepers in India. The utility of PIRS has been further demonstrated in sleep problems in a number of patient population like type II diabetes,[26] insomnia[23,24,25] and insomnia associated with depression.[27] Even though, it is a widely used scale in research and clinical practice, its psychometric properties are not well established. More over the only study which has reported the psychometric properties of this scale used patients suffering from insomnia whereas; we have included patients who have not reported to a physician for sleep problems. This is a group with no diagnosed sleep problems. The age group that we had worked with is 18 to 40 years in which most of the student populations and young employees of the university were covered. A number of studies have reported poor sleep quality in this population.[39,40] Hence, there is a need to find a robust scale to assess their sleep problems for early identification of sleep disorders.

ICC_2,1 value ranged from 0.70 to 0.93 for the total as well as the component scores of PIRS. ICC_2,1 value of more than 0.75 has been reported to be good to excellent. ICC_2,1 for the total score was reported to be 0.93 which is indicative of excellent reliability. There is a lack of any scientific evidence to support the same as no other study has reported it. But the results of our study justify the use of PIRS in the specified population. The ICC_2,1 for distress score is 0.90 which again points towards high reliability. Sleep parameters component has ICC_2,1 of 0.70 and Quality of life component is 0.71, both indicate high reliability thus making the scale a reliable tool for practice. Similar; values have been obtained Pearson's coefficient too but ICC_2,1 is supposed to be a more robust estimate of association. The standard error of measurement (SEM) estimates how recurrent measures of an individual's score on the same instrument tend to be distributed around his or her “actual” score. This actual score is always an unknown entity. No measure can be created that can provides an actual score. SEM is directly associated with the reliability of a test. As the values of SEM increase reliability of the test decreases. The reported values of SEM here are the lowest for sleep parameters and quality of life scores suggesting that these score demonstrate good absolute reliability.

In our study PIRS and PSQI showed a moderate to low correlation. Moul et al.,[21] reported a Pearson product moment correlation of 0.73. The difference here may be attributed to the specificity of the population. They have used a population of insomniacs whereas we have had a population of poor sleepers. PIRS and ISI have a moderate correlation. This can be explained by the fact that they both are assessing similar construct of insomnia. Thus PIRS seems to have good construct validity.

Although we have achieved our aims, we would like to suggest some future studies to address the certain important issue. Firstly as there is no gold standard tool to measure the condition the available valid and reliable tools were used to establish the concurrent validity. PIRS could have been correlated with polysomnographic (PSG) readings too. Secondly study can be done to establish the sensitivity and specificity of the scale by including some normal individuals and MDC also can be reported in future studies. The results of the study may not be generalized to other age groups or other ethnicities. Thirdly studies should be done on larger population with broader inclusion criteria.

Conclusion

This preliminary study found two repeated Pittsburgh insomnia rating scale scores, obtained by a single clinician during two test session for twenty five Indian university population of poor sleepers, to be highly reliable, and valid. Therefore, it may be considered reliable and valid for the described conditions in this study. As psychometric properties and normative data have been established for PIRS, use of this scale may prove to be of great value to the clinician and the researcher working in the area of sleep.

Competing Interests

The authors have no conflicts of interest to declare in their work. We certify that no party or group having a direct interest in the results of the research supporting this article has or will confer a benefit on us or any organization with which we are associated.

Author's Contribution and Acknowledgement

Each author has contributed to conception and design of the review drafting the article or revising it for important intellectual content and final approval of the version to be published. The authors would like to acknowledge the work and assistance of Dr Zafar Azeem (PT) towards data dissemination, critical appraisal and content development. We would also like to acknowledge the University Grants Commission for the financial assistance in the form of major research grant.