The Pain and Sleep Questionnaire three-item index (PSQ-3): A reliable and valid measure of the impact of pain on sleep in chronic nonmalignant pain of various etiologies

Abstract

BACKGROUND:

Sleep disturbance is among the more common complaints reported by chronic pain patients. Because pain-related sleep disturbance may serve as a marker for the assessment of responses to treatment for chronic pain, inclusion of a measure designed to assess the impact of pain on sleep in clinical trial protocols is important, if not necessary. Measures typically used for this purpose lack scales specifically designed for the assessment of the impact of pain on sleep or are based on a single item. Single-item scales lack reliability and, therefore, validity.

OBJECTIVES:

To investigate the psychometric properties of the five-item Pain and Sleep Questionnaire (PSQ) Index, which is embedded in the eight-item inventory, by applying an accepted methodology using retrospective analyses in controlled clinical trials in which the measure had been administered among patients with chronic nonmalignant pain.

METHODS:

Data were pooled from nine independent, single-site, double-blind, randomized placebo-controlled clinical trials conducted over a period of approximately 10 years, the majority of which were cross-over designs. A cross-validation approach was adopted with exploratory and confirmatory factor analyses conducted to evaluate the underlying structure and dimensionality of the measure. Internal consistency reliability was evaluated using Cronbach’s alpha coefficient. Mean score differences were used to assess the ability of the index to detect important treatment changes. Correlation coefficients were calculated between index scores and scores from other health-related outcome measures to evaluate the criterion validity of the index. Finally, predictive validity was assessed using multiple regression analyses.

RESULTS:

Pooling the data resulted in a sample of 605 patients (65.5% female; mean age 55.7 years). Findings suggested a revised three-item PSQ Index (PSQ-3). The PSQ-3 demonstrated high internal consistency across samples (ranging from 0.82 to 0.93) and was sensitive to detecting meaningful treatment effects within different chronic pain categories. Moderate to strong correlations (r>0.40) between the PSQ-3 and other health-related outcome measures provided preliminary evidence for criterion-related validity. Results of multiple regression analyses demonstrated that the PSQ-3 accounted for between 29% and 40% of the variance in scores from other health-related outcome measures.

CONCLUSIONS:

Results support the scoring of a revised three-item index for the assessment of the impact of pain on sleep. The revised index demonstrated acceptable levels of internal consistency and preliminary support for the structural, criterion-related and predictive validity of the index was achieved.

Sleep disturbance is characterized by difficulties with sleep onset, sleep maintenance and poor sleep quality, and is among the more common complaints reported by chronic pain patients. Chronic pain patients experience more cyclic fluctuations in sleep due to frequent arousals, as well as longer awakenings, shifts between sleep stages and periodic body (ie, limb) movements (1). As a result, understanding the relationship between chronic pain and sleep has been the topic of numerous theoretical and empirical investigations. There is now substantial evidence that a mutually reinforcing relationship exists, whereby pain contributes to sleep disturbance, and sleep disturbance, in turn, contributes to pain (2–9). Furthermore, research has demonstrated that relief from pain is associated with a reduction in self-reported sleep problems and, conversely, alleviation of sleep problems is associated with a reduction in pain symptoms (10). Because pain-related sleep disturbance may serve as a marker for the assessment of responses to treatment for chronic pain (11), inclusion of a measure designed to assess the impact of pain on sleep in clinical trial protocols is important, if not necessary. Surprisingly, however, few studies include such a measure, and researchers have relied instead on instruments that assess sleep problems more broadly defined without reference to the impact of pain on sleep.

A recent literature review (4) explored which of the various self-reported measures of sleep disorders available best captured the types of sleep disturbance important to patients with chronic pain. The Pittsburgh Sleep Quality Index (PSQI) and Medical Outcomes Study (MOS) Sleep Scale were found to be the most frequently used. The PSQI consists of 19 items and measures seven different domains of sleep problems including sleep latency, sleep duration, sleep efficiency, sleep quality, sleep disturbances, medication use and daytime dysfunction (12). The sum of scores for these seven components yields one global score. The MOS Sleep Scale, on the other hand, is a 12-item measure developed using patients with chronic illness and designed to assess a total of five domains of sleep problems: initiation, maintenance, respiratory problems, quantity, perceived adequacy and somnolence (13). A sleep problems index can also be scored based on summarizing information across nine items on the MOS Sleep Scale (14). However, despite their popularity, neither the PSQI nor MOS have scales that directly assess the impact of pain on sleep, making them inadequate as tools to be used for determining a causal link between pain and sleep.

One way researchers have attempted to circumvent this issue is by including a single item in their protocols that directly asks about the impact of pain on sleep. In one study, patients were asked to indicate the frequency of sleep disturbance due to pain ranging from ‘undisturbed’ to ‘more than five times per night’. Change in the frequency pre- and post-treatment was used as an indicator of treatment effectiveness (15). Another study had patients rate “pain disturbed night rest” on a 100 mm visual analogue scale (VAS) and used changes on this measurement as an indication of treatment effectiveness (16). The Daily Sleep Interference Scale is a single-item measure that was developed specifically to quantify sleep interference due to pain (17). The Daily Sleep Interference Scale has an 11-point response scale (ranging from 0 ‘did not interfere with sleep’ to 10 ‘completely interfered with sleep – unable to sleep due to pain’) and asks patients to select the number that best describes how much their pain has interfered with their sleep over a period of 24 h (17). Similarly, some multi-item pain scales have one item that measures pain interference with sleep (eg, the Multidimensional Pain Inventory and the Brief Pain Inventory) and this single item is then used as an outcome measure for treatment efficacy (18,19). Unfortunately, single-item measures lack reliability and their use is generally not recommended (20–23). There is considerable measurement error associated with a single item, and it is unlikely that a single item can adequately capture the complex relationship between sleep disturbance and pain.

One measure that does include multiple items that attribute sleep problems to pain is the Chronic Pain Sleep Inventory (CPSI). The CPSI consists of five items measured on a 100 mm VAS assessing: trouble falling asleep due to pain (CPSI1), the need for sleep medication (CPSI2), awakening by pain during the night (CPSI3) and in the morning (CPSI4), and overall sleep quality (CPSI5) (24). The CPSI provides a brief assessment of the impact of chronic pain on pain-related sleep problems, and psychometric testing has supported the reliability and validity of an abbreviated Sleep Problems Index (SPI) derived from the CPSI. The SPI is determined by calculating the average score across three items that specifically attribute sleep problems to pain (CPSI1, CPSI3 and CPSI4) (24). Before the identification and validation of the SPI, there was confusion among researchers about how to use the CPSI in clinical research as a measure of the impact of pain on sleep. Some studies only included the CPSI score for overall sleep quality (CPSI5) (25), while others report using all five items on the inventory as separate end points (26,27). Confusion appears to continue concerning the scoring rules for the measure given that, as recently as 2010, a study was published that reported selecting four items from the CPSI for inclusion in its protocol (28). The SPI does not appear to have been widely adopted; however, because we could identify only three studies that have reported using the measure since the publication of the validation report (28–30).

Another multi-item measure designed to assess the impact of pain on sleep in chronic pain patients is the Pain and Sleep Questionnaire (PSQ) (31). The PSQ is an eight-item questionnaire developed to assess the impact of pain on quality of sleep. Six of the eight items are scored on a 100 mm VAS (ranging from 0 [‘never’] to 100 [‘always’]) and asks respondents to rate how often they have trouble falling asleep (PSQ1); how often they need pain medication to fall asleep (PSQ2); how often they need sleeping medication to fall asleep (PSQ3); how often they are awakened by pain during the night (PSQ4) and in the morning (PSQ5); and how often their partner is awakened (PSQ6). The seventh item is also scored using a VAS; however, it uses different anchor points (0 [‘very poor’], 100 [‘excellent’]) and asks individuals to rate the overall quality of their sleep. The final item asks individuals to indicate, using a number that can range from 1 to 24, the average number of hours of sleep they get each night. Typically, a single index is created from summing the scores on the first five items and using this composite score as an overall measure of the impact of pain on quality of sleep (referred to as the Pain and Sleep Index) (32–37). Interestingly, all of the items on the CPSI are included in the PSQ. Although the PSQ has been used as a clinical end point, limited information is available on the psychometric qualities of this measure – or the commonly scored and utilized PSQ Index – to support its use for evaluating treatment effectiveness among chronic pain patients.

The purpose of the present retrospective study was to evaluate the psychometric properties of the PSQ Index to provide preliminary evidence for the reliability and validity of the scale.

METHODS

Participants

The data for the present study were pooled from nine independent double-blind, randomized placebo-controlled studies, of which eight were cross-over designs, conducted over a period of approximately 10 years. Studies were selected from clinical trials that were all conducted by Purdue Pharma, Canada. Protocols and informed consent were approved by research ethics boards at all participating centres and each patient gave written informed consent before participating in the respective study.

All studies in the Purdue Pharma database that included the PSQ as an outcome measure were included in the present sample. Excluded were individuals who were suffering from acute trauma or cancer-related pain because each of these categories is regarded to be unique from other forms of nonmalignant chronic pain. If pain became overwhelming, rescue medication was available at all times, regardless of treatment assignment, in all studies.

The pooled data resulted in a sample of 605 patients (65.5% female, 34.5% male) 19 to 88 years of age with a mean (± SD) age of 55.7±12.6 years. The majority of participants were low back pain patients (n=287); however, a total of four pain classification categories were represented in the sample (Table 1). The study population was based on the full analysis dataset, and missing values were not replaced.

TABLE 1.

Patient classification according to sample

Classification of pain	Total (n=605)	Baseline		Treatment (n=573)
		Sample 1 (n=279)	Sample 2 (n=326)
Low-back pain	287	139	148	278
Osteoarthritis of hip or knee	157	67	90	148
Chronic noncancer-related pain	103	54	49	91
Neuropathic pain*	58	19	39	56
Age, years, range	19–88	25–85	19–88	19–88
Age, years, mean ± SD	55.7±12.6	55.9±12.1	55.5±13.0	55.8±12.6
Sex, % female	65.5	66.3	64.7	65.3

Data presented as n unless otherwise indicated. There were 32 subjects who had only baseline data available, resulting in the total sample being slightly larger than the treatment sample.

^*Neuropathic pain group includes diabetic neuropathy

Data from baseline measurements (n=605) and measurements taken at the completion of the treatment arm of the studies (n=573) were used in the analyses. The baseline data were randomly split into two independent samples, n=279 and n=326, so that a cross validation approach to data analysis could be applied. The two baseline samples were similar in age and sex (Table 1).

Measures

PSQ:

As noted above, the PSQ is an eight-item questionnaire developed to assess the impact of pain on quality of sleep. Seven of the eight items are scored using 100 mm VAS, while the remaining item asks individuals to indicate the average number of hours of sleep they get each night. Typically, the first five items on the scale are summed and used as an overall measure of the impact of pain on quality of sleep (38,39).

Western Ontario and McMaster Universities Osteoarthritis Index:

The Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) is a 24-item scale that assesses the three dimensions of pain, disability and joint stiffness in the knee and hip of osteoarthritis patients using a 100 mm VAS (40). It has been widely used in the evaluation of knee and hip osteoarthritis, and has been demonstrated to be a reliable and responsive measure of outcome in diverse clinical and interventional environments. Higher scores are associated with greater pain, stiffness and disability.

36-item Short-Form Health Survey:

The 36-item Short-Form Health Survey (SF-36) is a multipurpose, short-form health survey consisting of 36 questions that yields two overall indexes: the Physical Component Summary score and the Mental Component Summary score. These two scores have been found to be reliable and valid (41), and have proven useful in surveys of general and specific populations comparing the relative burden of diseases and in differentiating the health benefits produced by a wide range of treatments. The summary measures are scored such that higher scores reflect better health and functioning. Information on the development and validation of the SF-36 physical and mental component summary measures are available in a technical manual, which also presents norms for each of the scales (42,43).

Pain Disability Index (44):

The Pain Disability Index (PDI) consists of seven items rated on a 10-point scale ranging from 0 (no disability) to 10 (total disability). The PDI is the algebraic sum of the seven items ranging from 0 to 70. Items on the scale ask about family home responsibilities, recreation, social activity, occupation, sexual behaviour, self-care and life support activity (33). The psychometric properties of the measure have been explored, and findings generally support the reliability and validity of the PDI as a brief measure of pain-related disability (44).

Activity Level Evaluation:

The Activity Level Evaluation (ALE) consists of one item measured on a 100 mm VAS, ranging from 0 (sedentary) to 100 (very active/daily vigorous activity). Respondents are asked to recall their last clinic visit and assess their level of activity.

Pain Intensity Questionnaire:

For the Pain Intensity Questionnaire (PIQ), individuals are asked to rate the intensity of their pain on a 100 mm VAS ranging from 0 (no pain) to 100 (excruciating pain).

Data analysis

When scoring the PSQ, the first five items are summed to form a composite index presumed to measure the impact of pain on sleep. The primary goal was to investigate the psychometric properties of this index. Rather than simply test the unidimensionality of the five-item index using a confirmatory factor analytical approach, an exploratory approach was adopted, whereby the underlying factor structure of the index could be explored. Best practices recommend that solutions based on exploratory factor analysis (EFA) be cross-validated in independent samples. As such, the pooled baseline data were randomly split into two independent samples. An EFA was conducted on the five-item PSQ Index to identify its underlying dimensionality, as well as on the full set of eight items appearing on the questionnaire to evaluate the structure of the measure as a whole. EFAs were conducted using baseline sample 1. The solution was cross-validated using confirmatory factor analysis (CFA) in baseline sample 2 and the treatment sample. For both the EFAs and CFAs, multiple complimentary fit indexes were used to evaluate model fit including the χ² summary statistic (nonsignificant), root mean square error of approximation (RMSEA, ≤0.08), Tucker-Lewis index (TLI, ≥0.90) and the comparative fit index (CFI, ≥0.90) (Cut-off values for fit indices were considered lower bounds for acceptable model fit. Good fit was assessed as RMSEA ≤0.06 and TLI and CFI ≥0.95 [45–49]). To evaluate the internal consistency reliability of the PSQ, coefficient alpha was estimated. An alpha coefficient in the range of 0.80 was considered to be satisfactory (50–52), with 0.70 acting as a lower bound for a minimally acceptable level of reliability (53). Interitem correlations were also computed and evaluated, with high interitem correlations suggesting that test items are all measuring the same construct, further supporting the internal consistency of the scale (54). To evaluate the ability of the measure to detect treatment effects, thus justifying it as an outcome measure in clinical research studies, descriptive statistics of the PSQ (mean values and SDs) were calculated at baseline and at the end of treatment and their differences were compared using paired-sample t tests. Zero-order correlations were calculated between PSQ scores and a selection of health-related outcome measures to evaluate the criterion-related validity of the PSQ. Cohen’s criteria for effect size interpretation were implemented to evaluate clinical meaningfulness (ie, r values <0.30 are small; r values of 0.30 to 0.49 are moderate; and r values ≥0.50 are large) (55). Based on previous research, it was hypothesized that moderate to large correlations would be observed between PSQ index scores and scores on all criterion measures (ie, ALE, PIQ, PDI, SF-36 Physical and Mental components, and WOMAC subscales) (28,29,56–59). The EFAs and CFAs were performed using Mplus 6 (Mplus, USA)(60). All other analyses were performed using SPSS version 19.0 (IBM Coporation, USA) (61).

RESULTS

Factor analyses (structural validity)

Table 2 presents the interitem correlations, eigenvalues, factor loadings and fit statistics from the exploratory factor analysis of the five-item PSQ composite score. Both a one- and two-factor model were explored, with an oblique rotation applied to the two-factor solution, which allows the factors to be correlated. The overall fit of the one-factor solution was poor, as indicated by the significant χ² statistic, RMSEA >0.08, and CFI and TLI ≤0.90 (Table 2). Item 3 (how often have you needed sleep medication to help you fall asleep) appeared problematic, having the lowest correlation with the other items in the set (ranging from 0.28 to 0.48), a lower factor loading (0.39) than was achieved with the other four items (ranging from 0.63 to 0.85) and the largest residual variance (0.85).

TABLE 2.

Interitem correlations, eigenvalues and factor loadings from exploratory factor analysis of the Pain and Sleep Questionnaire (PSQ) five-item Index in baseline sample 1

	Interitem correlations					Eigenvalue	One-factor solution		Two-factor solution
									Factor loadings		Residual variance
	PSQ1	PSQ2	PSQ3	PSQ4	PSQ5		Factor loadings	Residual variance	1	2
PSQ1	1.00					2.97	0.78	0.40	0.26	0.58	0.43
PSQ2	0.59	1.00				0.91	0.63	0.60	1.03	0.00	−0.05
PSQ3	0.31	0.48	1.00			0.51	0.39	0.85	0.42	0.10	0.77
PSQ4	0.65	0.48	0.28	1.00		0.35	0.85	0.28	−0.03	0.92	0.18
PSQ5	0.58	0.45	0.28	0.73	1.00	0.26	0.80	0.36	0.02	0.79	0.36
Model fit indexes
χ2 (df)							59.83 (5)		0.27 (1)
P							<0.001		0.60
RMSEA (90% CI)							0.20 (0.16–0.25)		0.00 (0.00–0.13)
CFI							0.90		1.00
TLI							0.80		1.01

PSQ abbreviated item content: PSQ1 Touble falling asleep; PSQ2 Need pain medication to fall asleep; PSQ3 Need sleep medication to fall asleep; PSQ4 Awakened by pain during the night; PSQ5 Awakened by pain in the morning. CFI Comparative fit index; RMSEA Root mean square error of approximation; TLI Tucker-Lewis Index. Bolded values indicate factor loadings >0.40

The two-factor solution fit the data well (Table 2). Item 2 and item 3 loaded on the first factor, and items 1, 4 and 5 all loaded on the second factor. The two factors were correlated (r=0.53). Item content suggested that factor 1 was assessing the need for medication (either pain medication or sleep medication) for sleep; whereas item content for factor 2 focused on sleep problems due to pain. Similar to the one-factor solution, estimated residual variance for item 3 (0.77) was larger than for the remaining items (ranging from −0.05 to 0.43) suggesting that the model does not account for a majority of variance associated with this item. The negative residual variance associated with item 2, coupled with its large factor loading (>1.00) suggested that too many factors were extracted and that the item should be eliminated. Eliminating items 2 and 3 would reduce the index to a unidimensional scale consisting of items 1, 4 and 5.

To determine whether additional items from the PSQ may contribute to the assessment of the impact of pain on sleep, an EFA of the full eight-item inventory was conducted testing the fit of one- to three-factor solutions. The interitem correlations for the eight items are presented in Table 3. Again, item 3 showed the weakest pattern of correlation with the other items on the inventory ranging from r=0.06 to r=0.48. With the exception of item 3, the majority of interitem correlations were moderate to strong (0.40 to 0.73). Although solutions for one through three factors were extracted, only the first two eigenvalues were >1.00. The first eigenvalue was significantly larger than the second (3.99 versus 1.29), suggesting that one dominant latent factor may account for the underlying structure of the items, although a two-factor solution may also fit the data. However, similar to results found with the five-item index, fit of the one factor model to the data was poor (Table 4). The two-factor model demonstrated adequate fit with an RMSEA=0.08 and CFI and TLI ≥0.95. The two factors were correlated (r=0.53). The two factors resembled those extracted from the five-item model, with the additional items from the PSQ (items 6, 7 and 8) loading on the second factor. Item 1 had a small secondary loading on factor 1, but otherwise the first factor continued to be identified by only two items. The three-factor solution was deemed inappropriate despite demonstrating good fit to the data across all indices, because the third factor was identified by only one item, and the factor loading was >1.0 and had a negative residual variance, suggesting that too many factors had been extracted. The presence of secondary loadings also made the three-factor solution difficult to interpret, justifying exclusion of this model from further consideration.

TABLE 3.

Pain and Sleep Questionnaire (PSQ) interitem correlations in baseline sample 1

	PSQ1	PSQ2	PSQ3	PSQ4	PSQ5	PSQ6	PSQ7	PSQ8
PSQ1	1.00
PSQ2	0.59	1.00
PSQ3	0.31	0.48	1.00
PSQ4	0.65	0.48	0.28	1.00
PSQ5	0.58	0.45	0.28	0.73	1.00
PSQ6	0.45	0.32	0.14*	0.52	0.58	1.00
PSQ7	0.52	0.25	0.14*	0.52	0.49	0.43	1.00
PSQ8	0.36	0.11*	0.06 (ns)	0.39	0.44	0.42	0.53	1.00

All correlations significant (P<0.001) unless otherwise indicated.

^*P<0.05. ns Not significant

TABLE 4.

Eigenvalues, factor loadings and residual variance from exploratory factor analysis of the Pain and Sleep Questionnaire (PSQ) in baseline sample 1

	Eigenvalues	One-factor solution		Two-factor solution			Three-factor solution
				Factor loadings		Residual variance	Factor loadings			Residual variance
		Factor loadings	Residual variance	1	2		1	2	3
PSQ1	3.990	0.77	0.41	0.34	0.55	0.38	0.45	0.55	−0.01	0.33
PSQ2	1.285	0.59	0.65	0.91	0.01	0.16	0.83	0.01	0.07	0.23
PSQ3	0.678	0.36	0.87	0.53	0.00	0.72	0.51	−0.08	0.10	0.71
PSQ4	0.584	0.85	0.28	0.12	0.77	0.30	0.21	0.41	0.34	0.35
PSQ5	0.463	0.92	0.33	0.09	0.77	0.33	−0.04	0.01	1.02	−0.01
PSQ6	0.436	0.64	0.59	0.00	0.66	0.56	0.04	0.39	0.28	0.59
PSQ7	0.313	0.62	0.61	−0.13	0.74	0.54	−0.01	0.87	−0.12	0.39
PSQ8	0.252	0.51	0.74	−0.27	0.72	0.61	−0.19	0.69	0.03	0.56
Model fit indexes
χ2 (df)		120.15 (20)		33.70 (13)			9.306 (7)
P		<0.001		<0.001			0.23
RMSEA (90% CI)		0.14 (0.11–0.16)		0.08 (0.05–0.11)			0.04 (0.00–0.09)
CFI		0.88		0.97			1.00
TLI		0.83		0.95			0.99

Primary loadings are indicated in bold; secondary loadings are indicated in italics. CFI Comparative fit index; RMSEA Root mean square error of approximation; TLI Tucker-Lewis index

Across all three solutions, item 3 demonstrated the highest residual variance estimates (ranging from 0.71 to 0.87), indicating that the majority of the variance in this item is not well accounted for by any of the solutions. Items 2 and 3 consistently loaded on the same factor across solutions; however, it is recommended that at least three items with substantial loadings are required to identify a factor (62). With only two items, it was decided that this factor was not viable.

Based on the results from the EFA in baseline sample 1, CFAs were conducted using baseline sample 2 and the treatment sample, testing the fit of a one-factor model consisting of items 1, 4, 5, 6, 7 and 8, and the alternative one-factor model solution consisting of items 1, 4 and 5. Results from the CFAs are presented in Table 5. Using baseline sample 2, the overall model fit of a one-factor solution consisting of items 1, 4, 5, 6, 7, and 8 was poor based on the χ² goodness-of-fit test (χ²=49.94, df=9; P<0.001) and RMSEA (0.12 [90% CI 0.09 to 0.16]); however, the CFI (0.95) and TLI (0.92) were indicative of adequate fit. Results of the fit of a one-factor model consisting of items 1, 4 and 5 returned a significant χ², but perfect fit according to the other indices (CFI=1.00; TLI=1.00; RMSEA=0.00). Similarly, the fit of a one-factor model to the six-item solution using the treatment sample was poor based on the χ² goodness-of-fit test and the RMSEA, but adequate based on the CFI and TLI (Table 5). Fit of the one-factor model to the three-item index retuned a significant χ², but perfect fit according to the other indexes. Taken together, the results suggest that the three-item index represents the best fit to the data across samples.

TABLE 5.

Summary of fit statistics from confirmatory factor analysis of the Pain and Sleep Questionnaire

Sample	Model description	χ2	df	P	CFI	TLI	RMSEA	RMSEA 90% CI
Baseline sample 2	6-item solution	49.94	9	<0.001	0.95	0.92	0.12	0.09–0.16
	3-item solution	0.00	0	<0.001	1.00	1.00	0.00	0.00–0.00
Treatment sample	6-item solution	66.22	9	<0.001	0.97	0.94	0.11	0.08–0.13
	3-item solution	0.00	0	<0.001	1.00	1.00	0.00	0.00–0.00

CFI Comparative fit index; RMSEA Root mean square error of approximation; TLI Tucker-Lewis Index

Sample statistics and reliability

Given the results of the factor analyses, all additional analyses were conducted based on the three-item composite score that consists of the algebraic sum of scores from items 1, 4 and 5 on the PSQ (herein referred to as the PSQ-3). Table 6 presents the internal consistency (coefficient alpha), mean values, and SD at both baseline measurement and completion of the treatment arm of the study for the PSQ-3. Scores are presented for the total sample, as well as by patient pain category. It has previously been reported that VAS scale changes (decreases) between approximately 30% and 35% indicate a meaningful change in pain to patients across patient populations (63). In the present study, decreases in mean scores on the PSQ-3 ranged from 30% to 57%. Results from a series of paired sample t tests revealed that the index was sensitive to treatment changes, with mean scores significantly higher (indicating poorer sleep) at baseline measurement compared with the end of treatment when considering the sample as a whole as well as according to patient pain group. Internal consistency reliability estimates ranged between 0.82 and 0.91 at baseline measurement, and between 0.88 and 0.93 at the end of treatment (Table 6).

TABLE 6.

Sample statistics and reliability of the Pain and Sleep Questionnaire-3 at baseline and after treatment for the total sample and within patient pain categories

Patient pain category	α	Mean±SD	Mean difference
Total sample
Baseline (n=572)	0.87	163.22±83.94
Treatment (n=556)	0.90	107.96±84.78	t=15.71, df=528; P<0.001
Low back pain
Baseline (n=257)	0.82	187.18±74.39
Treatment (n=262)	0.88	131.77±83.78	t=9.97, df=235; P<0.001
Osteoarthritis of hip or knee
Baseline (n=154)	0.85	125.51±82.02
Treatment (n=148)	0.89	81.00±73.74	t=6.64, df=146; P<0.001
Chronic noncancer-related pain
Baseline (n=103)	0.89	159.86±84.83
Treatment (n=91)	0.91	106.10±84.64	t=7.48, df=90; P<0.001
Neuropathic pain
Baseline (n=58)	0.91	163.12±91.33
Treatment (n=55)	0.93	70.15±84.04	t=8.33, df=54; P<0.001

Reported sample sizes do not correspond with those in Table 1 due to missing data

Criterion-related and predictive validity

Table 7 presents the bivariate correlations among PSQ-3 scores and scores from other health-related outcome measures at baseline and after treatment. Results are presented for the total sample, as well as by pain category where possible (Different outcome measures were used in the different study protocols, thus, only subsets of the samples completed certain measures. For example, only the osteoarthritis of the hip or knee patients completed the WOMAC at baseline and after treatment). As hypothesized, the majority of correlations between the PSQ-3 scores and scores on other health-related outcomes measures were moderate to large in magnitude. Contrary to hypotheses, the majority of the correlations between the PSQ-3 scores and scores on the ALE and SF-36 mental component were either not significant or small in magnitude.

TABLE 7.

Correlations between scores on the Pain and Sleep Questionnaire-3 and scores on other health-related outcome measures at baseline measurement and after treatment across pain categories

Pain category	ALE	PIQ	PDI	SF-36		WOMAC
				Physical	Mental	Pain	Function	Stiffness
Total sample
Baseline	−0.20	0.50	0.48	−0.42	−0.21
n	361	451	511	394	394
Treatment	−0.24	0.62	0.63	−0.56	−0.31
n	363	537	495	365	365
Low back pain
Baseline	−0.15*	0.49	0.43	−0.28	−0.26
n	254	242	257	252	252
Treatment	−0.20	0.62	0.55	−0.48	−0.26
n	261	247	261	241	241
Osteoarthritis of hip or knee
Baseline	−0.23*	0.42	0.50	−0.41	−0.10 (ns)	0.60	0.52	0.50
n	93	153	93	89	89	154	154	153
Treatment	−0.27*	0.53	0.50	−0.45	−0.32**	0.62	0.52	0.40
n	88	148	88	77	77	148	147	148
Chronic noncancer-related pain
Baseline			0.50
n			103
Treatment		0.63	0.72
n		87	91
Neuropathic pain
Baseline	−0.32 (ns)	0.62	0.41	−0.48	−0.25 (ns)
n	14	56	58	53	53
Treatment	−0.44 (ns)	0.77	0.79	−0.68	−0.53
n	14	55	55	47	47

All correlations are significant (P≤0.001) unless otherwise indicated.

^*P≤0.05;

^**P≤0.01.

ALE Activity Level Evaluation; ns Not significant; PDI Pain and Disability Index; PIQ Pain Intensity Questionnaire; SF-36 Short Form Health Survey; WOMAC Western Ontario and McMaster Universities Osteoarthritis Index

Although the exploratory and confirmatory factor analyses supported the use of the shorter three-item index, a series of multiple regression analyses were conducted to explore the amount of additional variance accounted for by the two remaining items of the original PSQ (five-item index) over and above the new PSQ-3 in the prediction of patient-reported outcomes. The PSQ-3 was entered simultaneously into a regression equation along with the two remaining items from the five-item index (item 2: How often have you needed pain medication to fall asleep; and item 3: How often have you needed sleep medication to help you fall asleep). Table 8 presents the amount of variance in the criterion measure that is accounted for by the set of predictors, the unstandardized and standardized regression coefficients for each predictor along with the corresponding t test, as well as the squared semipartial correlation that indicates the amount of variance accounted for by a given predictor over and above the variance accounted for by the remaining predictors in the set. Results demonstrated that the PSQ-3 was the only significant predictor of WOMAC: Function and WOMAC: Stiffness scores, accounting for 34% and 35% of the variance, respectively, in osteoarthritis of the hip or knee patients. For the remaining patient-reported outcome measures, the PSQ-3 accounted for the majority of the variance (ranging from 29% to 40%); however, item 2 incremented the Index in the prediction of health-related outcome scores, accounting for 9% to 13% of additional variance. Item 3, asking about the need for sleep medication to assist with sleep, did not contribute to the prediction of any of the criterion measures included in this set of analyses.

TABLE 8.

Multiple regression analyses predicting scores on patient-reported health related outcomes with the Pain and Sleep Questionnaire-3 (PSQ-3) and the additional items of the original five-item PSQ index at baseline

Health-related outcome measure	Predictors	R2	F	β		t	P	Squared semipartial correlation
				Unstandardized	Standardized
SF-36 Physical	PSQ-3			−0.05	−0.37	−6.23	<0.001	−0.29
	Item 2	0.19	29.26	−0.04	−0.13	−1.99	0.05	−0.09
	Item 3			0.02	0.07	1.32	0.19	0.06
Pain intensity	PSQ-3			0.09	0.41	8.03	<0.001	0.33
	Item 2	0.28	56.15	0.06	0.12	2.30	0.03	0.09
	Item 3			0.04	0.08	1.63	0.10	0.07
Total pain and disability	PSQ-3			0.07	0.39	7.97	<0.001	0.31
	Item 2	0.25	54.56	0.06	0.14	2.60	0.01	0.10
	Item 3			0.01	0.02	0.53	0.59	0.02
WOMAC: Pain	PSQ-3			0.55	0.48	6.23	<0.001	0.40
	Item 2	0.41	33.41	0.47	0.17	2.09	0.04	0.13
	Item 3			0.39	0.12	1.70	0.09	0.11
WOMAC: Function	PSQ-3			1.65	0.41	4.88	<0.001	0.34
	Item 2	0.29	20.23	1.47	0.15	1.74	0.08	0.12
	Item 3			0.85	0.07	0.98	0.33	0.07
WOMAC: Stiffness	PSQ-3			0.22	0.42	4.91	<0.001	0.35
	Item 2	0.28	18.39	0.19	0.15	1.69	0.09	0.12
	Item 3			0.00	0.00	0.02	0.99	0.00

Significant predictors are indicated in bold. PSQ-3 = Sum of items 1, 4 and 5 on PSQ: Item 2 = “How often have you needed pain medication to fall asleep?”; Item 3 = “How often have you needed sleep medication to help you fall asleep?” SF-36 Short-form Health Survey; WOMAC Western Ontario and McMaster Universities Osteoarthritis Index

DISCUSSION

The majority of chronic pain patients (50% to 70%) report experiencing pain-related sleep disturbance on a regular basis (28,65). As a result, the literature increasingly stresses the importance of devoting more attention to the reciprocal relationship between pain and sleep disturbance from both a diagnostic and treatment perspective (28). This requires that there be a valid and reliable means for researchers and clinicians to directly assess the impact of pain on sleep. The purpose of the present study was to examine the psychometric properties of the pain and sleep questionnaire and, in particular, the PSQ index, and to provide an assessment of the reliability and validity of the index for assessing the impact of pain on sleep.

When used in clinical trials as an outcome measure, the first five items of the measure are typically summed to form a composite index assessing the impact of pain on sleep. Results from an EFA did not support the use of the inventory in this fashion as optimal. A one-factor model did not explain the underlying structure of the items. Instead, a correlated two-factor model demonstrated better fit, with one factor consisting of three items and assessing sleep disturbance attributed to pain, and the other factor indicated by two items assessing the need for medication to assist with sleep. The first factor, however, had an item with a negative residual variance and the other item loading on this factor had a large residual variance. For these reasons this factor was not seen as viable. A second EFA was conducted to explore the structure of all eight items on the questionnaire. The additional items from the inventory all loaded with items 1, 4 and 5 on a factor assessing sleep disturbance. Two competing models were cross-validated using CFA. Across samples, the three-item factor consisting of items 1, 4 and 5 demonstrated the best fit to the data and was determined to be the best scoring option from the items on the inventory for assessing the impact of pain on sleep.

A qualitative inspection of the content of each item was also supportive of the three-item index. A strong rationale for not including the additional items could be made for each of the remaining items on the inventory. Of the items not included in the PSQ-3, item 2 (which asks about the need for pain medication to initiate sleep) is the only item that, at face level, could reasonably be hypothesized to belong in a coherent set with the other three items as a measure of sleep disturbance attributable to pain. The other five items on the PSQ do not specifically ask about pain and its relation to sleep. Item 3 asks about the need for sleep medication to help sleep, but does not mention pain. One could imagine a number of reasons for why sleep medication might be required to help with sleep (eg, physical and/or psychological dependency on sleeping pills; insomnia due to hormonal changes), of which pain is only one among many possible explanations. Interestingly, this was the most problematic item across samples, demonstrating the poorest psychometric properties (weak correlations with other items on the questionnaire, high residual variance across factor solutions, and a lack of predictive validity), calling into question the importance of this concept in patients with chronic nonmalignant pain. This result is not unprecedented because a similar study investigating the psychometric properties of the CPSI (whose items all appear in identical form on the PSQ) found this item to have poor psychometric properties, demonstrating elevated floor effects, skewed score distributions, and weak associations with other items on the inventory and with measures of related constructs (24).

Items 6, 7 and 8 also do not refer to pain specifically. Item 6 asks: “if sleeping with a partner, how often has the partner awakened from his or her sleep”, but does not ask if the awakening was a result of the patient’s pain-related sleep disturbance. There are a number of alternative explanations for why one’s partner may awaken from sleep, many of which may have nothing to do with the patient’s pain-related sleep disturbance. For example, the patient may snore loudly causing the partner to awaken in the night. The item only asks about sleep awakening on the part of the partner, without linking the item back to the cause of the awakening or the target construct of the impact of pain on sleep. Item 7 (average number of hours of sleep per night) and item 8 (rating of overall quality of sleep) on the PSQ also do not specifically address the impact of pain on sleep. Instead, both are more related to the construct of sleep effectiveness than to the direct impact of pain on sleep.

Focusing on the three items that formed a coherent set (the PSQ-3), the results of this evaluation support the use of the index as a brief and efficient measure of the impact of pain on sleep. The index achieved strong internal consistency reliability coefficients at both baseline measurement and after treatment in the total sample, as well as across patient pain categories. Supporting its use as an outcome measure in clinical trials, mean scores were significantly lower after treatment than at baseline measurement, demonstrating the sensitivity of the PSQ-3 for capturing positive treatment effects. Preliminary evidence of the criterion-related validity of the index was demonstrated through the moderate to large correlations achieved between PSQ-3 scores and scores on measures of theoretically similar but separate constructs. In particular, it was found that higher scores on the PSQ-3 were associated with greater levels of pain, more intense pain, and increased disability resulting from pain, joint stiffness and poor physical functioning. Only small or nonsignificant relationships were observed between the PSQ-3 and scores on the ALE and SF-36 Mental Functioning scale. Although previous research has suggested that sleep disturbance and pain-related sleep disturbance are associated with decreased mental functioning (56) and reduced activity level (4), results from the present study suggest that mental functioning and activity level are not associated (or demonstrate only weak to moderate associations) with pain-related sleep disturbance for the majority of chronic pain groups included in the present study (Table 7). Finally, a series of multiple regression analyses demonstrated the predictive validity of the PSQ-3. These analyses further demonstrated that little, if any, additional variance was accounted for in the prediction of other health-related outcome measures by including the two additional items typically scored in the original five-item PSQ Index. Therefore, it should be noted that this outcome does not negate any previous results that have been acquired with the five-item index, but simply suggests that the predictive power of the inventory, or the degree of association among outside measures and the PSQ composite score, were largely driven by the newly identified coherent three-item core, with no additional variance accounted for by item 3, and only a small percentage of unique variance accounted for by item 2 (≤13%). In other words, the same or similar results would have been achieved if only the three items were summed. Thus, because the five-item index does not form a unidimensional structure, and little additional variance in patient reported outcomes is accounted for by including the additional items, the practice of summing the first five items on the PSQ to form a composite score should be abandoned and replaced by the use of the more psychometrically sound PSQ-3.

It is a notable strength of the present study that it converges with results from previous research (24) investigating a similar measure (the CPSI) that also resulted in an identical three-item index (the SPI) for the assessment of the impact of pain on sleep. Together, these two studies provide strong support for the validity of the index. As noted previously, the CPSI is subsumed within the PSQ, with all five of the CPSI items appearing on the PSQ. Although the impact of the validation of the SPI has been limited, having been cited only five times since its publication according to the Web of Science Citation Index, it is our hope that, given the popularity of the PSQ and the preliminary evidence for the reliability and validity of the PSQ-3 presented here, that the Index will become a preferred measure for assessing the impact of pain on sleep.

The development of the PSQ-3 in the present article is intended to be timely to recent developments on the relationships between sleep and pain. Sleep disturbance is one of the most common comorbid problems for chronic pain patients. The association between the two phenomena has long been recognized, but the nature of the relationship is not well-understood. In a recent review of epidemiological, clinical and laboratory studies (64), a positive relationship emerged between poor sleep and increased pain. Sleep deprivation also seemed to attenuate analgesic effects of medications.

Independently, this same association emerged in a literature review of studies with control group designs that investigated sleep deprivation (total or partial form) as an independent and pain as a dependent variable. In five of six studies, sleep deprivation lead to an increase in bodily complaints as measured by self-rating tests and a decrease of pain inhibition (65).

Recently, the relationship between sleep disruption and pain states has also been prospectively studied. Twenty patients with a somatoform pain disorder according to the International Classification of Diseases, 10th Revision diagnosis criteria were sleep deprived for one night, followed by one recovery night. Clinical pain complaints (VAS), detection and pain thresholds (temperature and pressure), as well as mood states (Profile and Mood States) were assessed on the day before the experiment, on the day after sleep deprivation and on the day after recovery sleep. We found a discrepancy (66) between significantly increased clinical pain complaints and unaltered experimental pain perception after sleep deprivation. Only the clinical pain complaints – but not the experimental pain thresholds – were correlated with tiredness-associated symptoms. Total mood disturbances decreased and feelings of depression and anger improved significantly after sleep deprivation; however, these changes were not correlated with a change in clinical pain perception.

Apart from direct associations between sleep disruption and pain, there is also evidence for a possible indirect relationship between short sleep duration and pain. This relates to the associations observed in the Quebec Family Study between short sleep duration and increased obesity. This finding was demonstrated for both children and adults (67,68). The link to obesity has also been independently demonstrated in epidemiological studies indicating that sleep restriction results in metabolic and endocrine alterations along with increased hunger and appetite (69).

In turn, obesity is associated with back pain. One insight into the potential implications of obesity for low back pain stems from a meta-analysis on the association between overweight/obesity and low back pain. In cross-sectional studies, obesity was associated with increased prevalence of low back pain in the previous 12 months, seeking care for low back pain and chronic low back pain. In cohort studies within this same meta-analysis, only obesity was associated with an increased incidence of low back pain in the previous 12 months (70). More recently, in a cross-sectional study of Chinese volunteers (71), individuals who were overweight and obese presented with a 1.3 and a 1.8 increase in the likelihood of having degenerative disc disease, respectively. Previous studies have cited that magnetic resonance imaging-verified, degenerative disc disease correlates with low back pain (72,73).

Thus, while the causal or associative relationships between sleep and chronic, noncancer pain continue to evolve, they now build on an existing base that is increasingly noteworthy and independently supported on various fronts.

Limitations and future directions

The present study has a number of strengths that are worth noting, including the use of a large heterogeneous sample of chronic non-malignant pain patients, a data set generated from independent double-blind, randomized placebo-controlled studies, the use of a cross-validation approach to data analysis and the inclusion of several criterion measures for the purpose of evaluating criterion-related validity. However, the present study is also not without its limitations. For example, criterion measures were not selected a priori, but rather were used because they were readily available in existing data sets, which meant that different criterion measures were completed by patients in the various pain categories. A different selection of measures may have resulted in different results. Furthermore, published evaluations of the psychometric properties of two of the measures included (the ALE and PIQ) are not available. The study was also limited by not including an independent and well-validated assessment of sleep problems for the purpose of assessing convergent validity. Ideally, convergent validity should be explored by examining the correlations between the PSQ-3 and other validated measures of sleep problems, such as the PSQI or MOS Sleep Scale. Exploring associations between scores on the PSQ-3 and polygraphic measurements of sleep disturbance should also be explored to reduce the influence of shared method variance on the assessment of validity. These examples, however, are not necessarily limitations of the current study as much as they are suggested procedures for future evaluations of the measure.

We also recognize that the impact of pain on sleep may vary for different chronic pain conditions. Therefore, we believe that future research should examine the performance of the index in large samples of different forms of chronic pain than those studied in the present sample. In the present sample, cancer-related pain and acute trauma related pain were purposely excluded from the sample because it is believed that each of these conditions have unique pain profiles that are believed to differ in many ways from other forms of chronic pain (such as low back pain, osteoarthritis pain, etc). For example, researchers have reported that patients with cancer report a greater affective component of their pain than patients with low back pain (74). This is consistent with the hypothesis that cancer pain may be associated with higher levels of affect than other types of chronic pain (63). Thus, whereas scores on the PSQ-3 demonstrated only weak associations with SF-36 mental functioning in the present sample, a stronger correlation between these measures may be found in a sample of cancer patients. We also believe that future research should consider variables other than type of chronic pain and the influence they may have on PSQ-3 scores and the generalizability of results. For example, age and sex represent two variables that can have an influence on the impact of pain on sleep, and as such, future evaluations of the psycho-metric properties of the PSQ-3 should include an evaluation of age and sex differences. Depression has also been shown to be correlated with chronic pain in the pain literature, suggesting its influence on PSQ-3 scores is worth consideration. Finally, we believe the current three-item index could be strengthened by the addition of items that assess the impact of pain on sleep more broadly, thus eliminating any possible concerns that high reliability of the index is attributable to possible redundancy of item content. Future studies may consider creating additional items that may cover the domain more broadly thus adding to the reliability and validity of the index. Future research should also consider exploring the test-retest reliability of the index to further support the use of the index as an outcome measure in clinical trials.

CONCLUSIONS

Results of the present study support the scoring and use of a psychometrically sound three-item Index (PSQ-3) as a direct measure of the impact of chronic pain on sleep. The PSQ-3 is scored by computing the algebraic sum of items 1, 4 and 5 on the PSQ. The PSQ-3 has demonstrated strong reliability and structural, criterion-related and predictive validity within a large heterogeneous sample of chronic nonmalignant pain patients from controlled clinical trials. The present study provides support for the use of this index in clinical trials as a brief outcome measure assessing the impact of chronic pain on sleep, an area of much interest given their strong reciprocal relationship. The brevity of the index means that it is easy to include in assessment protocols, providing an opportunity for researchers to directly assess the impact of pain on sleep, something that is not possible with many of the existing, well-validated sleep measures. Funding agencies and ethics boards often require that inventories included in a clinical trial are empirically supported and validated before approval or funding for the trial is granted. Results from the present study provide researchers and clinicians with preliminary evidence that supports and justifies the inclusion of the PSQ and, in particular, use of the PSQ-3, as an outcome measure in future research in the management of chronic noncancer pain.