Internal consistency and factor structure of Jenkins Sleep Scale: cross-sectional cohort study among 80 000 adults

Abstract

Objectives

To assess the internal consistency and construct validity of the Finnish translation of the Jenkins Sleep Scale (JSS) in a large healthy working-age population with diverse work characteristics.

Design

Survey-based cross-sectional cohort study.

Setting

Survey conducted by an institute of occupational health.

Participants

Employees of 10 towns and 6 hospital districts.

Primary and secondary outcome measures

The internal consistency defined by a Cronbach’s alpha. Exploratory and confirmatory factor analyses to evaluate the construct structure of the JSS.

Results

Of 81 136 respondents, 14 890 (18%) were men and 66 246 (82%) were women. Their average age was 52.1 (13.2) years. Of the respondents, 41 823 (52%) were sleeping 7 or less hours per night. The mean JSS total score was 6.4 (4.8) points. The JSS demonstrated high internal consistency with an alpha of 0.80 (lower 95% confidence limit 0.80). Exploratory factor analysis supported a one-factor solution with eigenvalue of 1.94. Confirmatory factor analysis showed that all four items were positively correlated with a single common factor explaining 44%–61% of common factor’s variance.

Conclusions

The Finnish translation of JSS was found to be a unidimensional scale with good internal consistency. As such, the scale may be recommended as a practicable questionnaire when studying sleep difficulties in a healthy working-age population.

Strengths and limitations of this study.

• To our knowledge, this was the first study on the psychometrics of the Finnish translation of the Jenkins Sleep Scale (JSS).

• The cohort of over 80 000 respondents represented a wide spectrum of occupations from managers to manual workers.

• Inequality in gender distribution may overestimate the prevalence of sleep difficulties in the studied cohort.

• The JSS may be recommended as an easy-to-do questionnaire instrument for the studying of sleep difficulties in a healthy working-age population.

Introduction

Several different questionnaires have been developed to assess the severity of sleep problems.¹ The Jenkins Sleep Scale (JSS), developed as a brief and standardised test for sleep disturbances in 1988, has been one of the most commonly used questionnaires in epidemiological studies.^1–4 Comparing with other similar measures, the JSS is a short questionnaire focusing on roughly recognising sleep difficulties. That is unlike to more complex scales, like Insomnia Severity Index, which quantify also the impact of sleep disturbance on the level of daily functioning. The JSS has been translated in several languages^5–10 and found to be valid and reliable among patients with different health problems including rheumatoid arthritis,¹⁰ psoriatic arthritis,⁹ ankylosing spondylitis,⁷ fibromyalgia,^{5 11} chest pain,¹² post-cardiac surgery patients,² patients with cognitive disorders¹³ and epilepsy.¹⁴ However, only a few studies have evaluated the psychometric properties of the JSS in large non-clinical populations.2 3 8 15 16

Previous studies have found the JSS to be internally consistent among patients with fibromyalgia,^{5 11} rheumatoid arthritis,¹⁰ ankylosing spondylitis⁷ and psoriatic arthritis⁹ as indicated by Cronbach’s alphas between 0.7 and 0.9. Several studies have assessed the internal consistency of the JSS in general and/or healthy populations similarly reporting good to excellent Cronbach’s alphas that vary between 0.8 and 0.9.2 3 6 8 15 16 Only three previous studies have assessed the factor structure of the JSS finding, the JSS to be a unidimensional scale.^{3 6 8} The construct structure of the JSS analysis has been assessed by a single study using a confirmatory factor that produced strong correlations with common factor for all four items.³ Like any brief screening instrument, the JSS has shortcomings, specifically the inability to address the spectrum of sleep difficulties. Hence, it can only be used as a preliminary screener of sleep disturbance.¹⁷

Overall, there is uncertainty concerning the psychometric behaviour of the JSS especially regarding its factor structure in healthy and/or general populations. Concerning a general population, previous research mostly focused on the internal consistency of JSS and its reliability. Instead, other important points, like for example, factors structure, remained practically unknown. Additionally, the psychometric properties of Finnish translation of the JSS have not been studied yet. To address this limitation, the aim of this study was to assess the internal consistency and construct validity of the Finnish translation of the JSS in a large healthy working-age population.

Methods

The data were derived from the Finnish Public Sector (FPS) study, an ongoing prospective cohort study of employees in the municipal services of 10 Finnish towns and 21 public hospitals. The eligible population from the register cohort of FPS (n=151 618) included those who had been employed for a minimum of 6 months at the participating organisations between 1991 and 2005. Employers’ records have been used to identify the eligible employees for a nested survey cohort to whom questionnaire surveys have been repeated every 4 years since 2000.¹⁸ For this study, the data were sourced from the survey in 2016–2017 administered to the FPS subcohorts (average response rate 70%). Individual‐level survey data cannot be made publicly available, but information on the data and analyses is available on request to the corresponding author.

Age was defined in full years at the time of survey response. Body mass index (BMI) was defined as weight in kg/height in m². The level of physical activity was calculated from the survey responses and converted into metabolic equivalent of task (MET). Alcohol consumption was obtained from the survey and converted into g/week. The respondents were asked about their usual amount of sleep hours per 24 hours with the following nine response alternatives: <6 hours, 6.5 hours, 7 hours, 7.5 hours, 8 hours, 8.5 hours, 9 hours, 9.5 hours and >10 hours.

JSS is a 4-item questionnaire to follow common sleep problems.² The frequency of sleep problems in the last month is evaluated using four items: the difficulty to fall asleep, wake up at night, difficulty to stay asleep and non-restorative sleep (ie, waking up after the usual amount of sleep feeling tired and worn-out). Each item is rated on a Likert-like scale from 0 to 5, where 0 is ‘never’, 1 is ‘1–3 days’, 2 is ‘about 1 night/week’, 3 is ‘2–4 nights/week’, 4 is ‘5–6 nights/week’ and 5 is ‘almost every night’. The total score is a simple sum of all four items’ scores and ranges from 0 (no sleep problems) to 20 (most sleep problems). The score of 11 is a cut-off—a score <12 is defined as little of sleep disturbances and a score >11 is understood as high frequency of sleep disturbances.¹⁹ Another way to dichotomise the JSS is considering sleep difficulties being present if there is at least one ‘yes’ response (>15 nights in the previous 4 weeks) to any item.

Patient and public involvement

Participants of the research were not involved in setting the study question and outcome measures, and were not involved in the design and implementation of the study or writing the manuscript.

Statistical analysis

Internal consistency and exploratory factor analysis

The internal consistency was defined by a Cronbach’s alpha reported along with its one-sided (lower) 95% confidence limit (95% CL). The α≥0.9 was considered excellent, ≥0.8 good, ≥0.7 acceptable, ≥0.6 questionable, ≥0.5 poor, and <0.5 was considered unacceptable.^{20 21} Exploratory factor analysis (EFA) was used to approximate the construct structure of the JSS. The goal was to determine whether the JSS measures only one latent trait (=sleep disturbances) or if there are other possible significant latent variables affecting the results. The results were analysed both numerically and graphically. EFA (principal factors) was applied with a minimum eigenvalue for retention set at >1.0 (Kaiser’s rule). The varimax rotation was applied. Retained and excluded factors were also explored visually on a scree plot along with the parallel analysis.

Confirmatory factor analysis

The estimation procedure used the maximum likelihood method considering covariances supplied as input being unbiased. For simplicity, the estimates were reported in standardised form as correlation coefficients. A correlation <0.2 was considered poor, from 0.21 to 0.4 fair, from 0.41 to 0.6 moderate, from 0.61 to 0.8 substantial, and >0.8 perfect.²² In addition, the coefficients of determination were calculated to show the proportion of variance in common ‘sleep disturbances’ construct that can be explained by the items. Finally, the coefficient of determination for the entire model was calculated.

In order to assess how well the model matches the observed data, the root mean square error of approximation (RMSEA) was used as a primary index. First, the model fit was tested assuming there were no covariances between unique factors. After that, the modification indices suggested by the software were used to add covariance between factors (double-headed arrows in figure 1) one at a time, each time testing the RMSEA closeness to the value of <0.05, or, at least, <0.08—the threshold for accepting the model fit. Every insertion was considered plausible if it made logical sense and did not violate the assumption that the common and the unique factors are uncorrelated. After achieving the RMSEA value of <0.05, no further covariances were imputed. The goodness of fit was assessed using a Χ² test. Also, the Akaike’s and Bayesian information criteria (AIC and BIC), Comparative Fit Index and the Tucker-Lewis Index were calculated. The AIC and BIC were considered good if they were close to 1.0.

Figure 1.

Confirmatory factor analysis of Jenkins Sleep Scale. ‘ε’ circles represent a measurement error associated with an observed variable (variance that is predicted by the latent factor). Estimates placed between ε errors and observed variables represent the amount of variance in higher level data that can be explained by a particular variable.

The analyses were performed using Stata/IC Statistical Software: V.16 (StataCorp, College Station, Texas, USA).

Results

Of 81 136 respondents, 14 890 (18%) were men and 66 246 (82%) were women. Their mean age was 52.1 (SD=13.2) years, BMI 26.2 (SD=4.7) kg/m², physical activity 29.4 (SD=25.3) METs/week and alcohol consumption 49.7 (SD=90.9) g/week (equivalent to 5 units of alcohol per week). Of the respondents, 41 823 (52%) were sleeping 7 or less hours per night. The mean JSS total score was 6.4 (4.8) points.

The JSS demonstrated a substantial internal consistency with alpha 0.80 (lower 95% CL 0.80). The exploratory factor analysis resulted in one retaining factor with eigenvalue of 1.94 based on Kaiser criterion (table 1 and figure 2). Three other factors had eigenvalues between −0.03 and −0.18 and thus explained variance less than the observed variables. The parallel analysis of scree plot confirmed the unidimensional structure of JSS (table 2).

Table 1.

Exploratory factor analysis of loadings of the Jenkins Sleep Scale items

Jenkins Sleep Scale items	Factor #1	Uniqueness
Trouble falling asleep	0.62	0.61
Waking up but no trouble falling asleep again	0.72	0.48
Waking up and trouble falling asleep again	0.79	0.37
Waking up feeling tired	0.63	0.60

Figure 2.

Exploratory factor analysis of Jenkins Sleep Scale—scree plot with parallel analysis.

Table 2.

Parallel analysis for factor analysis (over 10 replications), eigenvalues

Factors	Factor analysis	Parallel analysis	Difference
1	1.94	0.01	1.93
2	−0.03	0.00	−0.03
3	−0.12	0.00	−0.11
4	−0.18	−0.01	−0.17

The confirmatory factor analysis showed that all four items positively correlated with a common factor explaining from 44% up to 61% of the variance of the common factor (table 3 and figure 1). The highest correlation 0.78 (r²=0.61) was observed for the third item ‘waking up and trouble falling asleep again’. Other items demonstrated similar and slightly lower correlations between 0.66 and 0.67 (r²=0.44–0.45). The model obtained a good fit after adding one covariance between second and third items: 0.26 (95% CI 0.25 to 0.28). After that, the RMSEA of the model was 0.03 (table 4).

Table 3.

Confirmatory factor analysis of Jenkins Sleep Scale—correlation between observed and predicted estimates (R) along with coefficient of determination (R²)

Estimates	Variance			R	95% CI		R2
	Fitted	Predicted	Residual		Lower	Upper
Trouble falling asleep	1.63	0.71	0.92	0.66	0.66	0.67	0.44
Waking up but no trouble falling asleep again	2.83	1.29	1.55	0.67	0.67	0.68	0.45
Waking up and trouble falling asleep again	2.40	1.47	0.92	0.78	0.78	0.79	0.61
Waking up feeling tired	2.28	1.02	1.26	0.67	0.66	0.68	0.45
Overall							0.78

Table 4.

Confirmatory factor analysis of Jenkins Sleep Scale—goodness of fit

Fit statistic	Value	Description
Likelihood ratio
Χ2	84.49	Model versus saturated
P value	<0.01
Population error
RMSEA	0.03	Root mean squared error of approximation
90% CI, lower bound	0.03
90% CI, upper bound	0.04
P value	1.00	Probability RMSEA ≤0.05
Information criteria
AIC	1 036 000	Akaike’s information criterion
BIC	1 037 000	Bayesian information criterion
Baseline comparison
CFI	1.00	Comparative Fit Index
TLI	1.00	Tucker-Lewis Index
Size of residuals
SRMR	0.01	Standardised root mean squared residual
CD	0.78	Coefficient of determination

Discussion

In this large cohort study, the JSS was found to be an internally consistent scale. EFA suggested the unidimensionality of the JSS. In addition, confirmatory factor analysis demonstrated a single-factor structure with only one mild aberration; the JSS item ‘waking up and trouble falling asleep again’ seemed to show higher coefficient of determination than any of the other items.

The generalisability of the results might be weakened by the gender disbalance of the studied cohort (women were predominated). This disbalance was due to the fact that fewer men are involved in the studied areas of public sector. Also, the mean age of study participants was 52 years and, therefore, the results described, in the first instance, people in the last third of their working life span. While it had been widely used for over two decades, the Finnish translation of JSS had never undergone a full linguistic validation process which might affect its equivalency with an English version. The response rate was 70% and there was no analysis of whether the non-respondents’ demographic characteristics might affect the results.

To our knowledge, this was the first study on the psychometrics of the Finnish translation of the JSS. The cohort of over 80 000 respondents represented a wide spectrum of occupations from managers to manual workers. However, the generalisability of the results might be compromised by the following aspects. The studied cohort was predominated by women. It has previously been stated that sleep problems are more common among women meaning that this inequality in gender distribution may overestimate the prevalence of sleep difficulties in the studied cohort.³ While overall working age could be understood as an age between early adulthood and the age of retirement, the mean age of the respondents was 52 years, covering mainly the last third of the working life span.

The results are in line with several previous studies that have found the JSS to be a unidimensional scale with excellent internal consistency.^{3 6 8} The Cronbach’s alpha of 0.8 seen in the present study was close to the estimates reported by previous research in both general population and populations of people with different health conditions.2 3 5–11 15 16 While the JSS has been studied by employing alpha and EFA by several studies, a confirmatory factor analysis has previously been used by only a single study (Tibubos et al). The correlations of four items with a common factor seen in study by Tibubos et al resembled the estimates observed in the present study with one exception. The present results demonstrated the greatest correlation for the item ‘waking up and trouble falling asleep again’, being in line with Tibubos et al, but, contrary to our result, item ‘waking up feeling tired’ had the smallest (out of four) estimate in their study. This difference might be explained by differences in the studied cohorts for example, in gender distribution and work status. Indeed, the present study represents a population that is probably healthier than general population. In addition, it is possible, though unlikely, that some differences might have occurred due to the linguistic variability between the two translations.

Further research may reveal more details on the JSS psychometrics, for example, its properties based on an item response theory analysis. Confirmatory factor analysis may especially be recommended for future research, as the knowledge on the JSS factor structure is still scarce.

Conclusions

The JSS was found to be a unidimensional scale with good internal consistency. As such, the JSS may be recommended as an easy-to-do questionnaire instrument for the studying of sleep difficulties in a healthy working-age population.