The Validity and Reliability of Turkish Version of the Jenkins Sleep Evaluation Scale in Rheumatoid Arthritis

Abstract

Objectives

This study aims to assess the validity and reliability of the Jenkins Sleep Evaluation Scale (JSS) when applied to a Turkish population with rheumatoid arthritis.

Patients and methods

The Turkish version of JSS (JSS-TR) was obtained after translation from English into Turkish, according to standard guidelines. The study included 61 patients of rheumatoid arthritis (13 males, 48 females; mean age 50.5 years; range 19 to 72 years) as defined by the American College of Rheumatology 2010 criteria. The internal consistency (Cronbach’s alpha) was assessed for reliability. Content and construct validity (convergent and divergent validities) were evaluated. The relationships between the JSS-TR and the Pittsburgh Sleep Quality Index, the Multidimensional Assessment of Fatigue scale, subgroups of the Nottingham Health Profile, and the Stanford Health Assessment Questionnaire were assessed for convergent validity. In addition, the relationships between the JSS-TR and age, disease duration, visual analog scale patient global score, and disease activity score 28 were assessed for divergent validity.

Results

The Cronbach’s alpha of JSS-TR was 0.80. All questions and the answer choices for the scale were understood well and related to some dimension of sleep demonstrating good content validity. The JSS-TR had good correlations with functional parameters (which are convergent), and poor or insignificant correlations with non-functional parameters (which are divergent). This implies that the JSS-TR had good construct validity in the context of this study. Overall, the JSS-TR had the best correlation with the Pittsburgh Sleep Quality Index (Spearman’s rank correlation coefficient=0.76).

Conclusion

The JSS-TR is a valid and reliable instrument. It is a simple and effective tool which can be used to evaluate sleep disturbances in rheumatoid arthritis patients in both daily practice and clinical research.

Introduction

Rheumatoid arthritis (RA) is a chronic, systemic, inflammatory disease which can affect joints, organs, and many bodily systems, including the eyes, kidneys, blood, and brain, as well as the integumentary, pulmonary, and cardiovascular systems.(1) Sleep disturbance is observed in 67% of RA patients.(2) Studies have shown that the sleep disturbances among RA patients include difficulties with falling and staying asleep, periodic limb movements during sleep, restless leg syndrome, various breathing disorders, and sleep fragmentation.(3-5)

The causes of sleep disturbance observed in RA patients are not well understood. However, there are some studies that showed that pain and depression may be causally related. Polysomnographic evaluation of RA patients has depicted fragmented sleep with numerous awakenings, as well as an enhanced chance of waking after the onset of sleep.(3,4) Abnormal electroencephalogram activities are also observed in these patients.(4) Poor sleep, fragmented sleep and other sleep disorders observed in RA patients may contribute to increased pain, fatigue, mood changes, and functional disability.(4,6-8)

There are many self-report questionnaires that can be used to evaluate the level of sleep disturbance in patients with RA.(9) However, most of these surveys are time consuming. And although these self-report studies are useful to researchers, using them to evaluate sleep disturbance in clinical practice is challenged. As a result, there is a need for simple instruments that can be used to evaluate sleep disturbance in RA patients.(10)

The Jenkins Sleep Evaluation Scale (JSS) is a well-known instrument which can be used to evaluate sleep disturbances in patients relative to the disease and the intensity of the disturbance they may experience. This sleep scale possesses both internal consistency and content validity, making it appropriate to be used in clinical trials and by occupational groups for health status surveillance.(11) It has been used to study several specific populations such as air traffic controllers, patients recovering from cardiac surgery, and patients with inexplicable chest pain.(11,12) As far as we know, the JSS has not been used to study RA patients. This simple and brief questionnaire can be applied to RA patients, potentially shortening patients’ evaluation period. Therefore, in this study, we aimed to assess the validity and reliability of the JSS when applied to a Turkish population with RA.

Patients and Methods

This is a cross-sectional, cross-cultural adaptation and validation study, including 61 patients of RA (13 males, 48 females; mean age 50.5 years; range 19 to 72 years). Inclusion criteria were age older than 18 years and meeting the 2010 American College of Rheumatology/European League Against Rheumatism classification criteria for RA. Patients with RA were recruited from Marmara University School of Medicine, Physical Medicine and Rehabilitation Department of Rheumatology Division Outpatient Clinics during May 2016 and July 2016. Sample size was calculated as 5-7 times of the number of items in the scale according to the literature.(13) Patients who were pregnant or had concomitant medical illnesses such as cancer, fibromyalgia, or psychiatric conditions such as psychosis or bipolar disorder were excluded. Demographic features such as age and sex were noted. The study protocol was approved by the Marmara University School of Medicine Ethics Committee. A written informed consent was obtained from each patient. The study was conducted in accordance with the principles of the Declaration of Helsinki.

Clinical data were obtained before the researchers applied the questionnaires. Findings from self-report questionnaires regarding pain, sleep, fatigue, functional disabilities, and health related quality of life were later reported.

Data regarding patients’ disease duration, resulting from their visual analog scales (VAS) for pain and VAS for patient global scales, were obtained. Disease activity score 28 (DAS28) was also implemented as a disease activity measurement.(14)

Two self-report questionnaires were used to assess patients’ sleep dysfunction. The Pittsburgh Sleep Quality Index (PSQI) assesses sleep quality over the past month.(15) It consists of 19 questions that are answered by the patient and five questions that are answered by the patient’s bed partner or roommate. These last five questions are only used for clinical purposes, and are not used for the patient’s overall sleep quality score. All 19 self-rated questions assess sleep quality, including estimates of sleep duration, latency, and of the frequency and severity of specific sleep-related problems. The questions are categorized into seven, subjective component scores. The seven component scores are then added to give a global PSQI score, which has a range of 0-21; higher scores indicate lower sleep quality. A total score >5 indicates poor sleep quality. The seven components include sleep quality, sleep latency, sleep duration, habitual sleep efficiency, sleep disturbances, use of sleeping medications, and daytime dysfunction. The PSQI was culturally adapted and validated in Turkish.(16)

The JSS is a research tool that assesses sleep disturbances for multiple diseases.(11) It was developed and validated by the analysis of 500 patients, who had had either coronary bypass surgery or cardiac valve surgery, in order to investigate sleep problems that may increase cardiovascular risk. The JSS is a four-item questionnaire that evaluates the number of days in the past month in which the patient has had sleeping problems. The respondents answer the questions using a six-point Likert-type scale (0=notatall,1=1to3days,2=4to7days, 3=8to14days,4=15to21days,5=22to 31 days). Total scores range from 0-20, and higher scores indicate a greater number of sleep problems. Sleep disturbances are considered when the mean score is equal or greater than 2, corresponding to at least one troubled night per week.(17)

The Multidimensional Assessment of Fatigue (MAF) scale consists of 16 items and evaluates fatigue using four dimensions: severity, distress, impact on activities of daily living, and timing.(18) Scores range from 0 (no fatigue) to 50 (severe fatigue). Turkish version of MAF was used as an assessment tool.(19)

The Stanford Health Assessment Questionnaire (HAQ) is a tool used frequently to evaluate physical functioning in patients with RA.(20) The HAQ is a 20-item questionnaire that has eight dimensions: dressing, rising, eating, walking, hygiene, reach, grip, and usual daily activities. Participants rate each activity on a 0-3 scale (0= without any difficulty, 1= with some difficulty, 2= with much difficulty, 3= unable to do). The maximum score in each dimension is added and then divided by eight to receive the “mean score”. The range of scores is 0-3. Its adapted Turkish format is also a reliable, validated questionnaire that is widely used in researches.(21)

The Nottingham Health Profile (NHP) has six health domains which include physical mobility (NHP-PM, eight items), pain (NHP-P, eight items), sleep (NHP-S, five items), social isolation (NHP-SI, five items), energy level (NHP-EL, three items) and emotional reactions (NHP-ER, nine items).(22) Patients answer yes or no to each item, and all items have specific weighted values. The sum of the items’ values within each domain is between 0-100. A higher score indicates worse quality of life. This has been validated in the Turkish population.(23)

The JSS was translated according to standard guidelines after permission was obtained from F. Guillemin.(24-26) Forward translation was achieved by two bilingual speakers who were separated during the translation process. A consensus version was derived after discussion between translators and authors of this study. Backward translation was achieved by two other bilingual also native English speakers who had not seen the original questionnaire. After translation, a committee composed of two rheumatologists, a physiotherapist, and four interpreters discussed the further elimination of any inconsistencies. The pre-final version of the JSS-Turkish population (JSS-TR) was obtained after a few minor changes done by the committee. In summary, the original meaning of the questionnaire remained unaltered.

The pre-final version of the JSS-TR was tested on 15 RA patients with cognitive interviews.(27) Each patient filled out the JSS-TR and was asked questions about what the patient thought was meant by each questionnaire item and their chosen response. These interviews revealed that all of the questions were accurately understood by the patients and that there were no ambiguous questions or missing data as a result of the translation. Consequently, no further adaptations were needed.(28) The JSS-TR is shown in the Appendix.

Statistical analysis

The data were analyzed using the IBM SPSS version 20.0 software for Windows (IBM Corp., Armonk, NY, USA). All descriptive data were either expressed as means, with standard deviations, or as percentages. The cutoff value of the JSS-TR for good and poor sleepers was calculated using the receiver operating characteristic (ROC) curve analysis.

The internal consistency of the JSS-TR was evaluated by Cronbach’s alpha coefficient (>0.70 was considered to be significant). Psychometric methods were used to confirm the validity of the JSS-TR in this population.(29) The content validity of the JSS-TR was assessed via cognitive debriefing interviews with 15 RA patients. The patients were asked to make note of specific items if any question was ambiguous or difficult to understand.

The measures that had direct implications for sleep were assessed for convergent validity. The relationship of the JSS-TR with the PSQI, the MAF scale, subgroups of the NHP (such as NHP-EL, NHP-P, NHP-ER, NHP-S, NHP-SI, NHP-PM), and the HAQ were assessed for convergent validity.

The relationship of the JSS-TR with age, disease duration, VAS patient global, and DAS28, which were not directly related to sleep, were assessed for divergent validity. Spearman’s rank correlation coefficient (rho) was used to assess the relationship between quantitative parameters. Correlation coefficients >0.50, 0.35-0.50 and <0.35 were considered strong, moderate or weak, respectively.(30) P<0.05 was accepted as statistically significant.

Appendix. Turkish version of Jenkins Sleep Evaluation Scale.

Results

The mean age at disease onset and disease duration of those who participated in the study was 50.5±11.4 years and 7.9 years, respectively. Demographic and clinical characteristics of the patients were outlined in Table 1.

Table 1. Demographic and clinical characteristics of patients (n=61).

	Mean±SD	Median	Min-Max
Age (year)	50.5±11.4
Disease duration (year)	7.9±76
Sedimentation	29.6±16.1
Disease activity score 28	3.0±1.0
Visual analog scale-patient global		5	0-10
Visual analog scale-pain		5	0-10
SD: Standard deviation; Min: Minimum; Max: Maximum.

Approximately one half of the patients (50.8%) had sleep problems. The percentages of sleep disturbance in male and female patients were almost identical (male: 53.8%, female: 50.0%). The floor and ceiling effects of patients were 11.5% and 6.5%, respectively. Mean values with standard deviations, as well as maximum and minimum scores of the measurements were depicted in Table 2. The cutoff value that differentiates between poor and good sleepers for the JSS-TR was determined to be 6.5. The overall sensitivity and specificity values of the JSS-TR, based on the cutoff point of 6.5, were 83% and 79%, respectively. An analysis of the ROC curve to find out the cutoff value was depicted in Table 3 and Figure 1.

Table 2. Descriptive data of measures used for validation of Jenkins Sleep Evaluation Scale-Turkish population.

	Mean±SD
Jenkins Sleep Evaluation Scale-Turkish population	7.5±6.1
Pittsburgh Sleep Quality Index-total	6.5±4.4
Nottingham Health Profile-Energy level	56.6±35.6
Nottingham Health Profile-Pain	44.6±32.9
Nottingham Health Profile-Emotional reaction	37.2±30.9
Nottingham Health Profile-Sleep	35.2±31.9
Nottingham Health Profile-Social isolation	21.3±25.2
Nottingham Health Profile-Physical mobility	28.3±22.3
Multidimensional Assessment of Fatigue	25.3±12.1
Stanford Health Assessment Questionnaire	0.7±0.7
SD: Standard deviation.

Table 3. Receiver operating characteristic curve analysis.

Area	Standard error	Significance	95% Confidence interval
			Lower bound	Upper bound
0,9	0,35	0	0,83	0,97

Figure 1. Receiver operating characteristic (ROC) curve analysis of Turkish version of Jenkins Sleep Scale.

The internal consistency (Cronbach’s alpha) of the JSS-TR was 0.80, demonstrating that the scale is reliable. Patients found that all questions were related to some dimension of sleep. In these interviews, patients were able to answer all of the questions easily. There were no missing data. According to these results, JSS-TR was found to have content validity. The JSS-TR had good correlations with functional parameters (which are convergent), and poor or insignificant correlations with nonfunctional parameters (which are divergent) (Table 4). This means that JSS-TR had good construct validity. Moderate-strong correlations between the JSS-TR and the subsets of the NHP (NHP-EL, NHP-P, NHP-ER, NHP-SI, and NHP-PM), subsets of the PSQI, and the MAF were detected. Among the subsets of NHP scale, the strongest correlation was found with NHP-ER (rho=0.60) component. The JSS-TR had the strongest correlation with the PSQI in comparison to other scales (rho=0.76).

Table 4. Comparison of Spearman’s rank correlation coefficient of Jenkins Sleep Evaluation Scale-Turkish population with other scales of rheumatoid arthritis.

	Spearman’s (rho)	Significance (p)Convergent validity of JSS-TR
Pittsburgh Sleep Quality Index-subjective sleep quality	0,49	<0.0001
Pittsburgh Sleep Quality Index-sleep latency	0,63	<0.0001
Pittsburgh Sleep Quality Index-sleep duration	0,38	0,002
Pittsburgh Sleep Quality Index-habitual sleep efficiency	0,4	0,001
Pittsburgh Sleep Quality Index-sleep disturbances	0,28	0,028
Pittsburgh Sleep Quality Index-use of sleeping medications	0,32	0,01
Pittsburgh Sleep Quality Index-daytime dysfunction	0,28	0,028
Pittsburgh Sleep Quality Index total	0,76	<0.0001
Nottingham Health Profile energy level	0,56	<0.0001
Nottingham Health Profile pain	0,57	<0.0001
Nottingham Health Profile emotional reactions	0,6	<0.0001
Nottingham Health Profile sleep	0,45	<0.0001
Nottingham Health Profile social isolation	0,55	<0.0001
Nottingham Health Profile physical mobility	0,56	<0.0001
Health Assessment Questionnaire	0,55	<0.0001
Multidimensional Assessment of Fatigue	0,47	<0.0001
Divergent validity of JSS-TR
Age	0,21	0,113
Disease duration	0,22	0,086
Visual analog scale-patient global	0,38	0,002
Disease activity score 28	-0,044	0,73
Rho: Spearman’s rank correlation coefficient; JSS-TR: Jenkins Sleep Evaluation Scale-Turkish population.

Discussion

This study sought to determine the validity and reliability of the JSS in a Turkish population using psychometric methods. The mean duration of patients filling out the JSS-TR was 1.5 minutes. The internal consistency of the JSS-TR was 0.80.

Rheumatoid arthritis patients are a population at risk for sleep disorders. Sleep and fatigue are emphasized at different Outcome Measures in Rheumatology (OMERACT) meetings as an important issue which influences the quality of life and well-being of RA patients. Validation of existing sleep scales in RA was recommended at both OMERACT 9 and 10 meetings.(31) Numerous tools and outcome measures have been designed to evaluate sleep disturbance in various populations. However, simple and user-friendly evaluations are needed to optimize clinical practice for RA patients. The JSS is a useful and simple measurement which has been validated in several other populations before it was applied to patients with RA.

Sleep problems were found in half of our RA patients, which is consistent with previous studies.(7,32) The internal consistency of the JSS-TR is higher than the internal consistency of the original JSS (0.63 for cardiac surgery recovery patients and 0.79 for a group of air traffic controllers).(11) The JSS has also been translated into Portuguese and used to study airline pilots. In this study using the Portuguese version, the Cronbach’s alpha was 0.84, which is similar to our results.(33) In other studies that have used the JSS for different patient groups, the Cronbach’s alpha was 0.78 for middle-aged females, 0.80 for patients with unexplained chest pain, and 0.77 for Japanese and British civil servants.(10,12,34) The internal consistency of the JSS-TR (0.80) shows that the questionnaire has good reliability.

The most significant correlation was present between the JSS-TR and the PSQI. This result was expected since both of these questionnaires are sleep outcome measurements. Previous studies used the PSQI as a sleep instrument tool for patients with RA, along with use of confirmatory factor analysis.(5,32,35) The JSS-TR had a moderate to strong correlation with most of the subsets of the NHP and the PSQI. The strongest one was with the NHP emotional reactions and pain subsets. This finding is consistent with the data from the preexistent literature, since pain and depression contribute to sleep loss in RA patients. Moderate correlations with the MAF and the HAQ show that sleep disturbance in RA patients can also decrease their pain threshold, quality of life, cause fatigue, depression, and increase functional disability. This is consistent with the findings of previous studies.(7,32) The findings of an earlier study, investigating the relationship between patients’ sleep quality and functional disability, showed that the PSQI is significantly associated with higher levels of depression (rho=0.52, p<0.001), fatigue (rho=-0.48, p<0.001) and pain (rho=0.48, p<0.001). These relationships have an indirect effect on patients’ greater functional disability (rho=0.41, p<0.001).(7) Furthermore, a study investigating 106 patients with RA indicated that pain and depressive symptoms contribute to sleep disturbance.(29) Indeed, pain and depression contribute to sleep disturbance reciprocally.(36-39)

Strong correlation found was between the JSS-TR and sleep latency of the PSQI. Increased sleep latency is one of the many sleep problems observed in RA patients.(40) The JSS-TR had poor or insignificant correlations with non-functional parameters such as age, sex, disease duration, and the DAS28, revealing its discriminative properties. In previous studies, no significant or weak correlations emerged between the PSQI results and sex, disease duration, or age, supporting the results of this study.(7,32) This study revealed only one divergent validity parameter: the VAS-patient global score had correlations with the JSS-TR. This may be due to the fact that RA patients with high levels of pain are more likely to have sleep disturbances.(32,36)

This study has some limitations. Test and retest reliability could be performed within three to four-week intervals. Also, further studies are required on responsiveness and sensitivity to change.(6)

In conclusion, the JSS-TR has good convergent and discriminant validities. Furthermore, JSS-TR has adequate sensitivity and specificity in its application. Scoring of the JSS-TR is extremely simple and it can be preferable to other sleep scales such as the PSQI. Adaptation of this simple sleep measurement tool will enable researchers, medical professionals, and occupational groups to use it easily in the future for epidemiological studies and in clinical settings.