Predicting SF-6D utility scores from the Neck Disability Index and Numeric Rating Scales for Neck and Arm Pain

Abstract

Study Design

Cross-sectional cohort

Objective

This study aims to provide an algorithm estimate SF-6D utilities using data from the NDI, neck pain and arm pain scores.

Summary of Background Data

Although cost-utility analysis is increasingly used to provide information about the relative value of alternative interventions, health state values or utilities are rarely available from clinical trial data. The Neck Disability Index (NDI) and numeric rating scales for neck and arm pain, are widely used disease-specific measures of symptoms, function and disability in patients with cervical degenerative disorders. The purpose of this study is to provide an algorithm to allow estimation of SF-6D utilities using data from the NDI, and numeric rating scales for neck and arm pain.

Methods

SF-36, NDI, neck and arm pain rating scale scores were prospectively collected pre-operatively, at 12 and 24 months post-operatively in 2080 patients undergoing cervical fusion for degenerative disorders. SF-6D utilities were computed and Spearman correlation coefficients were calculated for paired observations from multiple time points between NDI, neck and arm pain scores and SF-6D utility scores. SF-6D scores were estimated from the NDI, neck and arm pain scores using a linear regression model. Using a separate, independent dataset of 396 patients in which and NDI scores were available SF-6D was estimated for each subject and compared to their actual SF-6D.

Results

The mean age for those in the development sample, was 50.4 ± 11.0 years and 33% were male. In the validation sample the mean age was 53.1 ± 9.9 years and 35% were male. Correlations between the SF-6D and the NDI, neck and arm pain scores were statistically significant (p<0.0001) with correlation coefficients of 0.82, 0.62, and 0.50 respectively. The regression equation using NDI alone to predict SF-6D had an R² of 0.66 and a root mean square error (RMSE) of 0.056. In the validation analysis, there was no statistically significant difference (p=0.961) between actual mean SF-6D (0.49 ± 0.08) and the estimated mean SF-6D score (0.49 ± 0.08) using the NDI regression model.

Conclusion

This regression-based algorithm may be a useful tool to predict SF-6D scores in studies of cervical degenerative disease that have collected NDI but not utility scores.

INTRODUCTION

Given the limited resources available, there has been an increasing use of cost-utility analysis to inform decisions regarding the relative value of different treatment interventions [1,2]. The most widely used measure for cost-utility analysis is the Quality-Adjusted Life-Year (QALY), wherein treatment outcomes are measured using a single utility score, anchored at 0 for death and 1 for perfect health, depending on the preference for a certain health state [3,4]. There are two main approaches in use to obtain utility scores. The direct approach uses preference elicitation methods such as standard gamble, time tradeoff, and visual analogue scale ratings to value health states of a representative sample. Alternatively, multi-attribute systems can be used, such as the Quality of Well Being Scale [5], the EuroQOL EQ-5D [6], SF-6D [7] or the Health Utilities Index HUI [8]. This approach includes the administration of a questionnaire to classify health states, and the application of an algorithm to estimate utilities. Most studies in the field of spine surgery use a generic measure of health such as the Short Form-12 or Short Form-36 (SF-36) [9] and a disease specific measure such as the Oswestry Disability Index [10] for low back disorders, the Scoliosis Research Society-22 [11] for spinal deformity or the Neck Disability Index (NDI) for cervical spine disorders to measure treatment outcomes [12]. Unless the research is specifically designed as a cost-utility study, preference-based measures are not usually administered as these may not be practical in the clinical setting due to lack of resources as well as an increased respondent burden.

Several studies have shown that health state values can be predicted using data from non-preference-based instruments in order to perform cost-utility analysis using available clinical research data [13–34]. The purpose of this study is to determine whether the widely used disease-specific measures for cervical spine disorders, the NDI, numeric rating scales for neck and arm pain may accurately predict SF-6D utility scores. Predicting SF-6D scores from these disease-specific measures may allow researchers to perform cost-utility analysis in studies of cervical degenerative disease that have collected NDI but not utility scores.

METHODS

Two separate cohorts were used in this study, a Development Sample to produce a regression model to calculate the SF-6D from the NDI; and a Validation Sample to test the external validity of the regression model derived from the Development Sample.

The Development Sample

As part of a multi-center database; the Cervical 2000, the Medical Outcome Study Short Form-36 (SF-36) [9], the Neck Disability Index (NDI) [12], neck pain scores (0 to 10) and arm pain scores (0 to 10) [35] were prospectively collected pre-operatively and at 12 and 24 months post-operatively in 2080 patients undergoing cervical fusion for degenerative disorders from January 2002 to June 2007.

The Validation Sample

To validate the estimation model, we used data from a second separate single center database which included patients undergoing cervical fusion for degenerative disorders from May 2008 to August 2009. The SF-36, NDI, neck and arm pain scores, were prospectively collected pre-operatively.

Outcome Measures

NDI

The Neck Disability Index is a 10 item self-administered questionnaire measuring disability in patients with neck pain. Each item is scored from 0 to 5 for a maximum score of 50 [12]; the higher the score, the greater the disability. Some authors may use a percentage score, in which case the range of scores would be from 0 to 100. In the current study, the original scoring system with a maximum score of 50 was used. Individuals with a score of 0 to 4 have no disability; 5–14 have mild disability; 15–24 have moderate disability; 25–34 have severe disability and over 34 are completely disabled.

SF-6D [36]

The SF-6D is derived from the SF-36 and is composed of six multi-level dimensions of health. It was constructed from a sample of 11 items selected from the SF-36 to minimize the loss of descriptive information and defines 18,000 health states. A selection of 249 states defined by the SF- 6D have been valued by a representative sample of the UK general population (n=611) using the standard gamble valuation technique. A regression model was used to estimate health state values for each of the health states defined by the SF-6D using the values for the subset of 249 health states. This algorithm is used to convert SF-36 data at the individual level to a preference-based index score or health state value.

Neck and Arm Pain Scores

These scores are based on a numeric rating scale for these two items, one each for neck pain and arm pain, on the survey that asks: “On a scale from 0 to 10, mark your level of neck (arm) pain discomfort, with 0 being none and 10 being unbearable.”[35]

Statistical Analysis

We calculated Spearman coefficients for paired observations from each time point between SF-6D and NDI and between SF-6D and neck and arm pain scores. We developed a Linear Regression model to predict SF-6D scores from the NDI, neck and arm pain scores. The R² and root mean square error (RMSE) estimate was used to assess the ability of the model to predict SF-6D scores.

To examine the accuracy of our prediction model, we applied the algorithm to data from a separate, independent database of patients undergoing cervical fusion surgery for degenerative disorders. These were paired observations of SF-6D, NDI and neck and arm pain scores. We estimated the SF-6D score for each subject using the regression algorithm and compared it to the actual SF-6D calculated from responses to the SF-36 in this independent dataset. SF-6D scores, or health state values for each individual paired observation were compared using paired t-tests.

RESULTS

There were 2118 patients in the development sample, of whom 33% were male, and the mean age was 50.4 ± 11.0 years. In the validation sample, consisting of 395 patients, the mean age was 53.1 ± 9.9 years and 35% were male. There were no statistically significant differences detected in the age or gender composition between the two samples. Tables 1 and 2 provide a summary of the outcome measures for the two samples. Correlations between the SF-6D and the NDI, neck and arm pain scores were statistically significant (p<0.0001). There was a strong correlation between SF-6D and NDI (r=0.82) and moderate correlations between SF-6D and neck pain scores (r=0.62), and between SF-6D and arm pain scores (r=0.50). Table 3 shows the mean SF-6D, neck and arm pain scores across the different Neck Disability Categories.

Table 1.

Summary statistics of all paired observations from the Development Sample

	SF-6D	NDI	Neck Pain	Arm Pain
Mean	0.53	23.44	6.01	5.16
SD	0.09	10.61	2.83	3.11
Minimum	0.30	0	0	0
Maximum	0.85	48	10	10

Table 2.

Summary statistics all paired observations from the Validation Sample

	SF-6D	NDI	Neck Pain	Arm Pain
Mean	0.49	28.54	6.74	5.90
SD	0.08	8.57	2.28	2.91
Minimum	0.32	0	0	0
Maximum	0.81	47	10	10

Table 3.

Mean scores for SF-6D, Neck Pain and Arm Pain across the different Neck Disability categories

Neck Disability Category	Mean SF-6D (SD)	Mean Neck Pain (SD)	Mean Arm Pain (SD)
None (0–4)	0.68 (0.04)	0.90 (1.24)	0.82 (1.38)
Mild Disability (5–14)	0.63 (0.04)	3.03 (2.18)	2.76 (2.65)
Moderate Disability (25–24)	0.56 (0.05)	5.55 (2.18)	4.65 (2.79)
Severe Disability (25–34)	0.49 (0.07)	7.43 (1.67)	6.32 (2.48)
Complete Disability (>35)	0.42(0.07)	8.57 (1.28)	7.47 (2.22)

The regression equation that defines the relationship between SF-6D and NDI in the development sample is:

$$\text{SF}-6\text{D}=0.78219-0.01005(\text{NDI})$$

The correlation coefficient of this equation is also 0.81, similar to the correlation coefficient of the regression equation that included neck and arm pain. The equation accounted for 66% of the variability of SF-6D with an RMSE of 0.056 (Table 4). This linear relationship can be seen in the SF-6D/NDI plot (Figure 1).

Table 4.

Comparison of Various Models Predicting the SF-6D

Stepwise Regression Results	R	R2	Adjusted R2	RMSE
NDI Alone	0.81	0.66	0.66	0.055536
NDI plus Neck Pain	0.81	0.66	0.66	0.055470
NDI, Neck Pain, Arm Pain	0.81	0.66	0.66	0.055510

Figure 1.

The results of the confirmatory analysis using the regression equation to estimate the SF-6D showed that the estimated SF-6D using the regression equation with NDI alone was very similar to the actual SF-6D in the validation sample. There was no statistically significant difference (p=0.961) between the actual SF-6D (0.49 ± 0.08) and the estimated SF-6D (0.49 ± 0.08) using the NDI regression model in the independent dataset (Figure 2).

Figure 2.

DISCUSSION

This is the first study to our knowledge that has provided a method to estimate health state values for cost-utility analysis from the Neck Disability Index. The results of this study of 2080 patients with cervical degenerative disorders show that the relationship between the SF-6D and the NDI is sufficiently robust to allow a valid estimation of SF-6D scores from the NDI using a regression equation. This is supported by the finding that the actual utility scores in an independent validation sample were similar to the estimated scores using the regression equation. Also, the RMSE value shows that the model defined by the regression model does well in predicting the SF-6D utility scores. The calculated R² in the model is greater than previous values reported for other models estimating state values from disease-specific measures [37]. The use of this regression model will allow researchers in the spine surgery field to use existing data to estimate utility scores in order to perform cost-utility analyses.

Although administering preference-based measures prospectively would be ideal to determine health state values in clinical studies, there are many instances when such data are not collected at the time of the clinical study. The use of the regression equation to estimate societal health state values using the NDI may facilitate the conduct of cost-utility analyses. Using the simple, 10-item NDI as an estimate of a health state may also reduce respondent and administrative burden.

Despite the advantages of using a regression-based model to estimate health state values from a disease-specific measure, there are limitations to this method. Previous studies have shown that health state estimated from mapping may have limited variability and measurement precision. [5, 8, 32]. Numerous studies have also shown that different approaches of measuring or converting health states into utility values may produce different estimates [5, 8, 32, 37]. However, our model produced the same estimated mean health state value as the mean health state value for the actual data in the validation group, indicating that it may be useful for cost-utility analysis.

The NDI has been found to have acceptable psychometric properties and is widely used as a measure of disability in patients with cervical spine disorders [12]. The SF-6D and NDI measure several domains that may be similarly affected by cervical spine pathology. The “Social Function” domain in the SF-6D and “Recreation”, “Reading” and “Driving” items in the NDI both measure the effect of neck pain on the individual’s social life; the “Role limitation” domain in the SF-6D and the “Personal Care” and “Work” items in the NDI address the impact of neck pain on the patient’s work and daily activities’ the “Bodily Pain” domain in the SF-6D and the “Pain Intensity” and “Headache” items in the NDI reflects the intensity and frequency of the patient’s pain symptoms.

The neck and arm pain scores showed only a moderate correlation with the SF-6D. This may be expected as, in contrast to the SF-6D and the NDI which measures how cervical pathology affects the patients’ quality of life, both the neck and arm pain scores address only pain intensity. The “Bodily Pain” domain in the SF-6D and the “Pain Intensity” item in the NDI may also make the use of neck and arm pain scores in the regression equation redundant. This lack of correlation between the SF-6D and neck and arm pain scores may be seen as an advantage, as the range of values used by clinicians and researchers vary. Some studies use a 0 to 10 scale while others may use a 0 to 100 scale. Other studies may include both the intensity and frequency of the pain, such that a pain scale is additive, 0 to 10 for intensity plus 0 to 10 for frequency, giving a scale range of 0 to 20. If there was a strong correlation between the SF-6D and neck and arm pain scores, such that they cannot be removed from the regression equation, it would have limited the general applicability of the regression equation.

Using the regression model to estimate SF-6D from the NDI in an independent dataset demonstrated that the estimated SF-6D was very similar to the actual SF-6D. There was no statistically significant difference between the actual SF-6D and the estimated SF-6D using the NDI regression model. The SF-6D should ideally be directly transformed from either the SF-36 or SF-12 [7, 38], as the regression model gives only an estimation of the actual SF-6D. This implies the presence of error and variation. However when SF-12 or SF-36 data are missing, or when items necessary to transform the SF-12 or SF-36 were not completed, the NDI regression model can be used to estimate SF-6D utility scores in studies of cervical degenerative disease. This data could then be used to develop economic models that assess cost burdens and effectiveness of treatments of cervical spine diseases.