Correlation between NDI, PROMIS and SF-12 in cervical spine surgery

Abstract

BACKGROUND CONTEXT:

As the focus in spine surgery has shifted from radiographic to patient-centric outcome, patient-reported outcomes measures (PROMs) are becoming increasingly important. They are linked to patient satisfaction, and are used to assess healthcare expenditure, determine compensation and evaluate cost-effectiveness. Thus, PROMs are important to various stakeholders, including patients, physicians, payers, and healthcare institutions. Thus, it is vital to establish methods to interpret and evaluate these outcome measures.

PURPOSE:

To evaluate the correlation between Neck Disability Index (NDI), Patient Reported Outcome Measurement Information System Physical Function (PROMIS-PF) and Short Form-12 Physical Health Score (SF-12 PHS) in cervical spinal surgery in order to determine the validity of PROMIS-PF in these patients.

STUDY DESIGN/SETTING:

Retrospective review of prospectively collected data.

PATIENT SAMPLE:

Consecutive patients who underwent cervical surgery for degenerative spinal pathology with a minimum of 3 months follow-up.

OUTCOME MEASURES:

Self-reported measures that is, PROMs including NDI, PROMIS-PF, and SF-12 PHS.

METHODS:

No funding was received for this study. The authors report no relevant conflict of interest. PROM collected preoperatively and at each follow-up were analyzed using Pearson product-moment correlation.

RESULTS:

Of the 121 patients included, 66 underwent anterior cervical discectomy and fusion, 42 cervical disc replacement, 13 posterior cervical decompression with or without fusion. A statistically significant improvement was achieved in all PROMs by 6 weeks and maintained at 1 year. Furthermore, the percentage of patients achieving an improvement greater than minimum clinically important difference was similar for NDI and PROMIS-PF, particularly at a follow-up of 3 months or more. A statistically significant negative correlation was seen between NDI and PROMIS-PF, which was moderate preoperatively and in the early postoperative period (r=−0.565 to −0.600), and strong at 3 months or longer follow-up (r=−0.622 to −0.705). A statistically significant, negative correlation was also seen between SF-12 PHS and NDI, which was moderate preoperatively and at 6 weeks (r=−0.5551 to −0.566); and strong at all other time-points (r=−0.678 to −0.749). There was a statistically significant positive correlation between SF-12 PHS and PROMIS-PF, which was strong to very-strong at all time-points (r=0.644–0.822), except at 2 weeks (r=0.570).

CONCLUSIONS:

Although NDI and SF-12 have been used for several years, PROMIS is a new outcome measure that is increasingly being implemented. The results of our study demonstrate the convergent and discriminant validity of PROMIS-PF, supported by the strong correlation between SF-12 PHS and PROMIS-PF at all time-points and the moderate correlation between NDI and PROMIS-PF preoperatively and in the early postoperative period, respectively. Thus, while PROMIS-PF may not be a good surrogate for disease-specific outcome measures, it may extend value as a precise and efficient general health tool.

Introduction

The unsustainable rate of growth of US healthcare expenditure, coupled with evidence that a significant portion of this expenditure is cost-ineffective [1] has led to emphasize on value-based care. Value in healthcare, from the global perspective, is determined by measuring benefit to the patient (patient perspective) per dollar spent (payor and society perspective) [2]. Thus measuring and tracking patients’ health with the use of PROMs is critical in enhancing the value of spine care and promoting evidence-based decision making.

Patient-reported outcomes measures (PROMs) are linked to patient satisfaction [3–5] and are increasingly being used to evaluate healthcare expenditure, assess cost-effectiveness of interventions [6,7] and determine compensation [8,9]. In addition, PROMs have also been used to identify mismatches in patient/provider perception of health status [9], and in conjunction with other parameters, risk-stratify patients [10] and predict outcomes [11,12]. Various governmental agencies in a number of countries have made the collection and research of PROMs a priority [13,14]. Thus, it is evident that PROMs are important to various stakeholders, including patients, physicians, payers, and healthcare institutions. As a result, it is vital to evaluate the validity of these outcome measures and determine their clinical relevance.

A number of PROMs are available for use in the spine population. Neck Disability Index (NDI), which measures functional disability due to cervical spine pathology, has become one of the principal condition-specific outcome measures in spine patients since its introduction in 1991 [15]. Despite its widespread adoption, studies on the psychometric properties of the NDI have shown varying results, with one systematic review of 36 articles reporting acceptable reliability but a large variability in intraclass correlation coefficients and inconsistency regarding the minimum clinically important difference (MCID) [16]. Despite these limitations, NDI continues to be the legacy PROM in these patients.

Short Form instruments, such as SF-36, SF-12, and SF-6D, which were created based on the RAND Corporation’s 1989 Medical Outcomes Study, measure a number of general health domains, and can be consolidated into two summary scores, a physical health score (PHS) and a mental health score. They are often used as an indicator of overall health and are not specific to a particular condition or intervention.

Patient-Reported Outcomes Measurement Information System (PROMIS), whose development was sponsored by the National Institutes of Health, uses computer adaptive testing (CAT) based instruments to measure health outcomes from the patient’s perspective. The CAT allows for far fewer questions to be asked of the patient, and subsequent questions are algorithmically chosen based on prior responses. Additionally, no more questions are asked once a high degree of measurement certainty is reached, with most patients reaching this threshold in 4–6 questions.

When completing questionnaires, if patients lose focus or energy or consider a question irrelevant to their symptoms they may alter their responses, skip questions, or stop completing questionnaires [17–20], as can occur when using the NDI. Thus utilizing CAT to administer the PROMIS questionnaire can limit patient burden, avoid missed questions, and may be less prone to floor and ceiling effects [17,19]. Although NDI historically has been the most commonly used PROM to evaluate cervical spinal disorders, PROMIS is a relatively new outcome measure that is increasingly being implemented. Thus, it is important to assess the true value and utility of this new outcome measure in common cervical spinal condition.

The purpose of this study is to assess the correlation between NDI, Patient-Reported Outcome Measurement Information System Physical Function (PROMIS-PF) and Short Form-12 Physical Health Score (SF-12 PHS) in patients undergoing cervical spinal surgery in order to determine the validity of PROMIS-PF in these patients.

Material and methods

Study design

Retrospective review of prospectively collected data from a single surgeon surgical database was performed.

Patient population

Patients who underwent cervical surgery for degenerative conditions of the spine were included. Patients who were less than 3 months postsurgery at the time of data analysis and those with a diagnosis of scoliosis, cancer, trauma, fracture, or infection were excluded. Additionally, patients who were non-English speaking were excluded.

Extracted data

Various PROMs, including NDI, SF-12, and PROMIS-PF were administered pre-operatively and at each follow-up as a part of the surgeon’s standard of care.

PROMs were administered by research personnel not directly involved in patient care. All PROMs, except PROMIS-PF were administered using paper forms, electronic forms sent via an electronic medical record – linked patient portal or a combination of the two, depending on patient preference. PROMIS-PF, which is a computer-adaptive questionnaire was administered only using REDCap (Research Electronic Data Capture) [21,22], a HIPAA-compliant web-based platform for electronic data capture. Electronic forms and a REDCap link were sent to patients 1 week before their clinic visit. If the patient did not complete these PROMs before the visit, NDI, VAS neck and arm pain, and SF-12 were administered in the waiting area using paper forms or electronic tablets, depending on patient preference, and PROMIS-PF was administered on a tablet, computer or mobile phone via a REDCap link. If a patient did not follow-up in clinic, PROMs were not collected.

The use of REDCap for this project was supported by the CTSC GRANT UL1 TR002384.

PROMs data collected preoperatively and at 2 weeks, 6 weeks, 3 months, 6 months, and 1 year postoperatively were extracted for analysis.

The NDI contains 10 topics relating to various aspects including intensity of pain, ability to care for oneself, interference with work, sleep quality, etc each followed by 6 statements describing different scenarios. The patient is instructed to select the statement which most closely resembles their situation. Each question is scored on a scale of 0–5. The scores for all questions answered are summed and multiplied by two to obtain the index (range 0–100). Zero is equated with no disability and 100 is the maximum disability possible [15].

The SF-12, a measure of Health-related Quality of Life, can be used across age, disease, and treatment groups. It is a shortened version of the SF-36 and contains 12 questions, each with two to five answer options. The patient is instructed to select the option most applicable to them. The SF-12, like the SF-36 covers eight dimensions – general health, physical functioning, role-physical, bodily pain, general health, vitality, social functioning, role-emotional, and mental health. A PHS and a mental health score, each ranging between 0 and 100, can be calculated using scoring algorithms. Scores are calibrated so that 50 is the average score or norm, and lower scores represent a worse and higher scores represent a better Health-related Quality of Life.

For spine patients, PROMIS-PF is the most relevant domain, which consists of a question bank of 121 items on the continuum of very low to very high function. By using CAT, wherein a question that a person is asked is selected from the question bank based on their response to a prior question, accuracy is often achieved with the delivery of 4–6 questions. PROMIS uses a T-score as the output measurement in which all PROMIS domains are normalized to the general population with the mean set to 50 and the standard deviation set to 10 points [23].

Statistics

Baseline characteristics, including patient demographics and operative data were summarized as means and standard deviations for continuous variables and percentages for categorical variables.

Paired Samples Student’s t test was used to analyze the change in PROMs from the preoperative visit to each follow-up. Percentage of patients having improvement greater than the MCID at each time-point was selected. Based on previous literature, the following MCID thresholds were used: 10 for NDI [15,24], 50% of baseline standard deviation for PROMIS-PF [25], and 8.1 for SF-12 PHS [26].

Pearson product-moment correlation was run to determine the correlation between NDI, PROMIS-PF, and SF-12 PHS at each time-point.

On the basis of the Pearson correlation coefficient (r), the strength of correlation was determined to be as previously described [27]: Very weak (0.00–0.20), Weak (0.21–0.40), Moderate (0.41–0.60), Strong (0.61–0.80), and Very Strong (0.81–1.00).

Statistical significance was defined with a p value set at <0.05. All analyses were performed using the IBM Statistical Package for the Social Sciences (SPSS) version 22 (IBM Corp., Armonk, NY, USA).

Results

A total of 121 patients, with a mean age of 52 years and a mean body mass index (BMI) of 27 kg/m² were included in this study. A majority of patients were males (63.6%), of white or Caucasian race (83.5%), had completed either a 4-year college (29.8%) or post-college (30.6%) education, were engaged in sedentary (21.5%) or light (28.1%) occupations and were privately insured (89.3%). 9.1% of patients were smokers, 18.2% were taking some type of opioid pain medication preoperatively and 83.5% were American Society of Anesthesiologists Class 2. Patient demographics are summarized in Table 1.

Table 1.

Patient demographics

Age (years)	52.08±11.68
BMI (kg/m2)	27.31±4.03
Gender
• Male	77 (63.6%)
• Female	44 (36.4%)
Race
• White or Caucasian	101 (3.5%)
• Black or African American	3 (2.5%)
• Asian	2 (1.7%)
• Other	7 (5.8%)
• Mixed	1 (0.8%)
• Unavailable/patient declined	7 (5.8%)
Ethnicity
• Hispanic or Latino	7 (5.8%)
• Not Hispanic or Latino	109 (0.1%)
• Unavailable/patient declined	5 (4.1%)
Insurance
• Medicare	11 (9.1%)
• Worker’s compensation	2 (1.7%)
• Private	108 (89.3%)
Educational level
• Less than high school	0 (0.0%)
• High school	14 (11.6%)
• Two year college	9 (7.4%)
• Four year college	36 (29.8%)
• Post-college	37 (30.6%)
• Unavailable	25 (20.7%)
Type of occupation
• Sedentary	26 (21.5%)
• Light	34 (28.1%)
• Medium	16 (13.2%)
• Heavy	8 (6.6%)
• Not employed/retired	12 (.9%)
• Unavailable	25 (20.7%)
Smoking status
• Current smoker	11 (9.1%)
• Former smoker	29 (24.0%)
• Never smoker	81 (66.9%)
Preoperative narcotic use	22 (18.2%)
Charlson comorbidity index (CCI)	1.25±1.42
American Society of Anesthesiologists (ASA) Class
• 1	15 (12.4%)
• 2	101 (3.5%)
• 3	5 (4.1%)

As seen in Table 2, a majority of procedures were primary surgeries (86.8%), with 66 patients undergoing anterior cervical discectomy and fusion, 42 undergoing cervical disc replacement and 13 undergoing posterior cervical decompression with or without fusion.

Table 2.

Operative details

Type of procedure
• Primary	105 (6.8%)
• Revision	16 (13.2%)
Type of surgery
• Anterior cervical discectomy and fusion (ACDF)	66 (54.5%)
• Cervical disc replacement (CDR)	42 (34.7%)
• Posterior cervical decompression with or without fusion	13 (10.7%)
Intraoperative complications	0 (0.0%)
Operative time (minutes)	69.78±25.38
Estimated blood loss (mL)	30.74±15.28
Return to OR during hospitalization	0 (0.0%)
In-hospital complications	4 (3.3%)
• Urinary retention requiring catheterization	1 (0.8%)
• Dysphagia	1 (0.8%)
• Infection	1 (0.8%)
• New onset sensory deficit	1 (0.8%)
Postsurgical length of stay (hours)	17.86±14.59

NDI decreased from 35.97±19.43 to 22.018±21.16, the PROMIS-PF score improved from 39.89±8.05 to 49.65± 11.56 and SF-12 PHS improved from 37.05±9.00 to 41.78±12.37 from the preoperative time-point to the 1-year follow-up. As seen in Table 3, a statistically significant improvement in all PROMs was seen at 6 weeks or longer follow-up. Additionally, the percentage of patients experiencing an improvement greater than the MCID on the NDI and PROMIS-PF increased from 30% to 35% at 2 weeks to 58% at 3 months, which was maintained up to 1 year. Of note, patients with a minimum of 3 months follow-up were included in the current analysis. Thus, although the number of patients available at the 6 month and 1 year follow-up is much smaller than at prior time-points, the follow-up percentage (ie, [number of patients who had completed at least 1 PROM/number of patients who had reached the corresponding follow-up time point] x 100) was 75% and 69.2% at the 6 month and 1 year time-points, respectively.

Table 3.

Mean PROM scores

	N	Follow up %	NDI		PROMIS-PF		SF-12 PHS
			Mean±SD	MCID %	Mean±SD	MCID %	Mean±SD	MCID %
Preoperatively	121		35.97±19.43		39.89±8.05		37.05±9.00
2 weeks	114	94.2	33.11±20.52	30.1	40.61±10.00	34.8	35.77±9.68	17.3
6 weeks	101	83.5	22.52±16.26*	48.5	45.47±9.59*	54.9	39.57±10.04*	22.4
3 months	97	80.2	21.58±18.13*	58.3	47.63±10.17*	58.2	42.84±10.83*	40.2
6 months	75	75.0	19.24±17.25*	58.7	48.55±9.60*	60.7	43.20±11.23*	35.6
1 year	45	69.2	22.08±21.16*	59.1	49.65±11.56*	58.3	41.78±12.37*	39.0

^*Indicates a statistically significant change from the preoperative time-point.

Table 4 shows the results of the correlation analyses. A statistically significant negative correlation was seen between NDI and PROMIS-PF at all time-points, which was moderate preoperatively and during the early follow-up (2–6 weeks), and strong thereafter. NDI and SF-12 PHS also demonstrated a statistically significant negative correlation, which was moderate preoperatively and at 6 week postoperatively, but strong at 2 weeks and at 3 months and longer follow-up. A statistically significant positive correlation was seen between PROMIS-PF and SF-12 PHS, which was strong to very strong at all time-points, except at 2 weeks.

Table 4.

Correlation between PROMs

	Number of observations (n)	Pearson correlation coefficient (r)	Strength of correlation	p Value
NDI and PROMIS-PF
Preoperatively	106	−0.565	Moderate	<0.0001
2 weeks	94	−0.600	Moderate	<0.0001
6 weeks	82	−0.598	Moderate	<0.0001
3 months	76	−0.689	Strong	<0.0001
6 months	60	−0.622	Strong	<0.0001
1 year	35	−0.705	Strong	<0.0001
SF-12 PHS and NDI
Preoperatively	117	−0.551	Moderate	<0.0001
2 weeks	111	−0.709	Strong	<0.0001
6 weeks	97	−0.566	Moderate	<0.0001
3 months	91	−0.678	Strong	<0.0001
6 months	73	−0.679	Strong	<0.0001
1 year	41	−0.749	Strong	<0.0001
SF-12 PHS and PROMIS-PF
Preoperatively	105	0.687	Strong	<0.0001
2 weeks	93	0.570	Moderate	<0.0001
6 weeks	80	0.729	Strong	<0.0001
3 months	73	0.729	Strong	<0.0001
6 months	60	0.644	Strong	<0.0001
1 year	34	0.822	Very strong	<0.0001

Discussion

The results of our study show that a statistically significant improvement was achieved in all PROMs by 6 weeks and was maintained at 1-year. Further, the percentage of patients achieving MCID was similar for NDI and PROMIS-PF, particularly at 3 months or longer follow-up, thus suggesting that the MCID threshold of 50% of the standard deviation for the study population established by Patel et al. [25] is equivalent to the threshold of 10 for NDI. In contrast, the percentage of patients achieving the MCID threshold of 8.1 for SF-12 PHS [26] was 15%–30% lower than the percentage of patients achieving MCID for NDI and SF-12, with only 39% of patients achieving MCID for SF-12 PHS at 1 year. Although numerous studies have established MCID thresholds for SF-36 in patients undergoing cervical spine surgery, with values ranging from 4 to 6 points [28–30], the study by Parker et al. [26], which reports a twice as high threshold of 8.1 points, is the only report on MCID for SF-12 in patients with degenerative cervical spinal pathology requiring surgical management. The use of different methods for MCID calculation in different studies, with a lack of definitive evidence regarding the best method of determining MCID could account for this difference. This is further highlighted by the fact that numerous studies have reported that MCID values in patients with cervical spinal disease vary greatly depending on the population, intervention and method used for calculation [30], with individual studies also reporting a wide range of values with various methods of calculation in the same population [26] and studies using various statistical methods to account for this variability when interpreting clinical results [31]. Although the reason for published studies establishing a higher MCID threshold for SF-12 than for SF-36 was not apparent, the use of this threshold for MCID analysis could explain the lower proportion of patients achieving MCID for SF-12 compared with NDI and PROMIS-PF.

Several studies have assessed the validity and utility of PROMIS in orthopedic patients [32,33], and specifically in those undergoing spine surgery [25,34–39]. Although a number of studies have reported a strong correlation between PROMIS-PF and NDI [36,40–45], Boody et al. [46] found only a moderate correlation between these measures. The findings of our study differ from those in the literature, with our results suggesting that the correlation between NDI and PROMIS-PF varies depending on the follow-up time-point – with only a moderate correlation seen preoperatively and in the early postoperative period, but a strong correlation seen beyond 3 months. Although we are unable to determine the reason for these findings, we believe that this could be attributable to a number of reasons – (1) PROMIS-PF is unable to capture the disease-specific disability caused by cervical spine pathology, particularly with respect to more severe symptoms and functional limitations that would be encountered preoperatively and during the early follow-up, but would not be apparent at mid-to-long term follow-up once the patient recovers from surgery and has relief of symptoms; or (2) NDI, which focuses on functional limitations due to pain may not actually be capturing the full spectrum of disability because it does not account for functional limitations due to other symptoms of cervical spinal pathology such as numbness, weakness, balance or gait disturbances, etc. PROMIS-PF, on the other hand, asks about the patient’s ability to perform physical tasks regardless of their symptoms, which may better capture functional limitations caused by symptoms other than pain. Regardless of the reason for this difference, PROMIS-PF may not be a good surrogate for NDI for the purpose of preoperative evaluation or short-term follow-up in patients following cervical spinal surgery.

We also found a strong correlation between SF-12 PHS and PROMIS-PF at all time-points, which to our knowledge has not been reported in the literature for this study population. Furthermore, even though the correlation between NDI and PROMIS-PF was strong at 3 months or longer follow-up, the correlation coefficient at all time-points was smaller than that for the correlation between SF-12 PHS and PROMIS-PF. Although it is not possible to determine the reason for the difference in our findings compared with other studies in the literature with respect to the degree of correlation between NDI and PROMIS-PF, we believe that our findings do accurately reflect psychometric properties of PROMIS-PF. Specifically, the results of our study highlight the convergent validity (ie, the degree to which the measure converges on other measures that it theoretically should be similar to) and the discriminant validity (ie, the degree to which the measure diverges from other measures that it theoretically should be not be similar to) of PROMIS-PF as evidenced by the following:

1. Like the SF-12, PROMIS is a general health measure that can be used across age groups, conditions and interventions. Thus, the strong correlation between these two PROMs demonstrated in our study provides evidence of the convergent validity of PROMIS-PF.

2. NDI, on the other hand is a condition-specific outcome measure. As a result, although the NDI is likely to correlate with PROMIS-PF, they essentially measure different domains of health with the NDI capturing only a subset of what is captured by PROMIS. Thus, the moderate correlation seen between these two measures supports the discriminant validity of PROMIS-PF.

Limitations

This study was a retrospective review of prospectively collected data and thus selection bias cannot be completely eliminated. Patients who answered their PROMs questionnaires may not necessarily be representative of all patients undergoing this surgical procedure. In addition, all patients did not have complete follow-up, and this could be a potential source of bias as well.

Other psychometric validation such as responsiveness, floor and ceiling effect, and efficiency was beyond the scope of this study and not performed as a part of this analysis. Despite this limitation, we believe that our findings are relevant in the context of PROMIS being increasingly implemented. Building on the results of the current study, a future direction would be to perform a full psychometric analysis and validation of PROMIS physical function and explore the applicability and relevance of other PROMIS domains in these patients.

Our study population comprised a majority of Caucasian males who were privately insured. Additionally patients included in the study were limited to those with degenerative conditions of the cervical spine who underwent spine surgery, and hence these findings may not be applicable to other populations. Due to the limited sample size, we were not able to further stratify by other factors such as number of levels, preoperative diagnosis or type of surgery that may impact PROM scores.

Conclusions

The results of our study demonstrate the convergent and discriminant validity of PROMIS-PF, supported by the strong correlation between SF-12 PHS and PROMIS-PF and the moderate correlation between NDI and PROMIS-PF, respectively. Thus, while PROMIS-PF may not be a good surrogate for disease-specific outcome measure, particularly in the preoperative and early postoperative period, it extends value as a precise and efficient general health measure. Larger studies, which allow for stratification by diagnosis and treatment, and focus on specific patient population, are warranted to evaluate the true utility and value of PROMIS in spine surgery patients, and methods to interpret the clinical significance of a change in PROMIS scores need to be established. Further research on the use of PROMIS to track outcomes in spine surgery will contribute to the development of evidence which can be used to guide patient-centered care based on patients’ perspectives while reducing patient-burden, and thus enhance the value of spine care.