Correlation of PROMIS Physical Function and Pain CAT instruments with Oswestry Disability Index and Neck Disability Index in Spine Patients

Abstract

Study Design

Prospective and retrospective cross sectional cohort analysis.

Objective

To show that PROMIS CAT assessments for physical function and pain interference can be efficiently collected in a standard office visit and to evaluate these scores with scores from previously validated Oswestry Disability Index (ODI)and Neck Disability Index (NDI) providing evidence of convergent validity for use in patients with spine pathology.

Summary of Background Data

Spinal surgery outcomes are highly variable, and substantial debate continues regarding the role and value of spine surgery. The routine collection of patient based outcomes instruments in spine surgery patients may inform this debate. Traditionally, the inefficiency associated with collecting standard validated instruments has been a barrier to routine use in outpatient clinics. We utilized several computer adaptive testing (CAT) instruments available through Patient-Reported Outcomes Measurement Information System (PROMIS) and correlated these with the results obtained using “gold standard” legacy outcomes measurement instruments.

Methods

All measurements were collected at a routine clinical visit. The ODI and the NDI assessments were used as “gold standard” comparisons for patient reported outcomes.

Results

PROMIS CAT instruments required 4.5 ± 1.8 questions and took 35±16 seconds to complete, compared with ODI/NDI requiring 10 questions and taking 188 ± 85 seconds when administered electronically. Linear regression analysis of retrospective scores involving a primary back complaint revealed moderate to strong correlations between ODI and PROMIS Physical function with r-values ranging from 0.5846 to 0.8907 depending on the specific assessment and patient subsets examined.

Conclusions

Routine collection of physical function outcome measures in clinical practice offers the ability to inform and improve patient care. We have shown that several PROMIS CAT instruments can be efficiently administered during routine clinical visits. The moderate to strong correlations found validate the utility of computer adaptive testing when compared to the gold standard “static” legacy assessments.

Introduction

Spinal surgery outcomes are highly variable, and substantial debate continues regarding the appropriate role for surgery. The routine use of patient based outcome instruments in spine surgery patients may help to inform this debate. Traditionally, the inefficiency associated with collecting standard validated instruments has been a barrier to routine use in outpatient clinics. Integrating routine collection of patient reported outcome measures (PROs) into clinical practice would facilitate the evaluation of treatment strategies based on direct feedback from the patient. Recently the National Institutes of Health (NIH) supported the development of the Patient-Reported Outcomes Measurement System (PROMIS). This system includes dynamic questionnaire based tools used to measure health outcomes from the patient’s perspective (http://www.nihpromis.org/default.aspx). While these instruments have the potential to provide quantitative outcome measures in spinal disorders, they have not been validated in a spine care setting nor have they been compared to previously validated legacy measures of neck and spine pain and disability. The Oswestry Disability Index (ODI) and the Neck Disability Index (NDI) are currently the “gold standard” PROs in low back and neck disability, respectively. In order to develop well-integrated, valid outcomes for spine surgery, practical protocols to evaluate these patients in the setting of a busy clinic must be established. This study assessed the feasibility of simultaneously obtaining PROMIS and ODI/NDI measurements from a cohort of patients seen in an orthopaedic spine clinic, and measured the correlation of specific PROMIS measures with the legacy instruments (ODI/NDI).

Back pain is a world-wide public health concern. Back pain has been shown to be the greatest reason for activity limitation in persons under 45 years of age.¹ Medical expenditures for individuals with back pain are approximately 60% larger than individuals who do not suffer from back pain. ^{2, 3} Previous studies have estimated that expenditures have increased 65% over a ten year period.⁴ Despite the prevalence of back pain and its large economic impact, there are surprisingly few evidence based treatment guidelines for this condition.⁵ Currently there are over 200 documented interventions for chronic lower back pain (CLBP), yet no one intervention has been shown to be superior.⁵

The failure to routinely measure validated patient based outcomes is an obstacle both for establishing best practices, and also for evaluating novel treatments and therapies. The PROMIS assessments were developed by a network of NIH-funded primary research sites and coordinating centers. They worked collaboratively to develop a series of dynamic tools to reliably measure PROs.^{6, 7} PROMIS provides reliable information on what patients are able to do and how they feel. The information is derived from patient responses to a set of rigorously designed questions about different aspects of health-related quality of life (pain, fatigue, anxiety, depression, social functioning, physical functioning, quality of sleep, etc.). Each measure was subjected to multi-stage development and testing to ensure that the information meets scientific standards of reliability.^6–8 The ultimate goal is to provide clinicians and researchers with access to efficient, precise, valid, and responsive indicators of a person’s health status. To increase the sensitivity and specificity of this instrument, a computer-adaptive testing (CAT) component has been developed. CAT utilizes the patient’s response to one question to queue the next question, which is presented. These measures are available for use across a wide variety of chronic diseases and conditions present in the general population.⁸ Several clinical studies have been published recently using PROMIS CAT as the primary outcome measure.^9–11

The Oswestry Disability Index has emerged as one of the most widely used spine specific outcome measures.^12–15 It was originally introduced in 1976 by Obrien et al and was published in 1980.¹⁶ A recent PUBMED search for (Oswestry Disability Index) returned over 2500 scientific articles involving the ODI. The ODI has been used to validate a multitude of low back specific pain/disability scales^17–20, and it has shown high correlation with general pain/function outcome measures.^{21, 22}

The Neck Disability Index was published by Howard Vernon in 1991.²³ Vernon modeled the NDI after the ODI with inclusion of several neck specific questions. A recent PUBMED search for Neck Disability Index returned over 1300 articles involving the NDI. The NDI has been used to validate a multitude of neck specific pain/disability scales, and has been extensively tested for repeatability and validity.²⁴

Very little research has been done using PROMIS assessments to study orthopaedic spine outpatients.²⁵ The primary hypothesis tested in this study is that these outcomes can be efficiently collected in a standard office visit in more than 80% of the patients who present for an appointment. The secondary hypothesis tested is that there is a significant correlation between a patient’s PROMIS (physical function and pain interference) scores and their ODI/NDI score.

Methods

The study was approved by the local IRB. Data was collected in two cohorts from a large academic hospital system. For the prospective cohort, all patients seeing the either of the 2 participating orthopaedic surgeons for an office visit who were at least 18 years old, with pain/disability associated with the spine were asked to participate in the study. Patients seeing the surgeon for concerns other than spine pathology, were not able to read or understand English, or were not able to provide informed consent were excluded. Patients who agreed to participate were evaluated by administering the PROMIS and ODI/NDI assessments on a tablet device (iPad 2) during a routine office visit. Patients were eligible for inclusion, regardless of the stage of treatment, as the questionnaires are purported to be broadly applicable to physical function and pain in general. Using the freely available access to PROMIS CAT instruments through the Assessment Center (www.assessmentcenter.net), a custom assessment was created to include both the physical function (version 1.2) and Pain interference (version 1.1); custom instruments were also created to include the ODI and the NDI separately. In this manner both of the PROMIS assessments and either the ODI or NDI questionnaire were completed in succession on the same tablet device during the patient’s office visit.

For the retrospective cohort, participants’ PROMIS pain and physical function CAT instruments had been previously collected as standard of care in the clinic for all providers for a defined historical period. Some surgeons in the department have also routinely collected ODI or NDI by paper for spine patients. Using the previously collected PROMIS data from one orthopaedic surgeon we identified those patients who had completed the PROMIS measures and reviewed their medical record to obtain ODI/NDI data collected on paper during the same visit. During the time period comprising this retrospective analysis two versions of the physical function assessment (version 1.0 and version 1.2) and two separate domains of pain assessment (pain behavior version 1.0 and pain interference 1.1) were administered in our clinic.

Statistical analysis

Differences based on demographics, assessment version, and cohort were tested by means of an individual univariate ANOVA. Those assessment versions found to be not statistically different were combined to provide a more robust correlative evaluation of the particular assessment type. All analyses used a significance level of 5%. Pain behavior and pain interference assessments were developed as separate constructs and therefore those correlations were analyzed separately. Pain behavior specifically addresses the actions of pain exhibited eg. “grimace”, “wince”, cringe”; while pain interference specifically addresses the aspects of life that have been disrupted eg. work, social life, sports. Correlation between legacy (ODI/NDI) assessments and individual PROMIS assessments were quantified using Pearson’s correlation coefficient. Separate correlation coefficients were calculated for subsets of patients based on sex, age, and operative state. Scatter plots of the data suggested a linear shape for models predicting ODI or NDI based on PROMIS assessments across all subjects as well as in each of the patient subsets.

Results

A total of 319 unique visits from 283 (146 Male; 137 Female) patients with an average age of 55.2 years were included in the analysis, with each office visit used as the unit of measure. Additional demographics are summarized in Table 1. Of the 118 patients approached during their clinical visit 114 were willing to participate, those who chose to participate completed all assessments, for an identical response rate of 96.6 % for each instrument. Each PROMIS instrument required 4.4± 1.9 (average +/− Stdev) questions and took 36±17 seconds, compared to ODI/NDI which are a static 10 questions taking 187 ± 86 seconds when administered electronically. Completion time for each of PROMIS assessments was significantly shorter than that of the ODI or NDI (p < 0.01), additionally combined times for both PROMIS assessments were also significantly shorter than the ODI or NDI (p< 0.01). A significant difference was found between males and females (p=0.041) on the ODI assessment with average scores of 40.1 ± 2.3 and 47.5 +/− 2.2 for males and females respectively, no other such difference based on sex or age were found for any of the other outcomes measured. No significant difference was found between the two versions of the PROMIS physical function assessment used. There was no significant difference found between the same assessments (ODI, NDI, PROMIS Physical Function, or PROMIS Pain interference) when collected either prospectively or retrospectively in this same spine clinic population. Therefore, separate PROMIS version and cohort scores were combined to give a more robust overall analysis. Linear regression analysis of scores involving a primary back complaint revealed a good linear fit between ODI and PROMIS Physical function with R² values of 0.683 (Figure 1), those involving a primary neck complaint also revealed a good linear fit between NDI and PROMIS physical function with R² values of 0.719 (Figure 2). PROMIS physical function and pain interference assessments given to post-operative patients showed an increased correlation with both the back complaint and neck complaint populations (Table 2). Those patients with a primary back complaint showed r- values of 0.847(post-op) vs. 0.760 (pre-op) and 0.891(post-op) vs. 0.613 (pre-op) for physical function and pain interference respectively. Those patients with a primary neck complaint showed r- values of 0.871 (post-op) vs. 0.833 (pre-op) and .8875(post-op) vs. 0.654 (pre-op) for physical function and pain interference respectively. A summary of the correlations for each assessment, along with subset sample sizes is given in Table 2. Regression analysis repeated for subjects based on age showed notably stronger correlations between ODI and PROMIS physical function assessment for those aged from 20–49 years of age (r = .870) when compared to those 50–69 (r = 0.795) or 70–89 (r = 0.712), see Table 3. Also, more moderate correlations were found between both legacy measures and PROMIS pain interference assessment in the 50–69 age range, with r = 0.702 and r = 0.685, for correlations with ODI and NDI respectively (Table 3). Regression analysis repeated for subjects based on gender showed a more moderate correlation between ODI and the PROMIS pain interference assessment for males (r = 0.698), an analysis by gender is given in Table 4. Insufficient numbers of retrospective patients with neck involvement that had taken the PROMIS pain behavior assessment prevented us from fully exploring the individual subsets of patients. A small number of 70–89 year old subjects completing the PROMIS pain interference assessment prevented examination of correlation in that patient subset.

Table 1. Demographics.

Descriptive demographic information for all visits evaluated patients are presented by the total number of unique patients, gender count, and average age for each gender. Averaged age includes standard deviation in parenthesis.

Cohort	Visits	Patients	Gender Male/Female	Average Age (stdev) Male / Female
All subjects	319	283	146 / 137	53.3(16.5) / 57.3(14.7)
Prospective	112	108	48 / 60	56.3(16.5) / 59.6(13.9)
Retrospective	207	175	98 / 77	51.9(16.4) / 55.5(15.2)

Figure 1. Linear regression analysis.

Linear regression plots are shown illustrating the correlation of ODI with PROMIS physical function CAT for all patient visits (A). A significant correlation between ODI and PROMIS was found with a combined correlation value of 0.8264 (p<0.001). Residual plot demonstrate the appropriate use of a linear fit for the regression analysis, with a random and equal distribution about the predicted line (B).

Figure 2. Linear regression analysis.

Linear regression plots are shown illustrating the correlation of NDI with PROMIS physical function CAT for all patient visits (A). A significant correlation between NDI and PROMIS was found with a combined correlation value of 0.8482 (p<0.001). Residual plot demonstrate the appropriate use of a linear fit for the regression analysis, with a random and equal distribution about the predicted line (B).

Table 2. Correlation by Operative State.

Associations between ODI and NDI among PROMIS physical function scores, pain interference scores, and pain behavior scores were quantified using Pearson’s correlation coefficient for all visits and are shown for between pre-operative and post-operative visits as well. All Correlations were found to be significant with p-values below 0.05. The number of observations at each time-point is given in in parenthesis (N). This analysis shows that both PROMIS physical function and pain interference are strongly correlated with the ODI and NDI post operatively, while Pre-operative correlations are more moderate. Pain behavior assessments are also more moderately correlated with both ODI and NDI.

	ODI			NDI
	All data	pre-op	post-op	All data	pre-op	post-op
Physical Function	0.8264(246)	0.7604(178)	0.8468(68)	0.8482(72)	0.8334(52)	0.8710(20)
Pain interference	0.7422(170)	0.6133(118)	0.8907(52)	0.7213(52)	0.6544(38)	0.8875(14)
Pain Behavior	0.7601(75)	0.7226(59)	0.8273(16)	0.5846(20)	0.7367(14)

Table 3. Correlation by Age.

Associations between ODI and NDI among PROMIS physical function scores, Pain Interference scores, and Pain Behavior scores were quantified using Pearson’s correlation coefficient for all visits and are shown for specific age ranges. All Correlations were found to be significant with p-values below 0.05. The number of observations at each time-point is given in in parenthesis (N). This analysis shows moderate to strong correlation for all three PROMIS CAT assessments across the ages examined.

	ODI			NDI
	20–49	50–69	70–89	20–49	50–69	70–89
Physical Function	0.8696(90)	0.7947(104)	0.7118(52)	0.8302(20)	0.8797(42)	0.8672(9)
Pain interference	0.7829(59)	0.7024(73)	0.8276(38)	0.7224(13)	0.6846(33)
Pain Behavior	0.7929(31)	0.7972(30)	0.6736(14)

Table 4. Correlation by Gender.

Associations between ODI and NDI among PROMIS physical function scores, Pain Interference scores, and Pain Behavior scores were quantified using Pearson’s correlation coefficient for all visits and are shown for each gender. All Correlations were found to be significant with p-values below 0.05. The number of observations at each time-point is given in in parenthesis (N). This analysis shows moderate to strong correlation for all three PROMIS CAT assessments for both genders.

	ODI		NDI
	Male	Female	Male	Female
Physical Function	0.8016(123)	0.7952(123)	0.8322(41)	0.8608(31)
Pain interference	0.6979(77)	0.8076(93)	0.7486(24)	0.7065(28)
Pain Behavior	0.7667(46)	0.7525(29)

Discussion

Validated outcomes instruments can be used to quantitatively assess the effects of reduced physical function and increased pain on an individual’s activities of daily living. Subjective measures can be quantified, and used to establish expected patient outcomes associated with specific injuries, disease states, or treatments at defined time points. Routine collection of validated measures in a clinic setting will serve to delineate these expectations. Once established, these expected outcomes can help to guide a patient’s individualized treatment plan.

In the present study, the use of PROMIS CAT provided for efficient collection of rigorous, valid descriptive outcome measures. We found moderate to strong correlations between the PROMIS physical function, pain interference, and pain behavior assessment and the ODI or the NDI depending on the region of primary complaint (Table 2). The correlation found between the PROMIS assessments and either of the legacy instruments provides convergent validity to the use of PROMIS_CAT in spine treatment research.²⁶ The strength of correlation observed in certain spine patient subsets is highly encouraging, particularly because the primary design of each PROMIS assessment, unlike the ODI and NDI, is more general and is not directly focused on disability due to spine disease. Anatomically distinct regions of disease or dysfunction may have differential effects on a patient’s total disability, and instruments designed to assess a specific anatomic region may not accurately reflect the overall disability experienced by the patient. The use of general outcome measures such as overall physical function and pain interference allows for use in a variety of spinal conditions, and may allow for broad comparisons between different disease states, different spine disorders, and different treatment paradigms including exercise, chiropractic, physical therapy, and surgery.

In the time-sensitive clinical environment in which many of today’s spine surgeons practice the incorporation of inefficient “legacy” PRO measures is untenable. The use of CAT administered PRO questionnaires is preferable as patients can easily complete these during waiting times. This study shows that using CAT administered PROs reduces the patient burden and has sufficient correlation with previously studied instruments in this population, and has the potential to provided added value for the both the clinician and the patient. The advantages of using PROMIS to collect PRO data also include the ease of administration, scoring, and tabulation of the collected data, while addressing the floor and ceiling effect found with other more specific PROs.^6–8 Automatic scoring and database storage eliminates the need for the additional steps of independent scoring, transcription, and data entry inherent in legacy PRO instruments. Each of these steps can increase the chance of human error. In present study, we made use of the PROMIS assessment center to collect ODI and NDI questionnaires on a tablet; however, independent scoring and data processing was required, taking up valuable support staff time. The workload impact of collecting these 2 PROMIS assessments in our facility is a minimal increase in staff time per patient, while providing invaluable information regarding patient outcomes. By developing reliable and valid PROs that are unobtrusive and efficient, the probability of widespread PRO data collection is increased.

The systematic collection of patient based outcomes in spine care will inform the debate regarding the efficacy of various treatments, and may also serve to clarify the time course of patient improvement during treatment. We also believe that it may be possible to identify patients who may be at high risk for poor outcomes through the use of such methodology. Previous evidence points to the use of pre-surgical and early post-surgical PROMIS assessment as a means by which to identify those patients at high risk of poor outcomes.¹¹ Those individuals with the lowest PROMIS scores at baseline were found to be more likely to have poor outcomes. Similar findings may hold in the spine population allowing for the development of specific pre-surgical interventions, specific patient education, specific surgical techniques, or post-surgical protocols that may help to decrease the risk of poor outcomes.

The present study utilized cross-sectional sampling of spine patients from three separate surgeons, and there was sufficient variation in each of the assessment values to produce reliable data for validation. However, this analysis gives no indication as to the utility of these assessments for a specific intervention strategy over the course of treatment. Previous research has shown that longitudinal tracking of PRO can identify patients likely to have a poor outcome following surgery.¹¹ It will be valuable to longitudinally track spine patients throughout episodes of care in the future. This value was recognized in a recent report on the NIH taskforce on the research standards for chronic low back pain. ²⁷ The physical function and pain interference assessment domains did not perfectly align with the assessment of disability that both the ODI and NDI quantified in this study. However, the moderate to strong correlations found show the impact that both physical function and pain had on the level of disability experienced by patients with spine pathology.

We have shown that PROMIS CAT assessments effectively identify both deficits in physical function and also the impact pain has on patients with spine pathology. We have demonstrated that there are moderate to strong statistical correlations between the PROMIS assessments and the “gold standard” legacy instruments widely used in spine care. Any patient based outcome measure, routinely employed, can help to inform physicians in individual patient cases, can identify improved outcomes associated with specific treatments, and can show overall practice trends. The results of this study suggest that using PROMIS-CAT as the preferred PRO instrument may prove to be more efficient and less burdensome than traditional legacy instruments in a busy practice.