Responsiveness of PROMIS® Global Health Short Form in Outpatients with Systemic Lupus Erythematosus

Abstract

Objective:

To evaluate the longitudinal responsiveness (sensitivity to change) of the Patient-Reported Outcomes Measurement Information Systems (PROMIS®) global health short form (PROMIS10) in outpatients with systemic lupus erythematosus (SLE).

Methods:

SLE outpatients receiving care at an academic medical center completed PROMIS10 at two visits a minimum of one month apart. Responsiveness of PROMIS10 global physical and mental health domains was evaluated in patients with improvement or deterioration of health status as measured by standard validated instruments. Effect sizes of changes in PROMIS10 scores between visits were evaluated using Kruskal-Wallis tests.

Results:

Two hundred twenty-three SLE patients enrolled and completed baseline surveys, with 186 (82%) completing a second set of questionnaires. PROMIS10 demonstrated mild to moderate responsiveness to patient-reported improvement (effect size 0.29) and worsening (effect sizes −0.27 and −0.54) of health status for both global physical health and global mental health. Changes in PROMIS10 correlated poorly with changes in physician-reported measures of disease activity.

Conclusion:

PROMIS10 showed responsiveness over time to patient-reported, but not physician-assessed changes in lupus health status. These data suggest that PROMIS10 can be used to efficiently measure and monitor important aspects of the patient experience of lupus not captured by standard physician-derived metrics. Further studies are needed to evaluate the role of PROMIS10 in optimizing longitudinal disease management in SLE and to determine its responsiveness in other chronic health conditions.

The accurate serial measurement of patient-reported outcomes (PROs) is a growing priority in rheumatology, where the care of chronic conditions requires validated tools to quantify and monitor the quality of life and the functional outcomes which patients prioritize.(1) The measurement of PROs in clinical care has rapidly increased during recent years not only to provide more patient-centered care, but also to meet the mandate of value-based health care initiatives in the United States such as the Centers for Medicare and Medicaid Services alternative payment models in orthopedics and oncology.(2,3) While numerous universal and disease-specific PRO measures are available, their use at the point of care has been limited by cost, burden of implementation, inadequate validation, and poor interpretability.(4)

The Patient-Reported Outcomes Measurement Information System® global health short form (PROMIS10) has emerged as an attractive universal PRO measure because it addresses many of these limitations. PROMIS10, a 10-item global PRO instrument, was developed as part of the National Institutes for Health Roadmap initiative to create a national resource for precise and efficient measurement of patient-reported symptoms, functioning, and health-related quality of life across medical conditions.(5) PROMIS10 combines items evaluating physical health, pain, fatigue, mental health, social health, and general health to provide a snapshot of global health through physical and mental health summary scores normalized to the general population.(6) As a freely available survey, PROMIS10 is increasingly being implemented throughout health systems as a “bottom line” indicator of health status, and to meet the reporting requirements of value-based health care initiatives.(7–10) Given the growing use of PROMIS10 as a default measure of patient-reported health status, understanding its performance characteristics, including responsiveness, or sensitivity to change, in specific diseases, would support its use as a valid tool for monitoring patient-centered outcomes in different clinical contexts. Despite the increased use of PROMIS10 in clinical settings, the responsiveness of PROMIS10 to changes in patients’ health status has not been extensively evaluated.

In this study, we evaluated the responsiveness of PROMIS10 to changes in health status in a cohort of patients with systemic lupus erythematosus (SLE). SLE patients suffer from severely diminished health related quality of life from their disease, its complications, and comorbidities.(11) We have previously demonstrated that PROMIS10 is psychometrically valid and reliable in outpatients with SLE, showing strong correlations with both comprehensive universal (SF-36 and PROMIS computerized adaptive tests) and lupus-specific (LupusQoL-US) instruments while taking less than two minutes to complete.(12) In the present study we examine the longitudinal performance of PROMIS10, focusing on its responsiveness to changes in SLE status using patient and physician-derived anchors.

Patients and Methods

Study Population:

English-speaking patients 18 years and older who received longitudinal care at the Lupus Center of Excellence of the Hospital for Special Surgery (New York, NY) and who met 1997 American College of Rheumatology SLE Classification Criteria were eligible to participate in the study. Patients on hemodialysis, or those with active malignancy, other than non-melanomatous skin cancer, were excluded.

Study Design:

This was a prospective longitudinal study. Eligible patients with SLE were identified through review of medical records and patients were enrolled at a routine outpatient visit. After written informed consent was obtained, subjects were registered in Assessment Center (www.assessmentcenter.net), a secure online research management tool maintained at the Northwestern University Research Data Center. Subjects completed online assessments onsite or remotely using a study-specific URL. They completed assessments at two consecutive regularly scheduled appointments at least one month apart. The study was reviewed and approved by the Hospital for Special Surgery Institutional Review Board (IRB# 14125).

Measures:

Patient-Reported Outcome Measures:

Subjects completed the PROMIS Global Health Short Form (PROMIS10) version 1.1, which consists of 7 questions related to general physical, emotional, and social health and 3 questions which examine emotional health, fatigue and pain in the previous 7 days. Subjects also completed the Short Form-36 (SF-36) Standard (US version 1.0), a validated and widely used 36 question survey of health-related quality of life with a 4-week recall period.(13) Both PROMIS10 and SF-36 were scored into the summary physical and mental health components. PROMIS10 global physical and mental health scores utilize a T-score metric with a US population mean of 50 and standard deviation of 10, with higher scores signifying better health-related quality of life. SF-36 physical component summary (PCS) and mental component summary (MCS) scores are also normalized around 50 with higher scores indicating better health status.

Patient Global Assessment of Change:

Subjects evaluated change in their health status at the time of their second assessment using a 7-point Likert scale describing the impact of lupus on their general health as “much better, somewhat better, a little better, about the same, a little worse, somewhat worse, or much worse” than at their baseline assessment.

Clinical Outcome Measures:

Treating physicians evaluated subjects’ SLE disease activity and damage at the time of each visit. Disease activity was measured with: 1) the physician global assessment (PGA) which uses a visual analog scale (score range 0 to 3); and 2) the SELENA-SLEDAI, a modified version of the SLE Disease Activity Index which consists of 24 weighted clinical and laboratory variables in 9 organ systems (score range 0 to 105).(14) Higher scores in each instrument reflect greater disease activity. SLE-related damage was assessed by treating physicians using the Systemic Lupus International Collaborating Clinics/ACR Damage Index (SDI) (score range 0 to 46, with higher scores indicating greater end-organ damage).(15)

Statistical Analysis:

The longitudinal responsiveness of PROMIS10 and SF-36 were evaluated utilizing an anchor-based approach in which external indicators were used to assign patients to groups of improving, worsening, or unchanging health status.(16) Subjects were categorized as “better,” “same,” or “worse,” in three different ways: 1) response to the patient global assessment of change; 2) change in PGA score; and 3) change in SELENA-SLEDAI score. “Better” was defined as a decrease in PGA of ≥ 0.5 or decrease in SELENA-SLEDAI of ≥ 3 points, while “worse” was defined by a score increase of ≥ 0.5 in the PGA or ≥ 3 in the SELENA-SLEDAI.(17) “Same” was defined by score change of < 0.5 in the PGA or < 3 in the SELENA-SLEDAI. Effect sizes, the ratio of the mean change in PROMIS10 or SF-36 score and the standard deviation of baseline scores, were calculated for groups of subjects who were better, same, and worse using each of the three anchors. Effect sizes of 0.2 are considered a small change, while those of 0.5 are considered to be a moderate change, and those of 0.8 represent a large change.(18) Standardized response means (SRM), in which the mean change in PROMIS10 or SF-36 scores are standardized by the standard deviation of the change in scores, were also calculated. Effect size and SRM are both standardized measures of responsiveness, but SRM is less affected by the heterogeneity of the sample because it accounts for variation in score changes.(19) Comparisons in effect sizes and SRMs were made between groups with Kruskal-Wallis tests given the non-parametric distribution of PROMIS10 scores. Mixed effects modeling, with patients treated as random effects, was used to analyze the changes in scores over time. The socio-demographic characteristics, baseline disease activity, and baseline PRO scores of subjects who completed to the study were compared to those lost to follow up with Wilcoxon rank sum tests and Fisher’s exact tests. Statistical analyses were performed with SAS version 9.3 (Cary).

Results

Participant Characteristics and Follow Up:

A total of 223 patients completed baseline assessments between November 2014 and September 2016 (Table 1). A majority (91.2%) of subjects were female and their median age was 37 years. Over 60% of subjects identified as non-white and more than one third were insured by Medicaid. The duration of SLE ranged from 0.1 to 47.7 years with a median of 10.6 years. SLE disease activity and damage was generally low, with a median PGA of 0.5, SELENA-SLEDAI of 4, and SDI of 1. Approximately 20% of participants were experiencing a lupus flare at the time of enrollment. The time between the baseline visit and completion of the initial assessment was a median of 0 (interquartile range 0, 1) days.

Table 1.

Baseline Characteristics of Study Participants

Characteristic	Participants (n = 223)
Age: median [IQR] years, (range)	37.0 [29.0, 49.0], (19 – 75)
Female: n (%)	208 (91.2)
Race: n (%)
White	87 (38.2)
Black	64 (28.1)
Asian	26 (11.4)
Other	51 (22.4)
Ethnicity: n (%)
Hispanic/Latino	65 (28.5)
Insurance: n (%)
Medicaid	82 (36.0)
Medicare	22 (9.7)
Commercial	124 (54.4)
SLE Characteristics
SLE Disease Duration: median [IQR] years, (range)	10.6 [5.7, 16.4], (0.1 – 47.7)
Physician Global Assessment: median [IQR], (range) [Range 0 to 3, higher worse]	0.5 [0.3, 1.0], (0.0 – 2.6)
SELENA-SLEDAI: median [IQR], (range) [Range 0 to 105, higher worse]	4.0 [2.0, 6.0], (0 – 14)
Flare: n (%)	45 (19.8)
SDI: median [IQR], (range) [Range 0 to 46, higher worse]	1.0 [0.0, 2.0], (0.0 – 8.0)

IQR = Interquartile Range

SELENA-SLEDAI = Safety of Estrogens in Lupus Erythematosus National Assessment-Systemic Lupus Erythematosus Disease Activity Index

SDI = Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index

Follow-up assessments were completed by 186 (82%) participants at a median of 3.7 (interquartile range 3.0, 12.5) months after baseline. Subjects who did not complete a second assessment did not significantly differ from those who completed the study in terms of their age, gender, race, insurance type, duration of disease or baseline PGA, SELENA-SLEDAI, or PROMIS10 scores. The change in SELENA-SLEDAI score between baseline and follow up visits was a median of 0 (interquartile range 0, 2).

Patient-Reported Outcomes:

Subjects’ baseline PROMIS10 and SF-36 scores reflected poor health-related quality of life (Table 2). The median PROMIS10 global physical health score was 39.8, more than one standard deviation worse than the US general population mean, while the median global mental health score was 43.5, over one-half standard deviation worse than the general population mean. Median SF-36 PCS and MCS scores were also decreased in comparison with the general U.S. population.

Table 2.

Baseline Patient-Reported Outcomes (n = 223)

Domain	Score
PROMIS10 Global Physical: median [IQR], (range) [Range 0 to 100, higher better]	39.8 [34.9, 47.7] (19.9 – 67.7)
PROMIS10 Global Mental: median [IQR], (range) [Range 0 to 100, higher better]	43.5 [36.3 – 48.3] (25.1 – 67.6)
SF-36 PCS: median [IQR], (range) [Range 0 to 100, higher better]	35.7 [28.0, 47.2], (8.0 – 61.0)
SF-36 MCS: median [IQR], (range) [Range 0 to 100, higher better]	44.7 [34.9, 52.5], (16.5 – 64.8)

IQR = Interquartile Range

SF-36 = Short Form 36

PCS = Physical Component Summary

MCS = Mental Component Summary

Responsiveness to Change:

PROMIS10 scores were responsive to changes in patients’ health status as measured by the patient global assessment of change (Table 3). Effect sizes for global physical and global mental health were small to moderate (−0.27 and −0.54 respectively) for patients reporting worsening of the impact of lupus on general health, and small (0.29) for those reporting improvement in the impact of lupus on their health status. The SRMs were slightly larger for both groups, at −0.37 and 0.41 for global physical and −0.68 and 0.36 for global mental health.

Table 3.

Responsiveness of PROMIS10 and SF-36 by Anchor [# of subjects]^*

A. Patient Global Assessment of Change
	Better	Same	Worse	p-value
PROMIS10 Physical				< 0.01
Effect Size	0.29 [72]	0.00 [76]	−0.27 [31]
SRM	0.41 [72]	0.00 [76]	−0.37 [31]
SF-36 PCS				0.01
Effect Size	0.15 [77]	−0.12 [75]	−0.15 [34]
SRM	0.22 [77]	−0.18 [75]	−0.23 [34]
PROMIS10 Mental				< 0.01
Effect Size	0.29 [59]	0.00 [72]	−0.54 [25]
SRM	0.36 [59]	0.00 [72]	−0.68 [25]
SF-36 MCS				0.06
Effect Size	0.14 [77]	0.00 [75]	−0.41 [34]
SRM	0.16 [77]	0.00 [75]	−0.49 [34]

B. Physician Global Assessment
	Better: ≥ 0.5 point decrease	Same: < 0.5 point change	Worse: ≥ 0.5 point increase	p-value
PROMIS10 Physical				0.22
Effect Size	0.00 [41]	0.00 [108]	−0.27 [29]
SRM	0.00 [41]	0.00 [108]	−0.37 [29]
SF-36 PCS				0.72
Effect Size	0.06 [42]	0.02 [113]	−0.34 [31]
SRM	0.09 [42]	0.03 [113]	−0.31 [31]
PROMIS10 Mental				0.23
Effect Size	−0.26 [39]	0.26 [90]	−0.29 [26]
SRM	−0.33 [39]	0.33 [90]	−0.36 [26]
SF-36 MCS				< 0.01
Effect Size	0.11 [42]	0.14 [113]	−0.34 [31]
SRM	0.13 [42]	0.17 [113]	−0.41 [31]

C. SELENA-SLEDAI
	Better: ≥ 3 point decrease	Same: < 3 point change	Worse: ≥ 3 point increase	p-value
PROMIS10 Physical				0.19
Effect Size	0.00 [36]	0.00 [122]	−0.27 [20]
SRM	0.00 [36]	0.00 [122]	−0.38 [20]
SF-36 PCS				0.21
Effect Size	0.08 [37]	0.02 [128]	−0.21 [21]
SRM	0.12 [37]	0.03 [128]	−0.31 [21]
PROMIS10 Mental				0.10
Effect Size	0.26 [33]	0.00 [105]	−0.31 [17]
SRM	0.33 [33]	0.00 [105]	−0.39 [17]
SF-36 MCS				0.03
Effect Size	−0.22 [37]	0.13 [128]	−0.15 [21]
SRM	−0.26 [37]	0.15 [128]	−0.18 [21]

SF-36 = Short Form 36

SRM = Standardized Response Mean

PCS = Physical Component Summary

MCS = Mental Component Summary

SELENA-SLEDAI = Safety of Estrogens in Lupus Erythematosus national Assessment-Systemic Lupus Erythematosus Disease Activity Index

SDI = Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index

^*Variable number of total subjects by domain due to skipped questions/non-scoreable assessments.

The SF-36 PCS demonstrated responsiveness to change in patients’ health status over time as measured by the patient global assessment, though effect sizes and SRMs were smaller (range 0.15 to 0.23) than those of PROMIS10 global physical health. The SF-36 MCS also showed smaller effect sizes and SRMs than those observed with PROMIS10 global mental health using this anchor (p = 0.06).

Using the PGA or the SELENA-SLEDAI as anchors, PROMIS10 global physical and mental health scales and SF-36 PCS did not demonstrate statistically significant differences in effect sizes or SRMs. The SF-36 MCS showed statistically significant differences in effect sizes and SRMs across groups with the use of PGA (p < 0.01) and SELENA-SLEDAI (p = 0.03), but values were generally small and the trend unreliable with values of similar magnitude in the “same” group and negative polarity in the SELENA-SLEDAI’s “better” group.

Mixed effects modeling found that time between assessments was not a significant variable associated with changes in PROMIS10 global physical (beta = −0.002 ± 0.009, p = 0.83) or global mental (beta = 0.005 ± 0.011, p = 0.66) health scores.

Discussion

In this study of a large socio-demographically diverse cohort of outpatients with SLE we found that PROMIS10 is responsive to patient-reported changes in health status. PROMIS10 global physical and mental health summary scores significantly increased in patients who reported improvement in the impact of lupus on their health, while these scores decreased in patients who reported deterioration of their SLE-related health status. Notably, PROMIS10 summary scores demonstrated greater responsiveness than the comparable PCS and MCS of the SF-36, a well-established generic measure of health-related quality of life.

To the best of our knowledge, this is the first study to evaluate the responsiveness of PROMIS10 in patients with SLE. Our findings suggest that PROMIS10 is a sensitive tool for detecting changes in patients’ perceptions of their health status, a crucial patient-centered outcome across medical conditions. We found that PROMIS10 was not sensitive to changes in physician-assessed SLE disease activity. Similar findings have been shown in other disease specific and universal PRO measures used in patients with SLE,(20,21). We suspect that the difference in performance of the anchors reflects the different latent variables measured by patient-reported outcome surveys and physician-evaluated disease activity instruments. Although differences in the reliability of anchors could account for the divergent findings, intraclass correlations coefficients for the PGA (0.88), SLEDAI (0.87), PROMIS10 global physical health (0.89), PROMIS10 global mental health (0.85), SF-36 PCS (0.92), and SF-36 MCS (0.91) are comparable, making this less likely.(12,22,23)

Few studies have examined the responsiveness of PRO measures to patient-reported changes in SLE. A study of 101 SLE patients in the United Kingdom who were followed at four week intervals for ten months, found small to moderate effect sizes (−0.11 to 0.53) for deterioration and improvement of patient reported health status in physical function and mental health domains of the SF-36 and the LupusQoL, a disease specific PRO measure.(21) Similarly, a study of 185 French patients with SLE who completed up to three surveys three months apart found low to moderate sensitivity to change using a patient-derived global assessment of disease impact for both the SF-36 PCS and MCS (SRMs −0.25 to −0.61) and related domains of the LupusQoL.(24) Our study corroborates these findings, identifying effect sizes of similar magnitude for the SF-36. Moreover, our study suggests that the overall performance of PROMIS10 is comparable to that of the SF-36, which is particularly notable given the brevity of PROMIS10, with 10 questions compared to the 36 questions of the SF-36.

By demonstrating the validity and responsiveness of PROMIS10 in this population of patients with SLE, our study supports its potential utility in clinical care. As a universal instrument with a single scoring metric, PROMIS10 has a lower burden of implementation and greater ease of use compared with the use of multiple disease-specific instruments. For patients with SLE, who are often seen and treated by many specialists due to the impact of this disease on multiple organ systems (e.g. rheumatology, nephrology, orthopedics, dermatology, ophthalmology), PROMIS10 provides a brief yet accurate “bottom line” measure of well being that can be followed longitudinally across the spectrum of health care. Its practicality and feasibility are, in part, related to its accessibility across conditions, which can be helpful to these patients, who often suffer from numerous disease related comorbidities and complications whose contributions to overall health are difficult to tease apart.

As a brief global measure, PROMIS10 also has limitations, specifically its precision. While PROMIS10 strongly correlates with physical function, pain, and emotional health domains in more comprehensive lupus-specific and universal PRO measures, due to its brevity, it may not capture subtle changes in specific domains.(12) The PROMIS family of instruments includes computerized adaptive tests (CATs), which are drawn from item banks developed based on item response theory enabling greater measurement precision than standard short form patient-reported outcome surveys.(25) PROMIS10 may be effectively utilized as a screening tool to identify domains of concern, which can then be evaluated in greater detail with more comprehensive and precise measures such as PROMIS CATs, which have construct validity and reliability in SLE.(26)

Limitations of our study include the relatively low or quiescent SLE activity of subjects, with two thirds experiencing no change in disease activity as measured by the SELENA-SLEDAI index. This lack of variability in disease activity may have limited our ability to detect responsiveness of PROMIS10 to changes in physician-derived instruments. Additional studies in patients with more active SLE are warranted to better understand the performance of PROMIS10 in this population, including its minimal important difference. The study is also limited by the loss of 18% of subjects to follow up. However, socio-demographic and clinical characteristics and baseline PROs did not significantly differ between those who did and did not complete the study, suggesting our findings are likely to be representative.

In conclusion, this study provides important new insights into the longitudinal performance of PROMIS10 in patients suffering from a chronic systemic condition. Our data suggest that PROMIS10 is a responsive tool for monitoring and detecting changes in global physical and mental health status related to SLE and understanding SLE patients’ longitudinal experience of disease. As a valid yet parsimonious universal instrument, PROMIS10 may be practical for use across the clinical settings SLE patients encounter, particularly as the survey continues to be implemented across health systems. Optimal clinical uses of PROMIS10 in SLE, including when it is expedient to incorporate additional instruments to measure disease-specific domains to ensure appropriate patient care, should be explored in future studies. Our findings are encouraging that PROMIS10 may accurately track important disease-related patient-centered outcomes despite its brevity and global outlook, and further work is necessary to establish its responsiveness in other disease conditions and health states.

Significance and Innovations.

• The accurate measurement of patient-reported outcomes is a priority for clinical care and value-based health care initiatives in rheumatology, but few validated measures are freely available for use at the point of care.

• PROMIS10, a brief, publicly available universal measure of patient-reported physical and mental health, is increasingly being implemented across health systems despite limited data on its performance characteristics.

• To our knowledge, this is the first study to report that PROMIS10 is responsive to patient-reported changes in health status over time in patients with a chronic disease.

• Our findings suggest that PROMIS10 may be a useful tool for monitoring important patient-centered outcomes in SLE, and may have similar utility in other chronic health conditions.