Responsiveness of PROMIS^® Pediatric Measures to Hospitalizations for Sickle Pain and Subsequent Recovery

Abstract

Background

The Patient-Reported Outcomes Measurement Information System^® (PROMIS^®) created pediatric self-report scales measuring a variety of health attributes (domains), but their responsiveness to changes in health status has not yet been determined in children with sickle cell disease (SCD).

Procedure

A convenience cohort of symptomatic SCD children, aged 8-17 years, were asked to complete PROMIS pediatric scales at an initial clinic visit, at the end of a subsequent hospitalization for sickle pain, at a subsequent clinic visit or at home 2-3 weeks after hospitalization, and at a clinic visit 1-2 years after their initial assessment.

Results

A total of 121 participants (mean age 12.5 ± 3.1 years, 56.2% female) participated in the study. Pain interference and fatigue domain scores were elevated at baseline, increased substantially during hospitalization, and largely returned to baseline by the recovery period, while the depressive symptoms, anger, and anxiety domain scores displayed a less pronounced elevation during hospitalizations and a slower return to baseline levels. The two physical functioning scales showed a substantial decline in response to hospitalization, but only modest improvements at the recovery assessment, likely representing incomplete recovery.

Conclusions

Several PROMIS Pediatric measures were responsive to changes in health status associated with occurrence and resolution of acute vaso-occlusive pain requiring hospitalization. The substantial differences in these domains during SCD-related pain exacerbations support their potential usefulness in clinical research or in clinical practice. Further studies to characterize variations in symptom patterns over time may provide insights into strategies for more effective management of sickle pain.

Introduction

Measures of health-related quality of life (HRQOL) are being increasingly used to assess the impact of treatment in clinical trials [1] and in clinical practice settings. [2] These measures of patient-reported outcomes (PROs) provide an important additional perspective on the impact of treatment effects or disease manifestations. Such measures may be particularly useful in chronic conditions with complex symptomatology that lack surrogate biomarkers to assess clinical outcomes, such as sickle cell disease (SCD).

The Patient-Reported Outcomes Measurement Information System^® (PROMIS^®; www.nihpromis.org) developed pediatric self-report scales to provide valid and reliable assessment of HRQOL domains for children. [3] The PROMIS measures were developed through an extensive review of the literature, expert review, and qualitative methods (focus groups and cognitive interviewing), [4-6] consistent with the FDA guidance on the development of such measures. [7] Subsequent quantitative analyses utilized item response theory (IRT) to develop item banks on a common HRQOL metric suitable for computer adaptive testing (CAT; fixed-length short forms were also developed as alternative modes of administration). [3] These measures were normed in a diverse sample of children and adolescents [8] and have been validated in several chronic childhood conditions and diseases. [9-13]

A number of generic HRQOL measures have been examined in children with SCD, including the Child Health Questionnaire (CHQ), the Peds QL ver 4.0™, and the PROMIS pediatric measures. [14] The Peds QL generic version has demonstrated significant reliability, and construct and discriminant validity in cross-sectional studies, [15,16] as have the PROMIS pediatric measures. [17] The Peds QL is also now available in a SCD-specific measure, [18,19] and clinically meaningful scale score ranges have been validated for use in clinical practice. [20] However, evidence of the responsiveness of any of these measures over time is limited.

A responsive measure is sensitive to improvements, deteriorations, and stability of health status over time, and assessing responsiveness is an important component of establishing validity. [21] A longitudinal study of patients undergoing treatment or experiencing exacerbations or remissions of their disorder is usually required for such evaluations. For example, the responsiveness of the PROMIS adult physical function measure (PF-20) has been evaluated in patients participating in the Arthritis, Rheumatism and Aging Medical Information Systems (ARAMIS) observational cohorts. [22] The PROMIS PF-20 was more responsive than two widely used (‘legacy’) measures, the SF-36 and the Health Assessment Questionnaire (HAQ).

As part of a larger study of the responsiveness of the PROMIS pediatric measures to changes in health status in childhood illnesses, including cancer, kidney disease, and asthma, we evaluated the responsiveness of selected PROMIS pediatric measures in a sample of children and adolescents with SCD. We studied children ages 8- to 17-years old during two routine, non-acute healthcare encounters over a 1-2 year interval; during an illness exacerbation (hospitalization for treatment of severe vaso-occlusive pain); and after a subsequent recovery period of several weeks. Based on previous studies of children [23] and adults, [24] levels of pain interference and fatigue were expected to increase with disease exacerbations (SCD pain episodes) and then decline to baseline levels upon recovery, while levels of physical functioning were expected to decline during disease exacerbations and improve upon recovery. Anxiety, depressive symptoms, anger, and peer relationship scores were expected to remain largely unchanged.

Methods

Recruitment

A convenience sample of SCD patients aged 8-17 years was recruited from 3 clinical sites in a large sickle cell program (Children's Healthcare of Atlanta). Only those children with one or more acute care visits for pain in the previous year were eligible, as we intended to assess both their usual state of health and an episode of severe pain, and thus excluded asymptomatic individuals. Eligible children and their parent/guardian also had to be able to read and speak English, possess functional computer skills (defined as the ability to see and interact with a computer screen, keyboard, and mouse), and be willing to give written assent/permission for study participation. Excluded were children and adolescents who had any concurrent medical or psychiatric condition that precluded study participation, or cognitive or other impairments (e.g., visual) that interfered with completing a self-administered computer-based questionnaire.

Data collection

After consent/assent was obtained by study staff, parents completed the demographic and SCD-related medical history items on the computer and then children completed the pediatric PROMIS measures. These assessments were completed at an initial routine healthcare encounter in the sickle cell clinic (baseline) and at a subsequent routine clinic visit one to two years later (follow-up). These measures were repeated prior to discharge from a subsequent hospitalization for pain (pain hospitalization) and after a subsequent recovery period of several weeks (recovery). Additional participants were enrolled during a pain hospitalization without a previous baseline clinic visit to facilitate completion of further hospitalization and recovery assessments. All pain hospitalization assessments were administered at the end of the hospitalization (on the day of discharge) when pain is typically well controlled.

Using a laptop or tablet in a private clinic exam room, a nearby conference/consultation room, or in their private hospital room, participants accessed the survey using Assessment Center (http://assessmentcenter.net/) via the Internet and completed the measures. Parents were invited to remain with their child if they preferred; however, if they remained, they were asked not to assist their child with responding and instead allow the study team member who remained in the room to assist if needed. To accommodate some families who were unable to return to clinic for hospital follow-up visits, paper versions (fixed short forms) of the PROMIS measures were sent to their homes and returned by mail. Children or adolescents received a $10.00 gift card for participating in each assessment.

Assenting children and adolescents were assigned unique identification numbers using a computer-based system. No other identifiers were collected by the computerized assessment. Only de-identified data were used in the analyses.

Measures

Parents completed a 16-item demographic form and a 17-item form detailing their child's SCD treatment, occurrence of chronic complications, frequency of acute complications, and the presence of pain experienced by their child during the previous 7 days. Medical information was verified, as needed, from medical records.

Eight PROMIS pediatric measures (Physical Functioning-Mobility, Physical Functioning-Upper Extremity, Pain Interference, Fatigue, Depressive Symptoms, Anxiety, Peer Relationships, and Anger) were administered. These measures elicited responses based on the previous 7 days using a 5-point response option ranging from “with no trouble” to “not able to do” for physical functioning measures and from “never” to “almost always” for other measures. Higher scores indicate more of the measured domain being assessed, which signifies worse HRQOL for depressive symptoms, anxiety, anger, fatigue, and pain interference, and better HRQOL for mobility, upper extremity functioning, and peer relationships. PROMIS pediatric measures are scored on a T-score metric, with a mean of 50 and a standard deviation of 10 based on the original reference sample of children and adolescents. [22]

These and other PROMIS pediatric measures are available at no cost for download at www.nihpromis.org or www.healthmeasures.net after a simple registration process. These websites also provide the characteristics of the measures, which are also available in associated publications. [23-28] Guidance for their use and scoring as fixed short forms or as computer adapted tests are available at these websites, as well as at www.assessmentcenter.net, which provides a software platform for administration and scoring as used in our study.

Children and adolescents also completed Global Ratings of Change (GRC) assessments for overall health, pain, fatigue, physical function, peer relationships, and anxiety as an anchor for establishing differences. Response format was better, worse, or the same. The GRC was administered at the pain recovery visit with reference to the preceding pain episode. The relatively short time frame between the hospitalized pain event and the recovery visits was expected to enhance the reliability of this response.

Statistical Analysis

All statistical analyses were produced via SAS version 9.3 and R version 3.1.1. [25,26] Descriptive statistics for each of the PROMIS domains were analyzed in tabular and graphical form. Mean values and corresponding standard errors were considered, in addition to the number of non-missing data points at each of the four possible assessments. Paired t-tests were used to compare subjects who had non-missing observations in each of the episodes considered. Alternately, a generalized linear model (GLM) with an identity link function and compound symmetric correlation structure was applied to jointly model the continuous outcomes for the four assessments for all subjects. [27] This technique is a generalization of ANOVA with unequal sample sizes with correlated outcomes. Missing data were assumed to be missing completely at random.

To further describe the marginal modelling approach, Let Y_i denote the observation vector of length n_i for participant i (1≤ n_i ≤ 4), where i = 1, ..., 121. The model is written as Y_i = X_i β + e_i, where e_i follows a multivariate normal distribution of dimension n_i with mean zero and covariance ϭ²V, where V is the (compound symmetric) population correlation matrix. The vector β=(μ_B, μ_P, μ_R, μ_A) contains the four unknown mean parameters, Baseline, Pain Event, Recovery, and Follow-up Visit, while the matrix X_i is a 4 × 4 identity matrix for subject i. Generalized estimating equations (GEE) were applied to estimate β and correlation parameters. Table II gives the estimates and standard errors of the domain marginal means for each assessment. Table III gives the estimates and p-values for differences between the selected means using the GLM. The test of the difference between two means using the GLM for repeated measures generalizes the paired t-test by using all observations from each participant (even if the participant had only one value) and also adjusting for the correlation between observations.

Table II.

Mean PROMIS domain scores at each assessment

Domain	Baseline Visit Mean (SE)	Follow-up Visit Mean (SE)	Hospital Episode Mean (SE)	Recovery Episode Mean (SE)
Pain interference	56.1 (1.2)	55.5 (1.1)	61.0 (1.2)	55.5 (2.1)
Fatigue	52.2 (1.5)	53.0 (1.5)	61.7 (1.8)	50.9 (2.6)
Depression	45.1 (1.1)	47.1 (1.1)	50.2 (1.2)	46.7 (1.6)
Anxiety	44.1 (1.3)	45.0 (1.1)	49.1 (1.5)	46.2 (1.8)
Anger	42.8 (1.3)	44.4 (1.3)	46.7 (1.5)	43.2 (2.2)
Mobility	48.2 (0.9)	47.1 (1.0)	41.5 (1.5)	43.8 (1.9)
Upper dexterity	50.0 (0.9)	48.2 (0.9)	43.1 (1.4)	45.3 (1.9)
Peer relationships	47.7 (1.2)	48.7 (0.8)	47.2 (1.3)	48.2 (1.4)

SE = Standard Errors

Table III.

Estimated mean differences in PROMIS domain scores between assessments

Domain	Baseline Visit to Follow-up Visit (SE)	Baseline Visit to Hospital Episode (SE)	Hospital Episode to Recovery Episode (SE)
Pain interference	0.6 (1.2)	5.5 (1.5)**	−5.5 (2.0)**
Fatigue	−0.8 (1.6)	8.6 (2.1)**	−10.7 (2.4)**
Depression	−2.0 (1.2)	3.1 (1.5)*	−3.5 (1.7)*
Anxiety	−0.9 (1.4)	4.2 (1.7)*	−3.0 (2.0)
Anger	−1.6 (1.6)	2.3 (1.7)	−3.5 (2.3)
Mobility	1.1 (1.1)	−5.6 (1.8)**	2.3 (2.3)
Upper dexterity	1.8 (1.1)	−5.1 (1.6)**	2.2 (2.2)
Peer relationships	−1.0 (1.3)	−1.4 (1.3)	1.0 (1.5)

The direction of the sign of the estimated mean difference indicates the direction that the scores are changing over time. For example, a positive difference in Column 2 indicates that the PROMIS score increased from the baseline to follow-up visit.

^**p-value <0.01

^*p-value <0.05

SE= Standard Errors

Results

Enrollment of Study Participants

Over a 20-month period from May 2011 to January 2013, a convenience sample of 121 children and adolescents were recruited. Of these, 91 (75%) were initially recruited at a routine clinic healthcare encounter while 30 (25%) were recruited during hospitalizations for pain management (Figure 1). Seventy individuals recruited in clinic and 9 recruited during hospitalizations provided an additional assessment, either in person or by mail, at a subsequent routine clinic visit at an interval of 1.5 ± 0.56 years.

Figure 1.

Twenty one of the 91 individuals recruited at a routine clinic healthcare encounter completed an assessment at the end of a subsequent hospitalization (16.6 ± 19.1 months from baseline visit), with 16 (76%) of those completing hospitalization assessments also completed pain recovery visits at a median interval of 20 days (range 7-67 days) from their hospitalization assessment. Recovery assessments were completed by 16 (53%) of participants enrolled at hospitalizations, which occurred at a median interval of 13 days (range 7-29 days) after the completion of hospitalization assessments.

Sample Characteristics

The 121 participants were equally represented in the 8-12 and 13-17 year old age groups (mean age 12.5 ± 3.1 years, 56.2% female). Almost all participants were self-identified by parents/guardians as African American and not of Hispanic ethnicity (Table I). Most participants had a homozygous S (SS) or Sickle B⁰ thalassemia phenotype (73.6%), while 24.8% had SC or Sickle B⁺thalassemia phenotypes. Children prescribed hydroxyurea (60% of those participants with SS or SB0thalassemia) had been prescribed this medication for at least 6 months prior to their initial PROMIS assessment.

Table I.

Demographic and clinical characteristics of study participants (n=121)

Characteristic	N (%)	Mean (SD)
Age (in years) at earliest study visit*		12.5 (3.1)
8 – 12	59 (49.6)
13 – 17	60 (50.4)
Gender
Male	53 (43.8)
Female	68 (56.2)
Race
Black or African American	120 (99.2)
Multiple races	1 (0.8)
Ethnicity
Hispanic	3 (2.5)
Non-Hispanic	118 (97.5)
Genotype
SS or SB0thalassemia	89 (73.6)
SC or SB+thalassemia	30 (24.8)
Other	1 (1.6)
Hydroxyurea usage reported at initial assessment^
No	65 (55.1)
Yes	53 (44.9)
Comorbidities reported at baseline visit**
None	43 (47.3)
One other health problem	27 (29.7)
Two or more health problems	21 (23.1)
Treated for pain at home in week prior to baseline visit^
No	52 (59.1)
Yes	36 (40.9)
Hip or joint problems reported at baseline visit^
No	76 (86.4)
Yes	12 (13.6)

^*2 missing

^**only available for 91 patients with baseline data

^{^}only available for 118 participants

Additional medical comorbidities were not uncommon with 52% reporting one or more co-occurring conditions at their baseline visit (Table I), the most common of which was a physician diagnosis of asthma. A recent hip or other joint problem that interfered with usual activities at the baseline visit was reported for 14% of participants, and 41% reported pain in the week prior to their baseline assessment.

PROMIS measures

At the initial baseline clinic visit, the pain interference mean score (55.9) was substantially above, while the fatigue mean score (52.1) was modestly above those of the original PROMIS reference sample norms with mean 50 (Table II), reflecting the prevalence of pain in this symptomatic SCD sample. As expected, mobility and peer relationships were somewhat below the reference sample averages, reflecting mild impairment from this chronic disease, while upper extremity physical function was similar to that of the reference sample. Interestingly, anger, anxiety, and depressive symptoms domain scores were lower than reference sample averages. The reference sample, which was used to calibrate the PROMIS pediatric item banks used in this study, comprised a diverse mix of children and adolescents, many of whom had health conditions; as a result, scores on the PROMIS measures do not reflect normative population scores. [8]

Responsiveness

Baseline clinic visit to annual follow-up visit comparisons

In general, PROMIS domain scores at the annual follow-up period were similar to those from the baseline, consistent with the absence of any major changes in therapy (Table II). The differences in comparison to baseline values were relatively small (1-2 points) for all domains and not statistically significant (Table III). The length of time between assessments (mean 1.5 years, range 0.65-2.74 years) was not significant when added as an additional variable in these marginal models. This suggests considerable stability in these domain scores over a modest period of time in the absence of major changes in symptom burden or treatment.

Initial clinic visit to pain hospitalization comparisons

As expected, given that these individuals were admitted for management of severe pain, pain interference and fatigue mean scores were substantially higher during pain episode hospitalizations (Table II), with large statistically significant differences (p<0.001) in comparison to scores from the initial clinic visits (Table III). These differences could be considered to be clinically significant as they were associated with hospitalization. Scores for depressive symptoms and anxiety had similar increases during the pain hospitalizations, which were also statistically significant. A comparable relationship was observed for the two physical functioning domains, reflecting impaired functioning during hospitalization with similar statistically significant differences in comparison to the initial clinic visit.

Pain hospitalization to recovery comparisons

As these individuals were admitted for management of severe pain, it was expected that the pain interference and fatigue domain scores would be high, reflecting the impact of the previous days of severe pain and its treatment, while scores would be lower at the recovery assessment, reflecting improvement in pain. As predicted, when compared to their prior hospitalization using the GRC, most participants reported at their recovery assessment that their overall health and pain were better (Table IV), with somewhat fewer reporting similar improvements in fatigue and physical activity, while anxiety and peer relationships were more likely to be reported as unchanged. Consistent with these global ratings of change, statistically significant improvements were noted for pain interference and fatigue scores using the GLM procedure comparing pain to recovery assessments (Table III). These changes remained statistically significant when only subjects with paired results were compared. Scores for depressive symptoms, anger, and anxiety improved (decreasing scores) comparing end of hospitalization to recovery assessments. Difference in depressive symptoms scores was statistically significant using the GLM procedure, but not when comparing paired data, likely reflecting the loss of power from the smaller sample size. A similar relationship was observed for the two physical functioning domains reflecting impaired functioning during hospitalization and increasing (improving) functioning by the time of the recovery assessment, but this difference was modest and not statistically significant. The length of time between assessments (mean 21.5 days, range 6-92 days) was not significant when added as an additional variable in these marginal models.

Table IV.

Child-reported Global Ratings of Change for Recovery Compared with Hospitalization

Characteristic	Better N (%)	About the same N (%)	Worse N (%)	Missing N (%)
Health	29 (76.3)	4 (10.5)	3 (7.9)	2 (5.3)
Pain	27 (71.1)	5 (13.2)	4 (10.5)	2 (5.3)
Energy	25 (65.8)	8 (21.1)	3 (7.9)	2 (5.3)
Physical Activity	21 (55.3)	13 (34.2)	2 (5.3)	2 (5.3)
Worries	17 (44.7)	17 (44.7)	2 (5.3)	2 (5.3)
Friendships	15 (39.5)	20 (52.6)	1 (2.6)	2 (5.3)

Question was worded: “How is your ____ compared to your last study visit?”

Responsiveness patterns

Of interest, a comparison of the hospitalization and recovery assessments to those at the baseline or follow-up assessments (Figure 2) suggest several different patterns of change for the PROMIS domain scores. Pain interference and fatigue domain scores were elevated at baseline, increased substantially during hospitalization, and largely returned to baseline by the recovery period (Figure 2A). Compared to pain interference and fatigue scores, the mental health-related domains of depressive symptoms, anger, and anxiety displayed a less pronounced elevation during hospitalizations and a slower return to baseline (Figure 2B). While showing a robust change (decline in functioning) in response to hospitalization similar to pain interference, the two physical functioning domains showed only a modest improvement at the recovery assessment, likely representing a slow or incomplete recovery of physical activity (Figure 2C). In contrast, peer relationship domain scores were relatively unchanged at each of the assessment points (Figure 2D).

Figure 2.

A Fatigue ───, Pain Interference – – –; B Anger ••••, Anxiety – • – •, Depression – – –; C Lower Extremity Functioning (Mobility) – • – • –, Upper Extremity Functioning • – • – •; D Peer Relationships – – –

Discussion

We experienced a number of challenges conducting this study that may be relevant to future studies. Measure completion was excellent when supervised by study staff in the clinical care setting using provided laptops/tablets for accessing the study website over the hospital wireless internet access system. For a number of reasons, some children were unable to return for follow-up clinic visits and their completion of recovery assessments was limited. Those participants unable to return to clinic were encouraged to complete the measures over the internet using the study website. However, completion rates improved when PROMIS pediatric short forms in paper format where mailed to homes suggesting that some children/families in our sample had limited computer resources or internet access at home, or found our study website difficult to use without assistance. Determining subject preferences for assessment response format at study entry might improve response rates. [28] Our interest in validating the available PROMIS pediatric domains in SCD required a modest participant burden (about 20-30 minutes to complete measures) that could be reduced in future studies by targeting measures to specific research or clinical questions being evaluated, or by using the shorter PROMIS pediatric profiles, which were not available at the time this study was conducted.

Our results continue to demonstrate the substantial impact of pain on numerous aspects of the lives of children and adolescents with SCD. Acute SCD pain sufficiently severe as to require hospitalization, was associated with increased levels of fatigue and deleterious consequences on depressive symptoms and physical functioning, similar to that reported using the PROMIS measures in adults with SCD, [29] or chronic noncancer pain disorders. [30] These results with the PROMIS pediatric measures were congruent with the literature that examined other symptom measurement scales, such as the Brief Pain Inventory in adults with SCD, [24] and the M. D. Anderson Symptom Inventory (MDASI) in adults with cancer pain. [31]

The large magnitude of changes for fatigue may reflect not only the impact of pain intensity on fatigue during hospitalization, but also the impact of worsening anemia typical of this disease exacerbation, and the sedating effects of intravenous opioids typically used for pain management in that setting. The rapid improvement in pain and fatigue scores following hospitalization with slower improvement in physical functioning scores suggests that different symptoms and various aspects of functioning may have different patterns of recovery following hospitalization.

In comparison to baseline values, pain interference, fatigue, depressive symptoms, anxiety, and both physical functioning measures were sensitive to the important clinical changes in symptoms experienced during hospitalization for severe sickle cell pain. These differences were similar or above the level of change (2-3 points) that healthcare providers and parents/children identified as recognizably different, often referred to as the minimally important difference (MID).[32] Not surprising in a clinical setting, we observed variability in intensity of symptoms and PROMIS pediatric domain responses among participants during the various clinical encounters; further appropriately designed studies are needed with these measures in SCD clinical populations to confirm relationships between PROMIS pediatric scores and clinical severity, as has been done for the Peds QL [20], or that may signal the need for clinical intervention, such as depression. [33] Evolving efforts to provide closer integration of PRO instruments with electronic medical record systems would facilitate these types of studies.[34,35]

Limitations

There are a number of limitations to our study. Although our study sample was diverse in age, sex, and type of hemoglobinopathies, we did not have a probability sample and findings cannot be generalized to all patients with SCD. We only studied episodes of pain sufficiently severe to require hospitalization, so the sensitivity of these measures to symptom changes caused by milder pain managed at home has not been established. This study had a complex set of missing values; for simplicity of analyses, we assumed all missing values were missing completely at random. Also, the GLM models used did not account for individual or disease-related factors that may be correlated to these PRO scores, such as demographic or hematologic variables.

In future research, we plan to assess the effect of missing data and also study demographic and other clinical explanatory variables that may impact these PRO scores. Further studies of symptom recovery patterns or of symptoms preceding acute care visits may be facilitated by measures with enhanced sensitivity over shorter periods of time, such as using 24 hour recall periods to assess daily changes in pain and other symptoms. [36] Successful PRO studies will provide careful attention to minimizing participant burden, facilitate measure completion using optimal formats for the assessment event, and maximize integration with clinical care.

Conclusion

Children and adolescents with SCD hospitalized for pain management, the most common cause for hospitalization in pediatric SCD, experienced substantial worsening of several symptoms such as pain, fatigue, depression, and considerable impairment of physical functioning, which improved to variable degrees as they recovered at home. PROMIS Pediatric measures were responsive to these changes in acute vaso-occlusive pain, fatigue, and in physical and emotional functioning. The reports of substantial levels of fatigue in SCD in our study has also has been reported by others [37-39] and suggests the need for more attention to this potentially distressing symptom, as does the apparent slow recovery in physical functioning after hospital discharge. The magnitude of changes for many of these domains, particularly pain interference and fatigue, suggest the PROMIS measures’ potential usefulness in clinical trials of novel therapies for vaso-occlusion. Further studies may find a role in using PROMIS measures in clinical practice settings to monitor a patient's health status over time.