Abstract
Objective:
Accurate and efficient measurement of patient-reported outcomes is key in cancer symptom management trials. The newer Patient Reported Outcomes Measurement Information System (PROMIS) and previously developed measures of similar conceptual content (legacy) are available to measure symptoms and functioning. This report compares the performance of two sets of measures, PROMIS and legacy, in a recently completed trial of a supportive care intervention that enrolled breast cancer patients and their friend or family caregivers.
Methods:
Patient-caregiver dyads (N = 256) were randomized to either reflexology delivered by caregivers or usual care control. Post-intervention, PROMIS and legacy measures of symptoms and functioning were analyzed in relation to trial arm, while adjusting for baseline values. Responsiveness of the two sets of measures was assessed using effect sizes and P-values for the effect of trial arm on patients’ and caregivers’ symptom and functioning outcomes.
Results:
Similar conclusions about intervention effects were found using PROMIS and legacy measures for pain, fatigue, sleep, anxiety, physical, and social functioning. Different conclusions were obtained for patient and caregiver depression: legacy measures indicated the efficacy of reflexology, while PROMIS depression measure did not.
Conclusion:
Evidence of similar responsiveness supports the use of either set of measures for symptoms and functioning in clinical and general populations. Differences between PROMIS and legacy measures of depression need to be considered when choosing instruments for use in trials of supportive care interventions and in clinical practice.
Keywords: cancer, legacy measures, measurement, PROMIS, symptom management
1 |. BACKGROUND
Symptoms and functioning are key patient-reported outcomes (PROs) that have gained prominence in research. One of the challenges in using PROs is the choice of measurement tools from many available instruments that gather patient perceptions on a given concept. For example, symptom severity scores may be based on single items or multi-item measures. The overall burden from multiple symptoms may be reflected in composite scores versus collections of separate symptom items that do not form scales. Similar issues apply to many of the existing measures of patient-reported physical, emotional, and social functioning.
The choice between single-item versus multiple-item measures and, more generally, between shorter and longer measures is challenging because on one hand, shorter measures carry less burden for both the patient as well as for the administrator. On the other hand, shorter instrument length may be insufficient to adequately cover the underlying construct and capture important differences among subgroups.
Further, the choice of general versus disease-specific measures is informed not only by the need to reflect PRO aspects unique to the disease, but also by the need to compare disease subgroups within the general population. Standardized measures that could be used with the general population allow for comparisons among multiple population subgroups and across studies, while disease-specific measures may cover clinically significant aspects of the construct that can be missed by disease-general measures.
A new set of instruments developed for the general population, the Patient Reported Outcomes Measurement Information System (PROMIS), provides publicly available brief standardized measures. PROMIS Profile1 measures reflect seven health-related quality of life domains: depression, anxiety, physical functioning, pain interference, fatigue, sleep disturbance, and satisfaction with participation in social roles. In this study, the shortest PROMIS-29 Profile v1.0 is compared with a collection of instruments of similar conceptual content but created prior to the PROMIS initiative and referred to as legacy measures. The legacy measures used in the initial PROMIS development and validation studies2–5 included the Medical Outcomes Study Short Form 36 (SF-36), the Center for Epidemiologic Studies-Depression (CESD) scale, the State Anxiety scale, and the Pittsburgh Sleep Symptom Questionnaire-Insomnia (PSSQ_I). In this study, the M.D. Anderson Symptom Inventory (MDASI) was also considered for symptoms experienced by patients with cancer. This set of legacy measures includes disease-general and disease-specific tools that have either longer or shorter length compared with the 4-item short forms that comprise PROMIS-29 Profile.1
While extensive evidence on the validity of PROMIS tools is available,2–4,6 their responsiveness, ie, the ability to discriminate groups, is beginning to be addressed in observational studies involving clinical populations.7–11 However, evidence of responsiveness of PROMIS measures from randomized controlled trials (RCTs) is lacking, with the exception of measures of pain interference in populations of patients with musculoskeletal pain or among stroke survivors.12,13 Responsiveness in the general or palliative care populations has not yet been evaluated in RCTs.14 This report shares the experience of using PROMIS and legacy measures in a recently completed RCT of reflexology for the management of symptoms experienced by women with advanced breast cancer during treatment. Friend or family caregivers were trained to deliver reflexology to women in the home setting as supportive care.
The attractiveness of PROMIS measures for this study stemmed from the ability to use these measures with both the clinical sample of breast cancer patients and the general population sample of their friend or family caregivers. Understanding both patients’ and caregivers’ outcomes resulting from engagement of friends or family members in patients’ symptom management is critical. Caring for the patient often has negative consequences to the caregiver’s family resources and health,15 and the added responsibility of reflexology delivery could have increased caregiver burden and worsened caregiver health. Conversely, meaningful involvement in patient care can improve caregiver health,16,17 and interventions that support the patient may also indirectly support the caregiver due to the interdependence in dyad’s outcomes.17,18 Thus, changes in symptoms and functioning could have occurred among both patients and caregivers who participated in the trial. We therefore asked, “Do PROMIS and legacy measures yield comparable findings when evaluating trial results for the patient-caregiver dyads?”
The primary aim of the parent study focused on the patient outcome of summed severity index of cancer-and treatment-related symptoms from the MDASI legacy instrument. The results of the primary analysis are reported elsewhere.19 An exploratory aim of the parent trial used a different set of measures for patient and caregiver symptoms and functioning to contribute to the growing evidence of psychometric properties of PROMIS measures by comparing their responsiveness to that of legacy measures.
1.1 |. Methods
1.1.1 |. Consent/enrollment
Participants (N = 256 patient-caregiver dyads) were recruited from two comprehensive cancer centers and five community-based oncology settings across the Midwest. The study was approved by investigators’ university including all recruitment sites’ Institutional Review Boards (file number: Michigan State University (MSU): C11–268; Northwestern University: STU00055139-MODCR0001) and registered on clinicaltrial.gov as NCT01582971. All patients and caregivers completed written informed consent.
1.1.2 |. Patient inclusion and exclusion criteria
Patient inclusion criteria were as follows: (1) ≥age 21; (2) diagnosis of stage III or IV breast cancer; (3) ability to perform basic activities of daily living; (4) receiving chemotherapy and/or hormonal therapy; (5) ability to speak and understand English; (6) access to a telephone; (7) ability to hear normal conversation; (8) cognitive orientation to time, place, and person (determined by recruiter); and (9) friend or family caregiver participating with them. Patient exclusion criteria were as follows: (1) diagnosis of major mental illness in the medical record; (2) residing in a nursing home; (3) bedridden; (4) currently receiving regular reflexology; or (5) symptoms of deep vein thrombosis or painful foot neuropathy.
1.1.3 |. Caregiver inclusion and exclusion criteria
The inclusion criteria for caregivers were as follows: (1) a friend/family member identified as a caregiver by the patient; (2) ≥age 18; (3) ability and willingness to provide the 30-minute reflexology protocol for four consecutive weeks; (4) ability to speak and understand English; (5) access to a telephone; and (6) ability to hear normal conversation. Exclusion criterion was unwillingness to perform a return-demonstration of the reflexology protocol with 90% accuracy according to training procedures.
1.1.4 |. Sample. The sample size of N = 256 dyads was set in the parent trial
where patient’s symptom severity index was the primary outcome. All 256 caregivers participated in the trial with patients and delivered reflexology to patients if randomized to the intervention group. Caregivers did not have to provide data about their own health but had an option to do so. The majority (70%) of the caregivers agreed to be interviewed about their own health outcomes. This report uses all patient and caregiver data collected at baseline (N = 256 patients, N = 180 caregivers) and week-5 post-intervention (N = 207 patients, N = 137 caregivers).
1.1.5 |. Protocol
After the baseline telephone interview, dyads were randomized to either reflexology or the control arm. The intervention arm received four weekly sessions of reflexology delivered to the patient at home by the caregiver trained to administer a manualized protocol, whereas the control group received usual health care only. Post-intervention telephone interviews were conducted at week-5.
1.1.6 |. Data collection
The following instruments were administered in baseline and week-5 via telephone interviews by trained interviewers who were blinded to participants’ group assignment. Patients and caregivers were interviewed separately about their own symptoms and functioning.
1.2 |. PROMIS measures
1.2.1 |. Item calibrations for PROMIS tools enable the generation of T-scores
that have a mean of 50 and standard deviation of 10 for the general United States (US) population.5 Higher scores indicate more of the construct being measured, eg, worse symptom experience or greater satisfaction. Both the PROMIS-29 Profile v1.0 and one additional PROMIS Short form v2.0 were included in these analyses. PROMIS guidelines were followed in scoring, and no missing item responses were present among patients and caregivers who completed baseline and week-5 telephone interviews.
1.2.2 |. PROMIS-29 Profile v1.01
(patients and caregivers). The PROMIS profile instrument covers seven health-related quality of life domains: depression, anxiety, physical functioning, pain interference, fatigue, sleep disturbance, and satisfaction with participation in social roles, with 4 items in each domain and a single pain intensity rating, for a total of 29 items.1 The internal consistency reliability at baseline was as follows: depression: 0.87 for patients, 0.91 for caregivers; anxiety: 0.88 for patients, 0.91 for caregivers; physical functioning 0.91 for caregivers; pain interference: 0.95 for patients, 0.93 for caregivers; fatigue 0.93 for patients, 0.90 for caregivers; sleep disturbance 0.85 for patients, 0.84 for caregivers; satisfaction with participation in social roles: 0.95 for patients, 0.92 for caregivers.
PROMIS Short Form v2.0—Ability to Participate in Social Roles and Activities20,21
(caregivers only) consists of 4 items reflecting trouble respondents have performing leisure, family, usual work activities, and activities with friends. The internal consistency reliability was 0.91 at baseline.
1.3 |. Legacy measures
The Medical Outcomes Study Short Form (SF-36) V222 Health Survey (one-week recall)
(patients and caregivers) measures eight health attributes. T-scores for the US general population were calculated, with higher scores reflecting better outcomes. Internal consistency reliability was not computed as scoring was proprietary by Quality Metric, Inc. In this report, five of the eight attributes were analyzed: physical functioning, bodily pain, vitality, mental health, and social role functioning. These constructs have a close conceptual mapping to the following constructs in PROMIS Profile v1.0: physical functioning, pain interference, pain severity, fatigue, depression, anxiety, satisfaction with participation in social roles (patients and caregivers), and Ability to Participate in Social Roles and Activities (caregivers).
M.D. Anderson Symptom Inventory (MDASI)23
(cancer-specific, patients only) evaluates severity of 13 symptoms: pain, fatigue, nausea, disturbed sleep, distress, shortness of breath, difficulty remembering, decreased appetite, drowsiness, dry mouth, sadness, vomiting, numbness/tingling, and the interference of these symptoms with daily life. Severity of each symptom is rated on the scale from 0 (not present) to 10 (as bad as you can imagine). Higher scores on the MDASI indicate worse outcomes. This report compares patient symptoms of pain, fatigue, disturbed sleep, sadness, and distress severity to the five related domains of PROMIS Profile v1.0: pain, fatigue, sleep disturbance, and depression.
Pittsburgh Sleep Symptom Questionnaire-Insomnia (PSSQ_I)24
(caregivers only because patients rated insomnia in the MDASI) is a 13-item instrument rating insomnia. Higher scores indicate better sleep outcomes. The internal consistency reliability at baseline was 0.86 for sleep quality, and 0.94 for sleep interference. This report examined sleep quality (range 0–15) and sleep interference (range 8–40) in relation to the PROMIS Profile v1.0 sleep disturbance domain score.
Center for Epidemiologic Studies-Depression CES-D25
(patients and caregivers) measures depressive symptomatology. Higher scores indicate worse depressive symptoms. The internal consistency reliability at baseline was 0.91 for each sample, patients, and caregivers. The CES-D score was compared with the PROMIS Profile v1.0 depression domain score.
Anxiety (State Anxiety)26
(patients and caregivers) includes 20 items. Evaluated are feelings of apprehension, tension, nervousness, and worry. Higher scores indicate greater anxiety. The internal consistency reliability at baseline was 0.94 for patients, and 0.95 for the caregivers. State anxiety score was compared with the PROMIS Profile v1.0 anxiety domain score.
1.4 |. Additional measures
Demographic data for patients and caregivers were collected at intake. Comorbid conditions were assessed using the Bayliss checklist of 21 conditions;27 omitting the cancer item for patients. Medical records provided data on patient disease and treatment.
1.4.1 |. Data analysis
The comparisons of PROMIS and legacy measures were conducted separately for the patient sample and the caregiver sample. Descriptive statistics for PROMIS and legacy measures at baseline were used to summarize their distributions in patient and caregiver samples, including percentages of observations at floor (worst) and ceiling (best). Correlations between pairs of measures of similar conceptual content were calculated. Responsiveness was assessed by comparing reflexology and control arms at week-5 on symptoms and functioning using PROMIS versus legacy measures. General linear models for this analysis included trial arm and baseline values as covariates. The least square (LS) means of each symptom and functioning outcome by trial arm were output from the models, and differences between them were tested. The adjusted effect sizes were computed as Cohen’s d, the differences between LS means divided by the square root of the mean squared error.
Whether a difference between trial arms for a particular outcome exists or not, the statistical conclusion about its existence should be similar based on the two sets of measures (PROMIS or legacy) of similar conceptual content. In the interpretation of results, the focus was not only on the statistical significance, but on the consistency of conclusions and magnitude of the effect sizes estimated using PROMIS and legacy measures. All statistical analyses were performed in SAS 9.4.28
2 |. RESULTS
The demographic characteristics of patients and caregivers were similar (Table 1), with a higher proportion of caregivers working full time, the majority being men, and having on average one less comorbid condition than patients. Cancer recurrence was present in 29% of patients, 63% had metastatic disease, and 82% were treated with chemotherapy or targeted therapy (with or without hormonal therapy), while 28% were on hormonal therapy alone.
TABLE 1.
Baseline characteristics of patients and caregivers
| Patients N = 256 | Caregivers N = 180 | |
|---|---|---|
| Mean (St. Dev.) | Mean (St. Dev.) | |
| Age | 56.44 (11.08) | 55.01 (15.30) |
| Number of comorbid conditions | 4.39 (2.95) | 3.17 (2.68) |
| N (%) | N (%) | |
| Sex | ||
| Male | 0 (0) | 102 (56.67) |
| Female | 256 (100) | 78 (43.33) |
| Race | ||
| American Indian or Alaska native | 1 (0.39) | 1 (0.56) |
| White | 218 (85.16) | 149 (82.78) |
| Black or African American | 28 (10.94) | 17 (9.44) |
| Asian | 4 (1.56) | 5 (2.78) |
| Refused/NA | 5 (1.95) | 8 (4.44) |
| Ethnicity | ||
| Hispanic or Latino | 12 (4.69) | 6 (3.33) |
| Not Hispanic or Latino | 240 (93.75) | 172 (95.56) |
| Refused | 4 (1.56) | 2 (1.11) |
| Marital status | ||
| Never married | 27 (10.55) | 26 (14.44) |
| Married or living with partner | 179 (69.92) | 136 (75.56) |
| Divorced/separated | 35 (13.67) | 12 (6.67) |
| Widowed | 15 (5.86) | 6 (3.33) |
| Employment | ||
| Full time | 71 (27.73) | 79 (43.89) |
| Part time | 21 (8.20) | 21 (11.67) |
| Retired | 69 (26.95) | 54 (30.00) |
| Disabled | 35 (13.67) | 2 (1.11) |
| Homemaker | 19 (7.42) | 3 (1.67) |
| Not employed/other | 36 (14.06) | 20 (11.11) |
| Refused | 5 (1.95) | 1 (0.56) |
Distributions of the T-scores from PROMIS and legacy measures (SF-36) were similar at baseline (Table 2) within each sample. The correlations between PROMIS and legacy measures of similar conceptual content were strong (r ≥ 0.6) or very strong (r ≥ 0.8), with the exception of correlations among measures of social functioning that were moderate (0.47 to 0.57). Floor and ceiling effects were not substantial in the patient sample, with the exception of PROMIS depression that had over one third of the observations at the ceiling (best scores, Table 2). In the caregiver sample, approximately two third of the observations were at the ceiling on PROMIS depression. Such ceiling effects were not seen on the legacy measures of depression for either the patient or the caregiver sample.
TABLE 2.
Descriptive statistics of PROMIS and legacy measures for patient and caregivers
| PROMIS | Patients Mean (St Dev) % at Floor (Worst); % at Ceiling (Best), Multi-Item Scales Only | Caregivers Mean (St Dev) % at Floor (Worst); % at Ceiling (Best), Multi-Item Scales Only | Legacy | Patients Mean (St Dev) % at Floor (Worst); % at Ceiling (Best), Multi-Item Scales Only | Caregivers Mean (St Dev) % at Floor (Worst); % at Ceiling (Best), Multi-Item Scales Only |
|---|---|---|---|---|---|
| PROMIS-29 physical function profile v1.0 (4 items) | 41.18 (7.84) 1.57%; 10.98% | 53.17 (6.94) 0%; 73.74% | SF-36 physical function (10 items) | 40.53 (9.52) 0.39%; 2.34% | 52.79 (7.89) 0%; 42.78% |
| PROMIS-29 pain interference profile v1.0 (4 items) | 55.80 (9.47) 2.35%; 23.92% | 47.05 (7.52) 0.56%; 61.11% | SF-36 bodily pain (2 items) | 45.01 (10.35) 2.35%; 11.76% | 54.60 (8.78) 0%; 44.44% |
| PROMIS-29 pain severity profile v1.0 (1 item) | 3.60 (2.71) | 1.63 (2.03) | MDASI pain severity (1 item, patient) | 3.57 (3.16) | - |
| PROMIS-29 fatigue profile v1.0 (4 items) | 58.76 (8.98) 4.31%; 2.35% | 47.28 (8.11) 0.56%; 13.97% | SF-36 vitality (4 items) | 43.92 (10.29) 1.57%; 0.78% | 55.67 (7.94) 0%; 3.89% |
| - | - | MDASI fatigue severity (patient, 1 item) | 5.83 (2.73) | - | |
| PROMIS-29 sleep disturbance profile v1.0 (4 items) | 53.37 (8.24) 2.75%; 2.35% | 45.56 (8.03) 0%; 12.78% | MDASI disturbed sleep (patient, 1 item) | 4.76 (3.26) | - |
| - | - | PSSQ-I sleep quality (caregiver, 5 items) | - | 8.07 (6.42) 1.12%;17.32% | |
| - | - | PSSQ-I sleep interference (caregiver, 8 items) | - | 13.87 (6.28) 0.70%; 9.79% | |
| PROMIS-29 depression profile v1.0 (4 items) | 50.46 (8.10) 0.39%; 33.86% | 45.14 (6.97) 1.11%; 67.22% | CES-D (20 items) | 15.13 (10.52) 0%; 2.76% | 6.89 (7.94) 0%; 13.33% |
| - | - | MDASI sadness severity (patient, 1 item) | 3.22 (3.13) | - | |
| - | - | MDASI distress severity (patient, 1 item) | 3.80 (3.05) | - | |
| - | - | SF-36 mental health (5 items) | 49.62 (9.30) 0%; 4.31% | 54.82 (7.88) 0%; 8.33% | |
| PROMIS-29 anxiety profile 1.0 (4 items) | 52.69 (8.82) 0.40%; 23.32% | 48.08 (8.48) 1.12%; 43.58% | State anxiety (20 items) | 34.33 (11.67) 0%; 9.06% | 28.97 (10.54) 0%; 22.22% |
| PROMIS-29 satisfaction with participation in social roles profile v1.0 (4 items) | 45.13 (8.40) 7.45%; 7.06% | 55.42 (7.78) 0.56%; 34.44% | SF-36 social functioning (2 items) | 43.79 (10.88) 2.35%; 22.75% | 53.66 (6.61) 0%; 67.78% |
| PROMIS short form v2.0—ability to participate in social roles and activities (4 items) | 56.37 (7.55) 0%; 39.66% |
Baseline characteristics of those who completed versus did not complete the week-5 interview are in Table 3. Patients who did not complete week-5 had worse physical functioning and pain, but otherwise did not differ from completers. Among caregivers, the dropouts had less depressive symptoms but otherwise did not differ from completers. Since the analysis of responsiveness adjusted for baseline values of each outcome, differences in attrition did not influence the conclusions about responsiveness of measures.
TABLE 3.
Baseline descriptive statistics of PROMIS measures for patient and caregivers stratified by having completed the 5-week interview or not
| Patients | Caregivers | |||||
|---|---|---|---|---|---|---|
| Outcome | Completed N = 191 Mean (St Dev) | Not Completed N = 65 Mean (St Dev) | P-Value | Completed N = 131 Mean (St Dev) | Not Completed N = 49 Mean (St Dev) | P-Value |
| PROMIS-29 physical function profile v1.0 | 41.98 (7.68) | 38.77 (7.88) | 0.01 | 53.24 (6.78) | 52.96 (7.41) | 0.81 |
| PROMIS-29 pain interference profile v1.0 | 54.86 (9.61) | 58.60 (8.51) | 0.01 | 47.10 (7.30) | 46.91 (8.15) | 0.88 |
| PROMIS-29 pain severity profile v1.0 | 3.40 (2.66) | 4.20 (2.78) | 0.04 | 1.65 (2.08) | 1.59 (1.90) | 0.87 |
| PROMIS-29 fatigue profile v1.0 | 58.59 (8.82) | 59.28 (9.49) | 0.59 | 47.03 (7.66) | 47.95 (9.24) | 0.50 |
| PROMIS-29 sleep disturbance profile v1.0 | 53.88 (8.05) | 51.87 (8.65) | 0.09 | 45.34 (8.00) | 46.14 (8.19) | 0.55 |
| PROMIS-29 depression profile v1.0 | 49.99 (7.75) | 51.82 (8.97) | 0.12 | 44.43 (6.23) | 47.05 (8.43) | 0.02 |
| PROMIS-29 anxiety profile v1.0 | 52.25 (8.25) | 53.98 (10.29) | 0.18 | 47.40 (8.04) | 49.88 (9.38) | 0.08 |
| PROMIS-29 satisfaction with participation in social roles profile v1.0 | 45.50 (8.43) | 44.03 (8.28) | 0.23 | 55.49 (7.22) | 55.24 (9.17) | 0.85 |
| PROMIS short form v2.0—Ability to participate in social roles and activities | 56.60 (7.31) | 55.74 (8.20) | 0.50 | |||
In the patient sample, the conclusions about intervention effects were similar using PROMIS and legacy measures of pain. Significant differences between trial arms were seen using PROMIS and legacy measures of pain severity with the effect size of one third of the standard deviation or higher. Differences between trial arms were smaller and not statistically significant on patient pain for the PROMIS interference measure and the SF-36 bodily pain, in which one of the two items queried about pain interference. No differences in pain measures were seen for the caregivers, with consistent small effect sizes across different measures (Table 4).
TABLE 4.
Summary of responsiveness in capturing intervention effect at week 5 for PROMIS and legacy measures: least square (LS) means, standard errors (SE), effect sizes (ES), and P-values
| Domain | Patient Measure | Reflexology Arm LS Mean (SE) | Control Arm LS Mean (SE) | Reflexology v. Control Arm ES (P-Value) |
|---|---|---|---|---|
| Physical function | PROMIS-29 physical function profile v1.0 | 41.93 (0.54) | 41.87 (0.52) | 0.01 (0.94) |
| SF-36 physical function | 43.17 (0.67) | 41.62 (0.64) | 0.24 (0.10) | |
| Pain | PROMIS-29 pain interference profile v1.0 | 52.90 (0.77) | 54.63 (0.74) | 0.23 (0.11) |
| SF-36 bodily pain | 48.34 (0.92) | 46.14 (0.89) | 0.25 (0.09) | |
| PROMIS-29 pain severity profile v1.0 | 2.87 (0.22) | 3.51 (0.21) | 0.31 (0.04) | |
| MDASI pain severity | 2.09 (0.22) | 3.06 (0.21) | 0.46 (<.01) | |
| Fatigue | PROMIS-29 fatigue profile v1.0 | 55.46 (0.75) | 57.35 (0.72) | 0.26 (0.07) |
| SF-36 vitality | 46.98 (0.81) | 45.45 (0.78) | 0.20 (0.18) | |
| MDASI fatigue severity | 3.52 (0.24) | 4.24 (0.23) | 0.31 (0.03) | |
| Sleep | PROMIS-29 sleep disturbance profile v1.0 | 51.21 (0.76) | 51.87 (0.73) | 0.09 (0.54) |
| MDASI disturbed sleep | 2.84 (0.26) | 3.11 (0.25) | 0.11 (0.45) | |
| Depression/mental health | PROMIS-29 depression profile v1.0 | 49.17 (0.63) | 49.50 (0.60) | 0.06 (0.70) |
| CES-D | 11.76 (0.77) | 14.09 (0.74) | 0.32 (0.03) | |
| MDASI sadness severity | 1.56 (0.21) | 1.94 (0.21) | 0.19 (0.20) | |
| MDASI distress severity | 1.68 (0.23) | 2.35 (0.22) | 0.31 (0.04) | |
| SF-36 mental health | 53.76 (0.76) | 50.08 (0.73) | 0.51 (<.01) | |
| Anxiety | PROMIS-29 anxiety profile v1.0 | 50.40 (0.71) | 52.42 (0.67) | 0.30 (0.04) |
| State anxiety | 30.93 (0.84) | 34.05 (0.80) | 0.39 (0.01) | |
| Social functioning | PROMIS-29 satisfaction with participation in social roles profile v1.0 | 47.45 (0.72) | 46.87 (0.70) | 0.08 (0.57) |
| SF-36 social functioning | 47.99 (0.88) | 45.41 (0.85) | 0.31 (0.04) | |
| Domain | Caregiver Measure | Reflexology Arm LS Mean (SE) | Control Arm LS Mean (SE) | Reflexology v. Control Group ES (P-Value) |
| Physical function | PROMIS-29 physical function profile v1.0 | 53.00 (0.55) | 52.74 (0.56) | 0.06 (0.74) |
| SF-36 physical function | 53.14 (0.33) | 53.00 (0.33) | 0.05 (0.76) | |
| Pain | PROMIS-29 pain interference profile v1.0 | 47.22 (0.72) | 47.01 (0.72) | 0.04 (0.84) |
| SF-36 bodily pain | 55.15 (0.74) | 54.21 (0.74) | 0.16 (0.37) | |
| PROMIS-29 pain severity profile v1.0 | 1.63 (0.19) | 1.55 (0.19) | 0.06 (0.76) | |
| Fatigue | PROMIS-29 fatigue profile v1.0 | 45.56 (0.77) | 48.15 (0.78) | 0.42 (0.02) |
| SF-36 vitality | 56.77 (0.76) | 55.54 (0.76) | 0.20 (0.26) | |
| Sleep | PROMIS-29 sleep disturbance profile v1.0 | 45.08 (0.89) | 45.44 (0.90) | 0.05 (0.78) |
| PSSQ-I sleep quality | 8.20 (0.63) | 8.12 (0.63) | 0.02 (0.93) | |
| PSSQ-I sleep interference | 14.73 (0.55) | 14.31 (0.56) | 0.11 (0.60) | |
| Depression/mental health | PROMIS-29 depression profile v1.0 | 44.48 (0.56) | 45.07 (0.56) | 0.13 (0.46) |
| CES-D | 5.35 (0.66) | 7.48 (0.66) | 0.40 (0.02) | |
| SF-36 mental health | 56.59 (0.66) | 54.70 (0.66) | 0.35 (0.05) | |
| Anxiety | PROMIS-29 anxiety profile v1.0 | 46.66 (0.73) | 47.88 (0.74) | 0.20 (0.25) |
| State anxiety | 27.20 (0.81) | 28.37 (0.81) | 0.18 (0.31) | |
| Social functioning | PROMIS-29 satisfaction with participation in social roles profile v1.0 | 56.60 (0.84) | 55.25 (0.85) | 0.20 (0.26) |
| PROMIS short form v2.0—Ability to participate in social roles and activities | 57.26 (0.73) | 55.82 (0.73) | 0.24 (0.16) | |
| SF-36 social functioning | 54.73 (0.76) | 52.90 (0.77) | 0.30 (0.09) |
For fatigue, the effects sizes in the patient sample ranged from 0.20 to 0.31 and were consistent across measures, reaching significance for severity from the MDASI tool. Also consistently, PROMIS and legacy measures indicated sizable significant effects of reflexology for the management of anxiety among patients, not among the caregivers, and no intervention effects on sleep for either patients or caregivers.
By contrast, very different conclusions about intervention effects were seen using legacy measures of depression compared with PROMIS depression. The CES-D and the SF-36 mental health for both patients and caregivers, and MDASI distress item for patients, all pointed to significant improvement in the reflexology group compared with control, with the effect sizes of one third to one half of the standard deviation, often deemed clinically significant in the PRO literature.29,30 However, with the PROMIS depression measure, there were no significant differences noted for either the patient or caregiver sample, and the estimated effect sizes were very small. Further, no significant differences were found for MDASI sadness among patients.
Two other differences in responsiveness were noted in the patient or caregiver sample but not in both. For patients, a larger effect size of 0.34 (P = .04) was seen with the SF-36 for social functioning compared with PROMIS satisfaction with participation in social roles d = 0.08 (P = 0.57). For caregivers, larger benefits of the reflexology intervention were seen with PROMIS fatigue (d = 042, P = 0.02) versus SF-36 vitality (d = 0.20, P = 0.26).
3 |. CONCLUSIONS
This report summarizes the experience of using PROMIS and legacy measures of symptoms and functioning in the same RCT with a clinical sample of advanced breast cancer patients and a community sample of their friend or family caregivers. Using the shortest available forms of PROMIS measures, the conclusions about the differences between trial arms were very similar to the conclusions obtained using the legacy measures, most of which were longer than the 4-item PROMIS forms. Different conclusions were obtained for patient and caregiver depression: legacy measures indicated the efficacy of reflexology, while the PROMIS depression measure did not. Short length with low respondent burden, applicability to both clinical and general populations and standardization of T-scores for the general US population make PROMIS 4-item forms an excellent choice for future trials of supportive care interventions.
Consistent with other studies that found no differences in the measurement properties of symptom scales based on the recall periods, this study found no differences in responsiveness on pain severity for PROMIS (7-day recall) versus MDASI (24-hour recall).31 Our findings also agree with reports of the validity of PROMIS pain measures in clinical populations3,32 other than advanced breast cancer, and with reported close mapping of PROMIS pain interference on the legacy Brief Pain Inventory (BPI) pain interference scores among patients with multiple sclerosis.33 In contrast to the present findings of very consistent performance of PROMIS and legacy measures on pain interference among women with advanced breast cancer and their caregivers, the responsiveness of the BPI and SF-36 was superior compared with the PROMIS pain interference short form among patients with persistent musculoskeletal pain.32
Beyond length and responsiveness, conceptual content is a key consideration in the choice of measurement tools. The titles of PROMIS measures of social functioning reflect the conceptualization of this construct by PROMIS developers as involvement in, and satisfaction with, one’s usual social roles in life’s situation or activities.3,5 This conceptual division into “ability to participate” and “satisfaction with participation” make the content of the PROMIS measures of social function somewhat different from the SF-36 social functioning subscale considered in this study as a legacy measure. This difference in content could potentially explain differences in responsiveness on social functioning noted in the patient sample. Similarly, lack of vitality in the SF-3634 may not be the same as fatigue in the PROMIS, and the MDASI’s sadness and distress are not an exact conceptual match to PROMIS depression.
By design, the PROMIS depression item bank does not contain somatic items and focuses on negative mood, decrease in positive affect, information processing deficits, negative views of self, and negative social cognition.5,35 Of these, the 4-item short form includes items reflecting negative affect (feeling worthless, helpless, depressed, and hopeless). Whereas, the 20-item CES-D includes 7 items reflecting somatic concerns, which are relevant to people with cancer36,37 and their caregivers.38 Further, the CES-D positively-worded items such as “I felt hopeful about the future” may be qualitatively different from PROMIS’, “I felt hopeless” due to differences between absence of positive affect versus elevated negative affect.39 The mental health subscale of the SF-36 includes 5 items reflecting positive and negative affect, anxiety, and loss of behavioral or emotional control. Greater responsiveness and conceptual content, not limited to negative affect and relevant for cancer survivors and their caregivers, strongly support the use of the legacy CES-D and the SF-36 mental health versus PROMIS depression 4-item short form in the populations reflected by this trial’s patient and caregiver samples. Present findings also indicate that the issue may not lie solely with length, but also in the content. The one-item MDASI distress item for patients demonstrated similar responsiveness to the longer CES-D and SF-36 mental health subscale, and the MDASI item of sadness yielded the same conclusion as a longer PROMIS depression measure reflecting negative affect. These findings suggest that the intervention did not reduce negative affect, but impacted depression by reducing other concerns reflected by the CES-D and SF-36 mental health for patients and caregivers, and MDASI distress for patients.
3.1 |. Limitations
Due to measurement and statistical hypothesis testing errors, it is unknown which set of measures produced a true conclusion of intervention differences on depression. However, because several legacy tools produced the same result which differed from the results obtained using PROMIS depression, and because this finding was seen in both patient and caregiver samples, it is likely that the legacy measures performed better in capturing intervention effects on depression. This finding is further supported by reductions seen in other patient symptoms and literature documenting associations of depression with other cancer-related symptoms.40
These findings apply to the reflexology intervention and not necessarily to other interventions. Generalizability of findings is also limited by lack of racial and ethnic diversity reflective of the populations served by the participating oncology settings. Despite these limitations, the study was unique in using both established legacy and newer PROMIS measures in a single clinical trial.
3.2 |. Clinical implications
These findings inform clinicians that the use of brief measures for symptoms does not lead to the loss of measurement validity compared with the use of longer measures. The exception is depressive symptoms, where the longer legacy measures have the advantage of covering not just negative affect, but also mood and somatic concerns that are often present in cancer populations. This report can inform choices in research and practice of supportive care interventions that benefit from reduced respondent burden, while maintaining measurement validity and precision.
ACKNOWLEDGEMENT
The research team would like to thank our dedicated participants who made this study possible.
This work was funded by the National Cancer Institute (1R01CA157459) and the MSU Clinical and Translational Sciences Institute (216).
Funding information
National Cancer Institute, Grant/Award Number: 1 R01CA157459; MSU Clinical and Translational Sciences Institute, Grant/Award Number: 216
Footnotes
CONFLICT OF INTEREST
None declared.
REFERENCES
- 1.PROMIS List of Adult Measures. Available from: http://www.healthmeasures.net. Accessed June 18, 2018.
- 2.Schalet BD, Hays RD, Jensen SE, Beaumont JL, Fries JF, Cella D. Validity of PROMIS physical function measured in diverse clinical samples. J Clinical Epidemiology. 2016;73:112–118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Cook KF, Jensen SE, Schalet BD, et al. PROMIS measures of pain, fatigue, negative affect, physical function, and social function demonstrated clinical validity across a range of chronic conditions. J Clin Epidemiol. 2016;73:89–102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cella D, Lai JS, Jensen SE, et al. PROMIS fatigue item Bank had clinical validity across diverse chronic conditions. J Clin Epidemiol. 2016;73:128–134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Cella D, Riley W, Stone A, et al. Initial item banks and first wave testing of the patient-reported outcomes measurement information system (PROMIS) network: 2005–2008. J Clinical Epidemiology. 2010;63(11):1179–1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Askew RL, Cook KF, Revicki DA, Cella D, Amtmann D. Evidence from diverse clinical populations supported clinical validity of PROMIS pain interference and pain behavior. J Clinical Epidemiology. 2016;73:103–111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hahn EA, Beaumont JL, Pilkonis PA, et al. The PROMIS® satisfaction with social participation measures demonstrate responsiveness in diverse clinical populations. J Clin Epidemiol. 2016;73:135–141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Jensen RE, Moinpour CM, Potosky AL, et al. Responsiveness of 8 patient-reported outcomes measurement information system (PROMIS) measures in a large, community-based Cancer study cohort. Cancer. 2017;123(2):327–335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hays RD, Spritzer KL, Fries JF, Krishnan E. Responsiveness and minimally important difference for the patient-reported outcomes measurement and information system (PROMIS®) 20-item physical functioning short-form in a prospective observational study of rheumatoid arthritis. Ann Rheum Dis. 2015;74(1):104–107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hung M, Saltzman CL, Greene T, et al. Evaluating instrument responsiveness in joint function: the HOOS JR, the KOOS JR, and the PROMIS PF CAT. J Orthopaedic Research. 2018;36(4):1178–1184. [DOI] [PubMed] [Google Scholar]
- 11.Lee AC, Driban JB, Price LL, Harvey WF, Rodday AM, Wang C. Responsiveness and minimally important differences for 4 patient-reported outcomes measurement information system short forms: physical function, pain interference, depression, and anxiety in knee osteoarthritis. J Pain. 2017;18(9):1096–1110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kean J, Monahan PO, Kroenke K, et al. Comparative responsiveness of the PROMIS pain interference short forms, brief pain inventory, peg, and SF-36 bodily pain subscale. Med Care. 2016;54(4):414–421. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Chen CX, Kroenke K, Stump TE, et al. Estimating minimally important differences for the PROMIS pain interference scales: results from 3 randomized clinical trials. Pain. 2018;159(4):775–782. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Aslakson RA, Dy SM, Wilson RF, et al. Patient and caregiver-reported assessment tools for palliative care: summary of the 2017. Agency for healthcare research and quality technical brief. J. Pain Symptom Manag 2017;54(6):961–972. [DOI] [PubMed] [Google Scholar]
- 15.Schulz R, Sherwood PR. Physical and mental health effects of family caregiving. Am J Nurs. 2008;108(9 Suppl):23–27. quiz 27 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Frambes D, Sikorskii A, Tesnjak I, Wyatt G. Caregiver health outcomes: the effects of providing reflexology for symptom-management for women with advanced breast cancer. Oncol Nurs Forum. 2017;44(5):596–605. [DOI] [PubMed] [Google Scholar]
- 17.Kim Y Given B. Quality of life of family caregivers of cancer survivors across the trajectory of the illness. Cancer. 2008;112(S11):2556–2568. [DOI] [PubMed] [Google Scholar]
- 18.Segrin C, Badger TA, Sikorskii A, Crane TE, Pace TWW. A dyadic analysis of stress processes in Latinas with breast cancer and their family caregivers. Psychooncology. 2018;27(3):838–846. [DOI] [PubMed] [Google Scholar]
- 19.Wyatt G, Sikorskii A, Tesnjak I, et al. A randomized clinical trial of caregiver-delivered reflexology for symptom management during breast cancer treatment. J Pain and Symptom Management. 2017;54(5):670–679. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Hahn EA, DeVellis RF, Bode RK, et al. On behalf of the PROMIS cooperative group. Measuring social health in the patient-reported outcomes measurement information system (PROMIS): item bank development and testing. Qual Life Res. 2010;19(7):1035–1044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Hahn EA, DeWalt DA, Bode RK, et al. New English and Spanish social health measures will facilitate evaluating health determinants. Health Psychol. 2014;33(5):490–499. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.SF-36v2 Health Survey Acute. 2000.
- 23.Cleeland C, Mendoza T, Wang XS, et al. Assessing symptom distress in cancer patients: the M.D Anderson symptom inventory. Cancer. 2000;89(7):1634–1646. [DOI] [PubMed] [Google Scholar]
- 24.Okun ML, Kravitz HM, Sowers MF, Moul DE, Buysse DJ, Hall M. Psychometric evaluation of the insomnia symptom questionnaire: a self-report measure to identify chronic insomnia. J. Clin. Sleep Med 2009;5(1):41–49. [PMC free article] [PubMed] [Google Scholar]
- 25.Radloff LS, Locke BZ. The Community Mental Health Assessment Survey and the CES-D Scale. New Brunswick: Rutgers. University Press; 1986. [Google Scholar]
- 26.Spielberger CD, Gorsuch RL, Lushene R, Vagg PR, Jacobs GA. Manual for the State-Trait Anxiety Inventory (Form Y). Mind Garden, Inc.: Palo Alto; 1983. [Google Scholar]
- 27.Bayliss EA, Ellis JL, Steiner JF. Seniors’ self-reported multimorbidity captured biopsychosocial factors not incorporated into two other data-based morbidity measures. J Clinical Epidemiology. 2009;62(5):550–557. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.SAS institute Inc. SAS 9.4 Cary, NC. 2013. [Google Scholar]
- 29.Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003;41(5):582–592. [DOI] [PubMed] [Google Scholar]
- 30.Sloan JA, Cella D, Hays RD. Clinical significance of patient-reported questionnaire data: another step toward consensus. J Clinical Epidemiology. 2005;58(12):1217–1219. [DOI] [PubMed] [Google Scholar]
- 31.Shi Q, Trask PC, Wang XS, et al. Does recall period have an effect on cancer patients’ ratings of the severity of multiple symptoms? J Pain Symptom Manage. 2010;40(2):191–199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Askew R, Kim J, Chung H, Cook KF, Johnson KL, Amtmann D. Development of a crosswalk for pain interference measured by the BPI and PROMIS pain interference short form. Qual Life Res. 2013;22(10):2769–2776. [DOI] [PubMed] [Google Scholar]
- 33.Cella D, Bullinger M, Scott C, Barofsky I, Clinical Significance Consensus Meeting Group. Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Mayo Clinic Proceeding. 2002;77(4):384–392. [DOI] [PubMed] [Google Scholar]
- 34.Deng N, Guyer R, Ware JE Jr. Energy, fatigue, or both? A bifactor modeling approach to the conceptualization and measurement of vitality. Qual Life Res. 2015;24(1):81–93. [DOI] [PubMed] [Google Scholar]
- 35.Schalet BD, Pilkonis PA, Yu L, et al. Clinical validity of PROMIS depression, anxiety, and anger across diverse clinical samples. J Clinical Epidemiology. 2016;73:119–127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Nelson CJ, Cho C, Berk AR, Holland J, Roth AJ. Are gold standard depression measures appropriate for use in geriatric cancer patients? A systematic evaluation of self-report depression instruments used with geriatric, cancer, and geriatric cancer samples. J Clinical Oncology. 2010;28(2):348–356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Tishelman C, Lovgren M, Broberger E, Hamberg K, Sprangers MA. Are the most distressing concerns of patients with inoperable lung cancer adequately assessed? A mixed-methods analysis. J Clinical Oncology. 2010;28(11):1942–1949. [DOI] [PubMed] [Google Scholar]
- 38.Kotronoulas G, Wengstrom Y, Kearney N. Sleep patterns and sleep-impairing factors of persons providing informal care for people with cancer: a critical review of the literature. Cancer Nurs. 2013;36(1): E1–E15. [DOI] [PubMed] [Google Scholar]
- 39.Watson D, Clark LA, Carey D. Positive and negative affectivity and their relation to anxiety and depressive disorders. J Abnormal Psychology. 1988;97(3):346–353. [DOI] [PubMed] [Google Scholar]
- 40.Miaskowski C, Dunn L, Ritchie C, et al. Latent class analysis reveals distinct subgroups of patients based on symptom occurrence and demographic and clinical characteristics. J Pain Symptom Manage. 2015;50(1):28–37. [DOI] [PMC free article] [PubMed] [Google Scholar]