Abstract
Background:
The importance of patient reported outcomes (PROs) in cellular therapies, including hematopoietic cell transplantation (HCT) was highlighted. Longitudinal collection of PROs in a registry is recommended for several reasons. Yet, to date there is no routine collection of PROs among HCT patients to augment clinical registry data.
Objective:
To determine the feasibility of electronic PRO data collection by a national clinical outcomes registry by assessing differences in who does and does not report PRO.
Study Design:
We conducted a cross-sectional pilot collection of PROs from HCT patients after treatment using computer adaptive tests from Patient Reported Outcome Measurement Information System (PROMIS). We implemented centralized data collection through the Center for International Blood and Marrow Transplant Research (CIBMTR) among patients who received an HCT for myelodysplastic syndromes (MDS), were at least six months post-HCT, and spoke English or Spanish. The main objective was identifying patient, disease, and transplant-related differences associated with completion of electronic PROs. Patients were excluded from analysis if they were determined to be ineligible (deceased, did not speak English or Spanish, refused to be contacted by CIBMTR).
Results:
163 patients were contacted and potentially eligible to participate, 92 (56%) enrolled, and 89 (55%) completed the PRO assessment. Largest source of incomplete surveys was inability to contact patients (n=88), followed by declining the study (n=37). There were no sociodemographic or age differences between those who completed PROs (n=89) and eligible non-responders (n=155). Patients were within 3 points of the U.S. average (50) for all symptoms and functioning except physical functioning.
Conclusions:
Responders and non-responders did not exhibit meaningfully different sociodemographic characteristics. Difficulty contacting patients posed the largest barrier and an opportunity for improvement. Once enrolled, survey completion was high. These results support standardizing centralized PRO data collection through the CIBMTR registry.
Keywords: patient reported outcomes, clinical outcomes registry, hematopoietic cell transplantation, cellular therapy, Patient Reported Outcome Measurement Information System (PROMIS)
Introduction
Cellular therapies (CT), including hematopoietic cell transplantation (HCT), are an established treatment for many hematological conditions.1 For over 45 years, the Center for International Blood and Marrow Transplant Research (CIBMTR) has operated as an observational outcomes registry collecting longitudinal clinical data for HCT recipients and, more recently, other cellular therapies. As of 2020, there are more than 550,000 patients represented in the registry with extensive data including patient, disease, treatment characteristics, exposures, clinical outcomes, and late effects. Additionally, the registry contains linked pre-transplant samples for >57,000 allogeneic HCT recipient and donor pairs. The CIBMTR registry is a valuable resource for clinical investigators, and provides a mechanism whereby important clinical questions posed by the community can be addressed in collaboration with CIBMTR scientific and statistical experts. CIBMTR has published >1,500 peer-reviewed publications since its inception.
This rich database, however, lacks patient reported outcomes (PROs), which are reports by patients of their symptoms and function without interpretation by others, such as a clinician.2 In recent years, the importance of PROs, both as indicators of outcomes that are important to patients, as well as valid predictors of clinical outcomes, was recognized by the HCT community and are now sometimes collected in individual treatment centers and on prospective clinical trials. 3–7 Longitudinal collection of PROs in a registry is recommended for several reasons.6 For example, PROs improve understanding of symptom trajectories, inform clinicians of patient variability in experience despite seemingly similar clinical outcomes, and expand available information for hypothesis driven analyses that include the patient experience and perspective.7 However, to date, there is no routine collection of PROs by the CIBMTR to augment clinical data. Clinical data are submitted to the CIBMTR by staff at the treating centers, who also obtain consent from the patient to allow their data to be included in research (https://www.cibmtr.org/About/dataprotection/Pages/default.aspx). Historically, CIBMTR has not had direct contact with patients.
Comprehensive interpretation of PROs in HCT is limited by the use of non-comparable measures across studies. The need to agree upon a set of measures that are free, easily accessible internationally, and address core domains of symptoms and function associated with HCT was recently highlighted.3 The use of Patient Reported Outcome Measurement Information System (PROMIS) was recommended for several reasons. There is strong evidence for the reliability and validity of PROMIS measures in multiple contexts.3 PROMIS measures are available across multiple formats (electronic and paper) and languages8 and scores can be compared across formats on a common metric. Previous studies demonstrate comparability to legacy instruments such as the SF-36 in the HCT population.9 A critical advantage of PROMIS is the ability to deliver measures using computer adaptive testing (CAT),10 where the questions a person answers are individually tailored to previous responses to reduce response burden and increase score precision.
The objectives of this study were to: (1) agree on a set of PRO measures for routine use in HCT recipients under a standard IRB-approved protocol; (2) develop an electronic PRO (ePRO) collection system; (3) centralize all consenting and operational functions within the CIBMTR to reduce burden on centers; (4) pilot the ePRO system in a population of patients previously enrolled on a prospective data collection study to assess the rate of participation and to determine whether there were differences in clinical and/or sociodemographic characteristics between patients who agreed to participate versus those that did not.
Materials and Methods:
Harmonized PRO measures
CIBMTR convened a panel of experts (including HCT physicians, PRO experts, clinical HCT researchers, patients and caregivers) to recommend the domains and measurement tools most relevant to the HCT population. PROMIS measures were selected with preferred delivery through an electronic system (though capacity for paper surveys was maintained). The domains recommended were: PROMIS CAT v1.0 - Depression11, PROMIS CAT v1.0 - Anxiety11, PROMIS CAT v2.0 - Emotional Support12, PROMIS CAT v2.0 - Physical Function13,14, PROMIS CAT v1.1 - Pain Interference15, PROMIS CAT v1.0 - Fatigue16, PROMIS CAT v1.0 - Sleep Disturbance17, and PROMIS CAT v2.0 - Ability to Participate in Social Roles and Activities12. We also included sociodemographic questions based on previous studies18 and socio-economic data fields (e.g. family income and education levels) which are typically only entered into the CIBMTR registry by transplant center data managers. All measures were available in English and Spanish.
Electronic PRO (ePRO) collection system
CIBMTR developed an ePRO system for routine PRO data collection to be utilized in the context of the registry. PRO surveys were developed in Qualtrics, communicating with an application programming interface (API) link to the PROMIS measures delivered using CAT technology. CIBMTR’s contact management systems are used to track and trigger PRO questionnaire dissemination. PRO data is stored in the CIBMTR database where they are linked to individual recipient clinical data using a unique CIBMTR identifier. Informed consent forms are built directly into the electronic system. The system is flexible and scalable to include any additional questionnaires. The protocol for PRO data collection was approved by the National Marrow Donor Program (NMDP) IRB. Participant’s informed consent in this protocol was obtained by CIBMTR staff. CIBMTR’s Survey Research Group (SRG) performed all functions related to PRO collection.
Pilot study to assess feasibility of the ePRO
Patient Selection and Enrollment Procedures
The cross-sectional pilot study leveraged an existing population of patients ≥55 years old who consented to a CIBMTR longitudinal study to understand the impact of age on outcomes of patients who underwent allogeneic HCT for myelodysplastic syndromes (MDS).19 The parent study focused on age differences and confirmed the hypothesis that older age (>65) was not associated with worse mortality following an allogeneic HCT, however no quality of life (QOL) outcomes were assessed. Six centers were selected to participate due to the numbers enrolled on the parent study protocol, race and ethnic diversity of enrolled patients at their sites and their interest in participating: Medical College of Wisconsin, The Ohio State University, H. Lee Moffitt Cancer Center, Stanford University, Karmanos Cancer Center/Wayne State University, and Memorial Sloan Kettering Cancer Center.
Inclusion criteria for our study were: ≥6 months from transplant to ensure patients had recovered from their initial transplant, English or Spanish speaking, an active email address, and consent to future contact by CIBMTR. Since CIBMTR did not have contact information for patients, lists of potentially eligible patients were sent to transplant centers to evaluate eligibility through electronic health records and provide contact details to CIBMTR. Some centers needed to contact patients to confirm their consent to be contacted for PRO research. Patients who were found to be deceased or ineligible (i.e. English or Spanish were not patient’s first language) – either at the time transplant centers were confirming eligibility or at contact attempt by SRG – were removed from analysis (Figure 1). The SRG phoned each potential participant to confirm eligibility and describe the study. For participants who verbally agreed to participate, the SRG then obtained electronic written consent for study participation and sent a link to the PRO measures. The SRG followed up with non-responders up to 12 times (average 3.6 times).
Figure 1. Flow Diagram for Analytic Sample, indicating Refusals, Implicit Refusals, and Unknown Eligibility.
Unknown eligibility = Patients were never able to be contacted to confirm eligibility; Refusal = Patients were contacted and verbally declined to be contacted about research or declined this study; Implicit Refusal = Patients eligibility was confirmed and initial contact was made, but patient did not respond to follow up contact or complete survey
Statistical Analysis
Descriptive statistics on patient, disease, transplant, and sociodemographic characteristics were performed for patients who completed ePROs and for those who were potentially eligible but did not complete ePROs. Ineligible patients removed from analysis included those deceased, whose first language was not English or Spanish, or withdrew from contact for research.
PROMIS domains were scored using response-pattern scoring through the Health Measures Assessment Center API. PROMIS domains use a T-score where 50 reflects the mean of the US population and 10 points represents one standard deviation among US population. Though specific for each domain, prior studies identify a 2-5 points as meaningful.20–22 As such, we considered 5 points (1/2 standard deviation) to be meaningful. Continuous variables were summarized with means and standard deviations.
Factors significantly associated with enrollment and completion of PROs were identified using logistic regression; adjustment for HCT performing center was done by treating it as a random effect in the model. To explore what factors were associated with each PRO domain score, linear regression was used. Factors considered for the logistic and linear regression included: (1) patient related (gender, age at transplant, Karnofsky Performance Score (KPS) at the time of HCT, race/ethnicity, HCT- comorbidity index (HCT-CI), time of PRO completion defined as time interval from HCT to the date of PRO survey submission; (2) socioeconomic related (education, annual income, insurance, marital status, employment, ability to pay monthly bills, need for a caregiver); (3) disease related (cytogenetics, secondary MDS, time from diagnosis to transplant); and, (4) transplant related (graft type, donor/recipient human leukocyte antigen (HLA) match, conditioning regimen, graft-versus-host-disease (GVHD) prophylaxis, anti-thymocyte globulin (ATG)/alemtuzumab as part of conditioning or GVHD prophylaxis, prior transplant, year of transplantation, history of grade 2-4 acute GVHD, chronic GVHD, and relapse). Since age was a primary variable in the parent study,23 age was forced into the logistic regression model assessing feasibility. For both regressions, variables were prescreened using univariate regression models and only those significant at 0.1 level were further considered for inclusion in the multivariable model. Only variables significant at 0.05 level of significance were retained and presented in the final model table.
Results:
Patients
Study recruitment and enrollment occurred between July 2018 and November 2019. Of 273 patients identified by CIBMTR as potentially eligible, transplant centers were able to provide contact information for 190 (70%) potentially eligible patients. Four patients (1.5%) actively declined participation at this phase. Of the 190 where contact was attempted by SRG, 163 (86%) could be contacted and had their eligibility confirmed; 92 patients (56%) enrolled in the study, of whom 89 (97%) completed the PRO assessment. Of 71 patients who did not enroll, 37 (52%) actively declined to participate (refusal), and 34 (48%) were unreachable or unresponsive to multiple SRG contact attempts (unknown eligibility, unable to be contacted) (Supplemental Appendix A). Reasons that patients declined included: not interested in research (n=17), too ill to participate or vision problems (n=8), technology barriers (n=8), and concerns with the study including data privacy concerns (n=4). In summary, for the 155 patients who did not complete ePROs, 41 actively declined to participate (refusal), while 77 had unknown eligibility because they were unable to be contacted and 37 were implicit refusals, either did not respond to multiple attempts to contact or did not complete surveys (Figure 1).
Contact Attempts of ePRO Collection
SRG metrics are shown in Figure 2. The median number of days from first day of recruitment outreach to survey completion was 22 days (range = 2 – 321), with 90% of patients completing in 68 days or less. Once CIBMTR obtained consent and provided patients with a survey link, the median time for completion was 5 days (range = 0 – 306 days), with 90% of patients completing in 26 days or less.
Figure 2:
The average number of days between milestones made by the CIBMTR survey research group at each stage of activity within the ePRO
Data Completeness and Respondent Burden
All patients who began the ePRO surveys completed them, taking an average of 18.3 minutes, with time to complete a single CAT PROMIS domain ranging from 0.7 to 1.2 minutes.
We compared data for sociodemographic variables collected by direct patient report (ePRO system) to data for the same patient characteristics entered on the CIBMTR clinical data entry forms by staff at the transplant centers. Patient reported data were more complete compared to the data entered by data managers at the centers (Supplemental Table 1). For example, marital status and job status were 100% complete in the ePRO system compared to the clinical registry (9% missing). Even more striking, data on income were reported by 87% of respondents but for only 6% via the clinical registry.
Predictors of ePRO completion
We compared the baseline characteristics between patients who completed ePRO (n=89) with those who were potentially eligible but did not complete ePROs (n=155) (Table 1). The groups were well balanced, with the main clinical differences related to variables that differed by transplant center (TC) and likely reflect practice variability (GVHD disease prophylaxis and year of transplant). There were differences by TCs, with the percent of patients progressing through the study to the final step of completing ePROs ranging from 20.6% to 54.3%. Additional analyses to assess differences between patients who actively refused to participate in ePRO completion and those who implicitly refused or were unable to be contacted demonstrated similar findings (Supplemental Appendix B). The median time from transplant to survey request was 32 months (range 9 – 95) for those completing ePROs with no differences from patients who actively (26 months; range 11-89) or passively (31 months; range 10-77) declined to participate. To assess post-transplant complications among patients who completed ePROs compared to those who declined, separate univariate analyses of relapse, acute GVHD, and chronic GVHD (models not shown) showed no statistically significant differences in completion for key adverse outcomes.
Table 1:
Baseline patient characteristics of patients based on completion of ePROs
| Variable | Completed ePRO, n(%) | Eligible and did not complete ePRO, n(%) |
|---|---|---|
| Patient related | ||
| Number of patients | 89 | 155 |
| Number of study centers | 6 | 6 |
| Centers | ||
| A | 19 (21) | 16 (10) |
| B | 15 (17) | 18 (12) |
| C | 7 (8) | 27 (17) |
| D | 27 (30) | 39 (25) |
| E | 9 (10) | 34 (22) |
| F | 12 (13) | 21 (14) |
| Gender | ||
| Male | 55 (62) | 96 (62) |
| Female | 34 (38) | 59 (38) |
| Age at transplant, years | ||
| 55-64 | 27 (30) | 55 (35) |
| 65+ | 62 (70) | 100 (65) |
| Karnofsky performance at HCT | ||
| 90-100 | 53 (60) | 85 (55) |
| 80-89 | 28 (31) | 41 (26) |
| <80 | 8 (9) | 28 (18) |
| Missing | 0 | 1 (<1) |
| Race/ethnicity | ||
| Caucasian, non-Hispanic | 82 (92) | 132 (85) |
| African-American, non-Hispanic | 3 (3) | 3 (2) |
| Asian, non-Hispanic | 2 (2) | 1 (<1) |
| Pacific islander, non-Hispanic | 0 | 1 (<1) |
| Caucasian, Hispanic | 0 | 8 (5) |
| Race unknown/missing | 2 (2) | 10 (7) |
| HCT-CI | ||
| 0 | 24 (27) | 24 (15) |
| 1-2 | 23 (26) | 44 (28) |
| 3+ | 42 (47) | 86 (55) |
| Missing | 0 | 1 (<1) |
| Disease related | ||
| Cytogenetics | ||
| Favorable | 40 (45) | 65 (42) |
| Intermediate | 23 (26) | 26 (17) |
| Poor | 25 (28) | 58 (37) |
| Not tested | 1 (1) | 5 (3) |
| Missing | 0 | 1 (<1) |
| Secondary MDS | ||
| No | 73 (82) | 128 (83) |
| Yes | 16 (18) | 26 (17) |
| Unknown | 0 | 1 (<1) |
| Time from diagnosis to transplant | ||
| <6 month | 25 (28) | 33 (21) |
| 6 month-1 year | 30 (34) | 60 (39) |
| >1 to 2 years | 16 (18) | 26 (17) |
| ≥2Y | 18 (20) | 36 (23) |
| Time from diagnosis to transplant, median (range), months | 8 (3 - 263) | 9 (3 - 179) |
| Transplant-related | ||
| Graft type | ||
| Bone marrow | 12 (13) | 11 ( 7) |
| Peripheral blood | 76 (85) | 142 (92) |
| Cord blood | 1 ( 1) | 2 ( 1) |
| Donor/recipient HLA match | ||
| Cord blood | 1 ( 1) | 2 ( 1) |
| HLA-identical siblings | 17 (19) | 43 (28) |
| Mismatched related | 6 ( 7) | 9 ( 6) |
| 8/8 unrelated | 61 (69) | 88 (57) |
| 7/8 unrelated | 4 ( 4) | 10 ( 6) |
| Unrelated (HLA match information missing) | 0 | 3 ( 2) |
| Conditioning regimen intensity | ||
| Myeloablative | 31 (35) | 58 (37) |
| RIC/NMA | 58 (65) | 97 (63) |
| GVHD prophylaxis | ||
| TCD | 3 ( 3) | 18 (12) |
| Post-CY +/− other(s) | 11 (12) | 13 ( 8) |
| Calcineurin inhibitor-based | 71 (80) | 110 (71) |
| Other GVHD prophylaxis | 1 ( 1) | 14 ( 9) |
| Missing | 3 ( 3) | 0 |
| ATG/CAMPATH GVHD prophylaxis | ||
| ATG alone | 20 (22) | 61 (39) |
| No ATG or CAMPATH | 69 (78) | 94 (61) |
| Prior transplant | ||
| No | 83 (93) | 149 (96) |
| Yes | 6 ( 7) | 6 ( 4) |
| Year of transplant | ||
| 2011-2014 | 24 (27) | 71 (46) |
| 2015-2018 | 65 (73) | 84 (54) |
| Socioeconomic (as reported by transplant center) | ||
| Education | ||
| Primary or elementary education | 0 | 2 ( 1) |
| Lower secondary education | 0 | 2 ( 1) |
| Upper secondary education | 14 (16) | 36 (23) |
| Post-secondary, non-tertiary education | 6 ( 7) | 8 ( 5) |
| Tertiary education, Type A | 17 (19) | 24 (15) |
| Tertiary education, Type B | 5 ( 6) | 9 ( 6) |
| Advanced research qualification | 7 ( 8) | 12 ( 8) |
| Not Answer/Missing | 40 (45) | 62 (40) |
| Household gross annual income | ||
| Less than $60,000 | 2 ( 2) | 3 ( 2) |
| $60,000 and over | 3 ( 3) | 2 ( 1) |
| Unknown/Not Answered/Missing | 84 (94) | 150 (97) |
ePRO = electronic patient reported outcome; HCT = Hematopoietic cell transplantation; HCT- CI = Hematopoietic cell transplantation comorbidity index; MDS= myelodysplastic syndromes; RIC/NMA= Reduced-intensity conditioning/ nonmyeloablative conditioning regimens; GVHD= graft-versus-host-disease; TCD= T cell depletion; Post-CY= post-transplant cyclophosphamide; ATG/CAMPATH= anti-thymocyte globulin/alemtuzumab; Tertiary education, Type A: Programs that provide education that is largely theoretical, lasting 3-4 years (US Equivalent: Includes university programs that last 4 years and lead to the award of a bachelor’s degree, and university programs that lead to a master’s degree)
Tertiary education, Type B: Programs that focus on practical, technical or occupational skills with a minimum duration of 2 years of full-time enrollment (US Equivalent: Programs typically offered at community colleges that lead to an associate’s degree)
In multivariable analysis (Table 2), patients who received a transplant more recently (OR 2.49, 95% CI 1.33-4.65, p=0.0045) or had an HCT-CI score of zero (compared to ≥1) prior to HCT (OR 2.38, 95% CI 1.19-4.77, p= 0.0145) were more likely to complete PROs. Four variables were significant at 0.1 level, with only one not remaining significant at 0.05 (ATC/CAMPATH) and were therefore not included in the model.
Table 2.
Logistic regression model showing factors associated with the completion of ePROs
| Characteristic | Group | N | OR (95% CI) | p-value |
|---|---|---|---|---|
| Year of HCT | 2011-2014 | 94 | 1.00 | 0.0024 |
| 2015-2018 | 149 | 2.56 (1.40-4.69) | ||
| HCT-CI | 1+ | 195 | 1.00 | |
| 0 | 48 | 2.40 (1.20 -4.80) | 0.0137 |
OR = Odds Ratio; HCT = hematopoietic cell transplant; HCT-CI = hematopoietic cell transplant comorbidity index
Patients were within 3 points of the U.S. average (50) for all symptoms and functioning except physical functioning (Table 3). The average physical function score was 44.3, indicating lower physical function compared to the U.S. average. Linear regression models were used to investigate the clinical and sociodemographic characteristics associated with PRO scores (Table 4). Of the sociodemographic factors, the need for a caregiver demonstrated significance across several domains: ability to participate in social roles, anxiety, depression, fatigue, pain interference and physical function. In all domains, those who need a caregiver reported at least a 6 point (1/2 standard deviation) worse functioning or increased symptoms compared to those who do not need a caregiver. For example, holding everything else constant, needing a caregiver was associated with significantly lower ability to participate in social roles, 11.5 points lower than for those who did not need a caregiver, which is >1 standard deviation. Those with higher incomes (≥$60,000/year) reported significantly greater ability to participate in social roles and significantly less anxiety, depression, fatigue, pain compared to lower income patients (<$60,000/year recipient’s combined household gross annual income). For example, holding everything else constant, household income ≥$60,000/year was associated with lower fatigue, 6.5 points lower than for those with lower income, which is >1/2 standard deviation. Few clinical characteristics were associated with PRO results. Pre-HCT cytogenetics (a marker of disease risk) were significantly associated with the ability to participate in social roles, pain interference, and physical function. Acute GVHD was associated with approximately one-half standard deviation lower emotional support (−5.1 points; p =0.0066). Secondary MDS was significantly associated with lower sleep disturbance by almost 6 points (−5.7 points; p=0.0143).
Table 3.
Patient reported outcomes (PROs), T-score means and standard deviations in parentheses
| Characteristic | Total (n=89) |
|---|---|
| Ability to Participate in Social Roles | 50.1 (9.3) |
| Emotional Support | 53.9 (8.6) |
| Physical Function | 44.3 (7.8) |
| Anxietya | 48.4 (8.2) |
| Depressiona | 47.7 (7.8) |
| Fatiguea | 50.8 (8.7) |
| Pain Interferencea | 49.2 (9.7) |
| Sleep Disturbancea | 49.3 (9.4) |
Higher numbers on these scores indicate worsening of the symptom or functioning (i.e. score of 50 means higher anxiety than a 40)
Table 4.
Linear Regressions models of factors independently associated with PRO scores
| Variable | Sample Size | Group | Estimate | Standard Error | p-value |
|---|---|---|---|---|---|
| Social Roles (n=88; 1 participant with missing cytogenetics removed) | |||||
| Intercept | 48.82 | 2.13 | 0.0004 | ||
| Cytogenetics | 40 | Favorable | 0.00 | . | 0.0012 (overall) |
| 23 | Intermediate | 2.57 | 2.13 | 0.2301 | |
| 25 | Poor | −6.57 | 2.08 | 0.0022 | |
| Household gross annual Income | 23 | Less than $60,000 | 0.00 | . | 0.0348 (overall) |
| 54 | $60,000 and over | 5.33 | 2.03 | 0.0103 | |
| 11 | Missing | 2.86 | 2.94 | 0.3324 | |
| Needs a caregiver | 80 | No | 0.00 | . | 0.0002 |
| 8 | Yes | −11.51 | 2.96 | ||
| Anxiety (n = 89) | |||||
| Intercept | 51.02 | 1.63 | <.0001 | ||
| Household gross annual Income | 23 | Less than $60,000 | 0.00 | . | 0.0392 (overall) |
| 55 | $60,000 and over | −4.20 | 1.92 | 0.0317 | |
| 11 | Missing | −6.45 | 2.84 | 0.0255 | |
| Needs a caregiver | 80 | No | 0.00 | . | 0.0002 |
| 9 | Yes | 7.30 | 2.72 | 0.0088 | |
| Depression (n=89) | |||||
| Intercept | 50.69 | 1.55 | <.0001 | ||
| Household gross annual Income | 23 | Less than $60,000 | 0.00 | . | 0.0233 (overall) |
| 55 | $60,000 and over | −5.07 | 1.83 | 0.0069 | |
| 11 | Missing | −4.60 | 2.70 | 0.0923 | |
| Needs a caregiver | 80 | No | 0.00 | . | . |
| 9 | Yes | 6.59 | 2.59 | 0.0128 | |
| Emotional support (n=89) | |||||
| Intercept | 50.44 | 2.39 | <.0001 | ||
| Time since HCT | 31 | 0-23 months | 0.00 | . | 0.0346 (overall) |
| 43 | 24-59 months | 2.25 | 1.89 | 0.2371 | |
| 15 | 60+ months | −4.14 | 2.52 | 0.1046 | |
| Paying bills | 18 | Slightly, somewhat, very, extremely, difficult | 0.00 | . | 0.0260 (overall) |
| 68 | Not at all difficult | 5.85 | 2.14 | 0.0078 | |
| 3 | Missing | 2.34 | 4.92 | 0.6364 | |
| aGVHD | 59 | No | 0.00 | . | 0.0215 (overall) |
| 27 | Yes | −5.13 | 1.84 | 0.0066 | |
| 3 | Missing | 0.93 | 4.71 | 0.8435 | |
| Fatigue (n=89) | |||||
| Intercept | 54.73 | 1.65 | <.0001 | ||
| Household gross annual Income | 23 | Less than $60,000 | 0.00 | . | 0.0033 (overall) |
| 55 | $60,000 and over | −6.47 | 1.94 | 0.0013 | |
| 11 | Missing | −7.22 | 2.86 | 0.0136 | |
| Needs a caregiver | 80 | No | 0.00 | . | . |
| 9 | Yes | 9.37 | 2.75 | 0.0010 | |
| Pain interference (n=88 as 1 participant with missing cytogenetics is removed) | |||||
| Intercept | 53.00 | 2.71 | <.0001 | ||
| Age | 27 | 55-64 | 0.00 | . | . |
| 62 | 65+ | −5.15 | 2.04 | 0.0136 | |
| Household gross annual Income | 23 | Less than $60,000 | 0.00 | . | 0.0106 (overall) |
| 54 | $60,000 and over | −6.62 | 2.23 | 0.0039 | |
| 11 | Missing | −2.13 | 3.21 | 0.5089 | |
| Needs a caregiver | 80 | No | 0.00 | . | . |
| 8 | Yes | 6.77 | 3.24 | 0.0397 | |
| Cytogenetics | 40 | Favorable | 0.00 | . | 0.0433 (overall) |
| 23 | Intermediate | −3.31 | 2.34 | 0.1606 | |
| 25 | Poor | 3.14 | 2.28 | 0.1722 | |
| Physical function (n=88 as 1 patient with missing cytogenetics is removed) | |||||
| Intercept | 47.66 | 1.09 | <.0001 | ||
| Cytogenetics | 40 | Favorable | 0.00 | . | 0.0002 (overall) |
| 23 | Intermediate | −0.68 | 1.73 | 0.6941 | |
| 25 | Poor | −7.07 | 1.69 | <.0001 | |
| Needs a caregiver | 80 | No | 0.00 | . | . |
| 8 | Yes | −11.49 | 2.46 | <.0001 | |
| Sleep disturbance (n=89) | |||||
| Intercept | 48.77 | 2.71 | <.0001 | ||
| Age at transplant | 27 | 55-64 | 0.00 | . | . |
| 62 | 65+ | −5.41 | 2.03 | 0.0094 | |
| Job | 11 | Part/full time employed | 0.00 | . | 0.0169 (overall) |
| 10 | Unemployed/on disability/other | 10.52 | 3.61 | 0.0046 | |
| 68 | Retired | 4.09 | 2.71 | 0.1358 | |
| Secondary MDS | 73 | No | 0.00 | . | . |
| 16 | Yes | −5.69 | 2.27 | 0.0143 | |
| Relapse | 63 | No | 0.00 | 0.0191 (overall) | |
| 22 | Yes | 5.10 | 2.04 | 0.0146 | |
| 4 | Missing | −4.57 | 4.28 | 0.2895 | |
Note: Only variables significant at 0.05 were retained in the final model; The following variables are patient reported: social roles, anxiety, depression, emotional support, fatigue, pain interference, physical function, sleep disturbance, household gross annual income, caregiver status, ability to pay bills, and job status
MDS= myelodysplastic syndromes
Discussion
We report here the development and implementation of a novel ePRO system, embedded within the operations of the CIBMTR clinical outcomes registry. We provide evidence for successful centralized ePRO collection, while also recognizing important opportunities for process improvements. Patient age (55-64 v. 65+ years old) and other demographics were not significantly associated with whether patients completed ePROs or not, and older age (65+ years old) at transplant was not associated with worse QOL.
Collecting PROs from patients on a large scale may be challenging due to logistic and other factors, but the utility of linking PRO to clinical outcome data for large populations is clear.24 This scaling of PRO data collection in an international registry setting is extremely challenging without an electronic system but others have reported high success rates of electronic PROs.25,26 In this study, we were able to successfully collect PROs for 89 patients using an electronic system that linked these data with clinical data routinely collected by CIBMTR. Although we were not able to collect data for all patients eligible for this study, the characteristics of participants and non-participants were very similar. We also identified several steps in the process from initial patient contact to PRO submission that led to loss of patient ePRO responses for which we propose solutions for, as described below.
We identified barriers in two main areas: those which could not be influenced directly by the CIBMTR and those that could. In the former, we found that transplant centers were unable to provide patient contact details to CIBMTR, due in part to time constraints of staff or IRB approvals. Similar to a previous feasibility study by CIBMTR6 where treating centers obtained informed consent from patients, we found barriers to decentralized IRB, recruitment, and consenting. To address this issue and relieve the burden on centers, we have since applied two solutions. First, we developed a protocol, approved by a centralized IRB, under which consenting for the ePRO collection can be done directly with the patient by CIBMTR staff. Second, we have put a mechanism in place to ask patients in advance if they agree to be contacted by CIBMTR. This allows CIBMTR to collect patient contact details at the first point of contact between the registries and the centers, and prior to the date the PRO is requested.
Barriers to enrollment was a second issue identified in the pilot. Of the patients the CIBMTR’s SRG attempted to contact, some either were never able to be contacted (17%), implicitly refused to participate (19%), and explicitly declined to participate (19%). Future solutions to mitigate this issue in routine PRO collection may include expanding the modes of recruitment, consenting, and data collection to reach those unable to be contacted. We intend to explore the impact of providing a postage-paid mail-in option on response rates. Future studies could explore other strategies, such as telephone-assisted collection27 for increasing response to routine PRO collection and reengaging patients who implicitly refuse.
A common barrier to enrollment reported from other cancer registries collecting PROs is differences by gender and socioeconomic status in responders compared to non-responders.24 In our study, in contrast, participation did not differ based on socio-demographic factors including age, sex or ethnicity, though with few non-Caucasian patients this finding should be interpreted with caution. Although there were some differences between responders and non-responders in clinical and transplant characteristics including comorbidities, cytogenetics, and GVHD prophylaxis, the small sample in this study preclude definitive conclusions. Potential reasons for these differences in responders and non-responders include a center effect, and a suggestion that patients less well or at higher risk prior to HCT may be less likely to respond to surveys.
Another reported barrier to PRO collection is respondent burden,7 which was reflected in some of the reasons patients declined to join our study. However, for those who completed the ePRO, patient burden appeared acceptable, with average time to completion for the entire survey at less than 20 minutes.
The study also pointed to some unique advantages of direct PRO collections. Transplant outcomes such as return to work and availability of a caregiver were captured in the PROs at significantly higher response rates than when submitted by a third-party (data professionals), suggesting this is a more efficient method to collect some important outcomes.
By using PROMIS measures we are able to assess functioning in the context of United States (US) averages, given PROMIS measures are normed with the US average translating to a score of 50. Mean PRO for this study fell within the normal limits for interpreting PROMIS T-score cut offs (i.e. did not demonstrate mild or moderate dysfunction).28 Physical function was the one domain in which patients reported significantly less function compared to US average physical function, which translates to mild functioning difficulty.28,29 Research on chronic diseases and QOL reports individuals with arthritis experience similar mild physical dysfunction.30 Results from this study demonstrated some age differences and post-HCT duration differences in symptom and functioning, though overall reports suggest little difference in ability to participate in social roles, anxiety, depression, or fatigue regardless of how old or how long ago patients had their transplant. Though future studies in larger, more diverse samples, are needed to confirm the findings of this study.
This study has important limitations. The sample is relatively small, drawing from only 6 centers in the US, did not include patients under the age of 55, and only included patients with MDS, thus limiting the generalizability of our findings. Additionally, PROs were offered only in English and Spanish, further limiting generalizability (although not many patients were lost in this US study due to this language restriction). The response rate of 56% was just under recommendations for oncology publications;31 yet importantly, there were no meaningful differences in patient sociodemographic characteristics between responders who completed PROs and potentially eligible patients who did not respond – and the response rate is felt to be more heavily influenced by our ability to make contact with patients than by patient refusal. Compliance with providing patient contact details was variable across sites. We believe many of these limitations can be alleviated by the identification of critical barriers to enrollment and robust strategies now in place, including upfront prospective collection of contact details by CIBMTR as early in the process as possible, and a central IRB-approved protocol obviating the need for center-specific IRB approval, which should result in higher enrollment in the routine PRO collection protocol. The CIBMTR is taking the knowledge gained from this pilot study to implement into a larger pilot expanding to other ages and diseases.
In conclusion, our study demonstrates that the incorporation of ePRO assessments into an international clinical registry is feasible and leverages the high quality clinical data available in the registry while adding the unique perspective that patients provide. Data suggest that except for physical function, average scores of patients are in keeping with US population. These provocative data are deserving of further study to better understand best practices for large scale PRO data collection and the valuable knowledge to be gained from patients perspectives of their symptoms and functioning. Based on these promising results, our next steps are to scale efforts to include a similar battery of PROs for routine collection by the CIBMTR.
Supplementary Material
Highlights:
Adding electronic patient reported outcomes to a clinical registry was feasible
No sociodemographic differences were found between responders and non-responders
Difficulty contacting patients posed largest barrier and opportunity for improvement
Once enrolled, survey completion was high
Acknowledgements:
Thank you to the participating sites: Medical College of Wisconsin, The Ohio State University, H. Lee Moffitt Cancer Center, Stanford University, Karmanos Cancer Center/Wayne State University, and Memorial Sloan Kettering Cancer Center.
Funding:
The CIBMTR is supported primarily by Public Health Service U24CA076518 from the National Cancer Institute (NCI), the National Heart, Lung and Blood Institute (NHLBI) and the National Institute of Allergy and Infectious Diseases (NIAID); HHSH250201700006C from the Health Resources and Services Administration (HRSA); and N00014-20-1-2705 and N00014-20-1-2832 from the Office of Naval Research; Support is also provided by Be the Match Foundation, the Medical College of Wisconsin, the National Marrow Donor Program, and from the following commercial entities: AbbVie; Accenture; Actinium Pharmaceuticals, Inc.; Adaptive Biotechnologies Corporation; Adienne SA; Allovir, Inc.; Amgen, Inc.; Astellas Pharma US; bluebird bio, inc.; Bristol Myers Squibb Co.; CareDx; CSL Behring; CytoSen Therapeutics, Inc.; Daiichi Sankyo Co., Ltd.; Eurofins Viracor; ExcellThera; Fate Therapeutics; Gamida-Cell, Ltd.; Genentech Inc; Gilead; GlaxoSmithKline; Incyte Corporation; Janssen/Johnson & Johnson; Jasper Therapeutics; Jazz Pharmaceuticals, Inc.; Karyopharm Therapeutics; Kiadis Pharma; Kite, a Gilead Company; Kyowa Kirin; Magenta Therapeutics; Medac GmbH; Merck & Co.; Millennium, the Takeda Oncology Co.; Miltenyi Biotec, Inc.; MorphoSys; Novartis Pharmaceuticals Corporation; Omeros Corporation; Oncopeptides, Inc.; Orca Biosystems, Inc.; Pfizer, Inc.; Pharmacyclics, LLC; Sanofi Genzyme; Seagen, Inc.; Stemcyte; Takeda Pharmaceuticals; Tscan; Vertex; Vor Biopharma; Xenikos BV. This project was supported by Medical College of Wisconsin Cancer Center.
Conflicts of Interest:
Dr. Muffly reports grants from Servier, grants from Adaptive, personal fees from Amgen, personal fees from Pfizer, personal fees from Kite, personal fees from Grail, grants from Baxalta, outside the submitted work.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Copelan EA Hematopoietic Stem-Cell Transplantation. N. Engl. J. Med 354, 1813–1826 (2006). [DOI] [PubMed] [Google Scholar]
- 2.Fayers PM & Machin D Quality of Life: The Assessment, Analysis and Interpretation of Patient-reported Outcomes. (John Wiley & Sons, 2013). [Google Scholar]
- 3.Shaw BE et al. Can we agree on patient-reported outcome measures for assessing hematopoietic cell transplantation patients? A study from the CIBMTR and BMT CTN. Bone Marrow Transplant. 51, 1173–1179 (2016). [DOI] [PubMed] [Google Scholar]
- 4.Pidala J, Anasetti C & Jim H Health-related quality of life following haematopoietic cell transplantation: patient education, evaluation and intervention. Br. J. Haematol 148, 373–385 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bevans MF et al. Health-related quality of life in patients receiving reduced-intensity conditioning allogeneic hematopoietic stem cell transplantation. Bone Marrow Transplant. 38, 101–109 (2006). [DOI] [PubMed] [Google Scholar]
- 6.Shaw BE et al. Centralized Patient-Reported Outcome Data Collection in Transplantation Is Feasible and Clinically Meaningful. Cancer 123, 4687–4700 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Gliklich RE, Dreyer NA & Leavy MB Use of Patient-Reported Outcomes in Registries. (2014). [Google Scholar]
- 8.Bjorner JB et al. Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. J. Clin. Epidemiol 67, 108–113 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Shaw BE et al. PROMIS measures can be used to assess symptoms and function in long-term hematopoietic cell transplantation survivors. Cancer 124, 841–849 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Cella D, Gershon R, Lai J-S & Choi S The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual. Life Res 16 Suppl 1, 133–141 (2007). [DOI] [PubMed] [Google Scholar]
- 11.Pilkonis PA et al. Item Banks for Measuring Emotional Distress From the Patient-Reported Outcomes Measurement Information System (PROMIS®): Depression, Anxiety, and Anger. Assessment 18, 263–283 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hahn EA et al. New English and Spanish social health measures will facilitate evaluating health determinants. Health Psychol. 33, 490–499 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Rose M, Bjorner JB, Becker J, Fries JF & Ware JE Evaluation of a preliminary physical function item bank supported the expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS). J. Clin. Epidemiol 61, 17–33 (2008). [DOI] [PubMed] [Google Scholar]
- 14.Rose M et al. The PROMIS Physical Function item bank was calibrated to a standardized metric and shown to improve measurement efficiency. J. Clin. Epidemiol 67, 516–526 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Amtmann D et al. Development of a PROMIS item bank to measure pain interference. Pain 150, 173–182 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Lai JS et al. How item banks and their application can influence measurement practice in rehabilitation medicine: A PROMIS fatigue item bank example. Arch. Phys. Med. Rehabil 92, S20–S27 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Buysse DJ et al. Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep 33, 781–792 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Anasetti C et al. Peripheral-blood stem cells versus bone marrow from unrelated donors. N. Engl. J. Med 367, 1487–1496 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Atallah E et al. Comparison of Patient Age Groups in Transplantation for Myelodysplastic Syndrome: The Medicare Coverage with Evidence Development Study. JAMA Oncol. 6, 486–493 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Chen CX et al. Estimating minimally important differences for the PROMIS pain interference scales: Results from 3 randomized clinical trials. Pain 159, 775–782 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Sandvall B, Okoroafor UC, Gerull W, Guattery J & Calfee RP Minimal Clinically Important Difference for PROMIS Physical Function in Patients With Distal Radius Fractures. J. Hand Surg. Am 44, 454–459.e1 (2019). [DOI] [PubMed] [Google Scholar]
- 22.Hays RD, Spritzer KL, Fries JF & Krishnan E Responsiveness and minimally important difference for the Patient-Reported Outcomes Measurement Information System (PROMIS) 20-item physical functioning short form in a prospective observational study of rheumatoid arthritis. Ann. Rheum. Dis 74, 104–107 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Atallah E et al. Comparison of patient age groups in transplantation for myelodysplastic syndrome: the Medicare Coverage with Evidence Development study. jamanetwork.comPaperpile [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.de Rooij BH et al. Cancer survivors not participating in observational patient-reported outcome studies have a lower survival compared to participants: the population-based PROFILES registry. Qual. Life Res 27, 3313–3324 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bush N et al. Development, feasibility and compliance of a web-based system for very frequent QOL and symptom home self-assessment after hematopoietic stem cell transplantation. Qual. Life Res 14, 77–93 (2005). [DOI] [PubMed] [Google Scholar]
- 26.Wood WA et al. Feasibility of Frequent Patient-Reported Outcome Surveillance in Patients Undergoing Hematopoietic Cell Transplantation. Biol. Blood Marrow Transplant 19, 450–459 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Lannin NA et al. Telephone follow-up was more expensive but more efficient than postal in a national stroke registry. J. Clin. Epidemiol 66, 896–902 (2013). [DOI] [PubMed] [Google Scholar]
- 28.Score Cut Points. Available at: https://www.healthmeasures.net/score-and-interpret/interpret-scores/promis/promis-score-cut-points. (Accessed: 17th September 2020)
- 29.Rothrock NE et al. Establishing clinically-relevant terms and severity thresholds for Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures of physical function, cognitive function, and sleep disturbance in people with cancer using standard setting. Qual. Life Res 28, 3355–3362 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Rothrock NE et al. Relative to the general US population, chronic diseases are associated with poorer health-related quality of life as measured by the Patient-Reported Outcomes Measurement Information System (PROMIS). J. Clin. Epidemiol 63, 1195–1204 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Parekh AD, Bates JE & Amdur RJ Response Rate and Nonresponse Bias in Oncology Survey Studies. Am. J. Clin. Oncol 43, 229–230 (2020). [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.