Improving the Care of Children With Advanced Cancer by Using an Electronic Patient-Reported Feedback Intervention: Results From the PediQUEST Randomized Controlled Trial

Abstract

Purpose

This study aimed to determine whether feeding back patient-reported outcomes (PROs) to providers and families of children with advanced cancer improves symptom distress and health-related quality of life (HRQoL).

Patients and Methods

This study was a parallel, multicentered pilot randomized controlled trial. At most once per week, children age ≥ 2 years old with advanced cancer or their parent completed the computer-based Pediatric Quality of Life and Evaluation of Symptoms Technology (PediQUEST) survey consisting of age- and respondent-adapted versions of the Memorial Symptom Assessment Scale (MSAS), Pediatric Quality of Life Inventory 4.0 Generic Core Scales (PedsQL4.0), and an overall Sickness question. In the intervention group (n = 51), oncologists and families received printed reports summarizing PROs; e-mails were sent to oncologists and subspecialists when predetermined scores were exceeded. No feedback was provided in the control group (n = 53). Primary outcomes included linear trends of MSAS, PedsQL4.0 total and subscale scores, and Sickness scores during 20 weeks of follow-up, along with child, parent, and provider satisfaction with PediQUEST feedback.

Results

Feedback did not significantly affect average MSAS, PedsQL4.0, or Sickness score trends. Post hoc subgroup analyses among children age ≥ 8 years who survived 20 weeks showed that feedback improved PedsQL4.0 emotional (+8.1; 95% CI, 1.8 to 14.4) and Sickness (−8.2; 95% CI, −14.2 to −2.2) scores. PediQUEST reports were valued by children, parents, and providers and contributed at least sometimes to physician initiation of a psychosocial consult (56%).

Conclusion

Although routine feedback of PROs did not significantly affect the child's symptoms or HRQoL, changes were in expected directions and improvements observed in emotional HRQoL through exploratory analyses were encouraging. Importantly, children, parents, and providers value PRO feedback.

INTRODUCTION

At any one time, thousands of children are living with advanced cancer, and nearly 1,400 children die annually from the disease.¹ These children experience high levels of suffering from physical and emotional symptoms and poor quality of life,^2–5 with long-term impact on surviving family members.⁶ Ensuring the best possible quality of life for children with cancer is a high national priority.^7–9

Experts have suggested that presenting providers and families with patient-reported outcomes (PROs) may improve patients' health-related quality of life (HRQoL) by enhancing communication and improving physician awareness and ability to monitor symptoms and HRQoL over time.^10,11 A few studies have assessed this type of intervention in adult patients with cancer^12–17; however, the evidence regarding their impact on patient outcomes is inconclusive.¹⁸ Only one study evaluating a Web-based feedback intervention (KLIK PROfile)¹⁹ in children was identified. By using a sequential cohort design, authors found that providing PRO feedback to rheumatology providers increased discussion of emotional and family topics.²⁰ No pediatric randomized controlled trials (RCTs) have evaluated the effects of feeding back PROs.^21,22

Pediatric Quality of Life and Evaluation of Symptoms Technology (PediQUEST) is a computer-based data collection system that collects patient- (or parent-) reported symptoms and HRQoL data and generates printed feedback reports and e-mail alerts. In this pilot study, we randomly assigned children with cancer that had advanced beyond initial treatment to evaluate the effects of feeding back summary PediQUEST data to oncologists and families. We hypothesized that the intervention would lead to decreased symptom distress and improved HRQoL and, among other secondary hypotheses, that family members and oncologists would value feedback. This article presents these two analyses.

PATIENTS AND METHODS

Design and Setting

The PediQUEST Study is a pilot parallel 1:1 RCT conducted at three large US pediatric cancer centers: Dana-Farber Children's Cancer and Blood Disorders Center (December 2004 to November 2009), Children's Hospital of Philadelphia (September 2006 to July 2009), and Seattle Children's Hospital (February 2008 to December 2009).

Participants

Eligible children were age ≥ 2 years old with at least a 2-week history of progressive, recurrent, or nonresponsive cancer or for whom there was a decision not to pursue cancer-directed therapy. Parents were required to speak English and be able to complete self-administered surveys. Patients were excluded if they had an isolated relapsed solid tumor treated with surgery or radiation alone, or a first relapse of a hematologic malignancy awaiting stem-cell transplantation with an identified donor. The study was approved by institutional review boards of participating sites.

Recruitment and Randomization

Candidate patients were identified through clinic rosters and communication with clinicians at clinics and wards. Oncologists (physicians or nurse practitioners) confirmed eligibility, introduced the study to families, and provided parent and child study brochures. Interested families were then approached for consent by the study team. Consecutive patients were enrolled until June 2009 (Fig 1). The target sample size (n = 120) and the actual sample size after 5 years of accrual (n = 104) were driven by practical considerations. Power calculations were not feasible because distribution estimates of scores were not available for this population. Further details are available in the Data Supplement. Block randomization was performed by site with patients as the unit of randomization (random assignment of providers was considered but rejected because patients could be seen by multiple providers). A random sequence of assignments was computer generated and integrated into the PediQUEST system, but it remained concealed to researchers, patients, and providers; allocation was done automatically once the patient completed the first PediQUEST survey. Blinding of intervention and outcome assessment was not feasible.

Fig 1.

The Evaluation of Pediatric Quality of Life and Evaluation of Symptoms Technology (PediQUEST) study flow diagram at 20 weeks of follow-up. (*) One site did a preassessment of eligibility; all patients from that site were eligible.

Study Procedures

All participants completed PediQUEST surveys in the clinic or ward at most once a week. Participants not attending the clinic completed PediQUEST surveys by phone once a month (only 11 [1.6%] were phone surveys). We initially planned for a 9-month follow-up period. However, an interim analysis of the first 29 patients showed that 75% were alive at 9 months, suggesting we were enrolling a healthier-than-expected cohort, and that the 9-month follow-up period may have discouraged sicker patients from enrolling. We therefore limited follow-up to 3 months with the option of multiple re-enrollments up to end of data collection (December 2009) or death. As a token of our appreciation, small nonmonetary incentives were provided to children, parents, and providers.

Study Instruments

Surveys were embedded in the PediQUEST system and administered through tablet computers. First, the PediQUEST survey (PQ survey) assessed symptoms, HRQoL, and overall sickness. The PQ survey included three tools: (1) the PQ Memorial Symptom Assessment Scale (PQ-MSAS), an adapted version of the validated MSAS^23–25 that assessed 24 physical and psychological symptoms; (2) the Pediatric Quality of Life Inventory 4.0 Generic Core Scales (PedsQL4.0),²⁶ a 22-item validated HRQoL measure^27–31; and (3) an overall sickness question (Sickness) developed de novo. The sickness question asked how the child had been feeling during the past week (anchors were “Not sick at all” and “Very sick”). Survey length, response options, respondent, and mode of administration varied by age (Table 1). See the Data Supplement for more on PQ instruments and interface. Scores were calculated according to authors' recommendations.^23,32 To increase comparability across age groups and with HRQoL scores, the original MSAS uses 0 to 3 and 0 to 4 scales,²³ but the PQ-MSAS scores were standardized to 0 to 100 scales (100 is the worst). PedsQL4.0 scores ranged from 0 to 100 (100 is the best), with a score change of four points considered a clinically important difference.³³ Sickness scores ranged from 0 to 100 (100 is the worst). PQ surveys were piloted in 23 families with good acceptability and comprehensibility (see Data Supplement). Second, child and parent surveys assessing satisfaction with the PQ intervention were adapted from existing questionnaires.^34–36 These satisfaction surveys were embedded in the PediQUEST system and were administered at the fourth and eighth administrations. Provider satisfaction with the PQ intervention and perceived usefulness of the PQ system were assessed through an online survey at the end of data collection.

Table 1.

PediQUEST Survey Versions (according to age group of respondents) and Respondents (parent or self-report)

Survey Characteristic	PQ-MSAS Versions				PedsQL4.0 Versions							Sickness
	Parent		Self-Report		Parent				Self-Report			Parent	Self-Report
	Full Proxy	Supplemental Proxy 7-12*	7-12	13-18	2-4	5-7	8-12	13+	5-7	8-12	13+	VAS	Faces	VAS
No. of items	24	17	8	24	22	22	22	22	22	22	22	1	1	1
Response options	L-5	L-4	L-4	L-5	L-5	L-5	L-5	L-5	FS-3	L-5	L-5	VAS	FS-3	VAS
Age group/respondent
2-4 years
Parent	X				X							X
5-6 years
Child + parent	X								X			X		X
Parent	X					X						X
7 years
Child + parent		X	X						X			X	X
Parent	X					X						X
8-12 years
Child + parent		X	X							X		X	X
Parent	X						X					X
≥ 13 years
Teen				X							X		X
Parent	X							X				X

NOTE. The table shows versions of the PediQUEST surveys administered (columns), according to survey length, number of response options, and age of the child and respondent (rows). Parents of children age 2 to 4 years answered the Full Proxy version. If child was age 5 to 12 years and chose to self-report, parents answered PQ-MSAS (Full Proxy and/or Supplemental Proxy) and the Sickness question. A research assistant read questionnaires out loud for children age 5 to 7 years. All other surveys for children age ≥ 8 years and all parents were self-administered. Proxy versions were in place to enable parents to respond when a child/teen did not want to or was unable to answer questions. Equivalence across all PediQUEST survey versions was assumed.

Abbreviations: FS-3, 3-point Faces scale; L-4, 4-option Likert type; L-5, 5-option Likert type; PedsQL4.0, Pediatric Quality of Life Inventory 4.0 Generic Core Scales; PQ-MSAS, PediQUEST-Memorial Symptom Assessment Scale; VAS, 100-mm Visual Analog Scale.

^*Parent Supplemental Proxy, PediQUEST survey used in children age 7 to 12 years. These children answered a shorter version of the PediQUEST survey, and parents were asked to complete the supplemental version, which included PQ-MSAS items that were not available in the child self-report version to allow for consistent measurement across the entire age range.

PediQUEST Intervention

The feedback intervention had two components: PQ reports and PQ e-mails. PQ reports are printed reports that were given to providers before the visit and to families immediately after survey completion. PQ reports (Data Supplement) consisted of bar plots of PedsQL4.0 and PQ-MSAS symptom scores from current and four prior administrations, a summary highlighting changes since the last report, and a list of available resources for symptom control (for families) or generic pain management recommendations (for providers). Training on how to interpret PQ reports was offered to families at enrollment and annually to providers. PQ e-mails were automatically generated if PQ-MSAS individual symptom scores were ≥ 70, or if on two consecutive administrations scores were ≥ 70 for any frequency, severity, or distress item, or for a PedsQL4.0 total score ≤ 40. E-mails were sent to providers (oncologist, nurse, and psychosocial clinician), the palliative care service and, when pain was reported, the pain service. No instructions were imparted on how to respond to e-mails.

Outcomes

The intervention effect was measured by using linear trends of child distress (PQ-MSAS total and PQ-MSAS physical and psychological subscale scores), HRQoL (PedsQL4.0 total and PedsQL4.0 physical and emotional subscales), and Sickness scores. Satisfaction with PediQUEST feedback was assessed in children, parents, and providers. See the Data Supplement for more on outcomes.

Statistical Analysis

To analyze the intervention effect, we included all patients who answered at least one survey after random assignment (n = 98; Fig 1); all patients were retained in the group to which they were allocated, and all surveys completed within the first 20 weeks of follow-up (704 surveys) were included in the analysis. We excluded surveys (n = 417) beyond 20 weeks to prevent undue influence of high compliers; only 50% of patients remained in the study after 20 weeks, and thereafter the number of surveys completed per patient was highly variable (from 1 to 43). Trends in PQ-MSAS, PedsQL4.0, and Sickness scores were modeled by using linear mixed-effects models with study group, time (weeks from study entry), and their interaction as fixed effects and patient as a random effect. Quadratic and higher order terms for time were also considered. The simplest model, with time as linear and no interaction terms (ie, parallel lines), fit the data as well as the more complex models for all outcomes. Estimated treatment effects and their 95% CIs are reported for these parsimonious models (Data Supplement). Because of the skewed distribution of PQ-MSAS scores, we ran a set of models by using log-transformed scores. Conclusions were essentially unaltered; therefore, we reported results on raw scores.

To assess potential bias in the estimated effects, two exploratory post hoc subgroup analyses were conducted among patients who survived beyond 20 weeks and among children age ≥ 8 years. The first aimed to control for the differential number of deaths in the two arms during the follow-up period. The second aimed to evaluate the influence of younger age group scores on the results. By design, this group had a higher proportion of parental reports (53% v 2% in children age ≥ 8 years) and used age-adapted administration techniques and response options (Table 1). Although for practical reasons we assumed score equivalence across age groups and respondents, we recognize that this heterogeneity may affect results by diluting the estimated effect and/or increasing variability. All analyses were performed with SAS software, version 9.2 (SAS Institute, Cary, NC). See the Data Supplement for details on the outcome measurement plan and missing data approach.

RESULTS

Of 339 patients assessed for eligibility, 235 were excluded: 22 providers did not complete the eligibility checklist and 21 declined permission to approach, 86 patients did not meet eligibility requirements, 15 were lost in the enrollment process, 48 died before being approached, and 43 declined participation. Of the 104 enrollees, 51 were randomly assigned to the intervention and 53 to the control arm; 49 patients in each arm completed at least one follow-up PQ survey (Fig 1) and were included in main analyses. Sixty-nine oncologists were enrolled: 88% physicians and 12% nurse practitioners. Each oncologist had on average 1.6 patients in the study (range, one to six).

No differences were observed between groups in demographic or clinical characteristics, baseline scores, length of follow-up, number of PQ surveys answered, or respondent (Table 2). Between random assignment and week 20, there were nine of 51 (intervention) and five of 52 (control) deaths (P = .22); four of the deaths in the control group were excluded from the analysis because they occurred before the second survey.

Table 2.

Characteristics of Children Enrolled Onto the PediQUEST Study

Characteristic	Control (n = 53)		Intervention (n = 51)		Pa
	No.	%	No.	%
Baseline demographic and clinical characteristicsb
Site					.99
1	12	23	12	23
2	30	56	29	57
3	11	21	10	20
Female sex	25	47	26	51	.70
Race/ethnicity					1.0c
Non-Hispanic white	47	89	46	90
Other	6	11	5	10
Child's age, years					.90
2-7	16	30	16	32
≥ 8	37	69	35	68
Diagnosis					.96
Hematologic malignancy	19	36	17	33
Brain tumor	5	9	5	10
Solid tumor	29	55	29	57

	(n = 49)		(n = 49)		Pe
Baseline scores and 20-week follow-up informationd
PQ-MSAS total score					.63
Mean	12.7f		13.5f
SD	8.6		8.8
PedsQL4.0 total score					.73
Mean	70.4g		71.5
SD	16.9		16.1
PedsQL4.0 physical subscale					.73
Mean	64.1g		65.9
SD	26.7		24.6
PedsQL4.0 emotional subscale					.18
Mean	73.8g		78.9
SD	17.3		19.2
20-week follow-up information
Weeks in study					.15h
Median	20		19
IQR	13-20		12-20
PQ surveys administered					.73h
Median	7		7
IQR	4-10		4-9
Total No.	348		351		—
Proportion of child self-reportsi
< 8 years old	35/50	70	40/60	67	.71a
≥ 8 years old	258/262	98	244/249	98	.75c

Abbreviations: IQR, interquartile range; PediQUEST, Pediatric Quality of Life and Evaluation of Symptoms Technology; PedsQL4.0, Pediatric Quality of Life Inventory 4.0 Generic Core Scales; PQ, PediQUEST; PQ-MSAS, PediQUEST-Memorial Symptom Assessment Scale; SD, standard deviation.

^aχ² test.

^bAll randomized patients (n = 104).

^cFisher's exact test.

^dPatients with at least one follow-up PediQUEST survey (n = 98).

^et test.

^fn = 48.

^gn = 47.

^hWilcoxon rank sum test.

ⁱTotal number of surveys answered by child over the total number of potential surveys that could be answered by the child (rather than a parent).

Figure 2 presents the estimated average score differences between intervention and control groups (see Data Supplement for effect estimates and 95% CIs). Providing feedback to families and providers did not significantly affect the average PQ-MSAS, PedsQL4.0, or Sickness scores during 20 weeks of follow-up. However, changes were in the expected directions (negative for PQ-MSAS and Sickness and positive for PedsQL4.0). In exploratory subgroup analyses, larger effects were observed for all outcomes, with significant changes in PedsQL4.0 emotional and Sickness scores. In children who survived beyond 20 weeks, feedback significantly improved PedsQL4.0 emotional scores by an average of +6 points (95% CI, 0.3 to 11.7) and Sickness scores by an average of −5.3 points (95% CI, −10.6 to 0.0). Among children age ≥ 8 years, similar results were observed that were even stronger among those who survived beyond 20 weeks (+8.1; 95% CI, 1.8 to 14.4) for PedsQL4.0 emotional scores and −8.2 (95% CI, −14.2 to −2.2) for Sickness scores.

Fig 2.

Symptom, health-related quality of life, and sickness average score changes in children receiving the Pediatric Quality of Life and Evaluation of Symptoms Technology (PediQUEST) intervention during 20 weeks of follow-up. The figure shows estimated average score differences between intervention and control groups for all children (blue bars), and exploratory subgroup analysis (gold, gray, and red bars). See Data Supplement for effect estimates and 95% CIs. PQ-MSAS, PediQUEST-Memorial Symptom Assessment Scale; PedsQL4.0, Pediatric Quality of Life Inventory 4.0. (*) .05 < P ≤ .10; (†) .01 < P ≤ .05; (‡) P ≤ .01.

Figure 3 shows satisfaction with PediQUEST reports from 41 of the 44 families (29 children and 24 parents) remaining in the intervention group at the fourth administration. Fifty-two percent of children and 71% of parents found reports easy to understand. Twenty-eight percent of the children thought reports helped them quite a bit/very much to talk to their doctors as did 54% of parents. Further, 75% of parents agreed that reports often or almost always helped them understand their child's feelings. Confidentiality was not an issue, and almost all would have liked to continue using the system. Results were stable at the eighth administration.

Fig 3.

Child, parent, and provider satisfaction with Pediatric Quality of Life and Evaluation of Symptoms Technology (PediQUEST [PQ]) feedback reports. Figure shows selected responses from 41 families (29 children age ≥ 8 years old, 24 parents) who answered the first satisfaction with PQ intervention survey and from 18 primary oncologists who answered the provider satisfaction online survey and received PQ reports. QoL, quality of life.

Twenty-one (64%) of 34 eligible providers answered the satisfaction with PQ feedback survey. Three providers reported never receiving reports and were excluded. Most providers agreed that reports were easy to understand, and they contained relevant symptom and HRQoL information (Fig 3). Half the providers found reports useful when speaking with patients and also found that their use did not increase total consult time. Providers agreed that reports supplied new information about psychosocial issues (61%) but supplied less information about physical symptoms (22%), and 78% felt that this information was in overall agreement with what they obtained during the clinical encounter. However, 56% said that if there was a discrepancy, they would rely more on their own interview. Reports seemed to have some influence on oncologist behaviors. According to providers, reports contributed at least sometimes to their decision to initiate a psychosocial (56%), pain (34%), social work (33%), or palliative care (29%) consult and to discuss goals with families (36%). Few agreed that reports changed their views about patient's treatment goals. Regarding overall usefulness, 72% agreed that PediQUEST reports were at least somewhat helpful, more so than e-mail alerts.

DISCUSSION

To the best of our knowledge, this is the first RCT designed to test a PRO feedback intervention among children and teens with advanced cancer. In this pilot study, routine collection and feedback of PROs using PediQUEST did not significantly affect the child's symptoms or HRQoL. However, most scores changed in the expected direction, and results from exploratory subgroup analyses suggest that PediQUEST may have a clinically significant effect on emotional HRQoL and Sickness scores. The observed outcome effects are supported by views that PQ reports facilitated child and parent conversations with providers and enhanced parental understanding about the child's well-being. Generalizability of study findings is limited to larger pediatric oncology centers; indeed, that is the location where a majority of children with cancer receive care.

The study has several strengths such as its multicenter, randomized design accompanied by process measurements. Findings are consistent with those reported in the literature for feedback interventions.^{12–16,18,20,21,37–42} A majority of the studies similarly showed small effects on patient outcomes with greater improvement in psychological outcomes. A reason for this differential effect could be that emotional distress may be less noticeable to oncologists and feedback enhances its visibility. Compared with feedback from adult patients, feedback from children may be of greater value since the tendency of children to disclose psychological distress to both parents and providers is developmentally less typical.^43,44

To aid with interpretation, it is worth noting several study limitations. First, design limitations may have decreased the effect size. Control arm procedures involved routine PRO measurement, which may have improved scores¹⁴ reducing the “feedback effect.” As explained in the Data Supplement, a true control group was not feasible because of ethical considerations, population size, and lack of independent outcome measures. In addition, a contamination effect cannot be ruled out. Future studies should consider a cluster RCT design.

Second, subgroup analyses suggest that effect estimates may have been biased toward the null. The larger proportion of patients in the intervention group who died during the 20-week follow-up period, which can hardly be interpreted as a result of receiving PQ reports, may have reduced the estimated effect and may explain the stronger effects observed in the survivor subgroup. Interestingly, some evidence suggests that feedback interventions work better among sicker patients⁴⁵; however, the low death rate observed precluded such analysis in this study. The inclusion of younger children may also have similarly biased results. In this age group, experience suggests that during the clinical encounter, parents rather than the child typically report on the child's wellbeing, and therefore feedback may have had little added value. Conversely, the effects observed in the subset of older children may reflect a true increase in parental and oncologist awareness about the child's condition as a result of receiving the reports. This hypothesis is supported by parent views about the value of feedback and also by the literature in which patient empowerment appears to be one of the mechanisms involved in feedback interventions.⁴²

Third, the intervention intensity may have been low. We limited the intervention to the provision of PQ reports in the clinic or ward and offered little guidance to oncologists regarding how to respond. This approach, which was most feasible, may have hampered the intervention in the following ways: (1) A symptom review is standard of care during the clinical encounter; thus, reports may have added attention only to less commonly examined symptoms such as psychological symptoms. Indeed, Cleeland et al⁴⁶ demonstrated a stronger effect of feedback when measuring pain from home, a setting in which symptoms are not typically monitored. (2) Providers' attitudes, beliefs, and behaviors toward symptom management may have hindered the intervention effect. For example, our results and those of others⁴⁷ reveal that clinicians seem to trust their own interview more than patient self-report, not taking into account that patients tend to report higher distress through anonymous methods than when talking with providers.⁴⁸ Providers may also believe that symptoms are an expected outcome of cancer and its treatment.⁴⁹ These, along with the recognized lack of education and training in palliative approaches among pediatric oncologists,^50,51 may have resulted in suboptimal symptom management. Further, our feedback intervention, not unlike others,⁵² may have lacked the leverage needed to affect clinician behaviors as evidenced by the low rate of referrals to other specialists, including palliative care, triggered by PQ reports.

Finally, the chosen outcomes may have limited our ability to detect a larger effect. We used a more pragmatic approach with multiple measurements per patient, as opposed to other feedback RCTs that used fixed measurement times. We hypothesized that the effect of feedback would be cumulative and that trends in symptom distress and HRQoL would be improved by the intervention. However, it is likely that trends over time are affected by myriad factors; instead, the effects of feedback may be better assessed by defining the episode of distress as the unit of analysis and evaluating outcomes within days of the event.

In conclusion, although routine feedback of PROs did not significantly affect the child's symptoms or HRQoL, improvements observed in emotional HRQoL through exploratory analyses are encouraging, especially in light of design limitations. Responses from families and providers suggest that PediQUEST is a valued communication enhancer and may affect specific care processes. Encouraged by these results, we updated the PQ system to a Web platform to permit more versatile administration and e-mail reporting, and we are conducting formative research to strengthen the intervention with additional components such as family empowerment and enhanced provider response to symptom control.

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The author(s) indicated no potential conflicts of interest.

AUTHOR CONTRIBUTIONS

Conception and design: Joanne Wolfe, Liliana Orellana, Jane C. Weeks, Veronica Dussel

Provision of study materials or patients: Joanne Wolfe, Christina Ullrich, Tammy Kang, Jeffrey Russell Geyer

Collection and assembly of data: Joanne Wolfe, Christina Ullrich, Tammy Kang, Jeffrey Russell Geyer, Chris Feudtner, Veronica Dussel

Data analysis and interpretation: Joanne Wolfe, Liliana Orellana, E. Francis Cook, Veronica Dussel

Manuscript writing: All authors

Final approval of manuscript: All authors