Importance ratings on patient-reported outcome items for survivorship care: comparison between pediatric cancer survivors, parents, and clinicians

Abstract

Purpose

To compare importance ratings of PROs items from the viewpoints of childhood cancer survivors, parents, and clinicians for further developing short-forms to use in survivorship care.

Methods

101 cancer survivors, 101 their parents, and 36 clinicians were recruited from St. Jude Children’s Research Hospital. Participants were asked to select eight items that they deemed useful for clinical decision-making from each of the four PROMIS Pediatric item banks. These item banks were pain interference (20 items), fatigue (23 items), psychological stress (19 items), and positive affect (37 items).

Results

Compared to survivors, clinicians rated more items across four domains that were statistically different than did parents (23 vs. 13 items). Clinicians rated five items in pain interference domain (ORs=2.33–6.01; p’s<0.05) and three items in fatigue domain (ORs=2.22–3.80; p’s<0.05) as more important but rated three items in psychological stress domain (ORs=0.14–0.42; p’s<0.05) and six items in positive affect domain (ORs=0.17–0.35; p’s<0.05) as less important than did survivors. In contrast, parents rated seven items in positive affect domain (ORs=0.25–0.47; p’s<0.05) as less important than did survivors.

Conclusions

Survivors, parents, and clinicians viewed importance of PRO items for survivorship care differently. These perspectives should be used to assist the development of PROs tools.

Introduction

The 5-year survival rate in pediatric cancers continues to improve [1], yet survivors frequently develop treatment-related late effects which significantly impact quality of life [2–4]. In 2004, the U.S. National Institutes of Health launched the Patient-Reported Outcomes Measurement Information System (PROMIS) project to develop tools for assessing patient-reported outcomes (PROs) [3, 4]. PROMIS measures have been validated in pediatric cancer patients but not yet in survivors [5–7].

PROMIS measures use item response theory to calibrate items on a common metric (i.e., item banks), which can be administered as short-forms or computerized adaptive tests. This also permits the creation of disease-relevant short-forms containing items that are face valid to respondents and clinicians. When collecting PRO data, patients should ideally complete the survey. However, when developing PRO tools, it is important to incorporate the perspectives of different stakeholders (patients, caregivers, clinicians, etc.) through the entire process [8, 9].

The purpose of this study was to contrast the ratings of items with respect to the perceived importance among pediatric cancer survivors, parents, and clinicians. When creating short-forms by utilizing items from the item banks, it is critical to include items relevant to survivorship care from the perspectives of survivors, parents, and clinicians. Although previous studies have evaluated discrepancies at the level of PRO domains between parents and children [10–12] and among children, parents, and clinicians [13, 14], none have evaluated importance ratings across PRO items relevant to survivorship care among children, parents, and clinicians. The specific item banks we examined assess experienced well-being (positive affect, defined as the momentary feelings of happiness and joy) [15], distress (psychological stress experiences) [16], and suffering (pain interference and fatigue) [17, 18]. These domains are vital for cancer survivorship because they capture the most prevalent treatment-relevant late effects [19, 20]. Screening pain interference, fatigue, and stress facilitates clinicians to administer interventions and empowers patients to achieve positive well-being.

Methods

Participants and Data Collection

Between August and December 2016, we recruited 101 survivors and 101 their parents from the After the Completion of Therapy Clinic at St. Jude Children’s Research Hospital during their follow-up care, and 36 St. Jude clinicians. Enrollment criteria for survivors included: ages 8 to 17.9 years; ≥2 years off anticancer therapy; ≥5 years since diagnosis; had not received a bone marrow transplant; had at least a third grade reading level in English; and had an intelligence quotient ≥70. Parents were eligible if they were the parents/legal guardians of eligible survivors and were able to read English. Participating clinicians worked at St. Jude and had ≥2 years of survivorship care expertise.

Using OptimalSort©, a well-established online program [21], participants were asked to identify eight most important items from each of the PROMIS Pediatric item banks (Online Supplement) that they regarded as useful for decision-making and communication in survivorship care. We instructed parents and children to act as independent raters. As an example, we asked parents: “Which of the following questions are most important to ask parents of a child who is a cancer survivor about how pain interferences with the child life? We asked children: “Which of the following questions are most important to ask children who are cancer survivors about how pain interferences with their lives?” Participants could compare the items being selected, unselect those items, and replace those with others before advancing to the next domain.

Measures

Four PROMIS Pediatric item banks were evaluated: pain interference, fatigue, psychological stress, and positive affect. The pain interference bank comprises 20 items assessing the consequences of pain on relevant aspects of a child’s life [17]. The fatigue bank comprises 23 items assessing a child’s experience of fatigue, ranging from a subjective feeling of tiredness to an overwhelming, debilitating, sustained sense of exhaustion, and impact of these feelings on daily functioning [18]. The psychological stress bank comprises 19 items assessing a child’s thoughts or feelings about self and the world in the context of environmental or internal challenges [16]. The positive affect bank comprises 37 items assessing a child's momentary positive or rewarding affective experiences [15].

Statistical Analysis

For each item in a specific domain, percentages of survivors, parents, and clinicians who rated it as importance were calculated. Odds ratios (ORs) of an item’s importance rating among respective clinicians and parents vs. survivors (as the reference) were calculated. Level of statistical significance was based on 2-sided tests with p-value <0.05.

Results

Table 1 describes participant demographics. Among survivors, the mean age was 13.9 years (SD=2.9), and 51% were female. Nearly half were treated for a non-central nervous system solid tumor. The majority of parents were female (85%). Among clinicians, 36% were oncologists, 39% were physician assistants, and 25% were psychologists or nurses.

Table 1.

Child, Parent, and Clinician Demographics

	Mean (SD)
Child’s Age at Evaluation	13.9 (2.9)
	N (%)
Child’s Sex
Male	50 (49%)
Female	51 (51%)
Child’s Race/Ethnicity
White, non-Hispanic	70 (69%)
Black, non-Hispanic	24 (24%)
Other	7 (7%)
Child’s Diagnosis
Non-central nervous system(non-CNS) solid tumor	46 (46%)
Leukemia	30 (30%)
CNS malignancy	16 (16%)
Lymphoma	9 (9%)
Parent’s Sex
Male	15 (15%)
Female	86 (85%)
Parent’s Race/Ethnicity
White, non-Hispanic	74 (72%)
Black, non-Hispanic	23 (22%)
Other	6 (6%)
Health Professional’s Position
Oncologist	13 (36%)
Physician assistant	14 (39%)
Psychologist	8 (22%)
Nurse	1 (3%)

Figure 1 displays the differences in item importance ratings between clinicians and survivors and between parents and survivors for pain interference and fatigue domains. Figure 2 displays the differences for psychological stress and positive affect domains. Within a domain, in general the disparity in importance item ratings was greater between clinicians and survivors (red color) than between parents and survivors (light blue color).

Fig. 1.

Differences (%) in Importance Ratings between Clinicians vs. Children and Parents vs. Children in Pain Interference and Fatigue Item Banks

Fig. 2.

Differences (%) in Importance Ratings between Clinicians vs. Children and Parents vs. Children in Psychological Stress and Positive Affect Item Banks

Table 2 shows the items within each domain whose importance rating was statistically different when comparing responses among clinicians or parents vs. children (the reference). Compared to survivors, clinicians rated a greater number of items differently than did parents (23 vs. 13 items). ORs in 14 out of 23 items were <1.0, meaning clinicians viewed these items as less important than did children. In contrast, ORs in 11 out of 13 items viewed differently by parents compared to children were <1.0.

Table 2.

Odds Ratio of Items rated as Importance by Clinicians vs. Children and by Parents vs. Children^†,^‡

	Clinicians vs. Children			Parents vs. Children
Pain Interference Item Bank	OR	p-value	95% CI	OR	p-value	95% CI
Waked carefully	6.01	<0.001	2.65, 13.65	-	-	-
Missed school	3.70	0.002	1.54, 8.87	-	-	-
Had trouble sleeping	2.93	0.018	1.17, 7.30	-	-	-
Needed to take medicine to treat it	2.50	0.031	1.07, 5.83	-	-	-
Was hard to pay attention	2.33	0.029	1.08, 5.03	-	-	-
Hurt all over body	0.36	0.023	0.14, 0.89	-	-	-
Was hard to stay standing	0.31	0.020	0.11, 0.86	-	-	-
Had to stop what was doing	0.13	<0.001	0.05, 0.35	-	-	-
Was hard to walk one block	-	-	-	0.37	0.018	0.16, 0.86
Felt angry	-	-	-	-	-	-
Had trouble doing schoolwork	-	-	-	-	-	-
Was hard to run	-	-	-	-	-	-
Was hard to have fun	-	-	-	-	-	-
Hurt a lot	-	-	-	-	-	-
Was hard to remember things	-	-	-	-	-	-
Was hard to get along with other people	-	-	-	-	-	-
Was hard to be away from home	-	-	-	-	-	-
Was hard to have fun with friends	-	-	-	-	-	-
Needed help walking	-	-	-	-	-	-
Was hard to do things with family	-	-	-	-	-	-
% of items with significantly different ratings	40%			5%
	Clinicians vs. Children			Parents vs. Children
Fatigue Item Bank	OR	p-value	95% CI	OR	p-value	95% CI
Needed to sleep during the day	3.80	0.001	1.70, 8.52	-	-	-
Was too tired to spend time with friends	2.85	0.011	1.26, 6.47	-	-	-
Was too tired to enjoy things like to do	2.22	0.037	1.04, 4.74	-	-	-
Was hard to pay attention	0.27	0.0014	0.11, 0.62	-	-	-
Felt weak	0.20	<0.001	0.08, 0.50	-	-	-
Kept from having fun	-	-	-	0.42	0.008	0.22, 0.80
Felt tired	-	-	-	0.37	0.001	0.21, 0.67
Got tired easily	-	-	-	-	-	-
Was hard to get out of bed	-	-	-	-	-	-
Felt tired when woke up	-	-	-	-	-	-
Was too tired to watch television	-	-	-	-	-	-
Was too tired to eat	-	-	-	-	-	-
Was too tired to take a bath or shower	-	-	-	-	-	-
Was too tired to do things outside	-	-	-	-	-	-
Was hard to focus on work	-	-	-	-	-	-
Was too hard to play or go out with friends	-	-	-	-	-	-
Had trouble starting things	-	-	-	-	-	-
Had trouble finishing things	-	-	-	-	-	-
Was too tired to go up and down stairs	-	-	-	-	-	-
Was too tired to go out with family	-	-	-	-	-	-
Was too tired to read	-	-	-	-	-	-
Was too hard to keep up with schoolwork	-	-	-	-	-	-
Was too tired to do sports or exercise	-	-	-	-	-	-
% of items with significantly different ratings	21.7%			8.7%
	Clinicians vs. Children			Parents vs. Children
Psychological Stress Item Bank	OR	p-value	95% CI	OR	p-value	95% CI
Had trouble controlling thoughts	0.42	0.049	0.17, 1.01	-	-	-
Was slow to react to things	0.22	0.038	0.05, 0.98	-	-	-
Felt unable to react to something bothered	0.14	0.003	0.03, 0.60	-	-	-
Felt concerned about what was going on	-	-	-	1.93	0.022	1.09, 3.41
Thoughts went very fast	-	-	-	0.33	0.009	0.14, 0.78
Felt stressed	-	-	-	-	-	-
Small things got upset	-	-	-	-	-	-
Everything bothered	-	-	-	-	-	-
Felt under pressure	-	-	-	-	-	-
Felt problems kept piling up	-	-	-	-	-	-
Felt overwhelmed	-	-	-	-	-	-
Forgot things	-	-	-	-	-	-
Felt like thinking was slower	-	-	-	-	-	-
Felt unable to remember answers	-	-	-	-	-	-
Could not remember what happened	-	-	-	-	-	-
Had trouble concentrating	-	-	-	-	-	-
Had trouble making decisions	-	-	-	-	-	-
Felt unable to manage things	-	-	-	-	-	-
Felt had too much going on	-	-	-	-	-	-
% of items with significantly different ratings	15.8%			10.5%
	Clinicians vs. Children			Parents vs. Children
Positive Affect Item Bank	OR	p-value	95% CI	OR	p-value	95% CI
Felt happy	2.93	0.007	1.33, 6.47	-	-	-
Felt comfortable	0.35	0.039	0.12, 0.98	-	-	-
Felt calm	0.28	0.025	0.09, 0.87	-	-	-
Felt active	0.21	0.007	0.06, 0.72	-	-	-
Felt appreciated	0.19	0.004	0.05, 0.66	-	-	-
Felt wide-awake	0.19	0.024	0.04, 0.84	0.47	0.045	0.23, 0.99
Felt refreshed	0.17	0.008	0.04, 0.76	0.43	0.021	0.20, 0.89
Felt content	-	-	-	2.25	0.048	0.99, 5.08
Felt great	-	-	-	0.43	0.021	0.20, 0.89
Felt satisfied	-	-	-	0.41	0.011	0.21, 0.83
Felt joyful	-	-	-	0.33	0.004	0.16, 0.72
Felt strong	-	-	-	0.30	0.001	0.15, 0.62
Felt delighted	-	-	-	0.25	0.027	0.07, 0.93
Felt peaceful	-	-	-	-	-	-
Felt grateful	-	-	-	-	-	-
Felt thankful	-	-	-	-	-	-
Felt positive	-	-	-	-	-	-
Felt carefree	-	-	-	-	-	-
Felt relaxed	-	-	-	-	-	-
Felt fulfilled	-	-	-	-	-	-
Felt respected	-	-	-	-	-	-
Felt proud	-	-	-	-	-	-
Had much to be proud about	-	-	-	-	-	-
Felt pleased	-	-	-	-	-	-
Felt cheerful	-	-	-	-	-	-
Had strong happy feelings	-	-	-	-	-	-
Smiled a lot	-	-	-	-	-	-
Laughed a lot	-	-	-	-	-	-
Was merry	-	-	-	-	-	-
Was in a good mood	-	-	-	-	-	-
Was in good spirits	-	-	-	-	-	-
Felt good	-	-	-	-	-	-
Felt blissful	-	-	-	-	-	-
Felt enthusiastic	-	-	-	-	-	-
Had a lot of energy	-	-	-	-	-	-
Felt full of pep	-	-	-	-	-	-
Felt healthy	-	-	-	-	-	-
% of items with significantly different ratings	18.9%			21.6%

^†Cells with null values mean ORs were not statistically different either between clinicians and children or between parents and children.

^‡For full content of each item, refer to Online Supplement.

For the pain interference, clinicians’ responses differed from children in eight items (40%). ORs of item endorsement ranged from 0.13 (95%CI=0.05–0.35) for “have to stop what was doing” to 6.01 (95%CI=2.65–13.65) for “walk carefully.” Parents rated only one item (5%) differently from children (“was hard to walk one block”); OR was 0.37 (95%CI=0.16–0.86). For the fatigue, clinicians differed from children in five items (21.7%); ORs ranged from 0.20 (95%CI=0.08–0.50) for “feel weak” to 3.80 (95%CI=1.70–8.52) for “need to sleep during the day.” In contrast, parents differed from children in two items (8.7%). For the psychological stress, clinicians and parents differed from children in three items (15.8%) and two items (10.5%), respectively. For the positive affect, clinicians differed from children in seven items (18.9%); ORs ranged from 0.17 (95%CI=0.04–0.76) for “feel refreshed” to 2.93 (95%CI=1.33–6.47) for “feel happy.” Parents differed from children in eight items (21.6%); ORs ranged from 0.25 (95%CI=0.07–0.93) for “feel delighted” to 2.25 (95%CI=0.99–5.08) for “feel content.”

Discussion

Our findings indicate that clinicians varied from children more than parents did in rating PRO items as importance for survivorship care. It is not surprising that clinicians emphasized the importance for the items in pain interference and fatigue domains more than those in psychological stress and positive affect domains as monitoring physical symptoms/wellbeing to identify cancer recurrence and late effects is a high priority in clinical practice. When inspecting item contents across domains, items that clinicians rated as more important tended to be objective (e.g., “walk carefully”) and clinical-oriented/treatable (e.g., “need to sleep during the day”). However, the items that clinicians or parents rated as less important than did children tended to be subjective/abstract and less clinical-oriented/treatable (e.g., “feel refreshed,” “feel delighted,” and “have to stop what was doing”).

When selecting items for developing PRO short-forms, the conventional practice includes using qualitative methods to elucidate themes/constructs of the domains, debriefing participants for content comprehensions, and evaluating psychometric properties (e.g., item difficulty, sensitivity to distinguish PRO levels, or measurement errors of the items) [16–18, 22–25]. However, the item selection process in previous studies did not contrast usefulness in survivorship care across children, parents, and clinicians. Our findings provide notable implications for PRO research. By using robust methodology of obtaining insights from children, parents, and clinicians regarding item contents that are relevant to survivorship care, our study complements the previous research in this area.

Despite the findings of diverse ratings, it is valuable to emphasize that when developing a short-form for use among children and proxies, we should consider the needs of all three stakeholder groups (survivors, parents, and clinicians), and the application of the tools rather than reconcile the differences. For developing survivorship-specific short-forms through PROMIS banks, our findings suggest that those items identified as equally important by all three groups (i.e., ORs for items not significant in Table 2) can be ideally selected for the short-forms. However, items rated more important by clinicians than did children may reflect issues that are of great clinical concern and could be weighted in the short-forms development, especially for condition-/disease-specific groups. In contrast, items viewed as less important by clinicians or parents than children, such as positive affect items, might reflect the issues that childhood cancer survivors would like to address or discuss during survivorship care.

This study did not evaluate importance ratings for social and cognitive functioning; brain cancer survivors often experience deficits in these domains. In addition, we did not conduct subgroup analyses for different ages (e.g., 8–12 vs. 13–18 years), cancer diagnoses, years since diagnosis, and types of clinicians (e.g., medical doctors vs. psychologists) given the current sample size. Importance ratings on specific items might differ across subgroups of clinicians due to specific clinical experience and responsibility. Future studies are warranted to address these limitations.

In summary, childhood cancer survivors, parents, and clinicians hold different opinions on the importance of PRO content for survivorship care, and each group should be assessed for the clinical relevance of PRO items. The heterogeneous views across these groups highlight the importance of considering unique perspectives when selecting PRO items for developing survivorship-specific short-forms.