Perceptions of Response Burden Associated with Completion of Patient-Reported Outcome Assessments in Oncology

ARTICLE SUMMARY:

We found that patients with cancer who completed a large presurgical battery of patient-reported outcomes questionnaires and interviews experienced minimal response burden.

Objective:

Patient response burden is often raised as a human subjects concern in consideration of the length or complexity of patient-reported outcome (PRO) instruments used in oncology. We sought to quantify patient response burden and identify its predictive factors.

Methods:

Data were collected pre-surgically during a prospective trial that employed a comprehensive symptom and health-related quality of life (HRQoL) PRO assessment. A subset of patients also completed HRQoL interviews. Response burden was captured using an internally developed 6-item instrument. Demographics, clinical characteristics, and HRQoL were examined as potential predictors using hierarchical regression. Response burden was used to predict participant drop-out at the first follow-up interval.

Results:

A total of 275 patients (mean age=67.5, 23.6% female) completed surveys (n=126) or surveys in addition to interviews (n=149). Patients experienced low response burden (M(SD)=12.19(11.65)). Repetitive questions were identified by 60 patients (21.8%), whereas 31.6% indicated additional information should be gathered; 35 patients (12.7%) identified repetitive questions and expressed a desire for additional items. Low self-reported cognitive function was a significant predictor of higher response burden (β; = −0.20; t(270) = −3.38; p = 0.01; Model Adjusted R² = 0.04). Response burden was not a significant predictor of study drop-out.

Conclusions:

Despite completing a large battery of PRO measures and interviews, patients reported minimal response burden, with nearly one third expressing that more questions should have been asked. Patients with lower cognitive function are more likely to report higher response burden when completing PRO measures. Further examination of patient characteristics related to response burden may reveal useful pathways for tailoring patient-centered interventions.

BACKGROUND

Patient-reported outcomes (PROs) are becoming widely accepted for use in clinical trials and routine cancer care as an indicator of the patient subjective experience, including their symptoms related to disease and/or treatment, quality of care, or health-related quality of life (HRQoL) (1-7). Despite this increased utilization of PRO measures, relatively little is known about the degree of response burden that is experienced by patients as they complete PRO questionnaires with respect to their time and expended effort, nor is there consensus for the most effective method of capturing response burden from patients.

Response burden can be conceptualized in several different ways, depending on the research setting of interest. For example, in the context of completing business surveys, researchers have found that response burden is not associated with time to completion of a survey or frequency of survey administration; burden is instead thought of as being related to the quality of data that is provided (8-10).

In the healthcare setting, especially in oncology, response burden is a particularly challenging concept to define. Sicker patients with advanced-stage disease might find that responding to PRO questionnaires is more burdensome than those who are diagnosed in earlier stages and are relatively healthier. The 2009 U.S. Food and Drug Administration (FDA) Guidance for Industry on the Use of PRO measures in Medical Development to Support Labeling Claims posits several potential factors that may be related to response burden (11). These include the length and/or formatting of the questionnaire or interview, issues with literacy level, issues related to the mode of administration (e.g., paper-, telephone-, or web-based surveys), issues related to sensitive content of items that participants may be unwilling to answer, or perceptions that an interviewer expects a specific response from a given patient.

A recent meta-analysis found that the length of a given PRO questionnaire may not necessarily be associated with participant response burden (12). As such, there may be other underlying factors that influence and predict patient response burden. For instance, there may be differences in response burden that depend on perceived difficulty of questionnaire completion due to the cognitive demand characteristics related to the mode of data capture (e.g., recognitionbased questionnaires versus recall-based interviews) (13). Additionally, various cancer disease types have a wide range of levels of severity and potential effects on a given patient’s health status. This necessitates the use of numerous PRO questionnaires and/or lengthy one-on-one interviews to capture all information relevant to the project’s domains of interest.

Because patients with cancer are considered a vulnerable population of interest, such requirements may make key stakeholders reticent with respect to including lengthy or complex PRO questionnaires in a given study (14-16). There is a strong bias toward “less is more” with respect to eliminating redundant or non-informative items (17-19), however the pressure to limit the number of measurement items can mean that aspects of HRQoL central to the patient experience may be ignored.

The purpose of the present study was to quantify levels of patient response burden associated with participation in a methodologically rigorous HRQoL study where a comprehensive battery of questionnaire-based PRO measures was collected, utilizing paper-and-pencil and interview modes of administration. Additionally, we sought to characterize potential predictors of burden associated with the PROs, as well as whether response burden was a predictor of study drop-out. Given the established willingness of patients to become more involved in their healthcare and treatment-related decisions (20-23), we anticipated that patients would be minimally burdened by the collection of this HRQoL information, with our results providing greater insight into whether demographic, disease experience, or appraisal factors are important predictors of perceived response burden.

METHODS

Patients

Data for this study were collected presurgically as part of a prospective trial that examined patient-reported symptoms and HRQoL of those undergoing radical cystectomy and urinary diversion for high-risk bladder cancer between 2008 and 2014 at Memorial Sloan Kettering Cancer Center (MSK). Patients were eligible to participate in the study if they had nonmetastatic bladder cancer and were scheduled for radical cystectomy and urinary diversion; were English speaking; at least 18 years of age; and able to provide informed consent. This study was approved by the MSK Internal Review Board (ClinicalTrials.gov NCT00745355).

Materials

PRO Questionnaires

All patients completed a brief questionnaire that captured demographic information (e.g., employment status, marital status). Additionally, patients were asked to complete a set of 14 validated and standardized PRO questionnaires in their entirety, with the exception of females being asked to complete the Female Sexual Function Index (24), and males being asked to complete the International Index of Erectile Function (25). These PRO measures assessed various symptom or HRQoL domains (Table 1). Females completed a total of 180 items, with males completing 176 total items.

Table 1.

Patient-Reported Outcomes Measures Administered by Domain Assessed

Patient-Reported Outcomes Measure	Domains Assessed
EORTC QLQ-C30 (35)	30-item measure of physical, role, emotion, and social functioning, as well as cognition, symptoms (e.g., fatigue, pain, dyspnea) and global quality of life in patients with cancer.
EORTC QLQ-BLM30 (36)	30-item assessment of urination, bowel functioning, and sexual functioning in patients with muscle-invasive bladder cancer.
EORTC QLQ-CR38 (37)	7-item bowel function subscale from a larger scale of quality of life concerns in patients with colorectal cancer.
Female Sexual Function Index (FSFI (24)) – Females Only	19-item measure of desire and subjective arousal, lubrication, orgasm, satisfaction, and pain/discomfort.
International Index of Erectile Function (IIEF (25)) – Males Only	15-item assessment of erectile function, orgasmic function, intercourse satisfaction, sexual desire/arousal and overall satisfaction.
Urinary Distress Inventory (UDI (38))	6-item assessment of the degree to which symptoms of incontinence cause distress.
Incontinence Impact Questionnaire (38)	7-item measure of the impact of urinary incontinence on activities and emotions.
AUASI (39)	7-item assessment of urinary functioning.
MSKCC Bowel Function Questionnaire (40)	18-item assessment of bowel functioning.
Decisional Conflict Scale (DCS (41))	16-item measure of the decision-making process in health care and levels of conflict attributed to treatment-related decisions.
Satisfaction with Life Scale (SWLS (42))	5-item assessment of global cognitive judgments of one's life.
Fear of Recurrence Questionnaire (FRQ (43))	22-item measure of cancer related fears and health concerns.
Mental Health Inventory (MHI (44))	5-item measure of anxiety, calmness, depression, happiness, and behavioral/emotional control.
FACIT-TS-G (45)	8-item measure of general treatment satisfaction.

Note: EORTC indicates European Organization for Research and Treatment of Cancer Quality of Life; QLQ-C30, Cancer quality of life questionnaire – 30 items; QLQ-BLM30, Cancer quality of life questionnaire – Muscle Invasive Bladder Module; QLQ-CR38, Cancer quality of life questionnaire – Colorectal Module; AUASI, American Urological Association Symptom Index; FACT-TS-G, Functional Assessment of Chronic Illness Therapy – Treatment Satisfaction – General.

Interview Measures

One-on-one idiographic interviews with patients included two key components: goal assessment and activity assessment (26-29). The basic structure of the research interaction asks patients to name the primary goals they would like to accomplish, what problems they would like to solve, what they would want to prevent or avoid, what they would want to keep the same as they are now, and what commitments that they would want to let go, what things that they want to be able to accept as they currently are, and what special events or milestones they are looking forward to reaching in order to have the most satisfying life possible. Each of these separate areas is followed by probes asking what the patient has been doing over the last month in order to reach these goals, as well as which activities matter the most in achieving these goals. These idiographic methods have been previously summarized (26, 28, 30).

Response Burden Questionnaire

Given the lack of a standard, validated measure to capture response burden, we developed a brief, 6-item PRO measure that captures key aspects of this multidimensional construct with respect to how the patient perceives: a) how well the questions related to their actual concerns, b) how comfortable the patients were with answering the questions, c) how well the interview characterized their health and well-being, d) the length of time to complete the questionnaires, e) whether questions seemed unimportant or repetitive, and f) what additional information should have been gathered (Table 2). Items 1-4 of the response burden questionnaire were reverse-scored. A composite score was then calculated to create a weighted representative index of concern, comfort, and well-being relative to time to completion (i.e., items 1, 2, and 3 were summed and multiplied by item 4) for a range of 0-72, with higher scores indicative of elevated endorsed response burden. For example, a patient may indicate that the PRO questions took too long to complete, but otherwise feel that the questions were relevant to their concerns (e.g., score range 0-10), whereas another patient may endorse that the amount of time to complete the PRO questionnaires was just right, but that they were very uncomfortable in responding to the questions, resulting in a similar score range. The open-ended questions of the response burden questionnaire were summarized thematically.

Table 2.

Patient Response Burden Items and Response Scale

Item	Response Scale
1. How well did these questions RELATE TO YOUR ACTUAL CONCERNS?	0 (Not at all related) - 10 (Very related)
2. How COMFORTABLE were you in answering questions?	0 (Not at all comfortable) - 10 (Very comfortable)
3. How well did this interview DESCRIBE YOUR HEALTH AND WELL BEING?	0 (Not at all) - 10 (Very well)
4. How did you feel about the LENGTH OF TIME to complete this section?	1 (Much too long), 2 (A bit too long), 3 (Just right/no problem)
5. What questions seemed UNIMPORTANT OR REPETITIVE?	Open-ended
6. What ADDITIONAL INFORMATION should we gather?	Open-ended

Note: To calculate a composite score, items 1-4 were first reverse-scored, followed by summing items 1-3 and multiplying them by item 4, for a range of 0-72. Items 5 and 6 are summarized thematically.

Procedure

Enrolled patients were sent the PRO battery via US Postal Service at baseline (pretreatment). Participants were asked to complete the measures and return them to the research team using postage-paid envelope. For the subset of patients who also completed the one-on-one idiographic interview, this took place via telephone.

Statistical Methods

We used multivariate hierarchical linear regression to determine predictors of response burden. Using a three-tiered backward-selection process. In order, the model included: patient demographic variables (i.e., age at surgery, gender, racial background, marital status, employment status), clinical characteristics (age adjusted Charlson Comorbidity score, cancer stage, body mass index, as well as binary variables for whether patients had prior neoadjuvant chemotherapy, intravesical treatment, or pelvic radiation treatment), and subscale scores from the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-C30 (31)) (i.e., Global Health Status, Physical Function, Role Function, Emotional Function, Cognitive Function, Social Function, Fatigue, Nausea-Vomiting, Pain, Dyspnea, Insomnia, Appetite Loss, Constipation, Diarrhea, and Financial Problems). EORTC QLQ-C30 subscale scores were selected for inclusion in the model due to its general assessment of key aspects of HRQoL. Model goodness-of-fit was assessed using adjusted R². Statistically significant (α = 0.05) predictors, if any, were included in a final combined predictive model. Differences in levels of response burden between those who completed both a PRO assessment and idiographic interview and patients who only completed the PRO questionnaires were explored using independent t-tests and non-parametric tests for dichotomous variables. General linear modeling was used to predict study-drop out at the first post-surgical assessment time point (i.e., three months). Drop-out was defined as having incomplete EORTC QLQ-C30 data at time 2. SPSS v.24 (32) was used for all quantitative analyses.

RESULTS

From the overall sample of 550, a total of 275 patients (50% response rate; 23.6% female, 4.4% non-white) had evaluable pre-surgical data and were included in the analysis. There were no instances of missing data in the completed surveys. The mean age at time of surgery was 67.48 (range 36.4 – 91.4 years), with mean age adjusted Charlson Comorbidity Score of 2.41 (SD 2.41, range 0 – 9). Those who completed ideographic interviews in addition to the PRO battery (n = 149) were significantly older, had higher mean age-adjusted Charlson Comorbidities, were less likely to be employed or to have underwent neoadjuvant chemotherapy, and had lower mean EORTC QLQ-C30 Nausea-Vomiting scores than those who only completed the PRO questionnaires (n = 126). A summary of demographic, clinical characteristics, and HRQoL scores for these patients is displayed in Table 3.

Table 3.

Demographic, Clinical Characteristics, and Health-Related Quality of Life Subscale Scores of Patient Sample

	Total (N=275)			Survey Only (n=126)			Survey + Interview (n=149)			Survey vs.Survey + Interview
Characteristics	n	Mean (SD)	%	n	Mean (SD)	%	n	Mean (SD)	%	t(df)	χ2	p
Age At Surgery (Years)		67.48 (9.29)			66.05 (8.98)			68.68 (9.40)		−2.36(272)		0.02
Age-Adjusted Charlson Comorbidity		2.41 (2.41)			1.91 (2.32)			2.82 (2.42)		−3.15(269)		0.01
Response Burden Composite Score		12.19 (11.65)			12.88 (12.15)			11.58 (11.07)		0.96(273)		0.33
EORTC QLQ-C30 Subscales
Global Health Status - QoL		73.43 (20.43)			72.47 (19.58)			74.26 (20.41)		−0.73(269)		0.46
Physical Function		90.93 (13.31)			90.11 (14.20)			91.62 (12.52)		−0.94(272)		0.35
Role Function		85.58 (23.18)			85.19 (22.73)			85.92 (23.63)		−0.26(272)		0.79
Emotional Function		74.73 (20.39)			74.60 (23.66)			74.85 (17.15)		−0.10(270)		0.92
Cognitive Function		86.70 (16.88)			86.51 (18.34)			86.87 (15.56)		−0.18(270)		0.86
Social Function		76.84 (25.90)			74.74 (26.89)			78.65 (24.97)		−1.25(270)		0.21
Fatigue		20.88 (18.83)			21.78 (19.94)			20.11 (17.86)		0.73(271)		0.47
Nausea-Vomiting		3.48 (10.26)			5.29 (13.28)			1.93 (6.33)		2.73(271)		0.01
Pain		11.92 (21.35)			11.90 (21.28)			11.94 (21.48)		−0.01(272)		0.99
Dyspnea		9.85 (17.24)			9.79 (17.42)			9.91 (17.15)		−0.06(272)		0.95
Insomnia		26.13 (27.73)			25.93 (27.29)			26.30 (28.20)		−0.11(271)		0.91
Appetite Loss		8.18 (17.68)			8.73 (19.40)			7.71 (16.11)		0.48(271)		0.64
Constipation		14.76 (22.84)			16.80 (23.04)			13.01 (22.62)		1.36(269)		0.17
Diarrhea		7.35 (17.51)			7.67 (18.46)			7.08 (16.71)		0.28(270)		0.78
Financial Problems		16.23 (26.17)			17.20 (26.02)			15.40 (26.36)		0.56(267)		0.57
Gender - Female	65		23.6	33		26.2	32		21.5		0.48	0.49
Racial Background - Non-White	12		4.4	4		3.2	8		5.4		0.69	0.41
Marital Status - Married/Partner	208		75.6	92		73.0	116		77.9		0.51	0.48
Employment Status - Employed	127		46.2	74		58.7	53		35.6		6.64	0.01
Cancer Stage
Ta, Tis, T1	121		44.3	49		38.9	72		49.0		0.90	0.34
T2-T4	152		55.7	77		61.1	75		51.0
Neoadjuvant Chemotherapy -Yes	123		44.9	66		52.8	57		38.3		4.72	0.03
Prior Intravesical Treatment -Yes	105		38.2	53		42.1	52		34.9		2.04	0.15
Prior Pelvic Radiation Treatment - Yes	24		8.7	15		11.9	9		6.0		1.98	0.16
Body Mass Index
Normal (18.5-25)	59		21.5	29		23.0	30		20.1
Overweight (25-30)	124		45.1	50		39.7	74		49.7		4.66	0.10
Obese (>30)	92		33.5	47		37.3	45		30.2

Note: EORTC QLQ-C30 indicates European Organization for Research and Treatment of Cancer Quality of Life Questionnaire

Response Burden

Overall mean response burden was 12.19 (SD 11.65) with a median of 8.0 and range 0-72. Levels of response burden did not significantly differ (t(273) = 0.96, p = 0.33) between those patients who only completed PRO questionnaires (M(SD) = 12.88(12.15)) and patients who completed idiographic interviews in addition to the PRO battery (M(SD) = 11.58(11.07)).

A total of 60 patients (21.8%; n = 29 PRO questionnaires only, n = 31 PRO questionnaires and idiographic interviews) indicated that there were unimportant or repetitive questions in the PRO battery. Eighty-seven patients (31.6%) reported via the response burden questionnaire that additional information should have been gathered as part of the PRO battery. These qualitative findings were not significantly correlated with overall response burden (Pearson r’s = 0.05 and −0.05, p’s = 0.40 and 0.39, respectively). A subset of these patients (n = 35, 12.7%) acknowledged that there were unimportant or repetitive questions in the PRO battery, but also suggested that additional information should have been included.

Predictors of Response Burden

In examining the multivariate hierarchical linear regression models of patient demographics, clinical characteristics, or HRQoL subscale scores as potential predictors of patient response burden (Table 4), low self-reported cognitive function was a significant predictor of higher response burden (β = −0.20; t(270) = −3.38; p = 0.01; Model Adjusted R² = 0.04). Clinical characteristics and other demographic and HRQoL subscale scores did not significantly predict response burden.

Table 4.

Final Hierarchical Regression Model

	Unstandardized Coefficients		Standardized Coefficients
Variable	B	Standard Error	β	t	p	Adjusted R2
Cognitive Function	−0.14	0.41	−0.2	−3.38	0.01	0.04

Predictors of Drop-Out

A total of 207 (75%) patients had complete EORTC QLQ-C30 data at the one-month post-surgical follow-up. Participant response burden was not significantly associated with participant drop-out (F(1,273) = 2.05, p = 0.15).

DISCUSSION

Despite the potential response burden associated with the completion of PRO measures in routine care and clinical trials, a growing number of patients with cancer are willing to self-report their experiences to not only to inform the care that they personally receive, but also to assist others who may share similar disease-related experiences in the future (3, 7, 21, 33, 34). We found that patients who complete a lengthy battery of PRO assessments experience minimal response burden as defined by our brief burden measure. No significant increase in response burden was observed for the subset of patients who also completed a 30-minute interview about their HRQoL experiences. While approximately 22% of our sample indicated that we asked questions that were repetitive or not relevant to their concerns, over half of these patients and a larger proportion of overall participants (32%) reported that we should have asked additional items about their condition. Additionally, participant response burden was not found to be a significant predictor of drop-out at the three-month post-surgical follow-up assessment. These findings should provide key stakeholders with confidence that the inclusion of a battery of psychosocial or behavioral PRO questionnaires in clinical trials or routine care are not perceived as burdensome, even by very sick cancer patients.

We also found that cognitive impairment, as captured by the EORTC QLQ-C30, is a significant predictor of higher patient response burden. This finding is intuitive, as individuals who have difficulty with memory or concentration may be more easily frustrated with questionnaire wording and length. Screening for cognitive impairment at trial entry or during an early routine care visit may facilitate the identification of patients who are more prone to experiencing burden. Future work should prioritize the utilization of interview-based PRO data collection with these impaired individuals may help to meet their important needs toward minimizing missing data and improve data quality in prospective studies.

Our internally developed measure of patient response burden was designed to include a number of key features. First, we wanted to avoid social desirability bias and intentionally did not ask about whether our specific measures were burdensome. We feel that the multidimensional item content (i.e., relationship of questions to actual concerns, comfort in answering questions, relevancy of questions to health, feelings about time to completion) encompasses key elements of response burden. The two open-ended questions serve as an avenue through which patients can express their desire for fewer or larger numbers of items. Lastly, we wanted this to be a brief assessment, so as to not introduce additional response burden. We acknowledge that this homegrown measure has not been previously validated. A future direction of this work is to provide formal validation of our response burden tool such that it can be deployed in similar contexts.

This study is not without a number of limitations. The study was completed at a single, tertiary cancer center with limited diversity of the patient population in terms of racial background, and only a single disease type was included. All PRO questionnaires were mailed to participants for home completion, thus not allowing us the opportunity to capture average length of time to completion, ultimately making it difficult to explore the association between perceived response burden and time to questionnaire completion. Based on the open-ended question responses, patients did not report being burdened by the questionnaire or interview aspects of the study and indicated that on average, these assessments did not take up a tremendous length of their time, however future studies of response burden should seek to quantify the length of time to complete questionnaires and/or interviews. Patient completion of these surveys at home rather than at the clinic site also removed the possibility of having study staff available to provide realtime assistance for questionnaire completion.

This is the first study that provides evidence that rigorous PRO assessment of multiple disease-related domains may not be perceived as burdensome to cancer patients as previously thought. While this work should be replicated across a broad range of patient and disease types, it is encouraging to find that patients are not only willing to complete these questionnaires and brief interviews regarding their disease, but are minimally burdened in doing so.