Can Electronic Web-Based Technology Improve Quality of Life Data Collection? Analysis of RTOG 0828

Abstract

Purpose

Missing data are a significant problem in clinical trials, particularly for quality of life (QOL), which cannot be obtained retrospectively. The purpose of this study was to evaluate the feasibility of an electronic web-based strategy for QOL data collection in a cooperative group radiation oncology trial setting.

Methods and Materials

RTOG-0828 was a prospective NCI cooperative group companion study of RTOG-0415, a randomized study of conventional versus hypofractionated radiation. Forty-nine English-speaking patients with favorable risk prostate cancer who enrolled on RTOG-0415 consented to using web-based technology for completing QOL. In RTOG-0415, using paper forms, the 6-month QOL compliance rate was 52%. The purpose of RTOG-0828 was to test the feasibility of a web-based strategy with the goal of increasing the 6-month QOL completion rate by 25% (from 52% to 77%) for a relative improvement of ∼50%. The web-based tool used in this study was VisionTree Optimal Care (VTOC), a Health-Insurance-Portability-Accountability-Act (HIPAA)-secure, online technology that allows real-time tracking and email reminders. The primary endpoint was the 6-month compliance rate for the validated QOL instrument, Expanded Prostate Index Composite (EPIC).

Results

The QOL completion rate at baseline was 98%. Compared to the prior 52% QOL completion rate at 6-months using paper forms, the QOL web-based completion rate at 6-months was 90% (two-sided p-value < 0.001). At 12-months, the EPIC completion rate was 82% (compared to 36% using paper forms).

Conclusions

This RTOG study suggests that a web-based strategy to collect QOL appears to be feasible in the cooperative group radiation oncology trial setting and is associated with an increase in the 6-month QOL compliance rate compared to the prior method of using paper forms. RTOG plans to further test this strategy in a head and neck cancer trial across all participating RTOG sites.

Introduction

Missing data have long been recognized as a significant problem affecting the validity of quality of life (QOL) studies ^1,2. Unlike data for traditional endpoints, such as survival, QOL data cannot be obtained retrospectively. Missing QOL data not only limits the statistical power of the analysis, but also may introduce bias which affects the interpretation of the study, necessitating a variety of involved methodologic approaches ^3,4. Clearly, the optimal strategy is to “prevent” missing QOL data from occurring in the first place.

Typically, in most clinical trials, patients fill out QOL forms on “hard” copies (paper forms). The Radiation Therapy Oncology Group (RTOG), one of the National Cancer Institute sponsored oncology research cooperative groups, provides Research Associates (RAs) with calendars for each registered patient. One of the many RA responsibilities is to periodically review each calendar and give participants the QOL forms in the clinic or mail them as needed. RTOG monitors forms due and informs sites of delinquencies which carry administrative disincentives. These strategies alone, however, have left room for improvements in decreasing missing QOL forms.

To develop additional strategies to reduce missing QOL data, the RTOG initiated a prospective study (RTOG-0828) to evaluate a HIPAA-secure web-based technology that allows patients to fill out QOL forms from any computer with internet access. The goal of the study was to test the feasibility of this technology as a tool to improve compliance of QOL data collection within a cooperative group radiation oncology trial setting.

Methods and Materials

RTOG-0415, the actively accruing “parent protocol” selected for this study, was a randomized study of conventional versus hypofractionated radiation for favorable-risk prostate cancer patients. This trial was registered at clinicaltrials.gov (identifier: NCT00331773). Of the initial 302 patients on RTOG-0415 who consented to participate in the QOL component of the study, >90% completed their baseline validated Expanded Prostate Index Composite (EPIC) QOL form. However, at the 6-month timepoint, the QOL paper form was only completed (within a 1-month window period) by 52% of the patients (and at 1-year, by only 36%). The most common reason for noncompliance was “institutional error” (>30%), such as the QOL form not being given or mailed to the patient. The purpose of this companion study (RTOG-0828) was to test the feasibility of an electronic web-based system in a cooperative group radiation oncology setting with the goal of improving EPIC QOL completion rate at 6 months by 25% (from 52% to 77%) for a relative improvement of ∼ 50%. RTOG 0828 was performed in accordance with the ethical standards of Institutional Review Boards (IRBs) and with the Helsinki Declaration of 1975, as revised in 2000. This study (RTOG-0828) focused on the top 20 accruing institutions within RTOG 0415. In addition to consenting to the parent study (RTOG-0415), patients also consented for this companion study (RTOG-0828).

The web based tool used in this study was VisionTree Optimal Care (VTOC), a HIPAA-secure, software system developed by VisionTree Software, Inc (San Diego,CA). The VTOC tool is compliant with the Title-21, Code of Federal Regulations and is encrypted (via 256-bit SSL) for security, privacy, and confidentiality. It is similar to the secure login commonly used when performing on-line banking.

To be eligible for this companion study, patients had to consent to the RTOG-0415 QOL component, be English-literate, and have an email address that they consented to use for the purpose of this study. E-mail addresses were necessary so that real-time reminders could be sent to patients to fill out QOL forms that are becoming due. If the site research associate (RA) received notice that forms had not been completed, the RA could contact the patient to remind him to complete the forms or to inquire why forms had not been completed. Patients who were interested in participating but did not yet have an e-mail address could obtain one for free from a number of sources.

Statistical Analysis

The EPIC completion rate at 6 months in RTOG 0415 (in these 20 institutions) was 52%. The hypothesis of RTOG-0828 was that the completion rate of EPIC at 6-months will improve by at least 25% in the cohort of patients using the electronic web-based system (VTOC). With a power of 85% and a significance level 0.05, the two-sided Z-test for one-sample proportions yields a minimum sample of size 33 ⁵. Hence, with a projected ineligibility rate of 5%, the sample required was 35 patients. This number does not include the run-in sample of 5 patients for RTOG to learn to use the VTOC system for a minimum accrual of 40 patients. All patients were included in the QOL completion rate analysis and all p-values are two-sided.

Results

Forty-nine patients enrolled onto RTOG-0828 between September 2008 and December 2009. The baseline characteristics for patients consenting to QOL using the web-based approach (VTOC) on RTOG-0828 compared to patients who consented to QOL using paper copies on RTOG-0415 are shown in Table 1. There were no significant differences, except that patients on RTOG-0828 had a median age of 4 years younger the patients on RTOG-0415 (63 vs 67 years old). Similarly, there were no significant differences in these characteristics between patients who did or did not consent to the QOL component of RTOG-0415 (data not shown).

Table 1. Pretreatment Characteristics (n=968).

Baseline characteristics of patients who consented to quality of life (QOL) study on RTOG 0828 (using VisionTree Optimal Care, VTOC) and those on the parent study, RTOG 0415 (who did not use VTOC).

	RTOG 0828 QOL (VTOC) Patients (n=49)	RTOG 0415 QOL Patients (No VTOC) (n=919)	p-value*
Age (years)			0.005†
Median	63	67
Race			0.906
American Indian/Alaska Native	0 (0.0%)	6 (0.7%)
Asian	1 (2.0%)	11 (1.2%)
African American or Black	7 (14.3%)	165 (18.0%)
Native Hawaiian or Other Pacific Islander	0 (0.0%)	2 (0.2%)
White	41 (83.7%)	726 (79.0%)
Unknown or not reported	0 (0.0%)	9 (1%)
Ethnicity			0.607
Hispanic or Latino	1 (2.0%)	28 (3.0%)
Not Hispanic or Latino	46 (93.9%)	818 (89.0%)
Unknown	2 (4.1%)	73 (7.9%)
Zubrod Performance Status			0.571
0	47 (95.9%)	852 (92.7%)
1	2 (4.1%)	67 (7.3%)
PSA			0.714
<4 ng/ml	11 (22.4%)	183 (19.9%)
4-<10 ng/ml	38 (77.6%)	736 (80.1%)
Gleason			0.999
2-4	0 (0.0%)	7 (0.8%)
5-6	49 (100.0%)	912 (99.2%)
Marital Status			0.366
Married or Partnered	38 (77.6%)	645 (70.2%)
Other	10 (20.4%)	215 (23.4%)
Not Reported	1 (2.0%)	59 (6.4%)
Education Level			0.121
Less than High School	3 (6.1%)	150 (16.3%)
High School or Some Collage	27 (55.1%)	405 (44.1%)
Bachelor's and/or Above	15 (30.6%)	238 (25.9%)
Other, Not Reported	4 (8.2%)	126 (13.7%)

^*Age based on Wilcoxon test; all other variables based on exact chi-square test; race and ethnicity were defined by the patients

^†Significant at the 0.05 level (two-sided p value)

The EPIC QOL compliance rates at baseline, 6 months (the primary endpoint) and 12 months are shown in Table 2. At baseline, the QOL completion rate was 98%. Compared to the 52% 6-month EPIC completion rate using paper forms (in RTOG-0415), the EPIC QOL web-based completion rate at 6 months was 90% (two-sided p-value < 0.001). At one year, the EPIC QOL web-based compliance rate was 82%. Reasons for noncompliance at 6 months with EPIC QOL via the web-based strategy were patient refusal (2%), patient could not be contacted (2%) or other reason (6%).

Table 2. EPIC Completion Rate (n=49).

Expanded Prostate Index Composite (EPIC) compliance rates using VisionTree Optimal Care (VTOC) at baseline, 6 months and 12 months.

	Baseline	6 Month	12 Month
Completed	48 (98%)	44 (90%)	40 (82%)
NC*, pt† refusal	0	1 (2%)	1 (2%)
NC*, pt† could not be contacted	0	1 (2%)	3 (6%)
NC*, other reason	1 (2%)	3 (6%)	5 (10%)

^*NC = not completed

^†Pt = patient

Discussion

This RTOG study suggests that a web-based strategy to collect QOL appears to be feasible in the cooperative group radiation oncology trial setting and is associated with a >70% increase in the 6-month EPIC QOL compliance rate from 52% to 90% by switching from paper forms to a web-based technology. Even at 12 months, the EPIC QOL web-based compliance rate was 82% (compared to only 36% using paper forms). These findings suggest that a web-based strategy can help reduce missing QOL data within a cooperative group setting. This strategy, utilizing real-time email reminders, appears to reduce institutional error as a cause of missing data (e.g., the QOL form was never given or mailed to the patient). Dr. Ann O'Mara, Head of Palliative Care Research, Division of Cancer Prevention (DCP), NCI, stated that “this study addresses a critical issue of missing data, which has been a key challenge over the years” (personal communication).

While computerized QOL monitoring is not new ⁶, this study, to our knowledge, is the first test of web-based technology to prospectively capture QOL within an NCI cooperative group oncology trial setting. Similarly, computerized adaptive testing (CAT) is an ongoing effort through the NIH Roadmap Initiative known as the Patient-Reported Outcomes Measurement Information System or PROMIS ⁷. In a recent survey of 1580 cancer patients, 55% of patients were willing to use an internet-based QOL tool ⁸. The authors suggest that “these data may serve as a basis for……projects to evaluate the implementation of internet-based regular assessment of QOL in cancer”, the experience presented here. Of note, prior studies have shown that paper and web-based surveys provide essentially equivalent information ⁹.

There are several limitations to this study. First, this was not a randomized study and was not designed to prove that an electronic strategy is indeed superior to using paper forms to collect QOL. Rather, RTOG-0828 was intended to be a feasibility study to prospectively test this approach in the cooperative group setting. Overall, the patient characteristics on this study (RTOG-0828) appear similar to those on the parent study (RTOG-0415), with the exception of younger age, for which the difference was only four years. Of note, after first consenting to the parent study (RTOG 0415), addressing the primary issue comparing conventional versus hypofractionated radiation, patients were next asked if they would secondarily consent to complete their QOL information. If they agreed, patients were then offered the option of also enrolling on this companion study (RTOG 0828) to collect QOL electronically (instead of using paper forms). It is therefore possible that patients on this companion study, which focused on the QOL completion rate, may have been more motivated to complete their QOL even than patients who consented to do so using paper forms on the parent study. The dramatic increase in compliance (from 52% to 90%), coupled in particular with the much lower rates of institutional error, suggest that this method has potential to reduce missing QOL data. Other strategies to reduce institutional error and more consistently deliver paper QOL forms to patients at the proper timepoints could also be considered. However, the protocol demands and logistic constraints on research associates is already a significant challenge. Second, this was a relatively small study involving 49 patients with prostate cancer from 20 institutions. In this regard, RTOG plans to next test this approach in a larger and more challenging population of head and neck cancer patients across all participating RTOG sites. Third, not all patients have the interest, access, or know-how to fill out forms on-line. It is interesting, however, that the average age in this study was 64 years, suggesting that patients in the 60′s and beyond are open to this novel approach. Nevertheless, patients can always opt to use paper forms, if they prefer. As more patients become computer-savvy, this web-based approach will likely become more popular over time.

Beyond the improvement in QOL compliance, there are other potential benefits to this web-based system. A survey of the research associates suggested that this system saved the research associates an average of 10 minutes per QOL form by not having to find patients in the clinic or mail out forms. Thus, an electronic option for patients to directly complete their QOL information on-line not only can reduce the risk of institutional error, but also can help free the research associates to perform other much needed protocol duties. Moreover, this technology allows for real-time tracking of the QOL compliance, so investigators can be informed early on if there's missing data. Additional features of such a web based system could ultimately be integrated into the routine clinical practice with real-time QOL feedback from the patients to their physicians ^10-12.