Abstract
Background/Aims
The use of Patient Reported Outcomes (PROs) in clinical trials is a focal point for research and policy. Non-compliance with planned questionnaires and missing data can threaten both internal validity and generalizability. This retrospective analysis was conducted to determine the extent of, and characteristics associated with, missing PROs.
Methods
Study characteristics, patient characteristics and adverse events, and reasons for non-compliance were compiled from 14 closed Alliance for Clinical Trials in Oncology, Mayo Clinic Cancer Center, or Mayo Clinic Cancer Research Consortium clinical trials. Compliance rates were calculated for each patient using the number of booklets completed while the patient was on trial divided by the number of booklets the patient was expected to complete. Frequency counts and summary statistics were compiled. Logistic regression techniques were employed.
Results
The 1640 included patients had a median age of 58 years, were mostly white (90.8%) and female (73.8%). Compliance rates per study ranged from 84.7% to 97.2%. The primary endpoint of overall compliance rate was 93.1%. A total of 1267 patients were compliant. Those non-compliant were slightly older (mean 58.6 vs 57.5, p=0.03) and had different types of cancer (p<0.01). There were no differences in compliance according to tumor status (p=0.66), clinical stage (p=0.81), baseline quality of life (p=0.42 for ≥8 vs <8 and p=0.12 for ≥6 vs <6) or maximum adverse event grade incidence (p=0.33 for grade 2+ incidence and p=0.36 for grade 3+ incidence). Reasons for non-compliance included patient refusal (N=136), booklet not administered to patient (N=199), no clinic visit at the scheduled time for booklet completion (N=40), and at-home completed booklet not returned (N=224). Logistic regression indicates gender (p<0.01), race (p<0.01), performance score (p=0.02), dose delay status (p=0.01) and incidence of grade 3 or higher adverse event (p=0.03) were correlates of compliance.
Conclusions
PROs have successfully been implemented into Alliance and Mayo Clinic trials with high rates of patient compliance. Further improvement in compliance can be made with staff commitment and education. Patients are typically noncompliant only when the task at hand is burdensome, unclear or logistically challenging. Existing tracking systems used for the other trial outcomes should be utilized to ensure successful capture of PRO outcomes.
Keywords: Patient reported outcomes, compliance rates, quality of life
Background
The use of Patient Reported Outcomes (PROs) in clinical trials continues to be a focal point for research and policy.1,2 Patient responses are utilized as measures of primary and secondary endpoints, and as such, completion of required assessments is necessary to draw proper conclusions.3 Numerous efforts have summarized various key aspects of including PROs in clinical trials;4 this report addresses the issue of patient compliance to ensure data completeness.
Non-compliance with planned questionnaires and resultant missing data can threaten both internal validity and generalizability.5 Compliance rates have varied in the literature. Osoba reported on average, the PRO rate for 7 completed clinical trials conducted by the Clinical Trials Group of the National Cancer Institute of Canada was 93% at baseline and during treatment, and 85% for off-treatment follow-up.6 Walker noted that for a study involving 159 patients with malignant glioma, only 20% of patients completed all 4 required PRO questionnaires. Compliance rates for admission and discharge questionnaires were in the 80 percentile range, but the rates dropped to the 50 and 60 percentile ranges for post-discharge assessments. Patient refusal accounted for only 6% of cases, while administrative failure (patient non-attendance, administrative errors or reluctance to assess ill patients, etc.) accounted for 72% of cases.7 Cuffe reported 84% compliance for patients having head and neck cancer.8
Patient agreement to comply with protocol PRO completion requirements in Mayo Clinic Cancer Center and North Central Cancer Treatment Group trials, now part of the Alliance for Clinical Trials in Oncology, has historically been high. However, a comprehensive audit of PRO completion has never been carried out to determine actual compliance rates or reasons for non-compliance. This retrospective meta-analysis was conducted in an exploratory manner to determine the extent of missing patient-completed forms and to determine if there are characteristics of the patient, clinical staff, or the parent trial that contribute to compliance rates. The ultimate goal is to use the information discovered to help ensure protocol procedures make it efficient and practical for patients to complete the PROs, thus resulting in more complete data and trial quality improvement.
Methods
This study was developed, approved and carried out under the North Central Cancer Treatment Group (now Alliance) guidelines and procedures (protocol number N0992, Institutional Review Board number 09-002846). Data were compiled from 14 closed Alliance (Legacy North Central Cancer Treatment Group), Mayo Clinic Cancer Center, or Mayo Clinic Cancer Research Consortium clinical trials where patient compliance in completing PROs was documented (Table 1). Each participant signed an Institutional Review Board-approved, protocol-specific informed consent in accordance with federal and institutional guidelines. In all studies, the PROs were compiled into booklets that included a cover page and a patient information sheet with an explanation of the content of the booklet and directions for booklet completion. As a part of routine data collection, a compliance form (Figure 1) was completed to document the reason for the non-compliance when a patient did not complete any part of a booklet as required per protocol.
Table 1.
Study Characteristics
| Protocol | Description | Projected Patients/Actual Accrual |
Site Specify | Assessment Schedule |
|---|---|---|---|---|
| MC0915 | Phase I Open Label, Dose Escalation Study of the Safety, Tolerability and Pharmacokinetic Properties of the Combination of Cilengitide and Paclitaxel in Patients with Advanced Malignancies |
32/12 | Breast | 3 booklets per patient: completed during cycle 1 |
| MC0995 | PAL-2: A Pilot Study of the Palliative ARNP (Advanced Registered Nurse Practitioner) Liaison Program (The PAL Program) Intended to Improve Patients' Quality of Life Through Targeted FACT-G Driven Palliative Interventions |
25/14 | Lymph nodes and bone marrow |
2 booklets per patient: 1 at baseline and 1 at 28 days post enrollment. |
| MC09C4 | Phase II Evaluation of S-Adenosyl-L-Methionine (SAMe) for the Treatment of Hot Flashes |
45/43 | N/A | 1 booklet per patient: hot flash diary completed daily and PROs completed weekly for 7 weeks |
| MC10C9 | Phase II Randomized Placebo-Controlled Exploration of a Topical Menadione-Containing Lotion to the Face for Prevention and Palliation of Epidermal Growth Factor Receptor Inhibitor-Induced Cutaneous Discomfort and Psychological Distress |
40/25 | Colorectal | 3 booklets per patient: 1 at baseline, 1 weekly for 4 weeks during treatment, and 1 weekly for 4 weeks during observation |
| MC11C4 | A Pilot Randomized, Placebo-Controlled, Double Blind Study of Venlafaxine to Prevent Oxaliplatin-Induced Neuropathy |
50/44 | Colorectal | Multiple booklets per patient: 1 daily for the first 6 days of every week cycle of treatment followed by 5 post-treatment |
| N07C4 | Phase II Randomized Trial Evaluating Two Non- pharmacologic Interventions in Cancer Survivors for the Treatment of Sleep-Wake Disturbances |
168/81 | N/A | 2 booklets per patient: 1 daily during week 1 and one daily during weeks 2–7 |
| N08C9 | Phase III Randomized Study of Sulfasalazine versus Placebo in the Prevention of Acute Diarrhea in Patients Receiving Pelvic Radiation Therapy |
140/84 | Endometrial | 5 booklets per patient: 1 at baseline, 1 weekly during treatment, 1 weekly for 6 weeks post treatment, and 2 during observation |
| N08CB | Phase III Randomized, Placebo-Controlled, Double-Blind Study of Intravenous Calcium/Magnesium in Two Different Versions to Prevent Oxaliplatin-Induced Sensory Neurotoxicity |
354/352 | Colorectal | Multiple booklets per patient: 1 daily for the first 6 days of every 2 week cycle of treatment followed by 5 post-treatment |
| N0949 | Phase III Randomized Trial of mFOLFOX7 or XELOX Plus Bevacizumab Versus 5-Fluorouracil/Leucovorin or Capecitabine Plus Bevacizumab as First-line Treatment in Elderly Patients with Metastatic Colorectal Cancer |
380/31 | Colorectal | Multiple booklets per patient: 1 prior to every treatment cycle, 1 every 3 months during treatment, and 1 during observation |
| N09C6 | Phase III Randomized Double-Blind Study of Doxepin Rinse versus Placebo in the Treatment of Acute Oral Mucositis Pain in Patients Receiving Radiotherapy with or without Chemotherapy |
198/156 | Head/Neck | Multiple booklets per patient: 2 on day 1, 2 on day 2, and 1 every week during optional continuation phase |
| N1031 | Phase II Randomized Study of Two Doses of Pixantrone in Patients with Metastatic Breast Cancer |
56/45 | Breast | Multiple booklets per patient: 1 prior to every cycle of treatment, during observation |
| N10C1 | Phase III Vaginal DHEA for Vaginal Symptoms: A Phase III Randomized, Double Blind, Placebo-Controlled Study |
456/451 | N/A | Up to 4 booklets per patient: 1 at baseline, 1 containing PROs to be completed at weeks 4, 6 and 8, 1 at week 12 and 1 at during continuation phase |
| N10C2 | Phase III Double-Blind, Placebo-Controlled Study of Magnesium Supplements to Reduce Menopausal Hot Flashes |
288/291 | N/A | Up to 4 booklets per patient: 1 daily during baseline week, 1 daily during weeks 2–5, 1 daily during weeks 6–9, and 1 daily during 4 weeks of continuation phase |
| RC11C3 | Pilot Placebo-Controlled Evaluation of Pregabalin as a Means to Prevent the Paclitaxel-Associated Acute Pain Syndrome |
46/43 | Breast | Multiple booklets per patient: 1 at baseline, 1 daily for every 1 week cycle of treatment followed by 5 post-treatment |
Figure 1.
Booklet Compliance Form
The current compliance form was developed to identify when missing PRO data would not be recoverable and to clarify reasons for non-compliance, so those issues could be addressed in the design of future trials. The previous version of the form collected patient refusal, unable to accommodate disability or language needs, staff unavailable, patient not given form by staff, patient did not like content of questions, site did not like content of questions and other. The Mayo Clinic Cancer Control and Quality of Life statistical research team and investigators agreed that the options on the previous form either weren’t appropriate or didn’t contain enough detail to identify how improve compliance and better serve the needs of the patient. The statistical team and investigators at Mayo Clinic Rochester collaborated with clinical research associates and quality assurance specialists to expand the non-compliance choices for better evaluation.
Data collected included study characteristics and baseline patient characteristics. In studies where a global measure of quality of life was asked of the patient, the response data were collected and scores were converted to a 0–10 theoretical scale so all assessments had the same range. The median score was calculated and patients were categorized as being less than median or greater than or equal to the median. Further, patients were categorized as having a score less than 6 or greater than or equal to 6. The selection of 6 as a meaningful quality of life score was the result of previous research where a baseline score of less than 6 resulted in poorer survival.9 Reasons for non-compliance, end of active treatment reasons and adverse event data were collected on all studies. Patients were categorized as having a maximum adverse event grade of 2 or higher and 3 or higher, and incidences of clinical events of a dose delay and a dose reduction at any time during treatment were recorded as yes/no variables.
Compliance rates per patient were calculated and patients were assigned a compliance status (yes for having complete data vs no for having non-complete data). Frequency counts and summary statistics were compiled. Simple logistic regression techniques were employed to determine if any patient characteristic, study characteristics, or clinical events influenced patient compliance.
The combined data sets were comprised of individual patient-level information. Simple un-weighted analytic procedures were performed. Since this study was exploratory in nature, no formal power statement is provided. Data collection and statistical analyses were conducted by the Alliance Statistics and Data Center. Data quality was ensured by review of data by the Alliance Statistics and Data Center and by the study chairperson following Alliance policies. All analyses were based on the study database frozen on 04/07/2016.
Results
Patients in this meta-analysis had a median age of 58 years, were mostly white (91%) and female (74%) (Table 2). They participated in clinical trials of any phase for any tumor type. Studies that used the specific compliance form and that were opened and closed to patient enrollment between April 2010 and July 2014 were included. Three of these trials were chemotherapy treatment trials, one was a quality of life specific study and 10 were cancer control studies.
Table 2.
Patient Characteristics by Compliant Status
| 100% Compliant (N=1267) |
Non Compliant (N=373) |
Total (N=1640) |
P-value | |
|---|---|---|---|---|
| Age | 0.031 | |||
| Mean (SD) | 57.5 (9.5) | 58.6 (11.6) | 57.7 (10.0) | |
| Median | 57.0 | 59.0 | 58.0 | |
| Range | (24.0–95.0) | (26.0–86.0) | (24.0–95.0) | |
| Race | <0.012 | |||
| White | 1168 (92.2%) | 321 (86.1%) | 1489 (90.8%) | |
| Black or African American | 68 (5.4%) | 41 (11.0%) | 109 (6.6%) | |
| Native Hawaiian or Other Pacific Islander | 1 (0.1%) | 0 (0.0%) | 1 (0.1%) | |
| Asian | 15 (1.2%) | 4 (1.1%) | 19 (1.2%) | |
| American Indian or Alaska Native | 4 (0.3%) | 4 (1.1%) | 8 (0.5%) | |
| Not reported | 11 (0.9%) | 3 (0.8%) | 14 (0.9%) | |
| Gender | <0.012 | |||
| Female | 985 (77.7%) | 226 (60.6%) | 1211 (73.8%) | |
| Male | 281 (22.2%) | 147 (39.4%) | 428 (26.1%) | |
| Unknown | 1 (0.1%) | 0 (0.0%) | 1 (0.1%) | |
| ECOG Performance Status | <0.012 | |||
| Unknown | 32 | 6 | 38 | |
| 0 | 981 (79.4%) | 259 (70.6%) | 1240 (77.4%) | |
| 1–2 | 254 (20.6%) | 108 (29.4%) | 362 (22.6%) | |
| Tumor Status | 0.662 | |||
| Unknown | 1035 | 252 | 1287 | |
| Resected with no residual | 80 (34.5%) | 41 (33.9%) | 121 (34.3%) | |
| Resected with known residual | 32 (13.8%) | 12 (9.9%) | 44 (12.5%) | |
| Unresected | 109 (47.0%) | 60 (49.6%) | 169 (47.9%) | |
| Recurrent | 11 (4.7%) | 8 (6.6%) | 19 (5.4%) | |
| Cancer Type | <0.012 | |||
| Unknown | 1 (0.1%) | 0 (0.0%) | 1 (0.1%) | |
| Breast | 404 (31.9%) | 54 (14.5%) | 458 (27.9%) | |
| Breast/Gynecologic | 397 (31.3%) | 45 (12.1%) | 442 (27.0%) | |
| GI | 268 (21.2%) | 230 (61.7%) | 498 (30.4%) | |
| Gyn | 7 (0.6%) | 9 (2.4%) | 16 (1.0%) | |
| Head and Neck | 139 (11.0%) | 20 (5.4%) | 159 (9.7%) | |
| Heme | 5 (0.4%) | 1 (0.3%) | 6 (0.4%) | |
| Lung | 10 (0.8%) | 3 (0.8%) | 13 (0.8%) | |
| Prostate | 15 (1.2%) | 7 (1.9%) | 22 (1.3%) | |
| Other | 21 (1.7%) | 4 (1.1%) | 25 (1.5%) | |
| Clinical Stage | 0.812 | |||
| Unknown | 1194 | 366 | 1560 | |
| Stage I | 23 (31.5%) | 3 (42.9%) | 26 (32.5%) | |
| Stage II | 24 (32.9%) | 2 (28.6%) | 26 (32.5%) | |
| Stage III | 19 (26.0%) | 2 (28.6%) | 21 (26.3%) | |
| Other | 7 (9.6%) | 0 (0.0%) | 7 (8.8%) | |
| Length of Time on Study (in Days) | <0.011 | |||
| Mean (SD) | 95.0 (102.4) | 112.3 (86.9) | 98.9 (99.3) | |
| Median | 87.0 | 102.0 | 88.0 | |
| Range | (1.0–2079.0) | (2.0–1071.0) | (1.0–2079.0) | |
| Overall QOL | 0.423 | |||
| >=8 | 267 (21.1%) | 71 (19.0%) | 338 (20.6%) | |
| <8 | 1000 (78.9%) | 302 (81.0%) | 1302 (79.4%) | |
| Overall QOL | 0.123 | |||
| >=6 | 397 (31.3%) | 101 (27.1%) | 498 (30.4%) | |
| <6 | 870 (68.7%) | 272 (72.9%) | 1142 (69.6%) | |
| AE Grade >=3 (Y/N) | 0.363 | |||
| Not Collected | 395 | 65 | 460 | |
| N | 825 (94.6%) | 296 (96.1%) | 1121 (95.0%) | |
| Y | 47 (5.4%) | 12 (3.9%) | 59 (5.0%) | |
| AE Grade >=2 (Y/N) | 0.333 | |||
| Not Collected | 395 | 65 | 460 | |
| N | 698 (80.0%) | 238 (77.3%) | 936 (79.3%) | |
| Y | 174 (20.0%) | 70 (22.7%) | 244 (20.7%) | |
| Dose Reduced (Y/N) | <0.013 | |||
| Y | 43 (3.4%) | 34 (9.1%) | 77 (4.7%) | |
| N | 1224 (96.6%) | 339 (90.9%) | 1563 (95.3%) | |
| Dose Delayed (Y/N) | <0.013 | |||
| Y | 263 (20.8%) | 132 (35.4%) | 395 (24.1%) | |
| N | 1004 (79.2%) | 241 (64.6%) | 1245 (75.9%) | |
| Study Type | <0.012 | |||
| Cancer Control | 1208 (95.3%) | 331 (88.7%) | 1539 (93.8%) | |
| QOL | 11 (0.9%) | 1 (0.3%) | 12 (0.7%) | |
| Treatment Trial | 48 (3.8%) | 41 (11.0%) | 89 (5.4%) | |
Kruskal Wallis
Chi-Square
Fisher Exact
Each of the 14 trials included in this analysis had their own set of assessments and their own schedule for completion. Further, the content of any booklet within any trial could have variability depending on when the booklet was administered. Thus completion of a booklet (not individual assessments within a booklet) was selected as our tool of measurement.
Compliance rates per study ranged from 84.7% to 97.2% (Table 3). The primary endpoint of overall percent of submitted booklets was 93.1% (10358 completed out of 11129 expected booklets). There were 1267 patients who were 100% compliant and 373 who were not. Those that were not compliant were slightly older (mean 58.6 vs 57.5, p=0.03), were on trial slightly longer (112 days vs 95 days, p<0.01) and had different cancer types (p<0.01). There were no differences in compliance according to tumor status (p=0.66), clinical stage (p=0.81), baseline quality of life (p=0.42 for ≥8 vs <8 and p=0.12 for ≥6 vs <6, the criteria indicating an effect on survival9) or maximum adverse event grade incidence (p=0.33 for grade 2+ incidence and p=0.36 for grade 3+ incidence). Females were more likely to be compliant than males (81.3% vs 65.7%, respectively, p<0.001) as were whites vs non-whites (78.4% vs 65.6%, p<0.01). Patients who were between the ages of 50 and 64 were more compliant (81.0%) than those less than 50 (73.4%) or those 65 or older 71% (p<0.01). Patients with an Eastern Cooperative Oncology Group performance status of 0 were more compliant than those with a performance status of 1–2 (79.1% vs 70.2%, p<0.01). Patients without dose reductions during treatment had a higher compliance rate than those without (78.3% vs 55.8%, p<0.01) as were those without having dose delays (80.6% vs 66.6%, p<0.01).
Table 3.
Compliance rates by study
| Protocol | N | Number of booklets expected |
Number of booklets completed |
Completion rate |
|---|---|---|---|---|
| MC0915 | 12 | 36 | 34 | 94.4 |
| MC0995 | 13 | 25 | 24 | 96.0 |
| MC09C4 | 43 | 43 | 41 | 95.3 |
| MC10C9 | 25 | 61 | 56 | 91.8 |
| MC11C4 | 47 | 543 | 498 | 91.7 |
| N07C4 | 81 | 161 | 152 | 94.4 |
| N08C9 | 84 | 755 | 672 | 89.0 |
| N08CB | 352 | 5033 | 4638 | 92.2 |
| N0949 | 31 | 449 | 382 | 85.1 |
| N09C6 | 148 | 829 | 806 | 97.2 |
| N1031 | 45 | 215 | 182 | 84.7 |
| N10C1 | 442 | 1353 | 1303 | 96.3 |
| N10C2 | 274 | 920 | 885 | 96.2 |
| RC11C3 | 43 | 706 | 685 | 97.0 |
If a patient did not complete a booklet due to not completing the trial and the patient did not have a clinic visit at the time of going off-study, that event was not considered an expected booklet. There were 94 of these cases. If a patient was seen at the clinic but the patient was not offered a booklet to complete when they went off-study, that event was counted as an expected booklet since the patients should have been offered the booklet as a part of the evaluation. Reasons for non-compliance were recorded for 609 of the 771 missing booklets. They were patient refusal (N=136), booklet not administered to patient (N=199), no clinic visit at the scheduled time for booklet completion (N=40), at home completed booklet not returned (N=224), the patient moved (N=8), the patient didn’t realize the booklet needed to completed (N=1), and unknown (N=1).
Within the patient refusal category, subcategory items selected to identify the reason for refusal were physical reasons (N=40), mental or emotional reasons (N=13), booklet was too long to complete (N=2), patient didn’t have time to complete the booklet (N=1), and no reason given or unable to identify the reason (N=80).
Within the not administered category, subcategory items selected were staff unavailable (N=32), staff did not give patient a booklet to complete (N=122), staff misinterpreted protocol (N=39), patient was given the wrong booklet (N=4), and the booklet was misplaced by the site (N=2).
Within the no visit category, subcategory items selected were there was no clinic visit due to treatment being held or delayed (N=7), there was no clinic visit scheduled by mistake (N=10), and the patient missed or cancelled the visit (N=23).
Within the at home booklet not returned category, subcategory items selected were patient did not have a booklet to complete (N=1), patient indicated the booklet was sent but it was not received by the site (N=57), patient did not complete, threw away or lost the booklet (N=30), and reason unknown (N=136).
The two studies having lower compliance than the others are indicated in Table 4. N0949 was a phase III trial for patients with colorectal cancer which had an 85.1% compliance rate. The main reasons for non-compliance were both patient driven (N=11) and study personnel driven (N=23). N1031 was a phase II trial for patients with breast cancer. The completion percentage was 84.7% which was distributed between patient driven reasons (N=22) and study personnel driven reasons (N=4). There was one patient on this study who refused to complete any booklet. If this patient was removed from the analysis, the completion percentage rose to 89.7%.
Table 4.
Non-Compliance Reasons for Studies With the Poorest Compliance Rates
| Study | Refusal Category | Frequency |
|---|---|---|
| N0949 | Patient Refusal | 9 |
| Booklet (or correct booklet) not given to patient | 23 | |
| At-home completed booklet not returned via mail | 1 | |
| Unknown | 1 | |
| N1031 | Patient Refusal | 18 |
| Booklet (or correct booklet) not given to patient | 4 | |
| At-home completed booklet not returned via mail | 4 |
Logistic regression results were predominantly consistent with the group comparisons, above. Males were 36% less likely that females to be compliant, whites were 2.3 times more likely to be compliant than non-whites, those with a performance score of 1–2 were 68% less likely to be compliant than those with a performance score of 0, those with a grade 3 or higher maximum adverse event grade were 2.4 times more likely to be compliant and those with dose delays were 1.5 times more likely to be compliant.
Discussion
Overall, the compliance rates for PRO booklet completion on these studies were high. With the exception of 3 studies, the compliance rate was above 90%. A perception existed that patient refusal to complete questionnaires was the primary reason for non-compliance.6 Our data suggest that while 22% of the noncompliance cases (136/609) indicated refusals, it was not simply a matter of patients not wanting to fill out the questionnaires. While the most common subcategory in this area was reason unknown (59%, 80/136), there were both physical (29%, 40/136) and emotional reasons (10%, 13/136). Another main reason for missing booklets was due to staff error (33% (199/609)). These indicate that the communication to staff about process was an underlying theme contributing to the non-compliance. This is important to recognize so that precautions may be taken to achieve compliance.
Patient burden has been indicated as a reason for non-compliance in other studies.10–12 In our responses, only 1 patient indicated that the booklet was too long to complete. For most studies, the compliance rates did not seem to have been effected by the number of booklets required nor the number of assessments within a specific booklet. For example, in study MC09C4, patients were required to complete a hot flash diary daily for 49 days plus complete a side-effects questionnaire at the end of each week for 7 weeks and complete 2 other assessments at both baseline and end of study. These were bound into 1 booklet and the study had a 95.3% compliance rate. In studies N08CB and MC11C4, patients were to complete a booklet for 6 days every cycle of chemotherapy (14 day cycles for 6 months) and at 5 post-chemotherapy time points. The compliance rates for these studies were 92.2% and 91.7%, respectively. In contrast, for study N0949, patients completed 1 booklet with 7 assessments at baseline, a booklet with 2 assessments every 14 or 21 day cycle during active monitoring, a booklet with 2 assessments every 3 months during active monitoring, and a booklet at 2 time points with 8 assessments during observation. The compliance rate for this study was 85.1%, but interestingly no one on N0949 commented that there were too many booklets or the booklets were too long. For our lowest compliance rate (84.7%) on study N1031, patients completed multiple booklets with only 2 assessments per booklet. On study N08CB, there was one patient who declared the booklet was too long at 2 different time points. This choice was not on the compliance form thus this response for patient refusal might be under reported.
A problem area that can be controlled is the case where patients are not provided with the PRO booklets to complete, for any reason. Education on the protocol conduct may be key and specific instructions should be written in the protocol. It should be ensured that patient completed booklets are available when patients are enrolled. Another area that may be improved is when patients do not complete required booklets outside of a clinic visit. This may occur when study participation has ended or when the patient is in follow-up. Phone calls or other forms of communication could be options for reminding patients to send in completed booklets. Note that this may not be feasible for large clinical trials. Some protocols in this analysis directed staff to encourage patients to complete booklets even if they did not complete the study while some protocols did not, thus there is a need for directives in the cases where patients do not complete study per protocol but booklets are still required. In this study, the return rate for booklets completed at home is not ideal (37% of non-compliant PROs, 224/609).
Langel et al reported that healthier populations perform better than sicker populations in regard to compliance in completing questionnaires, however, they also stress that reasons for successful questionnaire completion may be due to key study success factors rather than better indications.13 Our high compliance rates would suggest that while illness will always be a potential barrier to patient participation, solid design and implementation practices can blunt this effect as suggested by Langel. Compliance for very ill patients may actually, in some situations, be superior as it is recognized that they get the needed assistance in completing assessments. Specific suggestions from Langel are to have user-focused PRO design, have efficient compliance management tools such as online reports and email notifications, and have an active study team who will utilize the monitoring tools to increase compliance.
Further, our results are similar to those reported by Ross and O’Gorman as part of a literature review on the usage of electronic PROs (ePROs).14 They indicated that patients would prefer to have simple directions and be trained. They identified reminder issues as main reason for non-compliance and that 80% of responders preferred email and 55% preferred text for reminders to complete ePRO entries.
Problem areas that do not have simple solutions are the cases where staff were unable to contact patients to remind them to send in completed booklets (8 (1%) of the 135 unknown reasons for at-home booklet not returned) or where the patient claimed the booklet was lost, thrown away or not completed (30 (14%) of the 223 not returned booklets). These issues contribute to the inherent reality of the existence of missing data as a result of conducting any clinical trial. The use of modern technology for data capture may reduce the frequency these issues. E-mail, cell phones, texting and multimedia applications provide multiple communication opportunities for the patient to make inquiries of the medical staff and for the medical staff to encourage the patient to complete PROs. PROs may be completed via tablets, iPads, cell phones, and computers after receiving a reminder and a link to an on-line survey (eg REDCap15 or Qualtriacs16). Basch et al have shown that success may be achieved in utilizing tablet computers to record patient-reported symptoms between visits.17 Modern technology can help, but it has its own issues. It does reduce physical communication and interaction between the patient and the medical staff. And the issue of staff engagement remains vital. If the PROs are to be completed at the clinic visit, the staff must give the device to the patient and there needs to be a mechanism in place to deal with technological problems. Non-compliance issues could occur just as easily as with paper. Finally, in a study presently accruing (Sloan R01 CA154537-4) comparing paper to computer to an interactive voice response system, it has been found that people are individualistic in their preference of assessment modality (unpublished data). In fact, some patients and staff simply prefer paper and enjoy a break from being on a computer.
Conclusion
We have been successful in identifying and resolving issues around patient compliance by utilizing booklets as a professional way to present the PROs to the patient, by explaining our expectations for booklet completion to the patient and providing clear instructions in the protocols on how to administer the booklets, but improvements are always possible. Staff commitment and education regarding PRO completion are vital to successful collection of data. Patients do want to provide their perspective, and are typically noncompliant only when the task at hand is burdensome, unclear or logistically challenging. Existing tracking systems used for the other trial outcomes should be utilized to ensure successful capture of PROs.
Acknowledgments
Funding: This study was supported, in part, by grants from the National Cancer Institute (CA31946) to the Alliance for Clinical Trials in Oncology (Monica M. Bertagnolli, M.D., Chair) and to the Alliance Statistics and Data Center (Daniel J. Sargent, Ph.D.) (CA33601), the North Central Cancer Treatment Group (NCCTG) grant (CA025224), the American College of Surgeons Oncology Group (ACOSOG) grant (U10CA 149950), and the National Cancer Institution Community Clinical Oncology Program (CCOP) grant (CA37404). The content of this manuscript is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute.
Abbreviations
- PRO
Patient Reported Outcome
- ePRO
Electronic Patient Reported Outcome
Footnotes
Declaration of conflicting interests
There are no conflicts of interest to disclose for any of the authors.
References
- 1.Washington EA, Lipstein SH. The Patient-Centered Outcomes Research Institute — Promoting better information, decisions, and health. N Engl J Med. 2011;365:e31. doi: 10.1056/NEJMp1109407. [DOI] [PubMed] [Google Scholar]
- 2.Halyard MY, Frost MH, Dueck A, et al. Integrating QOL assessment for clinical and research purposes. Curr Probl Cancer. 2006;30:319–330. doi: 10.1016/j.currproblcancer.2006.08.009. [DOI] [PubMed] [Google Scholar]
- 3.Ibrahim JG, Chu H, Chen MH. Missing data in clinical studies: issues and methods. J Clin Oncol. 2012;30:3297–3303. doi: 10.1200/JCO.2011.38.7589. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Snyder CF, Aaronson NK, Choucair AK, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 2012;21:1305–1314. doi: 10.1007/s11136-011-0054-x. [DOI] [PubMed] [Google Scholar]
- 5.Bernier J. Head and Neck Cancer: Multimodality Management. Springer Science & Business Media; [2011, accessed 21 July 2015]. http://www.springer.com/us/book/9781441994639. [Google Scholar]
- 6.Osoba D, Zee B. Completion rates in health-related quality-of-life assessment: approach of the National Cancer Institute of Canada Clinical Trials Group. Stat Med. 1998;17:603–612. doi: 10.1002/(sici)1097-0258(19980315/15)17:5/7<603::aid-sim807>3.0.co;2-d. [DOI] [PubMed] [Google Scholar]
- 7.Walker M, Brown J, Brown K, et al. Practical problems with the collection and interpretation of serial quality of life assessments in patients with malignant glioma. J Neurooncol. 2003;63:179–186. doi: 10.1023/a:1023900802254. [DOI] [PubMed] [Google Scholar]
- 8.Cuffe S, Hon H, Tobros K, et al. Cancer Patients’ acceptability of incorporating an epidemiology questionnaire within a clinical trial: a patient preference study and subanalysis of the NCIC clinical trials group HN.6 clinical trial. J Clin Oncol. 2014;32:5s. doi: 10.1177/1740774514568689. (suppl; abstr 1586). ( http://meetinglibrary.asco.org/content/127556-144) [DOI] [PubMed] [Google Scholar]
- 9.Qi Y, Schild SE, Mandrekar SJ, et al. Pretreatment quality of life is an independent prognostic factor for overall survival in patients with advanced stage non-small cell lung cancer. J Thorac Oncol. 2009;4:1075–1082. doi: 10.1097/JTO.0b013e3181ae27f5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Rolstad S, Adler J, Ryden A. Response burden and questionnaire length: is shorter better? A review and meta-analysis. Value Health. 2011;14:1101–1108. doi: 10.1016/j.jval.2011.06.003. [DOI] [PubMed] [Google Scholar]
- 11.Sloan JA, Berk L, Roscoe J, et al. Integrating patient-reported outcomes into cancer symptom management clinical trials supported by the National Cancer Institute-sponsored clinical trials networks. J Clin Oncol. 2007;25:5070–5077. doi: 10.1200/JCO.2007.12.7670. [DOI] [PubMed] [Google Scholar]
- 12.Kropf ME, Blair J. Eliciting survey cooperation: incentives, self-interest, and norms of cooperation. Eval Review. 2005;29:559–575. doi: 10.1177/0193841X05278770. [DOI] [PubMed] [Google Scholar]
- 13.Langel K. Patient reported outcomes: A key to compliance success factors. J for Patient Compliance. 2011;1:12. [Google Scholar]
- 14.Ross J, O’Gorman H. Addressing the patient factor in ePRO to improve compliance. J for Patient Compliance. 2011;1:34–37. [Google Scholar]
- 15.Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Qualtrics. [2016, last accessed 29 April 2016]; https://www.qualtrics.com/ [Google Scholar]
- 17.Basch E, Deal AM, Kris MG, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized control trial. J Clin Oncol. 2016;34:557–565. doi: 10.1200/JCO.2015.63.0830. [DOI] [PMC free article] [PubMed] [Google Scholar]