Abstract
Background
Data about factors driving accrual to radiation oncology trials are limited. In oncology, 30%–40% of trials are considered unsuccessful, many because of poor accrual. The goal of the present study was to inform the design of future trials by evaluating the effects of institutional, clinician, and patient factors on accrual rates to a randomized radiation oncology trial.
Methods
Investigators participating in sabr-comet (NCT01446744), a randomized phase ii trial open in Canada, Europe, and Australia that is evaluating the role of stereotactic ablative radiotherapy (sabr) in oligometastatic disease, were invited to complete a survey about factors affecting accrual. Institutional ethics approval was obtained. The primary endpoint was the annual accrual rate per institution. Univariable and multivariable linear regression analyses were used to identify factors predictive of annual accrual rates.
Results
On univariable linear regression analysis, off-trial availability of sabr (p = 0.014) and equipoise of the referring physician (p = 0.014) were found to be predictive of annual accrual rates. The annual accrual rates were lower when centres offered sabr for oligometastases off-trial (median: 3.7 patients vs. 8.4 patients enrolled) and when referring physicians felt that, compared with having equipoise, sabr was beneficial (median: 4.8 patients vs. 8.4 patients enrolled). Multivariable analysis identified perceived level of equipoise of the referring physician to be predictive of the annual accrual rate (p = 0.023).
Conclusions
The level of equipoise of referring physicians might play a key role in accrual to radiation oncology randomized controlled trials. Efforts to communicate with and educate referring physicians might therefore be beneficial for improving trial accrual rates.
Keywords: Oligometastases, stereotactic radiotherapy, clinical trials, accrual
INTRODUCTION
Randomized controlled trials (rcts) are the “gold standard” of evidence-based medicine, but many rcts fail to accrue. In oncology specifically, estimates suggest that 30%–40% of oncology trials are unsuccessful1,2. From a health care perspective, several negative consequences ensue from failed clinical trials, including waste of money, institutional resources, and time. From a patient perspective, failed trials might be even more consequential: patients who enrolled in failed trials were exposed to some risk (by potentially undergoing unproven treatments), and yet the information gained from their participation cannot benefit future patients because the study remains incomplete.
The evaluation of new technologies within the fields of radiation oncology and oncologic surgery encounters unique challenges inherent to conducting rcts3,4. Within the field of medical oncology, investigators and patients are usually blinded to treatment allocation in trials of new systemic therapies, and they continue to be unaware of the treatment arm after randomization. That approach is considered ethically acceptable, because it allows for a fair assessment of the new therapy, and in theory, either treatment arm (that is, drug or placebo) could be better. In contrast, most surgical and radiation trials are open-label, with physicians and patients both being aware of treatment allocation. In many cases, these technologically innovative treatments can also be available off-trial. If new interventions are presumed by patients or physicians to be beneficial, then the combination of an open-label design with off-trial availability of the intervention could lead to unwillingness to pursue randomization, or trial drop-out with pursuit of the off-trial intervention upon randomization to a standard treatment arm. Currently, very little is known about the factors that drive accrual to trials of new interventions in radiation oncology, and yet such information could be useful in allowing investigators to design trials with a higher likelihood of success.
One example of a new technology being evaluated in clinical trials is stereotactic ablative radiotherapy (sabr) for oligometastatic disease. In the United States and world-wide, sabr has been widely adopted for the treatment of oligometastases5,6, but to our knowledge, no randomized evidence has yet indicated that sabr improves overall survival in this patient population. Stereotactic Radiotherapy for the Comprehensive Treatment of Oligometastatic Tumors (sabr-comet) is an international trial assessing the effect of sabr on overall survival. The trial recently completed accrual, with diverse accrual rates at the various participating institutions. The goal of the present study was to better inform the design of future radiation oncology trials by evaluating the effect of institutional factors and participant and patient beliefs on accrual rates.
METHODS
Study Design
In the present study, we surveyed investigators participating in the randomized phase ii trial sabr-comet (NCT01446744), whose design has been published7. In brief, patients with up to 5 metastatic lesions from a controlled primary solid tumour were randomized to either standard-of-care treatment (that is, palliative systemic therapy or radiotherapy, or both, at the discretion of the treating physicians) or to sabr to all sites of known metastatic disease, with palliative systemic therapy at the discretion of the treating physicians. The primary endpoint of the study was overall survival, and secondary endpoints included quality of life, toxicity, progression-free survival, lesion control rate, and number of cycles of further chemotherapy or systemic therapy. Patients were accrued in 4 countries between February 2012 and August 2016, reaching a total sample size of 99 patients.
Questionnaire
To determine the factors affecting accrual to the sabr-comet trial, a questionnaire was developed by the study’s principal investigators. The questionnaire was not part of the original sabr-comet protocol, and therefore separate institutional ethics approval was obtained for the project. The self-administered, Web-based survey (Table i) consisted of 10 questions related to physician, patient, and institution factors that were hypothesized to affect study accrual, including institutional experience with sabr, off-trial availability, and levels of equipoise on the part of patients, treating physicians, and referring physicians. Questions pertaining to levels of equipoise were designed on a scale from −5 to +5, with −5 indicating a strong belief of harm, +5 indicating a strong belief of benefit, and 0 indicating a neutral belief.
TABLE I.
The SABR–COMET accrual survey
| 1. | Please enter the name of your institution | |||||||||||
| ____________________________________________________________________________________________________________________________________________________________________________________________ | ||||||||||||
| 2. | For how many years has your centre been treating extracranial oligometastases with stereotactic radiation? | |||||||||||
| ____________________________________________________________________________________________________________________________________________________________________________________________ | ||||||||||||
| 3. | At your centre, can SABR for oligometastases be delivered off-trial? | |||||||||||
| □ Yes □ No | ||||||||||||
| 4. | How many clinicians in your department can treat patients with SABR for oligometastases? | |||||||||||
| ____________________________________________________________________________________________________________________________________________________________________________________________ | ||||||||||||
| 5. | Does your centre have a uniform policy regarding treatment of extracranial oligometastases? | |||||||||||
| ____________________________________________________________________________________________________________________________________________________________________________________________ | ||||||||||||
| 6. | In the rows below, please indicate the levels of equipoise regarding the use of SABR for oligometastatic disease who would be candidates for COMET. | |||||||||||
|
| ||||||||||||
| SABR is harmful for patients −5 | −4 | −3 | −2 | −1 | SABR is neither harmful nor beneficial 0 | +1 | +2 | +3 | +4 | SABR is beneficial for patients +5 | ||
|
| ||||||||||||
| Your own beliefs | □ | □ | □ | □ | □ | □ | □ | □ | □ | □ | □ | |
| The beliefs of radiation oncologists at your centre | □ | □ | □ | □ | □ | □ | □ | □ | □ | □ | □ | |
| The beliefs of physicians who refer to you | □ | □ | □ | □ | □ | □ | □ | □ | □ | □ | □ | |
|
| ||||||||||||
| 7. | Are there differences between tumour groups in their equipoise in randomizing patients to COMET? | |||||||||||
| □ Yes □ No | ||||||||||||
| If yes: In your opinion, which tumour groups feel that SABR is beneficial? | ||||||||||||
| ____________________________________________________________________________________________________________________________________________________________________________________________ | ||||||||||||
| 8. | In patients whom you have approached for entry into the trial, including both patients who have enrolled and those who have declined, what is your perception of their treatment preferences? | |||||||||||
|
| ||||||||||||
| Strongly prefer to receive palliative measures only −5 | −4 | −3 | −2 | −1 | No preference 0 | +1 | +2 | +3 | +4 | Strongly prefer SABR +5 | ||
|
| ||||||||||||
| Patient views toward SABR | □ | □ | □ | □ | □ | □ | □ | □ | □ | □ | □ | |
|
| ||||||||||||
| 9. | In general at your centre, if a patient undergoes SABR for 1–5 oligometastases, as opposed to palliative radiation or observation, does the remuneration to the radiation oncologist increase? | |||||||||||
| □ Yes □ No | ||||||||||||
| 10. | Are there any other factors that you feel have influenced the ability to accrue to COMET (making accrual easier or harder)? | |||||||||||
| ____________________________________________________________________________________________________________________________________________________________________________________________ | ||||||||||||
The questionnaire was distributed to local principal investigators at the 10 participating centres, with instructions to circulate the questionnaire to any colleagues eligible to accrue patients to the trial. The questionnaire was sent to investigators 3 months before the trial completed accrual.
Statistical Analyses
The primary endpoint was the annual accrual rate per institution, defined as the average number of patients accrued to the trial per year at a specific institution, during the time that the trial was open to accruals at that institution.
Descriptive statistics were generated to describe survey responses. Univariable linear regression was used to identify factors predictive of the annual institutional accrual rate. To allow for equal weighting for each institution despite a variable number of responses per institution, all regression models were inversely weighted by the number of participants from each institution. Variables with p values less than 0.10 in the univariable linear regression were selected for multivariable linear regression modelling that used forward stepwise techniques by sequentially adding variables beginning with the most significant predictors (lowest p values) and continuing until all eligible variables were considered. Variables with p values exceeding 0.10 after addition to the model were removed in the subsequent step before a new variable was added. All statistical analyses were performed using the SAS software application (version 9.4: SAS institute, Cary, NC, U.S.A.), with two-sided statistical testing at the 0.05 significance level.
RESULTS
The 30 responses that were received from the 10 centres participating in the trial included at least 1 respondent from each centre (range: 1–8 respondents per centre). All respondents were radiation oncologists or clinical oncologists. One institution was not included in any of the analyses, because at that institution, the trial opened only 2 months before overall trial completion, and the timeframe was considered too short to accrue any patients regardless of patient or physician motivations.
Institution Factors
Table ii summarizes the survey results. Physicians reported that their centres had been treating oligometastases with sabr for an average of 4.4 ± 3.0 years. Almost three quarters of respondents (n = 22, 73%) indicated that sabr was offered off-trial at their centre, and only a few (n = 9, 30%) reported that their centre had a uniform policy about the treatment of extracranial oligometastases. In most centres (77%), remuneration to the radiation oncologist did not increase if a patient were to undergo sabr.
TABLE II.
Questionnaire results
| Question | Respondents (n) | Results | |
|---|---|---|---|
|
| |||
| n (%) | Median (min, max) | ||
| Years treating extracranial oligometastases with SABR | 29 | 5.0 (1.0, 13.0) | |
| SABR for oligometastases delivered off-trial | 30 | 22 (73.3) | |
| Clinicians in department treating oligometastases with SABR | 29 | 5 (1, 11) | |
| Standard institutional policy for treating extracranial oligometastases? | 30 | 9 (30.0) | |
| Equipoisea: Own beliefs | |||
| −5 to −1 (“SABR is harmful for patients”) | 29 | — | |
| 0 (“SABR is neither harmful nor beneficial”) | 4 (13.8) | ||
| +1 to +5 (“SABR is beneficial for patients”) | 25 (86.2) | ||
| Equipoisea: Beliefs at institution | |||
| −5 to −1 (“SABR is harmful for patients”) | 29 | 2 (6.9) | |
| 0 (“SABR is neither harmful nor beneficial”) | 6 (20.7) | ||
| +1 to +5 (“SABR is beneficial for patients”) | 21 (72.4) | ||
| Equipoise a: Referring physician’s beliefs | |||
| −5 to −1 (“SABR is harmful for patients”) | 30 | 1 (3.3) | |
| 0 (“SABR is neither harmful nor beneficial”) | 5 (16.7) | ||
| +1 to +5 (“SABR is beneficial for patients”) | 24 (80.0) | ||
| Equipoise: Between tumour groups | 29 | 17 (58.6) | |
| Tumour groups viewing SABR as beneficial | 30 | ||
| Gastrointestinal | 16 (53.3) | ||
| Lung | 5 (16.7) | ||
| Renal | 5 (16.7) | ||
| Melanoma | 3 (10.0) | ||
| Sarcoma | 3 (10.0) | ||
| Breast | 2 (6.7) | ||
| Head and neck | 1 (3.3) | ||
| Pancreatic | 1 (3.3) | ||
| Testicular | 1 (3.3) | ||
| Patient SABR preferences | |||
| 0 (“No preference”) | 29 | 1 (3.5) | |
| 1 | 3 (10.3) | ||
| 2 | 4 (13.8) | ||
| 3 | 9 (31.0) | ||
| 4 | 6 (20.7) | ||
| 5 (“Strongly prefer SABR”) | 6 (20.7) | ||
| Remuneration increase for SABR for oligometastases | 30 | 7 (23.3) | |
Questions pertaining to levels of equipoise were designed on a scale from −5 (strong belief of harm) to +5, (strong belief of benefit), with 0 indicating neutral belief.
SABR = stereotactic ablative radiotherapy.
Level of Equipoise
With respect to the use of sabr for oligometastatic disease, respondents were asked about their personal level of equipoise, the level of equipoise of radiation oncologists at their institution, and finally, the level of equipoise of referring physicians. As Figure 1 shows, most respondents endorsed a belief that the use of sabr was beneficial and reported that other radiation oncologists and referring physicians largely considered sabr beneficial. Only a very few responses indicated a neutral belief or a belief that sabr could be harmful. Nearly all respondents (97%) indicated that their patients had a preference to be treated with sabr.
FIGURE 1.
Reported levels of equipoise for (A) radiation oncologists, (B) institutions, and (C) referring physicians.
Physicians were asked to enumerate the tumour groups within their centre whose beliefs were most likely to support the use of sabr for oligometastatic disease. Gastrointestinal tumour groups (53%) were most commonly reported to support the use of sabr, followed by those concerned with primary cancers of lung (17%) and kidney (17%) and with sarcoma (10%) and melanoma (10%).
Univariable and Multivariable Analysis
On univariable linear regression analysis (Table iii), the off-trial availability of sabr for oligometastases (p = 0.014) and the perceived equipoise of the referring physician (p = 0.014) were found to be predictive of annual accrual rates. The annual accrual rate was lower when sabr was offered off-trial for oligometastases in the centre (median: 3.7 patients enrolled when sabr was available off-trial vs. 8.4 enrolled when sabr was not available off-trial) and when referring physicians were perceived to believe that sabr was beneficial (median: 4.8 patients enrolled when referring physicians perceived benefit vs. 8.4 enrolled when referring physicians had equipoise). On multivariable analysis, the perceived level equipoise of the referring physician was the only factor that remained predictive of the annual accrual rate (p = 0.023).
TABLE III.
Results of the univariable analysis
| Characteristic | Estimate | p Value | Patients/year | |
|---|---|---|---|---|
|
| ||||
| Median | Range | |||
| Years treating extracranial oligometastases with SABR | −0.34 | 0.080 | — | — |
| SABR for oligometastases delivered off–trial | ||||
| No | 4.94 | 0.014 | 8.42 | (4.80, 8.42) |
| Yes | 0.00 | 3.74 | (0.00, 10.70) | |
| Clinicians in department treating oligometastases with SABR | −0.37 | 0.187 | — | — |
| Institutional policy for extracranial oligometastases | ||||
| No | ∓0.26 | 0.858 | 4.80 | (0.00, 10.70) |
| Yes | 0.00 | 4.80 | (1.24, 8.42) | |
| Equipoisea: Own beliefs | ||||
| −5 to −1 (“harmful”) | — | 0.058 | — | — |
| 0 (“neutral”) | 5.04 | 6.61 | (4.80, 10.70) | |
| +1 to +5 (“beneficial”) | 0.00 | 4.80 | (0.00, 10.70) | |
| Equipoisea: Beliefs at institution | ||||
| −5 to −1 (“harmful”) | 1.71 | 0.920 | NR | NR |
| 0 (“neutral”) | 0.26 | 4.80 | (1.05, 8.42) | |
| +1 to +5 (“beneficial”) | 0.00 | 4.80 | (0.00, 10.70) | |
| Equipoisea: Referring physician’s beliefs | ||||
| −5 to −1 (“harmful”) | −1.73 | 0.014 | NR | NR |
| 0 (“neutral”) | 6.34 | 8.42 | (8.42, 10.70) | |
| +1 to +5 (“beneficial”) | 0.00 | 4.80 | (0.00, 10.70) | |
| Equipoise: Between different tumour groups | ||||
| No | 0.01 | 0.995 | 3.54 | (0.00, 10.70) |
| Yes | 0.00 | 4.80 | (0.33, 10.70) | |
| Patient SABR preferences | −0.04 | 0.937 | — | — |
| Remuneration increase for SABR for 1–5 oligometastases | ||||
| No | 2.28 | 0.087 | 4.80 | (0.57, 10.70) |
| Yes | 0.00 | 1.05 | (0.00, 10.70) | |
Questions pertaining to levels of equipoise were designed on a scale from −5 (strong belief of harm) to +5, (strong belief of benefit), with 0 indicating neutral belief.
SABR = stereotactic ablative radiotherapy; NR = not reported (n < 3).
Qualitative Results
Participants were also given the opportunity to describe additional factors that influenced the ability to accrue to the trial. The factor most commonly cited as a barrier to accrual was the availability of sabr off-trial. Other themes that emerged were the trial inclusion criteria and the unwillingness of patients to be randomized.
DISCUSSION
Failure rates in radiation oncology rcts are high in the modern era, and analyses of factors influencing accrual could allow for improvements in study design for future trials. The analysis reported here suggests that the perceived equipoise of referring physicians is associated with trial accrual, a phenomenon that appears to have previously been unrecognized in the clinical trial literature. In four previously published systematic reviews reporting on factors affecting accrual, none identified the beliefs of the referring physician as an important predictor8–11.
The mechanisms underlying the relationship between the perceived equipoise of referring physicians and trial accrual are unclear. We hypothesize that they might relate to one of two phenomena:
-
■
The expectations of the patient are influenced by the initial discussions about sabr with the referring physician, affecting the likelihood that the patient will accept enrolment
-
■
The perceived expectations of the referring physician affect whether the radiation oncologist will offer the trial or merely treat off-trial
Our finding underlines the importance of fostering good medical relations between referring physicians and treating oncologists.
The results of the present study can help in proposing new strategies to potentially improve accrual to rcts in radiation oncology. First, although rcts are commonly advertised to referring centres, additional outreach and education for referring doctors that focuses on the uncertainty of the benefit of the experimental arm might be beneficial. Second, involvement of various specialties in the conception and implementation of radiation oncology trials might allow for more consistent equipoise among referring and treating physicians. Finally, multidisciplinary tumour board discussion of prospective patients would allow for continuous reassessment, discussion, and education about institution-level equipoise.
Although our study presents novel insights into the factors influencing clinical trial accrual, its results should be considered in the context of its limitations. First, the survey data were gathered from an unvalidated questionnaire; however, to our knowledge, no validated questionnaire for this purpose is available. Second, the sample size is small and was drawn only from academic centres, which could limit its generalizability. However, all institutions participating in sabr-comet provided at least 1 response. Given the limited number of responses from each centre, we are unable to comment on variability between the centres. Third, the survey was distributed to all accruing centres by e-mail, and we were unable to determine the exact number of physicians receiving the survey and therefore the true response rate. Finally, the questionnaire was completed only by radiation oncologists (or clinical oncologists within the United Kingdom), and the findings pertaining to patient and physician equipoise are based solely on the perceptions of the physicians completing the survey, which could be subject to recall bias. However, the fact that, in this study, perception of the level of equipoise of the referring physicians affected accrual highlights the importance of collaboration between referring and accruing physicians to optimize the success of clinical trials in oncology.
CONCLUSIONS
The level of equipoise of referring physicians might play a key role in accrual to radiation oncology rcts. Efforts to communicate with and educate referring physicians about the pros and cons of treatment might therefore be beneficial for improving accrual rates to clinical trials in radiation oncology.
CONFLICT OF INTEREST DISCLOSURES
We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare the following interests: SS has received research funding and a speaker’s honorarium from Varian Medical Systems and sits on the advisory board for Lilly Oncology. AVL has received a speaker’s honorarium from Varian Medical Systems. The remaining authors have no conflicts to disclose. No direct funding was available or used for this study.
REFERENCES
- 1.Schroen AT, Petroni GR, Wang H, et al. Achieving sufficient accrual to address the primary endpoint in phase iii clinical trials from U.S. Cooperative Oncology Groups. Clin Cancer Res. 2012;18:256–62. doi: 10.1158/1078-0432.CCR-11-1633. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Cheng SK, Dietrich MS, Dilts DM. A sense of urgency: evaluating the link between clinical trial development time and the accrual performance of cancer therapy evaluation program (nci-ctep) sponsored studies. Clin Cancer Res. 2010;16:5557–63. doi: 10.1158/1078-0432.CCR-10-0133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Potter S, Mills N, Cawthorn SJ, Donovan J, Blazeby JM. Time to be brave: is educating surgeons the key to unlocking the potential of randomised clinical trials in surgery? A qualitative study. Trials. 2014;15:80. doi: 10.1186/1745-6215-15-80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Rooshenas L, Elliott D, Wade J, et al. Conveying equipoise during recruitment for clinical trials: qualitative synthesis of clinicians’ practices across six randomised controlled trials. PLoS Med. 2016;13:e1002147. doi: 10.1371/journal.pmed.1002147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Pan H, Simpson DR, Mell LK, Mundt AJ, Lawson JD. A survey of stereotactic body radiotherapy use in the United States. Cancer. 2011;117:4566–72. doi: 10.1002/cncr.26067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lewis SL, Porceddu S, Nakamura N, et al. Definitive stereotactic body radiotherapy (sbrt) for extracranial oligometastases: an international survey of > 1000 radiation oncologists. Am J Clin Oncol. 2017;40:418–22. doi: 10.1097/COC.0000000000000169. [DOI] [PubMed] [Google Scholar]
- 7.Palma DA, Haasbeek CJ, Rodrigues GB, et al. Stereotactic ablative radiotherapy for comprehensive treatment of oligometastatic tumors (sabr-comet): study protocol for a randomized phase ii trial. BMC Cancer. 2012;12:305. doi: 10.1186/1471-2407-12-305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Tournoux C, Katsahian S, Chevret S, Levy V. Factors influencing inclusion of patients with malignancies in clinical trials. Cancer. 2006;106:258–70. doi: 10.1002/cncr.21613. [DOI] [PubMed] [Google Scholar]
- 9.Grand MM, O’Brien PC. Obstacles to participation in randomised cancer clinical trials: a systematic review of the literature. J Med Imaging Radiat Oncol. 2012;56:31–9. doi: 10.1111/j.1754-9485.2011.02337.x. [DOI] [PubMed] [Google Scholar]
- 10.Mills EJ, Seely D, Rachlis B, et al. Barriers to participation in clinical trials of cancer: a meta-analysis and systematic review of patient-reported factors. Lancet Oncol. 2006;7:141–8. doi: 10.1016/S1470-2045(06)70576-9. [DOI] [PubMed] [Google Scholar]
- 11.Townsley CA, Selby R, Siu LL. Systematic review of barriers to the recruitment of older patients with cancer onto clinical trials. J Clin Oncol. 2005;23:3112–24. doi: 10.1200/JCO.2005.00.141. [DOI] [PubMed] [Google Scholar]