Prospective review of radiotherapy trials through implementation of standardized multicentre workflow and IT infrastructure

Abstract

Objective:

We sought to develop a process that would allow us to perform a prospective review of outlining in trials using expert reviewers based in multiple centres.

Methods:

We implemented a specific information technology infrastructure and workflow that could serve all organizations involved in the radiotherapy quality assurance (RTQA) process.

Results:

Data were processed and packaged in the computational environment for radiotherapy research (CERR) binary format and securely transmitted to the expert reviewer at the designated remote organization. It was opened and reviewed using the distributed CERR-compiled application, and a standardized report was sent to the respective centre. Centres were expected to correct any unacceptable deviations and resubmit outlining for approval prior to commencing treatment. 75% of reviews were completed and fed back to centres within 3 working days. There were no delays in treatment start date.

Conclusion:

Our distributed RTQA review approach provides a method of prospective outlining review at multiple centres, without compromising the quality, delaying the start of treatment or the need for significant additional infrastructure resources. Future progress in the area of prospective individual case review will need to be supported by additional resources for clinician time to undertake the reviews.

Advances in knowledge:

Trial groups around the world have formulated different approaches to address the need for the prospective review of radiotherapy (RT) data with clinical trials, in line with available resources. We report a UK solution that has allowed the workload for outlining review to be distributed across a wider group of volunteer reviewers without the need for any additional infrastructure costs and has already been adopted within the UK RT trials community.

INTRODUCTION

It is now well established that radiotherapy (RT) trial outcomes are related to protocol compliance and quality of the delivered RT.^1–7 The detrimental effects of non-compliance to protocol can be minimized by a robust radiotherapy quality assurance (RTQA) programme.^3,8 UK RT trials are encouraged to undertake some form of pre-accrual assessment of outlining⁷ and planning, which can take the form of a benchmark case or a dummy run.⁹ While this approach is associated with improved protocol compliance,³ to ensure the ongoing quality of the RT delivery within the trial, depending on the complexity of the RT, it should be complemented by some form of on-trial assessment [individual case review (ICR)].¹⁰

The UK National Radiotherapy Trials Quality Assurance (RTTQA) group operates from four sites, ours being one of the host sites. Much of the ICR for trials supported by this group has been retrospective, in keeping with many other international trials. This does not allow protocol deviations to be identified and corrected prior to an individual patient commencing treatment. ARISTOTLE (http://www.ctc.ucl.ac.uk/TrialDetails.aspx?TrialID=48), a Phase III trial for locally advanced rectal cancer, was the first trial in our RTTQA centre to undertake prospective ICR (prior to start of the treatment), but it was reliant on a single clinician working in the centre to perform all the reviews. The NeoSCOPE¹¹ trial was reintroducing neoadjuvant chemoradiotherapy prior to oesophagectomy into the UK and there were concerns about increasing post-operative morbidity and mortality with this approach. In order to ensure that non-protocol-compliant RT was not the cause for any excess toxicity, prospective ICR of all cases up to the first toxicity assessment (after 20 patients had completed treatment) was undertaken. This necessitated a more sustainable approach to prospective ICR, allowing multiple reviewers not necessarily based at one of the RTTQA centres to participate in the process, without compromising on quality. Here, we report on how we achieved this.

METHODS AND MATERIALS

For the purpose of ICR, we implemented a specific information technology (IT) infrastructure and workflow that could serve all organizations involved in the RTQA process. The main requirements that we identified for the former are outlined in Table 1. The computational environment for radiotherapy research (CERR) met all of these requirements. It was custom built and validated for the purpose of analyzing and sharing RT data for research purposes¹² and our RTTQA group had previous experience of its use.¹³ A compiled version (a stand-alone version without the need for licence or specific IT infrastructure) was provided by the Database and IT Solutions subgroup of the UK RTTQA group (http://www.rttrialsqa.org.uk/rttqa/). It can be downloaded at a remote site within minutes and a user guide is also made available.

Table 1.

Summary of the main requirements for the UK radiotherapy trials quality assurance (RTTQA) workflow and software to be used for real-time review of multicentre radiotherapy trials

Workflow	Software
Comply with good clinical practice and data protection regulations Decentralized or hybrid data-pooling model Accessible and secure data transfer Network speed and reliability Validation of completeness and quality of data Standardization of data format and consistent quality review moving from a single to a multiple site basis Simplified process that meets prospective ICR requirements	Distributable to both RTTQA and non-RTTQA centres Operating system independent Stand-alone with no additional equipment required Read in digital data exported from multiple TPS and PACS (DICOM, DICOM-RT, RTOG formats) Easy to use with functionalities and tools similar to those found in TPS Centrally developed and maintained Locally managed (IT remote organization)

DICOM, digital imaging and communications in medicine; ICR, individual case review; IT, information technology; PACS, picture archiving and information system; RT, radiotherapy; RTOG, Radiation Therapy Oncology Group; TPS, treatment planning system.

RESULTS

The workflow process

In Step 1, all centres were asked to submit the outlining data (all target volumes including organs at risk) in digital imaging and communications in medicine (DICOM) format along with the relevant diagnostic information (CT, endoscopic ultrasound and positron emission tomography reports). The former was performed with the centre-specific treatment planning system. Data were anonymized at the treating centre, as per trial regulations and exported in DICOM format, and securely transmitted to the RTTQA centre. All transferred data were stored locally on a dedicated RTTQA drive.

Centres were encouraged to submit outlining as soon as it was available and given the option to either wait for feedback or proceed with the planning on the understanding that the outlines may need to be modified. In Step 2, anonymized DICOM data were received by our RTTQA centre and were checked for integrity and validated by the trial quality assurance (QA) contact. This consisted of ensuring that all data were complete and anonymized and could be processed in CERR, along with a visual check of the outlines and correlation with the plan assessment form. Data were then processed and packaged in the CERR¹² binary format and securely transmitted to the expert reviewer at the designated remote organization. In Step 3, the expert reviewer received the binary package, which was opened and reviewed using the distributed CERR-compiled application. The review was then carried out, a standardized report prepared and submitted to our RTTQA centre and sent to the respective centre. Dialogue between the reviewer and the centre was undertaken if an unacceptable deviation or a data query was identified. A detailed review of the planning was also undertaken but is beyond the scope of this article.

Use of the workflow in NeoSCOPE

The ICRs for outlining in this trial were undertaken by five upper gastrointestinal clinical oncologists (SG, TC, SM, MH, GR, all NeoSCOPE Trial Management Group members) on a rota basis, only one of whom was based at the RTTQA centre. Reviews for patients outlined using three-dimensional or four-dimensional CT (both allowed in the trial) were assigned according to the expertise of the reviewers. In order to minimize inter-reviewer variation, all of the reviewers had been part of the RT protocol development group and had undertaken the pre-accrual assessment process in their respective centres. A standardized pro forma for review and feedback was used, which detailed pre-agreed acceptable and unacceptable deviations for outlining.

Prospective ICR was undertaken for the first 20 patients recruited to the trial as described above, regardless of the recruiting centre and previous performance. However, as recruitment from the 15 participating centres was not at the same rate, prospective ICR was also extended to the first recruited case from each centre, repeating the process if there were any unacceptable deviations, until there was a satisfactory submission. Centres were expected to correct any unacceptable deviations and resubmit outlining for approval, prior to commencing treatment. The intention was to feed back to centres within 3 working days and this was achieved in 75% of reviews with a median of 2 days. There were no delays in treatment start date. Subsequent cases were subject to “timely retrospective review”, with review within 2 weeks (10 out of 25 fractions) of the start of RT. This approach was intended to minimize the burden of the initial rigorous QA requirements on departments which were performing well, but still allow identification of a significant error in sufficient time for correction to be made to at least half the number of remaining fractions. This target was achieved in 93% of reviews, with 40% before the start of treatment. 39 (47%) real-time and 44 (53%) timely retrospective reviews were undertaken. 9 cases required resubmission, 6 (67%) cases were real time and 3 (33%) were timely retrospective. The compiled version of CERR has now been made available to the other RTTQA centres for trials involving a range of anatomical sites.

DISCUSSION

Historically, most RTQA of gastrointestinal RT trials has been retrospective.² Given the recent evidence for the relationship between protocol deviations and poor outcome, we no longer believe it is acceptable to simply collect the data on deviations retrospectively, since there is no opportunity to correct prior to, or during the course of, the treatment or to provide feedback to the clinician involved. However, the move to prospective ICR is both labour and resource intensive.^14,15 If prospective ICR is to be performed in a timely manner and avoid delays in the patient treatment, feedback to centres is needed within 3 working days. The RTTQA group has limited funding to provide the expertise for outlining review and the four centres are not expected to offer high-level clinical expertise in outlining, which is viewed to be the responsibility of the chief investigator or nominated RT lead. In reality, the workload and timescales are too onerous for a single clinician in most cases and there has been an urgent need to develop a robust process to involve multiple reviewers, who would often be based outside of the four RTTQA centres.

The first trial to use prospective ICR in our RTTQA centre was ARISTOTLE. In this trial, a single clinician reviewed all the outlines and provided feedback to centres within 48 h. While feasible for this particular trial, as one of the lead members of the Trial Management Group was based at the RTTQA centre, this is not a model that can be applied to all trials and a move towards a multiclinician approach is recommended. We have been able to implement a distributed review process through the use of a secure, fast and reliable IT infrastructure and a distributed application specifically designed to review RT data. The application used in this work is platform independent, enabling the review to be conducted under the same conditions as if the review was conducted in the RTTQA centre.

Recently, Skripcak et al¹⁶ reported on challenges and possible solutions for the strategic development of international research data exchange framework in radiation therapy and oncology and identified three major classes of data-pooling models: centralized, decentralized and hybrid. Our approach used a decentralized model in which data are collected, validated and processed at the designated RTTQA centre by expert trial QA staff. These data are then securely transmitted to the review sites, as appropriate, in anonymized form. In this model, the applications needed to technically access the data and review the clinical case are distributed and installed at the review centre and managed by the local IT department, following local rules and security protocols. Furthermore, our approach minimizes possible interoperability issues between the different clinical IT solutions implemented in institutions participating in the trial. The procedure that we implemented ensures completeness and quality of the data submitted for review to individual centres because the information is processed, validated and standardized in a single file format that is checked for integrity before leaving the RTTQA centre. This also facilitates and speeds up the real-time review process.

Different approaches have been adopted to address the issue of prospective ICR.^{14,15,17–19} The European Organisation for Research and Treatment of Cancer have reported the development of a digital central review facility, where centres upload data through a secure website and then the reviewers, who may not be based at the central RTQA centre, are able to remotely access the data through a terminal server.²⁰ The International Society of Paediatric Oncology (Europe) high-risk neuroblastoma group have developed a web-based platform which allows remote uploading of radiology and RT-related data and images from the participating centres, allowing remote review by clinicians in a timely fashion, without the need to meet in one place at one time.¹⁹ In the USA, the leading QA centres serving the current National Cancer Institute clinical trial programme have been brought together to form a single organization administered by the American College of Radiology Clinical Research Centre in Philadelphia. This new organization will be known as the Imaging and Radiation Oncology Core Group.¹⁸ Real-time reviews are conducted by a faculty from the University of Massachusetts. Study chairs wishing to perform retrospective reviews have a secure virtual private network connection to the database and can access the information (T Fitzgerald, Quality Assurance Review Centre, University of Massachusets, personal communication). SWAN¹⁷ was developed in Australia with the aim of facilitating objective analysis, QA and review of digital treatment planning data from Trans Tasman Radiation Oncology Group multicentre RT trials. It is utilized for central review, pre-accrual benchmark cases and credentialing. It has unique properties in that it can perform automatic reviews and reporting which allows specific fields in a data export to be examined for adherence to protocol criteria. While not removing the need for manual review, it can reduce the time needed for review by identifying errors in data submission prior to being sent to a clinician for review.

The trend is for trials to become more complex and more costly.¹⁴ Continued progress requires dedicated resources. Plan reviews and dosimetry audits have been supported by funding of dedicated physicist time, but across Europe, most of the outlining reviews are undertaken by clinicians (e.g. chief investigators or other members of the trial management group) on a voluntary basis or by clinical fellows supported by grants from various sources.¹⁴ The Radiation Therapy Oncology Group have shown that in four of their trials, including the most recent trial of intensity-modulated radiotherapy for anal cancer, the major deviations were inaccurate target volume delineation³ and additional resources are needed to ensure that we can continue to provide high-quality review of outlining for future trials.

CONCLUSION

Our distributed RTQA review approach provides a method of prospective outlining review at multiple centres, without compromising the quality, delaying start of treatment or need for significant additional resources. Future progress in the area of prospective ICR will need to be supported by additional resources for clinician time to undertake the reviews.

FUNDING

ARISTOTLE and NeoSCOPE trials were funded by the Cancer Research UK project grants CRUK/08/032 and C44694/A14614, respectively.