Clinical Audit of the Radiotherapy Process in Rectal Cancer: Clinical Practice Guidelines and Quality Certification Do Not Avert Variability in Clinical Practice

Introduction

The therapeutic approach to cancer is complex and multidisciplinary [1], [2], [3], and radiotherapy is among the essential treatments, whether used alone or in conjunction with other therapies (e.g., chemotherapy, surgery). About 50% of cancer patients need radiotherapy as part of their treatment plan [4], according to scientific evidence. To guarantee its quality, safety, and effectiveness, providers must apply strict quality control measures during administration of the therapy to minimize risks to the patient. Ideally, there should be monitoring systems in place throughout the radiation oncology process in order to check critical points with regard to the clinical appropriateness, safety, and effectiveness of treatments with ionizing radiation. Numerous scientific societies and organizations specifically recommend conducting audits in these settings as a way to evaluate these quality criteria [5], [6]. Moreover, variability in clinical practice [7] is a relevant factor that can affect patients’ clinical results.

Some authors have proposed general indicators to assess the quality of the radiotherapeutic process [8], [9] or specific ones related to rectal cancer [10], and recently, the International Consortium for Health Outcomes Measurement [11] has also put forward a set of indicators focused on the impact of treatment and quality of life in patients with colorectal cancer. However, for the most part, there are no available indicators with international validity that could serve as a reference for comparing radiation oncology services.

This study aims to design a clinical audit model that defines a set of specific indicators for each of the four priority pathologies (cancers of the rectum, prostate, lung, and cervix) that enables the assessment of key areas of the treatment process: diagnosis, treatment, posttreatment follow-up, and clinical outcomes. In this way, we intended to obtain information on the clinical efficacy of the treatments, evaluate adherence to agreed protocols, and measure the variability among the three participating departments of radiation oncology; our findings will inform the development of objectives to improve therapeutic quality based on mutual learning. This paper presents the results obtained for rectal cancer.

Methods

Study Framework

This study took place in the Catalan Institute of Oncology (ICO in its Catalan abbreviation), an organization specialized in cancer care, with three centers offering radiation oncology treatments in cooperation with three university hospitals. Additionally, ICO works in a network with 17 local hospitals, making it the reference center in cancer care for 2.5 million people (40% of the population of Catalonia). Its care model is characterized by its multidisciplinary focus by pathology and the standardized care based on clinical practice guidelines, developed through corporate consensus. The three radiation oncology departments are ISO-certified (2003, 2005, and 2008). At the time of the study, clinical practice guidelines for colorectal cancer were in place (approved in December 2012), with two radiotherapy treatments detailed: short-course radiotherapy (SCRT), indicated for patients aged 70 years or older and those with tumors in the middle third of the rectum or without involvement of the mesorectal fascia, and long-course radiotherapy (LCRT), indicated for all other cases.

Design

This is a multicenter retrospective cohort study in a representative sample of patients diagnosed with rectal cancer. To design the study, we formed a working group of radiation oncologists, medical physicists, and technologists from the three participating centers, led by the senior clinician in rectal cancer. We identified the key areas for assessment, including the diagnostic, treatment, and follow-up phase, based on the reviewed literature [8], [9] and a previous study that ICO performed in collaboration with the Wielkospolskie Centrum Onkologii group [10].

The set of indicators we developed assessed the appropriateness of the tests performed and the quality of the diagnostic reports, particularly the notation of the distance from the tumor to the mesorectal fascia (rCRM), based on high-resolution magnetic resonance imaging (MRI). Other indicators included the presentation of cases to a multidisciplinary committee to validate the treatment, appropriateness and degree of adherence to the prescribed treatment (dose and duration of radiotherapy), number of imaging verifications conducted during the radiotherapy sessions, the postoperative circumferential resection margin (ypCRM), the clinical follow-up carried out in the center, adverse effects recorded over the course of treatment [and whether these were classified according to the Common Terminology Criteria for Adverse Events (CTCAE) version 4.0] [12], and overall survival at 4 years.

Clinical Audit

The second phase of the study was the clinical audit, which consisted of a review of a random sample of 40 clinical histories for each center. The target population for the study was patients diagnosed with rectal cancer (RE: CIM-9:154.1) who in 2013 received neoadjuvant radiotherapy with a curative intent. The audit took place from June to September of 2015 and was carried out by evaluators who were not affiliated with the participating centers in order to avoid bias in interpreting the information contained in the clinical histories and to maintain consistency in the data collection. The form used to collect the data was prepiloted to test its validity. The source of the data consisted of the electronic medical records used in the centers throughout the care pathway and which are available to all professionals working in all centers. The specific information system for radiotherapy was also accessed to complete the prespecified data collection process. To calculate overall survival, in February 2017, we updated our data on the vital status of all patients included in the study. To complete the audit and with the aim of assessing the overall functioning of each center, we collected additional information on organizational indicators (existence of committees and clinical sessions), quality certification systems, availability of consensus-based protocols, research and care activity, available equipment, perceived quality, and patient satisfaction. We obtained this information from corporate information systems and meetings between the auditing team and the participating services.

We performed the statistical analysis during the fourth quarter of 2015 using SPSS statistical software (IBM, Armonk, NY). Sample size was calculated to achieve a level of confidence of 0.95 (alpha 0.05), assuming a finite population of about 100 cases per center and an attrition rate of 50%. To compare results between centers, we used the χ² test for qualitative variables and the ANOVA test for qualitative variables. To calculate survival, we used the Cox regression model, adjusted for center, sex, age group, and tumor stage at diagnosis.

Results

Table 1 presents the profile of overall functioning, the organization and protocolization of care, and research and treatment activity in each center. The clinical characteristics and results for the indicators in the diagnostic stage are described in Table 2, while Table 3 shows those for the treatment and follow-up phases. The demographic profile, histology, and staging of patients were similar, and we did not observe any significant differences between centers. Overall, 71.7% of patients were men and 28.3% women, and the mean age was 67.23 years. Virtually all patients (99.2%) had an invasive carcinoma, and 78.2% a stage III tumor. For locoregional staging, 95% of the patients underwent an MRI, with no significant differences between centers, although there were qualitative differences in the clinical information recorded in the corresponding report. One center recorded the distance from the tumor to the anal verge in 97.5% of the cases compared to 85% and 22.5% in the other centers. The reports did not include the rCRM in 38.3% of the cases; by center, this value was recorded in 82.5%, 57.5%, and 45% of the reports. Ninety percent of the patients received LCRT, while just 12 patients (10%) were treated with SCRT.

Table 1.

General Indicators of Center Operation (2014 Data)

	Center A	Center B	Center C
Quality management systems
ISO standard 9001 (EBRT, BQT, clinical dosimetry, physical dosimetry processes)	2003-2016	2008-2016	2005-2016
Care protocols and organization
Existence of a multidisciplinary committee to agree on therapeutic approach	Yes	Yes	Yes
Number of members in multidisciplinary committee	Variable between 3 centers
Existence of a clinical session to evaluate the indication for radiotherapy	Yes	Yes	Yes
Number of members in clinical session	Variable between 3 centers
Existence of clinical practice guidelines by pathology	Yes, unified between centers
Existence of a protocol for imaging verification	Yes, partially. Not unified between centers.
Existence of a protocol for treatment interruptions	Yes, partially. Not unified between centers.
Perceived quality
Overall satisfaction with care received is ≥8/10	92.14%	91.85%	92.55%
Center operation perceived as perfect or very good	94.25%	83.3%	84.58%
Professionals (do not include brachytherapy)
Ratio external beam radiotherapy treatments per radiation oncologist	188.07	169.33	191.86
Equipment (does not include brachytherapy)
Number of accelerators (monoenergetic + multienergetic)	6	3	3
Care activity (does not include brachytherapy)
External beam radiotherapy treatments	2539	1524	1343
Percentage of complex treatments	18.6%	7.1%	13.8%
Brachytherapy	986
Radiosurgery	119
Scientific activity
Impact factor (articles published per year)	13,624.00	3607.00	2635
Percentage of patients included in clinical trials with participation of RT and/or medical physics service(s)	1.41%	0.6%	0.01%

EBRT, external beam radiotherapy; BQT, brachytherapy.

Table 2.

Characteristics of Cases Reviewed and Quality Indicators (2013)^a†

Rectal Cancer		Center A		Center B		Center C		Total ICO
		n	%	n	%	n	%	n	%	P
Total cases included		40	33.3%	40	33.3%	40	33.3%	120	100%
Sex	Men	30	75.0	29	72.5	27	67.5	86	71.7	0.750
	Women	10	25.0	11	27.5	13	32.5	34	28.3
Age (years)	<60	28	23.3	6	15.0	9	22.5	13	32.5	0.532
	60-69	42	35.0	15	37.5	16	40.0	11	27.5
	70-79	40	33.3	14	35.0	13	32.5	13	32.5
	≥80	10	8.3	5	12.5	2	5.0	3	7.5
Histology	Squamous cell carcinoma	1	2.5	·	·	·	·	1	.8	0.365
	Invasive adenocarcinoma	39	97.5	40	100	40	100	119	99.2
Histological grade	Well differentiated	17	42.5	9	22.5	17	42.5	43	35.8	0.005*
	Moderately differentiated	10	25.0	3	7.5	7	17.5	20	16.7
	Poorly differentiated	3	7.5	1	2.5	·	·	4	3.3
	Missing†	10	25.0	27	67.5	16	40.0	53	44.2
Percentage of patients undergoing MRI	Yes	36	90.0	38	95.0	40	100.0	114	95.0
	No	4	10.0	2	5.0			6	5.0	0.122
Distance from tumor to anal verge	0-5 cm	9	22.5	2	5.0	13	32.5	24	20.0	<0.001*
	5-10 cm	19	47.5	7	17.5	15	37.5	41	34.2
	>10 cm	6	15.0			11	27.5	17	14.2
	Missing†	6	15.0	31	77.5	1	2.5	38	31.7
Distance from tumor to mesorectal fascia (mm) (rCRM)	≤1 mm	6	15.0	9	22.5	14	35.0	29	24.2	0.013*
	>1 mm	12	30.0	14	35.0	19	47.5	45	37.5
	Missing†	22	55.0	17	42.5	7	17.5	46	38.3
TNM staging⁎ (presurgical)	II	3	7.7	6	15.0	5	12.5	14	11.8	0.666
	III	33	84.6	29	72.5	31	77.5	93	78.2
	IV	3	7.7	5	12.5	4	10.0	12	10.1
	Missing†	1						1
Percentage of patients diagnosed in ICO	Yes	5	12.5	29	72.5	10	25	44	36.7	<0.001*
	No	31	77.5	10	25.0	30	75.0	71	59.2
	Missing†	4	10.0	1	2.5	0	0.0	5	4.2
Percentage of cases presented in multidisciplinary committee to evaluate initial treatment plan	Yes	19	47.5	29	72.5	28	70.0	76	63.3
	No	20	50.0	11	27.5	12	30.0	43	35.8	0.1
	Missing†	1	2.5					1	.8
Percentage of patients diagnosed at ICO and presented at multidisciplinary committee to evaluate initial treatment plan	Yes	4	80.0	25	86.2	10	100.0	39	88.6
	No	1	20.0	4	13.8			5	11.4	0.402
Percentage of patients presented at clinical session in radiation oncology service prior to initiating RT	Yes	36	90.0	37	92.5	40	100.0	113	94.2
	No	2	5.0	3	7.5			5	4.2	0.134
	Missing†	2	5.0					2	1.7

^aTNM Classification of Malignant Tumours (UICC) (Five edition, 1997)

^†No data found in the medical records.

Table 3.

Treatment Phase and Oncological Follow-Up. Quality Indicators (2013)^g

Rectal Cancer		Center A		Center B		Center C		Total ICO
		n	%	n	%	n	%	n	%	P
Treatment phase
Distribution of patients according to treatment received	Chemo + concomitant RT + surgery	39	97.5	37	92.5	37	92.5	113	94.2	0.225
	RT + surgery	1	2.5	3	7.5	1	2.5	5	4.2
	RT + chemo					2	5.0	2	1.7
Type of radiotherapy	SCRT	8	20.0	4	10.0	·	·	12	10.0	0.012*
	LCRT	32	80.0	36	90.0	40	100	108	90.0
Total prescribed dose (Gy)	SCRT (25.00 Gy)	8	20.0	4	10.0	·	·	12	10.0	<0.001*
	LCRT (45.00 Gy)	·	·	36	90.0	·	·	36	30.0
	LCRT (50-54 Gy)	32	80	0	0	40	100	72	60
Treatment technique indicated	3D	37	92.5	40	100	40	100	117	97.5	0.188
	VMAT	1	2.5	·	·	·	·	1	.8
	IMRT	2	5.0	·	·	·	·	2	1.7
Percentage of patients completing planned treatment (dose)	Yes	39	97.5	38	95.0	39	97.5	116	96.7	0.772
	No	1	2.5	2	5.0	1	2.5	4	3.3
Percentage of patients suffering interruptions according to the days treatment was prolonged (LCRT)	<3 days	12	37.5	19	52.7	31	77.5	62	57.4	0.006*
	3-5 days	12	37.5	11	30.5	9	22.5	32	29.6
	6-10 days	8	25.0	3	8.3	0	0	11	10.2
	>10 days	0	0.0	3	8.3	0	0	3	2.8
Percentage of patients undergoing restaging MRI	Yes	8	20.0	0	0.0	31	77.5	39	32.5	0.001*
	No	32	80.0	40	100.0	9	22.5	81	67.5
Percentage of patients with indication for surgical treatment following primary treatment (chemo and/or RT)	Yes	39	97.5	40	100.0	37	92.5	116	96.7	0.087
	No					3	7.5	3	2.5
	Missing⁎	1	2.5					1	.8
Percentage of patients according to radial margin result (ypCRM)	Positive	1	2.6	1	2.5	3	8.1	5	4.3	0.001*
	Negative	28	71.8	15	37.5	28	75.7	71	61.2
	Missing⁎	10	25.6	24	60.0	6	16.2	40	34.5
Percentage of patients undergoing surgery in ICO who had been previously presented in multidisciplinary committee to assess response to primary treatment	Yes	8	100	23	69.7	21	72.4	52	74.3
	No			10	30.3	8	27.6	18	25.7	0.203
Follow-Up Phase
Percentage of patients experiencing serious (>grade II) RT-related adverse effects	Yes	1	2.5	3	7.5	2	5.0	6	5.0	0.591
	No	39	97.5	37	92.5	38	95.0	114	95.0
Percentage of patients with recorded follow-up in ICO	Yes	36	90.0	37	92.5	39	97.5	112	93.3	0.392
	No	4	10.0	3	7.5	1	2.5	8	6.7

Chemo, chemotherapy; RT, radiotherapy; VMAT, volumetric arc therapy; IMRT, intensity-modulated radiation therapy.

^gNo data found in the medical records.

Most (96.7%) patients received the planned dose, and 57.4% received it at the planned time (±2 days). There were significant differences with regard to the latter indicator, with one center never lengthening the treatment period for more than 5 days and the other two centers doing so in 25% and 16.6% of the cases, respectively. In the 46 cases where there was an interruption in the treatment, the most common reasons were public holidays (78.7%) and technical difficulties (31.9%). None of the patients who received SCRT experienced interruptions.

Surgery followed neoadjuvant treatment in 96.7% of the patients. Among this group, postoperative CRM was recorded in 65.5% (n = 76) of the cases and was negative in 93.4% of these. With regard to the 34.5% (n = 40) of cases where no CRM value was stated, there were differences between the centers: 83.3% of the reports from one center contained this information, while 74.4% and 40% of the reports from the other two centers did. Of the total sample, clinical histories showed that 5% of the patients experienced adverse events of grade III or higher according to the CTCAE (v 4.0)¹². Mean follow-up was 3.4 (SD 0.6) years, and overall survival at 4 years was 81.7%. After adjusting for age, sex, and tumor stage, there were no statistically significant differences between centers in this regard (P = .223, Table 4, Figure 1).

Table 4.

Analysis of Time to Death or Date of Last Follow-Up (Stage I Cases Omitted)¹

		Exitus
				Multivariate⁎
		n total	% exitus	HR (95%CI)	P
Center	Center A	40	20.0	1	0.223
	Center B	40	12.5	0.47 (0.15-1.45)	0.191
	Center C	40	22.5	1.24 (0.47-3.23)	0.666
Sex	Men	86	22.1	1
	Women	34	8.8	0.33 (0.10-1.15)	0.081
Age (years)	<60	28	21.4	1	0.272
	60-69	42	9.5	0.34 (0.09-1.29)	0.113
	70-79	40	25.0	1.04 (0.35-3.06)	0.949
	≥80	10	20.0	1.06 (0.19-5.86)	0.943
Stage	II	14	28.6	1	0.573
	III	93	15.1	0.59 (0.18-1.88)	0.372
	IV	12	33.3	1.30 (0.30-5.58)	0.723
	Missing	1	0.0	(-)	0.981

Dash (-) used when HR and 95% CI could not be calculated.

¹Specification cox model (statistical). This model collect the mortality for 4 years adjusted by center, sex, age group and stage.

Figure 1.

Survival functions with Cox multivariate model.

Discussion

This study brings to light the existence of considerable variability in clinical practice [7], [13] among three centers working with the same care protocols, demonstrating that consensus-based clinical practice guidelines and ISO certification are not enough; systematic monitoring and evaluation of each planned change are required to achieve the desired adherence.

For rectal cancer, the therapeutic strategy includes determining which patients are at the most risk, and high-resolution MRI is the most accurate technique, allowing us to identify the mesorectal fascia and its relation with the tumor front with a specificity of 92% [14]. The distance from the tumor to the mesorectal fascia (rCRM) is predictive of local recurrence. The results obtained for this indicator show considerable room for improvement: even though all the patients had an MRI, the medical records of 38.3% did not contain information on this parameter either in the diagnostic report or over the clinical course, making it a prime target for improvements. The relevance of this indicator has been recognized, for example, in the work of the Mercury Group [15], which reported that the rate of recurrence ranges from 53% in tumors with a potential margin of less than 1 mm of the CRM to 8% in patients with an rCRM of 8 mm or more.

In the treatment phase, adherence to clinical practice guidelines regarding the type of radiotherapy prescribed (LCRT versus SCRT) was rather low: 12 patients (10%) were indicated for SCRT, but 6 did not meet the pertinent criteria. Of the 9 patients who did meet the criteria for SCRT, only 6 (66.7%) received it. Although the indication of one technique over another has not been linked to local control, survival, late toxicity, or quality of life [16], the advantage of SCRT over LCRT lies in the simplicity of its administration and patients’ reduced dependency on the hospital. Moreover, regarding adherence to the planned treatment, the results are not optimal either, especially in one center. We observed that 13% of the cases were prolonged more than 5 days over the initial plan. Numerous studies directly relate local control of the tumor with the prolongation of radiotherapy [17], [18], [19], [20]; Bese [21] found that for cancer of the head and neck, each day that treatment was interrupted led to a decrease in local control of 1.4%. There is less evidence for rectal cancer, but we consider that administration of radiotherapy should rigorously adhere to the treatment plan and that it is unacceptable to prolong it more than 5 days, especially when the main reasons are avoidable organizational factors. As described in the Royal College of Radiologist guidelines [22], which also apply here, most interruptions are due to public holidays that fall on weekdays and technical issues (machine inspections and breakdowns). These factors are generally known well in advance, and staff should take them into account when planning treatment. A range of options is available to compensate for the negative effect of these interruptions, but essentially, it is a question of developing and implementing a protocol tailored to each center’s circumstances. Updating the protocol and regularly monitoring this indicator are actions that we use to improve adherence to planned radiotherapy treatment. Overall, the degree of adherence to clinical practice guidelines could be significantly better. We believe that a reasonable mechanism to improve it would be automated alerts and reminders—referring to consensus-based indicators—sent to professionals at the clinical station in order to support the decision-making process. This approach has led to improvements in other clinical settings [23], [24].

Finally, the other indicator considered crucial due to its prognostic value in patients’ evolution is the result of the ypCRM [25]. We observed that only 4.3% of the patients had a positive postoperative ypCRM margin, compared to the 8% to 13% in other published reports [26], [27], suggesting good results. However, and as with the rCRM value, the ypCRM values for 34.5% of the patients who underwent surgery are unknown. While other studies have reported similar percentages [28], we have launched several actions for improvement. We believe that the tumor board sessions are invaluable, and in addition to using this time to reach a consensus on each patient’s treatment plan, it should be used to emphasize the importance of everyone on the multidisciplinary team being able to access CRM data, which in this case implies that the anatomical pathology and diagnostic imaging services should add these data to their respective reports.

With regard to the clinical results, it is difficult to assess the adverse effects, in part because the design of the questionnaire did not allow any differentiation between acute and chronic events but mostly because the poor quality of the notes taken over the clinical course made grading the severity of the effects difficult. Without a doubt, adopting a registration system with structured data would produce more objective and internationally comparable data [29]. Furthermore, we confirmed the scant references to certain adverse effects, especially those related to the sexual sphere or sphincter control; the presence (or absence) of these effects should clearly be noted in the clinical histories. In fact, the questionnaire does not include any items related to quality of life, which we consider to be a limitation of the study to be rectified in future audits. There is no question that awareness among professionals should be raised with regard to monitoring patients’ quality of life, and we believe that the use of a validated questionnaire in these patients could be useful for systematizing an essential dimension of clinical results [11], [30]. Other study limitations relate to the design of the audit itself, as the sample of just 40 medical histories limits our assessment of the results and the comparison between centers, particularly for indicators on clinical results. Moreover, the period analyzed, 2013, was chosen so that we could assess all phases of the process, including follow-up and 4-year survival; however, some results have changed considerably since then. As with any review of clinical histories, the data are derived from the information contained in clinical registries, and any considerations not recorded there are also absent from the audit.

In conclusion, the audit we performed revealed a suboptimal degree of adherence to clinical practice guidelines for rectal cancer. Significant variability between centers exists from a clinical perspective but especially with regard to organization and process. In that sense, the value of a clinical audit like the one performed resides in its power to shed light on the need for improvement in areas that may otherwise have been considered resolved. Additionally, the audit raises questions about how to improve adherence to guidelines, what value professional consensus confers, and how to translate that consensus to practice during patient care. Clearly, the approval of a document by a group of experts is insufficient. We believe that using information and communication technologies in the form of automated guidelines to aid professionals at the clinical station could be the way to improve adherence. In any case, measuring and evaluating performance through a clinical audit are the first steps for improving clinical processes for the benefit of our patients.

Funding

No specific funding for this study.