The quantitative impact of joint peer review with a specialist radiologist in head and neck cancer radiotherapy planning

Abstract

Objectives:

Radiologist input in peer review of head and neck radiotherapy has been introduced as a routine departmental approach. The aim was to evaluate this practice and to quantitatively analyse the changes made.

Methods:

Patients treated with radical-dose radiotherapy between August and November 2020 were reviewed. The incidence of major and minor changes, as defined by The Royal College of Radiologists guidance, was prospectively recorded. The amended radiotherapy volumes were compared with the original volumes using Jaccard Index (JI) to assess conformity; Geographical Miss Index (GMI) for undercontouring; and Hausdorff Distance (HD) between the volumes.

Results:

In total, 73 out of 87 (84%) patients were discussed. Changes were recommended in 38 (52%) patients: 30 had ≥1 major change, eight had minor changes only. There were 99 amended volumes: The overall median JI, GMI and HD was 0.91 (interquartile range [IQR]=0.80–0.97), 0.06 (IQR = 0.02–0.18) and 0.42 cm (IQR = 0.20–1.17 cm), respectively. The nodal gross-tumour-volume (GTVn) and therapeutic high-dose nodal clinical-target-volume (CTVn) had the biggest magnitude of changes: The median JI, GMI and HD of GTVn was 0.89 (IQR = 0.44–0.95), 0.11 (IQR = 0.05–0.51), 3.71 cm (IQR = 0.31–6.93 cm); high-dose CTVn was 0.78 (IQR = 0.59–0.90), 0.20 (IQR = 0.07–0.31) and 3.28 cm (IQR = 1.22–6.18 cm), respectively. There was no observed difference in the quantitative indices of the 85 ‘major’ and 14 ‘minor’ volumes (p = 0.5).

Conclusions:

Routine head and neck radiologist input in radiotherapy peer review is feasible and can help avoid gross error in contouring.

Advances in knowledge:

The major and minor classifications may benefit from differentiation with quantitative indices but requires correlation from clinical outcomes.

Introduction

Systematic quality assurance or peer review is encouraged in all aspects of oncological care. The Royal College of Radiologists (RCR) first alluded to the importance of radiotherapy planning peer review in 2008 and went on to publish the official guidance almost a decade later.^1,2 Single-centre prospective studies on peer review process have demonstrated the value of this practice.^3–7 In international multicentre trials, the lack of radiotherapy peer review has contributed to a higher rate of treatment failure.^8–10 Unfortunately there are various limiting factors to peer review being a routine practice.^6,11

Head and neck radiotherapy techniques have evolved over the past two decades. The ability to precisely deliver high-dose radiation to the target with intensity modulated radiotherapy (IMRT) has allowed improved normal tissue sparing and quality of life.^12,13 There is however the least consensus on head and neck gross tumour volumes (GTV) and clinical target volumes (CTV).^14–16 Although there is now international consensus on head and neck contouring,^17–19 it is not possible to produce a guidance that encompasses all clinical scenarios. Peer review of the radiotherapy based on established guidelines will reduce interindividual variability and the risk of geographical miss.^2,20

Head and neck oncology is heavily dependent on interpretation of radiology. With the technological advances in imaging modalities, the ability of oncologists to consistently distinguish tumour from normal tissue is not always straightforward. Several studies have highlighted the importance of specialist radiologists as part of quality assurance.^21–23 There was also great heterogeneity of head and neck target delineations by oncologists compared with radiologists.²¹ This raises the question whether head and neck radiotherapy contouring can be solely performed by oncologists without input from radiologists.

As a departmental approach, a specialist head and neck radiologist has been routinely incorporated into the weekly radiotherapy peer review since 2019. The aim of this study was to evaluate the practice and to quantitatively analyse the changes recommended to the GTV/CTV by peer review.

Methods and materials

Treatment approach

Tumour biopsies, magnetic resonance imaging (MRI) of the neck and computed tomography (CT) of the chest were the standard investigations for all patients. Positron emission tomography (PET-CT) was requested for patients with suspected unknown primary, or high risk of distant metastasis e.g. Stage IVa/b disease, or nasopharynx cancer. Standard MRI sequences included T1, T2, T1 with contrast, T2 short tau inversion recovery (STIR), diffusion-weighted imaging (DWI), and apparent diffusion coefficient (ADC) mapping. Wherever possible, patients with oropharynx or nasopharynx cancer received additional MRI-simulation in their radiotherapy immobilisation masks. The MRI-simulation sequences included isometric T1 & T2 at 1 mm slices, dynamic contrast-enhanced (DCE) T1, and 3mm-sliced axial DWI.

The standard dose fractionation for definitive head and neck radiotherapy was 65 Gray (Gy) in 30 fractions (f).²⁴ The primary tumour 65 Gy CTV (CTVp_6500) and 54 Gy CTV (CTVp_5400) were typically 5 mm and 10 mm expansion of the primary GTV (GTVp) respectively.¹⁷ The CTVp was edited off structures not at risk of disease spread. The nodal 65 Gy CTV (CTVn_6500) was obtained by 5 mm expansion of the gross nodal disease (GTVn) or 10 mm if there was significant extra nodal extension.¹⁸ The elective 54 Gy nodal CTV (CTVn_5400) was delineated according to the established atlas.^19,25 The same dose fractionation was used for post-operative radiotherapy: Areas of residual disease, involved margin(s) or extranodal extension in the neck received 65 Gy in 30f. The remaining surgically operated bed at risk of harbouring microscopic disease received 60 Gy in 30f (CTVp_6000 and CTVn_6000). Regional unoperated but clinically at-risk areas of occult metastasis received 54 Gy in 30f.²⁶

Data acquisition

The IMRT volumes of head and neck patients treated with radical intent between August and November 2020 were evaluated. Any changes from the peer review were prospectively recorded. The one-hour weekly peer review was conducted remotely a week prior to radiotherapy start dates. Cases that were reviewed “off-line”¹⁵ outside the meeting were not included in this analysis. The meeting was attended by a minimum of one (out of two) consultant head and neck oncologists, trainees, a physicist, and a consultant head and neck radiologist who was present throughout the study period. The radiotherapy volumes had either been created by an oncologist or revised by the oncologist if delineated by trainees. During peer review, the clinical examination findings, histopathology and radiology reports, were presented by the responsible oncologist. The radiologist, who might not necessarily have been involved with the initial diagnostic process, reviewed the diagnostic and simulation images. The contouring guidelines were accessible to all during peer review.

The major and minor changes classification as per the RCR guidelines was used² i.e. a major change was defined as one that required alteration to prevent a geographical miss, or a change in the treatment paradigm. A minor change was defined as a change without which the original volumes would still have been deemed clinically acceptable, e.g. excluding an uninvolved neck muscle from CTV. The decision regarding which classification was made by the clinicians at peer review and was recorded prospectively.

To allow the quantitative analyses to be performed, the pre-peer review original IMRT volumes were saved prior to any alterations. The amended volumes were then retrospectively compared with the original volumes using the Jaccard conformity index (JI), Vant Riet (VR) and Dice similarity coefficient (DICE) to assess conformity^27,28 ; Geographical Miss Index (GMI) to assess undercontouring; Discordance Index (DI) for overcontouring; and Hausdorff distance (HD) to assess the maximum distance between the volumes in three-dimension. Computational environment for radiotherapy research (CERR) software was used to perform these functions.²⁹ Only the amended volumes were included in the quantitative analyses which were performed independently by a single-blinded observer. The difference in the quantitative analyses of ‘major’ and ‘minor’ volumes was assessed using Mann–Whitney U-test.

Results

During the 4-month study period, 104 patients (58 definitive, 29 adjuvant and 17 palliative) were planned for IMRT. Eighty-four percent (73 out of 87) radical patients were reviewed, constituting 50 and 23 definitive and post-operative IMRTs, respectively. Details of patient demographics, disease sites and the proportion of changes are shown in Table 1. Sixteen percent (14 out of 87) patients were not included in the study. They were either discussed outside peer review due to oncologists’ expected absence, or not formally reviewed due to lack of time when more complex review was required for other cases.

Table 1.

Patients and disease demographics with the changes

	Definitive IMRT (n=50)	Post-operative IMRT (n=23)	All IMRT (n=73)
Age
Median (Range)	61 (36 – 90)	64 (32 – 75)	62 (32 – 90)
Sex
Male	43	11	54
Female	7	12	19
Stage (8th edition)
I – II	18	4	22
III – IVb	22	19	41
Tumour Site
HPV-Mediated Oropharynx	26 (4*)	0	26 (4*)
Oral Cavity	5 (1*)	17 (4*)	22 (5*)
Larynx	10	1	11
Oropharynx (non-HPV)	5 (2*)	0	5 (2*)
Hypopharynx	1	1	2
Unknown Primary	1	0	1
Nasopharynx	1	0	1
Major salivary glands	0	1 (1*)	1 (1*)
Thyroid Cancer	0	2	2
Skin	0	1 (2*)	1 (2*)
Nasal Cavity/Paranasal Sinus	1	0	1
Changes Recommended
HPV-Mediated Oropharynx	13 (50%)	0	13 (50%)
Oral Cavity	4 (80%)	10 (59%)	14 (64%)
Larynx	5 (50%)	0	5 (45%)
Oropharynx (non-HPV)	4 (80%)	0	4 (80%)
Hypopharynx	1 (100%)	0	1 (50%)
Unknown Primary	1 (100%)	0	1 100%)
Total	28 (56%)	10 (44%)	38 (52%)
≥1 Major	22 (44%)	8 (35%)	30 (41%)
≥1 Minor only, no major	6 (12%)	2 (9%)	8 (11%)

^aNumber of patients not formally reviewed. HPV – Human Papilloma Virus.

The peer review recommended modifications in 38 out of the 73 patients (52%); 28 out of 50 definitive IMRT patients (56%) and 10 out of 23 postoperative IMRT cases (44%). In the definitive IMRT cohort, 22 of the 28 patients (79%) underwent at least one major change. A similar pattern was also observed in the post-operative IMRT cohort where 8 out of the 10 patients (80%) required major changes. Eight patients (six definitive, two post-operative) underwent minor changes only.

The highest number of ‘major’ changes in the definitive IMRT cohort were in the GTVp (35%) and the subsequent expansion CTVp_6500 (35%), as shown in Table 2. There were two definitive IMRT patients without GTVp as one was an unknown primary and the other a neck recurrence two years after the primary surgery and neck dissection for an oral cavity cancer. The proportion of GTVn changes (18%) was not as high as that of GTVp (35%). The proportion of changes for postoperative IMRT patients was slightly lower than that of the definitive IMRT cohort, though it had a smaller number of patients. In the post-operative IMRT cohort, there was no GTVp post-surgical resection, but two patients had GTVn changes as they were thought on review to have gross nodal recurrence after the neck dissection.

Table 2.

The distribution of amended radiotherapy volumes

Definitive IMRT	GTVp	CTVp_6500	CTVp_5400	GTVn	CTVn_6500	CTVn_5400
Major	17 (35%)	17 (35%)	15 (31%)	7 (18%)	9 (23%)	5 (10%)
Minor	0	3 (6%)	4 (18%)	0	1 (3%)	3 (6%)
No change	31	28	29	33	30	42
n/a	2	2	2	10	10	0
Total	48	48	48	40	40	50
Post-operative	CTVp_6500	CTVp_6000	GTVn	CTVn_6500	CTVn_6000	CTVn_5400
Major	3 (25%)	3 (15%)	2 Recurrences post-operatively	3 (30%)	0	4 (24%)
Minor	0	1 (5%)	0	0	1 (7%)	1 (6%)
No change	9	16	0	7	14	12
n/a	11	3	21	13	8	7
Total	12	20	2	10	15	17

There were in total 99 peer-review amended volumes available for the quantitative analyses, with 81 and 18 from the definitive IMRT and post-operative IMRT cohort, respectively. The overall changes to individual volumes from both cohorts are shown in Table 2: The amended volumes included 17 GTVp, 23 CTVp_6500, 23 CTVp_6000/5400, 9 GTVn, 13 CTVn_6500, and 14 CTVn_6000/5400. Of these amended volumes, the overall median JI was 0.910 (interquartile range [IQR]=0.795–0.965); VR: 0.909 (IQR = 0.795–0965); DICE: 0.953 (IQR = 0.889–0.982); GMI: 0.057 (IGR = 0.023–0.177); DI: 0.003 (IQR = 0.000–0.035); and HD was 0.423 cm (IQR = 0.200–1.166 cm).

The breakdown of the quantitative indices of each amended volume is shown in Table 3. The GTVp, despite undergoing ‘major’ changes, generally had median conformity indices JI, VR and DICE of more than 0.9. The median GMI and DI for the primary volumes were less than 0.1. The HD were generally less than 0.5 cm. Although there were comparatively fewer recommended number of changes to the nodal GTVn (n = 9, 21%), there was a wider range of variations as shown in the box and whisker plots of Figure 1. The JI and GMI of GTVn and CTVn_6500 were relatively higher than that of primary volumes. The HD of the amended nodal volumes was also considerably more than that of primary volumes (Figure 2). Reasons for this included a case of considerably undercontoured node (Figure 3), two cases of missed-identification of involved ipsilateral retropharyngeal nodes (Figure 4), and a case of growing contralateral involved nodes deemed highly suspicious for progressive malignancy (Figure 5).

Table 3.

The quantitative indices of the amended radiotherapy volumes

		Median	Interquartile	Range	Std Dev
			Q1	Q3
JI	GTVp	0.941	0.832	0.965	0.075
	CTVp_6500	0.885	0.809	0.962	0.142
	CTVp_6000/5400	0.924	0.873	0.961	0.209
	GTVn	0.888	0.436	0.951	0.347
	CTVn_6500	0.778	0.589	0.903	0.288
	CTVn_6000/5400	0.942	0.885	0.985	0.265
VR	GTVp	0.940	0.831	0.965	0.076
	CTVp_6500	0.882	0.802	0.962	0.144
	CTVp_6000/5400	0.924	0.871	0.961	0.211
	GTVn	0.888	0.406	0.951	0.352
	CTVn_6500	0.776	0.589	0.903	0.289
	CTVn_6000/5400	0.941	0.885	0.985	0.265
DICE	GTVp	0.970	0.908	0.982	0.042
	CTVp_6500	0.939	0.894	0.981	0.093
	CTVp_6000/5400	0.960	0.932	0.980	0.201
	GTVn	0.941	0.607	0.975	0.331
	CTVn_6500	0.875	0.741	0.949	0.279
	CTVn_6000/5400	0.970	0.939	0.993	0.261
GMI	GTVp	0.039	0.014	0.069	0.076
	CTVp_6500	0.068	0.023	0.140	0.144
	CTVp_6000/5400	0.049	0.022	0.116	0.256
	GTVn	0.112	0.047	0.506	0.342
	CTVn_6500	0.201	0.066	0.311	0.284
	CTVn_6000/5400	0.028	0.006	0.101	0.266
DI	GTVp	0.008	0.000	0.032	0.044
	CTVp_6500	0.030	0.001	0.066	0.045
	CTVp_6000/5400	0.022	0.000	0.041	0.202
	GTVn	0.000	0.000	0.004	0.332
	CTVn_6500	0.004	0.001	0.188	0.299
	CTVn_6000/5400	0.000	0.000	0.003	0.070
HD	GTVp	0.36 cm	0.22 cm	0.45 cm	0.339
	CTVp_6500	0.42 cm	0.24 cm	0.68 cm	0.432
	CTVp_6000/5400	0.36 cm	0.16 cm	0.48 cm	0.387
	GTVn	3.71 cm	0.31 cm	6.93 cm	3.280
	CTVn_6500	3.28 cm	1.22 cm	6.18 cm	2.522
	CTVn_6000/5400	0.20 cm	0.00 cm	1.87 cm	1.818

Figure 1.

The Jaccard conformity index (Top) and Geographical miss index (Bottom) of the amended volume structures

Figure 2.

The Hausdorff distance (centimetre) between the amended and original volumes in ascending order

Figure 3.

Peer review GTVn amendment (outer highlighted red contour) to the original nodal delineation (inner red contour) JI: 0.43, GMI: 0.51

Figure 4.

Retropharyngeal nodes identified at peer review. Top row patient – missed solitary node (JI: 0.00, GMI: 1.00). Bottom row – retropharyngeal node in an N2b neck (JI: 0.41, GMI: 0.59, HD 6.9 cm)

Figure 5.

Top left – Pre-peer review GTVn (red). Top Right – Post-peer review amended GTVn (JI 0.82, GMI: 0.18 HD: 7.09 cm). Bottom left – Pre-peer review CTVn_6500 (magenta). Bottom Right – Post-Peer review CTVn_6500 (JI: 0.69, GMI: 0.31, HD 7.1 cm)

There were in total 85 amended radiotherapy volumes classified as ‘major’, and 14 as ‘minor’. Although markedly different in the subjective clinical context of the major and minor classification, there was no statistical difference in the distribution of quantitative analyses between them (Figure 6).

Figure 6.

Distribution of ‘major’ and ‘minor’ changed volumes

Discussion

The overall results showed there was a high proportion of volumes changed as a result of peer review, affirming the importance of quality assurance. One of the experiences from this study was that it was not uncommon for oncologists to realise obvious misses or mistakes at the peer review, despite meticulous care being taken during contouring. Our institution is considered a medium-sized tertiary centre but with a high ratio of head and neck patients per oncologist. Although there may be a volume–quality relationship in the outcomes of head and neck surgical cases,^30,31 there is no evidence to support ‘quality’ in radiotherapy centres without established peer review. As evidenced from the high proportion of volumes changed, having the experience of a high number of head and neck IMRT cases per oncologist did not exclude the need for robust peer review.

Prior to the current format of peer review, the oncologists had involved head and neck radiologists on an ad-hoc basis, but radiologists had not always been readily available, and some patients therefore missed out important radiology input. Our institution is not immune to the national shortfall of oncologists and radiologists.^11,32 Given the patient volume it was not deemed practical to have radiologists available for all oncologists’ radiotherapy planning sessions. Instead, creating a fixed weekly peer review session with a head and neck radiologist, one week prior to radiotherapy start dates, has turned out to be a feasible local practice. Whilst the time spent per individual case was not prospectively recorded, 84% of the radically treated cases (average 4 cases per session) were constructively reviewed without the meetings being substantially rushed or overrun. Physicists present at the meeting had adequate time for the planning and no radiotherapy was delayed because changes from peer review.

The presence of a head and neck radiologist in the peer review may be a reason for a higher number of alterations compared to some of the reported head and neck peer review series.^3,5,15 The 52% change from this study is similar to the 55% change by Braunstein et al who also had neuro-radiology support.²¹ The experience from the study process was that oncologists were not always able to fully appreciate subtle but crucial details on functional MRI sequences (e.g., DWI, ADC mapping) to delineate the GTV, until these were specifically pointed out by the radiologist at peer review. Functional MRI sequences are increasingly gaining significance in radiotherapy planning.^33,34 The Danish Head and Neck Group (DAHANCA) recently published their guidance on using the right radiological information in delineating head and neck cancer.³⁵ Utilising the functional MRI components in delineating tumour, as well as routine radiologist input in locating the complex nerve distribution in cancer with perineural invasion, may well become the routine practice for head and neck oncologists.

Contouring techniques for head and neck IMRT have become more volumetric than the traditional field-based approach of conventional radiotherapy. Some institutions may define the therapeutic high-dose CTVp as a 1 cm isometric expansion of GTVp, with or without additional compartmental elective dose coverage of the primary subsite e.g. encompassing the whole oropharynx or larynx. As a departmental practice, the “5 + 5” consensus for definitive IMRT^17,18 has been implemented since 2018. This approach could improve the therapeutic ratio compared to whole subsite irradiation, but the target volumes require demarcating accuracy. This may explain the high proportion of GTVp and CTVp changes in this study. Nonetheless, our GTVp still had a high conformity and low geographical miss, consistent with quality assurance studies of clinical trials which showed a median conformity index of 0.7.^36,37

The “5 + 5” approach was also adopted for the nodal delineation,^18,25 with identification of each GTVn individually^38,39 and 5 mm expansion as the high dose CTVn in the definitive setting. Some centres may routinely treat the whole anatomical neck level with therapeutic high dose when there is a single involved node within the particular level. Delineation of each individual node can be more prone to interobserver error, as reflected in the results of this study. Despite having the diagnostic radiology reports available for contouring, there were cases of involved nodes that had not been specifically reported prior. One of the two cases of missed retropharyngeal nodes (top row of Figure 4) had not been previously described. The example shown in Figure 5 was an oropharynx cancer patient who had initially been reported as having ipsilateral involved nodes only. The enlarging contralateral nodes noted on the subsequent CT simulation resulted in recommendation of treating them as pathological by peer review. Overall, the quantitative analyses show that the magnitude of changes made to our nodal volumes was larger than the changes made to the primary volumes, although the number of cases requiring changes was not as high.

In this study, the decisions for any changes made to the IMRT volumes were made jointly by all clinicians at the peer review. Every query on any volume was considered based on clinical and radiological merits. It is appreciated that there is normally an inherent difference in volume delineation between oncologists and radiologists, with Giroud et al reported that radiologists tended to delineate smaller GTVs compared to oncologists.^21,40 When it comes to radiotherapy planning for head and neck cancer, clinical examinations (e.g., transoral examination, nasopharyngoscopy, neck palpation) are an essential part of volume contouring.^17,35 Disease extension along the oral mucosa or skin, which may appear as erythema, is not always fully illustrated radiographically but requires inclusion in the treatment volume. Functional imaging, however informative, is not a complete substitute for head and neck clinical examinations. Regrettably, there were restrictions in performing aerosol generating procedures during the study period which was in the first year of Covid-19 pandemic. This meant nasopharyngoscopic examination was not performed on every patient by the treating oncologists. Inadequate clinical findings, especially in the event of uncertain radiology (e.g., image artifacts), could lead to larger radiotherapy volumes. As evidenced by the higher GMIs than DIs, the peer review tended to recommend expanding the treatment volumes to compensate for potential geographical miss rather than be concerned about overcontouring. This was not a unique problem to any particular institution during the pandemic, but the joint peer review has demonstrated the value of collaborative volume delineation and not about performance of individual clinicians.

When reporting changes to the volumes, there has not been a consistency of how ‘major’ or ‘minor’ changes are defined in the literature. One method^3,5 defines changes made to GTV or high dose CTV as ‘major’, and alterations to the elective CTV as ‘minor’. A potential limitation of any major or minor classification is the lack of objective quantitative measurement, as a slight adjustment to the GTV could be considered as ‘major’ but entire omission of the contralateral elective neck coverage could be considered as ‘minor’. One of the proposed suggestions for a unified classification by Lewis et al is any edit to the GTV or CTV by more than 1 cm to be considered as ‘major’, and modification less than 1 cm as ‘minor’.⁴¹ Instead of a one-dimensional cut-off though, integration of multiple quantitative and conformity indices as part of the classification could be beneficial. As shown in Figures 1 and 2, the low JI and high GMI & HD values of the nodal volumes may be collectively utilised as a combined single metric to indicate a “gross” or “major” change better than one particular measurement. Conformity and quantitative indices have been evaluated with artificial intelligence algorithms for detecting gross error.⁴² Different weightings may be placed on the indices as part of the algorithm design. Although there was no observed difference in the ‘major’ and ‘minor’ quantitative analyses in this study, the total number was too small to make conclusions about the true impact of the indices.

Conversely, it is possible that even a major change clinically can be a small correction that is not necessarily reflected by the quantitative indices. The indices may fail to capture the topographic element of the volume i.e. the anatomy of the head and neck. For instance, a laryngeal/glottic tumour, although locally advanced, can be small in volume. Any increase in volume within the larynx, such as a correction to cover the involved anterior commissure, is unlikely to produce a higher HD value than covering the nodal disease of the contralateral neck. Moreover, the indices do not always reflect the underlying clinical thought process. For example, increasing the CTV to cover a large indeterminate area as a precaution will result in a lower JI and higher GMI than a slight correction to cover a small but obvious tumour miss.

The major criticism of this study is the lack of clinical outcome data to assess the peer review or the significance of the quantitative results. The recognisable limitation of the quantitative indices, like any current major and minor classifications, is their intrinsic inability to translate into meaningful clinical impact e.g. pattern of recurrence. Human clinical judgement, however subjective, is still required in clinical practice of peer review. When used in the right context though, the quantitative indices can support clinicians with numerical objectiveness to assess the extent of changes.

Conclusions

Incorporating a specialist radiologist to assist oncologists in head and neck radiotherapy peer review is a feasible and valuable practice. Quantitative conformal indices are useful in illustrating the magnitude of volume change. The major and minor change classification may benefit from inclusion of quantitative indices but ultimately this should be correlated with clinical outcomes.