A single center, inter-observer evaluation of vestibular schwannoma stereotactic radiosurgery and its dosimetric impact

Abstract

The aim of this work was to evaluate the inter- and intra-observer variation in contouring vestibular schwannoma (VS) and the organs-at-risk (OAR), and its dosimetric impact in Volumetric Modulated Arc Therapy (VMAT). Three VS typical cases were contoured by four clinicians. The Agreement Volume Index (AVI) appeared to be notably higher in VS than in OARs, such that the dose coverage of VS is fairly robust. In OARs, the largest variation was +1.02Gy in dmax for the brainstem, +0.78Gy in dmean for the cochlea and +1.05Gy in dmax of the trigeminal nerve. Accordingly, it was decided that all VS delineations for stereotactic radiosurgery (SRS), and all frame-based SRS contouring in general, should always be reviewed by a second physician. In addition, the retrospective presentation of VS cases at daily peer review meetings has also been adopted to ensure that the consensus is constantly updated, as well as for training purposes.

INTRODUCTION

Vestibular schwannoma (VS) is a benign tumor that originates from the vestibular portion of the eighth cranial nerve [1]. The therapeutic options to manage VS are surgery or stereotactic radiosurgery (SRS), the latter representing a well-consolidated approach to treat small to mid-sized VS with remarkably high rates of local control and minimal toxicity [1-8].

VS represents a specific scenario for highly modulated SRS VMAT (Volumetric Modulated Arc Therapy), in particular considering the proximity of these tumors to OARs like the brainstem, trigeminal nerve and cochlea.

The comparison of the contours produced by several observers with themselves (intra-observer variation), as well as with each other (inter-observer variation), is known as an important source of systematic error in radiotherapy procedures. As inter-observer variations are potentially much larger than intra-observer variations, this issue has received more attention and numerous studies have addressed this phenomenon for a wide range of existing treatments and locations [9]. For SRS in particular, the inter-observer variability when defining the target volume and the OARs has been studied for a benign tumor adjacent to the optic tract as a quality assurance exercise in a multicenter study of GK users [10-12]. However, to our knowledge, no study has contemplated the dosimetric impact of intra/inter observer variation in the case of VS that specifically includes OARs [9].

The treatment of VS is especially critical given that maximal OAR tolerances are employed at the doses used and most of the volumes considered are remarkably small. The aim of this study was to evaluate the inter- and intra-observer variation in the contouring of the OARs and VS, and the dosimetric significance of this when performing VMAT.

MATERIALS AND METHODS

Treatment technique, planning and prescription

First, the Brainlab (BrainLAB AG, Feldkirchen, Germany) head frame was placed on the patients under local anesthesia and a cranial Computed Tomography (CT) image was obtained on a Philips Brilliance Big Bore scanner (Philips HealthCare, Best, Netherlands) at the finest resolution available (0.8 mm).

Contrast medium was administered into the inferior vena cava (IVC) to enhance the tumor visibility. Visualizing the vascular structures in the posterior fossa makes it possible to assess the co-registration between Magnetic Resonance Images (MRI) and CT planning with the frame placed on the patient.

Volume contouring was performed with the iPlan treatment planning system (v.4.5.5, Brainlab), and it is based on the Signa HDtx (General Electric, Milwaukee, WI, USA) MRI images in three different acquisition modes: T1:3D Neuro-Navigator (NN), overlapping slices of 0.5 mm width; T1, Fast Acquisition with Multiphase Elliptical fast gradient echo (FAME); and T2, Fast Imaging Employing Steady State Acquisition (FIESTA).

The VS is defined on the basis of the T1:NN and T1:FAME sequences, whereas the T2:FIESTA sequence is useful to define the relationship between the lateral end of the tumor and the fundus of the internal acoustic channel and cochlea.

Co-registration of the CT and MRI was performed in iPlan, where the VS is defined as the Gross Tumor Volume (GTV) and as no extra margin is given due to the headframe, the GTV is equal to the Planning Target Volume (PTV) [13, 14].

VMAT was employed, which in the case of Varian is specifically called RapidArc® (RA). The plans were optimized and calculated in Eclipse v.15.5 (Varian, Palo Alto, CA) using 6MV FFF RX. An AAA algorithm (v.15.511) was used with a 1 mm grid resolution and VMAT was optimized with the PO algorithm (v.15.5.11). Four non-coplanar homolateral partial arcs were used with a couch angle separation range from 0 to 70 degrees. The length of the arcs was about 100 to 120 degrees.

The beams were administered in a TrueBeam linear accelerator (linac) v.2.5 (Varian, Palo Alto, CA) with high-definition Multi-Leaf Collimator (MLC) (2.5 mm width at the isocenter). Once the frame is fixed to the couch, the final patient set-up is established using a 6D couch after Cone Beam Computed Tomography (CBCT) relative to the planned CT matching. The typical treatment time (imaging plus delivery) is 12-15 min.

The aims of the SRS VS protocol planning are:

1. VS coverage such that D95% (where Dx is the minimum dose in Gy to reach x% of the target volume) receives the prescription dose of 12 Gy, keeping VS homogeneity within 107%.

2. Maximum brainstem dose of 12 Gy.

3. Regarding the trigeminal nerve, when the distance from the nerve to the VS is >2 mm the maximum dose should be <10 Gy. For shorter distances, the dose is limited to D10% <10 Gy.

4. The mean dose to the cochlea must be <4 Gy if preserving hearing is mandatory. In cases of a incomplete or non-useful hearing, this preservation is secondary to the VS conformity, with typical reference value of 5 Gy.

5. In some cases when the VS volume is large and in close proximity to the brainstem, an internally defined overdose volume was used for VMAT optimization to facilitate tumor shrinkage. This was the case in patient #3 (P3) in the present inter-observer study, whereby D95% was established as 11.5 Gy with an inner volume at 12.5 Gy.

According to the ICRU-91 [15], the so-called near-to-maximum dose should be taken as the maximum dose (Dmax), which is set at the dose to the hottest volume of 0.035 cc (D0.035cc). This was applied in all cases except for the cochlea due to its extremely small size, in this case the maximum pure dose and the hottest 1% dose were used.

Intra- and Inter-observer variation

This study centered on three typical cases of VS that had undergone SRS, selected from the patients treated and covering three different distances of the VS to the brainstem, all lying within a typical range. Indeed, they reflected the range of target sizes and shapes most often observed when SRS is performed at our center.

Four clinicians (1 Neurosurgeon and 3 Radiation Oncologists) participated in this study in a blind manner, all of whom have experience in the planning and treatment of VS. For the inter-observer evaluation, each observer worked on the set of MRI images in which the contours of the other observers were hidden, as was the contour used for treatment. Similarly, all the contours were masked to evaluate the intra-observer variation and the contouring was repeated with an interval of a least one week from the first delineation to reduce any bias. All observers contoured the same image modalities, mainly T1:NN with refinement according to each observer’s criteria with T1:FAME and T2:FIESTA. The registration was the same for all the users and was that used in the treatment plan.

The inter-observer differences of the delineated structures were evaluated using a methodology proposed previously [12,16]. The following parameters were calculated for all the structures: the union volume AV100/N; the intersect volume AV100; and the Agreement Volume Index (AVI). The AVI is the ratio between the AV100 and AV100/N, and it adopts a value between 0 and 1, reflecting the level of agreement with respect to the volume, size and shape.

The approach adopted in this inter-observer study was to consider the dosimetric implications of both VS coverage and the maximum dose to the OARs, in addition to an analysis of the resulting contours and their spatial location. For every set of contours produced by each observer, the treatment plan applied to the patient was downloaded and the resulting dosimetric parameters were analyzed. The treatment plan was originally contoured by one of the four clinicians in this study and hence, this clinician only contoured that case once to evaluate the intra-observer variation. The dosimetric parameters extracted from the treatment planning dose volume histogram were:

• VS (in this case GTV or PTV): D98%, D95%, Dmean, D50%, D10%, D2%. Also, according to ICRU-91 [15] the Dnear-min and Dnear-max were also reported, established as the dose to the volume minus 0.035cc and the hottest 0.035cc, respectively.

• Brainstem: D0.1cc, D0.035cc and V12Gy (Volume that receives a dose ≥12 Gy).

• Trigeminal nerve: D0.1cc, D0.035cc and D10%.

• Cochlea: Dmean, Dmax and D1%.

• Conformity index (CI) [17]

• Paddick Conformity Index (PCI) [17]

• Homogeneity Index (HI) [17]

• Gradient Index (GI) [17]

RESULTS

The superposition of all the contours is presented for the three patients, including the intra and inter-observer variations (Figure 1). The two selected axial and coronal planes attempt to represent the situation of VS and brainstem, cochlea and trigeminal nerve.

Figure 1.

Superposition of all the contours (VS in orange, brainstem in green, cochlea in cyan and trigeminal nerve in yellow) for the three cases: P1, P2, P3. On the left is the axial plane and on the right the coronal plane. The same scale was used for all images

Notably, the variability between the observers is more evident in the OARs than in the VS. Moreover, in the case of the brainstem much more attention was paid to the volume adjacent to the VS than to rest of the organ. Logically, this adjacent volume is that which will condition the dosimetry given that the constraint is the maximum dose.

In terms of the inter-observer evaluation, the mean, median and range of the contoured volumes, as well as the AV100, AV100/N and AVI, are summarized in Table 1. It should be noted that the inter-observer AVI values were notably higher in the VS than in the OARs, which in turn increase as the volume augments. The AVI was noticeably lower in the smaller OARs (cochlea and trigeminal nerve), which is logical due to their size.

Table 1.

Summary of the VS and OAR inter-observer variability for the four observers with each of the three cases (P1–P3). All volumes are in cc

P1
	VS	Brainstem	Cochlea	Trigeminal nerve
Mean volume	0.35	25.92	0.05	0.11
Median volume	0.36	27.44	0.04	0.10
Range	(0.29-0.38)	(20.58-28.21)	(0.04-0.06)	(0.07-0.17)
AV100	0.27	18.63	0.02	0.05
AV100/N	0.42	32.85	0.08	0.17
AVI	0.64	0.57	0.25	0.29
P2
	VS	Brainstem	Cochlea	Trigeminal nerve
Mean volume	1.53	26.09	0.06	0.09
Median volume	1.53	26.17	0.05	0.09
Range	(1.44-1.62)	(24.44-27.58)	(0.04-0.11)	(0.06-0.13)
AV100	1.29	19.80	0.01	0.04
AV100/N	1.76	32.31	0.07	0.16
AVI	0.73	0.61	0.14	0.25
P3
	VS	Brainstem	Cochlea	Trigeminal nerve
Mean volume	4.50	22.91	0.05	0.05
Median volume	4.49	23.38	0.06	0.04
Range	(4.28-4.73)	(20.04-24.83)	(0.03-0.07)	(0.03-0.08)
AV100	4.02	17.02	0.01	0.01
AV100/N	5.02	28.63	0.09	0.10
AVI	0.80	0.59	0.11	0.10

The AVI values of the two delineations to assess the intra-observer variation for the four observers and the three patients are shown in Table 2. Again, the value of the AVI is higher in the VS than in the smaller OARs (cochlea and trigeminal nerve). The AVI resulting from the intra-observer study is notably higher than that for the inter-observer study. Hence, the subsequent dosimetric impact analysis focused on the inter-observer data.

Table 2.

Intra-observer evaluation: AVI of the VS and OARs for the four observers (OBS 1-4) for each of the three cases (P1-P3)

		VS	Brainstem	Cochlea	Trigeminal nerve
	P1	0.90	0.95	0.40	0.71
	P2	0.86	0.96	0.18	0.57
OBS1	P3	0.92	0.95	0.50	0.29
	Mean AVI	0.89	0.95	0.36	0.52
	P1	0.80	0.81	0.57	0.55
	P2	0.83	0.88	0.33	0.36
OBS2	P3	0.84	0.85	0.55	0.33
	Mean AVI	0.82	0.84	0.48	0.41
	P1	0.85	0.94	0.60	0.50
	P2	0.91	0.89	0.60	0.63
OBS3	P3	0.92	0.84	0.40	0.40
	Mean AVI	0.89	0.89	0.53	0.51
	P1	0.81	0.78	0.57	0.64
	P2	0.85	0.81	0.50	0.60
OBS4	P3	0.92	0.77	0.63	0.20
	Mean AVI	0.86	0.79	0.57	0.48

The patient’s treatment plan was downloaded for each contour set in the inter-observer study, recording the dosimetric parameters established for the VS (Table 3). The maximum deviations with respect to the reference plan in D95% were -0.23 Gy (P1), -0.12 Gy (P2) and -0.31 Gy (P3). Notably, despite the variability in the contours the impact on VS coverage amounted to a worst-case variation of -2.7%. Similarly, the different indices of conformity, homogeneity and gradient were very close.

Table 3.

VS dosimetric parameters and indices for the three cases (P1-P3): the prescribed doses (D95%) are in red (Ref. values correspond to treated values)

	P1			P2			P3
	Ref. value	Mean	Range	Ref. value	Mean	Range	Ref. value	Mean	Range
Mean [Gy]	12.30	12.25	12.22–12.30	12.28	12.25	12.24–12.28	12.52	12.44	12.40–12.52
Dnear-min [Gy]	12.06	11.88	11.59–12.06	11.93	11.77	11.71–11.93	11.18	10.31	8.80–11.18
D98% [Gy]	11.92	11.68	11.52–11.92	11.93	11.79	11.71–11.93	11.24	10.95	10.45–11.24
D95% [Gy]	12.00	11.86	11.77–12.00	12.00	11.92	11.88–12.00	11.50	11.34	11.19–11.50
D50% [Gy]	12.30	12.27	12.25–12.30	12.28	12.26	12.25–12.28	12.62	12.59	12.56–12.62
D10% [Gy]	12.53	12.52	12.51–12.53	12.51	12.50	12.49–12.51	13.09	13.09	13.08–13.09
D2% [Gy]	12.65	12.65	12.64–12.65	12.61	12.61	12.60–12.61	13.22	13.21	13.21–13.22
D0.035cc [Gy]	12.51	12.51	12.51–12.52	12.60	12.60	12.60–12.60	13.26	13.26	13.26–13.26
CI	1.31	1.28	1.27–1.31	1.15	1.16	1.15–1.16	1.01	1.01	1.01–1.01
PCI (Paddick)	0.74	0.76	0.74–0.79	0.87	0.86	0.81–0.88	0.89	0.85	0.82–0.89
HI	1.04	1.04	1.04–1.04	1.05	1.05	1.05–1.05	1.11	1.11	1.11–1.11
GI	7.03	7.03	7.03–7.03	4.96	4.96	4.96–4.96	4.04	4.04	4.04–4.04

The maximum differences for the near min doses were -0.23 Gy (P1) and -0.12 Gy (P2), while the larger size of the maximum variation of -2.38 Gy in P3 stood out. Indeed, the reference plan for this case displayed the widest range.

The inter-observer variability in the dosimetric parameters for the OARs were established relative to the reference plan (Table 4). For the brainstem, V12Gy was found to be 0 in all cases. In turn, the maximum doses (in 0.035 cc) for the largest variations resulted in a higher dose than the reference dose of +0.53 Gy (P1), +1.02 Gy (P2) and +0.65 Gy (P3). The value that deviated most corresponding to P2, although this was still within the maximum allowable value, i.e.: 12 Gy. In terms of the cochlea, the largest deviations in the mean dose were +0.78 Gy, +0.1 Gy, and +0.76 Gy for P1 to P3, respectively. The highest deviation observed would have resulted in a dose higher than the established limit of 4 Gy for P1 and especially, for P3 (4.73 Gy). Finally, the largest variations in the maximum doses (in 0.035 cc) for the trigeminal nerve resulted in a higher dose than the reference dose of +0.68 Gy and +1.05 Gy for P2 and P3, respectively. However, these values were well below the established limits.

Table 4.

OAR dosimetric parameters for the three cases (P1-P3: the ref. values correspond to the treated values

Brainstem
	P1			P2			P3
	Ref. value	Mean	Range	Ref. value	Mean	Range	Ref. value	Mean	Range
D0.1cc	4.62	4.92	4.47–5.09	9.43	9.34	7.94–10.42	10.60	10.77	10.60–11.01
D0.035cc	5.86	6.21	5.56–6.39	10.40	10.39	9.16–11.42	10.93	11.18	10.93–11.58
V12Gy	0.00	0.00	0.00–0.00	0.00	0.00	0.00–0.00	0.00	0.00	0.00–0.00
Cochlea
	P1			P2			P3
	Ref. value	Mean	Range	Ref. value	Mean	Range	Ref. value	Mean	Range
Mean [Gy]	3.40	3.61	3.26–4.18	3.19	3.18	3.07–3.29	3.97	4.25	3.82–4.73
Dmax	8.85	9.20	7.75–10.99	5.54	0.01	4.61–5.58	8.15	8.53	8.15–9.59
D1%[Gy]	3.26	8.19	6.86–9.91	3.07	4.59	4.23–5.10	4.47	7.70	7.30–8.63
Trigeminal nerve
	P1			P2			P3
	Ref. value	Mean	Range	Ref. value	Mean	Range	Ref. value	Mean	Range
D0.1cc	0.42	0.14	0.00–0.42	0.00	0.37	0.00–1.48	0.00	0.00	0.00–0.00
D0.035cc	0.91	0.74	0.60–0.91	2.58	2.51	1.90–3.26	5.45	5.90	5.31–6.95
D10%[Gy]	1.37	1.59	1.37–1.74	4.28	3.83	3.43–4.47	8.52	8.39	8.21–8.52

DISCUSSION

Single-dose or fractionated radiotherapy can be used to treat VS, although the decision to employ either technique must take into account several patient and tumor related factors, the most important of which is tumor size. The typical size is limited to 1.5 to 3 cm given the greater risk of side effects to normal tissue as the VS size increases [18].

Here a single center study was carried out to assess the planning of VS by experienced neurosurgeons and radiation oncologists. The aim was to focus on quality control, assessing the treatment plan and evaluating the consensus in VS contouring by each physician. The study aimed to isolate all sources of variability, except the intra- and inter-observer variability, ensuring that the features of the VMAT treatment plan, optimization, imaging modalities and even, MRI-CT registration are uniform. In addition to the volume comparison, attention was paid to the dosimetric impact of their variation. However, the small sample size given the single center nature of this study is a limitation when considering the resulting AVIs.

From the results obtained, the dose coverage of VS appears to be robust to inter-observer variations, although the impact on some OARs may exceed the established limits. In this study, observers appear to pay more attention to the zone of the OARs limiting that of the VS than to the OARs as a whole, particularly in the case of the brainstem.

We believe that this exercise in quality assurance is particularly useful to ensure quality control at centers where several physicians perform contouring for techniques as sensitive as SRS, especially when dealing with VS given the small volumes involved. This is particularly critical when handling several imaging modalities, as in this case where three MRI modalities and a CT approach are employed.

Although there is a procedural consensus among physicians, the results of this modest assessment show that PTV coverage was strong, yet this is not the case for some small OARs, requiring a reinforcement of the quality control program in this regard. In our oncological radiation department, daily clinical review meetings are held to ensure the quality of radiation care prior to the initiation of treatment, and based on the feedback received in a team environment [19]. In these daily meetings, all case contours to be planned are reviewed but in the case of SRS with a head frame, this is not possible as the entire procedure is performed on the same day. Hence, and in the light the data presented here, it has been decided that an additional physician with experience in VS planning will always review all SRS VS delineations prior to the dosimetry task. In addition, regular meetings of the physicians involved in the planning of VS and retrospective presentation of VS cases at the daily peer review meeting have also be adopted to keep the consensus always up to date.

An important limitation of this study, in addition to the small number of cases evaluated, is the absence of a re-evaluation of intra- and inter-observer variation once the quality measures have been adopted. Even if there is total agreement on the contours by the two physicians, it is clear that it is not an objective measure. Thus, the present study just highlights the need to implement quality control procedures to reduce the variation in contouring in our department.

The impact of variability should also be characterized through its radiobiological effect rather than by any physical dose deviation, i.e. its effect on tumor control probability (TCP) and normal tissue complication probability (NTCP). Soltys et al. [4] showed the limitations in extracting information for biological analysis of SRS for VS, which are mainly driven by the poor quality reporting of dosimetric data associated with statistically valid and well-defined definitions of tumor control. As an example, we present an estimation using values extracted from a previous study Soltys et al. [4]. The estimated range of TCP for the three patients studied here would be about 94.2%-95.4% (P1), 94.5%-95.0% (P2) and 87.1%-93.1% (P3) when the LQ_L method is used, due to the range of variation of Dnear-min, in all cases relatively high values except for that of the worst case (P3). In terms of the NTCP, in a study by Woods et al. [20] comparing SRS with another technique, they found that a reduction of the mean cochlear dose from 6.29 Gy to 4.25 Gy that led to sensorineural hearing loss, and a NTCP reduction from 40.8% to 30.8% at 3 years and from 61.7% to 43.3% at 5 years. In the light of this data, the range of cochlear values observed here does not imply a significant variation in the NTCP. For the brainstem, maximum doses taking into account inter-observer variability are in all cases below 12 Gy (the maximum dose received is 11.98 Gy for P3 and one observer), so the risk of symptomatic radionecrosis would be expected to be low (<5%) [21].

CONCLUSIONS

For the resulting inter-observer volumes, the AVI value is notably higher in the VS than in the OARs and in turn, it augments as the volume increases. Due to their size, the AVI is noticeably lower in the smaller OARs (cochlea and trigeminal nerve). The AVI resulting from the intra-observer study is notably higher than that of the inter-observer study. VS dose coverage is fairly robust to inter-observer variation, although the impact on some OARs causes the established limits to be exceeded in some cases.

In the light of this study, it has been decided in our radiation oncology department to establish that all SRS VS delineations, and in general all frame-based SRS contouring, will always be reviewed by an additional physician with experience in VS planning. In addition, regular clinical meetings to address VS planning and retrospective reviews of VS cases at daily meetings also ensure that consensus is up to date.