Analysis of regional radiotherapy dosimetry audit data and recommendations for future audits

Abstract

Objectives

Regional interdepartmental dosimetry audits within the UK provide basic assurances of the dosimetric accuracy of radiotherapy treatments.

Methods

This work reviews several years of audit results from the South East Central audit group including megavoltage (MV) and kilovoltage (kV) photons, electrons and iodine-125 seeds.

Results

Apart from some minor systematic errors that were resolved, the results of all audits have been within protocol tolerances, confirming the long-term stability and agreement of basic radiation dosimetric parameters between centres in the audit region. There is some evidence of improvement in radiation dosimetry with the adoption of newer codes of practice.

Conclusion

The value of current audit methods and the limitations of peer-to-peer auditing is discussed, particularly the influence of the audit schedule on the results obtained, where no “gold standard” exists. Recommendations are made for future audits, including an essential requirement to maintain the monitoring of basic fundamental dosimetry, such as MV photon and electron output, but audits must also be developed to include new treatment technologies such as image-guided radiotherapy and address the most common sources of error in radiotherapy.

There are numerous recommendations specifying that radiotherapy departments should take part in external quality control audits. These include radiation incident reports [1], guidance to improve safety in radiotherapy [2] and UK radiotherapy performance measures [3]. However, there is little evidence-based guidance as to the nature and scope of the required audits, their frequency and the achievable tolerance, nor a detailed analysis of the value of a substantial body of interdepartmental audit results or where auditing activity should be focused in the future. Indeed, the latest radiotherapy peer review measures [3] simply state that radiotherapy departments “should have taken part in the external quality control programme, as agreed by the network”, leaving individual networks to agree both the scope and the frequency of audit. The 2008 joint Royal College of Radiologists publication “Towards safer radiotherapy” [2] advocated the use of dosimetric audit networks.

The South East Central Regional Audit Group (also known as Group E, and previously the Oxford Group) is the largest in the UK, now comprising 11 radiotherapy centres, listed in Table 1, and has an active programme of audits and a very high compliance rate, almost 100% over the last 5 years. This paper analyses a previously unpublished comprehensive set of interdepartmental dosimetry audit results from this group over the last 6 years and compares them with some historic audits. The results provide information on the agreement of basic radiation dosimetry parameters between centres in the audit group, for the majority of treatment modalities currently in clinical use: megavoltage (MV) and kilovoltage (kV) photons, electrons and low dose-rate permanent iodine-125 seed implants, and how this agreement has varied with time. These data therefore contribute to the body of evidence on the consistency of delivered radiotherapy treatment doses between centres in the UK. This adds to the data published by the South West Regional Group [4]. More importantly, the data are also used as a basis to make recommendations for future audits. Data from audits within the south east central region for specific clinical trials, or for ad hoc audits on new equipment installations, are not included in the analysis. However, data from the audit measurement programme of the National Physical Laboratory (NPL), Teddington, UK, are included when directly relevant to the interdepartmental audits.

Table 1. Radiotherapy departments within the South East Central Regional Audit Group.

Imperial College Healthcare NHS Trust, Charing Cross and Hammersmith Hospitals

The Cromwell Hospital, London

The Royal Surrey County Hospital NHS Trust, Guildford

East and North Hertfordshire NHS Trust, Mount Vernon Cancer Centre

Oxford Radcliffe Hospitals NHS Trust, Churchill Hospital

Portsmouth Hospitals NHS Trust, Queen Alexandra Hospital

Royal Berkshire NHS Foundation Trust, Reading

The Royal Marsden NHS Foundation Trust, Chelsea, London

The Royal Marsden NHS Foundation Trust, Sutton, Surrey

Southampton University Hospitals NHS Trust, Southampton General Hospital

The London Clinic, London

The central purpose of interdepartmental dosimetry audits is to establish the variation between centres in fundamental dosimetry parameters used in radiotherapy treatments, with the aim of detecting any systematic errors, investigating further any smaller discrepancies between centres and improving overall accuracy in radiotherapy. A key outcome of the process is to demonstrate the consistency and absence of discrepancies between centres. There are two possible methods of audit which may be employed: (a) the single centre (reference centre) approach, in which audits are performed by only one department, which conducts audits at each of the other departments in turn; and (b) the round-robin (peer-to-peer) approach, in which all departments conduct and receive audits, with every department performing an audit at another department according to a pre-defined schedule. The latter method has been used for the majority of audits considered in this paper within the South East Central Regional Group. The only exception is the iodine-125 seed audits in which a calibrated seed was transferred between centres, rather than an audit by a visiting centre. In some cases, a traceable calibration to the NPL national standard has been available and is presented, but this is frequently not the case.

The effectiveness of dosimetry audits in their present form in detecting machine calibration-related errors has been previously reported [5,6]; however, there is limited evidence to show the impact of audits on the rest of the radiotherapy chain. In this paper we consider proposals for expanding the scope of audits to cover more aspects of the radiotherapy process. We also consider whether and how the scope of future audits may be modified to re-align with current advanced treatment techniques and technology, and address auditing requirements that may arise in response to the most common radiotherapy-related incidents. We also comment on how improved compliance may be achieved for interdepartmental audits and discuss minimum audit standards.

Methods and materials

The majority of the regional audit schedules were arranged such that every radiotherapy centre would visit another centre to perform an audit; thereby each centre would take part in audits as both visitor and host. The alternative single-centre audit approach [7] may enable improved consistency of results and easier analysis, but at the expense of significantly increased audit burden for one centre and increased overall time to complete the audit, and was therefore considered impractical in the south east central region. The former approach, often termed “round-robin”, is common among the regional audit groups in the UK. The only exception is the iodine-125 seed audits, in which the seed itself was transported between the centres.

Measurements were made by the visiting centres using ionisation chambers and electrometers with traceable calibrations to the national standard at NPL, in accordance with the relevant UK code of practice (CoP) for the energy and modality considered. Host centre measurements were made with equipment used locally, also with traceable calibrations to the NPL. Ancillary checks were also performed during all audits, including independent temperature and pressure measurements, and set-up alignment checks.

Where audit measurement results exceeded the defined protocol tolerance, supplementary investigations were always immediately conducted by host and visiting centre to identify the source of discrepancy. It is the host centre’s responsibility to determine the necessity for, and arrange, further investigations for any results that are not explained by the initial examination on the day of the audit.

During 2006 to 2010, NPL reference dosimetry audits have been used to “tie-in” regional audit programmes with a “gold-standard” measurement. The NPL measurement programme aimed to perform an MV, kV and electron audit at one centre in each regional audit group each year, although such audits have been less frequent in recent years.

MV audits

MV photon audits have been conducted within the audit group for many years. Each audit has used Farmer-type thimble ionisation chambers connected to a secondary standard grade electrometer [8], with the combination’s calibration being directly traceable to the NPL primary standard. Calibration factors for the visiting centre’s equipment used in the host centre’s treatment beam are calculated by interpolating the auditor’s calibration factors as a function of quality index using a polynomial fit to the data to derive calibration factors for the host’s treatment unit. A protocol acceptable tolerance of ±2% agreement between auditor and host MV standard output measurements has been used since 2005, and ±3% used prior to 2005.

Basic audits were conducted in 1993, 1995, 1999, 2000 and 2008, comprising simply measurement of the linear accelerator’s output under standard conditions. More complex audits were conducted in 2005, 2006 and 2007, as described below.

In the 2005 audit, clinically typical multileaf collimator (MLC)-shaped fields were planned on the host’s treatment planning system and compared with doses measured in solid water by the visiting centre. MLCs were fitted to 6 fields (shapes adapted from [9]), which included a 10 × 10 cm square field, 5 × 5 cm square field, circle of diameter 5.6 cm, Y-shape 15 cm long and T-shape 12 cm long, the last one planned with and without a wedge (at the planned collimator angle only). An acceptable tolerance of ±2% agreement was used.

In 2006 and 2007, an MV standard output measurement and a rectangular field calculated by the host’s treatment planning system were audited. An open rectangular field was used in 2006, and a wedged rectangular field in 2007. The audit in 2007 was completed as part of the UK national photon audit and included a measurement of photon energy (TPR20/10) and ion recombination. Acceptable tolerances of ±4% and ±3% were used for the planned field in 2006 and 2007, respectively.

kV audit

kV treatment units were available in six radiotherapy centres in the South East Central Audit Group during the period of the audit in 2008 and 2009. The centres audited had between one and three kV energies in clinical use. A total of 10 treatment beams were audited, in the energy range 1.4 mm Al to 2.3 mm Cu half-value layer (HVL), with accelerating voltages of between 70 and 300 kV. The audit consisted of a HVL consistency check, standard output measurement in the reference field (10 × 10 cm or equivalent), an applicator factor measurement in a non-reference field which was significantly different from the standard, and a calculated treatment using another applicator with stand-off and a shielding cut-out.

The audit schedule was designed such that auditors would make measurements at similar HVLs to their own equipment, enabling measurements within an appropriate range of their own chamber calibration factors, which were all based on the in-air CoP [10]. Energies calibrated via the in-phantom CoP [11] were excluded from the audit, as this would cause inconsistencies in dosimetric comparisons. An audit spreadsheet was used to facilitate rapid data analysis and minimise calculation error. The audit protocol stated acceptable agreement limits between auditor and host of ±3% for HVL, ±3% for standard output, ±2% for applicator factor and ±5% for planned field. In addition, the NPL measured the standard output and HVL at one centre within the region as part of their annual reference dosimetry programme, coinciding with the regional kV audit.

Electron audits

In 2000, an audit of centres calibrating linear accelerators via the 1996 electron dosimetry CoP [12] was undertaken. A single NACP-02 parallel plate ionisation chamber, electrometer and water equivalent phantom, loaned from 1 of the region’s radiotherapy centres, were passed between 8 centres resulting in a total of 19 measurements at 6 and 15 MeV. The chamber and electrometer were cross-calibrated with a NPL-calibrated secondary standard chamber at the first centre, this value for N_d,air was used at all centres, although the cross-calibration was repeated at the fourth centre to check the calibration factor. Correction factors for each nominal energy, including polarity and ion recombination, were measured at the initial centre, and these factors were used throughout the audits. The visiting and host centres both measured the standard output on the day of the audit.

In 2006 and 2007, an audit of four centres that calibrated linear accelerators via the 2003 Institute of Physics, Engineering and Medicine (IPEM) CoP for electron dosimetry [13] was undertaken. Measurements were made at two energies using the visiting centre’s parallel plate ionisation chamber, electrometer and water equivalent material, the depth of which was adjustable to the nearest millimetre. Calibration factors were determined from the visiting centre’s calibration factors (N_D,w) for the chamber, by plotting N_D,w against R_50,D, the depth at which the dose is 50% of the maximum for the depth–dose curve in water, for a range of nominal energies. The visiting and host centres both measured the standard output on the day of the audit, at reference depth, d_ref. The host centre’s value for R_50,D was verified as the basis for their d_ref calculation via the auditor’s measurements of the depth of maximum dose, d_max, and R_50,I, the depth at which the ionisation is 50% of the maximum for the depth–ionisation curve in water. All parameters are as defined in the 2003 IPEM CoP for electron dosimetry [13]. An acceptable tolerance of ±2 mm between host and auditor values for R_50,D was used. Ion recombination corrections, evaluated using the two-voltage technique, were also measured by the auditing centre. This was done to account for any affect on chamber calibration of differences in dose per pulse between centres. All measurements were taken at the usual treatment distance and reference field size for the host centre, at least 10 × 10 cm. An acceptable tolerance of ±2% was set for output measurements.

In 2009, electron audits were performed according to the UK national electron audit protocol. This was again based on the 2003 IPEM CoP for electron dosimetery [13] and was similar in method to the 2006/7 audit described above (having been based on this audit protocol). However, measurements were made in water where possible and R_50,D measurements were made at a larger field size as recommended in the CoP.

Iodine-125 seed audits

Audits using a manufacturer-calibrated iodine-125 seed were performed in 2007 and 2009 by six centres within the group that offer prostate brachytherapy treatment services using permanent seed implants. In both audits a calibrated seed was purchased by one of the participating centres and sent round to each centre to perform source strength measurements using their well chambers and relevant calibration factors. The ionisation chambers used in the audits included the PTW GmbH Source Check, Standard Imaging Inc. HDR 1000 Plus and IVB 1000 well chambers. The calibration factors used for the well chambers by different centres are either NPL-derived or manufacturer supplied, and are specific to the seed type used clinically at each centre.

An iodine-125 Oncura Oncoseed was used in 2007, and an iodine-125 Isotron SelectSeed was used in 2009. In the latter audit the seed was also sent to the NPL for source strength measurement. Comparisons were made to the manufacturer’s source certificate and to the NPL measurement. A tolerance of ±5% was expected for agreement between the audit centre results and the NPL measurement.

Results

For the MV, kV and electron audits, only the magnitude of the percentage difference between auditor and host centre is reported, rather than the sign of the difference. The mean difference is therefore calculated as the average deviation from exact agreement, e.g. the mean deviation calculated for one centre with −1% deviation and another with +1% deviation would be 1%, rather than 0%. Similarly, only the magnitude of the deviation of NPL measurements compared with the visited centre’s results are included in the data. However, for the iodine-125 seed audit results, where an absolute “gold standard” is known for comparison at each audit, the sign of the difference is also included in the data.

MV audits

The results in Figure 1 show that the agreement between host and visiting centres over the last 8 years has had a maximum deviation of 2.7% for standard output measurements and is frequently within 1.5%; thus, always within the tolerance set for the regional audit protocols. There is an indication of improving agreement with time, although only a limited set of measurements were made in 2008.

Figure 1.

Results of the megavoltage (MV) photon audits completed between 1993 and 2008, showing individual audit results, mean audit results and National Physical Laboratory (NPL) audits in 2006. (Limited data for 2008 as an MV audit was only undertaken as an alternative to the kV audit conducted that year).

Figure 2 shows the results of the treatment planning system-calculated MLC shapes and rectangular field audits in 2005, 2006 and 2007. The 2005 MLC-shaped planned field results had a mean deviation of 1.5%, with the majority of results within the protocol tolerance of 2.0%, and five results greater than 2.0% but less than 3.3%. The MLC shapes were more complex than the standard rectangular fields used in other audits, which may contribute to the differences reported. The results that exceeded tolerance were further investigated by the host centre after the audit via a review of their treatment planning system commissioning data and additional measurements close to MLC field edges.

Figure 2.

Results of the megavoltage audit for treatment planning system calculated multileaf collimator-shaped fields (2005) open rectangular field (2006) and wedged rectangular field (2007).

The 2006 open-rectangular planned field results show good agreement, with a mean difference of 0.9% and the majority of differences less than 1.5%. In 2007, the wedged rectangular fields had a mean difference of 1.3%, with the majority of results within 2.0%, and one result exceeding the protocol tolerance value at 3.3% difference. The last result was further investigated by the host centre after the audit via a review of their treatment planning system commissioning data and additional measurements under wedged fields.

The TPR20/10 measurements undertaken in 2007 showed good agreement between host and visiting centre with a mean difference of 0.4%, standard deviation of 0.3 and all results within 1.2%. Temperature and pressure measurements were all within the tolerances of ±3 mmHg and ±1.0°C.

kV audit

Figure 3 shows the results of the 2008–9 audit programme of kV treatment units. The final results of all audits were within the acceptable tolerances stated in the measurement protocol. Three audits required immediate investigation by host and visiting centre on the day of the audit to resolve discrepancies that exceeded tolerance. In each case, audit process errors were identified and resolved, with only the final audit results being presented here.

Figure 3.

Results of kilovoltage audits conducted during 2008 and 2009 showing individual audit results for the four measured parameters and National Physical Laboratory (NPL) audit results where available. HVL, half-value layer.

For standard output measurements, the majority of the auditor’s results agreed with the host centre’s measurement within 1%, with only two energies at one centre greater than this but less than 3%. A greater spread of results for HVL was observed, although all the differences were within 2.5%. Applicator factor differences were all within 1.4%. The auditor measurements of planned dose were all within 2.1% of the expected value when corrected for daily output variation.

The results of the NPL audit at one centre are also shown in Figure 3. The three energies audited were all within 1.1% of the host centre measurement for output and 0.8% for HVL.

Electron audits

Figure 4 provides the results of the 2000, 2006 and 2009 electron audits. In 2000, the maximum difference in the measured standard output between visiting and host centres was 4.2%. The second cross-calibration produced a value for N_d,air that was +1.4%, different from the one used for the audits. In 2006 and 2009, the maximum differences were 1.5% and 1.8%, respectively. An NPL audit of one of the centres in 2006 showed all electron energies were within 0.9%.

Figure 4.

Results of electron audits conducted during 2000, 2006 and 2009. (For clarity, only the mean National Physical Laboratory (NPL) audit result is plotted in 2006).

Iodine-125 seed audits

In both the 2007 and the 2009 audits, all results were within 2.8% of the manufacturer’s source certificate values, as shown in Figure 5. The 2007 results had a mean value of −0.4% and the 2009 results had a mean of 0.7%. The lines in Figure 5 link the individual centre results for the two audits conducted. The trend shows a relatively higher measured seed activity for most centres in the 2009 audit. This may be due to uncertainties in the manufacturer’s quoted seed activity. There was up to 4.9% difference between the audit centre results and the NPL result in the 2009 audit, with the audit centre results all being higher than the NPL measurement.

Figure 5.

Results of the iodine-125 calibrated seed audits conducted in 2007 and 2009 showing individual centre results, their mean and the National Physical Laboratory (NPL) calibration result. The results from each individual centre are linked on the diagram.

Sources of uncertainty

In general, interdepartmental audit involves comparing two identical dosemeters with calibrations traceable back to a common calibration standard and using the same dosimetric protocol. In this situation, most systematic uncertainties due to the calibration process will cancel, leaving only random uncertainties or systematic errors not inherent in the calibration protocol. The possible sources of uncertainty in the MV, kV and electron audits include chamber-positioning errors, temperature and pressure measurement uncertainties, monitor unit (mu) fluctuations and uncertainty in the calibration factor derivation. For the MV planned dose there are also uncertainties associated with the treatment planning algorithm or tabulated data used. Ion recombination uncertainties also exist for MV photons and electrons. Differences in solid-water material can be significant for electrons, as well as the omission of ion recombination measurements in host beams with earlier measurement protocols. For HVL measurements, the purity of the filters used in the measurement needs to be established. The uncertainties in the iodine-125 audit are due to the manufacturer’s seed calibration accuracy, the calibration factor used, the seed positioning in the chamber and axial correction factor, as well as the source strength at the time of measurement. The different sources of uncertainty affecting the reported audit results are presented in Table 2. The total uncertainty was estimated by adding the individual uncertainties in quadrature to give total uncertainties of 1.3% for MV photons, 1.7% for kV photons, 1.6% for electrons and 2.2% for the iodine-125 seed audits.

Table 2. Percentage uncertainty estimates for megavoltage (MV), kilovoltage (kV), electron and iodine-125 seed audits.

	Uncertainty estimate (%) (95% confidence level)
Source of uncertainty	MV photons	kV photons	Electrons	Iodine-125 seeds
Chamber/seed positioning	0.8	1.4	1.2	1.5
Temperature and pressure	0.5	0.5	0.5	0.5
Exposure fluctuations (mu, time, strength)	0.5	0.5	0.5	1.0
Calibration factor	0.7	0.5	0.7	1.2
Ion recombination and polarity	0.3	0.3	0.4	–
Chamber/meter response	0.2	0.2	0.2	0.2
Differences in solid water phantoms	–	–	0.2	–
Total (root mean square) (%)	1.3	1.7	1.6	2.2

mu, monitor units.

A balance has to be achieved between making additional or more comprehensive measurements to reduce uncertainties with that of minimising the time required to undertake the audit to ensure it is practical within the regional audit group system and that a high completion rate is achieved.

Discussion of results

MV audits

The results of all basic standard output MV audits have been within protocol tolerances, indicating consistent MV dosimetry between centres since the 1990s. Considering the number of audits that have taken place at MV photon energies, there has been a very low incidence of detection of any systematic error, or any significant variation between host and visiting centre measurements.

In 2005 a significant increase in complexity was introduced in measuring the dose predicted by the treatment planning system for a field defined by a series of MLC shapes. A minority of results were outside the protocol tolerance level and these were investigated further by the host centres. The results of these additional measurements indicated the discrepancies were the result of minor inaccuracies of the treatment planning system model for dose point calculations at positions within 3 cm of MLC shielding edges.

In 2006, an incorrectly calibrated mercury-in-glass barometer was discovered during an MV audit, leading to an error of 6 mmHg. The host centre had corrected the instrument reading to a calibration value from the UK Meteorological Office. However, no account had been taken of the height of the radiotherapy centre above sea level in using the calibration value. The barometer checks at the same centre the following year showed good agreement with the visiting centre.

The audits carried out in 2006 and 2007 included measurements of the dose predicted by the treatment planning system for a wedged field. All results were within protocol tolerances except one, at 3.3%, which was further investigated by the host centre. An inaccuracy of the treatment planning system model at large off-axis positions in wedged fields, an issue already known to the host centre, was attributed as the cause of the discrepancy.

Based on the achieved MV audit results between 1993 and 2008, an investigation tolerance of ±1.5%, with a maximum acceptable deviation of ±2%, is proposed for future audits.

kV audits

Only one set of comprehensive kV audit data was available within the group; however, every parameter measured in each audit gave results within the audit protocol tolerances, demonstrating good consistency between radiotherapy centres of absolute dosimetry at kV treatment energies.

The results of three audits were immediately investigated further by the host and visiting centre due to initial results being larger than the protocol tolerance value. In each case, the discrepancies were resolved on the day of the audit and attributed to suboptimal purity of HVL material, accidental omission of temperature and pressure correction factor, and inaccurate quotation of reference HVL value by host centre.

Electron audits

No tolerance was set for the audit in 2000, but all results were within 4.5%. All results in 2006 and 2009 were within protocol tolerances. There is a statistically significant improvement in the audit results in 2009 compared with 2000 (t-test p-value of 0.0018), perhaps as a result of the implementation of the newer 2003 CoP [13] during this period. The newer CoP provides a much simpler formalism and significant reduction in the uncertainties involved in the determination of absorbed dose in reference conditions compared with the previous approaches based on air kerma chamber calibrations. There was no significant variation with energy. The audit results were verified by the NPL in 2006.

The significant difference in N_d,air at the second cross-calibration was found to be mainly due to variations in the ion recombination factor; 1.007 compared with 1.020, a difference of 1.3%, in the 2009 audit. The ion recombination correction varies with dose per pulse and is often different between centres, ranging from 1.005 to 1.022 in the electron audit results within this group over the 3 years. This may account for some of the variation in output between visiting and host centres.

In 2009 the results are consistent with those recorded in 2006. This is as expected over a gap of only 3 years with no major changes in dosimetry techniques. However, more measurements were carried out at higher energies in 2009 and there is a statistically significant increase in the magnitude of difference between host and auditor measurements at the higher energies with R_50,D greater than 6.60 cm compared with those with less than or equal this value (t-test p<0.0001). All electron calibrations are traceable to the secondary standard calibration at the NPL. The NPL provides a table of N_D,w against R_50,D for the calibrated chamber, but this only covers a range up to 6.60 cm, requiring extrapolation by both visiting and host centres for higher energies, using stopping power ratios and assuming no change in chamber perturbation corrections [13]. Although sensitivity analysis indicates that this should not have a significant affect on the overall uncertainty, further work is required to confirm and establish the cause of larger discrepancies in audits at higher energies.

Results have consistently been within the tolerances set; however, even with the 2003 CoP there is a greater potential for error with electron dosimetry than with MV photons owing to the steep dose gradients, varying energy with depth and possibility of larger chamber correction factors such as polarity and ion recombination.

1-125 seed audits

The iodine-125 seed audit results were all within the required protocol tolerances from the NPL calibration in 2009 and from the manufacturer’s source certificates in 2007 and 2009. A tolerance of ±7% is normally quoted by the manufacturers for seed activity, and in practice all seed activity measurements fall within this tolerance.

The relatively large spread in audit results between centres for the calibrated seeds can be attributed to several factors: many of the audit centres’ well chambers were calibrated for different types of seeds to the one measured in the audit. There is also currently a lack of consistency in the source of calibration factors in use at each centre. The absence of definitive UK national guidance on the calibration of well chambers for measuring iodine-125 seeds has led to the use of either a manufacturer-supplied chamber calibration factor or an NPL-derived calibration factor. The main challenge in this type of audit has been the requirement to get the seed to all participating centres while it is still of high source strength, and this has been difficult to achieve in some cases, leading to measurement of a low activity seed by a few centres. This uncertainty has to be taken into account in interpreting these results. Well chambers in routine use are designed to accurately measure a strand containing either 5 or 10 seeds at the same time. The measurement of a single seed requires the chamber’s axial response to be accurately known at the point of measurement. Uncertainties in the axial response correction and in the source placement relative to the point of correction may lead to inaccuracies in the measured seed activity.

Further investigation of these sources of uncertainty, with calibration guidance and consistency should improve the situation and reduce the differences between centres’ measurements of iodine-125 seed activity in the future.

Comparison to other interdepartmental audits

In 2007 a UK national MV audit was undertaken, which was implemented by the existing regional audit groups [14]. The majority of comparisons of standard outputs between host and visiting centres gave results within 1.0%, with a maximum deviation of 2.1%. The results presented in this paper for the South East Central Group are consistent with this national picture.

A kV audit conducted in the South West Regional Group in 2005 [7], in which a single centre audited nine other radiotherapy units, showed a maximum standard output difference between visitor and host of 2.5% with a mean difference of 0.9%. Differences in measured-to-predicted applicator factors had a maximum of 3.6% and a mean of 1.1%. The results of the South East Central Group kV audit presented in this work show similar agreement values between host and visitor, even though the round-robin approach may be expected to result in greater deviations than the single-centre approach undertaken in the South West report.

Results of electron audits conducted in the South West Regional Group since 1993 were published in 2002 [4], and showed differences between host and visitor standard output measurement of up to 3.0%. While deviations of up to 4.5% were observed in the South East Central group during the 2000 electron audit, maximum errors of up to 2.0 %, with the majority within 1.5%, have been reported in the 2006 and 2009 electron audits.

There have been no results of audits involving iodine-125 seeds published in the literature. The results presented in this work, of up to ±3% deviation from the NPL calibrated value, may be taken as representative of expected tolerances in clinical use.

Reporting of results for round-robin audit systems

Using a simple analytical model, the effect of “who audits whom” in the peer-to-peer, round-robin style of audit was assessed. Nine theoretical centres were assigned random output calibration errors of between ±2%, and then two different audit schedules, “audit schedule 1” and “audit schedule 2”, that were randomly devised. As Figure 6 shows, in the absence of an NPL validation, significantly variable results are obtained depending on the audit schedule used. The model results show that the output range between centres can either be half or twice the true NPL validated range depending on the audit scheduled. In the absence of an NPL measurement to standardise the results, an absolute difference-reporting methodology rather than including the sign of the difference should be adopted as it provides a more representative indication of the limit of the information that can be inferred from the result. It can also be misleading to quote mean values for agreements in audits, which is frequently done in other published work on audits. The mean is not a sensitive indicator where both positive and negative deviations are quoted, it is much better to take the mean of the magnitude of the audit results.

Figure 6.

Results of an analytical model showing the possible variability in audit results due to audit schedule. A National Physical Laboratory (NPL) audit is required to standardise and deduce the correct variation of centre results.

To prevent the reporting of misleading results, it is recommended that all audits should include an NPL measurement. This has implications for the ad hoc auditing of new radiotherapy equipment in the UK, which usually falls outside the national dosimetry audit schedule, and hence may not have a direct link to an NPL measurement for standardisation. Prior to such audits the visiting centre should, if possible, verify the variation of their own centre measurement to an NPL measurement. Ideally only centres with recent NPL audits should perform audits at other centres. If a link to NPL audits is not possible, then the limitations of the centre-to-centre auditing approach should be clear.

Discussion of future audits

MV audits have traditionally been the mainstay of an interdepartmental audit protocol for a number of reasons: all centres have MV treatment equipment, MV audits are generally more straightforward than other audits such as kV and electron audits, calibration factors for MV audits are generally well known with many years’ experience of MV dosimetry and MV treatments make up the majority of radiotherapy activity. However, the quantity of previous MV audits, and the proven agreement between centres in the presented results, should not undermine the value of continuing basic MV dosimetric comparisons in the future, as the potential consequences of any significant systematic error far outweigh the likelihood of such an error being present.

There are, however, options to change the scope of an MV audit from a simple “physics-based” dosimetry audit to full multidisciplinary systems analysis. A recent MV audit was completed at Portsmouth Trust involving the whole treatment chain, including CT scanner, treatment planning system, linear accelerator delivery, imaging and in vivo diode confirmation, as part of the commissioning of a new radiotherapy centre. Oncology staff as well as physics staff were involved in the audit and the results showed agreement between prescribed and measured dose within 2%. This format of comprehensive audit is recommended for MV audits in the future.

In electron audits a significant proportion of audit time is dedicated to measuring and verifying dosimetric parameters such as R_50,D, D_max, Z_ref, etc. This verification could be performed prior to the audit by theoretical derivations from percentage depth–dose profiles and comparisons to previous audit information for the same type of machine and energies. For such an application previous audit results should be used to create a database of dosimetric information. Prior machine and energy parametric comparisons and trend analysis would be required, as well as subsequent regular updates.

With the expansion of permanent prostate brachytherapy services in the UK, a number of treatment planning systems using TG43-based calculation algorithms are now in use. Future iodine-125 seed audits could involve an independent check of the planning system dose calculation, possibly using a simple single seed geometrical calculation. There is also a need to include high-dose rate (HDR) treatment planning systems and treatment delivery units in future audit intercomparisons.

Where possible, the involvement of the national dosimetry standards laboratory, NPL, should be included within regional interdepartmental audits. This is to ensure the absolute accuracy of a particular department’s audit results can be established, by reference to a “gold standard” rather than limiting audit results to only an indication of the relative agreement between two centres.

There are also options to re-align audits to check the accuracy of parameters that contribute most frequently to treatment errors. A National Patient Safety Agency report [15], based on a sample of 197 radiotherapy-related incidents, indicated that 48% of radiotherapy incidents were due to actual or potential inaccuracies in radiotherapy delivery. Of these, alignment errors and calculation errors were most frequently reported. While it is assumed that these are not the result of systematic errors, and therefore are difficult to detect on sample audits, any comprehensive auditing programme should check the integrity and reliability of alignment and calculation methodologies.

It is also important to ensure interdepartmental audits are complementary to any other dosimetry audits that may take place, such as those provided by clinical trials quality control (e.g. [16]) or other processes.

Conclusion

It is essential that the external audit is undertaken on a regular basis to ensure consistency of radiotherapy dosimetry and minimise the likelihood of errors, and we are fortunate in the UK to have an established regional audit group network to undertake this work. Such audits provide reassurance, confidence and demonstrable quality in absolute dosimetry for radiotherapy patients. It is valuable to review audit data from regional groups over time and identify changes to processes that will improve the service. Publication of audit results promotes audit activity and provides a wider appreciation of the agreement in basic radiation dosimetry parameters between centres.

It is possible to achieve high audit compliance rates, as evidenced within the South East Central region, if realistic and relevant objectives are set and audits are carefully prepared. The data presented in this report demonstrate what is typically achievable from regional audit groups. All results have been within tolerance values for MV, kV, electrons and iodine-125 seeds, with a small number of minor systematic errors also having been detected and investigated through the audit process.

Practical audit recommendations

To enable practical implementation of an audit within busy radiotherapy centres, and ensure good compliance rates, the procedure should take no more than 4 h of machine time and use comprehensive protocols and recording spreadsheets. It is recommended that audits should comprise basic checks on absolute dosimetry under standard conditions, which should be repeated each year at different energies and on different treatment units. The audit should then have additional elements to investigate relevant parameters; those critical to patient safety, known areas of potential problems, technological developments or aspects related to common radiotherapy errors.

The round robin audit approach has limitations but has been shown to provide sufficient accuracy for the detection of minor errors, and will certainly report any significant systematic errors in radiotherapy dosimetry.

New radiotherapy technology

The current regional audit process is biased towards the recurrent auditing of existing systems and techniques, and needs to be extended to meet the challenges of advanced radiotherapy treatment techniques and technology. New treatment delivery systems such as four-dimensional gating, helical tomotherapy and volumetric modulated arc therapy require specific auditing procedures, in addition to the requirements of modern treatment planning techniques including intensity modulated radiotherapy (IMRT). Audits of image-guided radiotherapy systems, ranging from ultrasound to kV and MV cone-beam CT, would assist in the standardisation of imaging protocols and the quantification and consistency of concomitant doses. Positron emission tomography (PET) MRI is now increasingly used in target volume delineation in radiotherapy, requiring new and specific quality control and dosimetric checks. Task groups should be set up within the national or regional audit groups to develop and trial audit protocols for these new systems.