Case Volume and Experience in Stereotactic Radiation: Analysis of a Prospective Peer Review Program

Abstract

Purpose

Stereotactic radiation therapy (SRT) is highly effective but carries the risk of significant toxicity. We identified factors associated with SRT plans that require revision in a comprehensive, prospective peer review program conducted across a network of affiliated radiation oncology centers.

Methods and Materials

Weekly peer review rounds were conducted to review SRT cases prior to the start of radiation. Revision recommendations were tracked. Univariate and multivariable logistic regression was performed to identify factors associated with case revision.

Results

From 2019 to 2024, 1172 SRT cases were reviewed at weekly rounds, including 313 brain stereotactic radiosurgery (SRS), 190 brain multi-fractionated SRS, and 669 stereotactic body radiation therapy. The yearly revision rate ranged from 19% in 2020 to 31% in 2024. There were 16 individual treating physicians with a median of 6 years of experience (range, 1-19 years), measured at the time of each SRT case review. Factors assessed for significance included SRT case volume in the 3 months preceding review (dichotomized as low- or high-volume), physician experience (≤2, 3-9, or ≥10 years), SRT technique (SRS, multi-fractionated SRS, or stereotactic body radiation therapy, and disease site. On multivariable logistic regression, revisions were less likely for high-volume physicians (odds ratio [OR], 0.58; 95% CI, 0.43-0.77), those with 3-9 years of experience (OR, 0.65; 95% CI, 0.44-0.96), and SRS technique (OR, 0.59; 95% CI, 0.41-0.84).

Conclusions

These data imply high value to prospective peer review for physicians with low SRT case volume and in their early career. Adequate case volume may be a critical factor for high quality SRT, analogous to the surgical literature. Annual revision rates in the program remained substantial over time, demonstrating the ongoing importance of an effective prospective peer review program for SRT.

Introduction

Peer review is a critical strategy for catching errors and reducing risks for harm to patients undergoing radiation treatment.1, 2, 3 Stereotactic radiation therapy (SRT) is a highly effective treatment modality for primary tumors and metastases but has risks for serious toxicity including brain radiation necrosis, pulmonary hemorrhage, and even death.4, 5, 6, 7 Consequently, peer review of SRT treatments may be especially important compared to other forms of radiation delivery for ensuring both quality and patient safety. As SRT is delivered in 1 to 5 fractions in the United States, prospective peer review prior to delivery of the first fraction is critical for catching errors that may potentially impact patient safety. The benefits of prospective review are substantial, but this can be labor- and time-intensive, while also being difficult to implement in a robust manner. As a result, it has not been implemented widely, with few exceptions.8, 9, 10

A comprehensive, prospective SRT peer review program was established across a network of community-based radiation centers in 2019. Here, we present methods for implementation of SRT prospective peer review and an analysis of factors associated with cases deemed to require revision prior to treatment.

Methods and Materials

Peer review methodology

SRT peer review rounds were conducted weekly for 45 minutes on a virtual platform to prospectively review SRT cases before radiation start. Rounds were attended by physicians, physicists, dosimetrists, radiation therapists, and physician assistants, from a network of 4 affiliated radiation oncology centers. A report listing all newly simulated stereotactic cases was generated from treatment planning software in preparation for the meeting. At the meeting, a physicist queued radiation prescriptions, image fusions, contours, plans, and tabulated dose metrics (as available) for review via video conference software. Case presentations were systematic and comprehensive, starting with a brief clinical history, followed by prescription intent, assessment of fusion quality (eg, diagnostic brain magnetic resonance imaging with planning computed tomography), and target volume and critical organ at risk contours. Contour review was typically led by the primary physician scrolling through the volumes and soliciting comments or suggestions. Radiation treatment plan review was led by a physicist, displaying the achieved metrics (or dose-volume histogram) for target volume and organs at risk and the radiation plan. Cases without major revision recommendations were typically reviewed within 1-2 minutes, whereas those eliciting extended discussions took up to 10 minutes.

Contours and plans were required to be peer reviewed prior to the first fraction of radiation. If peer review at the weekly meeting was not possible (eg, because of the timing of treatment initiation), an offline review of contours and/or plan was performed by an appropriately credentialed peer physician for that specific body site outside of the weekly meeting. To become credentialed, one had to be proctored by a credentialed physician in contour and plan review for a set number of cases (ranging from 3-6, depending on body site). If contours were ready for review at rounds but not the plan, the plan was asynchronously reviewed either at the following week’s meeting or offline. Occasionally both contours and plan were reviewed offline (often due expedited start dates, meeting cancellation for holidays, staff unavailability, etc.). Revision recommendations for such cases were not rigorously tracked, and therefore, these cases were excluded from analysis.

Peer review rounds revision recommendations were logged prospectively. Completion of contours, planning, and peer review tasks were tracked and recorded within the treatment planning software. Approval for treatment at final physics check was contingent on verification of peer review completion.

Factors associated with revision recommendations

Recommendations for revision on peer review were evaluated for factors including body site, SRT technique, physician experience, and physician case load. Body sites included brain, head and neck (H&N), lung, abdominopelvic, spine, and nonspine bone. SRT techniques included stereotactic body radiation therapy (SBRT), single fraction stereotactic radiosurgery (SRS) to the brain, and multi-fractionated stereotactic radiosurgery (fSRS) to the brain. Physician experience was defined as years from completion of radiation oncology residency to date of peer review meeting. Case load was measured by the total number of SRT cases treated by the physician in the 3 months preceding the review date and dichotomized as less than or greater than the median. Factors were analyzed on a per-case basis. All cases were assumed to be independent.

Statistics

The odds ratio of case revision (and the corresponding 95% CI) with factors above were assessed using univariate and multivariable logistic regression. Significance was set at P < .05. Those variables significant in univariate analysis were included in the multivariable model. Analyses were performed in R version 4.4.0 (R Foundation for Statistical Computing).

Results

From March 2019 to April 2024, 1440 SRT cases were prospectively reviewed. Of these, 268 had both contour and plan review performed outside of formal weekly SRT rounds and were excluded. There were 1172 remaining cases, including 313 brain SRS, 190 brain fSRS, and 669 SBRT. Revision was recommended in 269 cases (23%). Yearly revision rates ranged from 19.1% in 2020 to 30.9% in 2024 (Fig. 1).

Figure 1.

Percentage of cases recommended for revision by year. Only first quarter data for 2024 were available at the time of analysis.

There were 16 treating physicians with a median of 6.3 years of experience (range, 0.8-18.8 years). Median year of graduation from residency was 2017. Physicians were categorized as junior (≤2 years of experience), mid-career (3-9 years), or senior (≥10 years), calculated by subtracting the date of graduation from the date of SRT peer review. Of the 1172 cases, 168 were treated by junior, 772 by mid-career, and 232 by senior physicians. The physician’s SRT case volume in the preceding 3 months (median 10 cases) was dichotomized as low volume (≤10) or high volume (>10).

Logistic regression results are shown in Table 1. Of note, only 4 cases of H&N SBRT were treated during the data collection period of this study. While H&N, lung, and spine anatomic sites showed significant associations on univariate analysis, anatomic site was not included on the multivariable analysis because of concern for bias from the low number of H&N cases. Lower revision odds were also found on univariate analysis for SRS technique, higher case load physicians, and mid-career physicians, and these factors remained significant on multivariable analysis. An exploratory multivariable analysis was performed including the anatomic site but excluding the 4 H&N cases (Table E1). Case load remained a statistically significant factor, while years of experience and technique no longer were.

Table 1.

Logistic regression results

	Univariate		Multivariable
	OR* (95% CI)	P	OR* (95% CI)	P
Technique
SBRT	Ref.		Ref.
SRS	0.52 (0.36, 0.73)	<.01	0.59 (0.41, 0.84)	<.01
fSRS	0.73 (0.49, 1.07)	.11	0.83 (0.55, 1.23)	.36
Site
Brain	Ref.
Head and neck†	13.72 (1.73, 279.11)	.02	-
Bone	0.95 (0.52, 1.64)	.85	-
Abdominal/pelvic	0.86 (0.32, 1.98)	.74	-
Lung	2.31 (1.71, 3.13)	<.01	-
Spine	0.32 (0.12, 0.69)	<.01	-
Case load in past 3 mo‡
≤10	Ref.		Ref.
>10	0.51 (0.38, 0.67)	<.01	0.58 (0.43, 0.77)	<.01
Years’ experience§
Junior (≤2)	Ref.		Ref.
Mid (3-9)	0.54 (0.38, 0.79)	<.01	0.65 (0.44, 0.96)	.03
Senior (≥10)	0.86 (0.56, 1.33)	.50	0.82 (0.53, 1.29)	.39

Abbreviations: fSRS, fractionated stereotactic radiosurgery; OR, odds ratio; SBRT, stereotactic body radiation therapy; SRS, stereotactic radiosurgery.

^⁎Reflects the odds ratio of making a revision compared to the reference group.

^†Only 4 cases for this location.

^‡Case load in past 3 months calculated by counting number of cases in the past 3 months from the date of current case for a given oncologist; 10 was the median case load in past 3 months.

^§Years of experience calculated by subtracting date of graduation from date case was reviewed for a given oncologist.

Revision rates for physicians with low versus high case load were 28.5% versus 16.8% (Fig. 2). Revisions rates for physicians ≤2, 2-10, and ≥10 years of experience were 31.1%, 19.7%, and 28.0%, respectively (Fig. 3). With respect to treatment technique, the revision rates for SBRT, SRS, and fSRS were 26.8%, 16.0%, and 21.1%, respectively (Fig. 4).

Figure 2.

Percentage of cases recommended for revision by treating physician’s case load. *Volume in past 3 months was calculated by counting the number of cases in the past 3 months from the date of peer review for a given treating physician; 10 was the median case load in the past 3 months, and low case load was considered ≤10 cases whereas high case load was >10.

Figure 3.

Percentage of cases recommended for revision by physician’s years of experience.

Figure 4.

Percentage of cases recommended revision by technique.

Discussion

The value of peer review rounds in radiation oncology is widely accepted for safety and quality purposes. In practice, these meetings often take place in the first or second week of radiation, after treatment has already started. Prospective review, prior to delivery of the first treatment, has the obvious advantage of allowing for revisions prior to treatment delivery but can be labor- and time-intensive. It may be of highest value for SRT plans with characteristically high dose per fraction, short courses, and high-grade toxicity. The purpose of this report is to share a methodology for implementing a prospective SRT peer review program, to identify characteristics of SRT cases more prone to revision recommendations and demonstrate the consistent value of such a program over time.

In this study, physicians with high case load volume (specifically, 10 or more cases treated in the prior 3 months) were less likely to have SRT cases flagged as requiring revision. Interestingly, physician case volume was an important predictor of revision rates independent of years’ experience. This suggests that programmatic attention to individual case load may be important for preventing errors in SRT. This is a familiar concept in the surgical literature where lower case volumes have been linked to higher complication rates or perioperative mortality across a variety of surgical procedures.11, 12, 13 Case volume was defined as the number of cases treated in the 3 months prior to review for several reasons. Analogous to the surgical literature, proficiency may relate to case volume.11, 12, 13 Furthermore, physician stereo case volume can vary over time for various reasons (eg, changing physician preferences, new referral patterns, or innovations in clinical practice, etc.), so a metric was chosen to represent physician case volume at the time of case review. In addition, labeling individual physicians statically as either high-volume or low-volume across the entire time may have led to confounding with physician seniority levels.

Notably, revision rates remained substantial even among the cohorts with the lowest revision levels, at 16.8% for physicians with higher stereotactic case load and at 19.7% for those with 3-9 years of experience. This confirms the importance of peer review even for more SRT-adept cohorts. Furthermore, overall revision rates remained stable over time, at about 20%-25%, demonstrating that maturity of the stereotactic radiation program did not reduce the need for peer review.

We found that mid-career physicians had lower odds of revision relative to junior physicians, although we acknowledge the relatively small number of physicians in this study (particularly for the junior and senior level subgroups). It is difficult to draw conclusions, but we can speculate on possible reasons. Senior physicians may have trained in an earlier era when SBRT and SRS were less common and may have less formalized instruction on these techniques, and junior physicians may have less experience with these techniques than mid-career physicians. Regardless, our data show the importance of peer review for all physicians.

Brain-directed SRS and fSRS cases were less likely to require revision in this study than SBRT, with SRS significantly so. It is difficult to draw firm conclusions from this, but SRS plans tend to be for small, discrete targets whereas fSRS cases are more likely to involve larger metastases, abutment of eloquent organs, or postoperative cavities where individual practices on cavity margins and target coverage can vary. The SBRT modality encompasses a wide variety of body sites, with some sites more complex to treat than others.

With brain site as reference, we found that lung SBRT cases were more likely to have suggested revisions whereas spine cases were less likely on univariate analysis, but these were not retained in the multivariable model. It is difficult to tease out with this type of analysis whether lung cases, for example, are intrinsically more error prone or rather that this is more a result of greater variation in physician preferences on how lung SBRT is performed, and similarly, whether there was greater spine SBRT uniformity or consensus. It would be difficult to draw any firm conclusions on H&N SBRT given the very limited number of cases treated. Of all SRT cases, 43% were brain SRS or fSRS, with the remainder split among the various other body sites. Numbers in these subgroups were likely too small to draw meaningful conclusions. Of note, anatomic site was excluded from the multivariable analysis because of concerns that the small number of H&N cases (3 of 4 required revision) would bias results. An exploratory multivariable analysis was performed including the anatomic site but excluding the 4 H&N cases, showing case load remained statistically significant whereas the odd ratios for junior versus mid-career and senior physicians, and treatment technique, no longer met significance.

Limitations of our work include the relatively small size of our experience, the performance of offline review, and the possibility of confounding factors. We were unable to quantify, for example, the amount of SRT-specific training individual physicians may have received during residency (or subsequently, for those who were not exposed to SRT in residency). Therefore, years of attending experience because residency may not be the best surrogate for SRT experience or expertise, though we did see a difference for the most junior physicians. Cases that underwent both offline contour and plan review were excluded from this study, limiting the analysis. Offline review was a practical necessity, because this was a weekly meeting, and for cases beginning treatment close to the day of the meeting or requiring more rapid turnaround (eg, brain-directed SRS and fSRS), peer review was prioritized over delaying treatment to match the logistics of the meeting. An additional limitation of this study is that reasons for revision and severity of potential errors were not consistently recorded (although they are in subsequent iterations of the peer review program). In future work, analysis of these details could potentially lend new insights into specific safety concerns during contouring or planning.

Although the study was limited to 16 physicians, the analysis was performed according to each individual case and revision, with a total of 1440 cases and 269 revisions. Therefore, each physician (with increasing seniority level and fluctuating case volume over the study period) contributed many useful datapoints to the analysis. In the end, the size of the data set was sufficient to show significant associations which may inform future work in peer review programs.

This SRT review program was initiated in 2019, prior to the COVID pandemic. It was conducted virtually from the outset, and as a result, the pandemic did not substantially impact its operation. Revision rates did not decrease over time, because one might expect given the growing experience with SRT of physicians and other team members during the period of the study. This suggests the ongoing importance of prospective peer review despite growing expertise in SRT. This may be explained by the context of ongoing advancements and changes including new technologies, evolving SRT research and best practices, and turnover of personnel.

There were additional, qualitative benefits to holding multidisciplinary prospective peer review rounds. The meeting created a venue for physicians, physicists, dosimetrists, and therapists to gather on a regular basis, teach each other, and discuss potential practice changes, allowing for practice evolution and alignment.

Accreditation organizations generally recommend weekly peer review rounds,¹⁴ and there is a growing movement for prospective peer review.¹⁵ As prospective peer review experience is gained, there may be room to consider it differently depending on the complexity of the contours or plan and the proficiency of the provider. It may not be feasible or palatable to reproduce this program in every practice setting given the time and resource requirements. In resource-limited settings, extra attention, and perhaps targeted review of cases with identified factors, may be a practical solution. This study suggests that providers with low case load may especially benefit from closer peer review. There remains a critical need for national or international evidence-based guidelines on peer review, and we hope that this work will aid in efforts to bring these forth.

Disclosures

Ayal Aizer reports a relationship with Varian Medical Systems Inc that includes: funding grants. Ayal Aizer reports a relationship with NH TherAguix that includes: funding grants. Ayal Aizer reports a relationship with Novartis that includes: consulting or advisory. Ayal Aizer reports a relationship with Seagen Inc that includes: consulting or advisory. John Phillips reports a relationship with Monteris that includes: consulting or advisory. John Phillips reports a relationship with UroGPO that includes: consulting or advisory. John Phillips reports a relationship with Ideology that includes: consulting or advisory. John Phillips reports a relationship with OneOncology LLC that includes: equity or stocks. John Phillips reports a relationship with Tennessee Oncology that includes: equity or stocks. John Phillips reports a relationship with Start up (unnamed) that includes: equity or stocks. Senior Editor for Online Longitudinal Assessment for the ABR – JGP. Travel support for attending ASTRO, SROA, and RO-HAC Retreat meetings – TK. Participates in Clarity PSO – TK. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.