Predictability and uncertainty in arteriovenous malformation radiosurgery

Abstract

Stereotactic radiosurgery (SRS) is an accepted management option for patients with cerebral arteriovenous malformations (AVM). The need for a system that accurately predicts patient outcomes after AVM SRS led to the development of the radiosurgery-based AVM score (RBAS). The original RBAS was simplified and been validated by numerous centers performing Gamma Knife, LINAC-based, and CyberKnife procedures. It is clear that no predictive method can be perfect due to the uncertainty that is inherent to complex, biologic systems.

“There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.”

Donald Rumsfeld

I would like to thank the International Stereotactic Radiosurgery Society (ISRS) for choosing me as the recipient of the 2013 Jacob I. Fabrikant Award. I am both delighted and humbled to have been chosen for this great honor. Over the past 30 years, the practice of stereotactic radiosurgery (SRS) has matured from an investigatory technique into an accepted treatment option for many patients with benign or malignant tumors, vascular malformations, and functional disorders. Moreover, SRS has expanded from a single-fraction, intracranial application to now include multi-session SRS and extra-cranial indications. Radiosurgery is a mandatory component of medical training for both neurological surgeons and radiation oncologists. I will not detail the illustrious history of the development of SRS and its parent organization, the ISRS, in this paper as these have been well documented elsewhere.[1] Nonetheless, it is safe to state that SRS has changed medical care forever and has transformed our knowledge so significantly that one can analyze the neurosurgical and radiation oncology literature into the time before and after the introduction of SRS.

As happens so often with people and their careers, my introduction to SRS was by chance. After graduating from college in 1984, I spent one year working at Genentech, Inc. in South San Francisco, to improve my skills and understanding of the exploding field of molecular biology. Convinced by my advisors at the time that a medical degree would help advance my research career, I applied and was accepted into medical school at the University of Pittsburgh with the intention of obtaining a combined M.D. and Ph.D. degree. However, it became clear shortly after starting my medical studies that a research career was not in my future. I was particularly impressed by the head of the department of Neurological Surgery, Dr. Peter J. Jannetta. Inspired by Dr. Jannetta and his colleagues, I would attend the weekly conferences held in the Neurosurgery department whenever possible, and by the beginning of my third year of medical school, I was determined to become a neurosurgeon.

It was during this period that I first met Dr. L. Dade Lunsford. Based on his foresight and great commitment, the first North American Gamma Knife (fifth world-wide) was installed at Presbyterian University Hospital in 1987.[2] Dr. Lunsford’s enthusiasm for this new procedure was palpable, and he did what all smart people do to succeed, which is to surround oneself with people equally driven and intelligent. During my residency, Dr. Lunsford became my mentor and friend, and over the years his generosity and support were invaluable. Working in the radiosurgery unit at Pittsburgh brought me into contact with a young assistant Professor from Toronto, Doug Kondziolka, and a rather quiet but intense Radiation Oncologist, John Flickinger. Over the next four years I completed my residency and started on staff at the University of Pittsburgh Medical Center (UPMC). Looking back, I did not appreciate at the time how fortunate I was to be working with this brilliant group of clinical researchers, but their teachings and lessons have been indelibly etched into my mind. The accomplishments and contributions of all three were recognized by the ISRS, with each being awarded the Fabrikant Award in 1997 (Dr. Lunsford), 2003 (Dr. Flickinger), and 2007 (Dr. Kondziolka). We truly do stand on the shoulders of giants.

It has been almost 20 years to the day since I made my first scientific presentation ever at the second ISRS meeting in Stockholm, Sweden, in 1993, on the use of SRS for patients with nonacoustic schwannomas.[3] I remember well my anxiety as Dr. Madjid Samii sat in the front row and critiqued this emerging technology that would alter forever the role of micro-neurosurgery for skull-base tumors. Two decades later, even Professor Samii has accepted the role of SRS in his management of neurosurgical diseases.[4] In this lecture, I would like to share some thoughts on making predictions about complex, biologic systems and how it relates to a personal interest of mine, arteriovenous malformation (AVM) SRS.

ANALYTICS, BIG DATA, AND UNCERTAINTY

“Predictability: does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?”

Edward Lorenz

Physicians are generally good at analyzing the past, decent when discussing the present, and poor when it comes to predicting the future. This is partly due to our lack of formal training in statistics and study design, but frequently relates to biases limiting our ability to objectively evaluate our practices and outcomes. However, a particular challenge for neurosurgeons and radiation oncologists who treat the central nervous system is that many neurologic disorders are rare. Consequently, determining the natural history and best treatment options for these rare conditions is often difficult if not impossible. In contrast, cardiologists in 1983 were able to determine with great confidence that aspirin therapy has a protective effect against acute myocardial infarction in men with unstable angina by performing a multicenter, double-blind, placebo-controlled randomized trial in 1266 patients.[5] Factors that made such a study possible was a common disorder to study (unstable angina), a simple intervention (aspirin therapy versus placebo), and a clear endpoint (myocardial infarction).

Now consider the problems associated with performing an “ideal” study that compares surgical resection and SRS for patients with vestibular schwannomas. First, approximately seven percent of adults over 20 years of age have coronary heart disease in the United States with almost 800,000 adults having a new coronary heart attack each year,[6] compared to approximately 4,000 new cases of VS that are diagnosed annually.[7] Second, it is essentially impossible to randomize patients to either a craniotomy and tumor resection or an outpatient-based procedure with minimal risk. Third, should the primary endpoint of this study be cranial nerve preservation or tumor control? If the latter is chosen, what would be considered an adequate follow-up period for these benign tumors? For these and other methodological reasons, Class I evidence will never exist to compare these techniques, so prospective case-control series have been performed as a practical surrogate to provide Class II evidence.[8,9] Still, although these studies have consistently shown that SRS is associated with a higher rate of cranial nerve preservation, no quality data exists to compare the longterm rate of tumor control. Consequently, the best management strategy for patients diagnosed with small- to moderate-sized VS remains controversial.

The vast majority of medical studies evaluate our past performance on a relatively small number of patients to find associations between patient and treatment factors with some desirable endpoint. Based on our findings, we then modify our practices in the hope of improving outcomes, and then repeat this cycle again at some later time frame. This approach is cheap and simple to execute, but has significant methodological shortcomings including selection bias, completeness of follow-up, and the quality of the analyzed data. Such “small data” analysis may show association between independent and dependent variables, but does not prove causation in most cases. Furthermore, the power or uncertainty of the conclusions relates not only to the sample size, but also the proportion of the study population observed to have a given outcome (Fig. 1). The rise of evidence-based medicine in recent times has emphasized the problems related to retrospective studies. Yet, randomized clinical trials cannot be performed to answer every question, may have poor external validity, and if a long period of time is required to complete the trial, the study hypothesis may no longer be relevant.[10]

Figure 1.

Graph showing the 95% confidence intervals for 2 observed proportions (60% versus 95%) based on different sized study populations. Note that the confidence interval at 95% with 50 patients is roughly equivalent to the confidence interval at 60% with 500 patients.

Analytics is defined as the science of logical analysis. Each day internet search engines, retailers, credit card companies, and data brokers gather information on billions of electronic transactions including age, martial status, level of education, political party, and income to compile profiles of individuals. This information has great value and can be sold to other interested parties. Analysis of such “big data” using predictive modeling techniques is able to examine your recent behavior and forecast your future purchases and activities. For example, someone buying maternity clothes may be interested in a mini-van or a larger house. Despite the potential abuses that this vast amount of information on individuals could be used in a punitive fashion (purchasing clothes in larger sizes may result in someone being classified as obese, clearly of interest to insurance companies), “big data” and advanced analytic techniques provide a benefit to all of us each and every day.

In 1870, Ulysses S. Grant signed a joint resolution of Congress authorizing the Secretary of War to establish the United States National Weather Service (NWS).[11] Weather observations were telegraphed to Washington D.C. from 22 stations providing the first real-time, systematic study of weather. Compilation of this data was one of the first steps to change weather forecasting from a reliance on historical averages to something approaching dynamic weather prediction.[12] Still, prediction of weather patterns with high accuracy remains stunningly complex. Prior to the introduction of supercomputers and “big data”, the NWS high temperature forecast three days in advance was wrong by an average of six degrees. Today, the average error has decreased for this same projection to three degrees, but the inability to precisely forecast the temperature is an example of the limitations predictive modeling for complex systems. Meteorologists have understood for many years that analysis of “big data” alone was rarely correct, and accepted the fact that random, seemingly unrelated events can influence the weather significantly (chaos theory).[13] Chaos theory explains why small differences in the initial state of a system can yield widely diverging outcomes for intricate systems such as the weather, thereby making precise and accurate long-term prediction essentially impossible. Therefore, uncertainty remains a fundamental component of weather prediction, and emphasizes the limitations we have in predicting the outcomes of complex systems, despite our advances in acquiring and analyzing “big data”.

DEVELOPMENT OF A RADIOSURGERY-BASED AVM GRADING SCALE

“Le mieux est l’ennemi du bien.”

(The best is the enemy of the good)

Voltaire

Clinical scientists exist in a world of observation, deduction, and logic. However, it is our curiosity that must sustain us through a career to find answers to complex medical problems. Prior to my research block of my neurosurgical residency, Dr. Lunsford and colleagues had already published one of the seminal papers on AVM SRS.[14] Nevertheless, a number of unanswered questions existed regarding AVM SRS and I was encouraged to re-evaluate the early UPMC AVM series. Over the next several years, we published studies that analyzed the hemorrhage rate before and after SRS,[15,16] the effect of dose on AVM obliteration,[17] factors associated with radiation-induced complications,[18] factors related to failed SRS,[19] and several other papers related to AVM SRS.[20-22] Our work combined with papers from other radiosurgical centers around the world established that SRS did not increase the bleeding rate of AVM,[23-24] showed the relationship between radiation dose and AVM obliteration,[25-27] and documented that AVM location and the amount of adjacent brain receiving radiation predicted complications after SRS.[28,29] The study of AVMs remained an area of great interest after I left UPMC and joined the Department of Neurological Surgery at the Mayo Clinic.

Over the past 16 years I have worked in collaboration with my friends at the UPMC to develop an AVM grading system based on factors relevant to SRS. The Spetzler-Martin grading scale is the most widely utilized grading system for cerebral AVMs.[30] Based on three factors (AVM size, location, and pattern of venous drainage), this grading scale is simple and predicts patient outcomes after resection of intracranial AVM. Despite the widespread use of the Spetzler-Martin scale, we felt that a valid instrument capable of accurately predicting outcomes after AVM SRS would be advantageous and help physicians compare the expected results of microsurgery and SRS for individual AVM patients. The first step involved changing how the results of AVM SRS were analyzed. Focusing on the separate components of AVM SRS (obliteration, radiation-related complications, and post-SRS hemorrhage) was problematic as it did not adequately describe the desired outcome after SRS, which is the chance that SRS would cause AVM obliteration without new neurologic deficits. Multivariate analysis using obliteration without new neurologic deficit as the dependent variable found patient age, AVM volume, AVM location, number of draining veins, and no prior embolization to be associated with successful AVM SRS.[31] In 1997, we published the Pittsburgh Arteriovenous Malformation Radiosurgery (PAR) grading scale,[32] but quickly realized it was too cumbersome for practical use.

Refinement of the PAR grading scale using regression analysis modeling resulted in the original radiosurgery-based AVM score (RBAS) (Table 1).[33] Importantly, the model based off the UPMC patients accurately predicted outcomes for the Mayo patients despite significant differences between the patient groups with regard to age, presentation, prior embolization, size, and different dose prescription guidelines. Since its publication in 2002, the RBAS has been validated by other Gamma Knife centers,[34-36] and has predicted outcomes after both LINAC-based and CyberKnife procedures.[37-39] The RBAS was simplified in 2008 using AVM location as a two-tiered variable (basal ganglia, thalamus or brainstem versus other) compared to the original three-tiered formula.[40] The modified RBAS has since been tested and also found to be predictive of outcomes after AVM SRS for patients having either Gamma Knife or LINAC-based procedures.[41-43] Since their publication in 2002 and 2008, the original and modified versions of the RBAS have been cited more than 175 times, and has been described as “the radiosurgical equivalent of the Spetzler-Martin grading system.”[44]

Table 1.

The Radiosurgery-based AVM Score (RBAS) Grading System

Radiosurgery-based AVM Score	Score = (0.1)(AVM volume, cm3) +
(Ref.#33)	(0.02)(patient age, yrs) +
	(0.3)(AVM location)*
Modified radiosurgery-based AVM Score	Score = (0.1)(AVM volume, cm3) +
(Ref. #40)	(0.02)(patient age, yrs) +
	(0.5)(AVM location)†

^*Location defined as: 0=frontal/temporal, 1=parietal/ occipital/intraventricular/ corpus callosum/cerebellum, 2=basal ganglia/thalamus/brainstem.

^†Location defined as: 0=hemispheric/corpus callosum/ intraventricular/cerebellum, 1=basal ganglia/thalamus/ brainstem.

In trying to develop a RBAS, the author has always appreciated the balance between accuracy and practicality. Although an integer-based grading scale analogous to the Spetzler-Martin system is simple, such an approach assumes that the component factors have an equal affect on outcomes. Thus, if AVM size was more important than age or location, giving them equal weighting would result in a simple, but potentially inaccurate scale. Conversely, relying on only three variables (age, volume, location), it is likely that including other known or potential factors into the RBAS would increase its accuracy (Table 2). However, inclusion of too many variables would make the RBAS more complicated and less likely to be utilized on a routine basis. For example, I agree with the conclusion of the University of Florida that a diffuse nidus is a negative predictor of obliteration after SRS,[45] but did not include AVM morphology in the modified RBAS. Similar to weather prediction, humans and their behavior can be thought of as complex, biologic systems so that alterations in unrecognized, but potentially significant variables, may partly determine a patient’s outcome after AVM SRS.

Table 2.

Non-Treatment Related Factors that May Affect Outcomes after AVM Radiosurgery

Proven	Suspected	Potential
Age	Prior hemorrhage*	Smoking
Size	Nidus architecture (compact vs. diffuse)	Hypertension
Location	Number of veins	Medications
	Blood-flow rate	Endocrine status
	Associated aneurysms	Immune status
		Coagulation status
		Ethnicity
		Diet
		Alcohol

^*Dependent on interval from the time of hemorrhage to the time of radiosurgery.

Lastly, it may be that the dependent variable we have always used (obliteration without new neurologic deficits) is not the most appropriate endpoint. A number of cerebro-vascular neurologists have put forth the argument that the natural history of unruptured cerebral AVM is more benign than traditionally believed, and that the treatment of patients with unruptured AVM is not justified based on the available information.[46] To that end, the composite risk of death or stroke and risk of death or clinical impairment, defined as a modified Rankin Score (MRS) ≥2, are the primary endpoints of the ongoing trial comparing the risk of observation versus treatment for patients diagnosed with unruptured brain AVM (A randomized trial of unruptured brain arteriovenous malformations, ARUBA). We have analyzed the results of AVM SRS using the MRS as the primary outcome measure, as well as more recently the results of SRS for ARUBA-eligible patients.[47,48]Overall, the RBAS correlated with a decline in MRS after SRS for patients with both ruptured and unruptured AVM. For the ARUBA-eligible patients, the 10-year risk of MRS ≥2 for patients with a RBAS ≤1.50 was 2%, compared to 18% for patients with a RBAS >1.50. Yet, despite the observed correlation between the RBAS and neurologic impairment after SRS, it is possible that different factors would be considered important or the relative weighting of the utilized variables would be altered if neurologic condition, without a consideration of the status of the AVM, were used as the dependent variable in our analyses.

CONCLUSIONS

The management of cerebral AVM continues to be of great interest to neurosurgeons and radiation oncologists. It is clear that no predictive method can be perfect due to the uncertainty that is inherent to complex, biologic systems.