Quantitative Analyses of Normal Tissue Effects in the Clinic (QUANTEC): An Introduction to the Scientific Issues

Abstract

Advances in dose/volume/outcome (or normal tissue complication probability, NTCP) modeling since the seminal Emami paper from 1991 are reviewed. There has been some progress with an increasing number of studies on large patient samples with three-dimensional dosimetry. Nevertheless, NTCP models are not ideal. Issues related to the grading of side effects, selection of appropriate statistical methods, testing of internal and external model validity, and quantification of predictive power and statistical uncertainty, all limit the usefulness of much of the published literature. Synthesis (meta-analysis) of data from multiple studies is often impossible due to sub-optimal primary analysis, insufficient reporting and variations in the models and predictors analyzed. Clinical limitations to the current knowledge-base includes the need for more data on the effect of patient-related co-factors, interactions between dose-distribution and cytotoxic or molecular targeted agents, and the effect of dose fractions and overall treatment time in relation to non-uniform dose distributions. Research priorities for the next 5 to 10 years are proposed.

1. Why QUANTEC?

Modern radiation therapy (RT) techniques generally yield non-uniform dose distributions in non-target tissues. The introduction of external beam megavoltage RT in the 1950’s shifted the most important side-effects from the skin and subcutaneous tissues to the deeper seated tissues. The ensuing wide adoption of parallel opposing field techniques led to improvements in target dose homogeneity, but typically led to whole or partial organ irradiation of the neighboring non-target tissues: a fractional volume of an organ at risk would essentially receive the prescribed target dose. Because of the limited capabilities to image the tumor extent, most RT fields included liberal margins.

CT-based diagnosis and RT planning in the 1980’s and 1990’s revolutionized target volume visualization and facilitated multiple-field and three dimensional (3D) conformal RT. Conceptual and technological advances have led to new RT technologies: e.g. intensity modulated radiation therapy (IMRT), rotational or helical delivery, robotic delivery, and proton therapy. These technologies typically deliver near-uniform doses to the target volume. However, the dose-distribution in the surrounding normal tissues is more variable.

Therefore, these new technologies provide the treatment planner with increased flexibility in determining which regions of normal tissue are to be incidentally irradiated. The treatment planner needs information to predict the risk of a normal tissue injury for competing 3D dose distributions, such that the therapeutic ratio can be optimized. One of the goals of QUANTEC is to summarize the available 3D dose/volume/outcome data. At the same time, increasing use of combined modality therapy has often increased the burden of early and late toxicities¹. Understanding the trade-off between an expected decrease in toxicity resulting from an improved dose distribution, and the possible increase in toxicity with systemic agents, is an increasingly pertinent, yet poorly researched, area.

2. Analyzing RT-related toxicity

Cancer survivorship issues have been gaining prominence, partly due to the increasing number of cancer survivors; a tripling in the USA² between 1970 and 2001. This increase is the result of early diagnosis, screening efforts, improved treatments, and an increased incidence of many cancers. Radiation oncologists have pioneered recording and analysis of late treatment sequelae and the available literature on late effects is much richer for this modality than for cytotoxic or surgical treatments. However, toxicity is often under-reported, and probably under-recorded, even in the more rigorous framework of prospective clinical trials^3–5. Clearly, this is a special concern in NTCP (normal tissue complication probability) modeling studies where the data analyzed often are retrospectively extracted from charts or databases.

The US National Cancer Institute Common Terminology Criteria for Adverse Events (CTCAE) v3.0 is a comprehensive dictionary for recording and grading of side effects of all major cancer therapies⁶. Widespread adoption of a common grading system for adverse events, such as CTCAE, would improve between-study comparability and is encouraged. However, CTCAE still combines multiple signs and symptoms into a single grade. While this may be convenient for routine studies and comparisons of therapies across studies, it is associated with a loss of specificity in toxicity-specific studies⁷. For such studies, including NTCP modeling studies, grades should be atomizedi.e. broken down to specific signs and symptoms that are likely to reflect specific radiation pathophysiologies. The SOMA (Subjective, Objective, Management, Analytic) scale explicitly distinguishes between objective signs and subjective symptoms. For toxicity-specific studies, a “SOMAtized” scale, that is a scale where these components of toxicity are kept separate, is preferable. Grouping several specific toxicities into a single composite endpoint is likely associated with a loss of statistical resolution^{3, 8}.

3. The Emami paper and early NTCP modeling

The paper by Emami et al.⁹ is the most frequently cited paper ever published in the International Journal of Radiation Oncology Biology Physics with 1,062 citations according to the ISI Web of Science^® (accessed February 3, 2009). This paper published the tolerance doses for irradiation of 1/3, 2/3 or the whole of various organs. As high quality clinical data were scarce, the task force took the bold approach to establish these doses by a simple consensus of clinical experience or opinions. In an accompanying paper, Burman et al.¹⁰ fitted a Lyman model¹¹ to the Emami consensus dose/volume data thereby facilitating the use of Emami’s constraints for an arbitrary fraction of a whole organ uniformly irradiated. Further, Kutcher et al.¹² proposed a method, a so-called dose-volume histogram reduction algorithm, for reducing an arbitrary non-uniform dose distribution into a partial volume receiving the maximum dose, effectively allowing the extrapolation of Emami’s constraints to any dose distribution. The mathematical method amounted to a common formula for taking a ‘generalized mean’, although this was not recognized at the time. This Lyman-Kutcher-Burman (LKB) model, combining Lyman’s model with the Kutcher-Burman DVH reduction scheme, remains the most widely used NTCP model. While the model claims no deep mechanistic validity, its mathematical form is sufficiently flexible to allow representation of various dose-volume dependencies. Within the structural resolution of current data sets, the LKB model can typically not be rejected as a good fit of the data, although it is not always the best model considered. Probabilistic models, studied in groundbreaking papers in the 1980’s by Schultheiss¹³ and Withers¹⁴, introduced concepts like serial and parallel tissue organization and functional sub-units and became conceptually influential but have played a relatively-modest role in actual data analyses except for The Relative Seriality Model¹⁵, that has found some use in analyzing clinical data.

4. Small animal models and limitations to a DVH based approach

DVH-based analyses inherently assume that organ function is uniformly distributed within an organ. Alongside the efforts to analyze clinical dose/volume/outcome data experimental animal studies of the volume effect have produced important proof-of-principle insights – but at the same time have had relatively little impact on clinical NTCP modeling so far. In 1995, Travis et al.^{16, 17} reported that partial organ irradiation of equal volumes of the mouse lung base was more likely to cause radiation pneumonitis than irradiating identical volumes of the apex or, even more pronounced, the middle regions of the lung. As the histological damage in the lung did not vary with location, this finding has been interpreted as a result of variation in the functional importance of different lung regions. However, some of the demonstrated effect may have also resulted from inadvertent inclusion of the central airways/vessels within the CT-defined lung. Attempts at modeling location effects in human lung have only been tried relatively recently, with mixed results, see the paper by Marks et al. in this issue. Location effects have also been demonstrated in partial volume irradiation of the parotid gland¹⁸, probably reflecting damage to the excretory ducts, blood vessels and nerves. Another example where DVH based analysis for the organ at risk may not be adequate is lung, where irradiation of the heart in addition to the lung has been shown in experimental animals to affect the risk of symptomatic radiation induced pneumonitis¹⁹.

Hopewell and Trott²⁰ analyzed experimental dose-volume data and concluded that: “Volume, as such, is not the relevant criterion, since critical, radiosensitive structures are not homogeneously distributed within organs”. Work by Trott and colleagues²¹ in 1995, documented a volume effect for functional damage after irradiation of the rat rectum but found no significant influence of volume on structural damage to the rectal wall. The theme of different radiation pathogenesis for rectal side-effects, and therefore varying radiobiological properties, has only relatively recently been systematically analyzed in patients by the group at the Netherlands Kanker Instituut²².

Extensive studies by van der Kogel in the late 1980’s showing that the probabilistic model did not correctly predict the probability of spinal cord injury after irradiation of two geometrically separated 4 mm segments of rat cervical spinal cord, undoubtedly discouraged further exploration of this model in the analysis of clinical data sets²³. Van der Kogel’s studies were subsequently expanded into an elegant, systematic study of dose-volume effects in the rat spinal cord, ending with the sobering conclusion that not any of the 14 mathematical models, tried by the authors, could fit all the data²⁴.

5. Progress on all fronts since 1991

Much has changed since 1991 (Table 1). Many, mainly retrospective, clinical studies have been published on dose-volume-outcome analysis of clinical data. The QUANTEC review identified >70 papers on radiation pneumonitis alone. Some of these studies are very large (e.g. a study of rectal effects in 1132 patients by Fiorini et al.²⁵). There are quantitative analyses of dose-volume-outcome relationships for >30 organs and tissues. More than a dozen mathematical dose volume models have been proposed.

Table 1.

Dose-volume relationships ca. 1990 and 2009+

ca. 1990	2009+
Treatment usually with parallel opposing fields or “box” techniques – 3DCRT gaining ground clinically in some centers	Widespread use of conformal techniques, including IMRT – often resulting in highly non-uniform dose distribution in organs at risk with large volumes receiving low doses
RT typically delivered as single-modality–spectrum of toxicities relatively well characterized	Many curative cases receiving combined modality therapy – many regimens are very toxic leading to problems with compliance
Conventional fractionation dominates – clinical trials of hyperfractionation and accelerated fractionation	Conventional fractionation dominates – clinical trials of hypofractionation in progress
Authors search for a “safe” dose-volume constraint	Increasing appreciation of the risk-benefit trade-off in an individual patient – a monotonic increase in toxicity risk with increasing dose/increasing volume
Early interest in NTCP modeling – Lyman model most widely used	Change from “more models” to “more data” – Lyman model still widely used, but new modeling strategies are being pursued
Analysis often based on groups of patients	Analysis of individual patient level data
Lack of consistency in contouring organs at risk among investigators	Lack of consistency in contouring organs at risk among investigators
Models often applied with parameters from the literature – no adjustment for patient or treatment characteristics	Statistical estimation of model parameters – often with adjustment for significant patient or treatment characteristics
Toxicity under-scored and under-reported in most studies	Toxicity under-scored and under-reported in most studies – despite attempts to define dictionaries for toxicity reporting such as CTCAE
A lack of quantitative, evidence-based dose-volume constraints – Emami et al. develops a ground-breaking set of consensus constraints for partial organ irradiation	A lack of quantitative, evidence-based dose-volume constraints – the QUANTEC group initiates a series of systematic literature reviews

One class of NTCP models reduces the 3D dose matrix to a scalar, often thought of as an effective volume or an effective dose received by a defined reference volume. This scalar is subsequently related to the incidence or risk of normal tissue toxicity through a sigmoid link function, typically a logistic or probit relationship. This model building strategy is similar to the one used originally by Lyman¹¹ and it may be reasonable classifying these as generalized Lyman models. The push from cell-killing based models towards heuristic models has been strengthened by novel insights into radiation pathogenesis of late effects²⁶ and an increased appreciation of the role of anatomical and physiological factors in normal tissue dysfunction.

Other modeling approaches have been used such as principal component analysis²⁷, contiguous (or cluster) damage model²⁸ and data-mining to build multivariate models²⁹. Other approaches include the use of artificial neural networks³⁰ and support vector machines³¹ as classifiers of patients with respect to the development of side effects. These methods are complementary to more traditional modeling and will undoubtedly be further explored in the coming years.

6. The QUANTEC initiative

It was on this background that the QUANTEC Steering Committee was formed. Stimulated by a proposal from the Science Council of the American Association of Physicists in Medicine (AAPM) to revise and update the Emami guidelines, the QUANTEC group was formed from a loose network of researchers with a long-standing interest in dose-volume modeling. The Steering Committee defined three aims for QUANTEC:

1. To provide a critical overview of the current state of knowledge on quantitative dose-response and dose-volume relationships for clinically relevant normal-tissue endpoints;

2. To produce practical guidance allowing the clinician to reasonably (though not necessarily precisely) categorize toxicity risk based on dose-volume parameters or model results;

3. To identify future research avenues that would help improve risk estimation and/or mitigation of early and late side effects of radiation therapy

A kick-off workshop with 57 invited participants from North America and Europe was held in Madison, Wisconsin in October 2007 with generous financial support from the AAPM and the Board of the American Society for Therapeutic Radiation Oncology (ASTRO). The main deliverable from the workshop was the formation of a number of working groups charged with producing organ-site specific overviews of quantitative dose-volume relationships as well as groups producing vision papers on future research avenues in the field. The results of these efforts are partly presented in this issue of the IJROBP, again made possible with generous support from ASTRO.

While overall progress has been real and substantial, research in the past two decades has also defined limitations to our current methods and the resulting knowledge. One of the main lessons from the literature overviews is that more uniform and comprehensive reporting would be a huge help when trying to combine data from multiple studies, see the paper by Jackson in this issue. Current best estimates of dose-volume parameters can in many situations be based on empirical data, in contrast to the consensus values proposed by Emami et al. However, there is still a lack of proper estimation of the uncertainty in these parameters in most cases. Clinically, the literature on patient-related risk factors is scattered and often inconsistent from one study to the next. When patient or treatment-related risk factors parameters are not listed as significant in a given paper, it is often not clear whether the factor has been tested or not. Therapeutically, RT is combined with drugs in more and more indications. While calculating the risk associated with the RT dose distribution alone may provide some guidance, it cannot generally be assumed that giving a drug together with radiation will even preserve the ranking of competing radiotherapy RT plans³². The increased use of hypofractionation, and the use of an increasing number of beam orientations (e.g. rotational delivery), results in a relatively large volume of normal tissue receiving a low total dose and dose per fraction. The available dose/volume/outcome data may not be applicable in this setting. There has been little discussion – and no consensus – on how models or dose-volume constraints should be adjusted if the fractionation scheme changes significantly. One study did adjust the individual bins in the dose volume histogram for dose per fraction³³, but the fits obtained with α/β=3 Gy, 10 Gy or infinity (=physical dose) were not statistically different for that given treatment fractionation scheme. However, the model may not be valid without correction if a significantly different fractionation scheme is used.

7. Model validation and data analysis

On the model side, there is a need for improved data analytical methods and a more critical appraisal of the various dimensions of model validity.

7.1 Face validity

The first screen when judging a model fit to a set of data is face validity. Is the probability of a side-effect a non-decreasing function of dose, dose per fraction and volume, given that two of these three variables are held constant? If the model includes patient characteristics, such as age, smoking history or co-morbidity, is the effect estimated using the model consistent with published clinical data? Are confidence intervals or standard errors of the estimates reasonable in view of the analyzed sample size and the number of events actually recorded?

7.2 Internal validity

Internal validity relates to whether the model actually provides a reasonable representation of the data to which it is fitted. To this end, a graphical representation of the fit to the data may be informative. This may be supplemented with a formal goodness of fit statistics, such as the chi-square test. The null hypothesis being tested is that the discrepancy between the observed toxicity incidence data and the data expected under the fitted model can be explained by chance alone. A test P-value <0.05 means that the null hypothesis can be rejected at the 5% significance level, i.e. the model “does not fit the data”. A non-significant P-value, however, may not be very informative as typical NTCP model fits to clinical data sets yield a relatively low statistical power of goodness of fit statistics. In other words, two alternative mathematical models may be quite divergent without either one of them being rejected based on the goodness of fit test.

The log-likelihood may also be used for comparing the fit of competing models to a data set, again studies have shown that competing models tend to produce very similar log-likelihood values for a given data set³⁴. For nested models, i.e. models that differ by the inclusion of one additional parameter, the difference in log-likelihood forms the basis for the likelihood ratio test, a robust test for the statistical significance of adding this parameter. For non-nested models the Akaike Information Criterion has been used by some authors, see for example Tucker³⁴.

Some authors look at NTCP models as classifiersi.e. as a way to separate patients who do or do not develop a given toxicity. This leads to a standard predictive testing framework, where sensitivity, specificity, negative and positive predictive values can be estimated. The area under the curve (AUC) of the Receiver Operating Characteristic (ROC) curve can be used as a figure of merit for comparing alternative models. Note, however, that a model reliably identifying subgroups of patients with, say, a 10% and a 40% risk of toxicity would most likely be clinically useful, but if the latter group is labeled as “responders” there would still be a 60% false positive rate. In this case a binned comparison of observed and expected toxicity may be more informative³⁵. Cross validation techniques have been suggested for NTCP modeling²⁹ but have so far not been widely applied.

7.3 External validity

External validity addresses how well the model explains the variability in response seen in an independent data set, preferably from another institution. Multivariate NTCP models are often over-fitted in the sense that they include too many parameters relative to the number of events analyzed. This may result in strongly correlated parameter estimates and while such a model may pass the test for internal validity with flying colors, it often has poor external validity. Differences between institutions in the scoring of reactions, in patient demographics, in the burden of co-morbidities as well as in treatment characteristics may all contribute to a reduced predictive power of a model when tested in an independent data set. Relatively little research has been performed on external validity of NTCP models. Bradley et al.³⁶ applied a radiation pneumonitis (RP) model fitted to data from 219 Washington University patients to an independent series of RP data from 129 patients enrolled in the RTOG 93-11 trial and concluded that the model “performed poorly” in the new data set. A model fitted to the two data sets combined was found to give an odds ratio of approximately two between the 33% of all patients with the riskiest plans and the 33% of patients with the safest plans, but much of the variability is still unexplained. Similar problems with generalizabilty are seen in studies applying different models on the same dataset: as an example Tsougos et al.³⁷ found that six published models predicted an incidence of grade 3+ RP ranging from 4% to 21% in a group of 47 patients.

One issue is that various dose-volume metrics often are strongly correlated within a given data set³⁸. This may lead to problems with multicollinearity which although it may not affect the internal validity of the model, can lead to reduced generalizability. This becomes particularly relevant for extrapolation in dose-volume space, i.e. if a model derived on basis of “similar” dose plans is applied to a very different dose distribution³⁹.

7.4 Clinical utility

Dose-volume constraints are used in routine dose planning as an integral part of the informal optimization of therapeutic ratio that inverse planning entails. Acceptable dose distributions are identified from a assessment of the risk:benefit ratio in an individual patient – often on the basis of clinical experience rather than on numerical estimates from dose-volume models. Population constraints are very important in this context but can obviously not stand alone. Careful consideration should be given not only to the numerical value of these constraints but also to their statistical uncertainty. Using these values directly in dose-plan optimization should be done with great caution.

The fact that dose-volume constraints or NTCP models are used in clinical practice does not in itself prove that they improve cancer care from an evidence-based medicine perspective. Ultimately, the clinical utility of NTCP modeling should be tested in randomized controlled trials. Phase I/II dose escalation trials in patients with non-small cell lung cancer (NSCLC), where the individual patient is assigned a dose based on a NTCP estimate⁴⁰, have been completed or are in progress for example at University of Wisconsin⁴¹, University of Michigan⁴² and the MAASTRO clinic in the Netherlands⁴³. The goal is to test these strategies in randomized phase III trials. This could potentially provide an evidence base for risk adaptive radiotherapy for NSCLC based on NTCP modeling.

8. Research priorities: beyond QUANTEC

Important research priorities, identified above as wells as in the QUANTEC thematic and organ-site reviews, include:

1. Development of tools and strategies for prospective recording of specific pathologies after RT alone or combined with drugs;

2. Wider application of methods adjusting for censoring when analyzing late effects

3. Quantification of the influence of physiologic factors and co-morbidities on the expression of toxicities;

4. The continued development of robust normal tissue endpoints including patient reported outcomes to further our understanding of the relationship between toxicity and quality of life.

5. Development of methods for synthesizing results across studies with appropriate estimation of prediction uncertainty;

6. Establishment of large continually growing data bases with full access to the 3D dose matrix and linkage with biomarkers and clinical outcome;

7. Prospective testing of model performance in independent data sets, preferably from clinical trials;

8. Improved understanding of the interaction between dose distribution on one hand and dose per fraction or administration of other modalities on the other;

9. Developing strategies for testing the clinical utility of NTCP models.

Adjustment for dose distribution remains a major challenge in clinical radiation research. A systematic effort, capable of winning competitive research funding, is required to take this field to the next stage.