Interpreting Trial Results in Light of Conflicting Evidence: A Bayesian Analysis of Adjuvant Chemotherapy for Non–Small-Cell Lung Cancer

Abstract

Purpose

When successive randomized trials contradict prior evidence, clinicians may be unsure how to evaluate them: Does accumulating evidence warrant changing practice? An increasingly popular solution, Bayesian statistics quantitatively evaluate new results in context. This study provides a clinically relevant example of Bayesian methods.

Methods

Three recent non–small-cell lung cancer adjuvant chemotherapy trials were evaluated in light of prior conflicting data. Results were used from International Adjuvant Lung Trial (IALT), JBR.10, and Adjuvant Navelbine International Trialist Association (ANITA). Prior evidence was sequentially updated to calculate the probability of each survival benefit level (overall and by stage) and variance. Sensitivity analysis was performed using expert opinion and uninformed estimates of survival benefit prior probability.

Results

The probability of a 4% survival benefit increased from 33% before IALT to 64% after IALT. After sequential updating with JBR.10 and ANITA, this probability was 82% (hazard ratio = 0.84; 95% CI, 0.77 to 0.91). IALT produced the largest decrease in variance (61%) and decreased the chance of survival decrement to 0%. Sensitivity analysis did not support a survival benefit after IALT. However, sequential updating substantiated a 4% survival benefit and, for stage II and III, more than 90% probability of a 6% benefit and 50% probability of a 12% benefit.

Conclusion

When evaluated in context with prior data, IALT did not support a 4% survival benefit. However, sequential updating with JBR.10 and ANITA did. A model for future assessments, this study demonstrates the unique ability of Bayesian analysis to evaluate results that contradict prior evidence.

INTRODUCTION

When pivotal trial results are published, one of the primary issues is whether the totality of accumulating scientific evidence is strong enough to support changing clinical practice. However, it is often unclear how to sequentially evaluate results from successive, and sometimes conflicting, trials. For example, before publication of the International Adjuvant Lung Trial (IALT) in 2004, no survival benefit had been found for adjuvant cisplatin-based chemotherapy in non–small-cell lung cancer (NSCLC): A 1995 meta-analysis found no statistically significant benefit,¹ subsequent trials were not positive^2,3 and the 2001 lung cancer consensus panel declared adjuvant chemotherapy unproven and experimental.^4,5 Therefore, the positive results of the IALT were controversial.^6–8 Subsequently, two similarly designed adjuvant chemotherapy trials published positive results: the National Cancer Institute of Canada Clinical Trials Group (JBR.10) in 2005⁹ and the Adjuvant Navelbine International Trialist Association (ANITA) in 2006¹⁰ (Table 1). However, the magnitude of the absolute 5-year overall survival (OS) benefit varied: 4.1%, 15%, and 8.6%, respectively for IALT, JBR.10, and ANITA. In light of this variability, earlier negative evidence, and the preliminary negative results for adjuvant carboplatin (2007),¹¹ clinicians may wonder how best to evaluate the conflicting evidence: Does adjuvant chemotherapy produce an OS advantage for NSCLC for each stage? And, if so, what is the magnitude of this benefit?

Table 1.

Trial Data Used in Bayesian Sequential Updating Analysis for Cisplatin-Based Adjuvant Chemotherapy for NSCLC

Trial and Publication Year	No. of Patients	Hazard Ratio	P	Absolute Overall Survival Benefit at 5 Years (%)	95% CI
Meta-analysis,(1) 1995	1,394	0.87	.08	5	−1 to 10
IALT,6 2004	1,867	0.86	< .03	4.1
JBR.10,9 2005	482	0.69	.04	15
ANITA,10 2006	840	0.80	.017	8.6

NOTE. Hazard ratio and absolute overall survival benefit for trials included in analysis are shown.

Abbreviations: NSCLC, non–small-cell lung cancer; IALT, International Adjuvant Lung Trial; ANITA, Adjuvant Navelbine International Trialist Association.

The task of integrating unexpected or conflicting results with prior evidence is often left to the clinician. Although discussions in published articles may qualitatively place trial findings within the context of available evidence, the crucial question is frequently not answered: What is the probability that the new result is correct? And how much does the study contribute?^12–17 Unfortunately, classical methods for iterative data incorporation are limited: (1) successive qualitative evaluations by individual clinicians may lead to heterogeneous conclusions, (2) appraisal by expert panels may lack transparency and timeliness, and (3) meta-analysis techniques generally evaluate a static set of trials, do not recognize time order, cannot incorporate diverse types of evidence, and are cumbersome to update.^18–20 For instance, systematic reviews of adjuvant chemotherapy for NSCLC do not describe the relative contribution of each study nor the probability that the survival benefits observed in each trial are true.^21,22

The scientific dilemma is how to sequentially evaluate and quantitatively integrate new trials with prior evidence to make informed decisions. Bayesian statistics is one solution that mathematically updates prior evidence with new data in a dynamic process.^{16,17,23–25} Bayesian methods are used in biostatistics, astrophysics, and genomics to quantify the reliability of results, to sharpen the assessment of risk, and to determine the amount of information contributed by a study.^25–31 These features facilitate sequential evaluation of trials. Furthermore, Bayesian approaches explicitly and quantitatively describe the data synthesis process, enhancing transparency, accuracy, and reproducibility.^{31a,31b,31c,31d}

Bayesian techniques are emerging in oncology research³² and have been used for trial design,^33–36 monitoring^35,37–43 and data evaluation,^33,44–47 pharmacokinetic evaluation,^48–50 prediction of phase III success,⁵¹ and models of cancer risk,^52–57 recurrence,^58–60 and mortality.^57,61,62 Clinicians frequently use Bayesian approaches during iterative clinical assessments to update the probability of a diagnosis. For example, a physician evaluating a 60-year-old patient with a cough may consider lung cancer more likely if a history of hemoptysis is later elicited. This “clinical judgment” can be formalized by calculating the positive predictive value (PPV) of lung cancer, as is commonly done for imaging studies.^28,42 The PPV calculation is derived from Bayes Theorem, the mathematical backbone of Bayesian statistics.^28,42 Similarly, Bayesian analysis can determine the PPV of a trial.

In contrast to classical statistical approaches, the results of Bayesian analysis are directly interpreted as the probability that a therapy produces a survival advantage,¹⁶ the finding most relevant to clinicians. Therefore, Bayesian analysis may be helpful in evaluating whether new results should change clinical practice and the focus of future clinical and basic science investigations.

Although cardiology trials have been appraised with Bayesian tools,^15,28 to our knowledge, this approach has not been applied to oncologic therapeutic trials and has not been used to evaluate successively published trials. We sequentially updated prior evidence with the results of IALT, JBR.10, and ANITA using Bayesian methods to (1) establish the magnitude of the OS benefit, (2) determine the amount of information contributed by each trial, and (3) explore the OS benefit supported for each stage. Also, we conducted an expert opinion survey and performed a sensitivity analysis on the prior probability of an OS benefit.

METHODS

Methodologic Approach

Bayes Theorem determines the probability (P) of an outcome (θ) given (|) new data (X)^16,23:

Therefore, the PPV for lung cancer, P(θ|X) depends on (1) the pretest probability of lung cancer (P(θ)) given the patient's symptoms, (2) a test result (X), and (3) the strength of the test result (P(X|θ)) as determined by the sensitivity and specificity.^16,23

In the current analysis, the updated probability (P(θ|X)) of a 5-year OS benefit for adjuvant chemotherapy (θ) was computed from the hazard ratio (HR) found in IALT (X) and the pre-IALT survival benefit data P(θ) (Fig 1 and Appendix, online only). The distribution of X and θ was obtained from the literature, such that P(X|θ) was the probability of the results of IALT given the existence or absence of an OS benefit. As additional trials were published, the probability of an OS benefit was updated, first with the results of JBR.10 and then ANITA.

Fig 1.

Flow diagram of method for updating the probability of a survival benefit. IALT, International Adjuvant Lung Trial; ANITA, Adjuvant Navelbine International Trialist Association.

Establishing the Probability of a Survival Benefit Before the IALT

The probability of an OS benefit for cisplatin-based adjuvant chemotherapy for NSCLC before the publication of the IALT was established through a systematic literature review. MEDLINE was searched from January 1965 through January 2004 using the terms “lung cancer,” “NSCLC,” and “chemotherapy” (adjuvant, cisplatin-based, or postoperative) and expanded by reviewing the references in these articles.

We identified the 1995 Non–Small Cell Lung Cancer Collaborative Group meta-analysis as the most comprehensive published quantitative summary of the data before the initiation of IALT. This meta-analysis did not show a statistically significant OS benefit for cisplatin-based adjuvant chemotherapy: the HR for death at 5 years was 0.87 (P = .08) and the absolute OS benefit was 5% (95% CI, 1% to 10%)(1) (Table 1).

Deriving the Prior Probability Curve

For the primary analysis (base case), uncertainty regarding the probability of an OS benefit derived from the meta-analysis was represented by a probability distribution curve constructed by converting the 5-year survival probability to the exponential distribution of the logarithm of the HR. The standard deviation (SD) was inferred from the CI.

The null hypothesis was the same as in the IALT: no OS benefit for adjuvant chemotherapy. Because there was uncertainty before IALT, and equipoise should exist to ethically initiate a trial,⁶⁷ the prior probability of the null was set at 50% (half the weight of the curve lay at zero; see Appendix, online only). The base case prior probability curve represents the probability of each survival benefit level being true given the meta-analysis result (solid line, Fig 2A and Table 2).

Fig 2.

(A) Prior probability of a survival benefit before IALT. x-axis = 5-year survival benefit level. y-axis = cumulative probability of the survival benefit level. (B) Updated probability of survival benefit after IALT. (C, D) Sequentially updated probability of survival benefit after IALT and JBR.10 (C), and after IALT, JBR.10, and ANITA (D). IALT, International Adjuvant Lung Trial; ANITA, Adjuvant Navelbine International Trialist Association.

Table 2.

Probability of a Survival Benefit for Cisplatin-Based Adjuvant Chemotherapy for NSCLC Before and After Sequential Updating: Base Case And Sensitivity Analysis

5-Year Survival Benefit	Probability of Survival Benefit (%)			Updated Probability of Survival Benefit (%)
	Before IALT			After IALT			After IALT and JBR.10			After IALT, JBR.10, and ANITA
	Base Case	Expert	Uninformed	Base Case	Expert	Uninformed	Base Case	Expert	Uninformed	Base Case	Expert	Uninformed
Any benefit	48	43	35	86	85	61	89	89	88	89	89	88
Survival decrement	1.5	6.7	15.4	0.5	0.1	0.6	0	0	0	0	0	0
≥ 2% benefit	44	34	33	84	78	57	88	86	85	89	88	87
≥ 4% benefit	33	21	30	64	52	43	75	66	73	82	77	81
≥ 6% benefit	18	10	28	24	16	21	34	26	42	43	35	51

NOTE. The probability of each survival benefit level before the results of IALT (base case and sensitivity analysis), the updated probability of survival benefit after IALT, after sequentially updating with JBR.10, and after sequentially updating with ANITA is shown.

Abbreviations: NSCLC, non–small-cell lung cancer; IALT, International Adjuvant Lung Trial; ANITA, Adjuvant Navelbine International Trialist Association.

Updating Prior Knowledge With the Results of the IALT, JBR.10, and ANITA

Using Bayes Theorem, the base case prior probability curve was updated with the logarithm of the HR for OS found in IALT (HR = 0.86; P < .03). This computation yielded the posterior probability of survival benefit after IALT. As subsequent trials were published, the probability of a survival benefit after IALT was similarly updated with JBR.10 (HR = 0.69; P = .04), and this new result was then updated with ANITA (HR = 0.80; P = .017; see Appendix, online only).

The Q statistic was calculated and assessed at the P < .05 level to evaluate statistical heterogeneity of the trial results for survival benefit (Appendix, online only).⁶⁸ Calculations were performed in SAS version 9.1 (SAS Institute Inc, Cary, NC) and the code is available on request.

Sensitivity Analysis

Sensitivity analysis was performed to evaluate the effect of different approaches to the synthesis of prior evidence.^28,69–72 The analysis was repeated with two approaches described in the literature that capture a range of possible prior probability curves: (1) expert opinion curve based on a survey of thoracic oncologists, and (2) modified uninformed curve reflecting pessimism and uncertainty about survival benefit (Fig 2).

Expert opinion curve.^69,70

Thoracic oncologists attending the International Novel Agents in the Treatment of Lung Cancer Conference in Cambridge, MA, were surveyed in September 2003. Opinions were elicited⁷³ about the 5-year OS benefit of cisplatin-based adjuvant chemotherapy before and after the results of the IALT were presented at the American Society of Clinical Oncology Annual Meeting in June 2003.⁷⁴ The respondents' ability to discriminate changes in opinion over time was assessed.^33,35

A distribution curve for the probability of a survival advantage before the presentation of IALT was constructed by converting survival estimates to HRs (exponential distribution). The null hypothesis was no survival benefit and equipoise was assumed (Appendix, online only).

Modified uninformed curve.

A prior probability curve reflecting uncertainty about the magnitude of survival benefit and pessimism about prior data was created to accommodate negative trials reported after the meta-analysis and potential limitations of the base case.^2,7 This modified uninformed curve was developed by adapting two approaches described in the literature²⁵: (1) a noninformative prior (horizontal line), indicating complete uncertainty, and (2) a downweighted base case, reflecting pessimism.

The modified uninformed curve was symmetric around zero (equal likelihood of benefit and harm). With a variance 75 times higher than the base case, the modified uninformed curve reflected extreme uncertainty and pessimism while avoiding substantial weight placed on improbable results. The same null hypothesis was assumed.

RESULTS

Prior Probability Curves

With a mean of 0.15% and SD of 0.08%, the base case prior probability curve had a higher probability of all positive survival benefit levels than the expert opinion curve (mean = 0.10%; SD = 0.09%). The modified uninformed curve was close to a straight line, with a 15.4% probability of survival decrement and about 30% probability for each positive survival benefit level less than 7% (Fig 2A and Table 2).

Base Case Analysis

After sequentially updating with IALT, JBR.10, and ANITA, the base case curve shifted toward a higher cumulative probability of survival benefit (Figs 2A through 2D). The Q statistic across all three trials (1.48; P = .85) was not statistically significant, suggesting it was appropriate to analyze the trials in combination.

For the base case, the probability of any OS benefit increased from 48% before IALT to 86% after IALT, and 89% after sequential updating with JBR.10 and ANITA (Table 2). The probability of a 5-year 4% OS benefit increased from 33% before IALT to 64% after IALT, and 82% after sequential updating. The probability of a survival decrement was less than 1% after IALT.

The HR was higher after sequential updating than when JBR.10 or ANITA were considered in isolation, but lower than for IALT (Fig 3). Uncertainty about the HR decreased after sequential updating and the 95% CI narrowed. IALT contributed the most information to the final HR estimate: the variance decreased by 61% after IALT (0.0025) and an additional 28% after JBR.10 and ANITA (0.0018; see Appendix, online only).

Fig 3.

Hazard ratio for death before and after updating with base case (gold), expert opinion (blue), and modified uninformed curves (gray).

Sensitivity Analysis

Expert opinion survey results.

With a response rate of 85% (n = 17), respondents had practiced a median of 11 years (interquartile range, 5 to 17; 85% North American, 15% European). Before IALT, 16.7% estimated an OS benefit for adjuvant chemotherapy (mean benefit = 2.9%; SD = 3%) and 11.1% offered it to patients. After the results of the IALT were presented, 72% estimated an OS benefit (mean benefit = 4.5%; SD = 0.51%).

Sequential updating sensitivity analysis.

The probability of any survival benefit was robust to the choice of the prior probability curve: after each updating, the curves move closer together (Fig 2). After sequential updating for the expert opinion (uninformed) curve, the probability of any survival benefit increased from 43% (35%) before IALT to 85% (61%) after IALT and 89% (88%). After IALT, the chance of a survival decrement remained less than 1% (Table 2).

In contrast to the probability of OS benefit, the probability of specific OS benefit levels did not consistently increase to more than 50% until the sequential updating of all three trials. For example, the probability of a 5-year 4% absolute OS benefit increased from 21% (uninformed curve, 30%) before IALT to 43% (uninformed curve, 52%) after IALT and to 77% (uninformed curve, 81%) after sequential updating for the expert opinion curve. Similar to the base case, sequential updating produced a precise HR for death, and IALT decreased the variance the most: by 65% (uninformed curve, 83%) after IALT (0.0028 [uninformed curve, 0.0020]) and an additional 29% (uninformed curve, 38%) after sequential updating (0.0020 [uninformed curve, 0.0026]) for the expert opinion curve.

Stage analysis.

Exploratory subgroup analysis using stage-specific priors from meta-analysis results suggested that even after sequential updating, the probability of any OS benefit for stage I NSCLC never exceeded 37% and the maximal OS benefit was 2% (Table 3). However, for stage II and III NSCLC, there was a more than 90% probability of a 6% survival benefit and a 50% probability of a 7% survival benefit.

Table 3.

Base Case (meta-analysis) Stage-Specific Probability of a Survival Benefit for Cisplatin-Based Adjuvant Chemotherapy for NSCLC Before and After Sequential Updating: Base Case by Stage

5-Year Survival Benefit	Probability of Survival Benefit (%)			Updated Probability of Survival Benefit (%)
	Before IALT			After IALT			After IALT and JBR.10			After IALT, JBR.10, and ANITA
	Stage I	Stage II	Stage III	Stage I	Stage II	Stage III	Stage I	Stage II	Stage III	Stage I	Stage II	Stage III
Any benefit	47	49	49	45	79	87	40	92	95	37	96	96
Maximum benefit with > 50% probability	2.0	2.0	2.0	1.0	5.5	6.3	1.0	6.6	7.0	1.0	7.0	7.1
Survival decrement	1.5	1.5	1.5	1.0	1.0	1.0	1.0	< 1.0	< 1.0	1.0	< 1.0	< 1.0
≥ 2% benefit	42	45	45	41	75	83	38	89	94	37	96	96
≥ 4% benefit	30	35	36	26	61	67	25	75	82	30	90	91
≥ 6% benefit	13	20	22	9.0	46	53	4.0	56	64	19	79	80

The probability of each survival benefit level before the results of IALT (base case by stage based on meta-analysis results), the updated probability of survival benefit after IALT using stage-specific results, after sequentially updating with JBR.10 using stage-specific results, and after sequentially updating with ANITA is shown.

Abbreviations: NSCLC, non–small-cell lung cancer; IALT, International Adjuvant Lung Trial; ANITA, Adjuvant Navelbine International Trialist Association.

DISCUSSION

To our knowledge, this study is the first Bayesian analysis to quantitatively evaluate the results of sequentially published oncology therapeutics trials in context with prior conflicting evidence. Our analysis supports a survival benefit for cisplatin-based adjuvant chemotherapy for resected NSCLC, consistent with other meta-analyses.^21,22,75 Uniquely, we show that although IALT contributed the most information, it was not sufficient to establish an OS benefit. The HR for death after sequential updating was confirmed by sensitivity analysis and was higher than JBR.10 or ANITA in isolation and lower than IALT in isolation. In addition, our results suggest there is a low probability of a clinically meaningful survival benefit for stage I NSCLC. Furthermore, exploratory analysis suggests the 5-year absolute OS benefit for stage II and III may be as high as, but is not likely to exceed, 7%.

The need for additional trials to substantiate the existence of a survival benefit is consistent with the controversy engendered by the publication of the IALT. The modest OS benefit found in our study is supported by the negative preliminary analysis of carboplatin-based adjuvant chemotherapy for stage IB NSCLC in the Cancer and Leukemia Group B protocol 963.⁷⁶ This finding emphasizes the need to consider the positive cisplatin-based chemotherapy trials in context with prior negative results.

Our study illustrates that Bayesian methods provide unique and complimentary information to other types of analyses. In contrast to frequentist methods, which calculate a P value for the probability of the trial result given no survival benefit, Bayesian analysis determines the probability of a survival benefit given the trial result, the relevant information for clinical decision making.^77,78 In addition, Bayesian methods provide the probability of each survival benefit level as opposed to the point estimate and CI produced by frequentist analysis. Furthermore, Bayesian approaches combine different types of evidence, borrowing strength from each to comprehensively evaluate a clinical question.^77,78 Finally, our Bayesian analysis quantitatively integrated data over time, mirroring the evaluation process performed by clinicians as new data are published.

Our findings must be considered within the limitations of the study. Although all three trials evaluated were randomized, controlled trials and tested the same hypothesis, there were differences between the trials, and summary statistics were used. Notably, JBR.10 evaluated stages I to II, whereas IALT and ANITA assessed stages I to III. Other differences were cisplatin dose, couplet choice, and radiation use.⁷⁹ Therefore, we evaluated this potential bias. First, the Q statistic was not significant (ie, the null hypothesis of homogeneity was not rejected). Second, the variance of the posterior was smaller than the variance of the prior, also indicating lack of statistical heterogeneity. Additionally, prior work suggests minimal information may be lost when summary statistics are based on large studies.⁸⁰ Therefore, we believe potential bias owing to dissimilar populations is minimal. (For further discussion, see Appendix, online only).

We acknowledge that the use of a prior probability distribution in Bayesian analysis has been criticized as subjective.⁸¹ Therefore, we used three approaches to control for potential biases: (1) the base case prior probability distribution was derived from data published before the initiation of the IALT, (2) a sensitivity analysis was performed, and (3) a modified uninformed curve was constructed to reflect extreme pessimism and uncertainty. The sensitivity analysis explicitly allowed a range of interpretations of the prior evidence to be formally expressed and evaluated.²⁸ In addition, the assumption of equipoise reduced the risk that a priori knowledge unduly influenced the final result. Finally, although the expert opinion survey was administered after IALT was presented, the full study had not been published,⁶ and the effect of timing appeared minimal: responses were biased against a positive benefit.

With the initial analysis undertaken at a time of great uncertainty, our results demonstrate that Bayesian analysis can highlight sources of the controversy: when considered in context with evidence available at the time of publication, the IALT did not support a 4% survival benefit, even though it did in isolation. Therefore, additional evidence was needed to strengthen the results. This need for additional data suggests that the evidence against the null hypothesis may be weaker than implied by the P value (ie, the P value can be much smaller than the posterior probability for the same hypothesis).^78,82 Therefore, Bayesian methods increase the quantitative rigor of trial evaluation and may decrease the number of false results accepted by the oncology community. Potential applications include clarifying the role of breast magnetic resonance imaging^83,84 and adjuvant chemotherapy for stage II colon cancer.^85,86 Bayesian analysis may be particularly important when new findings may change clinical practice, the comparator arm of future trials, or the focus of basic science research. This study offers a potential model for evaluating future studies whose results contradict prior evidence.

Appendix

Methodologic Details

Given the large sample size of each trial and the fact that individual level data were not available, the hazard ratio (HR) for each trial was assumed to follow a lognormal distribution (equivalently, the logarithm of the HR follows a normal distribution), and all calculations were based on HR. The mean and the standard deviation (SD) of the HR for each trial was inferred from the reported CIs using one of the following equations, which describe the lower (LL) and upper (UL) limits of the CIs for the HR:

For ease of clinical interpretation, results are presented as HRs and as 5-year survival increments. Because the exponential distribution has a single parameter (death rate), it is possible to convert the HR to 5-year survival probabilities without requiring additional information or making further assumptions.

We weighted trials with more precise information (ie, less variability) such that they contributed more information to the analysis. To achieve this weighting, we derived the variance of the probability distribution for the HR from the SE. This approach is similar to weighting the HR inversely proportional to the SEs.

Formulating the Prior Probability Curve

The base case prior probability curve formulations follow the description in Methodologic Details above: the log(HR) of the prior was assumed to have a normal distribution with the mean equal to the mean log(HR) of the meta-analysis and the SD inferred from the CIs of the meta-analysis. Conversion from HR to 5-year survival probabilities was performed using the exponential distribution.

Given that the null hypothesis (θ) of the adjuvant chemotherapy trials (and this Bayesian analysis) was “no survival difference between the treatment and control groups” (θ = 0), modification to Bayes formula was required. A continuous prior for θ implies a probability of zero for θ = 0, the posterior probability (after updating) of θ = 0 will also be zero. For this reason, we chose a mixture prior which allocates a portion (π₀) of the prior probability to the null hypothesis (θ = 0) and distributes the rest to all the possible values of HR according to a lognormal distribution, an approach extensively studied in the literature [Berger JO: Statistical Decision Theory and Bayesian Analysis (ed 2). New York, NY, Springer-Verlag, 1985]. Because equipoise is generally required to ethically initiate a trial and there was uncertainty in the literature about the survival benefit of adjuvant chemotherapy, the probability of the null hypothesis (π₀) was assumed to be 50%. With 50% of the weight at 0%, the remaining probability (1−π₀,) was distributed over the real line using a continuous prior, f(θ). Prior information about θ is summarized as:

Updating With Trial Results

Conversion of trial results into the logarithm of the HR.

In large samples, the distribution of the HR remains skewed, whereas the logarithm of the HR can be approximated by a normal distribution. Therefore, we assigned θ to be logarithm of the HR with a normal distribution, an approach successfully used by Faraggi and Simon (Faraggi D, Simon R: Stat Med 16:2573-2585, 1997). Therefore,

and θ̂ is used for X (the “new data”) in the analysis. Bayes theorem now appears as:

Bayesian calculation.

Using Bayes theorem, the above prior using π₀ and f(θ) translate into the following posterior distribution:

In this expression, p₀ is the posterior probability of the null hypothesis and is given by:

where Bayes Factor (BF) is a common Bayesian summary statistic for hypothesis testing. BF is calculated as:

Using the above formulas, BF and then p₀ were calculated (Kass RE: J Am Stat Assoc 90:773-795, 1995). In addition, the entire prior and posterior probability functions (equations 3 and 6, respectively) were used to plot the probability of benefit.

We considered γ to be known and fixed. Because it is difficult to sequentially update γ in the Bayesian sequential updating framework without access to the full data, our decision was dictated the type of data available.

Sensitivity Analysis Prior Probability Curves

Expert opinion prior survey.

Survey respondents were asked: “Before the American Society of Clinical Oncology 2003 meeting, what would have been your best estimate of the effect of cisplatin-based adjuvant chemotherapy on overall 5-year survival (for patients with resected non–small-cell lung cancer in absolute percentage)?” In addition, four separate questions ascertained the probability of each specific survival benefit level: “Before the American Society of Clinical Oncology 2003 annual meeting, how would you have assessed the chances of any (a 3%, a 5%, or a negative) 5-year survival benefit for patients receiving cisplatin-based adjuvant chemotherapy (non–small-cell lung cancer)?”

Construction of the expert opinion prior probability curve.

The mean response for each specific survival benefit and for the estimated average absolute benefit was calculated. A cumulative normal distribution was fitted to these means and the distribution was obtained by minimizing the sum of squares between the fitted line and the observed means using the OPTIM function in R (http://www.cran.r-project.org).

Evaluation of Heterogeneity Between Trials

The Q statistic was calculated and evaluated at the P = .05 level as described in the text. To further assess for trial heterogeneity at each sequential update, we compared the variance of the posterior with the variance of the prior. In the setting of sequential updating, heterogeneity between trials results in a posterior variance that is larger than the prior variance. In our analysis, the posterior variance was smaller than the prior variance at each update. Therefore, we considered the level of trial homogeneity to be acceptable within the Bayesian framework.

It is plausible that future applications of Bayesian sequential analysis may include situations with strong evidence against homogeneity. Potential strategies to address this issue are discussed in the Bayesian meta-analysis literature, and different situations may require unique solutions (McMahon PM, Zaslavsky AM, Weinstein MC, et al: Med Decis Making 26:497-511, 2006; Sutton AJ, Abrams KR: Stat Methods Med Res 10:277-303, 2001; Sutton AJ: Methods for Meta-Analysis in Medical Research. Chichester, NY, J. Wiley, 2000; White IR, Higgins JP, Wood AM: Stat Med 27:711-727, 2008; Conlon EM: Funct Integr Genomics 8:43-53, 2008; Schmidt FL, Raju NS: J Appl Psychol 92:297-308, 2007; Prevost AT, Mason D, Griffin S, et al: Psychol Methods 12:434-450, 2007; Baker R, Jackson D: Health Care Manag Sci 11:121-131, 2008; Brannick MT: J Appl Psychol 86:468-480, 2001). As a potential advantage of a Bayesian meta-analysis, covariates can be investigated for sources of heterogeneity (Dixon DO, Simon R: Stat Med 11:13-22, 1992; Sutton AJ, Kendrick D, Coupland CA: Stat Med 27:651-669, 2008; Nam IS, Mengersen K, Garthwaite P: Stat Med 22:2309-2333, 2003; Warn DE, Thompson SG, Spiegelhalter DJ: Stat Med 21:1601-1623, 2002; Nixon RM, Thompson SG: Health Econ 14:1217-1229, 2005).

Contribution of the Prior

One commonly used method to measure the contribution of the prior to the final results is to calculate the ratio of the prior precision to the posterior precision (precision is defined as the reciprocal of the variance). A high value for this ratio indicates a high contribution from the prior (Gelman A: Bayesian Data Analysis (ed 2). Boca Raton, FL, Chapman & Hall/CRC, 2004). For our analysis, this ratio was 0.29 for the base case, 0.33 for the expert opinion, and 0.03 for the skeptical prior. Although there are no established thresholds against which to compare these ratios, they all point to the strong influence of the data on the posterior. This finding supplements our conclusion that our analysis was relatively robust to the choice of prior.

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Although all authors completed the disclosure declaration, the following author(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.

Employment or Leadership Position: Thomas G. Roberts Jr, Noonday Asset Management, L.P (C) Consultant or Advisory Role: Thomas J. Lynch, Astra Zeneca (C), Genentech (C), OSI (C), Roche (C), Chugai (C), Merk-Sorono (C), Millenium (C), Sanofi (C), Lilly (C), Xelixis (C) Stock Ownership: None Honoraria: None Research Funding: None Expert Testimony: None Other Remuneration: Thomas J. Lynch, Genzyme, Inc

AUTHOR CONTRIBUTIONS

Conception and design: Rebecca A. Miksad, Mithat Gönen, Thomas J. Lynch, Thomas G. Roberts Jr

Administrative support: Rebecca A. Miksad

Provision of study materials or patients: Rebecca A. Miksad, Thomas G. Roberts Jr

Collection and assembly of data: Rebecca A. Miksad, Thomas G. Roberts Jr

Data analysis and interpretation: Rebecca A. Miksad, Mithat Gönen, Thomas J. Lynch, Thomas G. Roberts Jr

Manuscript writing: Rebecca A. Miksad, Mithat Gönen, Thomas G. Roberts Jr

Final approval of manuscript: Rebecca A. Miksad, Mithat Gönen, Thomas J. Lynch, Thomas G. Roberts Jr