Prognosis, Effect Modification, and Mediation

One of the first questions cancer patients ask on diagnosis is: “What are my chances, doctor?” On hearing more about treatment options, they may further ask: “Yes, but will any of those work better or worse for me?” A sophisticated patient might then question: “But why does that treatment work better?” Epidemiologists describe these questions in terms of prognosis, effect modification, and mediation.

Prognosis

A good example of a prognostic factor is Gleason score. A study of radical prostatectomy patients might report that men with a high Gleason score have a greater risk of metastasis (eg, 10% at 10 yr) than men with an intermediate Gleason score (3% at 10 yr). A common mistake is to assume that an adverse prognostic factor is an adverse effect modifier. For example, investigators might conclude from the surgery study that because patients with high Gleason score have poor outcomes after surgery, they are not good surgical candidates and should receive an alternative treatment approach. This is to confuse outcome with response [1]: if a group of patients have a poor outcome with surgery, say, a 10% risk of metastasis, that does not imply that they responded poorly, for instance, risk of metastasis without surgery might be considerably higher than 10%. The relative benefits and harms of different treatments for different subtypes of prostate cancer can only be derived from a study that compares different treatments [2,3]. Gleason score would only be an effect modifier if the effects of a treatment such as radical prostatectomy differed between men depending on their Gleason score.

Effect modification

One of the most common ways to evaluate effect modification is to conduct a subgroup analysis of a randomized trial, and this has been done for prostate cancer surgery. The SPCG4 trial [4] randomized patients with localized cancer to surgery or conservative management. The investigators reported a relative risk of 0.56 for the endpoint of prostate cancer mortality. However, they also went on to conduct subgroup analysis, reporting relative risks of 0.54 (p = 0.17), 0.38 (p < 0.001), and 0.87 (p = 0.8) for National Comprehensive Cancer Network (NCCN) low, intermediate, and high-risk groups, respectively.

These findings illustrate the two key problems with subgroup analysis of randomized trials. First, they are underpowered. For instance, note that the relative risk for low-risk disease (0.54) is almost identical to that for patients overall (0.56); however, differences are nonsignificant (p = 0.17), likely because there are fewer patients and events in the low-risk subgroup. Second, subgroup analyses can often be confounded by variables not balanced between arms. For example, the reason why there is only a small relative risk reduction in high-risk patients is likely because high-risk patients tend to be older and so more often die of other causes before they die of prostate cancer.

One alternative to subgroup analysis is to create a multivariable model to predict the benefit of treatment. This has been attempted for the SPCG 4 trial [5]. The approach was to create a statistical model predicting the risk of death for patients in the control group based on age, stage, grade, and prostate-specific antigen. A second model was created for patients in the surgery group. The difference between the estimates from each model gives an estimate of benefit for the individual patient (Fig. 1). Depending on patient characteristics, this varied from a 25% absolute risk reduction (younger patients, with high Gleason and stage T2) to 0% risk reduction (older patients with low-risk disease).

Fig. 1.

Absolute reduction in risk of death at 10 yr associated with surgery by age. Grey line: Gleason 8+; Dashed line: Gleason 7; Black line: Gleason 6.

It is straightforward to demonstrate that a prognostic factor must also be an effect modifier, either in terms of absolute risk reduction or relative risk reduction, or both. In Figure 2, risk for the control group in a randomized trial is compared with two treatments, one of which is associated with a constant relative reduction and the other with a constant absolute risk reduction. Note that if the relative risk reduction is constant, the absolute risk reduction varies (eg, from 5% for a low level of the prognostic factor to 20% for a high level). On the other hand, if absolute risk reduction is constant, the relative risk reduction varies (eg, 50% at low levels of the prognostic factor to a little over 10% at high levels).

Fig. 2.

Probability of an adverse outcome (eg, death) in a randomized trial plotted against the level of an adverse risk factor (eg, prostate-sdoubling time at recurrence). Black line: Control group. Light grey: Treatment group where treatment has a constant relative risk reduction, a 50% reduction in risk. Dashed line: Treatment group where treatment is associated with a constant absolute risk reduction of 5%. Note that if relative risk reduction is constant different levels of the prognostic factor, absolute risk reduction varies and vice versa.

It is scientifically more challenging to evaluate a claim that a variable is an effect modifier without being a prognostic factor, or that effect modification is in excess of prognostic impact. In the typical set of subgroup analyses for say, an anticoagulant trial, investigators might look at age, gender, and diabetes and conclude that the drug lowered the risk of a heart attack for younger men, or older women with diabetes. Of course, there is no particular reason why an anticoagulant should work better for men than for women or that its protective effects are modified by age or diabetes. To avoid this type of problem, investigators need to carefully consider the scientific literature and then specify a limited number of hypotheses about effect modification upfront. These hypotheses then need to be tested directly, for instance, by using an interaction analysis rather than by informally comparing two p values from subgroups with and without the modifying factor.

Effect modification has become of particular interest in oncology in recent years due to the development of targeted agents. There are generally no strong reasons to believe that the effects of a cytotoxic agent would work better or worse for one or other group of patients. But the effectiveness of targeted agents might well depend on host or tumor factors.

Mediation

An answer to the third question asked by patients, “Why does that treatment work better?” would be in terms of mediating factors. A mediating factor describes the mechanism of an effect. If one group of surgeons outperforms another group of surgeons concerning preservation of sexual health, the mechanism may be the degree of preservation of the neurovascular bundles. Based on the information the research team has beforehand, a statistical analysis plan may define variables that reflect possible mediating factors and variables that reflect possible confounding factors. The distinction guides the interpretation when comparing the results of univariable and multivariable analyses or when comparing two multivariable analyses that include different predictors. For instance, if we looked at erectile function in patients treated by the two groups of surgeons above and adjusted for tumor factors, age, baseline function, and comorbidity, we would find better erectile function in one group. If differences between groups were smaller when nerve preservation was added into the model, then we would conclude that nerve preservation was a mediating factor.

The text box shows the difference between prognostic, effect-modifying, and mediating factors in simple mathematical terms.

Text box

For didactic purposes, the examples below are simplified, with risk in arbitrary units.

Prognostic factor

$$\begin{gathered} \text{Risk of prostate cancer metastases}={\beta }_1\text{Surgery}(\text{vs no surgery})+{\beta }_2\text{NCCN risk group} \\ (\text{hgih vs low/intermediate})+c \end{gathered}$$

Here, risk group is a prognostic factor because it helps to predict a future event. For instance, if the coefficient for risk factor β₂) was 4, risk of metastases would be 4 points higher for NCCN high risk compared to NCCN low or intermediate risk.

Effect-modifying factor

$$\begin{gathered} \text{Risk of prostate cancer metastases}={\beta }_1\text{Surgery}+{\beta }_2\text{Risk group}+{\beta }_3\text{surgery}\times \text{Risk} \\ \text{group}+c \end{gathered}$$

Here, risk group is both a prognostic factor and an effect-modifying factor. Imagine that the coefficient for surgery (β₁) is −2, the coefficient for risk group (β₂) is 4, the coefficient for surgery × risk group (β₃) is −1, and the constant is 5. Risk would be as shown in Table 1.

Table 1.

Risk of prostate cancer metastases

Treatment	NCCN risk group
	High	Low/intermediate
Surgery	6	3
No surgery	9	5

NCCN = National Comprehensive Cancer Network.

There is a bigger difference between surgery and no surgery for the high compared to the low/intermediate-risk group and hence, risk is an effect modifier.

Mediating factor

To assess a mediating factor, investigators often run two models.

$$\begin{gathered} \text{Model 1}:\text{Risk of erectile dysfunction}={\beta }_1\text{Surgery type}+{\beta }_2\text{Age}+{\beta }_3\text{Comorbidity}+ \\ {\beta }_4\text{Baseline}+c \end{gathered}$$

$$\begin{gathered} \text{Model 2}:\text{Risk of erectile dysfunction}={\beta }_1\text{Surgery type}+{\beta }_2\text{Age}+{\beta }_3\text{Comorbidity}+ \\ {\beta }_4\text{Baseline}+{\beta }_5\text{Never resection}+c \end{gathered}$$

If the coefficient for surgery type (β₁) is statistically significant in model 1 but closer to zero in model 2, then we might conclude that nerve resection explains at least some of the effect of surgery type on outcome.