Risk calculators are useful but…

Lobectomy is typically considered the optimal treatment of clinical stage I non-small cell lung cancer (NSCLC) [1]. The risk of lobectomy has been extensively studied and models for morbidity have been developed [2]. In this current study, Samson and colleagues have evaluated the performance of a risk calculator developed from the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) by retrospectively examining their own clinical practice [3]. They found that the NSQIP tool underestimated morbidity. They also found that risk predicted by the NSQIP tool was not necessarily aligned with their institution's actual treatment selection for clinical stage I NSCLC.

This study potentially has important clinical implications. Not all patients with stage I NSCLC will tolerate lobectomy from a physiologic standpoint and alternative therapy must be considered. Perhaps more importantly, both sublobar resection and non-surgical treatment with stereotactic body radiation therapy (SBRT) have been suggested to be oncologically adequate for some subsets of patients with clinical stage I NSCLC [4-5]. Randomized trials that will hopefully provide evidence on if and when alternatives to lobectomy should be considered are currently being performed [6] but clearly the availability of alternatives to lobectomy must be factored into the risk and benefit assessment for specific patients. This present study has shown that even a robust, well-managed and designed tool from NSQIP does not adequately stratify surgical risk. Importantly the study's results suggest that using the NSQIP tool may not have impacted the therapeutic decision between wedge resection or SBRT at their own institution, which has extensive experience with both modalities. Their analysis implies that the treatment decision made by the institutional clinicians is optimal.

However, several factors limit the rigor of their findings. The retrospective analysis was based on a small single-institution database. Moreover, the utility of the NSQIP risk score was evaluated by checking how well it distinguished patients who received surgery or SBRT. The lackluster performance of the NSQIP score is understandable, as it was not designed to optimally differentiate patients who benefit most from surgery or SBRT. Randomized clinical trials or well-controlled prospective observational studies are needed to develop and validate specific predictive tools for optimal treatment selection. These models must consider not only treatment morbidity but also the cost of possible recurrence with each therapy. Decision-theoretic framework [7-9] can evaluate a treatment selection signature Y (e.g. the NSQIP score or a new treatment selection index). Assuming the target event (D=1) is recurrence within 3 years after surgical resection or SBRT, surgical resection is the standard treatment (T=1) and SBRT is the alternative treatment (T=0); we define the following utilities dependent on the value of the treatment selection signature Y:

• C_D(Y) is the burden of the targeted event (recurrence) in the presence of SBRT (D = 1, T = 0).

• C_T(Y) is the burden of surgical resection (morbidity) in the absence of the targeted event (recurrence) (D = 0, T = 1).

• C_DT(Y) is the burden (recurrence + morbidity + additional cost) having both the surgical resection and the targeted event (D = 1, T = 1).

• C(Y) is the burden of no surgical resection and no targeted event (D = 0, T = 0) and it is set to zero without loss of generality.

The optimal treatment rule is that a patient receives surgical resection (i.e. Y+ = 1) if the benefit of surgical resection exceeds the burden of surgical resection, i.e., if Δ_s(Y) = P(D = 1|T = 0, Y)C_D(Y) − P(D = 1|T = 1, Y)(C_DT(Y)- C_T(Y)) > C_T(Y) [7]. The population expected benefit of the treatment selection signature can be evaluated and is the difference in the expected burden associated with treating those who benefit from surgery (Y+ = 1) with surgery versus treating everyone with surgery: E(Δ_s(Y)-C_T(Y)|Y⁺)∙P(Y⁺), where E() denotes an average taken over the population. The optimal treatment rule (Y+ = 1) if Δ_s(Y) > C_T(Y) maximizes the expected benefit.

Until such a specific predictive tool is developed and validated, the findings of this current study cautions on basing clinical decisions on data-driven tools that are inherently limited by the variables available for their models. One important NSQIP limitation is that specific pulmonary function is not used to characterize risk of lung resection [10]. The NSQIP tool also highlights that simple categorization of characteristics such as functional status, steroid use, and diabetes cannot replace a clinician's personal assessment of whether an elderly patient is independent but frail, whether chronic steroid use actually increases surgical risk, or whether diabetes is poorly controlled. Perhaps the most important conclusion that can be drawn from this present study is that current risk assessment tools can be helpful but cannot replace evaluation by clinicians for whom all management options are available when therapy is chosen for a specific patient.

Central Message.

Current surgical risk assessment tools are helpful but cannot replace clinical evaluation that considers all therapeutic options for early-stage NSCLC.