Plasma-only ctDNA-guided MRD Detection in Patients with CRC - Response

We appreciate the comments from the Letter to the Editor authored by Natera (maker of the Signatera MRD test) and collaborating investigators at Arhaus University. While we respectfully disagree with the criticisms raised, as detailed below, we believe that this discussion highlights the need for the academic community to create a consensus on what analyses and endpoints should be considered standard when evaluating new approaches for the detection of minimal residual disease (MRD).

First, the authors of this Letter to the Editor suggest that the endpoint used in our study is “unconventional” as it represented analysis of a landmark post-definitive therapy timepoint, defined as ~30 days post-surgery for patients undergoing surgery alone or ~30 days post-adjuvant therapy for patients receiving surgery and adjuvant therapy. We strongly disagree. A post-definitive therapy timepoint has been used in multiple previous MRD studies, including several of the key foundational studies in the MRD field(1–3). In fact, Reinert et al also used a “post-definitive-treatment” endpoint as the primary analysis in their study (Figure 2D and eFigure 7, Reinert et al, JAMA Onc 2019)(4). The rationale for a post-definitive therapy analysis is that for each group of patients, this is the point from which curative-intent therapy is complete and patients undergo standard observation only(5). Therefore, whether a patient develops tumor recurrence or not following this key assessment is purely dependent on whether residual disease existed at the time of the post-definitive therapy analysis, and importantly is not potentially affected by any intervening therapy. If one were to utilize a post-operative timepoint only, then for patients who went on to receive adjuvant therapy, one could not know for sure if a patient did not recur because they did not have disease post-operatively, or if they were subsequently cured by adjuvant therapy. Notably, we also provide analyses for each group separately (Supplementary Table 4). For the authors of this Letter to characterize the use of this endpoint as “inappropriate and misleading”, when it has been used previously in multiple studies, and as the primary analysis in their own study, is both meritless and irresponsible.

To further the discussion around standards in MRD studies, we believe that “landmark” (~30 day post-intervention) sensitivity should be a standard endpoint in MRD studies, as the first 1–2 months after completion of surgery or other definitive therapy represents the window in which clinical decisions based on MRD testing—whether to proceed with observation, or whether to escalate or deescalate therapy—are ideally made. Thus, the percentage of eventual recurrences that can be predicted at this early point is most clinically relevant. This is why the majority of MRD studies have included or focused on a landmark or 30-day timepoint analysis and also why we chose to utilize this endpoint for the primary analysis of our study(1–3). Of note, though they claim our study endpoint is “unconventional”, Reinert et al authored one of the few MRD manuscripts that does not directly report a landmark sensitivity number. The reader is forced to calculate the landmark 30-day sensitivity, which from the data they provide in Fig. 2A is 41% (7/17 recurrences detected). We believe reporting a landmark sensitivity should be a standard endpoint for all MRD studies due to its key clinical relevance.

The second criticism raised by the authors of this Letter is factually inaccurate: Patient TPS1473 had a clinical follow up of only 292 days post-completion of definitive therapy (not more than a year, as they claim). Given that the patients in our cohort had varying lengths of clinical follow up post-definitive therapy, we approached specificity in two ways. First, we reported specificity for all patients on the study (95.4%). However, it was noted that the only two patients that had detectable ctDNA post-definitive therapy (of 17 total) but did not recur by the data cut had limited follow up of less than 1 year. Thus, it was not possible to determine whether these patients represented actual false positives or if these patients might go on to recur with additional follow up. Accordingly, we also reported that the specificity value for patients with at least 1 year of clinical follow up post-definitive therapy was 100%. These methods are clearly noted in the manuscript.

Finally, the authors of this Letter raise issue with the exploratory surveillance analysis performed in the study, which was based on the very methods from their study, in which they excluded 50 patients without surveillance samples (Table 1, Reinert et al, JAMA Onc 2019)(4) from their comparable analysis. As we noted in our manuscript, the lack of systematic, regularly scheduled serial draws for most patients in our cohort limited our ability to perform a comprehensive assessment of longitudinal monitoring, and thus these analyses were not the primary focus of the study. However, to best approximate the surveillance analysis performed in Reinert et al, we assessed a cohort of patients with a serial draw within 4 months of recurrence, since the majority of patients with longitudinal draws in Reinert et al had systematic draws every 3–6 months during the study and all but 2 patients who recurred (~90%) had a draw within ~4 months of recurrence(4). Importantly, as the basis for their analysis, Reinert et al EXCLUDED from their surveillance analysis at least 7 patients (specifically, patients 20, 89, 92, 99, 103, 108, 125) who were false negatives in the overall analysis but who lacked surveillance draws, and this is the ONLY reason we similarly excluded such patients in our surveillance analysis in an effort to replicate these methods. If the authors of this Letter believe that doing so “artificially inflates sensitivity”, then this criticism should be first directed at their own study, as this method increased the sensitivity in their study from ~41% at the landmark timepoint (as calculated above) to the 88% reported in their surveillance analysis.

Furthermore, to ensure clarity of our analysis, we also reported the sensitivity including longitudinal draws, but without excluding patients who lacked surveillance draws (which we termed the longitudinal sensitivity and which was not done in Reniert et al), as has been done previously(3). We also provided a dedicated figure (Figure 3B) to help the reader understand how each value was calculated(5). It is also worth noting that Reinert et al scored one patient as “ctDNA positive” in their surveillance analysis (patient 77, eFigure 7), even though all of this patient’s draws were ctDNA-negative up until the time of recurrence and did not turn positive until a single specimen obtained 2 draws (~5 months) after recurrence, after having surgery and initiating chemotherapy(4). As the clinical utility of detecting MRD through a ctDNA assay ~5 months after the patient has already experienced a clinical recurrence and undergone surgical and systemic therapy is negligible, it is hard to see why it would be appropriate to count this patient as “detected” in this analysis, and without this result, longitudinal sensitivity might be more appropriately reported by Reinert et al as 81% (13/16), instead of 88%. This is another example of how consistent standards in data reporting for MRD studies would be valuable.

Overall, we believe that ctDNA-guided MRD detection has the potential to have a transformative impact on cancer care. The purpose of our study was to perform an initial assessment of the feasibility of plasma-only MRD detection and not to endorse a tumor-uninformed or tumor-informed approach over the other. As we note in our manuscript, our study is limited by modest sample size. Further evaluation of this assay, as well as other MRD assays (whether tumor-informed or uninformed), in larger observational or prospective cohorts are needed to more thoroughly assess performance characteristics of each assay. However, this discussion highlights the need for the academic community to propose a consensus on the standard analyses and endpoints to be used in evaluating novel MRD technologies. One example of this is a recent whitepaper published by the National Cancer Institute Colon and Rectal-Anal Task Forces, in which an expert panel of academic investigators was assembled to provide guidance on the development and integration of ctDNA technologies(6). Among several recommendations, this panel highlighted the importance of landmark MRD assessments 2–8 weeks after surgery or completion of all curative-intent therapy. Continued academic efforts of this nature to set standards for this important and rapidly developing field would be valuable.

In closing, we believe it is also important to note that while the above Letter to the Editor was co-authored by Natera, a company in the MRD space, our response was written without any input from companies whose technology was assessed in or related to our study.

Abbreviations:

ctDNA

circulating tumor DNA

MRD

minimal residual disease