Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Nov 14.
Published in final edited form as: JCO Precis Oncol. 2025 Nov 6;9:e2500245. doi: 10.1200/PO-25-00245

Functional Characterization and a Real-World Clinical Laboratory Pilot of the Foundation for the National Institutes of Health Circulating Tumor DNA Quality Control Materials

Cloud P Paweletz 1, Thomas D Forbes 2, Laura Yee 3, Grace Heavey 1, Hua-Jun He 4, Zhiyong He 4, Dana Connors 5, Daniel Stetson 6, Susan Keating 7, Kenneth D Cole 4, Li Chen 2, Rini Pauly 2, Hua Bao 8, Xue Wu 8, Gary A Pestano 9, Amanda L Weaver 9, Manish Kohli 10, Sabine Hellwig 11, Adam S Corner 12, Andrew M Prantner 12, Stephen Q Wong 13, Stephen B Fox 13, Chelsee A Hewitt 13, Rainier Arnolda 13, Greg R Jones 14, Jonathan Craft 15, Melissa McConechy 16, Mawath A Qahtani 16, Florian Klemm 17, Fuad Mohammad 17, Caroline Sigman 7, Liang-Chun Liu 18, Qiang Gan 18, Yves Konigshofer 19, Russell Garlick 19, Ephrem Chin 20, Gary Kelloff 21, P Mickey Williams 2, Robert McCormack 22, Chris Karlovich 2
PMCID: PMC12614329  NIHMSID: NIHMS2110674  PMID: 41197083

Abstract

Purpose:

We previously developed quality control materials (QCMs) to aid in the development of circulating tumor DNA (ctDNA) assays. In this study, we further characterize the performance of the QCMs relative to clinical samples.

Methods:

QCMs were provided by 3 manufacturers. To functionally characterize the QCMs, we (1) evaluated EGFR L858R and ex19del (range 0.5–5.0% variant allele frequency [VAF]) in QCMs compared to clinical samples by droplet digital polymerase chain reaction (ddPCR), targeted-amplicon sequencing (Tag-seq), and hybrid capture next-generation sequencing (NGS); and (2) evaluated the QCMs and clinical samples near the Tag-seq limit of detection (LOD). A clinical pilot was also conducted in 11 clinical labs spanning 4 continents.

Results:

For functional characterization, part 1, QCM VAFs for hybrid capture were similar to ddPCR for EGFR L858R but lower for ex19del. In contrast, hybrid capture results for EGFR L858R clinical samples showed a positive trend compared to ddPCR. For amplicon NGS, QCMs performed similarly to clinical samples for both variants. For part 2, observed hit rates approximated expected values. In the clinical pilot, median ex19del VAF was higher for Tag-seq than hybrid capture for both 1.0% and 0.5% QCM formulations. Median QCM L858R VAFs were similar for Tag-seq and hybrid capture, with greatest inter-lab differences observed for Thermo Fisher Scientific (TFS) QCMs. For non-EGFR variants, we observed assay and QCM-dependent trends, with no particular QCM or assay driving these trends.

Conclusion:

This project revealed unexpected differences in performance of both assays and QCMs. These findings highlight the need for further validation across diverse alteration types and merit consideration by laboratories that rely on QCMs to develop and perform ctDNA assays for diagnostic applications.

Context Summary

Key Objective

To evaluate the performance of commercially available quality control materials (QCMs) from three manufacturers by comparing them directly to clinical circulating tumor DNA (ctDNA) samples across multiple assay platforms from different vendors, addressing the need for reliable and scalable validation materials for non-invasive cancer genotyping using liquid biopsies.

Knowledge Generated

The study demonstrates that the herein described QCMs can closely approximate the performance of clinical ctDNA samples. This comparison revealed assay- and laboratory-specific sources of variability, emphasizing the utility of QCMs in rigorous analytical validation.

Relevance

As ctDNA testing becomes more widely adopted in clinical oncology, the scarcity and heterogeneity of clinical specimens present significant challenges to assay validation. This work supports the broader use of standardized reference materials for assay development, validation, and quality assurance, facilitating more robust and reproducible ctDNA testing across laboratories and platforms.

Introduction

In patients with advanced cancer, ctDNA is now being primarily used to identify clinically actionable alterations for treatment selection, with several novel applications being developed, including the determination of minimal residual disease,1 therapy response monitoring,2 emergence of resistance,3 and early detection of cancer.4 A primary challenge for ctDNA assay development for noninvasive genotyping is the acquisition of clinical specimens in suitable quantity for analytical validation. Reference materials or QCMs fulfill a critical need for validation studies like LOD because pools of variants can be formulated in large quantities at precise levels. Ideally, these QCMs would have performance characteristics similar to clinical samples.

These QCMs could help address the discordance that has been observed previously when comparing between different ctDNA assays.5,6 A joint review from the American Society of Clinical Oncology and the College of American Pathologists highlighted the need for a standardized set of reference materials to facilitate cross-assay comparisons of analytical validation studies.7 In recognition of this need, several groups, including ours,8 have characterized commercially available standards that could potentially be used by all labs.912 In the initial phase of our project, we worked with 3 manufacturers of reference materials to manufacture a set of QCMs having variants shared by each, which we designated as the FNIH QCMs (Table 1).8 We confirmed fit for purpose of FNIH QCMs in 4 core laboratories using 3 NGS-targeted panels and 2 ddPCR assays.8

Table 1.

FNIH QCMs Variant List

Gene Variant Variant Type COSMIC ID Manufacturer
AKT1 E17K SNV 33765 HD; SC; TFS
ALK G1202R SNV 144250 SC; TFS
EML4-ALK EML4-ALKv1 Translocation COSF463 SC
EML4-ALKv3 Translocation COSF474 HD
BRAF V600E SNV 476 HD; SC; TFS
BRCA1 K654fs*47 del fs 1383519 SC; TFS
BRCA2 R2645fs*3 del fs 1738242 SC; TFS
EGFR L858R SNV 6224 HD; SC; TFS
EGFR T790M SNV 6240 HD; SC; TFS
EGFR E746_A750 del del in frame 6223 SC; TFS
E746_A750 del del in frame 6225 HD
ERBB2 A775_G776insYVMA ins in frame 20959 HD; SC; TFS
ERBB2 Amplification CNV N/A HD; SC; TFS
KRAS G12D SNV 521 HD; SC; TFS
CD74-ROS1 CD74-ROS1 Translocation COSF1201 HD; SC
PIK3CA H1047R SNV 775 HD; SC; TFS

Abbreviations: CNV, copy number variation; COSMIC, Catalogue of Somatic Mutations in Cancer; del, deletion; FNIH, Foundation for the National Institutes of Health; fs, frameshift; HD, Horizon Discovery; ID, insertions and deletions; ins, insertion; N/A, not applicable; QCM, quality control material; SC, SeraCare; SNV, single-nucleotide variant; TFS, Thermo Fisher Scientific.

Here, we report on a 2-part functional characterization study comparing the performance of the FNIH QCMs to clinical samples both near the LOD and within the quantitative range by ddPCR and NGS, with a focus on single-nucleotide variant (SNV) and insertion-deletion (indel) variant classes. We also report a real-world assessment of the FNIH QCMs by 11 clinical laboratories spanning 4 continents.

Methods:

Clinical samples were obtained from fully consented patients enrolled in TIGER-X (NCT01526928). VAFs were confirmed by ddPCR at the National Institute of Standards and Technology (NIST). QCMs were provided by Revvity Mimix (formerly Horizon Discovery; HD), TFS, and LGC Clinical Diagnostics (SeraCare; SC) and were formulated as cell-free DNA (cfDNA) in buffer for functional characterization and in either healthy donor plasma (TFS) or synthetic plasma (HD, SC) for the clinical pilot. QIAsymphony DSP Circulating DNA Kit (Qiagen, Germantown, MD, USA) was used for sample extraction.

Functional Characterization, Part 1: Quantitative Range Comparison

Nineteen unique clinical samples were separately prepared for EGFR L858R and ex19del. EGFR ex19del samples were from 4 isoforms (COSM6223, 6225, 12370, and 12384). Samples were diluted in pooled healthy donor cfDNA to 0.5–5.0% VAFs. Additionally, 1 unique sample for each variant type was serially diluted to 5, 2.5, 1, and 0.5% VAF, and 4 replicates of each dilution were used for the serial dilution testing. Samples were distributed to Dana-Farber Cancer Institute (DFCI) for Genexus-Oncomine and ddPCR for EGFR L858R and ex19del,13 NIST for EGFR L858R and ex19del ddPCR, and the Molecular Characterization Laboratory at Frederick National Laboratory for Cancer Research (FNLCR) for TruSight Oncology 500 (TSO500) ctDNA (Supplementary Methods).

Functional Characterization, Part 2: LOD Testing

Clinical samples were diluted in healthy donor cfDNA and QCMs in wild-type QCMs, provided by the respective QCM vendors, down to LOD95 (~0.25% VAF), LOD70, LOD50, and LOD20. Samples were distributed to DFCI for Genexus-Oncomine and to NIST for EGFR L858R and ex19del ddPCR (Supplementary Methods).

Clinical Pilot

Anonymized cfDNA from EGFR L858R and EGFR ex19del clinical samples were diluted in healthy donor cfDNA to 0.5% or 1.0% VAF. QCM samples in healthy donor plasma (TFS) or synthetic plasma (HD, SC) were provided pre-diluted by the vendors to 0.5% or 1.0%. Ten replicates of each sample type were provided to all pilot sites. Pilot sites were blinded to sample type (QCM vs. clinical) and the VAF of the samples. Clinical pilot sites using ddPCR were notified of the ex19del isoform present in the QCMs and clinical samples to ensure they used a relevant ddPCR assay. Each clinical pilot testing site used its respective extraction (for QCMs) protocols and assays (Supplementary Methods).

Results:

Functional Characterization Study Overview

The aim of the functional characterization study was to determine how similarly the QCMs perform relative to clinical samples. We evaluated EGFR L858R and EGFR ex19del as representatives of SNVs and indels, respectively, and conducted the study in 2 parts. We first compared QCMs to clinical samples from 0.5–5.0% VAF by TSO500 ctDNA, a hybrid capture NGS assay, Genexus-Oncomine, a Tag-seq assay, and ddPCR. Informed by these data, we chose Genexus-Oncomine for a second functional characterization study near the LOD.

Functional Characterization from 0.5–5.0% VAFs

Results for EGFR L858R are shown in Figure 1A and Supplementary Table 1. For TSO500, VAFs were similar to ddPCR for HD and SC QCMs, with TFS QCM VAFs trending slightly lower compared to ddPCR. By contrast, for the clinical sample dilution series, VAFs as determined by TSO500 were reproducibly higher than ddPCR. For Genexus-Oncomine, the clinical sample dilution series performed more similarly to the QCM dilution series, with TFS QCMs having the greatest similarity.

Figure 1.

Figure 1.

Functional characterization from 0.5–5.0% VAFs. (A) Results for EGFR L858R, the representative SNV profiled in the study. (B) Results for EGFR ex19del, the representative indel. The dashed black line in each graph represents the identity line where ddPCR and NGS VAFs are equal. “Clinical” in each graph legend refers to a dilution series made from a single clinical sample in healthy donor cfDNA. cfDNA, cell-free DNA; ddPCR, droplet digital polymerase chain reaction; indel, insertion/deletion; NGS, next-generation sequencing; SNV, single-nucleotide variant; Thermo Fisher, Thermo Fisher Scientific; VAF, variant allele fraction.

Results for EGFR ex19del are shown in Figure 1B and Supplementary Table 2. For TSO500, VAFs for the QCMs were all lower than observed by ddPCR. By contrast, a positive trend was observed for the clinical sample dilution series, where VAFs as determined by TSO500 were reproducibly higher than for ddPCR. For Genexus-Oncomine, the clinical sample dilution series of EGFR ex19del performed similarly to the QCM dilution series for the HD and SC QCMs. While a slight negative trend was seen in the TFS QCM dilution series compared with the clinical sample dilution series, ddPCR and Genexus-Oncomine data showed greatest similarity for TFS QCMs. The trends observed for both the EGFR L858R and ex19del dilution series were confirmed in a separate experiment, in which a total of 38 clinical samples harboring EGFR L858R and ex19del variants between 0.5% and 5.0% VAFs were tested (Supplementary Figure 1).

We further analyzed the size distribution of sequenced mutant and wild-type fragments from samples in the functional characterization study on an exploratory basis (Supplementary Figures 2, 3, and 4). No gross differences between clinical samples and QCMs were identified, which might have explained some of the trends observed in the functional characterization study (e.g., mutant fragments of a single size in some materials vs. mutant fragments having a distribution of sizes in clinical samples).

Functional Characterization near Assay LOD

Based on the similarity in performance between ddPCR and Tag-seq, we chose Genexus-Oncomine to further characterize the QCMs near the assay LOD (Supplementary Methods). For the LOD assessment, we made 4 dilutions, starting at ~0.25% VAF (LOD95, Dilution A), the presumed LOD95 for the assay, and extending down to what we expected would be LOD70 (Dilution B), LOD50 (Dilution C), and LOD20 (Dilution D) for each clinical sample and the QCMs. LOD95 represents the highest VAF at which at least 95% of replicates tested are expected to be detected. The hit rates are shown in Figure 2A and Supplementary Table 3 for EGFR L858R. Probit analyses of the data are shown in Supplementary Figure 5. Notably, the HD L858R hit rate at Dilution A was 10% (95% confidence interval [CI], 0–45%), which was much lower than expected. Additionally, the Dilution A hit rate for the L858R clinical sample did not trend with the other dilutions in the series. A separate reformulation of Dilution A, not presented in Figure 2A, showed close agreement with the expected higher hit rate (data not shown). This strongly suggests a formulation error was responsible for the lower-than-expected hit rate seen with the original formulation of Dilution A.

Figure 2.

Figure 2.

Functional characterization near assay LOD. (A) Dilution series hit rates for EGFR L858R. (B) Dilution series hit rates for EGFR Ex19del. Results were generated with the amplicon NGS platform. In each graph, points represent the proportion of 10 replicates with tested variant reported by the assay data analysis pipeline. Vertical bars represent Clopper-Pearson 2-sided 95% CIs. The starting dilution, Dilution A, was prepared at 0.25%, the presumed LOD95 of the amplicon NGS assay. Horizon, Horizon Discovery; LOD, limit of detection; NGS, next-generation sequencing; QCM, quality control material; Thermo Fisher, Thermo Fisher Scientific; VAF, variant allele fraction.

Hit rates for EGFR ex19del are shown in Figure 2B and Supplementary Table 3, and probit analyses of the data are shown in Supplementary Figure 6. Notably, the Dilution C hit rate for both the ex19del clinical sample and SC QCMs did not trend with the other dilutions in their respective series. To gain insight into the SC ex19del results, we looked at hit rates for other variants in the SC QCM pool and observed that all but 1 variant had progressively lower hit rates with each successive dilution, as expected (Supplementary Figure 7). We therefore believe the higher-than-expected hit rate for SC Dilution C, and likely for Dilution C of the clinical sample, to be a chance outcome associated with the wide CI of testing only 10 replicates at each dilution.

A Real-World Clinical Pilot of the FNIH QCMs

To understand the real-world applicability of the QCMs, we enlisted 11 laboratories across 4 continents to compare QCM performance to clinical samples across a variety of assays. Among these, 3 used targeted hybrid capture NGS, 5 used ddPCR, and 3 used Tag-seq (Assays 1–12; Supplementary Figure 8). Seven laboratories were affiliated with commercial diagnostic companies, and 4 were within academic institutions. Laboratories were asked to report on any of the 14 variants present in the QCMs and given the opportunity to analyze contrived clinical specimens containing EGFR L858R or ex19del at VAF of 1.0% or 0.5% (Supplementary Table 4). Each laboratory was provided enough material to run 10 replicates each.

We first sought to characterize the variability of EGFR L858R and EGFR ex19del between QCMs and clinical specimens. For EGFR L858R ddPCR, VAF tracked similarly for the QCMs and clinical materials, with the median VAF for clinical samples reported as 0.53% (n=58; interquartile range [IQR] 0.38–0.69) and 1.06% (n=60; IQR 0.71–1.35) for 0.5% and 1.0% formulations, respectively. This compared to the reported median EGFR L858R ddPCR VAF for QCMs formulated by HD of 0.55% (n=13; IQR 0.49–0.64) and 1.1% (n=14; IQR 0.67–1.15), for SC of 0.6% (n=18; IQR 0.46–0.69) and 1.22% (n=17; IQR 1.0–1.28), and for TFS of 0.51% (n=25; IQR 0.36–0.7) and 1.0% (n=26; IQR 0.65–1.2) (Figure 3; Supplementary Tables 5 and 6).

Figure 3.

Figure 3.

Clinical pilot study results for EGFR L858R and ex19del. (A) EGFR L858R results for clinical sample and QCMs. (B) EGFR ex19del results for clinical sample and QCMs. Results for the 8 evaluable assays among 11 laboratory clinical pilot participants are shown. QCMs were formulated at expected VAFs of 0.5% and 1.0%. In each graph, boxes represent the IQR of the distribution of reported variants from 25% to 75%. The horizontal line within each box represents the median. Vertical lines represent data points outside the IQR but less than 1.5× IQR. ddPCR, droplet digital polymerase chain reaction; Horizon, Horizon Discovery; Hyb, hybrid; IQR, interquartile range; MFG, manufacturer; QCM, quality control material; Thermo, Thermo Fisher Scientific; VAF, variant allele fraction.

The variability in the ddPCR cohort is largely driven by 1 assay (Assay 10). Similarly, the median VAF and IQR for Tag-seq were 0.51% (n=9; IQR 0.39–0.55) and 0.85% (n=10; IQR 0.71–1.00) for clinical samples, 0.39% (n=11; IQR 0.33–0.51) and 0.89% (n=11; IQR 0.67–0.98) for HD QCMs, 0.47% (n=13; IQR 0.41–0.59), 0.76% (n=13; IQR 0.67–0.83) for SC QCMs, and 1.12% (n=14; IQR 0.49–1.67) and 1.93% (n=14; IQR 1.03–2.25) for TFS, at 0.5% and 1.0%, respectively. Of note, the variability observed in the TFS specimens was driven solely by Assay 9, which reported VAF roughly twice as high. For hybrid capture NGS, median VAF and IQR were 0.61% (n=15; IQR 0.49–0.66) and 0.85% (n=13; IQR 0.71–1.00) for clinical samples, 0.47% (n=30; IQR 0.40–0.54) and 1.10% (n=30; IQR 1.00–1.35) for HD QCMs, 0.59% (n=30; IQR 0.52–0.67) and 1.05% (n=30; IQR 0.89–1.10) for SC QCMs, and 0.76% (n=21; IQR 0.59–0.89) and 1.56% (n=20; IQR 1.30–1.77) for TFS, at 0.5% and 1.0%, respectively (Supplementary Tables 5 and 6).

Similar to our EGFR L858R observations, VAF tracked similarly between QCMs and clinical materials for EGFR ex19del (Figure 3; Supplementary Tables 5 and 6), with the only assay-dependent variations for SC-formulated QCMs (n=28, median VAF 1.05%, IQR 0.65–2.13; n=30, median VAF 1.76%, IQR 1.33–2.32, at 0.5% and 1.0%) contributed by Assay 10 (ddPCR) and for TFS QCMs contributed by Assay 9 (Tag-seq) (n=14, median VAF 1.06%, IQR 0.45–1.56; n=14, median VAF 2.75%, IQR 1.06–3.34, at 0.5% and 1.0%). Of note, a negative QCM-dependent trend was observed for HD QCMs (n=26, median VAF 0.36%, IQR 0.30–0.46; n=30, median VAF 0.65, IQR 0.42–0.90, at 0.5% and 1.0%) for all hybrid capture NGS assays (Figure 3). Assay 1 did not detect either EGFR alteration at any VAF in the TFS QCMs. Notably, all laboratories employed an allele-specific assay targeting the specific EGFR ex19del present in the QCMs. One laboratory additionally utilized an EGFR drop-off assay in conjunction with the allele-specific assay, but only for clinical samples. No differences in performance were observed between the drop-off and allele-specific assays.

We next explored the variance of non-EGFR-activating alterations. Out of the 3523 results, 483 were by ddPCR (SNVs only), 509 by Tag-seq (493 SNVs and 16 insertions), and 1355 by hybrid capture NGS (919 SNVs, 120 insertions, 40 frameshifts, 216 translocations, and 60 CNVs; Supplementary Figure 8). As with EGFR L858R and ex19del, Assay 1 detected none of the TFS QCMs at either VAF. Overall, we observed assay- and QCM-dependent trends across the data, with no particular QCM or assay being the sole source for the variance (Figure 4A and B). Notably, all assays reported BRAF V600E for HD at twice the expected VAF, and all assays reported KRAS G12D for TFS at half the expected VAF. ERBB2 (p.A775_G776ins YVMA) was reported at twice the VAF by hybrid capture NGS assays for TFS (Assay 6) and HD (Assays 1 and 6), but not for SC-manufactured QCMs (Figure 4; Supplementary Table 6). Clear positive trends were observed for Assay 6 for ROS and ALK translocations for both SC and HD QCMs. Translocations in the TFS materials were not assessed (Figure 4C).

Figure 4.

Figure 4.

Clinical pilot study results for all variants assessed. (A) Indels reported by 8 labs that tested at least 1 indel. Assay 4 was an EGFR drop-off assay, whereas assays 7, 10, and 11 were allele-specific. (B) SNVs reported by 11 labs that tested at least 1 SNV. (C) Fusions reported by the 2 labs that reported fusion results. QCMs were formulated at expected VAFs of 0.5% and 1.0%. In each graph, boxes represent the IQR of the distribution of reported variants from 25% to 75%. The horizontal line within each box represents the median. Vertical lines represent data points outside the IQR but less than 1.5× IQR. ddPCR, droplet digital polymerase chain reaction; Horizon, Horizon Discovery; Hyb, hybrid; indel, insertion/deletion; IQR, interquartile range; QCM, quality control material; SNV, single-nucleotide variant; Thermo, Thermo Fisher Scientific; VAF, variant allele fraction.

Discussion:

A European Society for Medical Oncology working group recently concluded that sufficient evidence now exists to support the use of ctDNA assays in routine clinical practice to direct patients with late-stage disease to molecularly targeted therapies.14 With that in mind, the FNIH Biomarkers Consortium set out to develop a source of well-vetted QCMs that can be used for the rapid development and analytical validation of ctDNA assays for therapy selection in advanced cancer patients.

In our phase 1 study, we characterized QCMs for cfDNA from 3 manufacturers in 4 core laboratories.8 Herein, we extended our characterization of these materials by comparing them to clinical samples in both a functional characterization study and a real-world clinical pilot. Many aspects of the functional characterization study design were modeled after Clinical & Laboratory Standards Institute guidelines for functional characterization.15

An important overall finding was that we observed significant discrepancies in the performance of different assay platforms on both clinical specimens and the QCMs. For example, in the functional characterization study, we observed higher VAFs in clinical samples bearing EGFR ex19del and L858R mutations by hybrid capture NGS as compared with ddPCR. Interestingly, this trend seemed unique to our hybrid capture assay because in the clinical pilot, the performance of the 3 hybrid capture NGS assays on clinical samples harboring those same mutations was more similar to ddPCR. Future studies should address the root cause of these differences and at which step(s) of the workflow of our hybrid capture NGS assay they may have been introduced (e.g., library preparation or bioinformatics analysis).

Notably, some trends seen in the functional characterization study were also observed in clinical pilot laboratories. For example, our hybrid capture NGS assay tended to undercall EGFR ex19del mutations in the QCMs compared to ddPCR and amplicon NGS, a trend reproduced in the clinical pilot. We postulate this finding may be related to the size-dependent higher amplification efficiency that is achieved with amplicon-based tests such as ddPCR or amplicon NGS on deletion-bearing fragments compared to matched wild type from the same sample.

A strength of our study was that we tested our QCMs in real-world laboratories and observed unexpected lab-specific differences in assay performance. In a notable example, the 2 amplicon NGS assays consistently identified the same variants in the TFS QCMs at different VAFs. Our analysis of the sizing profile of sequenced mutant and wild-type fragments identified no obvious difference between the TFS QCMs and QCMs from the other manufacturers, which might have explained this discrepancy. Instead, we postulate that differences in primer design and amplicon size may be responsible.

One surprising finding of the clinical pilot was the heterogeneity and variance between laboratories, assays, and QCMs. Though some variance could clearly be attributed to a specific assay (e.g., Assay 9 for EGFR L858R), other trends were QCM-specific (e.g., BRAF V600E for HD), while others still were unique to an assay and a particular QCM (e.g., Assay 1 detected no variants for any TFS QCMs). These observations suggest that QCMs are not yet identical to cfDNA isolated from plasma and that labs should not rely solely on these materials for assay development.

Other groups have reported on reference materials for cfDNA.9,10,12 As part of the Blood Profiling Atlas in Cancer (BLOODPAC) project, Hernandez et al. characterized reference materials developed by SC and HD, 2 of the same vendors described herein.16 The authors concluded that their materials performed similarly when tested at 9 independent BLOODPAC laboratories by ddPCR. Similarly, He et al. evaluated a reference set from SC that contained 40 genetic alterations and showed consistent quantification across labs, with reliable detection down to 0.125% VAF.10 Importantly, He et al. mainly focused on analytical validation of the assays, while we focused on fit for purpose of the QCMs.

Our study had several limitations. Challenges in sourcing CNVs and translocations in sufficient quantities limited our functional characterization study to SNVs and indels, the 2 most prevalent variant types observed in cancer genomes. By focusing our evaluation on EGFR L858R and EGFR ex19del, we did not comprehensively evaluate performance in homopolymer regions and other challenging genomic contexts. Additionally, due to resource limitations, we only tested 10 replicates in the functional characterization LOD study. This led to wider CIs in the hit rates and at least 2 instances of an aberrant result that was likely due to chance. We strongly recommend that future LOD studies be performed with a substantially larger number of replicates (e.g., 20).9

Going forward, we believe the major stakeholders in the clinical application of liquid biopsies—manufacturers, laboratorians, clinicians, regulators, and payors—should develop a consensus approach for the development, validation, and implementation of QCMs for cfDNA. Such an approach could include the materials characterized herein. Indeed, a standard set(s) of well-vetted QCMs adopted for use by all laboratories would greatly facilitate cross-laboratory comparisons and regulatory review.

Supplementary Material

PV Data Supplement_1
PV Data Supplement_2
PV Data Supplement_3
PV Data Supplement_4
PV Data Supplement_5
PV Data Supplement_6
PV Data Supplement_7

ACKNOWLEDGMENTS

The authors would like to thank Diana Kimono for project management and operations support; Dr. James Doroshow at NCI for his encouragement and support; Revvity Mimix (formerly Horizon Discovery), LGC Clinical Diagnostics (SeraCare), and Thermo Fisher Scientific for providing QCMs for this project; Blakely Swain and Nicole Abramowitz at CCS Associates for editorial support; and Andy Simmons at Clovis Oncology for providing clinical samples with EGFR L858R and ex19del mutations.

This manuscript is dedicated to the memory of Mickey Williams, Alexander Dobrovic, and Caroline Sigman. Mickey founded the Molecular Characterization Laboratory at FNLCR in 2010 and led it until his retirement in February of 2024. He was a superb translational scientist with a unique creative vision. In support of clinical trials sponsored by NCI, his lab developed several cutting-edge genomic assays that had an enduring impact on the lives of patients with cancer. Among his many scientific achievements, he stood up a network of 5 laboratories that screened the first 6000 patients for the NCI-MATCH trial, demonstrating for the first time that high concordance among NGS labs running the same harmonized workflow was indeed possible. In support of NCI’s Cancer Moonshot Biobank study, he helped bring molecular testing to underserved communities. He was also a humanist, a mentor, and a beloved friend to many at NCI, the Frederick National Laboratory, and beyond until his recent untimely passing. He will be deeply missed.

Alexander was renowned for his pioneering work in personalized medicine and for developing innovative diagnostic methodologies, including tests for detecting KRAS, BRAF, and TP53 mutations. He was the head of the Molecular Pathology Laboratory at the Peter MacCallum Cancer Centre and led the Translational Genomics and Epigenomics Laboratory at the Olivia Newton-John Cancer Research Institute.

A passionate scientist, visionary leader, and tireless advocate for advancing public health, Dr. Caroline Sigman founded CCS Associates in 1985 with a singular mission: to help others achieve the breakthroughs that could cure some of society’s most intractable health challenges, particularly cancer. She was a pioneer in the development of biomarkers and among the first to recognize their transformative potential in identifying disease earlier and intervening more effectively. Her leadership helped shape the future of biomarker science and expanded access to powerful tools for disease prevention and treatment. To those who had the privilege of working with her, Dr. Sigman was more than a leader—she was an inspiration.

SUPPORT

Financial support was provided by AstraZeneca Pharmaceuticals LP; Genentech, a member of the Roche Group; Janssen Research & Development LLC; Merck Sharp & Dohme Corp.; Pfizer Inc.; the Expect Miracles Foundation (C.P.P.); the Robert A. and Renee E. Belfer Family Foundation (C.P.P.); R01 CA240592 (C.P.P.); and the National Cancer Institute (NCI), NIH, under contract HHSN261200800001E (C.K., L.C., R.P., M.W., and T.F.).

Conflict of Interest (COI) Statement

C.P.P. is a consultant for XSphera Biosciences and has stock and other ownership interests in XSphera Biosciences, has received honoraria from Thermo Fisher Scientific and Agilent Technologies Inc., and has sponsored research agreements with Daiichi Sankyo Company Ltd., Transcenta Holding, Bicara Therapeutics, AstraZeneca Inc., Pfizer Inc., Takeda Oncology, Bristol Meyers Squibb Company, and Merck & Co. Inc. (all institutionally). H.B. and X.W. report full-time employment with Geneseeq, as well as other support from Geneseeq outside the submitted work. G.A.P. and A.W. report full-time employment with Biodesix, as well as other support from Biodesix outside the submitted work. A.S.C. and A.M.P. report full-time employment with Bio-Rad, as well as other support from BioRad outside the submitted work. G. R. J. reports full-time employment with NeoGenomics, as well as other support from NeoGenomics outside the submitted work. J.C. reports full-time employment with Sysmex, as well as other support from Sysmex outside the submitted work. M.M. and M.A.Q. report full-time employment with Canexia Health during the study. No other COIs were reported.

Footnotes

Previous Presentation

AACR meeting in San Diego, 2024, abstract and poster were presented

DISCLAIMER

Certain commercial equipment, instruments, and materials are identified to specify the experimental procedure. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment are necessarily the best available for the purpose.

References

  • 1.Tie J, Wang Y, Tomasetti C, et al. : Circulating tumor DNA analysis detects minimal residual disease and predicts recurrence in patients with stage II colon cancer. Sci Transl Med 8:346ra92, 2016 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Vega DM, Nishimura KK, Zariffa N, et al. : Changes in circulating tumor DNA reflect clinical benefit across multiple studies of patients with non-small-cell lung cancer treated with immune checkpoint inhibitors. JCO Precis Oncol 6:e2100372, 2022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Thress KS, Paweletz CP, Felip E, et al. : Acquired EGFR C797S mutation mediates resistance to AZD9291 in non-small cell lung cancer harboring EGFR T790M. Nat Med 21:560–2, 2015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Liu MC, Oxnard GR, Klein EA, et al. : Sensitive and specific multi-cancer detection and localization using methylation signatures in cell-free DNA. Ann Oncol 31:745–759, 2020 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Stetson D, Ahmed A, Xu X, et al. : Orthogonal comparison of four plasma NGS tests with tumor suggests technical factors are a major source of assay discordance. JCO Precis Oncol 3:1–9, 2019 [DOI] [PubMed] [Google Scholar]
  • 6.Torga G, Pienta KJ: Patient-paired sample congruence between 2 commercial liquid biopsy tests. JAMA Oncol 4:868–870, 2018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Merker JD, Oxnard GR, Compton C, et al. : Circulating tumor DNA analysis in patients with cancer: American Society of Clinical Oncology and College of American Pathologists joint review. J Clin Oncol 36:1631–1641, 2018 [DOI] [PubMed] [Google Scholar]
  • 8.Williams PM, Forbes T, S PL, et al. : Validation of ctDNA quality control materials through a precompetitive collaboration of the Foundation for the National Institutes of Health. JCO Precis Oncol 5:910–920, 2021 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Godsey JH, Silvestro A, Barrett JC, et al. : Generic protocols for the analytical validation of next-generation sequencing-based ctDNA assays: A joint consensus recommendation of the BloodPAC's Analytical Variables Working Group. Clin Chem 66:1156–1166, 2020 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.He HJ, Stein EV, Konigshofer Y, et al. : Multilaboratory assessment of a new reference material for quality assurance of cell-free tumor DNA measurements. J Mol Diagn 21:658–676, 2019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Lindeman NI, Cagle PT, Aisner DL, et al. : Updated molecular testing guideline for the selection of lung cancer patients for treatment with targeted tyrosine kinase inhibitors: Guideline from the College of American Pathologists, the International Association for the Study of Lung Cancer, and the Association for Molecular Pathology. Arch Pathol Lab Med 142:321–346, 2018 [DOI] [PubMed] [Google Scholar]
  • 12.Deveson IW, Gong B, Lai K, et al. : Evaluating the analytical validity of circulating tumor DNA sequencing assays for precision oncology. Nat Biotechnol 39:1115–1128, 2021 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Oxnard GR, Paweletz CP, Kuang Y, et al. : Noninvasive detection of response and resistance in EGFR-mutant lung cancer using quantitative next-generation genotyping of cell-free plasma DNA. Clin Cancer Res 20:1698–1705, 2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Pascual J, Attard G, Bidard FC, et al. : ESMO recommendations on the use of circulating tumour DNA assays for patients with cancer: A report from the ESMO Precision Medicine Working Group. Ann Oncol 33:750–768, 2022 [DOI] [PubMed] [Google Scholar]
  • 15.Clinical & Laboratory Standards Institute (CLSI): Evaluation of Commutability of Processed Samples; Approved Guideline - Third Edition. CLSI document EP14-A3 (Final Draft) Vol 34, No. 11, 2014 [Google Scholar]
  • 16.Hernandez KM, Bramlett KS, Agius P, et al. : Contrived materials and a data set for the evaluation of liquid biopsy tests: A Blood Profiling Atlas in Cancer (BLOODPAC) community study. J Mol Diagn 25:143–155, 2023 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

PV Data Supplement_1
PV Data Supplement_2
PV Data Supplement_3
PV Data Supplement_4
PV Data Supplement_5
PV Data Supplement_6
PV Data Supplement_7

RESOURCES