Abstract
Background and Aims
The multitarget stool DNA (mt-sDNA) assay is a noninvasive average-risk colorectal cancer (CRC) screening test. A new biomarker panel was developed for a next-generation test to improve specificity while maintaining/increasing sensitivity. We aimed first to establish an algorithm and cutoff for the next-generation mt-sDNA test and then to validate it using archived samples from the pivotal DeeP-C study (NCT01397747) of the first-generation test.
Methods
Algorithm development and cross-validation included 3011 samples from 2 specimen collection studies (NCT03821948 and NCT03789162). The algorithm and cutoff were locked before validation. Validation test set samples included 57 CRC, 583 advanced precancerous lesions (APLs), and 7022 samples negative for CRC or APLs from the DeeP-C study, which prospectively enrolled average-risk, asymptomatic adults aged 50–84 years before screening colonoscopy. Next-generation biomarkers included methylated DNA markers ceramide synthase 4 gene, leucine-rich repeat-containing protein 4 gene, serine/threonine-protein phosphatase 2A 56 kDa regulatory subunit gamma isoform gene, and zinc finger DHHC-type containing 1 gene (reference marker), and fecal hemoglobin. Primary validation end points were CRC sensitivity and specificity for the absence of advanced neoplasia. Secondary end points included APL sensitivity and specificity for non-neoplastic findings or negative colonoscopy.
Results
Cross-validation and best-fit results from algorithm development closely matched, confirming algorithm reliability and reproducibility. For the test set, next-generation mt-sDNA test sensitivity was 93.0% (95% confidence interval [CI], 83.0%–98.1%) for CRC and 48.4% (95% CI, 44.2%–52.5%) for APLs. Specificity was 88.5% (95% CI, 87.7%–89.2%) for the absence of advanced neoplasia and 90.4% (95% CI, 89.5%–91.2%) for the combination of non-neoplastic findings or negative colonoscopy.
Conclusion
Based on archived samples, the next-generation mt-sDNA test demonstrated promising CRC screening performance characteristics that will be further assessed in a prospective clinical validation study (BLUE-C; NCT04144738).
Keywords: Colorectal Cancer, Colorectal Cancer Screening, Algorithm Validation, Stool Test
Graphical abstract
Introduction
Colorectal cancer (CRC) is the fourth most common cancer in the United States, accounting for 7.8% of all incident cancers, with an estimated 153,000 new cases in 2023,1 and is the second most common cause of cancer-related deaths. Screening for CRC reduces its incidence and mortality.2, 3, 4 Although screening is recommended for average-risk adults aged 45–75 years,5 participation rates have not achieved the 80% target endorsed by the National Colorectal Cancer Roundtable,6,7 suggesting that there are barriers to guideline-recommended CRC screening, some of which could be reduced by noninvasive screening options.8
The first-generation multitarget stool DNA (mt-sDNA) test is approved by the US Food and Drug Administration9 for noninvasive, average-risk screening and is included in clinical guidelines from the United States Preventive Services Task Force, American Cancer Society, and other national organizations.5,10,11 The mt-sDNA test analyzes stool samples for KRAS mutations, methylated DNA markers (MDMs), β-actin, and fecal hemoglobin, and was evaluated in the pivotal DeeP-C study of 9989 asymptomatic average-risk participants (ClinicalTrials.gov identifier: NCT0139774712), demonstrating CRC sensitivity of 92.3% and 42.4% sensitivity for advanced precancerous lesions (APLs).13 In the same study, the mt-sDNA test demonstrated specificity of 86.6% for the absence of advanced neoplasia and 89.8% for negative colonoscopy.13
As part of ongoing innovation efforts, biomarker discovery research was conducted to develop an optimized biomarker panel of MDMs for a next-generation mt-sDNA test. MDM discovery proceeded via next-generation sequencing of DNA extracted from primary CRC, APL, and non-neoplastic tissues.14 Biomarker down-selection and early-phase verification studies were then performed with archival stool samples from multiple case/control and prospective studies. In this preliminary assessment, the next-generation biomarker panel performed well and showed high sensitivity for CRC and APLs, along with high specificity for the absence of advanced neoplasia.15, 16, 17 The primary aim for a next-generation test was to improve specificity while simultaneously maintaining or increasing sensitivity for CRC and APLs (larger and/or more histologically advanced adenomas or sessile serrated polyps with greater risk for progressing to CRC). The optimized biomarker panel includes the MDMs ceramide synthase 4 gene (LASS4), leucine-rich repeat-containing protein 4 gene (LRRC4), serine/threonine-protein phosphatase 2A 56 kDa regulatory subunit gamma isoform gene (PPP2R5C), and the reference marker zinc finger DHHC-type containing 1 gene (ZDHHC1), as well as fecal hemoglobin, and excludes KRAS gene mutations included in the first-generation mt-sDNA assay.
In the present study, an optimized algorithm for the next-generation mt-sDNA was first developed, trained, and cross-validated using archived samples from 2 sample collection studies. The algorithm and next-generation test were then evaluated using an independent set of archived samples available from the pivotal, prospective DeeP-C study.12
Methods
Algorithm Development
Study design and participants for algorithm development
Stool samples from 3011 participants (100 CRC, 242 APLs, 813 nonadvanced adenomas, and 1856 negative samples) were selected from 2 sample collection studies: Act Bold (ClinicalTrials.gov identifier: NCT0382194818) and Act Fast (ClinicalTrials.gov identifier: NCT0378916219). Stool samples were collected prior to initiation of bowel preparation for colonoscopy. As indicated in the study eligibility criteria, the Act Bold study prospectively enrolled participants aged ≥ 40 years prior to colonoscopy presenting for CRC screening or surveillance colonoscopy and who had no symptoms requiring immediate or near-term referral for diagnostic or therapeutic colonoscopy.18 The Act Fast study enrolled asymptomatic or symptomatic participants aged ≥40 years who underwent colonoscopy and were found to have either CRC of any stage confirmed with tissue biopsy or a colorectal lesion ≥1 cm suspicious for adenoma, sessile serrated polyps, or CRC on a pre-enrollment colonoscopy that remained ≥1 cm postcolonoscopy and required additional surgical excision or complex polypectomy.19 Inclusion and exclusion criteria for the Act Bold and Act Fast samples used for algorithm development are provided in the Supplementary Appendix. The final analysis population included 2994 participants with valid next-generation mt-sDNA test results, and a sample distribution that closely matched colonoscopy and histopathology results from the mt-sDNA pivotal DeeP-C study (ClinicalTrials.gov identifier: NCT01397747),13 including CRC stage; CRC lesion size; and frequency of nonadvanced adenomas, non-neoplastic findings, and samples with negative or no colonoscopy findings (Table A1). Case samples were obtained from participants with CRC or APLs based on colonoscopy and histopathologic results. Control samples were obtained from participants without CRC or APLs. For both Act Bold and Act Fast, the study protocols, all amendments, informed consent forms, and other relevant study documents were reviewed and prospectively approved by the Institutional Review Board/Ethics Committee. Trained laboratory personnel performing the next-generation mt-sDNA testing were blinded to the results of colonoscopy and histopathologic diagnosis.
Analyses for algorithm development and cross-validation
For algorithm development, ZDHHC1 was used as the reference marker to normalize the MDM signal based on input DNA. For all training set samples, the number of strands for each marker was calculated and log10 transformed, and the log10 strand numbers for LASS4, LRRC4, and PPP2R5C were plotted against log10 strand numbers for ZDHHC1 to calculate normalization slopes. The reference gene ZDHHC1 was also used to establish a cut-off value for sample adequacy. To standardize MDMs, negative samples with non-neoplastic findings or negative colonoscopy findings (category 6) were used. MDMs were normalized to the reference marker. Standardized values were then calculated as the distance from the mean in units of standard deviation.
Next, correlations were determined between each of the panel markers, as well as between the panel markers and the reference standard of colonoscopy/histopathology results. Because each of the MDMs was more strongly correlated with one another than with colonoscopy/histopathology results, they were combined to create a single predictive input variable based on a median weighted average of the MDMs. To capture all stool marker data, the quantitative results from the MDMs and fecal hemoglobin were incorporated into modeling analyses to produce the final algorithm.
Logistic regression, general additive, neural network, and random forest models were fitted to the training data. Ultimately, the logistic regression model provided the highest accuracy (ie, highest specificity or sensitivity) with the least complexity and was used to identify the optimal threshold, with values above the determined threshold denoting a positive result and values at or below the threshold denoting a negative result.
Algorithm performance was then cross-validated in silico to minimize overfitting the model. For cross-validation, 25% of the samples were held back as test sets and the model was trained on the remaining 75% of the samples; 5000 splits of the dataset were performed.
Algorithm Validation on DeeP-C Test Set Samples
Study design and participants
A total of 7662 archived samples were available for study inclusion. Samples were prospectively collected from 2011 to 2013 during enrollment for the pivotal DeeP-C study and were archived until this analysis.13 Study participants were asymptomatic adults aged 50–84 years at average risk for CRC who were scheduled for screening colonoscopy and provided stool samples prior to colonoscopy preparation. The specific inclusion and exclusion criteria for the DeeP-C study have been described previously.13 None of these DeeP-C samples were used in previous algorithm development for the next-generation mt-sDNA test. Prior to validation on archival DeeP-C samples, the next-generation algorithm and cut-off value were locked based on results from the algorithm development study described above.
Participant samples were categorized based on the most advanced histopathology found on colonoscopy as follows: CRC (category 1), APLs (category 2), and nonadvanced adenomas (categories 3–5); samples with non-neoplastic findings (category 6.1) or negative colonoscopy with no tissue taken (category 6.2) comprised category 6 (Table A1).
Sample processing and biomarker detection
The archived buffered stool sample homogenates and hemoglobin samples were stored at −80 °C until use. Stool homogenates were thawed and processed to remove solids and assay inhibitors. Samples then underwent target-specific DNA capture, bisulfite treatment, and MDM quantification. Hemoglobin was quantified via an enzyme-linked immunosorbent assay. Details are available in the Supplementary Material. Laboratory personnel performing the next-generation mt-sDNA testing were blinded to the pathology and demographic information for patient samples as well as to fecal immunochemical testing and first-generation mt-sDNA results from the DeeP-C study. Operators analyzing the study data were also blinded to assay results until study completion.
Study end points for validation with archival DeeP-C samples
The primary study end points were sensitivity for CRC (category 1) and specificity for the absence of advanced neoplasia (categories 3–6, which include nonadvanced adenomas, non-neoplastic findings, and negative colonoscopy). Secondary study end points included sensitivity for APLs (category 2) and specificity among participants with non-neoplastic findings or negative colonoscopy (category 6). For comparative purposes, original results from the first-generation mt-sDNA test were analyzed for the subset of DeeP-C samples available in the present study. Receiver operating characteristic curves of sensitivity for CRC or any advanced neoplasia (CRC or APLs) vs 1-specificity (absence of advanced neoplasia) were plotted. Exploratory analyses also assessed impacts of age, CRC stage, APL subtype, and lesion size on the sensitivity and specificity of the next-generation mt-sDNA test.
Statistical analyses for validation with archival DeeP-C samples
All available samples from the DeeP-C study set were tested for algorithm validation and assessment of the next-generation mt-sDNA test characteristics. For the primary end points, CRC sensitivity was tested against the null hypothesis of 65% and specificity for the absence of advanced neoplasia was tested against the null hypothesis of 85.9%. Null hypotheses and power calculations for primary and secondary end point hypotheses are detailed in Table A2. All performance estimates are reported with 2-sided 95% confidence intervals (CIs) using the exact binomial method. If the lower bound of the 2-sided 95% CI was more than the null hypothesis, then the null was rejected, and the end point was considered to have been met. In exploratory analyses, next-generation mt-sDNA test performance was compared with the original first-generation mt-sDNA test results for the paired sample set using McNemar’s tests.
Results
Algorithm Development and Cross-Validation
The overall proportion of invalid specimens for analysis by the next-generation mt-sDNA test was low (17 of 3011 samples; 0.6%). Among the 2994 participants included in the training set with valid next-generation mt-sDNA test results, 2655 had nonadvanced adenomas, non-neoplastic findings, or no findings on colonoscopy, and 339 were positive for CRC or APLs (Table 1 and Supplementary Results provide a summary of study participant characteristics). Results of in silico cross-validation closely matched the results obtained by logistic regression. Point estimates for CRC sensitivity were 91.9% for both logistic regression results and cross-validation (Table 2). For APL sensitivity, point estimates were 41.3% and 40.7% for logistic regression results and cross-validation, respectively (Table 2). Test specificities for the absence of advanced neoplasia in the logistic regression model and cross-validation were 88.7% and 88.5%, respectively; and test specificities for non-neoplastic findings or negative colonoscopy were 91.2% and 90.9%, respectively (Table 2). Specificity was 89.0% for hyperplastic polyps <10 mm (category 6.1) and 92.1% for negative colonoscopy with no tissue biopsy taken (category 6.2). The high degree of agreement between logistic regression results and cross-validation suggests negligible overfitting with this simple classification model. Additional data on the performance of the next-generation test on the algorithm training set are included in the Supplementary Results.
Table 1.
Demographic Characteristics for Participants in the Algorithm Training and DeeP-C Test Sets
Characteristic | Training set (valid samples) N = 2994 |
DeeP-C test set N = 7662 |
|||
---|---|---|---|---|---|
Controla,b (n = 2655) | Caseb,c (n = 339) | All samples | |||
Age, mean (SD), y | 62.1 (9.2) | 64.0 (9.4) | 62.3 (9.3) | 63.8 (8.2) | |
Sex, n (%) | |||||
Female | 1598 (60.2) | 164 (48.4) | 1762 (58.9) | 4260 (55.6) | |
Male | 1057 (39.8) | 175 (51.6) | 1232 (41.2) | 3402 (44.4) | |
Race, n (%) | |||||
White | 2285 (86.1) | 290 (85.5) | 2575 (86.0) | 6697 (87.4) | |
Black | 268 (10.1) | 37 (10.9) | 305 (10.2) | 702 (9.2) | |
Asian | 66 (2.5) | 11 (3.2) | 77 (2.6) | 129 (1.7) | |
Other | 35 (1.3) | 1 (0.3) | 36 (1.2) | 131 (1.7) | |
Missing | 1 (<0.1) | 0 (0.0) | 1 (<0.1) | 3 (<0.1) | |
Ethnicity, n (%) | |||||
Hispanic/Latino | 228 (8.6) | 17 (5.0) | 245 (8.2) | 676 (8.8) | |
Non-Hispanic/Latino | 2415 (91.0) | 322 (95.0) | 2737 (91.4) | 6983 (91.1) | |
Other | 12 (0.5) | 0 (0.0) | 12 (0.4) | 3 (<0.1) | |
Smoking, n (%) | |||||
Current | 206 (7.8) | 52 (15.3) | 258 (8.6) | 691 (9.0) | |
Former | 793 (29.9) | 105 (31.0) | 898 (30.0) | 2695 (35.2) | |
Never | 1656 (62.4) | 182 (53.7) | 1838 (61.4) | 4276 (55.8) | |
First-degree family history of CRC, n (%) | |||||
Yes | 381 (14.4) | 44 (13.0) | 425 (14.2) | NA | |
No | 2274 (85.6) | 294 (86.7) | 2568 (85.8) | NA | |
Missing | 0 (0.0) | 1 (0.3) | 1 (<0.1) | NA |
APL, advanced precancerous lesion; CRC, colorectal cancer; NA, data not available.
Control samples were those with nonadvanced adenomas, non-neoplastic findings, and no findings on colonoscopy (categories 3–6).
In an analysis comparing demographic characteristics between case and control groups, the following P values were obtained: age, sex, and smoking, P < .001; race, P = .547; ethnicity, P = .018; first-degree family history of CRC, P = .151.
Case samples were those positive for CRC or APLs.
Table 2.
Performance of the Next-Generation mt-sDNA Test in Algorithm Development and Cross-Validation
Endpoint | Samples, N | Best fit, % | Cross-validation, % | 95% CI |
---|---|---|---|---|
Sensitivity | ||||
CRC | 99 | 91.9 | 91.9 | 84.9–95.8 |
APL | 240 | 41.3 | 40.7 | 34.8–47.1 |
Specificity | ||||
Absence of advanced neoplasiaa | 2655 | 88.7 | 88.5 | 87.2–89.7 |
Non-neoplastic findings or negative colonoscopyb | 1845 | 91.2 | 90.9 | 89.5–92.1 |
APL, advanced precancerous lesion; CI, confidence interval; CRC, colorectal cancer; mt-sDNA, multitarget stool DNA.
Includes all nonadvanced adenomas, non-neoplastic findings, and negative colonoscopy (categories 3–6).
Category 6 (6.1 or 6.2).
Participants in the DeeP-C Test Set
The algorithm test set comprised 7662 participants with valid next-generation mt-sDNA test results, including 7022 without advanced neoplasia and 640 who were positive for CRC (N = 57) or APLs (N = 583). Among these samples, the mean (standard deviation) participant age was 63.8 (8.2) years and 55.6% were female (Table 1). Most participants were White (87.4%) and 9.2% were Black. The most common ethnicity was non-Hispanic/Latino (91.1%). Participant demographics and characteristics were comparable to those included in the primary analysis of the pivotal study.13
Algorithm Validation in the DeeP-C Test Set
Sensitivity of the next-generation mt-sDNA test for CRC was 93.0% (95% CI, 83.0%–98.1%), which was identical to that of the first-generation mt-sDNA test (P > .99; Table 3). Sensitivity for APLs was significantly higher for the next-generation mt-sDNA test (48.4% [95% CI, 44.2%–52.5%]) compared with the first-generation test (41.2% [95% CI, 37.1%–45.3%]; P = .0003). Additional sensitivity comparisons for APL subsets are provided in Table 3. Approximately one-quarter (25.9% [151/583]) of APLs were reported as 10 mm in size.
Table 3.
Performance of the Next-Generation mt-sDNA Test Compared With Historical Performance of the First-Generation mt-sDNA Test in the DeeP-C Test Set
Most advanced colonoscopy finding | Test set (N = 7662) |
P valueb | ||
---|---|---|---|---|
Valid DeeP-C results, n | Next-gen mt-sDNA test sensitivity, % (95% CI) | First-gen mt-sDNA test sensitivity,a % (95% CI) | ||
CRC | 57 | 93.0 (83.0–98.1) | 93.0 (83.0–98.1) | >.99 |
APLs | 583 | 48.4 (44.2–52.5) | 41.2 (37.1–45.3) | .0003 |
High-grade dysplasia | 30 | 76.7 (57.7–90.1) | 63.3 (43.9–80.1) | .16 |
Villous histology | 194 | 56.7 (49.4–63.8) | 44.3 (37.2–51.6) | .0002 |
Adenoma ≥10 mm | 284 | 41.9 (36.1–47.9) | 35.6 (30.0–41.4) | .02 |
Serrated or hyperplastic lesion ≥10 mm | 75 | 40.0 (28.9–52.0) | 45.3 (33.8–57.3) | .43 |
Valid DeeP-C results, n | Next-gen mt-sDNA test specificity, % (95% CI) | First-gen mt-sDNA test specificity, % (95% CI) | ||
---|---|---|---|---|
Absence of advanced neoplasiac | 7022 | 88.5 (87.7–89.2) | 86.9 (86.1–87.7) | <.0001 |
Non-neoplastic findings or negative colonoscopyd | 4859 | 90.4 (89.5–91.2) | 88.6 (87.6–89.4) | <.0001 |
Negative colonoscopye | 3454 | 91.6 (90.7–92.5) | 90.0 (89.0–91.0) | .003 |
APL, advanced precancerous lesion; CI, confidence interval; CRC, colorectal cancer; first-gen, first-generation; mt-sDNA, multitarget stool DNA; next-gen, next-generation.
Historical results on the same subset of samples available for testing with the next-gen mt-sDNA test.
P values calculated using McNemar’s test.
Includes all nonadvanced adenomas, non-neoplastic findings, and negative colonoscopy (categories 3–6).
Category 6 (6.1 or 6.2).
Includes no findings on colonoscopy (category 6.2).
For the primary specificity outcome of the absence of advanced neoplasia, the next-generation mt-sDNA test significantly outperformed the first-generation test (88.5% [95% CI, 87.7%–89.2%] vs 86.9% [95% CI, 86.1%–87.7%], respectively; P < .0001; Table 3). Similarly, secondary specificity for non-neoplastic findings or negative colonoscopy was significantly higher for the next-generation mt-sDNA test compared with the first-generation test (90.4% [95% CI, 89.5%–91.2%] vs 88.6% [95% CI, 87.6%–89.4%]; P < .0001). Specificity for negative colonoscopy (ie, no tissue taken) was also significantly higher for the next-generation mt-sDNA test (91.6% [95% CI, 90.7%–92.5%]) vs the first-generation test (90.0% [95% CI, 89.0%–91.0%]; P = .003).
The next-generation mt-sDNA test maintained high sensitivity for CRC stage I (92.0% [95% CI, 74.0%–99.0%]), stage II (100.0% [95% CI, 83.9%–100.0%]), stage III (83.3% [95% CI, 35.9%–99.6%]), and stage IV (75.0% [95% CI, 19.4%–99.4%]) (Table 4). The first-generation and next-generation mt-sDNA tests had equivalent performance across CRC stages (Table 4), with concordant results except 2 stage III CRC cases. Among APL subgroups, next-generation mt-sDNA test sensitivity was highest for adenomas with high-grade dysplasia (76.7% [95% CI, 57.7%–90.1%]), followed by adenomas with villous histology (56.7% [95% CI, 49.4%–63.8%]), adenomas ≥10 mm in size (41.9% [95% CI, 36.1%–47.9%]), and serrated or hyperplastic lesions ≥10 mm in size (40.0% [95% CI, 28.9%–52.0%]; Figure 1A). For APLs <10 mm in size, sensitivity was 37.3% (95% CI, 24.1%–51.9%) and increased with increasing lesion size (Figure 1B). Sensitivity was higher for APLs of 10–19 mm (44.7% [95% CI, 40.0%–49.4%]), 20–29 mm in size (68.3% [95% CI, 55.0%–79.7%]), and ≥30 mm in size (80.6% [95% CI, 62.5%–92.5%]). Test specificity was highest for participants aged 50–64 years, with reduced specificity in older age groups (P value for trend <.0001; Table 5).
Table 4.
CRC Sensitivity by Index Lesion Stage for the Next-Generation mt-sDNA Test Compared With the First-Generation Test in the DeeP-C Test Set
CRC lesion stage | Test set (N = 7662; CRC, n = 56a) |
||
---|---|---|---|
Valid DeeP-C results, n | Next-gen mt-sDNA test sensitivity, % (95% CI) | First-gen mt-sDNA test sensitivity,b % (95% CI) | |
Stage I | 25 | 92.0 (74.0–99.0) | 92.0 (74.0–99.0) |
Stage II | 21 | 100.0 (83.9–100.0) | 100.0 (83.9–100.0) |
Stage III | 6 | 83.3 (35.9–99.6) | 83.3 (35.9–99.6) |
Stage IV | 4 | 75.0 (19.4–99.4) | 75.0 (19.4–99.4) |
CI, confidence interval; CRC, colorectal cancer; first-gen, first-generation; mt-sDNA, multitarget stool DNA; next-gen, next-generation.
One CRC sample could not be staged and was not included in this analysis.
Historical results on the same subset of samples available for testing with the next-gen mt-sDNA test.
Figure 1.
Sensitivity of the next-generation mt-sDNA test for advanced precancerous lesions by (A) type and (B) size in the DeeP-C test set. APL, advanced precancerous lesion; mt-sDNA, multitarget stool DNA.
Table 5.
Next-Generation mt-sDNA Test Specificity by Age Group in the DeeP-C Test Set
Age group, y | Specificity for the absence of advanced neoplasiaa |
Specificity for non-neoplastic findings or negative colonoscopyb |
||
---|---|---|---|---|
N | Next-generation mt-sDNA test specificity, % (95% CI) | N | Next-generation mt-sDNA test specificity, % (95% CI) | |
50–54 | 1548 | 94.9 (93.7–95.9) | 1183 | 95.6 (94.3–96.7) |
55–59 | 584 | 91.8 (89.3–93.9) | 421 | 94.5 (91.9–96.5) |
60–64 | 563 | 90.2 (87.5–92.6) | 391 | 91.6 (88.4–94.1) |
65–69 | 2647 | 87.1 (85.8–88.4) | 1750 | 89.4 (87.9–90.8) |
70–74 | 1206 | 84.5 (82.3–86.5) | 791 | 85.1 (82.4–87.5) |
75–79 | 398 | 79.1 (74.8–83.0) | 267 | 83.1 (78.1–87.4) |
≥ 80 | 76 | 78.9 (68.1–87.5) | 56 | 80.4 (67.6–89.8) |
CI, confidence interval; mt-sDNA, multitarget stool DNA.
Includes all nonadvanced adenomas, non-neoplastic findings, and negative colonoscopy (categories 3–6).
Category 6 (6.1 or 6.2).
For the next-generation mt-sDNA test, the area under the receiver operating characteristic curve for CRC was 0.97 (95% CI, 0.94–0.99) (Figure 2A) and was 0.78 (95% CI, 0.76–0.80) (Figure 2B) for advanced neoplasia (CRC or APLs); for the first-generation mt-sDNA test, areas under the receiver operating characteristic curves were 0.94 (95% CI, 0.91–0.97) and 0.74 (95% CI, 0.71–0.76), respectively. The differences in area under the receiver operating characteristic curves for the next-generation mt-sDNA test vs the first-generation mt-sDNA test were 0.03 (95% CI, 0.004–0.05; P = .02) for CRC and 0.04 (95% CI, 0.02–0.06; P < .0001) for any advanced neoplasia.
Figure 2.
ROC curves and AUC for (A) CRCa (sensitivity category 1) and (B) any advanced neoplasiab (CRC or APLs, sensitivity categories 1–2) vs 1-specificity for the absence of advanced neoplasia (categories 3–6) for the first-generation and next-generation mt-sDNA tests in the DeeP-C test set. APL, advanced precancerous lesion; AUC, area under the curve; CRC, colorectal cancer; mt-sDNA, multitarget stool DNA; ROC, receiver operating characteristic. aDifference in area under the ROC curve for the next-generation mt-sDNA test vs the first-generation mt-sDNA test: 0.03 (95% CI, 0.004–0.05); P = .02. bDifference in area under the ROC curve for the next-generation mt-sDNA test vs the first-generation mt-sDNA test: 0.04 (95% CI, 0.02–0.06); P < .0001.
Discussion
Since its approval in 2014,9 the first-generation mt-sDNA test has become a widely used noninvasive option for average-risk CRC screening.20 Compared with fecal immunochemical testing, the first-generation mt-sDNA test has higher sensitivity for CRC and APLs but also has lower specificity in single-application screening.13 A mt-sDNA test with higher specificity would improve testing efficiency by decreasing false-positive results.7 Recently, novel CRC-associated MDMs were identified14 and evaluated for their ability to discriminate for CRC.15 In a recently published identification and verification study, MDM markers LASS4, LRRC4, and PPP2R5C and the reference marker ZDHHC1 exhibited high sensitivity for CRC and APLs and specificity for colorectal neoplasia, and were chosen for the next-generation mt-sDNA test, along with fecal hemoglobin.17 The oncogene KRAS (DNA mutation marker) was removed, allowing all tissue and DNA input to be allocated to MDM quantification, thus aiding in improved analytical sensitivity.17 In the present study, we developed and cross-validated an algorithm and clinical cut-off value for the next-generation mt-sDNA CRC screening test using the optimized biomarker panel. We then evaluated the next-generation mt-sDNA test performance using this locked, optimized algorithm and cut-off in an analysis of archived samples that were prospectively collected for the pivotal DeeP-C study. In this early performance evaluation, the next-generation mt-sDNA test demonstrated promising performance characteristics for CRC screening.
The results of this study support the need for multimarker inclusion in the development of stool-based CRC screening tests. Notably, both current and next-generation mt-sDNA screening tests, which incorporate molecular and hemoglobin markers, have shown greater sensitivity for detecting CRCs and APLs in comparison to fecal hemoglobin alone.13
In comparing the original results of the first-generation mt-sDNA test with the performance of the next-generation mt-sDNA test on the same samples, we found that sensitivity for CRC was the same between the 2 tests, while the next-generation mt-sDNA test had statistically significantly higher sensitivity for APLs. While the CRC sensitivities were equal, each version of the mt-sDNA test missed one CRC that the other detected. Furthermore, the next-generation mt-sDNA test had clinically important and statistically significantly higher specificity compared with the first-generation test for the primary and secondary end points. The results support an improvement in specificity of the next-generation mt-sDNA test from the first-generation test, with CRC sensitivity maintained. Even small improvements in test performance can have important, favorable clinical and economic consequences for population-level CRC screening. At the population level, higher specificity will result in fewer false positive tests, reducing referrals for follow-up colonoscopy and lowering patient and provider concerns regarding the need for additional diagnostic evaluation or more intensive monitoring. To this end, modeling studies are underway to demonstrate the potential impact of improved test performance on population-level screening outcomes.
In the DeeP-C test set, sensitivity for APLs with high-grade dysplasia was higher for the next-generation mt-sDNA test (76.7%) than for first-generation test (63.3%). It is important to note that this estimate for the first-generation test differs from the sensitivity for high-grade APLs reported in the pivotal DeeP-C study (69.2%),13 because there were 9 APL samples with high-grade dysplasia from the DeeP-C study that were not available for inclusion in the current analysis, 8 of which were positive in the original DeeP-C study.
The next-generation mt-sDNA test had the greatest specificity (>90%) among participants aged 50–64 years, with reduced specificity among older participants, a finding consistent with the first-generation mt-sDNA test. The age-related difference in specificity may be due to the types and sizes of lesions present in participants of different ages or to changes in biomarker levels from age-related background DNA methylation.
Limitations of this study require comment. Samples positive for CRC that were used in algorithm training were collected postcolonoscopy, and a portion of these samples (41.4% [41 of 99 valid CRC samples]) were collected from symptomatic patients. Therefore, these cases may not represent the spectrum of CRC from an average-risk CRC screening population. The DeeP-C samples were collected more than 10 years prior to this analysis, and only a subset of, rather than all, archived samples were available for evaluation. Based on an analysis of a small subset of specimens, we verified that sample integrity was maintained prior to testing the algorithm test set samples. Furthermore, there is no reason to believe that the available specimens differ from those specimens that were not available for this study. Additionally, historical results of the first-generation mt-sDNA test on the same sample set (rather than repeat testing with the commercially available mt-sDNA test) were used for comparison with the next-generation test. There is also a statistical limitation to using the same sample set to evaluate both the first-generation and next-generation mt-sDNA tests: overall type 1 error is no longer controlled across all hypothesis testing. Finally, racial/ethnic diversity was relatively limited for both the training set and test set samples. Further validation of the algorithm and locked clinical cut-off value for the next-generation mt-sDNA test will be obtained from a broader, more diverse cohort of participants enrolled in the prospective BLUE-C pivotal study (ClinicalTrials.gov identifier: NCT04144738).21
In summary, in this study, we first used samples from specimen collection studies to train the algorithm and lock the clinical cut-off value of the next-generation mt-sDNA test. This necessary step of the development process ensures that the next-generation mt-sDNA test will be evaluated with the current algorithm and cut-off value. In a preliminary and independent validation analysis on archived, prospectively collected samples, we then showed that the next-generation mt-sDNA test has improved specificity while maintaining or increasing sensitivity for detecting CRC and APLs. These improved test characteristics may help increase the efficiency of CRC screening and improve clinical outcomes among average-risk individuals.
Acknowledgments:
The authors thank Madeline Scheel, Evan Ragland, Brittany Bloom, Heather Cohen, Brenda Neis, Kong Xiong, Alycia Riehl, Katie Schmidt, Maddie Hettinger, and Katie Draheim for their contributions toward marker evaluation and algorithm methodology. Statistical programming support was provided by Varun Sama, MS, who also verified the results. Clinical study support (Act Fast, Act Bold) was provided by Julia Zella, PhD and Amber Cheng, PhD. Medical writing support was provided by William J. Kelley, PhD, of Lumanity Communications Inc, and was funded by Exact Sciences. Editorial review and support were provided by Feyza Sancar, PhD, and Rukaiyya Khan, PhD, of Exact Sciences.
Authors' Contributions:
Thomas F. Imperiale, MD: Conceptualization, methodology, investigation, writing–original draft, writing–review and editing, study supervision. Zubin D. Gagrat, BS: Conceptualization, investigation, methodology, project administration, resources, writing–review and editing, study supervision. Martin Krockenberger, PhD: Formal analysis, software. Kyle Porter, MAS: Conceptualization, formal analysis, data curation, writing–review and editing, visualization. Emily Ziegler, BS: Investigation, resources, data curation, writing–review and editing, project administration. Christine M. Leduc, BS: Investigation, resources, data curation, writing–review and editing, project administration. Michael B. Matter, MS: Investigation, resources, data curation, writing–review and editing, project administration. Marilyn C. Olson, PhD: Conceptualization, methodology, resources, writing–review and editing, study supervision. Paul J. Limburg, MD: Conceptualization, methodology, investigation, writing–review and editing, project administration, funding acquisition, study supervision.
Footnotes
Conflicts of Interest: The authors disclose the following: Thomas F. Imperiale has received research grants from Exact Sciences. Zubin D. Gagrat, Martin Krockenberger, Kyle Porter, Emily Ziegler, Christine M. Leduc, Michael B. Matter, Marilyn C. Olson, and Paul J. Limburg are employees of Exact Sciences and own stock or stock options. Zubin D. Gagrat and Michael B. Matter have intellectual property interests in patents owned by Exact Sciences.
Funding: This study was funded by Exact Sciences. Exact Sciences also designed the study, performed data collection and analysis, and was involved in data interpretation.
Ethical Statement: Samples for this analysis were from the Act Bold (NCT03821948) and Act Fast (NCT03789162) studies. For each, the study protocols, all amendments, informed consent forms, and other relevant study documents were reviewed and prospectively approved by the Institutional Review Board/Ethics Committee. Archived samples were also used from the DeeP-C study (NCT01397747); at the time of DeeP-C sample collection, the study was approved by the institutional review board at each site, and all participants provided written informed consent.
Data Transparency Statement: Individual participant data that underlie the results reported in this manuscript may be shared after deidentification from 2 years and ending 4 years after publication. The study protocol, statistical analysis plan, and informed consent form may also be shared. Proposals for access to data should be directed to clinicaltrials@exactsciences.com; researchers requesting data should provide a methodologically sound proposal with their request and are required to sign a data access agreement as well as to obtain necessary institutional review board approval as applicable to conduct their research.
Reporting Guidelines: Reporting Guidelines were not applicable for this article type.
Material associated with this article can be found, in the online version, at https://doi.org/10.1016/j.gastha.2024.05.002.
Supplementary Materials
References
- 1.Siegel R.L., Giaquinto A.N., Jemal A. Cancer statistics, 2024. CA Cancer J Clin. 2024;74:12–49. doi: 10.3322/caac.21820. [DOI] [PubMed] [Google Scholar]
- 2.Lin J.S., Perdue L.A., Henrikson N.B., et al. Screening for colorectal cancer: updated evidence report and systematic review for the US Preventive Services Task Force. JAMA. 2021;325:1978–1998. doi: 10.1001/jama.2021.4417. [DOI] [PubMed] [Google Scholar]
- 3.Schoen R.E., Pinsky P.F., Weissfeld J.L., et al. Colorectal-cancer incidence and mortality with screening flexible sigmoidoscopy. N Engl J Med. 2012;366:2345–2357. doi: 10.1056/NEJMoa1114635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Atkin W., Wooldrage K., Parkin D.M., et al. Long term effects of once-only flexible sigmoidoscopy screening after 17 years of follow-up: the UK Flexible Sigmoidoscopy Screening randomised controlled trial. Lancet. 2017;389:1299–1311. doi: 10.1016/S0140-6736(17)30396-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.US Preventive Services Task Force. Davidson K.W., Barry M.J., et al. Screening for colorectal cancer: US Preventive Services Task Force recommendation statement. JAMA. 2021;325:1965–1977. doi: 10.1001/jama.2021.6238. [DOI] [PubMed] [Google Scholar]
- 6.Joseph D.A., King J.B., Dowling N.F., et al. Vital signs: colorectal cancer screening test use - United States, 2018. MMWR Morb Mortal Wkly Rep. 2020;69:253–259. doi: 10.15585/mmwr.mm6910a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.American Cancer Society National Colorectal Cancer roundtable. 80% in every community. https://nccrt.org/80-in-every-community-2/
- 8.Inadomi J.M., Vijan S., Janz N.K., et al. Adherence to colorectal cancer screening: a randomized clinical trial of competing strategies. Arch Intern Med. 2012;172:575–582. doi: 10.1001/archinternmed.2012.332. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Exact Sciences FDA approves Exact Sciences' Cologuard®; first and only stool DNA noninvasive colorectal cancer screening test, 2014. https://investor.exactsciences.com/investor-relations/press-releases/press-release-details/2014/FDA-Approves-Exact-Sciences-Cologuard-First-and-Only-Stool-DNA-Noninvasive-Colorectal-Cancer-Screening-Test/default.aspx
- 10.Wolf A.M.D., Fontham E.T.H., Church T.R., et al. Colorectal cancer screening for average-risk adults: 2018 guideline update from the American Cancer Society. CA Cancer J Clin. 2018;68:250–281. doi: 10.3322/caac.21457. [DOI] [PubMed] [Google Scholar]
- 11.Rex D.K., Boland C.R., Dominitz J.A., et al. Colorectal cancer screening: recommendations for physicians and patients from the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol. 2017;112:1016–1030. doi: 10.1038/ajg.2017.174. [DOI] [PubMed] [Google Scholar]
- 12.Exact Sciences Multi-target colorectal cancer screening test for the detection of colorectal advanced adenomatous polys and cancer (DeeP-C). ClinicalTrials.gov identifier: NCT01397747. https://clinicaltrials.gov/study/NCT01397747
- 13.Imperiale T.F., Ransohoff D.F., Itzkowitz S.H., et al. Multitarget stool DNA testing for colorectal-cancer screening. N Engl J Med. 2014;370:1287–1297. doi: 10.1056/NEJMoa1311194. [DOI] [PubMed] [Google Scholar]
- 14.Taylor W.R., Kisiel J.B., Yab T.C., et al. 109 discovery of novel DNA methylation markers for the detection of colorectal neoplasia: selection by methylome-wide analysis. Gastroenterology. 2014;146 [Google Scholar]
- 15.Domanico M., Kisiel J., Gagrat Z., et al. Novel multi-target stool DNA marker panel yields highly accurate detection of colorectal cancer and premalignant neoplasia. Am J Gastroenterol. 2019;114 [Google Scholar]
- 16.Kisiel J.B., Gagrat Z.D., Krockenberger M., et al. Can second-generation multitarget stool DNA panels reliably detect colorectal cancer and advanced precancerous lesions? J Clin Oncol. 2022;40(4_suppl):63. [Google Scholar]
- 17.Gagrat Z.D., Krockenberger M., Bhattacharya A., et al. Next-generation multi-target stool DNA panel accurately detects colorectal cancer and advanced precancerous lesions. Cancer Prev Res (Phila) 2024;17(3):119–126. doi: 10.1158/1940-6207.CAPR-23-0285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Exact Sciences Blood and stool sample collection in subjects participating in colorectal cancer screening: Act Bold. ClinicalTrials.gov identifier: NCT03821948. https://clinicaltrials.gov/study/NCT03821948
- 19.Exact Sciences Blood and stool sample collection in subjects with a diagnosis of colorectal cancer or colorectal lesion: Act Fast. ClinicalTrials.gov identifier: NCT03789162. https://clinicaltrials.gov/study/NCT03789162
- 20.Fisher D.A., Princic N., Miller-Wilson L.A., et al. Utilization of a colorectal cancer screening test among individuals with average risk. JAMA Netw Open. 2021;4 doi: 10.1001/jamanetworkopen.2021.22269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Exact Sciences Clinical validation of an optimized multi-target stool DNA (mt-sDNA 2.0) test, for colorectal cancer screening “BLUE-C.” ClinicalTrials.gov identifier: NCT04144738. https://clinicaltrials.gov/study/NCT04144738
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.