Algorithm Development and Early Performance Evaluation of a Next-Generation Multitarget Stool DNA Screening Test for Colorectal Cancer

Abstract

Background and Aims

The multitarget stool DNA (mt-sDNA) assay is a noninvasive average-risk colorectal cancer (CRC) screening test. A new biomarker panel was developed for a next-generation test to improve specificity while maintaining/increasing sensitivity. We aimed first to establish an algorithm and cutoff for the next-generation mt-sDNA test and then to validate it using archived samples from the pivotal DeeP-C study (NCT01397747) of the first-generation test.

Methods

Algorithm development and cross-validation included 3011 samples from 2 specimen collection studies (NCT03821948 and NCT03789162). The algorithm and cutoff were locked before validation. Validation test set samples included 57 CRC, 583 advanced precancerous lesions (APLs), and 7022 samples negative for CRC or APLs from the DeeP-C study, which prospectively enrolled average-risk, asymptomatic adults aged 50–84 years before screening colonoscopy. Next-generation biomarkers included methylated DNA markers ceramide synthase 4 gene, leucine-rich repeat-containing protein 4 gene, serine/threonine-protein phosphatase 2A 56 kDa regulatory subunit gamma isoform gene, and zinc finger DHHC-type containing 1 gene (reference marker), and fecal hemoglobin. Primary validation end points were CRC sensitivity and specificity for the absence of advanced neoplasia. Secondary end points included APL sensitivity and specificity for non-neoplastic findings or negative colonoscopy.

Results

Cross-validation and best-fit results from algorithm development closely matched, confirming algorithm reliability and reproducibility. For the test set, next-generation mt-sDNA test sensitivity was 93.0% (95% confidence interval [CI], 83.0%–98.1%) for CRC and 48.4% (95% CI, 44.2%–52.5%) for APLs. Specificity was 88.5% (95% CI, 87.7%–89.2%) for the absence of advanced neoplasia and 90.4% (95% CI, 89.5%–91.2%) for the combination of non-neoplastic findings or negative colonoscopy.

Conclusion

Based on archived samples, the next-generation mt-sDNA test demonstrated promising CRC screening performance characteristics that will be further assessed in a prospective clinical validation study (BLUE-C; NCT04144738).

Graphical abstract

Introduction

Colorectal cancer (CRC) is the fourth most common cancer in the United States, accounting for 7.8% of all incident cancers, with an estimated 153,000 new cases in 2023,¹ and is the second most common cause of cancer-related deaths. Screening for CRC reduces its incidence and mortality.2, 3, 4 Although screening is recommended for average-risk adults aged 45–75 years,⁵ participation rates have not achieved the 80% target endorsed by the National Colorectal Cancer Roundtable,^6,7 suggesting that there are barriers to guideline-recommended CRC screening, some of which could be reduced by noninvasive screening options.⁸

The first-generation multitarget stool DNA (mt-sDNA) test is approved by the US Food and Drug Administration⁹ for noninvasive, average-risk screening and is included in clinical guidelines from the United States Preventive Services Task Force, American Cancer Society, and other national organizations.^5,10,11 The mt-sDNA test analyzes stool samples for KRAS mutations, methylated DNA markers (MDMs), β-actin, and fecal hemoglobin, and was evaluated in the pivotal DeeP-C study of 9989 asymptomatic average-risk participants (ClinicalTrials.gov identifier: NCT01397747¹²), demonstrating CRC sensitivity of 92.3% and 42.4% sensitivity for advanced precancerous lesions (APLs).¹³ In the same study, the mt-sDNA test demonstrated specificity of 86.6% for the absence of advanced neoplasia and 89.8% for negative colonoscopy.¹³

As part of ongoing innovation efforts, biomarker discovery research was conducted to develop an optimized biomarker panel of MDMs for a next-generation mt-sDNA test. MDM discovery proceeded via next-generation sequencing of DNA extracted from primary CRC, APL, and non-neoplastic tissues.¹⁴ Biomarker down-selection and early-phase verification studies were then performed with archival stool samples from multiple case/control and prospective studies. In this preliminary assessment, the next-generation biomarker panel performed well and showed high sensitivity for CRC and APLs, along with high specificity for the absence of advanced neoplasia.15, 16, 17 The primary aim for a next-generation test was to improve specificity while simultaneously maintaining or increasing sensitivity for CRC and APLs (larger and/or more histologically advanced adenomas or sessile serrated polyps with greater risk for progressing to CRC). The optimized biomarker panel includes the MDMs ceramide synthase 4 gene (LASS4), leucine-rich repeat-containing protein 4 gene (LRRC4), serine/threonine-protein phosphatase 2A 56 kDa regulatory subunit gamma isoform gene (PPP2R5C), and the reference marker zinc finger DHHC-type containing 1 gene (ZDHHC1), as well as fecal hemoglobin, and excludes KRAS gene mutations included in the first-generation mt-sDNA assay.

In the present study, an optimized algorithm for the next-generation mt-sDNA was first developed, trained, and cross-validated using archived samples from 2 sample collection studies. The algorithm and next-generation test were then evaluated using an independent set of archived samples available from the pivotal, prospective DeeP-C study.¹²

Methods

Algorithm Development

Study design and participants for algorithm development

Stool samples from 3011 participants (100 CRC, 242 APLs, 813 nonadvanced adenomas, and 1856 negative samples) were selected from 2 sample collection studies: Act Bold (ClinicalTrials.gov identifier: NCT03821948¹⁸) and Act Fast (ClinicalTrials.gov identifier: NCT03789162¹⁹). Stool samples were collected prior to initiation of bowel preparation for colonoscopy. As indicated in the study eligibility criteria, the Act Bold study prospectively enrolled participants aged ≥ 40 years prior to colonoscopy presenting for CRC screening or surveillance colonoscopy and who had no symptoms requiring immediate or near-term referral for diagnostic or therapeutic colonoscopy.¹⁸ The Act Fast study enrolled asymptomatic or symptomatic participants aged ≥40 years who underwent colonoscopy and were found to have either CRC of any stage confirmed with tissue biopsy or a colorectal lesion ≥1 cm suspicious for adenoma, sessile serrated polyps, or CRC on a pre-enrollment colonoscopy that remained ≥1 cm postcolonoscopy and required additional surgical excision or complex polypectomy.¹⁹ Inclusion and exclusion criteria for the Act Bold and Act Fast samples used for algorithm development are provided in the Supplementary Appendix. The final analysis population included 2994 participants with valid next-generation mt-sDNA test results, and a sample distribution that closely matched colonoscopy and histopathology results from the mt-sDNA pivotal DeeP-C study (ClinicalTrials.gov identifier: NCT01397747),¹³ including CRC stage; CRC lesion size; and frequency of nonadvanced adenomas, non-neoplastic findings, and samples with negative or no colonoscopy findings (Table A1). Case samples were obtained from participants with CRC or APLs based on colonoscopy and histopathologic results. Control samples were obtained from participants without CRC or APLs. For both Act Bold and Act Fast, the study protocols, all amendments, informed consent forms, and other relevant study documents were reviewed and prospectively approved by the Institutional Review Board/Ethics Committee. Trained laboratory personnel performing the next-generation mt-sDNA testing were blinded to the results of colonoscopy and histopathologic diagnosis.

Analyses for algorithm development and cross-validation

For algorithm development, ZDHHC1 was used as the reference marker to normalize the MDM signal based on input DNA. For all training set samples, the number of strands for each marker was calculated and log₁₀ transformed, and the log₁₀ strand numbers for LASS4, LRRC4, and PPP2R5C were plotted against log₁₀ strand numbers for ZDHHC1 to calculate normalization slopes. The reference gene ZDHHC1 was also used to establish a cut-off value for sample adequacy. To standardize MDMs, negative samples with non-neoplastic findings or negative colonoscopy findings (category 6) were used. MDMs were normalized to the reference marker. Standardized values were then calculated as the distance from the mean in units of standard deviation.

Next, correlations were determined between each of the panel markers, as well as between the panel markers and the reference standard of colonoscopy/histopathology results. Because each of the MDMs was more strongly correlated with one another than with colonoscopy/histopathology results, they were combined to create a single predictive input variable based on a median weighted average of the MDMs. To capture all stool marker data, the quantitative results from the MDMs and fecal hemoglobin were incorporated into modeling analyses to produce the final algorithm.

Logistic regression, general additive, neural network, and random forest models were fitted to the training data. Ultimately, the logistic regression model provided the highest accuracy (ie, highest specificity or sensitivity) with the least complexity and was used to identify the optimal threshold, with values above the determined threshold denoting a positive result and values at or below the threshold denoting a negative result.

Algorithm performance was then cross-validated in silico to minimize overfitting the model. For cross-validation, 25% of the samples were held back as test sets and the model was trained on the remaining 75% of the samples; 5000 splits of the dataset were performed.

Algorithm Validation on DeeP-C Test Set Samples

Study design and participants

A total of 7662 archived samples were available for study inclusion. Samples were prospectively collected from 2011 to 2013 during enrollment for the pivotal DeeP-C study and were archived until this analysis.¹³ Study participants were asymptomatic adults aged 50–84 years at average risk for CRC who were scheduled for screening colonoscopy and provided stool samples prior to colonoscopy preparation. The specific inclusion and exclusion criteria for the DeeP-C study have been described previously.¹³ None of these DeeP-C samples were used in previous algorithm development for the next-generation mt-sDNA test. Prior to validation on archival DeeP-C samples, the next-generation algorithm and cut-off value were locked based on results from the algorithm development study described above.

Participant samples were categorized based on the most advanced histopathology found on colonoscopy as follows: CRC (category 1), APLs (category 2), and nonadvanced adenomas (categories 3–5); samples with non-neoplastic findings (category 6.1) or negative colonoscopy with no tissue taken (category 6.2) comprised category 6 (Table A1).

Sample processing and biomarker detection

The archived buffered stool sample homogenates and hemoglobin samples were stored at −80 °C until use. Stool homogenates were thawed and processed to remove solids and assay inhibitors. Samples then underwent target-specific DNA capture, bisulfite treatment, and MDM quantification. Hemoglobin was quantified via an enzyme-linked immunosorbent assay. Details are available in the Supplementary Material. Laboratory personnel performing the next-generation mt-sDNA testing were blinded to the pathology and demographic information for patient samples as well as to fecal immunochemical testing and first-generation mt-sDNA results from the DeeP-C study. Operators analyzing the study data were also blinded to assay results until study completion.

Study end points for validation with archival DeeP-C samples

The primary study end points were sensitivity for CRC (category 1) and specificity for the absence of advanced neoplasia (categories 3–6, which include nonadvanced adenomas, non-neoplastic findings, and negative colonoscopy). Secondary study end points included sensitivity for APLs (category 2) and specificity among participants with non-neoplastic findings or negative colonoscopy (category 6). For comparative purposes, original results from the first-generation mt-sDNA test were analyzed for the subset of DeeP-C samples available in the present study. Receiver operating characteristic curves of sensitivity for CRC or any advanced neoplasia (CRC or APLs) vs 1-specificity (absence of advanced neoplasia) were plotted. Exploratory analyses also assessed impacts of age, CRC stage, APL subtype, and lesion size on the sensitivity and specificity of the next-generation mt-sDNA test.

Statistical analyses for validation with archival DeeP-C samples

All available samples from the DeeP-C study set were tested for algorithm validation and assessment of the next-generation mt-sDNA test characteristics. For the primary end points, CRC sensitivity was tested against the null hypothesis of 65% and specificity for the absence of advanced neoplasia was tested against the null hypothesis of 85.9%. Null hypotheses and power calculations for primary and secondary end point hypotheses are detailed in Table A2. All performance estimates are reported with 2-sided 95% confidence intervals (CIs) using the exact binomial method. If the lower bound of the 2-sided 95% CI was more than the null hypothesis, then the null was rejected, and the end point was considered to have been met. In exploratory analyses, next-generation mt-sDNA test performance was compared with the original first-generation mt-sDNA test results for the paired sample set using McNemar’s tests.

Results

Algorithm Development and Cross-Validation

The overall proportion of invalid specimens for analysis by the next-generation mt-sDNA test was low (17 of 3011 samples; 0.6%). Among the 2994 participants included in the training set with valid next-generation mt-sDNA test results, 2655 had nonadvanced adenomas, non-neoplastic findings, or no findings on colonoscopy, and 339 were positive for CRC or APLs (Table 1 and Supplementary Results provide a summary of study participant characteristics). Results of in silico cross-validation closely matched the results obtained by logistic regression. Point estimates for CRC sensitivity were 91.9% for both logistic regression results and cross-validation (Table 2). For APL sensitivity, point estimates were 41.3% and 40.7% for logistic regression results and cross-validation, respectively (Table 2). Test specificities for the absence of advanced neoplasia in the logistic regression model and cross-validation were 88.7% and 88.5%, respectively; and test specificities for non-neoplastic findings or negative colonoscopy were 91.2% and 90.9%, respectively (Table 2). Specificity was 89.0% for hyperplastic polyps <10 mm (category 6.1) and 92.1% for negative colonoscopy with no tissue biopsy taken (category 6.2). The high degree of agreement between logistic regression results and cross-validation suggests negligible overfitting with this simple classification model. Additional data on the performance of the next-generation test on the algorithm training set are included in the Supplementary Results.

Table 1.

Demographic Characteristics for Participants in the Algorithm Training and DeeP-C Test Sets

Characteristic	Training set (valid samples) N = 2994			DeeP-C test set N = 7662
	Controla,b (n = 2655)	Caseb,c (n = 339)	All samples
Age, mean (SD), y	62.1 (9.2)	64.0 (9.4)	62.3 (9.3)	63.8 (8.2)
Sex, n (%)
Female	1598 (60.2)	164 (48.4)	1762 (58.9)	4260 (55.6)
Male	1057 (39.8)	175 (51.6)	1232 (41.2)	3402 (44.4)
Race, n (%)
White	2285 (86.1)	290 (85.5)	2575 (86.0)	6697 (87.4)
Black	268 (10.1)	37 (10.9)	305 (10.2)	702 (9.2)
Asian	66 (2.5)	11 (3.2)	77 (2.6)	129 (1.7)
Other	35 (1.3)	1 (0.3)	36 (1.2)	131 (1.7)
Missing	1 (<0.1)	0 (0.0)	1 (<0.1)	3 (<0.1)
Ethnicity, n (%)
Hispanic/Latino	228 (8.6)	17 (5.0)	245 (8.2)	676 (8.8)
Non-Hispanic/Latino	2415 (91.0)	322 (95.0)	2737 (91.4)	6983 (91.1)
Other	12 (0.5)	0 (0.0)	12 (0.4)	3 (<0.1)
Smoking, n (%)
Current	206 (7.8)	52 (15.3)	258 (8.6)	691 (9.0)
Former	793 (29.9)	105 (31.0)	898 (30.0)	2695 (35.2)
Never	1656 (62.4)	182 (53.7)	1838 (61.4)	4276 (55.8)
First-degree family history of CRC, n (%)
Yes	381 (14.4)	44 (13.0)	425 (14.2)	NA
No	2274 (85.6)	294 (86.7)	2568 (85.8)	NA
Missing	0 (0.0)	1 (0.3)	1 (<0.1)	NA

APL, advanced precancerous lesion; CRC, colorectal cancer; NA, data not available.

^aControl samples were those with nonadvanced adenomas, non-neoplastic findings, and no findings on colonoscopy (categories 3–6).

^bIn an analysis comparing demographic characteristics between case and control groups, the following P values were obtained: age, sex, and smoking, P < .001; race, P = .547; ethnicity, P = .018; first-degree family history of CRC, P = .151.

^cCase samples were those positive for CRC or APLs.

Table 2.

Performance of the Next-Generation mt-sDNA Test in Algorithm Development and Cross-Validation

Endpoint	Samples, N	Best fit, %	Cross-validation, %	95% CI
Sensitivity
CRC	99	91.9	91.9	84.9–95.8
APL	240	41.3	40.7	34.8–47.1
Specificity
Absence of advanced neoplasiaa	2655	88.7	88.5	87.2–89.7
Non-neoplastic findings or negative colonoscopyb	1845	91.2	90.9	89.5–92.1

APL, advanced precancerous lesion; CI, confidence interval; CRC, colorectal cancer; mt-sDNA, multitarget stool DNA.

^aIncludes all nonadvanced adenomas, non-neoplastic findings, and negative colonoscopy (categories 3–6).

^bCategory 6 (6.1 or 6.2).

Participants in the DeeP-C Test Set

The algorithm test set comprised 7662 participants with valid next-generation mt-sDNA test results, including 7022 without advanced neoplasia and 640 who were positive for CRC (N = 57) or APLs (N = 583). Among these samples, the mean (standard deviation) participant age was 63.8 (8.2) years and 55.6% were female (Table 1). Most participants were White (87.4%) and 9.2% were Black. The most common ethnicity was non-Hispanic/Latino (91.1%). Participant demographics and characteristics were comparable to those included in the primary analysis of the pivotal study.¹³

Algorithm Validation in the DeeP-C Test Set

Sensitivity of the next-generation mt-sDNA test for CRC was 93.0% (95% CI, 83.0%–98.1%), which was identical to that of the first-generation mt-sDNA test (P > .99; Table 3). Sensitivity for APLs was significantly higher for the next-generation mt-sDNA test (48.4% [95% CI, 44.2%–52.5%]) compared with the first-generation test (41.2% [95% CI, 37.1%–45.3%]; P = .0003). Additional sensitivity comparisons for APL subsets are provided in Table 3. Approximately one-quarter (25.9% [151/583]) of APLs were reported as 10 mm in size.

Table 3.

Performance of the Next-Generation mt-sDNA Test Compared With Historical Performance of the First-Generation mt-sDNA Test in the DeeP-C Test Set

Most advanced colonoscopy finding	Test set (N = 7662)			P valueb
	Valid DeeP-C results, n	Next-gen mt-sDNA test sensitivity, % (95% CI)	First-gen mt-sDNA test sensitivity,a % (95% CI)
CRC	57	93.0 (83.0–98.1)	93.0 (83.0–98.1)	>.99
APLs	583	48.4 (44.2–52.5)	41.2 (37.1–45.3)	.0003
High-grade dysplasia	30	76.7 (57.7–90.1)	63.3 (43.9–80.1)	.16
Villous histology	194	56.7 (49.4–63.8)	44.3 (37.2–51.6)	.0002
Adenoma ≥10 mm	284	41.9 (36.1–47.9)	35.6 (30.0–41.4)	.02
Serrated or hyperplastic lesion ≥10 mm	75	40.0 (28.9–52.0)	45.3 (33.8–57.3)	.43

	Valid DeeP-C results, n	Next-gen mt-sDNA test specificity, % (95% CI)	First-gen mt-sDNA test specificity, % (95% CI)
Absence of advanced neoplasiac	7022	88.5 (87.7–89.2)	86.9 (86.1–87.7)	<.0001
Non-neoplastic findings or negative colonoscopyd	4859	90.4 (89.5–91.2)	88.6 (87.6–89.4)	<.0001
Negative colonoscopye	3454	91.6 (90.7–92.5)	90.0 (89.0–91.0)	.003

APL, advanced precancerous lesion; CI, confidence interval; CRC, colorectal cancer; first-gen, first-generation; mt-sDNA, multitarget stool DNA; next-gen, next-generation.

^aHistorical results on the same subset of samples available for testing with the next-gen mt-sDNA test.

^bP values calculated using McNemar’s test.

^cIncludes all nonadvanced adenomas, non-neoplastic findings, and negative colonoscopy (categories 3–6).

^dCategory 6 (6.1 or 6.2).

^eIncludes no findings on colonoscopy (category 6.2).

For the primary specificity outcome of the absence of advanced neoplasia, the next-generation mt-sDNA test significantly outperformed the first-generation test (88.5% [95% CI, 87.7%–89.2%] vs 86.9% [95% CI, 86.1%–87.7%], respectively; P < .0001; Table 3). Similarly, secondary specificity for non-neoplastic findings or negative colonoscopy was significantly higher for the next-generation mt-sDNA test compared with the first-generation test (90.4% [95% CI, 89.5%–91.2%] vs 88.6% [95% CI, 87.6%–89.4%]; P < .0001). Specificity for negative colonoscopy (ie, no tissue taken) was also significantly higher for the next-generation mt-sDNA test (91.6% [95% CI, 90.7%–92.5%]) vs the first-generation test (90.0% [95% CI, 89.0%–91.0%]; P = .003).

The next-generation mt-sDNA test maintained high sensitivity for CRC stage I (92.0% [95% CI, 74.0%–99.0%]), stage II (100.0% [95% CI, 83.9%–100.0%]), stage III (83.3% [95% CI, 35.9%–99.6%]), and stage IV (75.0% [95% CI, 19.4%–99.4%]) (Table 4). The first-generation and next-generation mt-sDNA tests had equivalent performance across CRC stages (Table 4), with concordant results except 2 stage III CRC cases. Among APL subgroups, next-generation mt-sDNA test sensitivity was highest for adenomas with high-grade dysplasia (76.7% [95% CI, 57.7%–90.1%]), followed by adenomas with villous histology (56.7% [95% CI, 49.4%–63.8%]), adenomas ≥10 mm in size (41.9% [95% CI, 36.1%–47.9%]), and serrated or hyperplastic lesions ≥10 mm in size (40.0% [95% CI, 28.9%–52.0%]; Figure 1A). For APLs <10 mm in size, sensitivity was 37.3% (95% CI, 24.1%–51.9%) and increased with increasing lesion size (Figure 1B). Sensitivity was higher for APLs of 10–19 mm (44.7% [95% CI, 40.0%–49.4%]), 20–29 mm in size (68.3% [95% CI, 55.0%–79.7%]), and ≥30 mm in size (80.6% [95% CI, 62.5%–92.5%]). Test specificity was highest for participants aged 50–64 years, with reduced specificity in older age groups (P value for trend <.0001; Table 5).

Table 4.

CRC Sensitivity by Index Lesion Stage for the Next-Generation mt-sDNA Test Compared With the First-Generation Test in the DeeP-C Test Set

CRC lesion stage	Test set (N = 7662; CRC, n = 56a)
	Valid DeeP-C results, n	Next-gen mt-sDNA test sensitivity, % (95% CI)	First-gen mt-sDNA test sensitivity,b % (95% CI)
Stage I	25	92.0 (74.0–99.0)	92.0 (74.0–99.0)
Stage II	21	100.0 (83.9–100.0)	100.0 (83.9–100.0)
Stage III	6	83.3 (35.9–99.6)	83.3 (35.9–99.6)
Stage IV	4	75.0 (19.4–99.4)	75.0 (19.4–99.4)

CI, confidence interval; CRC, colorectal cancer; first-gen, first-generation; mt-sDNA, multitarget stool DNA; next-gen, next-generation.

^aOne CRC sample could not be staged and was not included in this analysis.

^bHistorical results on the same subset of samples available for testing with the next-gen mt-sDNA test.

Figure 1.

Sensitivity of the next-generation mt-sDNA test for advanced precancerous lesions by (A) type and (B) size in the DeeP-C test set. APL, advanced precancerous lesion; mt-sDNA, multitarget stool DNA.

Table 5.

Next-Generation mt-sDNA Test Specificity by Age Group in the DeeP-C Test Set

Age group, y	Specificity for the absence of advanced neoplasiaa		Specificity for non-neoplastic findings or negative colonoscopyb
	N	Next-generation mt-sDNA test specificity, % (95% CI)	N	Next-generation mt-sDNA test specificity, % (95% CI)
50–54	1548	94.9 (93.7–95.9)	1183	95.6 (94.3–96.7)
55–59	584	91.8 (89.3–93.9)	421	94.5 (91.9–96.5)
60–64	563	90.2 (87.5–92.6)	391	91.6 (88.4–94.1)
65–69	2647	87.1 (85.8–88.4)	1750	89.4 (87.9–90.8)
70–74	1206	84.5 (82.3–86.5)	791	85.1 (82.4–87.5)
75–79	398	79.1 (74.8–83.0)	267	83.1 (78.1–87.4)
≥ 80	76	78.9 (68.1–87.5)	56	80.4 (67.6–89.8)

CI, confidence interval; mt-sDNA, multitarget stool DNA.

^aIncludes all nonadvanced adenomas, non-neoplastic findings, and negative colonoscopy (categories 3–6).

^bCategory 6 (6.1 or 6.2).

For the next-generation mt-sDNA test, the area under the receiver operating characteristic curve for CRC was 0.97 (95% CI, 0.94–0.99) (Figure 2A) and was 0.78 (95% CI, 0.76–0.80) (Figure 2B) for advanced neoplasia (CRC or APLs); for the first-generation mt-sDNA test, areas under the receiver operating characteristic curves were 0.94 (95% CI, 0.91–0.97) and 0.74 (95% CI, 0.71–0.76), respectively. The differences in area under the receiver operating characteristic curves for the next-generation mt-sDNA test vs the first-generation mt-sDNA test were 0.03 (95% CI, 0.004–0.05; P = .02) for CRC and 0.04 (95% CI, 0.02–0.06; P < .0001) for any advanced neoplasia.

Figure 2.

ROC curves and AUC for (A) CRC^a (sensitivity category 1) and (B) any advanced neoplasia^b (CRC or APLs, sensitivity categories 1–2) vs 1-specificity for the absence of advanced neoplasia (categories 3–6) for the first-generation and next-generation mt-sDNA tests in the DeeP-C test set. APL, advanced precancerous lesion; AUC, area under the curve; CRC, colorectal cancer; mt-sDNA, multitarget stool DNA; ROC, receiver operating characteristic. ^aDifference in area under the ROC curve for the next-generation mt-sDNA test vs the first-generation mt-sDNA test: 0.03 (95% CI, 0.004–0.05); P = .02. ^bDifference in area under the ROC curve for the next-generation mt-sDNA test vs the first-generation mt-sDNA test: 0.04 (95% CI, 0.02–0.06); P < .0001.

Discussion

Since its approval in 2014,⁹ the first-generation mt-sDNA test has become a widely used noninvasive option for average-risk CRC screening.²⁰ Compared with fecal immunochemical testing, the first-generation mt-sDNA test has higher sensitivity for CRC and APLs but also has lower specificity in single-application screening.¹³ A mt-sDNA test with higher specificity would improve testing efficiency by decreasing false-positive results.⁷ Recently, novel CRC-associated MDMs were identified¹⁴ and evaluated for their ability to discriminate for CRC.¹⁵ In a recently published identification and verification study, MDM markers LASS4, LRRC4, and PPP2R5C and the reference marker ZDHHC1 exhibited high sensitivity for CRC and APLs and specificity for colorectal neoplasia, and were chosen for the next-generation mt-sDNA test, along with fecal hemoglobin.¹⁷ The oncogene KRAS (DNA mutation marker) was removed, allowing all tissue and DNA input to be allocated to MDM quantification, thus aiding in improved analytical sensitivity.¹⁷ In the present study, we developed and cross-validated an algorithm and clinical cut-off value for the next-generation mt-sDNA CRC screening test using the optimized biomarker panel. We then evaluated the next-generation mt-sDNA test performance using this locked, optimized algorithm and cut-off in an analysis of archived samples that were prospectively collected for the pivotal DeeP-C study. In this early performance evaluation, the next-generation mt-sDNA test demonstrated promising performance characteristics for CRC screening.

The results of this study support the need for multimarker inclusion in the development of stool-based CRC screening tests. Notably, both current and next-generation mt-sDNA screening tests, which incorporate molecular and hemoglobin markers, have shown greater sensitivity for detecting CRCs and APLs in comparison to fecal hemoglobin alone.¹³

In comparing the original results of the first-generation mt-sDNA test with the performance of the next-generation mt-sDNA test on the same samples, we found that sensitivity for CRC was the same between the 2 tests, while the next-generation mt-sDNA test had statistically significantly higher sensitivity for APLs. While the CRC sensitivities were equal, each version of the mt-sDNA test missed one CRC that the other detected. Furthermore, the next-generation mt-sDNA test had clinically important and statistically significantly higher specificity compared with the first-generation test for the primary and secondary end points. The results support an improvement in specificity of the next-generation mt-sDNA test from the first-generation test, with CRC sensitivity maintained. Even small improvements in test performance can have important, favorable clinical and economic consequences for population-level CRC screening. At the population level, higher specificity will result in fewer false positive tests, reducing referrals for follow-up colonoscopy and lowering patient and provider concerns regarding the need for additional diagnostic evaluation or more intensive monitoring. To this end, modeling studies are underway to demonstrate the potential impact of improved test performance on population-level screening outcomes.

In the DeeP-C test set, sensitivity for APLs with high-grade dysplasia was higher for the next-generation mt-sDNA test (76.7%) than for first-generation test (63.3%). It is important to note that this estimate for the first-generation test differs from the sensitivity for high-grade APLs reported in the pivotal DeeP-C study (69.2%),¹³ because there were 9 APL samples with high-grade dysplasia from the DeeP-C study that were not available for inclusion in the current analysis, 8 of which were positive in the original DeeP-C study.

The next-generation mt-sDNA test had the greatest specificity (>90%) among participants aged 50–64 years, with reduced specificity among older participants, a finding consistent with the first-generation mt-sDNA test. The age-related difference in specificity may be due to the types and sizes of lesions present in participants of different ages or to changes in biomarker levels from age-related background DNA methylation.

Limitations of this study require comment. Samples positive for CRC that were used in algorithm training were collected postcolonoscopy, and a portion of these samples (41.4% [41 of 99 valid CRC samples]) were collected from symptomatic patients. Therefore, these cases may not represent the spectrum of CRC from an average-risk CRC screening population. The DeeP-C samples were collected more than 10 years prior to this analysis, and only a subset of, rather than all, archived samples were available for evaluation. Based on an analysis of a small subset of specimens, we verified that sample integrity was maintained prior to testing the algorithm test set samples. Furthermore, there is no reason to believe that the available specimens differ from those specimens that were not available for this study. Additionally, historical results of the first-generation mt-sDNA test on the same sample set (rather than repeat testing with the commercially available mt-sDNA test) were used for comparison with the next-generation test. There is also a statistical limitation to using the same sample set to evaluate both the first-generation and next-generation mt-sDNA tests: overall type 1 error is no longer controlled across all hypothesis testing. Finally, racial/ethnic diversity was relatively limited for both the training set and test set samples. Further validation of the algorithm and locked clinical cut-off value for the next-generation mt-sDNA test will be obtained from a broader, more diverse cohort of participants enrolled in the prospective BLUE-C pivotal study (ClinicalTrials.gov identifier: NCT04144738).²¹

In summary, in this study, we first used samples from specimen collection studies to train the algorithm and lock the clinical cut-off value of the next-generation mt-sDNA test. This necessary step of the development process ensures that the next-generation mt-sDNA test will be evaluated with the current algorithm and cut-off value. In a preliminary and independent validation analysis on archived, prospectively collected samples, we then showed that the next-generation mt-sDNA test has improved specificity while maintaining or increasing sensitivity for detecting CRC and APLs. These improved test characteristics may help increase the efficiency of CRC screening and improve clinical outcomes among average-risk individuals.