Comparison of Laboratory-Developed Tests and FDA-Approved Assays for BRAF, EGFR, and KRAS Testing

Abstract

Importance

The debate about the role of the Food and Drug Administration (FDA) in the regulation of laboratory-developed tests (LDTs) has focused attention on the analytical performance of all clinical laboratory testing. This study provides data comparing the performance of LDTs and FDA-approved companion diagnostics (FDA-CDs) in proficiency testing (PT) provided by the College of American Pathologists Molecular Oncology Committee.

Objective

To compare the analytical performance of LDTs and FDA-CDs on well-characterized PT samples and to compare the practice characteristics of laboratories using these assays.

Design, Setting, and Participants

This comparison of PT responses examines the performance of laboratories participating in the College of American Pathologists PT for 3 oncology analytes for which both FDA-CDs and LDTs are used: BRAF, EGFR, and KRAS. A total of 6897 PT responses were included: BRAF (n = 2524; 14 PT samples), EGFR (n = 2216; 11 PT samples), and KRAS (n = 2157, 10 PT samples). US Food and Drug Administration companion diagnostics and LDTs are compared for both accuracy and preanalytic practices of the laboratories.

Main Outcomes and Measures

As per the College of American Pathologists PT standards, results were scored and the percentages of acceptable responses for each analyte were compared. These were also broken down by the specific variants tested, by kit manufacturer for laboratories using commercial reagents, and by preanalytic practices.

Results

From analysis of 6897 PT responses, this study demonstrates that both LDTs and FDA-CDs have excellent performance overall, with both test types exceeding 97% accuracy for all 3 genes (BRAF, EGFR, and KRAS) combined. Rare variant-specific differences did not consistently favor LDTs or FDA-CDs. Additionally, more than 60% of participants using an FDA-CD reported adapting their assay from the approved procedure to allow for a greater breadth of sample types, minimum tumor content, and instrumentation, changing the classification of their assay from FDA-CD to LDT.

Conclusions

This study demonstrates the high degree of accuracy and comparable performance of both LDTs and FDA-CDs for 3 oncology analytes. More significantly, the majority of laboratories using FDA-CDs have modified the scope of their assay to allow for more clinical practice variety, rendering them LDTs. These findings support both the excellent and equivalent performance of both LDTs and FDA-CDs in clinical diagnostic testing.

This data analysis compares the analytical validity of US Food and Drug Administration–approved companion diagnostics and laboratory-developed tests for 3 genetic analytes using uniform reference materials from the College of American Pathologists Molecular Oncology Committee.

Key Points

Question

Are there performance differences between laboratory-developed tests (LDTs) and US Food and Drug Administration–approved companion diagnostics (FDA-CDs [also known as in vitro diagnostics])?

Findings

In 6897 proficiency testing responses, both LDTs and FDA-CDs exceed 97% accuracy combined across all comparable molecular oncology proficiency testing samples. In addition, more than 60% of participants using FDA-CDs report modifying the approved procedure to broaden clinical practice, rendering them LDTs.

Meaning

This study supports the accuracy and comparable performance of LDTs and FDA-CDs and indicates that the majority of laboratories purchasing in vitro diagnostics for FDA-CDs are in fact using them as LDTs.

Introduction

Recent public debate has focused on the regulation of clinical tests.^1,2,3,4,5 This includes regulatory practices to ensure the highest quality patient care, with the US Food and Drug Administration (FDA) recommending greater oversight of all in vitro testing. In cancer molecular testing, there are several FDA companion diagnostics (FDA-CDs) sold under the designation of in vitro diagnostics (IVDs). There are many more clinical laboratory-developed tests (LDTs) that are designed, validated, and performed in a single laboratory. Proprietary commercial assays can also fall under the LDT umbrella. This study compares analytical validity of FDA-CDs and LDTs using uniform reference materials from the College of American Pathologists Molecular Oncology Committee.

Methods

Sample Definition and Timeframe

The College of American Pathologists is a well-known provider of external proficiency testing (PT) materials that provide a mechanism for laboratories to fulfill the Clinical Laboratory Improvement Amendments requirement for laboratories to assess the analytical validity of clinical assays during initial development and ongoing clinical use. The College of American Pathologists BRAF PT data from the second half of 2011 through the first half of 2015 were included in this comparison of PT responses (14 samples and 8 mailings). For EGFR, data from the first half of 2013 through the first half of 2015 were examined (11 samples and 5 mailings). For KRAS, data from the second half of 2013 through the first half of 2015 were examined (10 samples and 4 mailings). Additional information detailing PT results categorized as “acceptable,” specific variants assessed, and select qualitative and quantitative preanalytic considerations are available in the Supplement. Based on the participant responses, the testing methodology was categorized as either FDA-CD or LDT. Laboratories using a kit manufactured by a vendor with FDA approval for that kit after the FDA approval date were analyzed as FDA-CDs. All other assays were analyzed as LDTs.

Statistical Analysis

A χ² test of association compared acceptability across FDA-CDs and LDTs for all tests. For testing acceptability across FDA and LDT by sample type, the Fisher exact test was used owing to counts of fewer than 5 in at least 1 table cell. A significance level of .05 was used.

Results

The overall BRAF College of American Pathologists PT acceptable rate was 96.2% with LDTs achieving a 96.6% acceptable rate while FDA-CD results were significantly lower at 93.0% (P = .002) (Table 1). The main cause of this discrepancy was p.V600K analysis, with LDT acceptable rates at 88.0% while FDA-CDs were 66.1% acceptable (P < .001). For EGFR, LDTs performed slightly less well than the FDA-CDs overall (97.6% acceptability for LDT vs 99.1% for FDA-CDs; P = .03) (Table 1). This discrepancy was driven by the detection of the EGFR p.L861Q mutation (91% for LDTs vs 100% of FDA-CDs; P = .04). For KRAS, there was no significant difference between LDTs and the FDA-CD acceptability rates overall or when the data were broken down by wild-type positions or individual variants.

Table 1. Acceptable Proficiency Testing Results of FDA Companion Diagnostics vs LDT for BRAF, EGFR, and KRAS.

Gene	FDA Companion Diagnositcs, No. (%)	Laboratory-Developed Tests, No. (%)	χ2 Test	P Value
BRAF
All	300 (93.0)	2224 (96.6)a	9.1800	.002b
Wild type	88 (100)	616 (99.0)	ND	.99
p.V600E	153 (99.4)	1300 (97.5)	ND	.25
p.V600K	59 (66.1)	308 (88.0)a	18.0775	<.001b
EGFR
All	549 (99.1)a	1667 (97.6)	4.6011	.03b
Wild type	169 (100)	383 (99.7)	ND	.99
Exon 19 del	109 (99.1)	254 (99.2)	ND	.99
p.G719A	82 (98.8)	479 (97.7)	ND	.99
p.L858R	101 (100)	280 (98.6)	ND	.58
p.L861Q	46 (100)a	129 (90.7)	ND	.04b
p.T790M	42 (92.9)	142 (93.0)	ND	.99
KRAS
All	331 (98.8)	1826 (97.4)	ND	.16
Wild type	34 (100)	175 (98.9)	ND	.99
p.G12A	34 (97.1)	171 (98.3)	ND	.52
p.G12C	68 (98.5)	344 (97.7)	ND	.99
p.G12R	67 (100)	359 (96.1)	ND	.14
p.G12S	65 (100)	365 (98.1)	ND	.60
p.G12V	32 (93.8)	190 (96.3)	ND	.62
p.G13D	31 (100)	222 (97.3)	ND	.99

Abbreviations: FDA, US Food and Drug Administration; ND, no data.

^aThis test is significantly higher.

^bThis P value is statistically significant.

For all 3 surveys, FDA-CD participants reported using off-label preanalytical practices more than 60% of the time. These off-label practices included accepting unapproved specimen and tumor types (Figure), accepting specimens with lower tumor content than are required for the approved assay, and not quantifying DNA before performing the assay (Table 2). These alterations of the FDA-approved procedure render these tests LDTs.

Figure. Methods of Performing Assays by Laboratories Based on Tissue Preparation and Tumor Type.

A, Percentage of laboratories performing their assays on FFPE tissue preparations exclusively vs laboratories that are inclusive of other types of tissue preparations. B, Percentage of laboratories performing their assays on FDA-approved tumor type (melanoma for BRAF) exclusively vs laboratories that are inclusive of other types of tumors. FDA indicates US Food and Drug Administration; FFPE, formalin-fixed paraffin-embedded; LDT, laboratory-developed tests.

Table 2. Respondents Reporting That Their Laboratory Performs the Following Preanalytic Steps.

Preanalytic Step	Respondent Reporting, %
	BRAF		EGFR		KRAS
	FDA	LDT	FDA	LDT	FDA	LDT
Pathologist review	99.0	93.1	95.2	91.2	91.5	93.2
DNA quantification	92.0	85.0	76.3	87.1	47.4	85.9
Tissue dissection	92.7	90.4	87.3	86.4	87.3	87.4

Abbreviations: FDA, US Food and Drug Administration; LDT, laboratory-developed tests.

Discussion

Our primary goal was to compare the accuracy of results between laboratories using FDA-CDs and LDTs. For 1 of 13 variants (BRAF p.V600K), LDTs performed statistically better than FDA-CDs, although 1 of the FDA-CDs is approved only for p.V600E and not other variants at that amino acid or adjacent loci. However, several of these other variants may also respond to BRAF-targeted therapies. For the second variant (EGFR p.L861Q), FDA-CDs performed slightly, but statistically, better than LDTs (Table 1).

This study also compared FDA-CDs and LDTs for preanalytical factors where specifically mentioned in the FDA-CD protocols, including specimen preparations used for testing, pathologist review, DNA quantification, and tissue dissection. The preanalytic questions highlight the fact that many FDA-CD laboratories conduct practices that are not in accord with their FDA-approved methods. Both FDA-CD and LDT laboratories accept a wide range of specimen preparations, as well as tumor types. Although this flexibility is advantageous for patient care, it is important to recognize that the use of specimens other than formalin-fixed paraffin-embedded samples of the specified tumor type for the FDA-CDs is off-label, resulting in reclassification of the assay as an LDT.

Limitations

There are several limitations of this study. First, each PT response was treated as an individual data point rather than grouping the responses by laboratory. Second, all participants using an assay produced by a vendor with an FDA-CD after the FDA approval date were categorized as FDA-CDs although some laboratories may have been using an alternate unapproved kit or had validated it as an LDT. Third, some laboratories failed to provide responses to all of the PT questions. This resulted in inconsistent numbers of data points for many of the variables, although the discrepancies are small and have minimal impact. Finally, the PT did not include questions about all aspects of the practice of each assay. Therefore, this survey cannot determine if there are additional ways that laboratories are using the FDA-CDs off-label.

Conclusions

We find no differences overall between FDA-CDs and LDTs in assay performance for these 3 analytes, with an average of over 97% accuracy from both types of assays more than the 3 surveys, although a technical limitation of one FDA-CD is noted. This study identified alterations from the FDA-approved procedure in greater than 60% of respondents using FDA-CDs, likely to allow for more clinical practice flexibility. Given the overall comparable performance of FDA-CDs and LDTs, as well as the significant off-label use of FDA-CDs, these data question the distinction between FDA-CDs and LDTs from a regulatory standpoint and note the greater clinically relevant applications of LDTs.

Supplement.

eMethods.