Temporary Regulatory Deviations and the Coronavirus Disease 2019 (COVID-19) PCR Labeling Update Study Indicate What Laboratory-Developed Test Regulation by the US Food and Drug Administration (FDA) Could Look Like

Abstract

The coronavirus disease 2019 (COVID-19) response necessitated innovations and a series of regulatory deviations that also affected laboratory-developed tests (LDTs). To examine real-world consequences and specify regulatory paradigm shifts, legislative proposals were aligned on a common timeline with Emergency Use Authorization (EUA) of LDTs and the US Food and Drug Administration (FDA)-orchestrated severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) labeling update study. The initial EUA adoption by LDT developers shows that the FDA can have oversight over LDTs. We used efficiency-corrected microcosting of our EUA PCR assay to estimate the national cost of the labeling update study to $0.3 to $1.4 million US dollars. Labeling update study performance data showed lower average detection limits in commercial in vitro diagnostic (IVD) assays versus LDTs (32,000 ± 75,000 versus 71,000 ± 147,000 nucleic acid amplification tests/mL; P = 0.04); however, comparison also shows that FDA review of IVD assays and LDTs did not prevent differences between initial and labeling update performance (IVD assay, P < 0.0001; LDT, P = 0.003). The regulatory shifts re-emphasized that both commercial tests and LDTs rely heavily on laboratory competence and procedures; however, lack of performance data on authorized tests, when clinically implemented, precludes assessment of the benefit related to regulatory review. Temporary regulatory deviations during the pandemic and regulatory science tools (ie, reference material) have generated valuable real-world evidence to inform pending legislation regarding LDT regulation.

One of the unanticipated consequences of the coronavirus disease 2019 (COVID-19) pandemic is that it forced the ongoing debate of diagnostic test regulation into the public consciousness (Health Affairs Blog, https://www.healthaffairs.org/doi/10.1377/hblog20200814.376610, last accessed August 29, 2021).1, 2, 3, 4 Specifically, the regulatory oversight of laboratory-developed procedures, known as laboratory-developed tests (LDTs), has been a contentious issue in the United States (Association for Molecular Pathology, https://www.amp.org/advocacy/advocacy-resources/laboratory-developedtesting-procedures-ldps, last accessed August 29, 2021; Diagnostics World, https://www.diagnosticsworldnews.com/news/2019/10/22/mixed-opinions-on-how-to-regulate-laboratory-developed-tests, last accessed August 29, 2021).5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33 LDTs are assays that are assembled, validated, and performed within a clinical laboratory.^{16,27,29,34,35} The laboratories are not required to submit data on the test for US Food and Drug Administration (FDA) review. Instead, laboratories must follow the regulations of the Clinical Laboratory Improvement Amendments (CLIA) of 1988. In contrast, test systems that are developed and sold by a manufacturer (ie, those that are distributed in interstate commerce) are regulated by the FDA under the Federal Food, Drug, and Cosmetic Act.³⁴ Once FDA authorized and sold, these in vitro diagnostic (IVD) devices are implemented, after appropriate verification of performance, in a CLIA-certified laboratory.^14,15,20,31 Changes to the nuanced regulatory balance between CLIA and FDA may critically affect US patient access to testing.^{4,13,24,29,32,34,}36, 37, 38 An ideal regulatory framework for oversight of tests and procedures must balance the need for accuracy and safety with ensuring that new tests are made available to patients within a rapid time frame.^{4,13,19,32,34,37,38} Congress has not expanded the FDA's authority to regulate LDTs; however, shortly following the introduction of the Verifying Accurate Leading-Edge IVCT (in vitro clinical test) Development Act (VALID Act) on March 5, 2020 (https://www.congress.gov/bill/116th-congress/senate-bill/3404/text, last accessed June 19, 2021), the United States plunged into the COVID-19 pandemic.

Diagnostic oversight took center stage in March 2020 because the FDA published guidance regulating severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) diagnostics.1, 2, 3, 4 In contrast with the prevailing paradigm that CLIA-certified laboratories can develop and validate LDTs to perform clinical testing without FDA oversight, the FDA outlined eligibility criteria for the use and authorization of LDTs (FDA, https://www.fda.gov/medical-devices, last accessed June 19, 2021). The basis for this decision was the public health emergency declaration (85 FR 17335) by the US Department of Health and Human Services that triggered section 564 of the Federal Food, Drug, and Cosmetic Act.^1,2 Although later rescinded (US Department of Health and Human Services, https://www.hhs.gov/coronavirus/testing/covid-19-diagnostic-data-reporting/index.html, last accessed June 19, 2021; College of American Pathologists, https://www.cap.org/covid-19, last accessed June 19, 2021), the FDA guidance documents effectively asserted authority over LDTs; and LDT developers had to undergo Emergency Use Authorization (EUA). In the early phase of the COVID-19 pandemic, this authority allowed the FDA to “help strengthen the nation's public health protections” (FDA, https://www.fda.gov/emergency-preparedness-and-response/mcm-legal-regulatory-and-policy-framework/emergency-use-authorization#abouteuas, last accessed June 19, 2021). The resulting scenario is interesting: the pandemic mandated diagnostic innovation to tackle disease identification, whereas the EUA requirement attempted to establish common ground regarding performance and accuracy of deployed systems. In an echo of the proposed legislation,^14,39,40 this temporary assertion of authority marked the first time that the FDA took practical steps to regulate LDTs.

In August 2020, parallel to US Department of Health and Human Services enabling laboratories to administer COVID-19 tests without FDA authorization, the FDA sent out agency-verified reference materials along with a study protocol to conduct an interlaboratory study (labeling update study). Practically speaking, all laboratories that received EUA for SARS-CoV-2 molecular diagnostics received an FDA notice:

“As part of the condition of authorization under the EUA it was listed that you will evaluate the analytical limit of detection and assess traceability of your product with any FDA-recommended reference material(s). After submission to FDA and DMD/OHT7-OIR/OPEQ/CDRH's review of and concurrence with the data, you will update labeling to reflect the additional testing. Such labeling updates will be made in consultation with, and require concurrence of, DMD/OHT7- OIR/OPEQ/CDRH. Through collaboration with CBER/FDA, a suitable reference panel is now available, and we are requesting you to test it."

(EUA communication from the FDA, https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/in-vitro-diagnostics-euas-molecular-diagnostic-tests-sars-cov-2#individual-molecular)

The COVID-19 pandemic resulted in two specific regulatory deviations: First, the temporary EUA requirement for COVID-19 diagnostics by the FDA represents a deviation from current practice of LDT regulation (ie, no FDA oversight). Second, the subsequent labeling update study represents a large-scale comparator study across manufacturers and LDT developers. Together, these temporary deviations provide a unique chance to examine what regulatory oversight of LDTs by the FDA could practically look like. To our knowledge, an examination of these LDT-related regulatory deviations has not been performed.

Herein, we review the timeline of regulating LDTs in the specific context of COVID-19, describe the regulatory deviations, and examine the labeling update study protocol, cost, and results. The COVID-19 pandemic required balancing innovation with patient safety and has led to several, temporary regulatory deviations. Ignoring the lessons from these regulatory paradigm shifts can be considered a missed opportunity. As Congress will debate legislative proposals to modernize diagnostic test regulation, an evidence-informed dialogue will be essential to align legislative intent with practical feasibility.

Materials and Methods

Study Design and Setting

The study design was a combination of literature review, laboratory data, and compilation of publicly available data. Two of the authors (H.M. and J.K.L.) discussed and selected key milestones related to LDTs and COVID-19 regulation. The milestones were sorted and placed on two separate timelines that were co-anchored on the introduction of the VALID and Verified Innovative Testing in American Laboratories Act (VITAL) Acts in March of 2020. All laboratory tests were performed in CLIA-certified laboratories of the Massachusetts General Hospital. Institutional review board approval was obtained for EUA-related experiments (2020P000895).

Initial EUA Review Time and Adoption Analysis

To compare our experience with EUA review time with laboratories achieving EUA status at the same time, one of the authors (M.M.M.) identified contacts from the FDA website and contacted laboratories (April 25, 2020) to collect initial submission date as well as EUA date. Data were separated between laboratories and manufacturers. To compare the adoption rate of EUA for manufacturers versus laboratories, the dates and numbers of authorized SARS-CoV-2 PCR tests were extracted.

SARS-CoV-2 Assay and Microcost Analysis

The labeling update experiments were performed using Massachusetts General Hospital's EUA real-time PCR assay for detection of SARS-CoV-2 RNA.⁴¹ Briefly, the assay targets SARS-CoV-2 N1, N2, and human RNaseP in separate wells. The cost analysis included nucleic acid extraction using the Total Nucleic Acid Isolation kit on the MagNA Pure 24 instrument (Roche, Basel, Switzerland), followed by PCR using the 2 (N1-/N2-) SARS-CoV-2 specific primer and probe mix (IDT, Coralville, IA) and TaqPath 4× Master Mix on a QuantStudio 7 real-time PCR instrument (ThermoFisher Scientific, Waltham, MA). The microcost analysis⁴² (Supplemental Table S1) included six components, and the unit price (% of total cost) per component was as follows: accessioning, $3.30 (6%); extraction, $9.29 (16%); wet-laboratory reagents, $13.58 (23%); laboratory consumables, $5.30 (9%); personnel time, $26.46 (45%); and information technology/storage, $0.43 (1%). The direct cost of $58.36/sample for each assay does not include indirect costs, costs for test validation, EUA submission, nasopharyngeal swabs/media/matrix, transport, or proficiency testing.

Cost and Effort Estimation of the Labeling Update Study

The FDA sent out agency-certified reference materials alongside a protocol with four experiments: reconfirmation of negatives (specificity), determination of the limit of detection (LOD; sensitivity), confirmation of the LOD (reproducibility), and testing of unknown/contrived samples (concordance). The total cost of the FDA labeling update study was calculated by multiplying the cost per sample. Our microcost was compared with the Centers for Medicare & Medicaid Services ruling that CPT87635 (Sars-cov-2 covid-19 amp prb) “shall be paid for at the rate of $100.” The cost of negative patient samples (required as a dilution matrix) was not accounted for; however, the volume of transport medium, master mix, primer/probe mix, and hours of labor (technologist time) were accounted for. To enable cost comparison with other settings, increased efficiency (eg, via laboratory automation, single-target assays, and/or pooled testing in other settings) was accounted for by using a factor of 4.3 (FDA, https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-issues-first-emergency-authorization-sample-pooling-diagnostic, last accessed June 19, 2021). The overall national cost was estimated by multiplying cost per laboratory by the total number of commercial- and laboratory-based sponsors.

Data Analysis and Statistical Analysis

The number of assay sponsors were extracted from the FDA's website (https://www.fda.gov/medical-devices/coronavirus-covid-19-and-medical-devices/sars-cov-2-reference-panel-comparative-data#table1, last accessed June 19, 2021). For each sponsor, laboratory versus manufacturer were assigned and initial and updated reference panel limit of detection value was noted. Specifically, the initial LOD values were extracted from the instructions for use documents accompanying each EUA (FDA, https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/in-vitro-diagnostics-euas-molecular-diagnostic-tests-sars-cov-2#individual-molecular, last accessed June 19, 2021). For statistical comparison of LOD values, Wilcoxon matched-pairs signed rank tests and U-tests were used and P < 0.05 was considered to be statistically significant.

Results

Timeline and Regulatory Deviations

The LDT and COVID-19 timelines were aligned in March 2020 when two proposed legislations coincided with the World Health Organization declaring a global pandemic (Figure 1). Despite seemingly opposing approaches (Figure 1), both acts, akin to the FDA oversight of COVID-19 tests, have one key intent: to ensure the availability of safe and accurate diagnostic tests with appropriate regulatory oversight. The anchor point of the timeline is positioned between February 4, 2020, when the CDC received the first EUA and subsequently warned laboratories about testing without EUA, and the US Department of Health and Human Services notice from August 19, 2020, clarifying that LDTs can be offered without EUA (Figure 1). The second notable timespan was between May 20 and October 9, 2020, representing the initial phase of the labeling update study (Figure 1). Depicting these regulatory deviations on top of the interactions between FDA, manufacturer, and laboratory shows that the EUA process temporarily bypassed the enforcement discretion of LDTs (Figure 2), whereas the labeling update study added a performance assessment to manufacturers and laboratories using LDTs (Figure 2).

Figure 1.

Selected events of the COVID-19 pandemic (top row) aligned with milestones of laboratory-developed test (LDT) regulations (bottom row). Regulatory events during the COVID-19 pandemic entail regulatory oversight over LDTs (yellow) and the labeling update study (turquoise). The official declaration of a global pandemic in March 2020 coincided with the introduction of two competing legislations: the Verifying Accurate Leading-Edge IVCT (in vitro clinical test) Development Act of 2020 (or VALID Act of 2020) is a 245-page, bipartisan legislation that aims to clarify the US Food and Drug Administration’s (FDA's) authority to regulate LDTs; the second, the Verified Innovative Testing in American Laboratories Act of 2020 (or VITAL Act of 2020) is a seven-page rebuttal that proposes updating the existing Clinical Laboratory Improvement Amendments (CLIA) regulations via the Clinical Laboratory Improvement Advisory Committee (CLIAC) with the primary aim of eliminating undue regulation that leads to delays in patient access. CMS, Centers for Medicare & Medicaid Services; DHHS, US Department of Health and Human Services; EUA, Emergency Use Authorization; Lab., laboratories; PHE, public health emergency; WHO, World Health Organization; w/o, without.

Figure 2.

Temporary regulatory deviations as part of the COVID-19 response. Deviation from enforcement discretion for laboratory-developed test (LDTs; yellow) and the SARS-CoV-2 labeling update study (turquoise) are portrayed atop of regulatory interactions between the US Food and Drug Administration (FDA), manufacturer, and laboratories, with ultimate responsibility for implementation still residing with the laboratory director (Lab.Dir.). Note: the labeling update study did not assess performance of FDA-approved in vitro diagnostic assays when implemented in clinical laboratories, depicted in faded gray to the right. CLIA, Clinical Laboratory Improvement Amendments.

Initial EUA Review Time

To undergo EUA entailed a set of validation experiments. The timeline of 14 initial EUA tests was examined from submission to authorization (Supplemental Table S2 and Supplemental Figure S1). The data indicate that EUA review took on average 17 ± 4 days (range, 4 to 56 days). Of note, the FDA requested EUA submission within 14 days of initiating clinical testing and allowed continued testing during the review. Effectively, the regulatory deviation was that the FDA served as an independent reviewer of prespecified validation documents (Figure 2). In isolation, the quality impact of this review remains unknown; however, in conjunction with the labeling update study, meaningful insights can be gained.

Labeling Update Study

The FDA sent out reference materials along with a detailed protocol (Supplemental Appendix S1). The first experiment required pooling of clinically negative samples for subsequent experiments (n = 20 tests). The second LOD experiment is a 10-fold dilution series of reference material T1 in a total of eight dilution steps, extracted and assessed in triplicate (n = 24 tests). The third experiment consisted of an LOD/sensitivity confirmation, assessing n = 20 replicates each of the previously determined LOD and concentrations threefold above and below the LOD (n = 60 tests). The fourth experiment consisted of testing six distinct concentrations using reference materials T2 through T5 in n = 5 replicates (n = 30 reactions). Simply put, the FDA asked each sponsor to perform a total of n = 134 SARS-CoV-2 assays (Figure 3A), or 402 individual PCRs (eg, when targeting viral N1, N2, and human RNAseP in three separate reactions).

Figure 3.

Overview of the SARS-CoV-2 labeling update study. A: The US Food and Drug Administration (FDA) sent reference material (T_1,2,3,4,5) to commercial- and laboratory-based molecular test sponsors. Each sponsor was asked to perform the following verification study: confirm 20 previously diagnosed SARS-CoV-2–negative specimens in the approved transport medium; then, pool the confirmed negative samples and perform spike-in experiments; using the reference material (T₁), determine assay-specific limit of detection (24 samples) and verify the limit of detection (60 samples); finally, assess the unknown inactivated reference materials (T_2-5) as a proficiency test (30 samples). B: Estimates for consumables for the entire study. C: Timeline of labeling update study from our own laboratory, from shipment of reference material to final submission to the FDA. D: Analysis of publicly available data from the FDA, delineating the status of the labeling update study. E: Assessment of the reported limit of detection from the 126 test sponsors who submitted data to the FDA; data sets were compared using Wilcoxon matched-pairs signed rank tests and two-tailed U-tests. F: Total number of SARS-CoV-2 assays that received Emergency Use Authorization (EUA), separated by manufacturer (blue) and laboratory-developed test (LDT, red). The dashed line marks the time point when the US Department of Health and Human Services (DHHS) clarified that EUA is not mandatory (Figure 1). ∗P < 0.05, ∗∗P < 0.01, and ∗∗∗∗P < 0.0001. CLIA, Clinical Laboratory Improvement Amendments; LOD, limit of detection; MERS, Middle East respiratory syndrome (a control); SARS, severe acute respiratory syndrome (herein, SARS-CoV-2).

Labeling Update Study Estimates

At the time of analysis, the FDA EUA website listed 140 commercial and 35 laboratory-based sponsors. Assuming 134 samples per protocol across 175 sponsors, the total consumption was estimated to 23,450 samples, 70,350 PCRs, 351.75 mL of PCR master mix, and 105.5 mL of primer/probe mix (Figure 3B). The hands-on workload in our laboratory was tracked at approximately 14 hours of technologist time and required, in our setting, overtime and weekend shifts (Figure 3C). This amounts to at least 2450 hours of labor or 1.2 full-time equivalents. Furthermore, assuming an assay price of approximately $58.36 in US dollars (USD), the requested experiments will cost each laboratory up to approximately $7800 USD in direct costs. Using a cost-reduction factor of 4.3 (eg, due to automation and/or pooling; $13.57 USD), the total direct cost for each laboratory was estimated at approximately $1800 to $7800 USD or $0.3 to $1.36 million in USD total.

Labeling Update Study Participation

Of the 195 total assays that received EUA, n = 17 sponsors (9%) did not participate and n = 26 sponsors (13%) did not return data. Of the remaining n = 152 sponsors (78%) that submitted data, n = 21 sponsors (14%) were undergoing interactive review, and n = 5 sponsors (3%) submitted uninterpretable data (Figure 3D). Data from the remaining n = 126 sponsors (83%) were used in the LOD results comparison (Figure 3E).

Labeling Update Study LOD Results and Comparison

Comparison of results using the FDA reference panel showed LODs ranging from 180 to 600,000 nucleic acid amplification tests (NAATs)/mL (Supplemental Table S3). The results were separated by manufacturers (n = 100) versus laboratories (n = 18). LODs obtained using the FDA reference materials were significantly higher in LDTs when compared with those obtained by manufacturers [LDT: 71,216 ± 147,134 NAATs/mL (median, 5400 NAATs/mL) versus 32,229 ± 75,060 NAATs/mL (median, 18,000 NAATs/mL); P = 0.039, U-test]. To examine this difference, the LODs originally submitted by each sponsor were also pulled [manufacturers versus laboratories, 3688 ± 11,178 (median, 1000) versus 4590 ± 5617 (median, 2300); P = 0.72, U-test]. LODs obtained using the FDA reference panel were significantly higher in both groups, manufacturers (P < 0.0001, Wilcoxon test) and laboratories (P = 0.0003, Wilcoxon test) (Supplemental Table S3).

EUA Adoption Pattern

The FDA continues to offer EUA review of LDTs. Assessment of the cumulative number of SARS-CoV-2 PCR assays that have received EUA shows that the number of authorized laboratories (ie, LDTs) and manufacturers ascended in parallel in the early phase of the pandemic. As manufacturers were catching up with test development, the number of additional EUA submissions by laboratories decreased (Figure 3F). A complete breakdown of the EUA timelines of laboratory sponsors (Supplemental Table S4) and commercial sponsors (Supplemental Table S5) is provided in the supplement. Briefly, manufacturers achieved 209 new EUAs after the US Department of Health and Human Services notice, whereas only 14 additional laboratories achieved EUA.

Discussion

Herein, we reviewed the timelines and examined specific regulatory deviations that occurred as a response to the COVID-19 pandemic in the United States. The specific changes entailed temporary discontinuation of the FDA's enforcement discretion of LDTs and a subsequent labeling update study using agency-verified reference materials (Figure 2). The initial EUA adoption pattern by LDT developers changed after the submission requirement was rescinded (Figure 3F). Commercial assays had, on average, higher sensitivities (Figure 3E); however, the labeling update data also indicate that FDA review did not prevent significant differences between initial and reference material derived LODs for both manufacturers and LDT developers (Figure 3E). The temporary regulatory paradigm shifts have generated valuable data that, when appropriately contextualized, can serve as real-world evidence to inform pending legislation.

The topic of regulating in-house diagnostic procedures has been controversial for a considerable amount of time (Figure 1). The framework established by CLIA of 1988, over which the FDA exercised enforcement discretion, was revolutionary at the time (Figures 1 and 2); however, concerns are mounting (Wayback Machine – Internet Archive, FDA, http://wayback.archive-it.org/7993/20171115144712/https://www.fda.gov/downloads/aboutfda/reportsmanualsforms/reports/ucm472777.pdf, last accessed June 19, 2021) and revisions are imminent (https://www.congress.gov/bill/116th-congress/senate-bill/3512; https://www.congress.gov/bill/117th-congress/senate-bill/1666/amendments, last accessed June 19, 2021).⁴⁰ For example, during the pandemic, members of Congress expressed concerns that “unregulated tests are flooding the market” (College of American Pathologists, https://www.cap.org/advocacy/latest-news-and-practice-data/august-25-2020, last accessed June 19, 2021). The concerns are, however, not restricted to the pandemic^9,13,37,43 or to the United States.44, 45, 46, 47, 48 In the European Union, a new regulation [termed IVD regulation (IVDR)] is going into full effect in May 2022.^46,47 The IVDR rules will affect 447 million individuals in 27 countries and have been called “the end of the laboratory developed test as we know it”⁴⁸ [IVDR(d)Art.5(5)]. In the United States, the VITAL Act recommends updating section 353 of the Public Health Service Act (42USC263a) “to reflect the current state of the field of clinical laboratory testing” (https://www.congress.gov/bill/116th-congress/senate-bill/3512, last accessed June 19, 2021; Association for Molecular Pathology, https://www.amp.org/advocacy/laboratory-developed-testing-procedures-ldps1/#clia, last accessed June 19, 2021). However, how to accomplish a meaningful legislative update in the absence of hard scientific data remains to be determined.

One commonly expressed opinion, evidenced by legislative proposals to increase oversight by the FDA (ie, VALID Act), is that FDA-reviewed tests have a higher diagnostic quality than LDTs. This notion requires careful contextualization. For this discussion, we propose a conceptual framework that distinguishes diagnostic tests, from diagnostic procedures, and diagnostic services (Figure 4A). In this framework, and in clinical practice, LDTs, modified IVDs, and unmodified IVDs may coexist in one laboratory and require operational integration and maintenance as diagnostic procedures. The performing CLIA personnel and the diagnostic procedures are carefully monitored using legally required competency assessment and proficiency testing, respectively.^{14,15,19,20,31,34} These diagnostic procedures interface with a health care delivery system to form diagnostic services to establish and maintain best clinical practices (Figure 4A). We acknowledge that diagnostic tests, procedures, and services (Figure 4A) cannot capture the maze of diagnostic test implementation (Figure 4B); however, the framework is appropriate to discuss regulatory changes.

Figure 4.

Conceptual diagnostic framework and test delivery maze. A: The diagnostic test, US Food and Drug Administration (FDA)–authorized in vitro diagnostic (IVD) assay or self-developed laboratory-developed test (LDT), lives in the clinical testing environment [Clinical Laboratory Improvement Amendments (CLIA) –certified laboratory] where personnel apply their skill, knowledge, and experience to perform diagnostic procedures. Competency (assessment): six specific assessment procedures are the minimal regulatory requirement for competency for all personnel performing laboratory testing [42 CFR 493.1451(b)(8), Legal Information Institute, https://www.law.cornell.edu/cfr/text/42/493.1451, last accessed July 6, 2021]. The team is depicted using exemplary professional degrees and years of experience after training of the least experienced team member serving in each role. Proficiency (testing): testing of unknown samples (eg, sent to a laboratory by a US Department of Health and Human Services (DHHS)–approved testing program) and submission of laboratory results to the administrator of the program. Proficiency testing is a legal requirement for each nonwaived test and CLIA certificate (42 CFR subchapter I). Proficiency testing is a tool to verify the accuracy and reliability of the entire testing process, including competency of the testing personnel. When an FDA-authorized IVD assay is implemented with a different (off-label) pre-analytic or postanalytic process, the entire diagnostic procedure requires establishing the performance (ie, validation), and the diagnostic test is considered an LDT. A laboratory typically has numerous diagnostic procedures (eg, using nasopharyngeal swab, anterior nasal swab, or saliva) that can be coupled to the same or different diagnostic tests (eg, LDT or IVD assay). The diagnostic procedures interface with a health care delivery system to form diagnostic services to establish and maintain best practices (including order entry and information about eligible sample types, service contracts, International Classification of Diseases, and Current Procedural Terminology coding processes, technical and professional billing, and payor operations). For any patient, the diagnostic quality relies on attributes of the diagnostic test, diagnostic procedure, and diagnostic service. B: To implement diagnostic tests (12 o'clock) for patient care (center court of the maze), laboratory directors must navigate through a maze of technical, professional, regulatory, and administrative hurdles. Depending on the diagnostic test and setting, the pathway will differ. The perimeter of the maze is formed by governing concepts, and the maze lists some of the relevant organizations. As depicted, the maze is solvable; however, when attempting to account for all relevant governing aspects of a specific diagnostic test, redundancy will be encountered. Redundancy can be regarded as an added benefit (eg, extra strength in case of failure) or as duplicative waste when not strictly necessary for functioning (eg, perceived as regulatory misalignment). AABB, American Association of Blood Banks; AAHHS, Accreditation Association for Hospital/Health Systems; A2LA, American Association for Laboratory Accreditation; ABMG, American Board of Medical Genetics; ABP, American Board of Pathology; ACLA, American Clinical Laboratory Association; ADASP, Association of Directors of Anatomic and Surgical Pathology; AMA, American Medical Association; AMP, Association of Molecular Pathology; AOA, American Osteopathic Association/Healthcare Facilities Accreditation Program (AOA/HFAP); API, Association of Pathology Informatics; APC, Association of Pathology Chairs; ASCP, American Society for Clinical Pathology; ASH, American Society of Hematology; ASHI, American Safety and Health Institute; DPA, Digital Pathology Association; CAP, College of American Pathologists; CMS, Centers for Medicare & Medicaid Services; CLFS, Clinical Laboratory Fee Schedule; COLA, Commission on Office Laboratory Accreditation; FDA, Food and Drug Administration; GDPR, General Data Protection Regulation; HFAP, Healthcare Facilities Accreditation Program; HIPAA, Health Insurance Potability and Accountability Act; IAP, International Academy of Pathology; IARC, International Agency for Research on Cancer; ICD-O, International Classification of Diseases for Oncology; IC3R, International Collaboration for Cancer Classification and Research; ICD-11, International Classification of Diseases 11th Revision; IFCC, International Federation of Clinical Chemistry and Laboratory Medicine; JCAH/JCAHO, Joint Commission on Accreditation of Hospitals/Joint Commission on Accreditation of Healthcare Organizations; MAC, Medicare Administrative Contractor; NYS/DH, New York State Department of Health; HCPCS, Healthcare Common Procedure Coding System; PFS, Physician Fee Schedule; PIcc, Pathology Innovation Collaborative Community; USCAP, United States and Canadian Academy of Pathology; VALID, Verifying Accurate Leading-edge IVCT Development Act of 2021; WHO, World Health Organization.

The main lessons learned from the temporary COVID-19 regulatory changes are as follows: i) The adoption pattern (Figure 3F) and initial review times (Supplemental Table S3) provide hard data that FDA oversight of LDTs, albeit temporary, is possible. ii) The temporary EUA oversight of LDTs provided clinical laboratory directors insight into the agencies' review process and demystified FDA authorization. For example, the fact that the requested data for both EUA and the labeling update study are the same as any credible laboratory would do as part of their own in-house validation. iii) The labeling update study represented a post hoc assessment of diagnostic procedures that allows concrete and publicly available comparisons using regulatory-grade data (Figure 3E). iv) The data of the labeling update study emphasize the value of agency-verified reference materials; this lesson goes beyond inactivated viral reagents and highlights the practical value of these regulatory science tools to establish and assess diagnostic performance metrics. Of note, the FDA has recently implemented an entire program dedicated to Medical Device Developmental Tools (FDA, https://www.fda.gov/medical-devices/science-and-research-medical-devices/medical-device-development-tools-mddt, last accessed June 19, 2021), although it remains to be determined whether these tools will become an integral part of future legislation. Nonetheless, the above lessons underline that the COVID-19 pandemic has accelerated regulatory developments.

There are also three additional, indirect lessons. First, future legislation should emphasize the laboratory context rather than adopt a narrow focus (ie, review of the analytical performance). For example, stringent regulation of the diagnostic test without ensuring balanced improvements to diagnostic procedures is the same as a highly accurate test without competent personnel. Simply put, balanced legislative approaches are key. Second, the FDA review governs a framework established by the Federal Food, Drug, and Cosmetic Act (21CFR 860) that establishes intended use, indication for use, and a risk classification. In clinical practice, however, the diagnostic procedure might be applied outside the stringent regulatory scope, which from a performance assessment perspective turns an IVD into an LDT. Indeed, LDT development comes with increased efforts of establishing performance; however, if an IVD is modified (eg, the laboratory does not own a specific piece of equipment specified in the label), the laboratory must establish performance akin to an LDT. These decisions fall back onto the laboratory directors, who are ultimately responsible for all aspects of the laboratory, irrespective of whether the test is an IVD or an LDT. During the pandemic, some laboratories developed their tests (Supplemental Table S3), whereas other laboratories chose commercial solutions (Figure 3F).^1,3,49,50 In other words, diagnostic quality is not only a function of IVD or LDT but clearly a function of the professional competence and available resources in the laboratory. Third, LDTs should be considered laboratory diagnostic procedures or even “professional medical services that utilize a laboratory examination in the context of clinical care” (https://www.congress.gov/bill/117th-congress/senate-bill/1666/text, last accessed June 19, 2021). The VALID Act is proposing a framework for procedural review (ie, so-called technology certification), and, as demonstrated by the FDA labeling update study (Figure 3A), procedural review (with carefully constructed protocols and reference materials) is possible and a move in the right direction. In combination, these indirect regulatory lessons emphasize the practical realization that the benevolent intent to increase diagnostic quality via legislation cannot be accomplished without taking the contextual laboratory framework into account.

Several limitations apply. We do not know whether the regulatory measures enacted by the FDA during the pandemic achieved the goals of ensuring safety and efficacy. There is good evidence^4,19,29; however, extrapolating from COVID-19 to other molecular diagnostic tests is an oversimplification.^{24,28,39,42,48,51} Our national estimate for the cost of the labeling update study (<$1.4 million USD) is, despite being efficiency corrected, likely inaccurate because of the various procedural differences in each laboratory. Similarly, cost estimation of revised legislation is difficult.⁵² Furthermore, should we consider the labeling update study redundant with proficiency testing mandates? The answer is tricky. On the one hand, the protocol clearly resembles proficiency testing (Figure 3A) under Centers for Medicare & Medicaid Services [42 U.S.C. § 263a(f)(3)]. On the other hand, the FDA study is distinct because there are, with few exceptions,⁵³ generally no FDA-certified reference materials and no centralization of data across the entities administering proficiency testing; and performance of specific laboratories is typically not made publicly available. In other words, the FDA has the appropriate standing to obtain such comprehensive comparison data, and we acknowledge the tremendous efforts in organizing what can be considered the largest diagnostic test performance comparison study during a pandemic.

That said, we caution that the findings of lower LODs should not be equated with better tests; highly sensitive assays are error prone, especially when many samples must be processed or if the operators lack sufficient experience.^54,55 Furthermore, we still do not know the clinically relevant level of the virus, and sampling quality is hard to standardize.⁵⁶ In fact, a highly sensitive assay may remain positive despite the patient being noncontagious, which may result in increased lengths of hospital stays (analytical validity versus clinical utility).^49,54,55,57

Probably the most important limitation is the lack of comparison data on FDA-authorized tests in the CLIA setting (Figure 2). The labeling update study did not focus on how FDA-approved test performance differs when performed in clinical practice rather than by the manufacturer (Figure 3E). Although this was not the aim, broadly speaking, FDA review of medical devices can only be a baseline assessment aiming to ensure safety and effectiveness through regulatory review. Whether the tests that underwent review achieve these goals when implemented clinically cannot be answered with the current data. This specific limitation can also be viewed as the identification of an important evidence gap that can be addressed in carefully designed follow-up studies. There are other COVID-19–related temporary regulatory deviations that enabled, for example, adoption of digital pathology,^3,51,58, 59, 60 and even a federal notice proposing to make all regulatory changes permanent (86 FR 4088); although the outcome of the former deviations is still pending, the latter was withdrawn (86 FR 20174).

Despite rivaling legislative proposals, we approached the topic of LDT regulation as neutrally and evidence based as possible. There is no rivalry except the intrinsic complexity of regulating LDTs. Albeit imperfect, we believe the real-world data generated during the pandemic have generated logically sound and meaningful insights. Regulators are actively soliciting input via improved communication and a network of expert programs (FDA, https://www.fda.gov/medical-devices/digital-health-center-excellence/network-digital-health-experts and https://www.fda.gov/about-fda/cdrh-strategic-priorities-and-updates/collaborative-communities-addressing-health-care-challenges-together, last accessed June 19, 2021). These communication channels provide a concrete opportunity to share data and provide direct input to prevent unrealistic guidance that would result in undue burden or failure to benefit patients. We are collectively tasked to generate scientific evidence to align regulatory intent with the clinical complexities of ensuring benefit to patients. Facilitating innovation while codifying regulatory oversight to maintain safety for patients is no simple task,²⁹ during a pandemic or otherwise. However, as outlined herein, the regulatory paradigm shifts have led to the generation of regulatory-grade data that should be used to inform legislative decision making. Our collective response to the challenges of the pandemic has thereby concretely accelerated regulatory developments.

Supplemental Data

Supplemental Figure S1.

Time to Emergency Use Authorization for selected laboratories and assay manufacturers (from initial submission to US Food and Drug Administration authorization).