Nationwide external quality assessment of SARS-CoV-2 nucleic acid amplification tests in Japan

Abstract

Objectives

We conducted a nationwide external quality assessment (EQA) study of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) nucleic acid amplification testing in Japan.

Methods

A total of 563 public health and private sector laboratories participated. The EQA samples comprised 6 RNA and full-process controls.

Results

The overall agreements were 99.3% and 97.9% for the RNA and full-process controls, respectively. A total of 530/563 (94.1%) laboratories reported correct results; public health laboratories had the highest accuracy. Thirty-three laboratories reported at least one incorrect result (26 laboratories of medical facilities, 5 commercial laboratories, 1 public health laboratory, and 1 other). Sixteen laboratories of medical facilities that used a fully automated assay system failed to detect the presence of the full-process control, due to inherent insufficiency in the limit of detection (LOD). Other causes of incorrect results included failure to ensure the LOD (n = 13), error in result judging or reporting (n = 3), and error in sample handling (n = 1).

Conclusions

Performance was mostly dependent on the laboratory category and assay evaluation, particularly the LOD. Guidance should be developed based on these results, particularly in the phase of new entry into laboratory services for SARS-CoV-2.

Introduction

In the ongoing outbreak of the coronavirus disease 2019 (COVID-19), access to reliable diagnostic assays and adequate testing capacity for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is essential for preparedness and response strategies worldwide (World Health Organization 2021b). In Japan, the standard method for the detection of SARS-CoV-2 RNA has been an in-house polymerase chain reaction (PCR) method developed by the National Institute of Infectious Diseases (NIID) (National Institute of Infectious Diseases 2020). However, governmental and public health laboratories have a limited ability to perform the volume of tests that is needed. As a result, diagnosis and isolation of patients, contact tracing, and even treatment of symptomatic patients may not be performed adequately. Therefore, rapid expansion of diagnostic testing for emerging pathogens into private sector laboratories has been demanded, which allows an increase in test volume and shorter turnaround times (Corman et al. 2020; Matheeussen et al. 2020b).

The government has promoted the use of nucleic acid amplification tests (NAATs), such as PCR, by implementing new measures such as emergency approval of in vitro diagnostic devices (IVDs) and coverage by national health insurance. A number of assay methods are covered, provided that the assay performance is comparable to that of the standard method, regardless of whether the test is an IVD or a laboratory-developed test (LDT). However, NAATs are used in various facilities, with a number of new entries into laboratory services, and the quality and diagnostic performance of these tests has not been adequately validated. Further, the nucleic acid extraction methods, NAAT reagents, and thermocyclers used differ among laboratories.

External quality assessment (EQA) is a fundamental element to ensure reliable test results, especially when using Emergency Use Authorization diagnostic kits for newly emerging pathogens (Matheeussen et al. 2020a; Sung et al. 2020; Wang et al. 2020; Fischer et al. 2021; US Food and Drug Administration 2021). The World Health Organization (WHO) has encouraged laboratories to participate in EQA schemes for this novel virus (World Health Organization 2021a). Here, we present the results of the first nationwide EQA on SARS-CoV-2 NAATs, which was a commissioned project by the Ministry of Health, Labour and Welfare of Japan. We also present data from a questionnaire survey that assessed the molecular testing capability of clinical laboratories conducting SARS-CoV-2 NAATs throughout Japan.

Methods

Organization and participants

This nationwide EQA was conducted as a commissioned project by the Ministry of Health, Labour and Welfare of Japan (National investigation for quality control of SARS-CoV-2 nucleic acid amplification tests). The Japanese Committee for Clinical Laboratory Standards was responsible for the EQA scheme plan and evaluation of the EQA results. All public health laboratories conducting SARS-CoV-2 tests were requested to participate, and these laboratories were included in the study by default. Private sector laboratories conducting SARS-CoV-2 tests were also recruited as participants.

A total of 128 public laboratories performing SARS-CoV-2 tests participated, including 65 regional health and environment institutes, 14 national quarantine stations, and 49 public health stations, along with 310 laboratories in medical facilities, 107 commercial laboratories (83 registered and 24 temporary), and 18 others, for example, research laboratories. EQA samples were shipped on November 15 and 16, 2020. All participants were asked to submit all data, including test results and other information, by November 21, 2020.

EQA panel and SARS-CoV-2 testing protocols

A positive RNA control and a pseudovirus-containing positive RNA full-process control were used as EQA samples to evaluate the process of amplification and detection and the entire process of extraction, amplification, and detection, respectively. The AcroMetrix^TM Coronavirus 2019 (COVID-19) RNA Control (Thermo Fisher Scientific Inc., Fremont, CA, USA) was used as a positive RNA control sample. It was prepared by formulating genomic RNA from SARS-CoV-2 into a proprietary buffer and quantified using Bio-Rad (Hercules, CA, USA) Droplet Digital^TM PCR (ddPCR). The AccuPlex SARS-CoV-2 Verification Panel (SeraCare LifeSciences Inc., Milford, MA, USA) was used as a positive full-process control sample. Non-replicative encapsulated recombinant viruses were used to evaluate test proficiency and accuracy through the entire process, including extraction and amplification, and viral RNA was quantified using ddPCR. The viruses were formulated in viral transport media that consisted of Tris-buffered saline with glycerol, antimicrobial agents, and human proteins. This material must undergo extraction, similar to that of the patient sample.

In order to ensure the quality of EQA samples, all samples were handled as specified by the manufacturers, which were certified under ISO 13485 (Medical devices — Quality management systems — Requirements for regulatory purposes). All samples were stored at −70°C until shipment. To validate the stability of EQA samples, one random panel was tested for confirmation of the property as specified by the manufacturers in the EQA round, i.e., 2 days after thawing (anticipated maximum shipping time) and after further storage at 4°C for 3 days (maximum expected storage time before testing at participating laboratories). To validate the homogeneity of EQA samples, three random panels were tested for confirmation of the property without significant differences among the first, the middle and last vials dispensed prior to distribution.

The EQA panel was composed of six samples. SARS-CoV-2 RNA control samples were diluted with a proprietary buffer to concentrations of 20 copies/μL (no. 1) and 10 copies/μL (no. 2). SARS-CoV-2 full-process control samples were diluted with a proprietary diluent to concentrations of 10 copies/μL (no. 4) and 20 copies/μL (no. 5). Nuclease-free water was used as a negative control (no. 3 and no. 6).

Protocols for SARS-CoV-2 NAATs varied among the participating laboratories regarding the assay components, including the RNA extraction platform, assay reagent, and instrument for amplification and detection. Participants were asked to choose one method used as a mainstay of routine tests in the laboratory and to test the material as they would clinical samples using their routine molecular assay workflows, with a minor modification for the EQA.

To ensure consistency of the EQA samples across this variety of combinations of assay components, three core protocols were prepared based on RNA extraction platforms, as follows. For column-based RNA extraction followed by amplification (i.e., PCR and Loop-Mediated Isothermal Amplification, LAMP), a set of SARS-CoV-2 RNA controls at total loads of 100 copies (20 copies/μL) and 50 copies (10 copies/μL) and one negative sample were input to evaluate the amplification process, and a set of SARS-CoV-2 full-process control samples at a total load of 2800 copies (20 copies/μL) and one negative sample were used to evaluate the entire process. For so-called direct PCR without extraction and purification followed by PCR amplification, the set of RNA controls and the set of full-process controls at a total load of 50 copies (10 copies/μL) each were input. For fully automated assay systems, SARS-CoV-2 full-process control samples at a total load of 2800 copies (20 copies/μL) with one negative sample were input. For SmartGene (MIZUHOMEDY Co. Ltd., Tosu, Japan), μTAS WAKO COVID-19 (FUJIFILM Wako Chemicals Co., Osaka, Japan), Aptima SARS-CoV-2 (Hologic, Inc., Tokyo, Japan), and TRCReady® SARS-CoV-2 (Tosoh Co., Tokyo, Japan) systems, a total load of 2800 copies was directly input, since the extraction efficiency of these assays was not disclosed at that time.

Distribution of EQA materials and collection of EQA results

Sysmex Corp. (Kobe, Hyogo, Japan), which is accredited by the Japan Accreditation Board as a proficiency testing provider under ISO/IEC 17043, was responsible for the preparation and transport of the EQA material and aggregation of the EQA results. EQA samples were shipped on dry ice with temperature monitoring by Timestrip UK Ltd (Cambridge, UK). Delivery to all laboratories was completed within 12-36 h. Upon receipt, the samples were stored at 2-8°C until use within 7 days, according to the manufacturer's instructions. After thawing, refreezing was not allowed.

Participants were asked to submit their results and the workflow, including those of the extraction/amplification reagents and instruments, through the EQA program website (https://crm.fjcl.fujitsu.com/certification/login_ini.do; CRMate, Fujitsu Ltd., Tokyo, Japan). In addition, information on routine quality management in the laboratory was also collected, including laboratory accreditation, qualification of personnel, standard operating procedures, evaluation of performance parameters at implementation, and routine quality control.

Evaluating the EQA results and statistical analysis

For qualitative evaluations, EQA participants were only asked to report whether the samples were SARS-CoV-2-positive or not. Correct results reported for each sample were evaluated for each peer group and laboratory category. A peer group was defined as over 10 laboratories using the same combination of assay kit and instrument. The laboratory categories performing SARS-CoV-2 tests were classified into public laboratories (regional institutes of health and environment, national quarantine stations, and public health stations), laboratories in medical facilities, commercial laboratories (registered and temporary), and others, for example, research laboratories.

Statistical analyses were conducted to examine whether the overall EQA performance correlated with the specific technical details provided by the EQA participants. Statistical analyses were performed by means of chi-square test using a standard software package (StatFlex, version 7.0; ARTEC Co. Ltd. (Osaka, Japan). When needed, the information about workflows and protocols provided by the EQA participants or manufacturers was reviewed to be conclusive regarding applied kits or tests. If needed, participants were contacted to confirm or specify the provided data and information. However, the reported information was incomplete for all the participants. Therefore, the utilized datasets differed slightly in size among the statistical analyses of the different variables.

Results

Participating laboratories

The extraction platforms in routine use included direct PCR (183, 32.5%), column-based (138, 24.5%), proprietary systems (120, 21.3%), simple direct for LAMP (54, 9.6%), magnetic bead-based (53, 9.4%), isopropyl alcohol-based (7, 1.2%), and unknown (7, 1.2%).

Assay reagents and instruments with regulatory approval were used in 304 (54.0%) and 300 (53.3%) laboratories, respectively. The rates of validation and verification of the assays performed at implementation were 56.0% (145/259) and 48.0% (146/304), respectively. The limit of detection (LOD) was evaluated in some regional institutes of health and environment (72.3%, 47/65), national quarantine stations (14.3%, 2/14), local public health stations (49.0%, 24/49), medical facilities (20.3%, 63/310), commercial laboratories (27.1%, 29/107), and others (44.4%, 8/18). The LOD was evaluated significantly more frequently in public health laboratories (57.0%, 73/128) than in private sector laboratories (23.0%, 100/435) (p < 0.05).

Qualitative reverse transcription polymerase chain reaction (RT-qPCR) results

The overall accuracies were 1243/1252 (99.3%) for the RNA control and 1168/1193 (98.0%) for the full-process control. The positive agreement in each sample ranged from 96.4% to 99.8%, and the negative agreement ranged from 99.4% to 99.8% (Table 1 ). A total of 530/563 (94.1%) laboratories reported correct results for all qualitative tests; the proportion was the highest in public health laboratories (99.2%, 127/128), followed by commercial laboratories (95.3%, 102/107), others (94.4%, 17/18), and laboratories of medical facilities (91.6%, 284/310) (Table 2 ).

Table 1.

EQA Panel and Reported Results

	RNA control			Full-process control
EQA Samples	Sample 120 cps/μL (100 cps /assay)	Sample 210 cps/μL (50 cps/ assay)	Sample 3Negative	Sample 410 cps/μL (50 cps/assay)	Sample 520 cps/μL (100 cps/assay)	Sample 6Negative
Responses (n)	417	418	417	192	503	498
Negative results (n)	1	7	416	4	18 (2)	495
Negative results (%)	0.2	1.7	99.8	2.1	3.6(0.4)	99.4
Positive results (n)	416	411	1	188	485	3
Positive results (%)	99.8	98.3	0.2	97.9	96.4	0.6
Correct results (%)	99.8	98.3	99.8	97.9	96.4	99.4

EQA: external quality assessment; cps: copies.

Table 2.

EQA Results in the Laboratory Groups

EQA Samples		RNA control and Reported Results												Full-process control and Reported Results
		Sample 120 cps/μL (100 cps/assay)				Sample 210 cps/μL (50 cps/assay)				Sample 3 Negative				Sample 410 cps/μL (50 cps/assay)				Sample 520 cps/μL (100 cps/assay)				Sample 6 Negative
		Response (n)	Correct (%)	Neg (n)	Pos (n)	Response (n)	Correct (%)	Neg (n)	Pos (n)	Response (n)	Correct (%)	Neg (n)	Pos (n)	Response (n)	Correct (%)	Neg (n)	Pos (n)	Response (n)	Correct (%)	Neg (n)	Pos (n)	Response (n)	Correct (%)	Neg (n)	Pos (n)
Public health sector laboratories (n=128)	Institutes of health and environment	61	100	0	61	62	100	0	62	62	100	62	0	17	100	0	17	65	100	0	65	64	98.4	63	1
	Quarantine stations	14	100	0	14	14	100	0	14	14	100	14	0	6	100	0	6	14	100	0	14	14	100	14	0
	Public health stations	49	100	0	49	49	100	0	49	49	100	49	0	19	100	0	19	49	100	0	49	49	100	49	0
Private sector laboratories (n=435)	Medical facility	193	99.5	1	192	193	97.9	4	189	193	100	193	0	87	97.7	2	85	258	93.8	16	242	254	99.6	253	1
	Commercial (Total)	87	100	0	87	87	97.7	2	85	86	100	21	0	56	96.3	2	54	100	100	0	100	100	99	99	1
	Commercial (registered)	66	100	0	66	66	98.5	1	65	65	100	65	0	41	97.6	1	40	78	100	0	78	78	98.7	77	1
	Commercial (temporary)	21	100	0	21	21	9	1	20	21	100	21	0	15	93.3	1	14	22	100	0	22	22	100	22	0
	Others	13	100	0	13	13	92.3	1	12	13	92.3	12	1	7	100	0	7	17	88.2	2	15	17	100	17	0

EQA: external quality assessment; cps: copies.

There were a total of 18 peer groups with over 10 laboratories using the same combination of assay kit and instrument (Table 3 ).

Table 3.

EQA results in the peer groups

		RNA Control and Reported Results									Full-process Control and Reported Results
	EQA Samples	Sample 120 cps/μL(100 cps/assay)			Sample 210 cps/μL(50 cps/assay)			Sample 3Negative			Sample 410 cps/μL(50 cps/assay)			Sample 520 cps/μL(100 cps/assay)			Sample 6Negative
Instrument	Reagent	Correct (%)	Neg(n)	Pos (n)	Correct (%)	Neg(n)	Pos (n)	Correct (%)	Neg(n)	Pos (n)	Correct (%)	Neg(n)	Pos (n)	Correct (%)	Neg(n)	Pos (n)	Correct (%)	Neg(n)	Pos (n)
ABI7500/7900/ViiA	NIID	100	0	19	100	0	19	100	19	0	100	0	1	100	0	20	100	19	0
BioRad CFX96	Ampdirect 2019-nCoV detection kit	100	0	25	100	0	25	100	24	0	100	0	24	100	0	24	100	24	0
cobas z 480/4800	Ampdirect 2019-nCoV detection kit	100	0	17	100	0	17	100	17	0	100	0	17	100	0	17	100	17	0
cobas z 480/4800	LightMix Module	100	0	15	93.3	1	14	100	15	0	100	0	1	100	0	18	100	0	18
GENECUBE	GENECUBE SARS-CoV-2	100	0	7	100	0	7	100	7	0	-	-	-	100	0	16	100	16	0
LightCycler 96/TaqMan48	Ampdirect 2019-nCoV detection kit	100	0	18	100	0	18	100	18	0	94.1	1	16	100	0	17	94.1	16	1
Loopamp EXIA/LF-160	Loopamp (SARS- CoV-2) detection kit	98.5	1	67	95.6	3	65	100	68	0	50.0	1	1	93.8	1	15	100	15	0
QuantStudio3/5/5Dx	Ampdirect 2019-nCoV detection kit	100	0	16	100	0	16	100	16	0	100	0	15	100	0	14	100	14	0
QuantStudio3/5/5Dx	SARS-CoV-2 Direct Detection RT-qPCR Kit	100	0	20	100	0	20	100	20	0	100	0	20	100	0	20	95.0	19	1
QuantStudio3/5/5Dx	NIID	100	0	37	97.3	1	36	100	37	0	-	-	-	100.0	0	37	100	37	0
StepOne/ StepOne Plus	NIID	100	0	13	100	0	13	100	13	0	-	-	-	100	0	13	100	13	0
Takara Dice/System	SARS-CoV-2 Direct Detection RT-qPCR Kit	100	0	21	95.2	1	20	95.2	20	1	100	0	21	100	0	21	100	21	0
BD MAX	BD MAXTM ExKTM TNA-3 /PCR Cartridges	-	-	-	-	-	-	-	-	-	-	-	-	100	0	13	100	13	0
FilmArray	FilmArray Respiratory panel 2.1	-	-	-	-	-	-	-	-	-	-	-	-	100	0	15	100	15	0
Gene Xpert	Xpert Xpress SARS-CoV-2	-	-	-	-	-	-	-	-	-	-	-	-	100	0	12	100	12	0
Smart Gene	Smart Gene nCoV Detection test cartridge	-	-	-	-	-	-	-	-	-	-	-	-	93.8	1	15	100	16	0
TRCReady-80	TRCReady SARS-CoV-2	-	-	-	-	-	-	-	-	-	-	-	-	100	0	26	100	25	0
μTAS WAKO g1	μTAS WAKO COVID-19	-	-	-	-	-	-	-	-	-	-	-	-	0%	16	0	100	16	0

NIID: The standard method for the detection of severe acute respiratory syndrome coronavirus 2 virus RNA, which is a PCR method developed by the National Institute of Infectious Diseases (NIID).

EQA: external quality assessment; cps: copies; polymerase chain reaction (PCR): polymerase chain reaction.

Inconsistent results in duplicate assays of the set of SARS-CoV-2 RNA control samples (nos. 1 and 2) were reported in 1.3% (2/150) and 5.3% (8/151) of tests, respectively. Assay reagents used included the Loopamp (SARS-CoV-2) detection kit (n = 4), SARS-CoV-2 Direct Detection RT-qPCR Kit (n = 3), and Ampdirect 2019-nCoV detection kit (n = 1).

A total of 33 laboratories reported at least one incorrect result. All 16 laboratories of one peer group, which were categorized as laboratories of medical facilities, used a fully automated cartridge assay system (μTAS WAKO g1 and μTAS WAKO COVID-19) and failed to detect the presence of the SARS-CoV-2 full-process control (no. 5) due to inherent insufficiency in the LOD. Therefore, we performed further assessment of the remaining 17 laboratories for qualitative analysis after omitting the results from these laboratories.

Analysis of submitted data and information acquired from interviews with the individual laboratories revealed that causes of incorrect results included error in pre- or post-examination processes, including errors in sample handling (n = 1) and result judging or reporting (n = 3). In 13 laboratories with incorrect results, a failure to ensure the LOD with or without resultant inappropriate judgement criteria led to incorrect results. Seven laboratories had false-negative results for EQA sample no. 2, including laboratories using the Loopamp EXIA/LF-160 and Loopamp (SARS-CoV-2) detection kit (n = 3), the Takara Dice/System and SARS-CoV-2 Direct Detection (n = 1), the cobas z 480 and LightMix Modular SARS-CoV (COVID-19) (n = 1), the QuantStudio5/5Dx and LightMix Modular SARS-CoV (COVID-19) (n = 1), and the QuantStudio5/5Dx and NIID (n = 1). The three assay systems yielding false-negative results for EQA sample no. 4 included the Loopamp EXIA/LF-160 and Loopamp (SARS-CoV-2) detection kit (n = 1), the 7500Fast (Thermo Fisher Scientific) and SARS-CoV-2 Direct Detection (n = 1), and the LightCycler 96/TaqMan48 and Ampdirect 2019-nCoV detection kit (n = 1). The assay system yielding a false-negative result for EQA sample no. 5 was the Smart Gene and Smart Gene nCoV Detection test cartridge, and two false-positive results for EQA sample no. 6 were produced by the Takara Dice/System and SARS-CoV-2 Direct Detection (n = 1) and the LightCycler 96/TaqMan48 and Ampdirect 2019-nCoV detection kit (n = 1). Most of these laboratories used a Ct cutoff of 40, following the manufacturer's instructions (5/7, 71.4%).

Discussion

In the ongoing outbreak of COVID-19, reliable detection of SARS-CoV-2 is essential for patient care and interruption of transmission chains. This increase in the need for testing has led to an expansion in the assay systems and laboratories used to perform tests, making continuing quality improvement essential. To this end, we conducted a nationwide EQA of SARS-CoV-2 nucleic acid amplification testing in Japan, which was a commissioned project by the Ministry of Health, Labour and Welfare. The EQA in this report is unique because of its nationwide scale, including both public health and private sector laboratories conducting SARS-CoV-2 testing. The result of this EQA was well-guided to support the responses of participant laboratories and others to minimize the ongoing outbreak. EQA of assay performance allows participating laboratories to assess the quality and identify the weaknesses and strengths of the currently used diagnostic methods (Matheeussen et al. 2020a; Sung et al. 2020; Wang et al. 2020; Fischer et al. 2021; US Food and Drug Administration 2021). Based on our results, a guidance document for internal process control in laboratory practice was developed (Ministry of Health Labour and Welfare of Japan, 2021).

This study included both public health and private-sector laboratories. The proportion of correct results was the highest in public health laboratories, followed by private sector laboratories. The intended purposes of tests in the public health and private sector laboratories are different. Public health laboratories have long been responsible for performing diagnostic tests for newly emerging pathogens and are familiar with using LDTs. By contrast, private sector laboratories, such as laboratories of medical facilities, have a history of performing diagnostic tests using IVDs approved by a regulatory body, since laboratory tests performed using IVDs are covered by the national health insurance in Japan. Therefore, these laboratories are not as familiar with LDTs as public health laboratories. The assay performance evaluation of LDTs and IVDs requires validation and verification, respectively. Performance was mostly dependent on the laboratory category and assay evaluation, particularly the LOD. In this context, the laboratory evaluation of the LOD was found to be significantly lower in private sector laboratories than in public health laboratories.

Most of the laboratories used a Ct cutoff value of 40, following the manufacturer's instructions. However, the Ct value can vary depending on the extraction and assay reagents and the instrument (Engelmann et al. 2021; Fischer et al. 2021). This could increase the risk of incorrect results in specimens with lower viral loads. Therefore, laboratories must evaluate the assay performance, including the appropriate cutoff value.

As the performance in an EQA is the outcome of many steps in the entire workflow, each step of the workflow should be assessed. These steps include pre-examination, examination, and post-examination, for example, resuspension of the panel samples, extraction method/kit used, change in nucleic acid concentration during extraction, and type of NAAT assay, including the genomic target and the number of different genomic targets or tests performed. In this EQA, a positive RNA control and a positive full-process control were used as EQA samples to evaluate the process of amplification and detection and the entire process of extraction, amplification, and detection, respectively. The RNA control sample with a lower viral load yielded more unstable results. This suggests that reliability in terms of repeatability of the assay comes partly from the amplification process itself.

Thirty-three laboratories reported at least one incorrect result; most were private-sector laboratories. Our results indicated that performance was mostly dependent on the assay evaluation, particularly on the LOD at implementation.

We developed guidelines for laboratory practice based on the results of the EQA as well as a questionnaire survey that assessed the molecular testing capability of clinical laboratories conducting SARS-CoV-2 NAATs. The content of the guidelines focused on compliance with requirements of the Medical Care Service Act amendment for quality assurance of laboratory tests, assay performance evaluation at implementation, quality monitoring, standard operating procedure, personnel training on quality assurance, and competence of molecular laboratories Ministry of Health Labour and Welfare of Japan, 2021; Quality Control for Molecular Diagnostics 2020). Laboratories that reported incorrect results on their qualitative tests were asked to take corrective actions by reevaluating their nucleic acid extraction protocols and internal quality control processes according to the laboratory guidance. A follow-up EQA is recommended to confirm improvement to acceptable results.

Fully automated systems are commercially available as point-of-care tests. The LOD can differ significantly among these systems. Therefore, although the extraction efficiency was not disclosed for such systems, a total load of 2800 copies was directly input for this EQA. Nevertheless, all 16 laboratories of one peer group using the fully automated cartridge assay system μTAS WAKO COVID-19 failed to detect the presence of the SARS-CoV-2 full-process control. The LOD for this system reported by the manufacturer was 150 copies/assay. Therefore, a cause of the false-negative results was attributed to inherent insufficiency in the LOD. In this study, SeraCare's reagents were used as the full-process control as a pseudovirus, which is classified as an alpha virus different from SARS-CoV-2. This could have affected the EQA results.

In Japan at the time of this study, principally an individual person did not qualify for free-cost testing unless presenting with COVID-19 symptoms or deemed to have been in close contact with a confirmed case. Therefore, when asymptomatic persons take a test at their own will, they have to pay for a test out of pocket in the clinic, which generally costs $200-400. Meanwhile, a number of low-cost facilities have become available, offering no-frills testing for a fraction of the price at a regular clinic. Since the service is provided with low cost and ease of use, it is proving very popular. However, whether the test has been adequately implemented and operated is unknown, and the quality of the test results is questionable. Those facilities were not the targets of this EQA. An EQA should enroll such low-cost facilities to assess their actual performance to confirm acceptable results. Continuous improvement of test quality of all laboratories involved in SARS-CoV-2 diagnostic testing, expansion of tests, and even a broad screening program for the population are needed to control the ongoing COVID-19 pandemic.

In conclusion, this report summarizes the nationwide EQA of SARS-CoV-2 NAATs, which included both public health and private sector laboratories. Performance was mostly dependent on the category of the laboratory and assay evaluation, particularly the LOD. This study indicates that EQAs should be performed for all laboratories involved in SARS-CoV-2 diagnostic testing on a regular basis for the evaluation of potential weaknesses in SARS-CoV-2 molecular testing procedures. Development and use of guidance for laboratory practice based on survey results should be considered, particularly during new entry into laboratory services for emerging pathogens, such as SARS-CoV-2. This action will help to increase the quality of test results and thus contribute to the quality of patient care and interruption of transmission chains.