Determining procalcitonin at point-of-care; A method comparison study of four commercial PCT assays

Abstract

Objective

Procalcitonin (PCT) testing adds value in the early detection of infection and sepsis, as well as in management of antibiotic therapy. We determined the analytical and diagnostic performance of four PCT assays at POC.

Methods

PCT assays on AQT90 FLEX, Getein 1100, mLabs, and Finecare POC analyzers, in whole blood and plasma, were analyzed for repeatability, linearity, accuracy and concordance, by comparing with our reference PCT assay on the Cobas E602 system.

Results

For all assays precision was found higher in plasma than in whole blood. AQT90 showed good performance in all analytical and diagnostic areas, irrespective of test matrix and PCT concentration. The other POC assays demonstrated at least one analytical weakness. The Getein assay showed adequate precision only in plasma at high PCT levels, the mLabs assay only in plasma at low PCT levels. Accuracy, as demonstrated by Bland-Altman and Passing-Bablok analysis, was found adequate only for the AQT90 FLEX and Getein 1100 assay. Diagnostic concordance at 0.5 ​ng/mL was found excellent for the AQT90 FLEX, Getein 1100, and Finecare assays, much lower for the mLabs assay. At 0.25 ​ng/mL, only the AQT90 FLEX and Finecare assays showed excellent concordance with the reference assay.

Conclusions

The AQT90 FLEX PCT assay demonstrated excellent analytical performance and diagnostic agreement with the Cobas E602 assay, allowing both stand-alone and side-by-side testing. The other assays demonstrated some analytical deficiencies, potentially limiting their diagnostic use. Any clinical use of PCT results should always be in combination with all other clinical signs and diagnostic information.

1. Introduction

Procalcitonin (PCT) is regarded as a valuable diagnostic tool in the diagnostic work-up of patients with suspected bacterial infection or sepsis [1]. It is also being evaluated to shorten the course of antibiotic therapy in patients with respiratory tract infections [2] and in septic patients during intensive care treatment [3,4]. Moreover, PCT is increasingly accepted as a biomarker to guide the appropriate and effective use of antibiotics, potentially lowering the risks of complications associated with antibiotic overuse, such as anaphylactic reactions [5] and the emerge of resistant bacterial infections [6]. Plasma PCT levels also correlate with increasing severity of sepsis and organ dysfunction, providing prognostic value and supporting risk stratification algorithms [7]. As such, PCT is typically used as an aid in critically ill patients at risk for infection and sepsis [8]. Finally, PCT testing helps in the early recognition of severe bacterial infections in patients admitted to the emergency department (ED) with suspected infection or sepsis [9].

The clinical manifestations associated with sepsis may vary, and not all patients with infections display the same or clear clinical symptoms [10]. A risk score based on bed-side physiological criteria, e.g. qSOFA, is often used as an early warning signal that sepsis might be developing in stationary patients. The qSOFA test is indicative of a high risk for sepsis when at least 2 of the following three criteria are fulfilled; breathing ≥22/minute, systolic blood pressure ≤100 ​mmHg, and altered cognition [11]. When untreated, sepsis can ultimately develop into multi-organ dysfunction with mortality rates up to 50% [11]. An early diagnosis of sepsis and undelayed start of effective antibiotic and hemodynamic therapy is therefore essential to reduce the risk of clinical escalation to more severe stages of sepsis with higher mortality [12].

In the ED, the use of POC assays has been widely accepted as a convenient and effective tool for appropriate diagnostic work-up and undelayed therapy measures such as the start of antibiotic treatment [12]. Especially when patients are admitted with a suspected bacterial infection, a sensitive PCT assay at POC can support the diagnosis or rule-out of bacterial infection. Consequently, the number of PCT assays at POC is increasing, also in the Asian market.

This study aimed to compare four commercial point-of-care PCT assays to a reference PCT assay method, as provided by the clinical lab. All four POC assays were designed for quantitative measurements of procalcitonin levels in whole blood or plasma specimens. These four POC platforms were: AQT90 FLEX (Radiometer, Denmark); mLabs (Micropoint, China), Finecare (Wondfo, China), and Getein 1100 (Getein Biotech Inc., China). The PCT reference method was performed with the automated Elecsys BRAHMS PCT assay on the Cobas e602 platform (Roche Diagnostics, Germany).

Aside some reports on the AQT90 FLEX PCT assay, little experience with these PCT assays in POC use has been published, nor on their analytical and diagnostic performance. Our study therefore focused on the analytical performance of these four POC assays, by determining variability and linearity across PCT concentrations, as well as on their diagnostic performance by determining concordance with the reference method, in an effort to assess their analytical and diagnostic quality for clinical practice at point-of-care.

2. Material and methods – test systems

2.1. Overview of the POC test systems in the study

• 1.The AQT90 FLEX analyzer (Radiometer Medical, Denmark) is a random-access bench-top immunoassay analyzer intended for the quantitative determination of cardiac, coagulation, and sepsis markers in whole blood samples. The immunoassays on this platform are typically sandwich assays with all the specific reagents provided in a dry stable form inside a biomarker dedicated test cartridge. The capture antibodies are biotinylated and pre-immobilized to the streptavidin surface of the test cartridge. The tracer antibodies used for detection of the antigen are labeled with europium chelate and are also lyophilized together with the capture antibodies. The system provides full connectivity to LIS and HIS. The assay being used in our study was the AQT90 FLEX Procalcitonin (PCT) assay; Reagent Lot: 14071.
• 2.The mLabs ImmunoMeter is a bench top analyzer, based on immunofluorescence and system-specific disposable microfluidic cartridges. Full connectivity to the hospital information system is provided. The system can be battery driven, supporting remote use. The assay being used in our study was the mLabs® PCT assay; Reagent Lots: 150619 (whole blood), and 991160 (plasma).
• 3.The Getein 1100 analyzer is a small bench top device, designed for near patient testing situations. Based on immunofluorescence the system requires only 3–4 drops of human specimens per assay. Results are stored on the device, for downloading or transmission to LIS and/or HIS. The assay being used in our study was the Getein PCT Fast Test Kit (Immunofluorescence Assay); Reagent Lot: Y08818020.
• 4.The Finecare FIA meter is also a small bench top device, based on immunofluorescence and designed for near patient testing situations. It offers dedicated microfluidic cartridges to measure biomarkers in the areas of infection, diabetes, cardiovascular disease, kidney disease and oncology. This analyzer also provides full connectivity to LIS/HIS. The assay being used in our study was the Finecare™ PCT (Procalcitonin) Rapid Quantitative Test; Reagent Lot: W21011808 A7.

An overview of the general characteristics of these 4 POC assays can be found in Table 1A. All four POC analyzer systems provide a PCT assay of which the claimed analytical performance, as detailed in the manufacturer product labeling, can be found in Table 1B. This table also provides the product specifications of the PCT assay provided by the Cobas e602 analyzer, our reference device. All testing was performed in the Kingmed Diagnostics Laboratory (Shanghai, China).

Table 1.

Assay specifications of the 5 PCT assays in this study.

	AQT90 FLEX	mLabs	Finecare	GeTein 1100	Cobas e602
A. General characteristicsa
TAT (min)	21	<8	15	15	18
Assay stability in months (storage temperature)	8 (2–8 ​°C)	12 (2–8 ​°C)	24 (RT)	24 (4–30 ​°C)	NA
Sample	WB or plasma	WB, plasma, serum, or finger tip	WB, plasma, or serum	WB, plasma, or serum	plasma or serum
Pipetting required	No	Yes	Yes	Yes	No
Sample volume	2 ​mL (for multi-testing)	250 ​μL	Pl, S: 50 ​μL WB: 75 ​μL	3-4 drops	collection tube (for multi-testing)
B. Analytical characteristicsa
Reportable range	Pl: 0.08–100 ​ng/mL WB: 0.12–100 ​ng/mL	0.02–100 ​ng/mL	0.1–100 ​ng/mL	0.1–50 ​ng/mL	0.02–100 ​ng/mL
LOD (Functional sensitivity)	Pl: 0.059 ​ng/mL WB: 0.072 ​ng/mL	0.02 ​ng/mL	0.1 ​ng/mL	0.1 ​ng/mL	0.06 ​ng/mL
Precision (within run); %CV around the cutoff 0.5 ​ng/mL	<8%	<10%	<10%	<10%	<5%

^aAll information drawn from current product labeling or manufacturer web-sites. WB: Whole Blood, Pl: Plasma, NA: Not Applicable.

3. Material and methods – test procedures and statistics

For this comparison study we focused on analytical and diagnostic performance. Analytical performance was quantified by determining precision (repeatability and reliability) at two levels of PCT concentration in plasma and whole blood, as well as by determining linearity across a broad range of procalcitonin concentrations. Adequate repeatability is a necessary condition for agreement between methods. If one or more methods in the method comparison do not give repeatable results, assessment of agreement between those methods is meaningless [13]. Reliability was determined by looking at the confidence interval of the results obtained in the repeatability testing, while linearity was determined by standardized dilutions of plasma at a low and high level of PCT concentration.

Diagnostic performance was evaluated by determining concordance of the four POC assays with the laboratory standard method at three ranges of PCT concentrations, including around the relevant diagnostic cutoff of 0.5 ​ng/mL. Finally, assuming a linear relationship between all PCT assays, the analytical agreement between the four POC methods and the reference method was determined through Passing-Bablok linear regression analysis.

Results were checked and corrected for outliers based on the Dixon-Reed test. This method for identifying outliers is based on the ratio between the absolute difference between the most extreme value of a data set (i.e. the possible outlier) and the next most extreme value, and the range of all values. If the ratio is equal to or greater than one third of the range, then the most extreme value is an outlier [14].

The work as described in this study has been carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans. All samples used for the analysis were derived from patients with variable levels of PCT, all designated for routine PCT testing with the reference method. All PCT results shown in this paper were, if not indicated otherwise, determined with the reference assay, e.g., the Elecsys® BRAHMS Procalcitonin (PCT) assay; Reagent Lot: 27885700, on the Cobas e602 analyzer.

The methods are detailed below:

• 1.
  Precision:

  • •
    in plasma – Analytical precision was quantified in plasma by determining within-run precision in two plasma samples, both tested with all 5 PCT assays. Repeatability and reliability were determined at low and high levels of PCT levels. For this, a low and a high plasma sample were used with PCT levels of around 0.5 ​ng/mL and 2.0 ​ng/mL respectively. The plasma samples were tested 22 times each, excluding the highest and the lowest value. The remaining 20 values were used to calculate the outcome. Results were expressed in mean, SD, and %CV.
  • •
    in whole blood – Analytical precision was quantified in whole blood by determining within-run precision in two whole blood samples, both tested with all 4 POC assays. (The Cobas assay is validated only for plasma/serum). For this, a low and high whole blood sample were used with PCT levels of around 0.5 ​ng/mL and 2.0 ​ng/mL respectively. The blood sample was tested 12 times each, excluding the highest and the lowest value. The remaining 10 values were used to calculate the outcome. Results were expressed in mean, SD, and %CV.
  • •
    A calculated CV less or equal to 10% was considered acceptable.
• 2.
  Linearity:

  • •
    Linearity was determined in various plasma mix-dilutions using pooled PCT-negative plasma (PCT levels ​< ​LOD) and two initial PCT-positive samples; one with a relatively low PCT concentration (4.53 ​ng/mL), and one with a high concentration (73.62 ​ng/mL), based on results with the Cobas assay. Standard dilutions were created by mixing the samples with PCT-negative pooled plasma using ratios of 1:2, 1:4, 1:8, 1:16, and 1:32. Based on these ratios theoretical PCT concentrations in the dilutions were calculated. The diluted samples were analyzed in duplicate and mean values used for analysis. A correlation coefficient of 0.995 or higher between assay results and the calculated theoretical results was considered acceptable.
• 3.
  Concordance:

  • •
    Concordance of the four POC assays with the reference method was determined in plasma samples (n ​= ​80) with a PCT concentration range of 0.05–4.0 ​ng/mL, of which 40 samples were found below the cutoff of 0.5 ​ng/mL and the other 40 ​at above 0.5 ​ng/mL, as based on the results with the reference assay. Concordance for the POC assays was calculated based on agreement with the reference method in classifying the samples having PCT levels under or above the 0.5 ​ng/mL cutoff. Similar mathematical exercises using the same database were performed at PCT cutoffs 0.25 and 2.0 ​ng/mL. An agreement level of 90% or better and a Cohen’s kappa value above 0.8 were considered acceptable.
• 4.
  Accuracy:

  • •
    We assessed accuracy by Bland-Altman analysis of results from 110 samples with PCT levels in the range of 0.05–15.0 ​ng/mL and comparing them to the reference method on Cobas. For clarity, we calculated percentage differences versus mean values, and used consistent scaling for all 4 assays. Finally, we performed Passing-Bablok linear regression analysis using the same results from the 110 plasma samples. The linear agreement between assays was described by determining axis-intercept and slope. The agreement was quantified by the R-squared value of the linear regression, as well as by the correlation coefficient Pearson’s P-value. Bias was calculated by the residuals, e.g., the differences between expected value based on the regression and the actual result, across the PCT concentration range (not shown). Linearity was analyzed by a 2-sided test. Adequate accuracy was verified when Pearson’s P-value was found ≥0.9, indicating excellent correlation of the assay with the reference method.

Statistics were calculated using JMP version 12.2.0 from SAS (Cary, NC, USA).

4. Results

Precision - Overall, precision was found higher in plasma than in whole blood and found to vary substantially between the 4 POC systems, both in plasma as well as in whole blood (Tables 2A and 2B).

• •In plasma, at low PCT levels around the diagnostic cutoff of 0.5 ​ng/mL, only the AQT90 FLEX PCT assay demonstrated a CV%< 5% (3.4%), all others significantly exceeded this threshold, with Getein, mLabs and Finecare demonstrating a CV% of 12.8%, 10.0%, and 8,7% respectively. At high PCT levels around 2 ​ng/mL, the precision of AQT90 FLEX and Getein improved (CV% of 2.5 and 4.6% respectively), while the Finecare and mLabs assays showed low precision (CV% of 11.2% and 19.6%, respectively).
• •In whole blood, only the AQT90 FLEX system demonstrated an acceptable precision at low and high levels of PCT levels, with CV% of 4,7% and 2.7%, respectively. All other systems demonstrated moderate to poor precision, irrespective of PCT levels. While the Finecare assay demonstrated a moderate precision at 5,2% and 10.0% respectively, for the Getein (23.9% resp. 12.5%) and mLabs (28.0% and 29.5%) assay the observed precision was inadequate at all levels of PCT.

Table 2.

Precision of the 4 POC assays in plasma or whole blood, at low and high levels of PCT.

A. Plasma
Low PCT Level	AQT90 FLEX	mLabs	Finecare	GeTein 1100
Mean ng/mL	0.66	0.67	0.59	0.42
N	20	20	20	20
SD	0.02	0.07	0.05	0.05
CV %	3.38	9.98	8.7	12.8
High PCT Level	AQT90 FLEX	mLabs	Finecare	GeTein 1100
Mean ng/mL	2.45	2.41	1.62	1.57
N	19	20	20	20
SD	0.05	0.47	0.18	0.07
CV %	2.1	19.64	11.21	4.6
B. Whole blood
Low PCT Level	AQT90 FLEX	mLabs	Finecare	GeTein 1100
Mean ng/mL	0.24	0.77	0.35	0.35
N	9	10	9	10
SD	0.01	0.21	0.02	0.08
CV %	4.66	27.96	5.25	23.9
High PCT Level	AQT90 FLEX	mLabs	Finecare	GeTein 1100
Mean ng/mL	1.96	2.01	4.38	12.9
N	9	9	10	10
SD	0.05	0.59	0.44	1.6
CV %	2.7	29.51	9.96	12.51

Despite the limited precision of especially the mLabs and Finecare assays we decided to further explore the diagnostic performance of all four POC PCT assays.

Linearity – Overall, a fair to excellent linearity was observed with all 4 POC assays (Fig. 1). The AQT90 FLEX and Finecare assay also showed a good agreement with the expected values at both low and high PCT concentrations (as based on results with reference assay). In contrast, the Getein assay demonstrated merely moderate agreement, while the mLabs assay was burdened with large discrepancies. In fact, at PCT levels above 1 ​mg/mL the mLabs assay produced results much higher than the expected results, resulting in up to 5 times as high results as observed with any of the other assays.

Fig. 1.

Linearity of the four POC assays in plasma at low or high PCT levels.

Concordance – Diagnostic agreement between the reference device and the four POC assays was determined in 80 plasma samples with a range of PCT levels between 0.05 and 4.0 ​ng/mL with all five PCT assays (Table 3). Samples were classified based on the result with the reference device, 40 samples with PCT levels ≤0.5 ​ng/mL and 40 samples with PCT >0.5 ​ng/mL. This level of 0.5 ​ng/mL was then used as the diagnostic cutoff in the 2 ​× ​2 concordance tables. Overall, we found moderate to perfect agreement between the methods. In accordance with Landis and Koch [14], the Cohen’s kappa values ranged from the moderate 0.6 for the mLabs assay, to the substantial 0.8 for the Finecare assay, and the almost perfect agreement values of 0.88 and 0.95 for the Getein and AQT90 FLEX assays, respectively. These results show that three of the POC assays demonstrate moderate to excellent performance in the detection of (non-)increased levels of PCT in plasma. Especially the AQT90 FLEX and Getein1100 PCT assay demonstrate excellent agreement with the Cobas reference method in the clinical lab. In contrast, the mLabs assay missed 16 out of 20 samples with PCT levels between 0.5 and 1.0 ​ng/mL. We also determined concordance using 0.25 ​ng/mL as the diagnostic cutoff, acknowledging that this cutoff is often applied in antibiotic therapy decisions. Concordance results using the cutoff of 0.25 ​ng/mL were found similar as obtained with the 0.5 ​ng/mL cutoff, albeit that the Getein assay performance at the 0.25 ​ng/mL was found lower (kappa 0.49 vs. 0.88).

Table 3.

Concordance (Cohen’s Kappa) and Agreement between POC assays and reference method.

Accuracy – Bland-Altman analysis for the four POC assays versus the reference assay demonstrated good agreement for the AQT90 assay without a significant trend across the range from 0.07 ​ng/mL to 15 ​ng/mL (R² ​< ​0.05). The other three POC assays did show a weak but significant trend across this interval, suggesting that results of the assays compared were different, at least at some levels of PCT (R² ​> ​0.05) (Fig. 2 A-D). Especially the mLabs demonstrated a different analytical performance then the PCT assay on Cobas, measuring lower at low PCT concentrations, and higher at high concentrations.

Fig. 2.

Bland-Altman analysis of PCT results in plasma for all 5 assays (n ​= ​110). Results with the Cobas assay were compared to the AQT Assay (A), mLabs assay (B), Finecare assay (C), and Getein assay (D.) In order to minimize the effect of outliers we calculated the differences at certain mean PCT concentration as percentages of that mean. To enable a visual comparison, scaling of both axes were kept consistent, causing a few values with means ​> ​10 ​ng/mL not being depicted, 1 for AQT90, 4 for mLabs, and 3 for both Finecare and Getein.

The analytical agreement between the four POC assays and the reference PCT assay on Cobas was also determined using Passing-Bablok linear regression analysis, again based on the results from 110 plasma samples with PCT levels in a concentration range from 0.07 ​ng/mL to 15 ​ng/mL. The analysis demonstrated a fair agreement of AQT90 and Finecare assay with the reference assay, and a much lower agreement for the mLabs and Getein assay (Fig. 3A – 3.D), consistent with the observations in the Bland-Altman analysis earlier. To better expose the higher variability of the mLabs, Finecare, and Getein assay compared to the AQT90 assay especially at low PCT concentrations, we show the same data also on a log/log scale.

Fig. 3.

Passing-Bablok Linear Regression. In the upper diagrams, red lines indicate the best linear fit between the assay and the PCT assay on the Cobas system. The light grey lines indicate the 95% CI around this line of agreement. The bold dotted line indicates total equivalence between the two assays. The stars indicate results outside of the scale of the diagram. Logarithmic scaling was applied at lower PCT levels in to better illustrate the analytical differences between the four POC assays (lower diagrams). (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

A linear relationship with the reference method could only be validated for the AQT90 FLEX assay (Table 4, 2-sided test; p ​> ​0.05). For the other three POC assays a linear relationship with the reference method could not be statistically validated (Table 4, p ​< ​0.05). When considering the calculated best “linear fit” and SD between the methods (Table 4), it is evident that the AQT90 FLEX assay demonstrates the best fit, measuring slightly higher than the reference method and a Pearson’s p-value of 0.96.

Table 4.

Passing-Bablok Linear Regression with “Best Fit” expression.

Y-Assay	X-Assay	Linear expressiona [Y] ​= ​intercept ​+ ​slope x [X]		Linearity Test (2-sided, α ​= ​0.05)	Correlation
		intercept (95% CI)	slope (95% CI)	p ​=	Pearson’s P-value	R2-value
AQT90	Cobas	−0.044 (−0.063 to −0.029)	1.184 (1.154–1.216)	0.056	0.96	0.92
mLabs	Cobas	−0.231 (−0.231 to −0.163)	1.550 (1.384–1.762)	<0.001	0.62	0.38
Finecare	Cobas	−0.021 (−0.053 to 0.015)	0.919 (0.864–0.972)	<0.001	0.64	0.41
Getein	Cobas	−0.129 (−0.176 to −0.079)	1.255 (1.173–1.338)	0.01	0.94	0.89

^aAll expressed in PCT concentrations (ng/mL).

The Getein assay shows an analytical profile similar to the AQT90 FLEX assay, but with slightly lower correlation values than observed for the AQT90 FLEX assay, resulting in an excellent Pearson’s P-value of 0.94. The mLabs assay generated lower results than the reference method at PCT concentrations <1.0 ​ng/mL, suggesting a lower sensitivity. In contrast, at PCT concentrations >1.0 ​ng/mL the mLabs assay generated higher results in combination with a lower correlation than observed with the any of the other assays. This phenomenon was also observed earlier during linearity and precision testing. Pearson’s P-value for mLabs was calculated at 0.62. The Finecare assay generated slightly higher results than the reference method especially in the higher concentration range, associated with a Pearson’s P-value of 0.64.

5. Discussion

This study focused on the analytical and diagnostic performance of four PCT assays on POC platforms to assess their analytical and diagnostic quality in clinical use. Aside specific qualities like linearity and precision of these POC assays, we assessed their relative diagnostic performance by comparing them to a reference PCT assay in the clinical laboratory, e.g., the Elecsys® BRAHMS Procalcitonin (PCT) assay as determined on the Cobas e602 device. For the comparison methods we have assumed all results obtained with this reference method are true, e.g., that the classification of <0.5 or >0.5 ​ng/mL in the concordance study is correct, and that all Cobas results as used in the Passing-Bablok linear regression analysis were correct. Consequently, any conclusions based on these assumptions should be seen in the light of these assumptions. As an example, we noticed a trend for all 4 POC assays to measure higher than the reference method at higher PCT concentrations, an observation which could also be described by the reference method measuring lower than the four POC assays, and so on. With that being said, we will now discuss the study results in more detail.

Based on all results, the analytical characteristics of the reference PCT assay on the Cobas e602 and the characteristics as observed with the AQT90 FLEX PCT assay were almost identical. For the AQT90 FLEX PCT assay we found an excellent agreement of 98% with the reference assay for diagnostic concordance around the cutoff of 0.5 ​ng/mL. The observed precision with a CV < 5% determined in whole blood as well as in plasma at both low and high levels of PCT, confirmed the assay performance as claimed in the product labeling. Finally, the excellent linear agreement and correlation with the Cobas reference method across the measurement range as shown by Bland-Altman and Passing-Bablok analysis demonstrates the analytical quality of this assay. The reported results and agreement with the reference assay are also consistent with data from an earlier report in which the AQT90 FLEX PCT assay was compared to the 4 automated clinical lab assays, including the Elecsys BRAHMS PCT assay as performed on Cobas [15]. Because the AQT90 FLEX assay performance can be adjusted automatically to any reference assay results by entering slope and intercept from the linear “best fit” expression into the AQT90 FLEX assay settings, the agreement and concordance with the reference method can be optimized even further. All this makes the AQT90 FLEX assay the only POC assay in our assessment showing no limitations in clinical diagnostic use, e.g. with a sensitivity and precision to confidently detect increased PCT levels and to aid in the diagnosis or rule-out of bacterial infection and sepsis. Moreover, the substantial equivalence with the Cobas assay would support side-by-side use of both assays and their results in clinical routine.

The other three POC assays all demonstrated minor or major deficiencies in their analytical performance and limitations in their diagnostic use. The most significant and general deficiency of the other POC assays was their relatively low precision. Without good precision any form of comparing the assay with other assays is affected, as every result obtained without sufficient precision will be burdened with some degree of uncertainty, e.g., wide confidence intervals. More specific, while a low accuracy can be compensated by a correction factor, e.g., calibration, a low precision cannot be corrected. Because the limited precision was found especially at lower PCT levels, the sensitivity of the other three PCT assay to detect clinically relevant levels of increased PCT levels might likely have been affected. Especially the mLabs and the Getein assay demonstrated limited precision across PCT levels, both in plasma as well as in whole blood samples. In fact, the precision claimed in the product labeling for the mLabs assay (CV<10% at 0.5 ​mg/mL), and for the Getein assay (CV <10% within run) could not be reproduced.

Although all four POC assays demonstrated good linearity for standard dilutions of plasma, only results obtained with AQT90 FLEX, Getein 1100, and Finecare assay demonstrate good agreement with the reference PCT assay, suggesting that such similarity was key during development of these three assays. In contrast, the mLabs assay generated results deviating substantially from the reference assay, also explaining the observations in Bland-Altman and Passing-Bablok analysis in which the mLabs results trended lower than the Cobas results at low PCT levels, and much higher at high PCT levels. We have no explanation for these surprising observations, also because a good correlation between the mLabs assay and an automated PCT assay (Vidas, bioMerieux) has been reported in the internet by the manufacturer [https://tombreunig.wixsite.com/micropointbio/pct-datasheet], while both the Cobas and the Vidas PCT assays are adaptations from the BRAHMS Kryptor assay and known to correlate well with Kryptor and each other [[16], [17], [18]]. The other three assays showed a much better agreement with the reference method suggesting that the questionable performance of the mLabs assay we observed was related to assay irregularities or to product performance issues of the specific lot we used for our results.

We also observed a rather wide-spread scatter of results with the Getein assay at low PCT levels, illustrative of a rather low precision and potentially affected sensitivity. The Finecare assay correlated well with the Cobas assay, we merely found a few outlier results at higher levels of PCT.

Considering the diagnostic use of the PCT assay as a test to aid in the detection or rule-out of bacterial infection and sepsis, we examined the assays’ performance around the most common diagnostic cutoff of 0.5 ​ng/mL. As mentioned earlier, the AQT90 FLEX showed almost perfect agreement with the reference test. Surprisingly, considering its limited precision, so did the Getein assay. We did observe a few “false positives” (>0.5 ​ng/mL) results with the Getein assay, based on negative Cobas results (<0.5 ​ng/mL), but such Type 1 errors would not cause any issues from a therapeutic perspective. In contrast, many samples showing positive results with the Cobas assay turned out negative with the Finecare and especially with the mLabs assay. Such “false negatives” or Type 2 errors could result in infections going undetected and, potentially, in under-treatment of the patient.

In conclusion, of the four assays for testing PCT at point-of-care, only the AQT90 FLEX assay demonstrated a substantially equivalent analytical and diagnostic performance when compared to the laboratory assay provided on the Cobas e602 platform. This substantial equivalence enables the use of the AQT90 FLEX assay in stand-alone as well as in side-by-side testing of PCT in clinical practice. Also, the Getein PCT assay appeared safe in using as a screening assay to look for increased levels of PCT in patients suspected having bacterial infection. However, the limited agreement of this assay with the automated reference assay does not support side-by-side use, while the modest precision might limit its use in disease monitoring by serial testing. We found performance of the Finecare assay borderline clinically useful, while the surprisingly low analytical and diagnostic performance found for the mLabs assay did not advocate its use in any clinical routine. We have not tested the systems in patients with a confirmed diagnosis of infection or sepsis, which would be the conclusive study to validate the clinical diagnostic value of especially the Getein, Finecare, and mLabs assays in various clinical settings such as ED, intensive care unit, and post-operative recovery room. Based on our data, the AQT90 FLEX PCT assay appears superior to the other three POC tests and can be recommended for its intended use.

Author statement

Yubin Xu: Design study concept, Methodology, Supervision, Writing- Reviewing and Editing. Chunbao Li: Data management, Methodology, Validation, Writing- Original draft preparation. Yaping Huang: Project administration, Visualization.

Declaration of competing interestCOI

The authors declare having no financial nor other conflicting interests.