Calibration of BK Virus Nucleic Acid Amplification Testing to the 1st WHO International Standard for BK Virus

ABSTRACT

Significant interassay variability in the quantification of BK virus (BKV) DNA precludes establishing broadly applicable thresholds for the management of BKV infection in transplantation. The 1st WHO International Standard for BKV (primary standard) was introduced in 2016 as a common calibrator for improving the harmonization of BKV nucleic acid amplification testing (NAAT) and enabling comparisons of biological measurements worldwide. Here, we evaluated the Altona RealStar BKV assay (Altona) and calibrated the results to the international unit (IU) using the Exact Diagnostics BKV verification panel, a secondary standard traceable to the primary standard. The primary and secondary standards on Altona had nearly identical linear regression equations (primary standard, Y = 1.05X − 0.28, R² = 0.99; secondary standard, Y = 1.04X − 0.26, R² = 0.99) and conversion factors (primary standard, 1.11 IU/copy; secondary standard, 1.09 IU/copy). A comparison of Altona with a laboratory-developed BKV NAAT assay in IU/ml versus copies/ml using Passing-Bablok regression revealed similar regression lines, no proportional bias, and improvement in the systematic bias (95% confidence interval of intercepts: copies/ml, −0.52 to −1.01; IU/ml, 0.07 to −0.36). Additionally, Bland-Altman analyses revealed a clinically significant reduction of bias when results were reported in IU/ml (IU/ml, −0.10 log₁₀; copies/ml, −0.70 log₁₀). These results indicate that the use of a common calibrator improved the agreement between the two assays. As clinical laboratories worldwide use calibrators traceable to the primary standard to harmonize BKV NAAT results, we anticipate improved interassay comparisons with a potential for establishing broadly applicable quantitative BKV DNA load cutoffs for clinical practice.

INTRODUCTION

BK virus (BKV) is a polyomavirus that causes significant pathology in transplantation, notably BKV-associated nephropathy (BKVN) in kidney transplantation and hemorrhagic cystitis in hematopoietic cell transplantation (HCT). BKV is the leading viral cause of premature kidney transplant failure in kidney transplant recipients, with 1 to 15% of recipients expected to develop BKVN and progressive kidney injury and graft loss (1). BKV is also a pervasive cause of hemorrhagic cystitis in patients who have undergone HCT. The cumulative incidence of BKV-associated hemorrhagic cystitis ranges from 5 to 20%, but can be as high as 40% in cord blood transplant recipients (2). The management of BKV in transplantation remains a challenge, particularly with determining when and how best to initiate therapeutic modifications. Current protocols rely on BKV DNA testing to guide clinical action. In HCT, BKV DNA testing directs the diagnosis and management of hemorrhagic cystitis. In kidney transplantation, screening for plasma BKV DNA enables early intervention with a preemptive reduction in immunosuppression for preventing BKVN development and graft failure progression and inherently relies on interpretable BKV DNA levels (3, 4).

Though the monitoring of BKV DNA is central to management protocols, significant interassay variability in quantifying BKV DNA complicates the interpretation of these results and precludes establishing broadly applicable thresholds for clinical intervention (1, 5). Quantitative results reported by one assay may be vastly different from those reported by a different assay, making it difficult to compare results across assays. Such variability has been well described with negative impact on patient care, particularly when two laboratories report discrepant viral loads and management is initiated or delayed based on inaccurate or discordant quantitative results (6–10). Among the most cited factors contributing to this variability are the use of different standards for quantitation and primer and probe designs that do not account for BKV genotypic variation (7–9, 11).

To address the absence of a common calibrator and to improve assay harmonization, the World Health Organization (WHO) released the 1st WHO International Standard for BKV, NIBSC code 14/212, for standardizing BKV nucleic acid amplification testing (NAAT) in 2016 (12). Similar to the standards previously released for cytomegalovirus (CMV) and Epstein-Barr virus (EBV), the 1st WHO International Standard for BKV is a quantified biological reference material expressed in an international unit (IU). The use of calibrators traceable to the WHO international standard has improved interassay variability for quantifying CMV and EBV DNA and holds promise to also improve that for BKV (13, 14).

Here, we describe the methodology for calibrating BKV NAAT to the 1st WHO International Standard for BKV (primary standard). We evaluated the Exact Diagnostics (EDX) BKV verification panel (secondary standard) for assay validation and calibration. Additionally, we evaluated the analytical and clinical performances of the Altona RealStar BKV assay (Altona) and compared its performance to that of a laboratory-developed assay (VP1 BKV) in copy numbers and IUs. Finally, we determined the conversion factors for reporting results in IUs. The calibration of BKV NAAT assays to the international unit will be increasingly employed in clinical laboratories worldwide and is a necessary step toward achieving broadly applicable quantitative BKV DNA load cutoffs for clinical practice.

RESULTS

Primary and secondary standard comparison.

Analysis of the WHO (primary) and EDX (secondary) standards revealed the linear regression equations for the primary standard (Y = 1.05X − 0.28, R² = 0.99) and the secondary standard (Y = 1.04X − 0.26, R² = 0.99), as well as the 95% confidence intervals for the slopes (primary standard, 1.02 to 1.08; secondary standard, 1.01 to 1.07) and intercepts (primary standard, −0.42 to −0.15; secondary standard, −0.41 to −0.10) (Fig. 1). The differences between the slopes (P = 0.71) and intercepts (P = 0.75) were not significant. Consistent with these findings, the copy-to-IU conversion factors on Altona were highly similar (primary standard, 1.11 IU/copy; secondary standard 1.09 IU/copy). Based on these results, we utilized the secondary standard for the remainder of the experiments described in this study.

FIG 1.

Comparison of the Exact Diagnostics BK virus verification panel and the 1st World Health Organization International Standard for BK virus on the Altona RealStar BK virus assay. The line of identity is shown as a dashed line.

Altona analytical performance.

The linear range of Altona extended from 7.3 to 2.3 log₁₀ IU/ml (Fig. 1). Within-run (repeatability), between-run (reproducibility), and within-laboratory (total imprecision) precisions were also determined for each secondary standard concentration level (Table 1). The percent coefficient of variation (CV) was less than 3.5% for each tested concentration, with the exception of the lowest concentration, 2.3 log₁₀ IU/ml, where the within-run, between-run, and within-laboratory CVs were 23.3%, 14.6%, and 24.0%, respectively. The 95% lower limit of detection (LLOD) was 66 IU/ml (Table 2).

TABLE 1.

Precision of the Altona RealStar BKV assay using the Exact Diagnostics BKV verification panel (secondary standard)

Concn (log10 IU/ml)		Within-run		Between-run		Within-laboratory
Nominal	Mean observed	SDa	CVb (%)	SD	CV (%)	SD	CV (%)
7.3	7.32	0.03	0.35	0.01	0.10	0.01	0.31
6.3	6.34	0.04	0.68	0.03	0.45	0.03	0.71
5.3	5.35	0.11	2.01	0.03	0.59	0.03	1.74
4.3	4.30	0.05	1.14	0.04	1.02	0.04	1.38
3.3	3.29	0.07	2.11	0.09	2.75	0.09	3.25
2.3	2.08	0.48	23.29	0.30	14.59	0.30	23.97

^aSD, standard deviation.

^bCV, coefficient of variation.

TABLE 2.

Lower limit of detection of the Altona RealStar BKV assay using dilutions of the Exact Diagnostics BKV verification panel

Concn (IU/ml)	No. of replicates	Detected
		No.	%
600	10	10	100
300	10	10	100
150	10	10	100
100	6	6	100
50	6	5	83.3
25	6	3	50.0

BKV detection and quantitation in clinical EDTA plasma samples.

One hundred sixty-one EDTA plasma samples previously tested on the VP1 BKV assay were tested for BKV DNA using Altona. The overall agreement between assays was 96.3% (155/161), with a kappa statistic of 0.91 (standard error, 0.03; 95% confidence interval, 0.85 to 0.98), indicating excellent qualitative agreement (Table 3). There were six qualitatively discrepant samples. Three were Altona positive and VP1 BKV negative, and these samples were detected at less than the Altona assay's lower limit of quantitation (200 IU/ml, 2.3 log₁₀ IU/ml). Similarly, the three samples that were VP1 BKV positive and Altona negative had low virus loads (504, 1025, and <280 IU/ml). Upon reextraction and retesting of each plasma specimen with both the Altona and VP1 BKV assays, concordant qualitative detection was obtained in four of six repeat samples, with one Altona-positive VP1 BKV-negative and one VP1 BKV-positive Altona-negative sample remaining.

TABLE 3.

Comparison of Altona RealStar and VP1 BKV assays for detecting BKV in clinical samples

Altona	VP1 BKV
	No. detected	No. not detected	Total
No. detected	107	3	110
No. not detected	3	48	51
Total	110	51	161

To investigate the quantitative agreement between the Altona assay and the VP1 BKV assay, the log₁₀ copies/ml and log₁₀ IU/ml concentrations of the 93 clinical samples originally quantifiable by both methods were plotted against one another and Passing-Bablok regression was performed (Fig. 2A and B). This analysis resulted in regression lines of Y = 1.02X − 0.76 in copies/ml and Y = 1.02X − 0.15 in IU/ml (Fig. 2B). A comparison of the assays in copies/ml revealed the 95% confidence intervals of the slope (0.97 to 1.07) and intercept (−0.52 to −1.01), indicating that Altona displayed no proportional bias and negative systematic bias compared with results from the VP1 BKV. In the comparison of assays in IU/ml, the 95% confidence intervals of the slope (0.97 to 1.07) and intercept (−0.36 to 0.07) included 1 and 0, respectively, demonstrating no proportional bias and resolution of the systematic bias between assays. Next, the differences in log₁₀ concentrations in copies/ml and IU/ml were plotted against the average values to generate Bland-Altman plots. This analysis more clearly revealed the negative bias in these assays when calibrated in copies/ml. The mean difference was −0.70 log₁₀ copies/ml (Altona − VP1 BKV) with 95% limits of agreement of −1.27 to −0.13 (Fig. 2C). Consistent with the regression analysis, in the comparison of assays in IU/ml, the mean difference was −0.10 log₁₀ IU/ml with 95% limits of agreement of −0.67 to 0.47 (Fig. 2D). Altogether, these results indicate that calibrating in IUs improved the quantitative agreement and minimized the bias between the two assays. One sample demonstrated a quantitative difference of more than ±1.0 log₁₀ IU/ml (−1.43 log₁₀ IU/ml, Altona − VP1 BKV). Reextraction of the primary specimen and retesting on both the Altona and VP1 BKV assays revealed values of 3.18 and 3.01 log₁₀ IU/ml, respectively, a difference of 0.17 log₁₀ IU/ml.

FIG 2.

Quantitative comparisons of clinical specimens tested with the Altona RealStar BK virus assay (Altona) and our prior laboratory developed assay (VP1 BKV). (A) Passing-Bablok regression analysis of 93 plasma specimens quantifiable by both the Altona and VP1 BKV and compared in log₁₀ copies/ml. The regression line (solid line) and 95% confidence intervals (dashed lines) are displayed. (B) Passing-Bablok regression analysis of the same 93 plasma specimens compared in log₁₀ IU/ml. The copy-to-IU conversion factor for VP1 BKV was 0.28. The regression line (solid line) and 95% confidence intervals (dashed lines) are displayed. (C) Bland-Altman plot comparing Altona with VP1 BKV in copies/ml. The bias (solid line) was −0.70 log₁₀ copies/ml. The 95% limits of agreement (gray dashed lines) and zero line (dotted black line) are also shown. (D) Bland-Altman plot comparing Altona with VP1 BKV in IU/ml. The bias (solid line) was −0.10 log₁₀ IU/ml. The 95% limits of agreement (gray dashed lines) and zero line (dotted black line) are also shown.

DISCUSSION

The 1st WHO International Standard for BKV was released in June 2016 as a common calibrator to improve harmonization of BKV NAAT and enable comparisons of biological measurements worldwide. While BKV NAAT is integral to the management of BKV in transplantation, quantification variability between assays has prevented comparisons of BKV viral load results across institutions, making result interpretation challenging and variable. Though current guidelines recommend a plasma viral load of ≥10,000 copies/ml as the threshold for clinical intervention in kidney transplantation based on a specificity of 93% for BKVN, some centers have reported that this threshold may underestimate the diagnosis of BKVN and suggest using lower viral load thresholds (1, 5, 15). In fact, such discrepancies may be appropriate realities for institutions using different assays that produce different quantitative results for a given sample. As seen with CMV and EBV (16), the use of an internationally accepted common calibrator offers promise for improving the harmonization of BKV NAAT results and the opportunity to define broadly applicable interinstitutional thresholds needed for optimizing the primary and secondary prevention of BKV-related end-organ disease in transplantation.

Consistent with reports of interassay variability in BKV NAAT, the Altona and the VP1 BKV assays yielded significantly different results in copies/ml. The mean difference between assays was −0.70 log₁₀ copies/ml, greater than the clinically acceptable range of variation, with the Altona assay reporting lower values compared with those from the VP1 BKV assay. However, the use of a common calibrator significantly improved the agreement between the assays. Comparison of Passing-Bablok regression and Bland-Altman analyses from testing clinical samples on both assays showed an improvement in the harmonization of results and resolution of systematic bias when results were reported in IUs. The mean difference between assays was reduced to −0.10 log₁₀ IU/ml, a difference that is not clinically significant.

We also evaluated the use of a secondary standard, the EDX BKV verification panel, for assay calibration. Commercial secondary standards traceable to the primary standard offer practical advantages over the primary standard. Notably, the secondary standard consists of panel members that span a wider dynamic range, do not require manual dilution that can introduce quantification variability, and can be purchased without a Centers for Disease Control and Prevention (CDC) import permit, offering improved accessibility. To confirm calibration to the primary standard as reported by the manufacturer, we compared the performance of the secondary standard with that of the primary standard on Altona. The primary and secondary standards revealed nearly identical linear regression equations (primary standard, Y = 1.05X − 0.28, R² = 0.99; secondary standard, Y = 1.04X − 0.26, R² = 0.99) and conversion factors (primary standard, 1.11 IU/copy; secondary standard 1.09 IU/copy). Using the secondary standard, the Altona demonstrated a linear range of 7.3 to 2.3 log₁₀ IU/ml, excellent precision, and an LLOD of 66 IU/ml. Given the comparable performance, wider dynamic range, and practical advantages, we found this secondary standard appropriate for use in analytic evaluation and IU calibration.

However, previous work has shown that, for a given nominal value, secondary standards do not always contain equal copy numbers of the targeted sequence (17). This applies to interstandard comparisons and may also apply within a given standard, for example, if a particular region of the genome is deleted or multiplicated. Though the EDX BKV secondary standard was calibrated against the WHO primary standard using digital PCR, whole-genome BKV sequencing of both primary and secondary standards for deletions, insertions, and point mutations, as well as digital PCR evaluation of multiple genomic regions targeted by common real-time quantitative PCR assays, may be necessary to ensure accurate calibration. Nevertheless, using the EDX BKV secondary standard, we successfully harmonized the Altona (targeting the large T antigen) and VP1 BKV assays to the international unit. In addition, future studies are required for evaluating the commutability of primary and secondary standards, the ability of a reference material to have interassay properties comparable to those demonstrated by authentic clinical samples (18), as noncommutable standards may significantly impact quantitative agreement (19, 20).

As BKV NAAT results are reported in IUs, clinicians managing BKV infections in immunocompromised patients need to understand the impact on result interpretation. At our institution, a sample that was previously quantified as 10,000 copies/ml by the VP1 BKV assay would now be reported as approximately 2,500 IU/ml on the Altona assay, a 4-fold decrease in result quantification compared to historical data. This value may prompt clinicians to hold off on therapeutic modifications that they previously might have initiated. As the degree and directionality of viral load quantification change are assay dependent, each center should be aware of the factors used to convert results to the IU for their BKV assay, and they should monitor patients closely for over- or undertreating BKV-related end organ disease as optimal action thresholds are evaluated and determined.

While the use of a common calibrator in BKV DNA quantitation will improve the agreement between assays, it is not likely to eliminate all result variability. A multi-institution evaluation of the first WHO international standard for CMV DNA demonstrated improved harmonization across assays, though clinically relevant variability remained (13). The factors previously cited to impact BKV NAAT quantification include the nucleic acid extraction method and the design of primers and probes that do not account for variation in BKV genotypes (7–9). The use of commercially available assays, such as Altona, may yield improved agreement between assays, as demonstrated in a study that found reliable and comparable performances of three commercial real-time BKV PCR assays (21). Thus, the use of commercial assays along with a standard calibrator may resolve many of the issues surrounding the heterogeneity in BKV NAAT results. The impact of the international standard on the harmonization of results, including factors contributing to any residual interassay quantification variability, remains to be determined.

Limitations of this study include the evaluation of one commercially available secondary standard and the comparison of two BKV NAAT assays performed at a single laboratory. Additionally, this analysis focused on plasma samples as the primary sample matrix. Similar to comparative studies on CMV and EBV NAAT, studies that evaluate the reliability of different secondary standards and test clinical samples at several laboratory sites using different assays with calibrators traceable to the primary standard are needed.

In conclusion, we discussed the steps taken in our clinical laboratory for validating the Altona RealStar BKV assay and calibrating results to the IU. We demonstrated comparable performances of the primary and secondary standards and provided support for the use of the Exact Diagnostics BKV verification panel for assay validation, reference range determination, and IU calibration. Additionally, we found improved reliability of the Altona and VP1 BKV PCR assays when using a common calibrator traceable to the primary standard. The release of the 1st WHO International Standard for BKV DNA offers the potential for improved interassay comparisons and the ability to establish broadly applicable viral load thresholds for clinical practice. Such harmonization opens new opportunities for standardizing the primary and secondary prevention of BKVN, for monitoring novel therapeutics and therapeutic strategies undergoing evaluation in clinical trials, and, ultimately, for managing BKV infection in transplantation.

MATERIALS AND METHODS

Ethics statement.

This study was reviewed and approved by the institutional review board of Stanford University.

BKV quantitative assay.

The Altona RealStar BKV assay (Hamburg, Germany) was performed according to the manufacturer's recommendations on a Rotor-Gene Q (Qiagen, Germantown, MD) real-time PCR instrument. Cycling conditions were as follows: 95°C for 10 min, and 45 cycles of 95°C for 15 s and 58°C for 60 s. Calibration was performed using DNA standards provided by the manufacturer. For all experiments, DNA was isolated from 200 μl of plasma using the EZ1 virus minikit, v2.0 on the EZ1 advanced XL instrument (Qiagen, Germantown, MD). The purified DNA was eluted into a final volume of 60 μl, and each PCR utilized 10 μl. An internal control was added to each primary sample prior to extraction, and amplification was performed with specific primers and hydrolysis probes contained in the master mix to ensure adequate extraction efficiency and the absence of inhibitors.

Primary and secondary standard panels.

The primary standard, comprising BKV subtype 1b-2, was obtained from the National Institute for Biological Standards and Control (Hertfordshire, United Kingdom) and diluted to 6.0, 5.0, 4.0, 3.0, and 2.7 log₁₀ IU/ml in BKV-negative EDTA plasma (SeraCare, Milford, MA) (12). The secondary standard (Exact Diagnostics, Fort Worth, TX) is a commercially available panel composed of heat-inactivated BKV subtype 1a formulated in EDTA plasma and calibrated to the WHO standard at concentrations of 7.3, 6.3, 5.3, 4.3, 3.3, and 2.3 log₁₀ IU/ml using digital PCR (22).

Primary and secondary panel comparison.

To evaluate the performance of the secondary standard, the primary and secondary standards were run on Altona in triplicate over 4 days. The means of the observed log₁₀ concentrations from both panels were plotted against the nominal log₁₀ IU/ml values, and ordinary least-squares regression was performed. The copy-to-IU conversion factors for both standards on Altona were calculated by averaging the ratios of expected IU to the observed copies at each tested concentration. Values greater than 1.5× the interquartile range (IQR) from the median were considered outliers and removed from final conversion factor calculations.

To aid the interpretation of historical results obtained on the VP1 BKV assay, we determined the conversion factor to IU by testing the secondary standard in triplicate on four separate days on the VP1 BKV assay. The copy-to-IU conversion was calculated as above and determined to be 0.28.

Altona analytical performance.

The secondary standard was tested in triplicate on four separate days to determine the linear range and precision of the Altona assay. The linear range was determined by fitting a best-fit line to the data by regression analysis and included the range where the R² value for this line was ≥0.98. Within-run, between-run, and within-laboratory precisions were calculated to establish the precision of Altona. To establish the 95% lower limit of detection (LLOD) of Altona, the secondary standard was diluted in EDTA plasma to 600, 300, 150, 100, 50, and 25 IU/ml. 10 replicates at each concentration were tested, and the 95% LLOD was calculated using probit analysis.

Altona and VP1 BKV assay comparison.

The clinical performance of Altona was compared with that of our previous BKV NAAT assay (VP1 BKV), an in-house laboratory-developed real-time PCR targeting the VP1 region of the BKV genome. VP1 BKV was performed on a LightCycler 2.0 instrument (Roche Applied Science, Indianapolis, IN) as previously described (23). The pBKV34-2 plasmid (ATCC 45025) containing the entire BKV Dunlop genome was used as the calibrator.

One hundred sixty-one deidentified clinical plasma specimens submitted to the Stanford Health Care (SHC) Clinical Virology Laboratory between June 2015 and February 2016 for BKV testing were tested on the Altona and VP1 BKV assays. Plasma specimens were collected in EDTA tubes and stored at −80°C prior to testing. Clinical samples were tested on both assays and the results were compared. The overall agreement and kappa statistic were determined by evaluating results that were qualitatively concordant by both methods. Assay quantitation in log₁₀ copies/ml and log₁₀ IU/ml was compared using Passing-Bablok regression and Bland-Altman analyses.

Statistical analysis.

Linearity was evaluated using ordinary least-squares regression in Prism version 6.0g (GraphPad, La Jolla, CA). Precision was analyzed in Excel (Microsoft, Redmond, WA), using the formula described by Chesher in 2008 (24). The 95% LLOD was calculated using probit analysis in SPSS software version 21 (IBM, Armonk, NY). Passing-Bablok and Bland-Altman analyses were performed in R using the mcr package (https://cran.rproject.org/web/packages/mcr/index.html), and plots were generated using Prism.