Analytical variation in factor VIII one‐stage and chromogenic assays: Experiences from the ECAT external quality assessment programme

Abstract

Background

Both one‐stage (OSA) and chromogenic substrate assays (CSA) are used to measure factor VIII (FVIII) activity. Factors explaining analytical variation in FVIII activity levels are still to be completely elucidated.

Aim

The aim of this study was to investigate and quantify the analytical variation in OSA and CSA.

Methods

Factors determining analytical variation were studied in sixteen lyophilized plasma samples (FVIII activity <0.01‐1.94 IU/mL) and distributed by the ECAT surveys. To elucidate the causes of OSA variation, we exchanged deficient plasma between three company set‐ups.

Results

On average, 206 (range 164‐230) laboratories used the OSA to measure FVIII activity and 30 (range 12‐51) used CSA. The coefficient of variation of OSA and CSA increased with lower FVIII levels (FVIII <0.05 IU/mL). This resulted in misclassification of a severe haemophilia A sample into a moderate or mild haemophilia A sample in 4/30 (13.3%) of CSA measurements, while this was 37/139 (26.6%) for OSA. OSA measurements performed with reagents and equipment from Werfen showed slightly lower FVIII activity (0.93, IQR 0.88‐0.98 IU/mL) compared to measurements with Stago (1.07, IQR 1.02‐1.14 IU/mL) and Siemens (1.03, IQR 0.97‐1.07 IU/mL). Part of this difference is explained by the value of the calibrator. For CSA, the measured FVIII levels were similar using the different kits.

Conclusions

In the lower range (<0.05 IU/mL), analytical variation of FVIII measurements is high in both OSA and CSA measurements. The variation in FVIII activity levels was partly explained by specific manufacturers. Further standardization of FVIII measurements and understanding of analytical variation is required.

1. INTRODUCTION

Correct classification of haemophilia A severity is important as treatment intensity is based on categorization.1 Severe (factor VIII [FVIII] activity levels <0.01 IU/mL) and some moderate (FVIII activity levels 0.01‐0.05 IU/mL) haemophilia patients receive prophylactic replacement therapy to prevent spontaneous bleeding in joints and muscles while mild haemophilia A patients (FVIII activity levels 0.05‐0.40 IU/mL) receive desmopressin or replacement therapy only in cases of trauma and/or surgery.1, 2 Measuring FVIII activity levels accurately and reproducibly in different laboratories is therefore essential. We recently showed that despite excellent performance in the ECAT external quality assessment programme, between‐laboratory variation may result in different FVIII levels, and consequently, in misclassification of haemophilia severity.4 Limited between‐laboratory variation in FVIII activity levels is also of importance for the monitoring of treatment in patients with haemophilia A, as specific target FVIII activity levels should be maintained around surgery and bleeding episodes.1, 2, 5

Two assays are widely used to measure FVIII activity: the one‐stage assay (OSA) and the two‐stage chromogenic substrate assay (CSA). Most laboratories use the OSA, which is based on the activated partial thromboplastin time (APTT), using the time until clot formation as its endpoint.6 In the CSA, the coagulation system is triggered resulting in the generation of factor Xa (FXa).7 In the second step of this test, FXa hydrolyses a chromogenic substrate causing a colour change, which reflects the amount of FVIII activity left in the patient sample. The endpoint in the CSA differs from that in OSA, as the CSA measures extinction at a plateau phase. Discrepancies in FVIII activity levels have been extensively reported between these two assays, depending on the mutation in F8 gene.8, 9

Nowadays, reagents and equipment to perform FVIII activity measurements are widely available. The use of varying products may partially explain the between‐laboratory variation in FVIII results. However, it is still unclear what the precise impact is of varying in reagents and equipment on the variability of FVIII activity measurements.11, 12 A possible explanation may be that particular companies provide the majority of products applied for the haemostatic testing which is standard in haemophilia. Most reports focus on the specific reagents of one company,12, 15, 16 rather than analysing a test system from one company which consists of calibrator, activator, deficient plasma and equipment. As this is often the case in real life situations, causal factors leading to the variation in FVIII activity levels should be investigated more extensively.

To improve quality of measurements in haemostasis laboratories, laboratories follow international guidelines and participate in external quality control surveys. The data from the ECAT external quality assessments indeed show that laboratories use all components for the FVIII assays from one company in a majority of cases. Therefore, ECAT data are highly suitable to investigate the influence of company set‐ups on FVIII activity level variation. The aim of this study is to investigate and quantify variation in FVIII activity when testing by OSA and CSA in surveys conducted by the ECAT foundation. In addition, we studied effects of replacement of selected reagents in the OSA with those from another company on FVIII results.

2. MATERIAL AND METHODS

2.1. Quantifying variation in FVIII activity measurements

More than 200 laboratories working in the field of haemostasis and thrombosis participate in the ECAT external quality assessment programme for FVIII. Four times per year, two lyophilized plasma samples are distributed. To quantify the variation in FVIII activity measurements, we selected sixteen samples (a) with FVIII activity levels between <0.01 and 1.94 IU/mL (consensus values), (b) measured by more than 10 laboratories by OSA or CSA and (c) measured between 2010 and 2016. As expected, we found that most laboratories use the calibrator, activator, deficient plasma and equipment from one company in the OSA. Therefore, three groups were created from the three largest companies to compare the CVs in the OSA: (a) Siemens, (b) Stago and (c) Werfen.

To investigate the impact of variation on hypothetical haemophilia severity diagnoses which are solely based on laboratory results, FVIII activity levels were subsequently classified according to severity type as stated by the World Federation of Haemophilia.1

2.2. Impact of test system on FVIII activity levels in the OSA

From the ECAT external quality assessment programme, four plasma samples were chosen with different FVIII activity levels to investigate the influence of the test system on the FVIII activity levels. To cover the range of FVIII activity measurements, the following samples from the ECAT surveys were chosen: (a) a severe haemophilia A patient sample (consensus value FVIII <0.01 IU/mL), a mild haemophilia A patient sample (consensus value FVIII 0.16 IU/mL), a borderline haemophilia A/low FVIII activity sample (consensus value FVIII 0.42 IU/mL) and a sample with normal FVIII activity levels (consensus value FVIII 1.00 IU/mL). The FVIII activity levels were measured by laboratories participating in the ECAT surveys. Next, groups were created of laboratories using calibrator, activator, deficient plasma and equipment from one company to investigate the impact of the test system on FVIII activity levels. When the reported FVIII activity levels were below 0.01 IU/mL, they were considered in the analysis as 0.005 IU/mL. To compare the FVIII activity levels between the three companies, we used the Kruskal‐Wallis test as the data were not normally distributed. All statistics were performed using SPSS statistics for Windows, version 24.0 (IBM Corp, Armonk, NY, USA). A P‐value of <0.05 was considered statistically significant.

2.3. Impact of test system on FVIII activity levels in the CSA

The impact of different test systems in the CSA was also investigated. FVIII activity levels were compared between Chromogenix Coamatic, Hyphen Biomed and a test system from Siemens in the four plasma samples as described under the subheading of “Impact of test system on FVIII activity levels in the OSA.” The Kruskal‐Wallis test was performed to analyse the data.

2.4. Contribution of deficient plasma and calibrator

As not all laboratories use complete packages from one manufacturer, deficient plasma or a calibrator from another company may explain the variation in FVIII results. Unfortunately, this could not be investigated in the ECAT surveys, as most laboratories use all the components in the test system from one company. For this reason, we varied in deficient plasma on three different machines and its reagents as shown in Table 1. Calibration curves were created in these set‐ups. Using these calibration curves, FVIII activity levels were measured in duplicate in three samples; one sample with normal FVIII activity levels (consensus value FVIII 1.00 IU/mL), mild haemophilia A (consensus value FVIII 0.34 IU/mL) and moderate haemophilia A (consensus value FVIII 0.04 IU/mL).

Table 1.

Set‐up of the different packages when varying in deficient plasma

	Company
	Siemens	Stago	Werfen
Calibrator	Standard Human Plasma	STA‐Unicalibrator	HemosIL Cal Plasma
Activator	FVIII Actin FS	STA‐CK Prest	APTT‐SynthASil
Deficient plasma	FVIII deficient	STA Immunodef VIII	FVIII Def. Plasma
Equipment	CS 5100 Sysmex	STA‐R Max	ACL TOP500

The influence of the calibrator was investigated by measuring the FVIII activity levels in duplicates from the calibrator of Werfen (HemosIL Cal Plasma) and Stago (STA‐CK Prest) in the Siemens set‐up as described in Table 1. As these calibrators have assigned values, we compared the measured FVIII activity levels of the calibrators with their assigned values.

3. RESULTS

3.1. Quantifying variation in FVIII activity measurements

In the different surveys, on average, 206 (range 164‐230) laboratories reported results from analyses that used the OSA to measure FVIII activity and 30 (range 12‐51) laboratories used the CSA. In surveys with lower FVIII activity levels, the CV was higher (Figure 1A). When comparing FVIII levels measured by OSA with the CSA, the CV was comparable between the OSA and the CSA. In addition, the median absolute FVIII activity levels in a sample from a severe haemophilia A patient were similar in the OSA and CSA, with FVIII activity levels of 0.005 IU/mL (IQR 0.005‐0.03 IU/mL) for the CSA and 0.005 IU/mL (IQR 0.005‐0.01 IU/mL) for the OSA. When comparing the CV between the laboratories using reagents from three companies for the OSA, similar patterns were observed. However, separation of products from different companies resulted in higher CVs than the overall CV with a CV up to 158% maximally for the Werfen package (Figure 1B).

Figure 1.

The coefficient of variation (CV) is higher when FVIII activity levels are lower. A, The CVs were calculated for both one‐stage assay (OSA) and chromogenic stage assay (CSA). The circles indicate the CVs calculated from measurements with the OSA. The squares reflect the CVs calculated from measurements with the chromogenic substrate assay (CSA). B, The CV of the OSA was also calculated when FVIII activity levels were measured with products from Siemens (circles), Stago (squares) and Werfen (triangles)

3.2. Impact of test system on haemophilia severity classification

The impact of this FVIII variability on haemophilia classification which is solely based on FVIII activity levels is significant. This is illustrated by the fact that the severe haemophilia A sample was classified as moderate in 37/139 (26.6%) of all OSA measurements (Figure 2D). When classification is differentiated according to company in samples tested with OSA, 9/45 (20.0%) of the laboratories working with Siemens classified this sample as moderate or mild haemophilia while these percentages were 18/38 (47.4%) for Stago and 10/56 (17.9%) for Werfen. Only a small number of laboratories measured FVIII activity levels with CSA. Overall with CSA, 4/30 (13.3%) classified the severe haemophilia A sample as moderate or mild. When results are differentiated according to company, misclassification was observed in 1/8 (12.5%) for Chromogenix, in 2/14 (14.3%) for Hyphen and in 1/8 (12.5%) for CSA testing with Siemens products. In conclusion, laboratories using CSA misclassified severe haemophilia A patients less often. However, the number of CSA measurements is small.

Figure 2.

The distribution of the FVIII activity levels measured by one‐stage assay (OSA). FVIII levels are shown when measured with company set‐ups from Siemens, Stago or Werfen

3.3. Impact of test system on FVIII activity levels in the OSA

Factor VIII activity levels were analysed for the three major companies and shown in Figure 3. In a sample from a healthy person (Figure 3A), FVIII activity levels measured with products from Werfen (median 0.93, IQR 0.88‐0.98 IU/mL) were lower than FVIII activity levels measured by products from Stago (median 1.07, IQR 1.02‐1.14 IU/mL) or Siemens (median 1.03, IQR 0.97‐1.07 IU/mL). We also observed this trend in a sample with 0.42 IU/mL FVIII (Figure 3B). The differences between the three manufacturers in the samples with lower FVIII activity levels were minimal; however, small differences may have a large clinical impact.

Figure 3.

Combination of deficient plasma, equipment, calibrator and activator from Werfen causes lower factor VIII (FVIII) activity levels when FVIII >0.40 IU/mL compared to Stago and Siemens. The red dots are the results from each laboratory. The black line represents the median. The error bars represent the interquartile range. Statistical significance is indicated as *P < 0.05, **P < 0.01, *** P < 0.001

We also investigated the influence of different activators in the set‐up of all products from Siemens. This company had an activator based on ellagic acid and one based on silica. In addition, phospholipid concentrations differ between these activators. We were able to compare these activators since enough participants in the ECAT survey used these activators. We observed equal FVIII activity values between the activators in all four plasma samples (Figure S1).

3.4. Impact of test system on FVIII activity levels in the CSA

For the CSA, three kits were most oftenly used: (a) Chromogenix Coamatic (n = 8‐13), (b) Hyphen Biomed (n = 14‐23) and (c) FVIII Chromogenic assay from Siemens (n = 7‐10). We compared the FVIII activity levels obtained by the three most commonly used kits and observed no consistent differences in FVIII activity levels between the kits (Figure 4). Some small differences were found as the kit from Siemens had higher FVIII activity levels in the normal sample (median 1.02, IQR 0.98‐1.09 IU/mL) compared to the kit from Hyphen Biomed (median 0.94, IQR 0.88‐0.98 IU/mL).

Figure 4.

No consistent differences in factor VIII (FVIII) activity levels between mostly wide used chromogenic assays. The red dots are the results from each laboratory. The black line represents the median. The error bars represent the interquartile range. Statistical significance is indicated as *P < 0.05, **P < 0.01, *** P < 0.001

3.5. Effect of deficient plasma on FVIII activity

A possible explanation for the variation in the OSA may be variation in the behaviour of the deficient plasma. Deficient plasma was therefore also exchanged between company set‐ups. We observed that using deficient plasma from another company did not influence FVIII activity levels in samples of a moderate haemophilia A patient or in samples containing FVIII activity levels around 0.40 IU/mL FVIII (Figure 5). However, in a sample from a healthy person, Stago deficient plasma causes slightly lower FVIII results. For example, the FVIII activity level in a Siemens set‐up using Stago deficient plasma results in a FVIII level of 1.00 IU/mL, while Siemens deficient plasma resulted in 1.11 IU/mL and Werfen in 1.09 IU/mL FVIII. More importantly, results obtained with Werfen equipment, were in general lower compared to FVIII results acquired from Stago and Siemens equipment. The average FVIII activity of the normal sample measured with Werfen equipment was 0.86 IU/mL while this was 1.08 IU/mL for Stago and 1.07 IU/mL for Siemens. This experiment shows that not only FVIII deficient plasma but other causes may have an effect on the variation in FVIII measurement.

Figure 5.

Exchange of deficient plasma into a system set‐up with equipment of another company does not change the factor VIII (FVIII) activity levels. Deficient plasma was exchanged and used in the one‐stage assay (OSA) set‐up of another company. Samples measured with Werfen equipment had lower FVIII activity levels compared to samples measured with Siemens or Stago. Triangles represent FVIII activity levels measured with a deficient plasma from Werfen. Squares represent FVIII activity levels measured with a deficient plasma from Stago. Circles represent FVIII activity levels measured with a deficient plasma from Siemens

3.6. Differences in calibrator

The influence of the calibrator was determined by measuring the FVIII activity in each calibrator and comparing the measured FVIII activity value to the assigned value from the manufacturer, based on the WHO international standard. The FVIII levels in both the STA‐Unicalibrator and the HemosIL calibrator plasmas were measured in duplicates on the Siemens set‐up as described in Table 1. The assigned calibration value was 1.10 and 0.98 IU/mL for the STA‐Unicalibrator and the HemosIL, respectively, while the measured FVIII activity levels of these calibrators were 1.21 and 1.12 IU/mL. As these values differed from the assigned value, it may be that the calibrator is one of the causes that results in the variation in FVIII activity measurements.

4. DISCUSSION

The aim of this study was to quantify and understand in more detail the variation in FVIII activity measurements when testing by OSA and CSA in surveys conducted by the ECAT external quality control. We showed that the CV in FVIII measurements has an inverse relationship with FVIII activity levels. In addition, measurements performed with OSA from the Werfen package showed lower FVIII activity levels compared to measurements with the Stago and Siemens package. The explanation may be due to differences in assigned values to the calibrator.

The results of this study showed that the variation between laboratories is higher when FVIII activity levels are lower, both in the OSA and CSA. These results are consistent with the results by Verbruggen et al12 in 2008, who also showed a J‐shaped relationship between FVIII activity levels and CV, for FVIII results predominantly from the OSA. In their study, the CV increased strongly below 0.20 IU/mL with a maximal CV between 30% and 40%. Our study demonstrated much higher CVs with a maximum of 121%. This may be due to the fact that Verbruggen et al showed the CVs for samples with FVIII activity levels between 0.10‐0.20 IU/mL and not lower. Furthermore, it may be that that haemophilia treatment centres may be more accurate in general and may more often perform both OSA and CSA. A subanalysis was performed comparing the variability of the two assays with the data from centres carrying out both assays, and no difference in CV was observed (Figure [Link], [Link]). The CV increases substantially in samples with low FVIII activity levels (Figure 1), although absolute differences in FVIII activity levels remain small. Therefore, it is important to realise, that although these differences are small, they have significant clinical consequences as early initiation of prophylactic treatment is largely dependent on test results and subsequent classification of haemophilia severity.

Factor VIII activity measurements were slightly lower when measured with products from Werfen, but statistically significant. It was impossible in the ECAT surveys to evaluate the cause of this lower FVIII activity by evaluating each component of the OSA separately, as laboratories often utilise calibrator, activator, deficient plasma and equipment from one manufacturer. We attempted to specify the cause of this variation in FVIII measurements by evaluating deficient plasmas from different companies (Figure 5) in separate experiments. No consistent differences were observed when exchanging deficient plasma, for example, deficient plasma from Stago in a Siemens set‐up. Despite the fact that small differences were found, results should be interpreted with caution. In general, a small amount of factor concentrate may still be present in plasma samples derived from severe haemophilia A patients due to prior treatment and an insufficient wash out period, thus influencing FVIII activity levels. In addition, the metrological traceability is only based on a consensus model and no golden standard is available for FVIII measurements. This again raises the question how to perform haemophilia classification based on the measured FVIII levels as it is still unclear which FVIII activity assay is most optimal.

Another cause for the variation in OSA FVIII measurements may be the calibrator. As we found a higher FVIII activity value of the Werfen calibrator in the Siemens set‐up, 1.12 IU/mL instead of the assigned 0.98 IU/mL, this may lead to an underestimation of FVIII levels in the Werfen package, explaining the lower FVIII activity results that we have observed. However, as previously mentioned, we do not know the true values. It is important to realise that despite the fact that companies calibrate their reference material against plasma FVIII international standards, differences may still be present in FVIII values between the various test systems.

Several other hypothetical explanations exist which may explain variation in both assays. Firstly, of course, preanalytical variables may influence the measurements.18, 19 However, in the ECAT surveys, these preanalytical variables are not applicable as all laboratories receive the same lyophilized plasma sample. Nevertheless, differences in dissolving lyophilized plasma may also be considered a preanalytical variable. Secondly, variation in characteristics of different batches of reagents, deficient plasmas and calibrators may also cause differences in FVIII activity levels. In the ECAT surveys, many different lot numbers were used by the different laboratories, and therefore, we do not expect that typical properties of a single lot will be able to influence the results from the ECAT surveys. Finally, previous studies have shown that some activators (STA Cephascreen [Stago] and Actin FS [Siemens]) are not optimal in diagnosing severe haemophilia A patients which may also have influenced the FVIII activity levels found in this study.12

High between‐laboratory CVs may influence diagnoses of haemophilia A patients between hospitals as reported previously.4 Already, small absolute differences in FVIII activity may result in misclassification and suboptimal treatment. This emphasizes the importance of the following three aspects in haemophilia management (a) performance of other relevant tests such as DNA mutation analysis aid in classification as well as repeated testing, taking lowest levels as basis for treatment; (b) adjustment of treatment is obligatory when test results do not correspond with clinical symptoms; and (c) treatment of haemophilia patients in certified and specialized centres in which (paediatric) haematologists specialized in rare bleeding disorders and the diagnostic criteria and clinical presentation of these disorders is of utmost importance. Laboratories should also be aware that incorrect patient diagnosis is still possible despite excellent analytical performance in quality control surveys. In addition, to reduce the large between‐laboratory CV both in the OSA and CSA, standardization is required for example by an external quality control as the ECAT foundation. Current developments in method harmonization may also reduce the large between‐laboratory variability.

In conclusion, FVIII activity levels are negatively associated with CV for both the OSA and CSA. The variation in the OSA may be attributed to the different components used in current FVIII assays. As no golden standard is available for FVIII measurements, it is not possible to judge which result is superior. Future studies focusing on standardization of FVIII measurements and in‐depth education on available tests are required to further improve haemophilia diagnosis and patient management.

DISCLOSURES

F.W.G. Leebeek received research support from CSL Behring and Shire for performing the Willebrand in the Netherlands (WiN) study, and is consultant for UniQure, Novo Nordisk and Shire, of which the fees go to the institution. M.H. Cnossen has received unrestricted research/educational and travel funding from the following companies: Pfizer, Baxter, Bayer Schering Pharma, CSL Behring, Novo Nordisk, Novartis and Roche, and serves as a member on steering boards of Roche and Bayer of which fees go to the institution. M.P.M. de Maat is a member of the supervisory board of the ECAT foundation and received unrestricted research/educational funding from Siemens, Werfen and Stago. The remaining authors stated that they had no interests which might be perceived as posing a conflict or bias.

AUTHOR CONTRIBUTIONS

I. van Moort and M.P.M. de Maat were responsible for protocol development and data analysis, and were main authors of the manuscript. P. Meijer supplied data from the ECAT foundation. D. Priem‐Visser was also responsible for protocol development and data collection. M.P.M. de Maat supervised the study with M.H. Cnossen. P. Meijer, F.W.G. Leebeek, N.C.V. Péquériaux and A.J. van Gammeren provided critical guidance during the project. All authors substantially contributed to the writing and critically revised the manuscript, with approval of the final draft.

Supporting information

 

 

 

van Moort I, Meijer P, Priem‐Visser D, et al. Analytical variation in factor VIII one‐stage and chromogenic assays: Experiences from the ECAT external quality assessment programme. Haemophilia. 2019;25:162–169. 10.1111/hae.13643