Reliability of International Normalized Ratio Results in the CoaguChek Pro II System in a Clinical Setting

Abstract

The National Institute of Cardiology has previously used the CoaguChek^® XS Plus system (Roche Diagnostics International Ltd), comparing capillary blood prothrombin time/international normalized ratio (PT/INR) results with those obtained using BCS-XP/Thromborel (Siemens). We assessed the reliability of PT/INR results using the third-generation CoaguChek Pro II system, the CoaguChek XS Plus system, and cobas^® t 411 for citrated plasma analysis. Venous and capillary PT/INR were measured (N = 204). Spearman's correlation, Bland-Altman, and concordance analysis between methods were conducted. Spearman's correlation coefficients between venous/capillary INR were high for CoaguChek Pro II versus CoaguChek XS Plus (r = 0.994), CoaguChek Pro II versus cobas t 411 (r = 0.967), and CoaguChek XS Plus versus cobas t 411 (r = 0.968). Good concordance was observed among capillary methods (concordance coefficient [κ] = 0.888) and remaining relationships (P < .001 for all): cobas t 411 versus CoaguChek XS Plus (κ = 0.696) and cobas t 411 versus CoaguChek Pro II (κ = 0.684). In conclusion, good agreement was observed between CoaguChek Pro II, CoaguChek XS Plus, and cobas t 411.

Introduction

For more than 50 years, vitamin K antagonists (VKAs) have been the cornerstone of anticoagulation therapy; in recent years, new therapies have become available, collectively known as direct oral anticoagulants or non-VKA oral anticoagulants. ¹ Despite these new treatment options, VKAs are frequently prescribed. ¹ Patients may be prescribed VKAs, rather than direct oral anticoagulants, due to instances where the latter treatment option is unsuitable for certain patients (eg, patients with mechanical mitral or aortic valve prostheses, patients with thrombotic antiphospholipid antibody syndrome, and patients with primary thrombophilia) and cost limitations.^2,3

Oral anticoagulant clinics provide treatment to patients who require VKA oral anticoagulants to maintain the patient for as long as possible with international normalized ratio (INR) values within the therapeutic range for greater than 50% of the time; a value that can reach 60% to 70% in developed countries. ⁴ Many variables influence the stability of the therapeutic INR value, including diet, adherence to treatment, concomitant use of multiple drugs, and cardiac conditions. Managing patients in oral anticoagulant clinics simplifies aspects of care including the control and standardization of prothrombin time (PT), INR calculation, therapy control frequency, the education of patients and their families, and communication between clinical staff and patients.

Handheld point-of-care (POC) devices have evolved as a result of miniaturization of traditional clinical laboratory instruments and procedures. ⁵ Handheld POC systems have demonstrated efficacy for anticoagulant treatment monitoring in elderly patients ⁶ and pediatric patients,^7,8 as well as in clinics staffed with well-trained personnel but lacking specialized laboratory professionals and where patients are supervised by specialists in remote centers thanks to telemedicine tools.

The Oral Anticoagulant Clinic of the National Institute of Cardiology Ignacio Chávez has over 6000 registered patients. The test for PT is a functional test that depends on thrombin generated from Factor VII activation by an exogenous thromboplastin; in the clinical laboratory, the clock stops with fibrin formation. Since 2011, the Clinic has used the CoaguChek^® XS Plus system (Roche Diagnostics International Ltd, Rotkreuz, Switzerland), which, according to an assessment conducted in more than 300 patients (data not published), was shown to provide fast and reliable results. The latest device, the CoaguChek Pro II system, is a rapid PT/INR test platform which uses reagent strips and can store up to 1000 independently identified results. Both devices were developed for clinical use. Each test strip contains an on-board humidity and temperature control system (resazurin), allowing the rejection of test strips that have undergone drastic humidity or temperature changes and may therefore provide erroneous results. The system has an internal quality control procedure, which is recommended to be carried out after every 20 tests. ⁹ Like previous-generation devices, the handheld POC CoaguChek Pro II system uses an electrochemistry method to detect thrombin generation. A constant current flow is established between a reference electrode and a working electrode as a result of an oxide-reduction reaction (in this case, by oxidation of thrombin-induced phenylenediamine). Simultaneously, the time is recorded; the time elapsed is converted via an algorithm into customary coagulation units. ¹⁰ Therefore, the device provides a PT result in seconds and the respective INR calculation. Essentially, the CoaguChek XS Plus system and the CoaguChek Pro II system use the same technology; however, the CoaguChek Pro II system has additional properties, such as an activated partial thromboplastin time (aPTT) test, an integrated 2D barcode reader for patients, user and quality control identification, wireless connectivity, and a quick response (QR) code function.

Clinical PT and INR monitoring allows for early detection of treatment deviations and the ability to take action before the occurrence of an adverse event associated with oral anticoagulant use. Various statistical strategies can be used to evaluate the impact that changes in methodology have on clinical decision-making. ⁷ The objective of laboratory equipment evaluation is to verify that clinical criteria are not significantly modified by the introduction of new procedures, and that decisions made regarding equipment do not affect the efficiency and safety of the treatment. ¹¹ Since PT/INR results are continuous variables, classical bivariate correlation calculation is recommended in the literature, as is the use of Bland-Altman plots, which provide a superior concordance value compared with simple correlation, as well as the constant κ.^12–15

The purpose of this study was to evaluate the reliability of the handheld POC CoaguChek Pro II system, compared with both the handheld POC CoaguChek XS Plus system and the automated cobas^® t 411 analyzer with PT Screen thromboplastin (Roche Diagnostics International Ltd). In evaluating this, we aimed to verify whether adequate results were obtained on the latest device, the CoaguChek Pro II system.

Materials and Methods

Study Population

Voluntary participation was requested from 202 patients on long-term anticoagulation therapy with acenocoumarol, who attended The Oral Anticoagulant Clinic of the National Institute of Cardiology Ignacio Chávez in a 1-month period. Of the 204 patients, 186 patients had INR values <8.0 and 18 patients had INR values >8.0 (considered to be a critical INR value) based on the POC system in use (CoaguChek XS Plus system). Patients with critical INR values were treated according to the “Critical INR Procedure” of the hematology laboratory.

Ethical Approval

According to the regulations of the National Institute of Cardiology Ignacio Chávez, this study was classified as an exercise to improve medical practice rather than a research protocol. As such, the approval of the Institutional Research or Ethics Committees was not required. All patients participated on a voluntary basis and provided written, informed consent prior to inclusion in the study.

Assessments

Capillary puncture was performed with disposable safety lancets with automatic retraction (BD Microtainer contact-activated lancet; Becton Dickinson and Co., Dublin, Ireland); the puncture was performed on the lateral side of the middle, central, or index fingers. The collected capillary blood was then placed on the appropriate area of the test strip of the capillary PT testing devices (the CoaguChek Pro II system and the CoaguChek XS Plus system) for processing. The order of testing on the CoaguChek Pro II system and the CoaguChek XS Plus system was alternated between patients. All tests were validated by the humidity and temperature controls within each test strip.

In the same session, a venous sample was obtained using a vacuum collection system in tubes containing 3.2% sodium citrate, with final anticoagulant to whole blood ratio of 1:9. Within 1 h, samples were centrifuged at 1500g for 15 min at room temperature, and an aliquot of plasma was used for the measurement of PT, aPTT, thrombin time, and fibrinogen tests using the cobas t 411 analyzer.

All testing was carried out by trained laboratory personnel (clinical chemists and laboratory technicians). Approximately 20 tests were conducted per day using the capillary PT testing devices and venous blood samples.

Four CoaguChek Pro II devices were used, and their results compared with those obtained from the cobas t 411 analyzer and the CoaguChek XS Plus system. This manuscript reports the PT and INR results, where INR in plasma is the geometric mean of the PT results raised to the power of the international sensitivity index provided by the manufacturer.

To ensure that measurements on the cobas t 411 analyzer were reliable and accurate, a verification phase was performed prior to the initiation of the study. PT and aPTT values were measured in 2 control samples (normal and abnormal), and preliminary precision and total precision were calculated. For the normal control sample, the coefficients of variation (CVs) for preliminary precision and total precision were 1.40% and 1.26%, respectively. For the abnormal control samples, the CVs for preliminary precision and total precision were 1.90% and 1.07%, respectively. Additionally, the PT test was monitored using 2 additional controls (normal and abnormal). The normal control had a commercially-assigned average value of 11.9 s; under our laboratory conditions, the mean value was 12.3 s (CV: 2.42%; standard deviation [SD]: 0.3 s). The abnormal control had a commercially-assigned average value of 39.9 s; under our laboratory conditions, the mean value was 38.1 s (CV: 2.87%; SD: 1.09 s).

To assess the reliability and accuracy of the CoaguChek XS Plus system, between-run precision was assessed prior to initiation of the study using 2 controls (normal and abnormal). The normal control had a mean INR value of 1.22 (CV: 3.24%) and the abnormal control had a mean INR value of 3.15 (CV: 2.10%). Moreover, a mean value of ∼12 s has been established by the manufacturer for normal samples, which corresponds to an INR value of 1.0.

Statistical Analysis

Spearman's correlation coefficients for comparisons between the following measuring devices were calculated: the CoaguChek Pro II system versus the cobas t 411 analyzer; the CoaguChek Pro II system versus the CoaguChek XS Plus system; and the CoaguChek XS Plus system versus the cobas t 411 analyzer. Bland-Altman plots were created and concordance coefficients (κ) obtained. According to Cerda et al, ¹⁶ a κ statistic value of 0 has a poor degree of concordance; κ = 0.01 to 0.20 has a slight degree of concordance; κ = 0.21 to 0.60 has a moderate degree of concordance; κ = 0.61 to 0.80 has a good degree of concordance; and κ > 0.81 has an almost perfect degree of concordance.

The correlations between the measuring devices (CoaguChek Pro II system vs the cobas t 411 analyzer; the CoaguChek Pro II system vs the CoaguChek XS Plus system; and the CoaguChek XS Plus system vs the cobas t 411 analyzer) were further analyzed by stratifying the patients according to their INR values, as measured using the cobas t 411 analyzer: sub-anticoagulated (INR <2.0); anticoagulated (INR 2.0 to 3.5); and over-anticoagulated (INR >3.5). Spearman's correlation coefficients between the measuring devices were calculated for each group.

The results were analyzed using Statistical Product and Service Solutions 17.0 (IBM, Chicago, IL, USA) and the MedCal application.

Results

Samples from 186 patients with INR values <8.0 (according to POC results) were used to analyze the relationships of the INR results obtained using the CoaguChek Pro II system, the CoaguChek XS Plus system, and the cobas t 411.

The correlations between the venous and capillary INR values, measured by the CoaguChek Pro II system versus the CoaguChek XS Plus system and the CoaguChek Pro II system versus the cobas t 411 analyzer, are shown in Figure 1. For lower INR values (INR ≤3.5), all relationships showed a similar trend, and all methods approximated the regression line with highly homogeneous behavior. For higher INR values (INR >3.5), the data points were scattered more broadly. Despite this distribution, Spearman's r values were >0.76 in all instances.

Figure 1.

Correlation (center line) and confidence intervals (outer lines) between (a) the CoaguChek Pro II system versus the CoaguChek XS Plus system INR (capillary method vs capillary method); (b) the CoaguChek Pro II system versus the cobas t 411 analyzer INR (capillary method vs venous method); N = 186. Abbreviation: INR, international normalized ratio.

Figure 2 shows the correlation between the CoaguChek Pro II system and the cobas t 411 analyzer in (a) patients with an INR <2.0 (sub-anticoagulated); (b) patients with an INR 2.0 to 3.5 (anticoagulated); and (c) patients with an INR >3.5 (over-anticoagulated). For all 3 groups, the data points were evenly distributed along the adjusted straight lines. Notably, there was good correlation between the CoaguChek Pro II system and the cobas t 411 analyzer in patients with an INR 2.0 to 3.5, with only 1 outlier. Corresponding Bland-Altman plots according to the degree of anticoagulation are shown in Figure 3, with the single outlier (±2SD) in the anticoagulated group clearly visible; this patient was monitored using the CoaguChek XS Plus system going forward. In general, very few values obtained using the CoaguChek XS Plus system and the CoaguChek Pro II system presented significant differences compared with the values obtained with the cobas t 411 analyzer. This was confirmed by results shown in Figure 4, in which Bland-Altman plots for all patients regardless of INR value showed that the overall differences between venous and capillary methods remained within the pre-specified dispersion criteria when INR was <3.5, while the greatest differences were shown at non-therapeutic ranges, when INR was >3.5.

Figure 2.

Correlation between the CoaguChek Pro II system and the cobas t 411 analyzer INR in (a) patients with an INR <2.0 (sub-anticoagulated); (b) patients with an INR 2.0 to 3.5 (anticoagulated); (c) patients with an INR >3.5 (over-anticoagulated). Abbreviation: INR, international normalized ratio. *INR reference measured in venous plasma on the cobas t 411 analyzer.

Figure 3.

Bland-Altman or difference plots in (a) patients with an INR <2.0 (sub-anticoagulated); (b) patients with an INR 2.0 to 3.5 (anticoagulated); (c) patients with an INR >3.5 (over-anticoagulated), for the venous method (cobas t 411 analyzer) versus capillary method (left panels: the CoaguChek XS Plus system; right panels: the CoaguChek Pro II system). Abbreviations: ACO, anticoagulated; INR, international normalized ratio; OverACO, over-anticoagulated; SD, standard deviation.

Figure 4.

Bland-Altman or difference plots of the cobas t 411 analyzer (venous method vs (a) the CoaguChek XS Plus system and (b) the CoaguChek Pro II system (capillary INR). The vertical red dotted line indicates the INR value (3.5) beyond which anticoagulation is necessary to reduce the risk of bleeding. Abbreviations: INR, international normalized ratio; SD, standard deviation.

Given the distribution of the variables, Spearman's correlation coefficients were calculated and highly significant values were obtained (P < .001; Table 1). Furthermore, the κ value between the CoaguChek Pro II system versus the CoaguChek XS Plus system showed very good concordance (κ = 0.888). Good concordance was also observed between the CoaguChek XS Plus system and the cobas t 411 analyzer (κ = 0.696), and the CoaguChek Pro II system versus the cobas t 411 analyzer (κ = 0.684), all with P < .0001.

Table 1.

Spearman's Correlation Coefficients Between INR Values Measured by the CoaguChek Pro II System Versus the Cobas t 411 Analyzer, the CoaguChek Pro II System Versus the CoaguChek XS Plus System, and the CoaguChek XS Plus System Versus the Cobas t 411 Analyzer.

	All patients INR (N) P-value	Patients with INR <2.0* (N) P-value	Patients with INR 2.0 to 3.5* (N) P-value	Patients with INR >3.5* (N) P-value
CoaguChek Pro II versus cobas t 411	0.967 (186) P < .001	0.920 (52) P < .001	0.876 (108) P < .001	0.812 (26) P < .001
CoaguChek Pro II versus CoaguChek XS Plus	0.994 (185) P < .001	0.932 (52) P < .001	0.984 (108) P < .001	0.764 (26) P < .001
CoaguChek XS Plus versus cobas t 411	0.968 (185) P < .001	0.967 (52) P < .001	0.879 (108) P < .001	0.780 (25) P < .001

Abbreviation: INR, international normalized ratio.

*INR reference calculated from the prothrombin time measured in venous plasma on the cobas t 411 analyzer.

Among the 18 patients with an INR >8.0 tested with the CoaguChek Pro II system, 10 (55.6%) had INR values >8.0 as determined by the venous method, while the remainder (44.4%) achieved INR 6.1 to 7.8, well above the therapeutic INR.

The results of the internal quality control assessments carried out on the cobas t 411 analyzer are shown in Supplemental Table 1. The SDs and CVs were within pre-specified criteria, based on the manufacturer's package insert, for all PT and aPTT measurements.

Discussion

This study investigated the relationship between the capillary INR to be introduced (INR CoaguChek Pro II system), the previous capillary INR (INR CoaguChek XS Plus system), and the reference method (cobas t 411 analyzer).

The correlation decreased as the INR value increased. This pattern was expected as the thromboplastin used in the CoaguChek test strip is recombinant, and its behavior with respect to tissue thromboplastins shows greater sensitivity to the reduction of vitamin K-dependent factors, mainly Factor VII, producing progressively longer results compared with tissue thromboplastins.^17–19

To examine this relationship further, the patients were stratified into 3 groups according to their INR. The correlation between the CoaguChek Pro II system and the cobas t 411 analyzer was strong in all 3 groups (sub-anticoagulated, anticoagulated, and over-anticoagulated), implying that the CoaguChek Pro II system can recognize whether patients are or are not within the therapeutic range. Few of the values obtained using either of the capillary POC systems presented a significant difference from the values obtained using the cobas t 411 analyzer. This was despite the presence of fewer patients in the sub-anticoagulated and over-anticoagulated groups compared with the anticoagulated group. These data are in alignment with the concordant κ values obtained for each anticoagulation interval between the cobas t 411 analyzer and the 2 POC systems.

It is important for INR to be accurately measured within the therapeutic interval, as patients with an INR 2.0 to 3.5 are considered “anticoagulated” and the clinician would not make any adjustments to the anticoagulant dose. The CoaguChek Pro II system INR measurements closely follow those of the cobas t 411 analyzer in the therapeutic interval. Within this group of patients classified as anticoagulated within the optimal INR range, one outlier was observed, with a significant difference between the values obtained using both POC systems and using the cobas t 411 analyzer. Neither the operator of the cobas t 411 analyzer nor those who operated the 2 POC systems were aware of the other values obtained, and no repetition of any of the tests was conducted. In this case, the value reported to the patient was that obtained from the INR CoaguChek XS Plus system, which was the standard test used at that time in the clinic.

When a patient has INR >4.0, the clinician should: review the factors that led to an increase in INR value, consider decreasing the anticoagulant dose, and suggest that the patient makes a clinical visit within several days to evaluate the impact of any treatment alterations on the INR value; clinical actions are required as the risk of bleeding increases up to 5-fold at INR values >4.5. ²⁰ Therefore, it is clear that clinical judgment would not be modified despite the potential difference in INR due to the use of the capillary versus venous method in INR values >4.0. Furthermore, the differences found between methods are within an acceptable range, with the greatest differences found only at higher INR values.

Concordance analysis and κ value in INR results obtained by handheld POC CoaguChek Pro II and CoaguChek XS Plus systems, compared with the venous system, provide support for the use of the third-generation capillary system. In general, the criteria for classifying patients as anticoagulated (INR 2.0 to 3.5), sub-anticoagulated (INR <2.0), or over-anticoagulated (INR >3.5) do not change; despite the κ value with the cobas t 411 analyzer, concordance was considered good, and low-impact differences between methods are to be expected.

Of the patient samples for which the CoaguChek Pro II system obtained an INR result of >8.0 (a critical value) in capillary blood, all were found to have an INR >6.0 in the plasma test. Since any patient with an INR >3.5 requires a clinical evaluation to reduce the risk of bleeding, the ability of the POC system to facilitate the recognition of patients at high risk for hemorrhage (INR >6.0), and for whom reversal measures are urgently needed, is extremely useful. POC systems for PT/INR testing have been a valuable tool in the management of patients treated with anticoagulants at The Oral Anticoagulant Clinic of the National Institute of Cardiology Ignacio Chávez. However, due to the cost of testing, the benefit of their application in resource-limited settings must be demonstrated, and cost analysis should take into account aspects including travel expenses and work absence for patients and caregivers. ²¹

Importantly, in the coagulation field, portable devices that function by electrochemistry methods, which support the care of patients being treated with VKAs, are useful in critical care units requiring fast and reliable results for timely clinical decision-making. In addition, it is necessary to have a clinical laboratory that covers process control features including internal quality control, adherence to international guidelines, membership in external quality control programs, and other requirements that allow close monitoring of quick methods.^22,23 This is especially important considering the impact of PT/INR results on the management of patients prescribed high-risk drugs such as VKA oral anticoagulants.

The CoaguChek Pro II system has now been incorporated into daily use within our clinic. In blood samples with a very high hematocrit (>55%), an error message instead of a value may be generated, which can be useful in alerting the treating physician to the increased production of erythrocytes in the patient. Alternatively, the error message can indicate a potential need for adjustment of the anticoagulant in the sample collection tubes prior to INR measurement, which can be confirmed via the direct method by measuring the hematocrit; this could prevent dosing decisions based on incorrect results from anticoagulant-patient plasma mismatch. ²⁴

Similar studies have assessed INR measured using POC systems versus INR measured using the venous method. Roco et al ²⁵ compared INR values obtained using the CoaguChek Pro II system versus the venous method; good correlation was observed (Spearman's correlation coefficients, r = 0.953 and r = 0.962). The study also compared INR values obtained using the CoaguChek Pro II system versus the Xprecia system (Siemens Healthcare, Malaysia) and the microINR system (iLine Microsystem, Donostia, Spain); Spearman's correlation coefficients were r = 0.953 and r = 0.962, respectively. Likewise to our study, superior correlation values were obtained between POC systems compared with a POC system versus the venous method. Notably, our study had a higher proportion of INR values >3.5.

Moreover, Moiz et al ²⁶ compared INR values obtained using the CoaguChek XS Pro system (comprising recombinant thromboplastin), which is a POC system from the CoaguChek family that was not evaluated in this study, versus INR values measured in venous plasma; good concordance was observed (κ = 0.916). Stronger correlations, and therefore κ values, are anticipated when the thromboplastins of the same origin are compared, as opposed to comparing tissue versus recombinant thromboplastin; this explains the lower κ value obtained in our study (κ = 0.696), which is still considered satisfactory and reliable. It is also important to mention that the agreement between devices/methods is anticipated to become more discordant as the INR value increases, ¹⁹ as evident from the Bland-Altman analysis in the over-anticoagulated group in our study.

The results discussed herein demonstrate that the CoaguChek Pro II system can be considered a reliable POC device for measuring INR, particularly when INR values are <3.5.

Abbreviations

ACO

anticoagulated

aPTT

activated partial thromboplastin time

CV

coefficients of variation

INR

international normalized ratio

κ

concordance coefficient

OverACO

over-anticoagulated

PT

prothrombin time

POC

point-of-care

QR

quick response

SD

standard deviation

VKA

vitamin K antagonist.