Prospective evaluation of heparin-induced thrombocytopenia expert probability and 4T scores in Chinese patients with suspected heparin-induced thrombocytopenia

Supplemental Digital Content is available in the text

Abstract

Background

Diagnosis of heparin-induced thrombocytopenia (HIT) is challenging. This study aimed to compare the diagnostic performance of HIT expert probability (HEP) and 4T scores, and to evaluate the inter-observer reliability for the 4T score in a clinical setting.

Methods

This prospective study included HIT-suspected patients between 2016 and 2018. Three hematologists assessed the HEP and 4T scores. Correlations between scores and anti-platelet factor 4 (anti-PF4)/heparin antibodies were evaluated. Receiver operating characteristic curves and area under the curve (AUC) were used to assess the predictive accuracy of these two scoring models. The intraclass correlation coefficient (ICC) was used to assess the inter-observer agreement of 4T scores between residents and hematologists.

Results

Of the 89 subjects included, 22 (24.7%) were positive for anti-PF4/heparin antibody. The correlations between antibody titer and either HEP or 4T scores were similar (r = 0.392, P < 0.01 for the HEP score; r = 0.444, P < 0.01 for the 4T score). No significant difference in the diagnostic performance was displayed between these two scores (AUC for the HEP score: 0.778 vs. AUC for the 4T score: 0.741, P = 0.357). Only 72 4T scores were collected from the residents, with a surprisingly low percentage of observers (43.1%) presenting the four individual item scores which made up their 4T score. The AUC of 4T score assessed by residents and hematologists was 0.657 (95% confidence interval [CI]: 536–0.765) and 0.780 (95% CI: 0.667–0.869, P < 0.05), respectively. The ICC of 4T score between residents and hematologists was 0.49 (95% CI: 0.29–0.65, P < 0.01), demonstrating a fair inter-observer agreement.

Conclusions

The HEP score does not display a better performance for predicting HIT than the 4T score. With the unsatisfactory completion rate, the inter-observer agreement of 4T score in a tertiary hospital is fair, underscoring the necessity for continuing education for physicians.

Introduction

Heparin-induced thrombocytopenia (HIT) is a prothrombotic and potentially fatal complication of heparin treatment caused by anti-platelet factor 4 (anti-PF4)/heparin antibodies of immunoglobulin G (IgG) class, affecting approximately 0.1% to 5% of patients receiving unfractionated heparin (UFH) or low-molecular-weight heparin (LMWH).^[1–3] As heparin is widely administered in clinical practice, HIT is still a reasonable differential diagnosis for patients with thrombocytopenia and heparin exposure.^[4]

The diagnosis of HIT is still challenging, especially in complicated and critically ill patients. In virtually all situations, physicians must make a primary clinical decision while waiting for the results of anti-PF4/heparin antibody test. Functional assays are the most accurate diagnostic tests, but they are time-consuming, technically complex, and expensive.^[5,6] Immunoassays, detecting both pathogenic and non-pathogenic antibodies, generally have high sensitivities and low specificities, resulting in the overdiagnosis of HIT.^[7] Although the performance of IgG-specific assays has improved, the potential for overdiagnosis still remains.^[7,8] Nevertheless, in several developing countries, including China, neither screening immunoassays nor specific functional tests are generally available, underscoring the importance of pre-test scoring systems to limit overdiagnosis.

The 4T and HIT expert probability (HEP) scores are the two principal scoring systems for HIT. The 4T score includes four variables (thrombocytopenia, the timing of platelet fall, thrombosis, and other causes of thrombocytopenia) and classifies patients as having a low, intermediate, or high risk of HIT.^[9] The HEP score, based on expert opinion, was proposed in 2010.^[10] It comprises eight clinical and biological criteria with corresponding positive and negative points. Both scores, but especially the 4T score, have a very high sensitivity, being capable of ruling out HIT in low-risk individuals.^[11] They both demonstrate good inter-observer agreements in their initial assessment. However, the reliability of HEP score in comparison to the 4T score remains uncertain. In previous studies, pre-test scoring systems were used retrospectively by hematologists and clinicians with expertise in HIT diagnosis.^[9–13] The determination of scores in these studies does not reflect real-life clinical practice.

This study was performed to validate the diagnostic performance of HEP score compared with the 4T score in heterogeneous patient populations from China, and to evaluate the inter-rater reliability for the 4T score in a clinical setting.

Methods

Ethical approval

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Peking Union Medical College Hospital (No. S-T369). Informed written consent was obtained from all patients or their guardians for the children prior to their enrollment in this study.

Study design

This was a single-center, prospective, observational study on HIT-suspected patients in the real-life setting of a tertiary hospital.

The sample size calculation was performed using MedCalc version 18.2.1 (MedCalc Software, Mariakerke, Belgium) based on a two-sided significance level (α) of 5% and a power (1 – β) of 80%. Referring to the findings of Cuker et al,^[10] 0.9 and 0.7 were assumed as the area under the curve (AUC) for the receiver operating curve (ROC) of HEP and 4T scores separately. Correlations in positive and negative groups were presumed to be 0.8,^[13] and the ratio of sample sizes in negative/positive groups was presumed to be 4.^[14] Then, the minimum required sample sizes for comparison of ROC curves were obtained, and at least 55 cases (11 positive and 44 negative) should be enrolled in the present study.

The HEP and 4T scores assessed by hematologists were compared, and the inter-observer reliability of 4T score between unintentionally trained frontline physicians and hematologists was examined.

Patients and samples

The records were collected, and citrated plasma from consecutive HIT-suspected inpatients was submitted to the hospital between May 17, 2016 and July 16, 2018. The inclusion criteria were the use of UFH or LMWH and the presence of thrombocytopenia or platelet count fall. The exclusion criteria included repeated tests, loss of sample, and chronic hemodialysis (as these patients had a rather high risk of developing asymptomatic anti-PF4/heparin antibodies^[15]) [Figure 1].

Figure 1.

Flow chart of the study. PF4: Platelet factor 4.

Clinical assessment

One hematologist (LS) and two HIT experts (SJW and YQZ) reviewed the clinical information of each participant and rated it independently using the 4T score [Table S1]^[13] and HEP score [Table S2].^[10] Each of the three hematologists produced independent scores for each given patient and discussed the results to achieve a final consensus.

For assessing the inter-observer agreement of 4T score, a structured questionnaire was given to every resident who was in charge of HIT-suspected patients but lacked intentional training.

The two scoring systems were determined before the antibody test.

Assay for anti-PF4/heparin antibody

The IgG-specific anti-PF4/heparin antibody was detected using the PF4 IgG enzyme-linked immunosorbent assay (ELISA) kit (GTI Diagnostics, Waukesha, WI, USA) according to the manufacturer's guidelines. Optical density as A_{(405-490 nm)} was recorded, using a cutoff value of 0.4.

Statistical analysis

Patients were classified into two groups on the basis of ELISA A_{(405-490 nm)}: HIT positive as A_{(405-490 nm)} ≥0.40 and HIT negative as A_{(405-490 nm)} < 0.40. Continuous variables were expressed as medians with quartiles. Categorical variables were presented as percentages. Student t test or Mann–Whitney U test was used to detect differences between continuous normal and non-normal variables, respectively, and the Chi-squared test was used to detect differences between categorical variables. A value of P ≤ 0.05 was considered to be statistically significant. Pearson product-moment correlation coefficient (r) was used to quantify the correlation between the scoring models and the ELISA A_{(405-490 nm)}. The ROC curve analysis was used to compare the diagnostic performance of scoring models for predicting IgG ELISA-based HIT. The intraclass correlation coefficient (ICC) was used to assess the inter-observer agreement between residents and hematologists. Statistical parameters were calculated mainly using the SPSS version 23.0 (IBM, Armonk, NJ, USA), with MedCalc version 18.2.1 (MedCalc Software, Mariakerke, Belgium) being used for the ROC curve and ICC analyses.

Results

Patients

The study included 89 HIT-suspected patients between 2016 and 2018. The demographic and clinical data of patients are summarized in Table 1. Most of the participants were admitted to the medical department (53.9%) or intensive care unit (ICU) (25.8%). The IgG anti-PF4/heparin antibody was positive in 22 patients (24.7%), with a median A_{(405-490 nm)} of 2.442. HIT-positive subjects were significantly older than HIT-negative ones and developed more thrombosis.

Table 1.

Clinical characteristics of 89 patients suspected with HIT.

Correlation between the scoring models and the HIT ELISA

Figure 2 shows the relationship between the clinical scoring models and the HIT ELISA. The patients were classified as HIT-positive or HIT-negative based on the result of the ELISA assay, with a cutoff of 0.4. The median 4T and HEP scores of HIT-positive individuals were both significantly greater than those of HIT-negative ones (5 vs. 3, Ζ = −3.47, P < 0.01 for the 4T score; 7 vs. 2, Ζ = −3.92, P < 0.01 for the HEP score) [Figure 2A].

Figure 2.

Relationship between scores and HIT ELISA. (A) Distribution of scores according to ELISA with a cutoff of 0.4. Boxes and whiskers represent medians with quartiles and limits, respectively. (B) Distribution of scores in the two scoring models according to ELISA. Lines and error bars represent median scores and quartiles, respectively. HEP: HIT expert probability; HIT: Heparin-induced thrombocytopenia.

The HIT-positive patients were then categorized into two groups [Figure 2B]. The operating characteristics were shown to be markedly improved when the A_{(405-490 nm)} was ≥1.00, without necessarily compromising diagnostic accuracy. Therefore, an A_{(405-490 nm)} of 1.00 was selected as the boundary between the two groups.^[16,17] Significant differences in median clinical scores were found between the A_{(405-490 nm)} ≥1.00 group and the A_{(405-490 nm)} < 0.4 group. The correlation coefficient (r) between the 4T score and the A_{(405-490 nm)} was 0.444 (P < 0.01), which was similar to that between the HEP score and the A_{(405-490 nm)} (r = 0.392, P < 0.01).

Diagnostic performance of HEP vs. 4T scores

The agreement between the HEP score and the 4T score was assessed by ROC curve analysis using HIT ELISA as the standard with a cutoff of 0.4 [Figure 3A]. At an AUC of 0.778 (95% confidence interval [CI]: 0.678–0.860), the HEP score did not exhibit a better diagnostic performance compared with the 4T score (0.741, 95% CI: 0.637–0.828, P = 0.357).

Figure 3.

Receiver operating characteristic curves of scoring models for the diagnosis of HIT using HIT ELISA as the standard. (A) Agreement between HEP and 4T scores (n = 89) assessed by hematologists. (B) Agreement of 4T scores between hematologists and residents (n = 72). AUC: Area under the curve; CI: Confidence interval; HEP: HIT expert probability; HIT: Heparin-induced thrombocytopenia.

Table 2 summarizes the operating characteristics of each model at the selected cutoffs. The 4T scores of 4 and 6, which were widely used as the boundaries among low, intermediate, and high risks of HIT, were chosen as the cutoffs for screening and diagnosing, respectively. The cutoff scores of HEP values were selected as 2 and 5 to assess their diagnostic performance, as used by Cuker et al.^[10] In the present study, the cutoff of the 4T scores of two yielded specificity and positive predictive value (PPV) of 2.99% and 0.25, respectively, whereas the cutoff of the HEP scores of –2 yielded specificity and PPV of 4.48% and 0.26, respectively, to achieve 100% sensitivity for the best performance of screening.

Table 2.

The diagnostic performance of each cutoff value for heparin-induced thrombocytopenia.

Inter-observer agreement of 4T score

Although every resident was asked to complete the 4T score while ordering for the HIT antibody test, the completion percentage was not satisfactory. Only 72 questionnaires (80.9%) were completed, with unexpectedly less than half presenting the four individual items of the 4T score (n = 31, 43.1%). The agreement of 4T score assessed by residents and hematologists was evaluated by ROC curve analysis using HIT ELISA as the standard [Figure 3B]. The AUC of residents was significantly lower than that of the hematologists (0.657, 95% CI: 0.536–0.765 vs. 0.780, 95% CI: 0.667–0.869, P < 0.05).

The inter-observer agreement between the two groups of doctors was further analyzed using the ICC. The ICC (95% CI) of total score was 0.49 (0.29–0.65, P < 0.01), demonstrating a fair inter-observer agreement. Among the four individual items of 4T score, “existence of other causes of thrombocytopenia” and “timing of thrombocytopenia” achieved lower ICCs with 0.36 (0.01–0.63, P < 0.05) and 0.57 (0.28–0.77, P < 0.01), respectively, whereas “magnitude of thrombocytopenia” and “presence of thrombosis” had excellent ICCs of 0.79 (0.62–0.90, P < 0.01) and 0.80 (0.63–0.90, P < 0.01), respectively.

Discussion

Despite the low incidence of HIT in clinical practice, it is a critical medical condition with a significant morbidity and mortality burden, which needs urgent clinical decision making.^[18] Diagnosis of HIT is still challenging, especially in patients from the medical department and ICU, accounting for nearly 80% of our subjects in the present study. The first reason is that the prevalence of thrombocytopenia in medical and critically ill patients is up to 58%,^[4] and heparin is frequently prescribed for these patients. Secondly, these patients usually have more complicated clinical conditions, including multiple causes of thrombocytopenia, resulting in atypical symptoms and problems with respect to diagnosis.

Functional tests are considered to be the golden standard for HIT diagnosis. However, they are time-consuming and expensive and require experienced expert personnel. Therefore, many countries, including China, have not yet developed these tests. Even in America and Europe, only a few laboratories are using these at present.^[19] Immunoassays are more commonly used in real-life clinical practice. Nevertheless, their diagnostic performances are limited due to their relatively low specificity, leading to the overdiagnosis of HIT.^[5] By detecting the specific IgG-class anti-PF4/heparin antibody, the specificity of ELISA can be improved up to 89.9%, without necessarily compromising sensitivity.^[7] However, because of the slow test turnaround time, IgG ELISA tends to be less clinically useful for urgent clinical decisions. Under such circumstances, the clinical scoring systems show their importance by providing pretest probabilities to guide whenever a biological assay is warranted.

The HEP score gave a more extensive definition of assessment criteria compared with the 4T score, thus exhibiting theoretical advantages over the latter method. Also, the performance of the HEP score was better in one center.^[10] However, other studies (similar to the present study) found that the HEP score was not superior, even worse in some cases, than the 4T score.^[13,20,21] Also, the correlation between the anti-PF4 assay results and the corresponding HEP scores in the present study (AUC = 0.778) and the studies by Beauverd et al^[20] (AUC = 0.85), Dore et al^[21](AUC = 0.69 and 0.714), and Uaprasert et al^[13] (AUC = 0.72) were both lower than that in the original report^[10] (AUC = 0.910). A possible explanation may be related to the study population. The study with better performance from the HEP score included mainly surgical patients, whereas the other studies (such as the present study) included mainly patients from the internal medicine department and ICU. These patients usually had multiple potential causes of thrombocytopenia and varied platelet counts compared with surgical patients, which sometimes resulted in several negative variables. Further validation of the HEP score in a large prospective study is warranted before it can be routinely used in clinical practice.

Despite a high correlation between the HEP and 4T scores, the 4T score was preferred as the pre-test scoring system in this study, considering that the HEP score is complex, cannot be easily remembered and calculated, and lacks precise cutoffs and clear guidance on the interpretation of its scores. However, the inter-observer reliability of 4T scores in a real-life setting remains unexplored. The present study investigated the inter-observer agreement between hematologists and junior physicians who were front liners in prescribing the anti-PF4/heparin test.

The ICC (95% CI) of total score was 0.489 (95% CI: 0.292–0.647, P < 0.01), demonstrating a fair inter-observer agreement between the hematologists and residents. However, the completion rate was not satisfactory because <50% of the residents would like to present their individual item scores while rendering a total score. Also, the diagnostic performance of 4T assessed by residents was significantly worse than that assessed by hematologists. Beauverd et al^[20] retrospectively investigated the 4T score in the HIT-suspected patients from the internal medicine department. They also observed that the 4T score was not commonly used, with only 13% actually documented in the medical records. In a community hospital in upstate New York, Samhouri et al^[14] also observed a mere 2.4% documentation rate of 4T scores, with an overused anti-PF4/heparin antibody test of 14.6%. It was speculated that junior physicians might have insufficient knowledge of 4T score and therefore they were uncertain about their scores. The second possible explanation was that they were unaware of the importance of clinical pre-test scoring systems. Similar to the institution of Beauverd et al,^[20] the hospital information system in this study allowed the ordering of laboratory tests without a compulsory requirement for the 4T score, and physicians could achieve results without hematologic consultations. The 4T score demonstrated a high negative predictive value of a low probability score (99.8%)^[11]and hence was considered to be an ideal pre-test system to exclude HIT. It can avoid unnecessary laboratory assays and cost in low-risk patients. Several hospitals and laboratories have already implemented a compulsory pre-test scoring system with the 4T score before the use of anti-PF4/heparin assays. However, because of the reasonable inter-observer agreement between HIT experts and frontline residents, the application of the 4T score is limited. The main disagreement was in two individual items of T4 (“existence of other causes of thrombocytopenia”) and T2 (“timing of thrombocytopenia”) with ICCs of 0.36 and 0.57, respectively, which was similar to the results reported by Nagler et al^[22] (raw agreement of 62% and 55%, respectively) and Dore et al^[21] (raw agreement of 54% and 63%, respectively). The lack of a clear definition of these two items remains an issue. Differences in clinical training and experience of raters may result in different interpretations. More education may help to improve the reliability of 4T scores. Hopefully, new on-demand diagnostic tools with greater sensitivity and specificity may overcome the diagnostic problems and be widely used.^[23,24]

This study had several limitations. Firstly, it was performed in a single center with a limited number of patients. Larger-sample studies should be conducted to allow the analysis of the diagnostic performance of scoring models in patients from the surgical department, internal medical award, and ICU separately. Secondly, the inter-observer agreement between hematologists could not be calculated because of their discussion to reach a final consensus. However, the pattern represented the real-world hematologic consultation in many countries wherein on-duty fellows discuss cases with their superiors to draw a final conclusion. Last but not least, only anti-PF4/heparin assay results were available for the patients in this study, inevitably increasing the false-positive rate of HIT. However, because of the unavailability of functional assays in many institutions, using IgG ELISA as a standard may be more practical for physicians to predict the performance of clinical scoring systems.

In conclusion, the HEP score does not improve the correlation with the anti-PF4/heparin antibody compared with the 4T score in Chinese patients. The inter-observer agreement of 4T score in a real-life setting is fair, albeit with an unsatisfactory completion rate. Chinese physicians should make greater efforts and pursue continuing education to use pre-test probability scores before testing the anti-PF4/heparin antibody concentration in HIT-suspected patients.

Conflicts of interest

None.