Accuracy of the Ottawa score in risk stratification of recurrent venous thromboembolism in patients with cancer-associated venous thromboembolism: a systematic review and meta-analysis

Abstract

In patients with cancer-associated venous thromboembolism, knowledge of the estimated rate of recurrent events is important for clinical decision-making regarding anticoagulant therapy. The Ottawa score is a clinical prediction rule designed for this purpose, stratifying patients according to their risk of recurrent venous thromboembolism during the first six months of anticoagulation. We conducted a systematic review and meta-analysis of studies validating either the Ottawa score in its original or modified versions. Two investigators independently reviewed the relevant articles published from 1st June 2012 to 15th December 2018 and indexed in MEDLINE and EMBASE. Nine eligible studies were identified; these included a total of 14,963 patients. The original score classified 49.3% of the patients as high-risk, with a sensitivity of 0.7 [95% confidence interval (CI): 0.6-0.8], a 6-month pooled rate of recurrent venous thromboembolism of 18.6% (95%CI: 13.9-23.9). In the low-risk group, the recurrence rate was 7.4% (95%CI: 3.4-12.5). The modified score classified 19.8% of the patients as low-risk, with a sensitivity of 0.9 (95%CI: 0.4-1.0) and a 6-month pooled rate of recurrent venous thromboembolism of 2.2% (95%CI: 1.6-2.9). In the high-risk group, recurrence rate was 10.2% (95%CI: 6.4-14.6). Limitations of our analysis included type and dosing of anticoagulant therapy. We conclude that new therapeutic strategies are needed in patients at high risk for recurrent cancer-associated venous thromboembolism. Low-risk patients, as per the modified score, could be good candidates for oral anticoagulation. (This systematic review was registered with the International Prospective Registry of Systematic Reviews as: PROSPERO CRD42018099506).

Introduction

Cancer is one of the most frequent risk factors for venous thromboembolism (VTE) and for VTE recurrence while on anticoagulation.^1,2 In patients with VTE and cancer, the rate of recurrent VTE despite anticoagulation can reach up to 20% after six months of therapy, but this rate highly depends on several patient and cancer characteristics.³ For example, age, residual thrombosis, previous history of VTE, surgical procedures within three months prior to VTE, cancer stage, and the site and histology of the malignancy all impact the 6-month rate of recurrent VTE.^4–8 The anticoagulant therapy used [e.g. vitamin K antagonist, direct oral anticoagulants or low molecular weight heparin (LMWH)] may also influence the rate of recurrent VTE. Reliable identification of which patients are at high or low risk of recurrent VTE recurrence must be performed to aid clinical decision-making regarding the type of anticoagulant therapy.

For this, the Ottawa score was designed to stratify the risk of recurrent VTE during the first six months of anticoagulant therapy in patients with cancer-associated VTE.⁹ Two scores were derived. The original score (female sex, lung cancer, and prior history of VTE each give 1 point; breast cancer gives a negative point; cancer stage I gives 2 negative points) dichotomizes patients into low (score ≤ 0) or high (score ≥ 1) risk for VTE recurrence. The modified score (female sex, lung cancer, and prior history of VTE each give 1 point; breast cancer and cancer stage I + II each give a negative point) classifies patients into low (score ≤−1), intermediate (score = 0), and high (score ≥ 1) risk for VTE recurrence. However, the accuracy of these two clinical models remains to determine.

To determine if the Ottawa risk score (i.e. original and modified) can reliably identify the risk of recurrent VTE during the first six months of anticoagulation in cancer patients with VTE, we conducted a systematic review and meta-analysis of the literature.

Methods

The guidelines of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement were followed. The systematic review was registered with the International Prospective Registry of Systematic Reviews (PROSPERO CRD42018099506).

Search strategy and study selection

We systematically searched Medline and Embase, using the following key words: recurrent venous thromboembolism AND cancer AND (decision tree OR clinical prediction rule OR clinical prediction score OR clinical decision rule OR management studies OR outcome studies OR decision support techniques), (venous thromboembolism recurrence) AND cancer AND (decision tree OR clinical prediction rule OR clinical prediction score OR clinical decision rule OR management studies OR outcome studies OR decision support techniques). The search was limited to English and French language studies. Literature search was restricted to 1^st June 2012 to 15^th December 2018, since the Ottawa score was published online in June 2012. To ensure a comprehensive literature search, we examined reference lists from retrieved articles and reference literature (guidelines and systematic reviews), and contacted experts in the management of cancer-associated VTE for possible missing studies. Eligible studies were those validating either the original or the modified Ottawa scores. If key data were missing, study authors were contacted to request the relevant data. Two investigators independently evaluated studies for possible inclusion (AD and SM). They independently assessed study quality and extracted the data on study design and patient characteristics. Disagreements about extracted data were resolved by consensus or by discussion with a third reviewer (MC).

Quality assessment and data extraction

Methodological quality of included studies was assessed independently by two observers (AD and SM) using the Hayden quality assessment tool specifically developed for systematic reviews of prognosis studies.¹⁰ This tool assesses six potential biases. 1) Is the population of interest represented in the study sample? 2) Are there cases of Loss to Follow Up that are not associated with key characteristics? 3) Is there adequate measurement of the prognostic factors? 4) Is there adequate measurement of the outcome of interest? 5) Are important confounders accounted for? 6) Has the appropriate statistical analysis been conducted? Regarding the study population criterion, we considered as representative cohorts those that included consecutive patients with documented cancer-associated VTE with at least six months of follow up.

We collected the following data for each study: year of publication, score evaluated (original or modified), data collection methods (retrospective or prospective), setting (outpatient, inpatient or both), geographic location, demographics (mean age, percentage of women), follow-up duration, overall prevalence of recurrent VTE, distribution of patients in each pre-test probability group, and prevalence of VTE in each pre-test probability group.

The primary outcome of the study was the pooled prevalence of recurrent VTE in each risk group after six months of anticoagulation.

Data analysis

Publication bias was explored by funnel plots and Egger’s test. We determined the 95% confidence intervals (CI) of the prevalence of recurrent VTE in the various clinical probability categories using the exact method. The prevalence of VTE in each level of clinical probability was separately assessed using the method of the inverse variance on the arcsine-transformed proportions. Heterogeneity was tested with the Cochran Q statistic and also quantified by the indicator I² (ranging from 0% for perfect homogeneity to 100% for extreme heterogeneity). In case of heterogeneity (Cochran Q test with a P-value < 0.10 or I2 > 50%), a random effects model was used.¹¹

All analyses were performed using STATA14 (StataCorp, College Station, TX, USA) using the metaprop command.¹²

Results

Study selection

The literature search identified 329 article records of which 229 were assessed for eligibility (Figure 1). Nine studies reporting data on 14,963 patients were eligible and were included in the analyses.^9,13–19

Figure 1.

Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement flow diagram.

Study characteristics

Study characteristics are depicted in Table 1. Of the reported studies, six were prospective, six were multicenter, and four used the original score. In total, seven studies were validation studies of the Ottawa score. Mean age of the patients was 58-69 years; female gender accounted for 45-59% of the patients.

Table 1.

Characteristics of the studies.

Risk of bias within studies

Risk of bias is summarized in Online Supplementary Appendix 1. For six studies, at least four out of six potential bias areas were judged satisfied.^9,14–16,18 Confounding measurement was not accounted for in all studies, and attrition was accounted for in only one study.¹⁸ A significant publication bias between the studies was observed (P=0.001) (Online Supplementary Appendix 2A). This bias was only observed in studies reporting on the modified score (Online Supplementary Appendix 2B and C).

Synthesis of results

All nine identified studies were included in the meta-analysis. Data were extracted from original publications for six studies^9,13–15,19 or, after that, additional data were provided by corresponding authors.^16–18 The overall 6-month rate of recurrent VTE varied from 4.3% to 18.1% in the different studies; the overall 6-month pooled rate was 9.8% (95%CI: 6.4-13.8; I²=96%).

The original Ottawa score was derived in one and validated in three studies.^9,14,15,19 The total number of patients included in these studies was 1,558 with an overall pooled 6-month rate of recurrent VTE of 12.7% (95%CI: 8.9-17.2, I²=81%). Overall, 763 (49.3%) patients were classified in the high-risk category with a pooled 6-month recurrence rate of VTE of 18.6% (95%CI: 13.9-23.9) (I²=64%, P=0.04) (Figure 2A). Of the remaining 795 patients (classified in the low-risk category), the pooled 6-month rate of recurrent VTE was 7.4% (95%CI: 3.4-12.5) (I²=81%, P<0.01) (Figure 2B). The estimated pooled sensitivity, specificity, and Area Under the Receiver Operating Characteristic curve (AUROC) of the original score to identify high-risk patients were 0.7 (95%CI: 0.6-0.8), 0.5 (95%CI: 0.5-0.6), and 0.7 (0.6-0.8), respectively.

Figure 2.

Pooled recurrence rates of venous thromboembolism for the original Ottawa score. (A) Pooled recurrence rates of venous thromboembolism for the original Ottawa score in high-risk patients. (B) Pooled recurrence rates of venous thromboembolism for the original Ottawa score in low-risk patients.

The modified score was derived in one and validated in four studies.^9,13,16–18 The pooled 6-month rate of recurrent VTE in the 13,419 studied patients was 7.85% (95%CI: 4.79-11.57) (I²=95%, P<0.01). The modified score classified 5,307 (39.5%) patients in the high-risk category, in which the pooled 6-month rate of recurrent VTE was 10.2% (95%CI: 6.4-14.6) (I²=89%, P<0.01) (Figure 3A). A total of 2,653 patients (19.8%) were classified in the low-risk category with a pooled 6-month rate of recurrent VTE of 2.2% (95%CI: 1.6-2.9) (I²=0%, P=0.51) (Figure 3B). For the remaining 5,459 patients in the intermediate-risk category, the pooled 6-month rate of recurrent VTE was 7.1% (3.8-11.3) (I²=90%, P<0.01) (Figure 3C).

Figure 3.

Pooled recurrence rates of venous thromboembolism for the modified Ottawa score. (A) Pooled recurrence rates of venous thromboembolism for the modified Ottawa score in high-risk patients (B) Pooled recurrence rates of venous thromboembolism for the modified Ottawa score in intermediate-risk patients. (C) Pooled recurrence rates of venous thromboembolism for the modified Ottawa score in low-risk patients.

The estimated pooled sensitivity, specificity, and AUROC of the modified score to identify high-risk patients were 0.5 (95%CI: 0.5-0.6), 0.6 (95%CI: 0.5-0.7), and 0.5 (95%CI: 0.5-0.6), respectively. For the identification of low-risk patients these characteristics were 0.9 (95%CI: 0.8-1.0), 0.2 (95%CI: 0.1-0.2), and 0.5 (95%CI: 0.5-0.7), respectively.

Pooled incidences of recurrent VTE for each point category using the original and the modified Ottawa scores are reported in Online Supplementary Appendix 3. The rates of recurrent VTE ranged from 0 to 37.1% (95%CI: 12.7-64.7) with a dose-effect association in studies applying the original score and ranged from 0 to 9.1% (95%CI: 0.2-24.7) in studies applying the modified score with a stepwise association.

Discussion

This systematic review and meta-analysis of nine studies involving a total of 14,963 patients with cancer-associated VTE, confirms that the Ottawa score is an accurate tool to stratify the risk for recurrent VTE within the first six months of anticoagulation. The original Ottawa score can reliably identify patients with cancer-associated VTE at high risk of recurrent events, whereas the modified score is best suitable for identifying cancer patients with low risk of VTE recurrence. The original score classified 49.3% of the patients into the high-risk group with a sensitivity of 71% and the modified score classified 19.8% of the patients into the low-risk group with a sensitivity of 92%.

The Ottawa scores (original and modified) are the only tools available to stratify the risk for recurrence of cancer-associated VTE. The scores can help identify patients with a rate of recurrent VTE >10%. This has been suggested as clinically relevant and considered as a “high-risk” category.²⁰ The Ottawa scores have better accuracy than the reported sensitivity, specificity or AUROC values, which were calculated based on crude rates of recurrent VTE as opposed to correct classification. If patients were classified a priori by risk categories (i.e. “high-risk”, “low-risk”) similar to how a diagnostic test would be reported as “disease”, or “no disease”, it is very likely that estimates of intrinsic properties of the Ottawa score would improve. Furthermore, when considering our meta-analysis across each sum of points, we could demonstrate a dose-effect relationship, either continuous (original score) or stepwise (modified score), which confirmed the accuracy of the risk classification of the Ottawa scores.

The accuracy of the modified Ottawa score to identify patients at low risk for VTE recurrence has a potential major therapeutic impact that should be considered for implementation into daily practice. The low 2.2% risk of recurrent VTE in this patient population closely mirrors the recurrent risk of the general VTE population^21,22 (refer to DOAC trials). In this setting, the potential advantages of LMWH over oral anticoagulation are clearly counterbalanced by their cost, their negative impact on quality of life, and the expected low absolute risk reduction of recurrent VTE (<2% based on a 50% relative risk reduction).^23,24 The use of oral anticoagulants in low-risk patients is, therefore, clinically relevant and could be systematically considered as first line in this specific risk group. In contrast, 49.3% of the patients were classified in the high-risk group by the original Ottawa score. The unacceptable 18.6% estimated rate of recurrent VTE, despite anticoagulant treatment, in this group warrants the urgent development of new therapeutic strategies.

Strengths of our study include its comprehensiveness and the large number of patients included; however, we acknowledge several limitations. First, there was a significant heterogeneity between studies and some publication bias, particularly in validation studies of the modified score. Nevertheless, most of the studies were of good quality. A major source of heterogeneity was the large difference in incidence rates of recurrent VTE across the studies. Most datasets were old and included patients receiving out-dated cancer therapies. These therapies may have exposed patients to higher risk of recurrent VTE, leading to an overestimation of the current risk. However, the most recent study using the original or modified scores reported an overall recurrence rate of VTE of 9.4 and 8.4%, respectively, which remains high and clinically relevant.^15,18 Specific center-related factors that could also potentially affect recurrence-rate of VTE include academic centers, age of included patients and their socio-economic level, number of comorbidities, higher stage disease, history of VTE, and ethnicity. The second limitation of our study was we were unable to account for the type of anti-coagulation used. None of the studied patients were treated with a direct oral anticoagulant. Data on exposure to LMWH or vitamin K antagonist was not available in any of the identified studies. In the derivation and in some of the validation studies, the type of anticoagulation was not a significant predictor for VTE recurrence. However, among all randomized trials, only one showed the superiority of LMWH over vitamin K antagonists to treat cancer-associated VTE.²⁵ Third, we could not assess the importance of other risk factors for VTE recurrence (e.g. interruption of anticoagulants for an invasive procedure, age, etc.) during follow up. However, the Ottawa scores were derived without accounting for these confounders and appeared to accurately classify patients. Fourth, patients classified by the original score in the low-risk category had an estimated rate of VTE recurrence of 7.4%, which cannot be considered as low. However, when the score was initially derived, the objective was to identify patients with an a priori risk for VTE recurrence during anticoagulation of <7%. In that instance, our data confirm the accuracy of the original score.

In conclusion, the Ottawa score, either in its original or modified form, is a useful tool to stratify the 6-month risk for VTE recurrence during anticoagulation in patients with cancer-associated VTE. Specifically, the original and modified Ottawa score can accurately identify patients with cancer-associated VTE at high and low risk of recurrent events, respectively. There is an urgent need for the development of new therapeutic strategies to prevent recurrent VTE in high-risk patients. The original score is an accurate tool to define inclusion criteria of future studies.