Abstract
Background
Genetics plays an important role in venous thromboembolism (VTE). Factor V Leiden (FVL or rs6025) and prothrombin gene G20210A (PT or rs1799963) are the genetic variants currently tested for VTE risk assessment. We hypothesized that primary VTE risk assessment can be improved by using genetic risk scores with more genetic markers than just FVL‐rs6025 and prothrombin gene PT‐rs1799963. To this end, we have designed a new genetic risk score called Thrombo inCode (TiC).
Methods and Results
TiC was evaluated in terms of discrimination (Δ of the area under the receiver operating characteristic curve) and reclassification (integrated discrimination improvement and net reclassification improvement). This evaluation was performed using 2 age‐ and sex‐matched case–control populations: SANTPAU (248 cases, 249 controls) and the Marseille Thrombosis Association study (MARTHA; 477 cases, 477 controls). TiC was compared with other literature‐based genetic risk scores. TiC including F5 rs6025/rs118203906/rs118203905, F2 rs1799963, F12 rs1801020, F13 rs5985, SERPINC1 rs121909548, and SERPINA10 rs2232698 plus the A1 blood group (rs8176719, rs7853989, rs8176743, rs8176750) improved the area under the curve compared with a model based only on F5‐rs6025 and F2‐rs1799963 in SANTPAU (0.677 versus 0.575, P<0.001) and MARTHA (0.605 versus 0.576, P=0.008). TiC showed good integrated discrimination improvement of 5.49 (P<0.001) for SANTPAU and 0.96 (P=0.045) for MARTHA. Among the genetic risk scores evaluated, the proportion of VTE risk variance explained by TiC was the highest.
Conclusions
We conclude that TiC greatly improves prediction of VTE risk compared with other genetic risk scores. TiC should improve prevention, diagnosis, and treatment of VTE.
Keywords: genetics, risk factors, tests, thrombosis, veins
Introduction
Thrombosis is the formation of a blood clot inside a blood vessel that obstructs the normal flow of blood. The clinical manifestations of thrombosis are myocardial infarction, stroke, and venous thromboembolism (VTE). The latter includes pulmonary embolism and deep vein thrombosis. VTE is a common cardiovascular illness associated with high mortality1 that affects ≈0.2% of the US and European population annually.1–2 Consequently, it is a considerable public health concern with a high economic burden.3–4
VTE is a multifactorial, complex disease that results from a combination of genetic and acquired risk factors. The heritability of VTE has been estimated at about 60%.5 The genetic factors underlying the risk of thrombosis include some well‐established mutations such as the FVL and PT mutations, which give rise to deficient anticoagulant or gain‐of‐function proteins.6 In addition, recent evidence shows several genetic variants that predispose someone to VTE by modifying components in the coagulation pathway.7–15 Moreover, several common low‐penetrance gene variants and unsuspected genes have been identified by genomewide association studies (GWASs) as contributing to a risk of thromboembolic disease.16–18 These new variants still require proper clinical validation.
This new knowledge of the genetic profile of VTE could be used to increase the ability to more accurately predict the risk of a thrombotic event. In current clinical practice, only FVL and PT mutations are used as markers to assess a patient's risk of VTE. In a genetic risk score (GRS) described by de Haan et al,19 5 of 31 single nucleotide polymorphisms (SNPs) linked to VTE were found to improve the predictive capacity of clinical factors including family history assessment. Moreover, the similar discriminative capacity observed by these authors with the use of 5 of 31 SNPs associated with VTE clearly indicates a need to identify an appropriate panel of genetic variants and to determine the predictive capacity of such a panel as a risk assessment score for this complex disease.
Our study was designed to compare the predictive capacity of a new GRS, Thrombo inCode (TiC), with a risk score based on family history alone and another based on FVL and PT mutations.
Methods
Study Populations
The SANTPAU case–control study, conducted on a Spanish population, has been described extensively.20 Briefly, 248 consecutive unrelated patients who had been referred to or had visited the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain) for thrombophilia screening were recruited over the period from November 1997 to April 2002. The inclusion criterion was having suffered a first thrombotic event at an age younger than 68 years. Patients were excluded if they had cancer or a history of chronic or acute liver disease or nephrotic syndrome. A medical history was obtained for each patient, including the site of thrombosis and acquired predisposing factors. The diagnosis of deep vein thrombosis of the lower limbs was established objectively by ultrasonography or ascending venography. Pulmonary embolism was diagnosed by ventilation–perfusion lung scanning, pulmonary angiography, or spiral computed tomography. Intracranial venous thrombosis was diagnosed by magnetic resonance imaging. A patient's family history was scored as positive if at least 1 other first‐ or second‐degree family member had a venous thrombosis. As controls, 249 unrelated, asymptomatic, and apparently healthy persons were recruited with no personal history of VTE or use of oral anticoagulants. The control group was matched to the patient group for age and sex. To avoid genetic stratification, both the case and control groups were recruited from the same geographical region; all participants were white, and all their family names were Spanish. The study protocol was approved by the hospital's institutional review board, and signed informed consent was obtained from each patient.
The Marseille Thrombosis Association study (MARTHA)21 is a case–control study including 1150 patients and 801 controls. The patients were unrelated and white; were recruited consecutively from the Thrombophilia Center, Hôpital de la Timone (Marseille, France) over the period from January 1994 to October 2005; had VTE; and were without known risk factors including antithrombin, protein C or protein S deficiency, homozygosity for FVL or for PT, or the presence of lupus anticoagulant. Thrombotic events including deep vein thrombosis and pulmonary embolism were documented by venography, Doppler ultrasound, spiral computed tomographic scanning angiography, and/or ventilation–perfusion lung scan. The control group was comprised of 2 subgroups: one included 475 healthy French white subjects with no personal history of cardiovascular disease (VTE was also considered) who were from the Marseille area and another group made up of 326 healthy French white heterozygotes for the FV Leiden or FII 20210A variants. In the MARTHA case–control study, controls were older than patients (47.4 versus 38.0 years, respectively), and the proportion of men was also higher among controls than among patients (47.8% versus 30.1%, respectively). To avoid the paradoxical (and confounded, by design) protective association between age and VTE and between sex and control, we matched patients and controls 1:1 by age and sex. Patients and controls were randomly matched 1:1 for the same sex and similar age (±5 years). In the original MARTHA study, 1148 patients and 801 controls were included, but for our study, we considered 477 cases and 477 controls.
SNP Selection and Genotyping
We performed a systematic review and meta‐analysis to select genetic variants that contribute to VTE risk (Table 1). Based on this information, we defined a panel, TiC (Table 1), with the variants rs6025 (F5, Factor V Leiden), rs118203906 (F5, Factor V Hong Kong), rs118203905 (F5, Factor V Cambridge), rs1799963 (F2, G20210A), rs5985 (F13, V34L), rs121909548 (SERPINC1, 384 Ala>Ser), rs2232698 (SERPINA10, 67 ARG>Stop), and rs1801020 (F12, 46 C>T) and the A1 carriers rs8176719, rs7853989, rs8176743, and rs8176750.22 It is important to note that all of these genetic variants have functional effects on the coagulation cascade.7–15 All except the A1 carriers are gain‐ or loss‐of‐function variants.
Table 1.
Genetic Variants Included in the Different Genetic Risk Scores Assessed and Coefficients (Weights) Assigned to Each Risk Factor
| SNP | Gene | Mutation | Risk Coefficient Assigned (ß) | GRS* | |||
|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | ||||
| rs6025, FV Leiden | F5 | R506Q | 1.589 | ||||
| rs118203905, FV Hong Kong | F5 | R306G | 1.589 | ||||
| rs118203906, FV Cambridge | F5 | R306T | 1.589 | ||||
| rs1799963 | F2 | G20210A | 0.293 | ||||
| ABO | ABO | A1 carriers | 0.956 | ||||
| rs8176719 | ABO | — | |||||
| rs1801020 | F12 | C46T | 1.633 | ||||
| rs5985 | F13 | V34L | 0.198 | ||||
| rs2232698 | SERPINE10 | R67X | 1.358 | ||||
| rs121909548 | SERPINC1 | A384S | 2.277 | ||||
| rs2036914 | F11 | 0.293 0.519 |
|||||
| rs2066865 | FGG | 0.344 | |||||
| rs710446 | KNG1 | 0.182 | |||||
| rs2289252 | F11 | 0.315 0.577 |
|||||
GRS indicates genetic risk score; SNP, single nucleotide polymorphism.
GRS 1, FVL+PT based on F5 and rs1799963 (F2, prothrombin); GRS 2: Thrombo inCode based on F5, rs1799963 (F2, prothrombin), ABO‐A1 carriers (rs8176719, rs7853989, rs8176743, rs8176750), rs1801020, rs5985, rs2232698, and rs121909548; GRS 3, de Haan et al based on F5, rs1799963 (F2, prothrombin), ABO (rs8176719), rs2036914, and rs2066865; GRS 4, expanded based on F5, rs1799963 (F2, prothrombin), ABO‐A1 carriers (rs8176719, rs7853989, rs8176743, rs8176750), rs1801020, rs5985, rs2232698, rs121909548, rs2036914, rs2066865, and rs2289252.
In addition, 3 different panels of genetic variants were defined (Table 1):
FVL+PT: rs6025 (F5, Factor V Leiden) and rs1799963 (F2, 20210 G>A). This panel represents the genetic variants most commonly tested in current clinical practice.
de Haan et al panel: rs6025 (F5, Factor V Leiden), rs1799963 (F2, 20210 G>A), rs8176719 (AB0), rs2066865 (FGG, 10034 C>T), and rs2036914 (F11, 7872 C>T).
Extended panel: All the TiC variants plus rs2289252 (F11, 22771 T>C), rs2036914 (F11, 7872 C>T), rs710446 (KNG1, Ile581Thr), and rs2066865 (FGG, 10034 C>T). These SNPs were added because of their relationships to VTE that were detected recently by GWAS.
DNA samples from the 2 populations were genotyped. With the SANTPAU samples, the Thrombo inCode kit (Ferer inCode) was used to identify the variants included in this panel, and the remaining variants were detected by Taqman assays run in an ABI 7500 instrument. The MARTHA samples were genotyped by allele‐specific polymerase chain reaction.
Genetic Risk Score
To take into account the association strengths between the selected SNPs and VTE, we created a weighted GRS for each of the panels described (Table 1). The weights assigned to each SNP were defined a priori and based on the results of published meta‐analyses in the case of FVL23 and PT24 or data from individual reports for rs2232698,13,25 rs1801020,20,26–27 rs59852,8,28–29 rs2289252,30–31 and rs203691431–32 or meta‐GWAS for the variants rs2066865,19,33–34 AB0,35 rs121909548,14 and rs71044636 (Tables S1 and S2). For FV Cambridge and Hong Kong, the same weights as FV Leiden were assigned. For the panel described by de Haan et al,19 we used the weights cited by the authors.
The genetic variants were introduced considering the genetic risk, and all were weighted in the same direction. The only genetic variant with a minor allele associated with lower thrombosis risk and odds was rs5985; in this case, we considered the common homozygote group as the risk.
Family History
For SANTPAU populations, data were compiled on the family history of VTE.
Statistical Analysis
Continuous variables are designated as means and standard deviations, and categorical variables are designated as proportions. Odds ratios (ORs) for the different variables linked to VTE and their 95% confidence intervals were calculated by conditional logistic regression.
We constructed different predictive models based on the FVL+PT GRS, the TiC GRS, the de Haan et al GRS, the extended GRS, or family history and combinations of these.
As previously mentioned, the weights assigned to all variables included in the models were defined a priori based on prior evidence.
The different scores were assessed according to the scientific statement of the American Heart Association, which describes the steps to be taken to evaluate novel risk markers in the cardiovascular field.37
To assess whether the strengths of association between clinical or genetic factors and thrombosis were different for those observed in the literature (expected) and those we observed in our study (observed), we compared coefficients (ie, logarithm of the ORs) by the z test statistic; in the equation z=(b[E]−b[0])/SE, b[E] and b[0] are, respectively, the coefficients expected and observed, whereas SE is the standard error of the difference in the coefficient.
We used different measures of performance to test the quality of fit in the GRS models. Discrimination measures the ability of the model to discriminate between participants who will and will not have a VTE. We quantified this by calculating the area under the receiver operating characteristic curve.38 This value represents an estimate of the probability that a model assigns a higher risk to those participants who will have a VTE than to those who will not have a VTE.
Reclassification measures how the inclusion of a new marker classifies as highest risk those participants with a VTE and as lowest risk those without a VTE. We used the methods described by Pencina et al.39–40 Integrated discrimination improvement (IDI) considers changes in the estimated VTE prediction probabilities as a continuous variable. The IDI increases when a new marker is added and thus enhances the estimate of the risk in those with VTE and decreases in those without VTE. Similarly, net reclassification improvement (NRI) requires the classification of the participants in risk categories and considers changes in the predicted probabilities of estimated VTE that imply a change from one category to another. Risk categories required to estimate the net reclassification improvement were established according to risk tertiles.
The sensitivity and specificity of the different GRSs were calculated41 using the cut points giving the highest sensitivity.
All tests were performed using R statistical software (version 3.0.1).42
Results
The sociodemographic, clinical, and genetic characteristics of the participants are listed in Table 2. In both populations, patients showed higher scores than controls in all of the GRSs examined.
Table 2.
Main Sociodemographic, Clinical, and Genetic Characteristics of the Study Participants
| Controls | Cases | P Value | |
|---|---|---|---|
| n=249 | n=248 | ||
| SANTPAU | |||
| Sex (male), n (%) | 109 (44.0) | 111 (44.6) | 0.960 |
| Age (y), mean (SD) | 49.0 (14.9) | 47.1 (14.0) | 0.145 |
| Smoker, n (%) | 101 (40.7) | 108 (43.7) | 0.559 |
| Diabetes, n (%) | 9 (3.7) | 14 (5.7) | 0.404 |
| Oral contraceptives, n (%) | 74 (29.8) | 83 (33.5) | 0.440 |
| Family history, n (%) | 45 (23.2) | 97 (40.9) | <0.001 |
| F5**, n (%) | 5 (2.02) | 32 (12.9) | <0.001 |
| F2: rs1799963*, n (%) | 7 (2.82) | 19 (7.63) | 0.027 |
| ABO‐A1 carriers/ABO*, n (%) | 87 (35.7) | 147 (59.0) | <0.001 |
| F12: rs1801020*, n (%) | 5 (2.02) | 15 (6.02) | 0.041 |
| F13: rs5985*, n (%) | 139 (56.5) | 146 (58.6) | 0.698 |
| SERPINE10: rs2232698*, n (%) | 4 (1.61) | 10 (4.02) | 0.178 |
| SERPINC1: rs121909548*, n (%) | 1 (0.40) | 4 (1.61) | 0.372 |
| F11: rs2036914 | |||
| Hetero, n (%) | 111 (46.2) | 119 (48.0) | 0.77 |
| Homo, n (%) | 54 (22.5) | 43 (17.3) | 0.189 |
| FGG: rs2066865*, n (%) | 92 (37.9) | 98 (39.4) | 0.804 |
| KNG1: rs710446b*, n (%) | 47 (19.5) | 49 (19.8) | 0.966 |
| F11: rs2289252 | |||
| Hetero, n (%) | 113 (47.1) | 122 (49.4) | 0.675 |
| Homo, n (%) | 39 (16.2) | 46 (18.6) | 0.568 |
| GRS 1*, mean (SD) | 0.04 (0.23) | 0.23 (0.54) | <0.001 |
| GRS 2*, mean (SD) | 0.56 (0.61) | 1.10 (0.89) | <0.001 |
| GRS 3*, mean (SD) | 0.78 (0.56) | 1.16 (0.76) | <0.001 |
| GRS 4*, mean (SD) | 1.23 (0.66) | 1.76 (0.95) | <0.001 |
| MARTHA | n=477 | N=477 | |
| Sex (male), n (%) | 198 (41.5) | 198 (41.5) | 1.000 |
| Age (years), mean (SD) | 44.2 (13.6) | 43.9 (14.0) | 0.681 |
| Smoker, n (%) | 143 (30.2) | 124 (27.7) | 0.447 |
| BMI (kg/m2), mean (SD) | 23.8 (3.8) | 25.0 (4.2) | <0.001 |
| Oral contraceptives, n (%) | 105 (22.1) | 187 (39.4) | <0.001 |
| F5**, n (%) | 103 (21.6) | 168 (35.2) | <0.001 |
| F2: rs1799963*, n (%) | 92 (19.3) | 86 (18.0) | 0.678 |
| ABO‐A1 carries/ABO*, n (%) | 28 (5.87) | 47 (9.85) | 0.030 |
| F12: rs1801020*, n (%) | 20 (4.19) | 29 (6.08) | 0.241 |
| F13: rs5985*, n (%) | 255 (53.5) | 283 (59.3) | 0.078 |
| SERPINE10: rs2232698*, n (%) | 8 (1.68) | 15 (3.14) | 0.205 |
| SERPINC1: rs121909548*, n (%) | 3 (0.63) | 1 (0.21) | 0.324 |
| F11: rs2036914 | |||
| Hetero, n (%) | 231 (49.7) | 236 (50.1) | 0.948 |
| Homo, n (%) | 118 (25.4) | 100 (21.2) | 0.155 |
| FGG: rs2066865*, n (%) | 178 (38.3) | 220 (49.5) | 0.001 |
| KNG1: rs710446b*, n (%) | 92 (19.6) | 82 (18.2) | 0.638 |
| F11: rs2289252 | |||
| Hetero, n (%) | 231 (48.4) | 225 (47.2) | 0.746 |
| Homo, n (%) | 72 (15.1) | 126 (26.4) | <0.001 |
| GRS 1*, mean (SD) | 0.40 (0.63) | 0.61 (0.73) | <0.001 |
| GRS 2*, mean (SD) | 0.67 (0.83) | 0.97 (0.92) | <0.001 |
| GRS 3*, mean (SD) | 0.85 (0.73) | 1.12 (0.83) | <0.001 |
| GRS 4*, mean (SD) | 1.34 (0.86) | 1.70 (0.94) | <0.001 |
BMI indicates body mass index; GRS, genetic risk score; hetero, heterozygosis; homo, homozygosis.
Carriers of the risk allele.
Carrier of any risk allele (Leiden, Hong Kong, or Cambridge).
Homozygotes for the risk allele.
GRS 1, FVL+PT based on F5 and rs1799963 (F2, prothrombin); GRS 2, Thrombo inCode based on F5, rs1799963 (F2, prothrombin), ABO‐A1 carriers, rs1801020, rs5985, rs2232698, and rs121909548; GRS 3, de Haan et al based on F5, rs1799963 (F2, prothrombin), ABO, rs2036914, rs2066865; GRS 4, expanded based on F5, rs1799963 (F2, prothrombin), ABO, rs1801020, rs5985, rs2232698, rs121909548, rs2036914, rs2066865, and rs2289252; ABO‐A1 carriers: rs8176719, rs7853989, rs8176743, rs8176750; ABO: rs8176719.
Regression coefficients (ie, logarithm of the ORs) and their standard errors for associations between the different variables and VTE are shown in Table S3. In our case, not all SNPs were associated with VTE, although we included all SNPs in the GRS calculations because of consistent reports in the literature of their correlation with VTE. In Table S3, we also provide the expected regression coefficients based on a literature review and our own meta‐analysis and P values for differences between observed and expected coefficients. We did not observe a significant difference between expected and observed coefficients except for mutations in the gene for FVL in the MARTHA population (expected coefficient 1.589 versus observed 0.805, P=0.028).
As shown in Table 3, in the SANTPAU population, the predictive model based on FVL and PT mutations showed an area under the curve of 0.575 (95% CI 0.547 to 0.604), which increased significantly when the model was based on the TiC GRS (0.575 versus 0.677, P<0.001). The de Haan et al GRS also was significantly better than the predicted model based on FVL and PT mutations (0.575 versus 0.645, P=0.015); however, this GRS did not improve the discriminative capacity of TiC (0.677 versus 0.645; P=0.346). Moreover, when we extended the TiC score (extended GRS) by adding 4 common VTE‐associated SNPs (F11 rs2289252 and rs2036914, KNG1 rs710446, and FGG rs2066865), no improvement was observed over the TiC GRS area under the curve (0.677 versus 0.671, P=0.848).
Table 3.
Predictive Capacities of the Different Models and Improvements Observed Including Different Genetic Variants Compared With the Simplest Model (FVL+PT)
| GRS 1 FVL+PT (95% CI) | GRS 2 TiC (95% CI) | GRS 3 de Haan et al (95% CI) | GRS 4 Extended (95% CI) | |
|---|---|---|---|---|
| SANTPAU | ||||
| Discrimination | ||||
| AUC | 0.575 (0.547; 0.604) | 0.677 (0.631; 0.724) | 0.645 (0.596; 0.694) | 0.671 (0.623; 0.719) |
| P value ΔAUC | NA | <0.001 | 0.015 | <0.001 |
| Reclassification | ||||
| IDI | NA | 5.49 (3.35; 7.63) | 2.43 (0.47; 4.39) | 2.57 (0.65; 4.49) |
| P value | NA | <0.001 | 0.015 | 0.009 |
| NRI | NA | 19.17 (7.01; 31.33) | −5.76 (−21.84; 10.32) | −8.66 (−25.61; 8.29) |
| P value | NA | 0.002 | 0.483 | 0.317 |
| MARTHA | ||||
| Discrimination | ||||
| AUC | 0.576 (0.544; 0.609) | 0.605 (0.570; 0.640) | 0.594 (0.557; 0.631) | 0.629 (0.592; 0.665) |
| P value ΔAUC | NA | 0.008 | 0.478 | 0.037 |
| Reclassification | ||||
| IDI | NA | 0.96 (0.02; 1.90) | −0.06 (−0.86;0.75) | 0.19 (−0.88; 1.25) |
| P value | NA | 0.045 | 0.889 | 0.730 |
| NRI | NA | 4.94 (−1.46; 11.33) | −5.98 (−13.98; 2.02) | −7.11 (−16.35; 2.13) |
| P value | NA | 0.130 | 0.143 | 0.131 |
AUC indicates area under the receiver operating characteristic curve; GRS, genetic risk score; IDI, integrated discrimination improvement; NA, not applicable; NRI, net reclassification improvement; TiC, Thrombo inCode.
The same approach was used to assess the validity and predictive improvement capacity of the different models when FVL and PT mutations (often used in clinical practice) were considered in addition to family history of VTE. The results in Table 4 indicate the capacity of each of the GRS models in addition to family history of VTE to improve the discrimination capacity when compared with family history of VTE alone. The discriminative capacity of TiC plus family history of VTE was not improved by the de Haan et al or extended GRSs plus family history of VTE.
Table 4.
Predictive Capacity of the Different Models and Improvements Observed When Including Different Genetic Variants With Respect to Family History in the SANTPAU Population
| Family History (95% CI) | GRS 1 FVL+PT (95% CI) | GRS 2 TiC (95% CI) | GRS 3 de Haan (95% CI) | GRS 4 Extended (95% CI) | |
|---|---|---|---|---|---|
| Discrimination | |||||
| AUC | 0.589 (0.545; 0.632) | 0.647 (0.602; 0.691) | 0.701 (0.652; 0.749) | 0.684 (0.633; 0.734) | 0.700 (0.649; 0.750) |
| P value ΔAUC | NA | <0.001 | <0.001 | 0.005 | 0.001 |
| Reclassification | |||||
| IDI | NA | 3.43 (2.10; 4.76) | 6.63 (4.45; 8.82) | 4.28 (2.18; 6.38) | 2.57 (0.40; 4.74) |
| P value | NA | <0.001 | <0.001 | <0.001 | 0.020 |
| NRI | NA | 16.04 (9.50; 22.57) | 29.42 (14.33; 44.53) | 6.92 (−11.40; 25.24) | 0.65 (−18.57; 19.88) |
| P value | NA | <0.001 | <0.001 | 0.459 | 0.947 |
AUC indicates area under the receiver operating characteristic curve; GRS, genetic risk score; IDI, integrated discrimination improvement; NA, not applicable; NRI, net reclassification improvement; TiC, Thrombo inCode.
Reclassification was improved by TiC, extended, or de Haan et al GRS when compared with the FVL+PT model, as measured by IDI (5.49, P<0.001; 2.56, P=0.009; and 2.43, P=0.015, respectively) (Table 3). The only GRS showing an improvement in reclassification (net reclassification improvement) over the simple FVL+PT model was the TiC GRS (19.17, P=0.002). Similar results were obtained when all GRSs plus family history of VTE were compared with family history of VTE alone (Table 4).
In addition, using the GRS cut points to obtain maximal sensitivity, the sensitivity of TiC in the SANTPAU population was significantly higher than that of FVL+PT (0.85 versus 0.20%, respectively) (Table 5). The specificity of the FVL+PT GRS was higher than that of TiC (0.95 versus 0.25, respectively).
Table 5.
Clinical Utility (Measured as the Sensitivity) and Specificity of TiC Compared With FVL+PT in the SANTPAU and MARTHA Populations
| Selected GRS Cut Points | SANTPAU | MARTHA | |||
|---|---|---|---|---|---|
| Sensitivity | Specificity | Sensitivity | Specificity | ||
| FV+PT | 0.147 | 0.20 | 0.95 | 0.53 | 0.59 |
| TiC | 0.099 | 0.85 | 0.25 | 0.85 | 0.26 |
GRS indicates genetic risk scores; TiC, Thrombo inCode.
More important, the variance in VTE risk explained by the different GRSs in the SANTPAU population were 7.1%, 15.1%, 9.9%, and 13.0% for FVL+PT, TiC, de Haan et al, and extended GRSs, respectively. It is noteworthy that the TiC score explained the greatest amount of variance in thrombotic risk; in fact, it was >2‐fold the variance explained by the conventional FVL+PT model.
Similar results were observed for the MARTHA population (Table 3). The AUC increased significantly with respect to the model based on FVL and PT alone when we used the TiC genetic variants (0.576 versus 0.605, P=0.008) and the extended genetic model (0.576 versus 0.629, P=0.037); however, the extended score did not improve the discriminative capacity of TiC (0.605 versus 0.629, P=0.361). Moreover, the de Haan et al GRS offered no improvement over the discriminative capacity of FVL+PT (0.576 versus 0.594, P=0.47). In addition, when analyzing the reclassification, only the TiC score improved the reclassification capacity of FVL+PT, as assessed by the IDI (0.96, P=0.045).
The sensitivity and specificity of TiC scores were similar in the MARTHA population (0.850 and 0.264, respectively); however, the sensitivity and specificity of the FVL+PT GRS in the MARTHA population differed from that found in the SANTPAU population (sensitivity 0.532 and specificity 0.591 in MARTHA).
It should be emphasized that in the MARTHA study, the extended GRS explained a high proportion of the variance in VTE risk (5.3%), whereas the FVL+PT, TiC, and de Haan et al GRSs explained 3.1%, 3.9%, and 3.7%, respectively. This observation could be attributable to the fact that the extended panel included the genetic variants included in TiC plus 4 common SNPs (F11 rs2289252 and rs2036914, KNG1 rs710446, and FGG rs2066865) reported to be associated with VTE in the MARTHA study.
Discussion
As knowledge of disease improves, new biomarkers and new tests are changing the traditional concept of risk assessment. Given that thrombosis is the final outcome of many systemic disorders, it is not surprising that interest in this disease is increasing rapidly; however, accurately predicting a person's risk of developing a complex disease is very difficult. This difficulty results, in large measure, from the many risk factors that exist for a given disease. Most of these factors and their interactions are unknown. VTE is a case in point because it has a large number of risk factors related to genetic variability.5 Despite this substantial genetic component of VTE and the new knowledge generated by GWASs, only 2 of these variants—FVL and PT—are used conventionally in clinical settings worldwide.
In our study, we assessed the predictive validity of 3 GRSs in 2 independent populations (SANTPAU and MARTHA) according to the American Heart Association's guidelines for the evaluation of novel cardiovascular risk markers.37 The proof of concept derived from our study is that the FVL+PT model can be greatly improved by using new genomic information. More important, by comparing the use of 3 GRSs on the same populations, we were able to show conclusively that their predictive capacities are greatly augmented using the TiC GRS.
In an initial step, we performed a systematic review of the literature and meta‐analysis to select genetic variants that contribute to VTE risk, and we assigned these variants a corresponding VTE risk coefficient (Tables 1, S1, and S2). The genetic variants selected have a direct functional effect on the blood‐clotting proteins,7–15 highlighting the role of coagulation in thrombosis risk. Based on these selected genetic variants, we defined 3 weighted GRSs and determined whether they could assess the risk of VTE better than a model based on family history or FVL+PT alone. The weights assigned to each genetic variant were defined a priori and based on the literature.
In the second step of our study, we selected 2 independent populations in which to compare the performance of these genetic scores: the Spanish population examined in the SANTPAU case–control study and the population of the French MARTHA study, with cases and controls that were rich in FV Leiden or FII 20210A mutations. Accordingly, the SANTPAU study participants better represented the general white population and formed the basis of our study, whereas the MARTHA study participants were considered a stricter sample in which to assess the role of new genetic markers because the effects of FV Leiden and FII mutations were overrepresented.
In this study, we were able to confirm the prior finding that FVL+PT improves the VTE predictive capacity of family history.43 We were also able to demonstrate that TiC, a GRS including 12 low‐frequency, high‐impact genetic risk factors, was the only score of the 3 examined that significantly improved the VTE predictive capacity of family history of VTE (Table 4) and FVL+PT (Table 3). This improvement consisted of better discrimination and reclassification in both the SANTPAU and MARTHA study populations.
One of the GRSs that we examined was reported by de Haan et al.19 Remarkably, these authors observed that a risk score including 31 SNPs independently associated with a VTE risk through GWAS showed a similar predictive capacity to a score including only 5 of these SNPs (FVL, PT, ABO blood group, FGG gene rs2066865, and F11 gene rs2289252). Three of these 5 genetic variants (FVL, PT, ABO blood group) reported by de Haan et al were included in the TiC score. Importantly, these matching markers are low‐frequency variants that show the highest individual ORs for VTE risk. This fact clearly indicates the limited value of risk scores composed of common SNPs that show low individual ORs, such as those identified by GWAS, when low‐frequency variants with high ORs are also considered. In effect, the addition of 4 common SNPs (rs710446, rs2289252, rs2066865 and rs2036914) identified through GWAS to TiC (extended GRS) failed to improve the area under the curve.
This point is important regarding GWASs because the findings of both the study by de Haan et al and our study indicate the better predictive capacity of a GRS that considers low‐ frequency variants that returns high ORs. It should be underscored that the TiC includes rare variants featuring high individual ORs (eg, SERPINC1 gene rs121909548 and SERPIN10 gene rs2232698 polymorphisms). Based on these results and in agreement with a previous GWAS,21 it is unlikely that common risk alleles identified through GWAS (showing a Minon Allele Frequency [MAF] >0.05 and a modestly increased VTE risk with an OR in the range 1.10 to 1.35) alone account for a large proportion of the familial risk of VTE and its clinical variability, as observed for most other human diseases investigated through GWASs.44
These findings emphasize the need to pay special attention to low‐frequency or rare variants rendering high ORs. In a recent study, the benefits of an in‐depth sequencing strategy were emphasized because it permitted the identification of a rare mutation responsible for familial cases of early‐onset VTE.45–46 The important predisposing role played by low‐frequency or rare variants in VTE was highlighted in a study based on multigenerational data by Zöller et al,47 who observed a high familial risk of VTE in a small number of siblings, suggesting segregation of rare but strong genetic risk factors.
In our study, the clinical utility (measured as the sensitivity of the GRSs) of the TiC GRS compared with FVL+PT was also examined. Using the cut point obtaining the highest sensitivity for FVL+PT, the sensitivity of FVL+PT was 0.20 and 0.53 of patients with VTE in SANTPAU and MARTHA, respectively, whereas using TiC at the cut point for maximal sensitivity, the sensitivity of TiC was 0.85 and 0.85 in SANTPAU and MARTHA, respectively. High sensitivity is important when the test is used to identify a serious but preventable or treatable disease such as VTE.41 This is especially true when low specificity may represent a genetic predisposition to VTE that will lead to VTE only when pro‐VTE clinical conditions are also present. The identification of genetic thrombophilia has several clinical applications because most guidelines48 advocate assigning a moderate risk of VTE in patients with thrombophilia.
Although our results point to a clear association between GRS and the risk of VTE, they should be interpreted with caution because of the limitations of our study. First, we did not include clinical data (apart from family history) in the scores because, as indicated by the guidelines,46 several clinical scenarios exist for VTE but the genetic basis of the disease is common to all of them. Consequently, we focused our study on genetic factors and will be examining the use of the TiC GRS in combination with several clinical variables in future studies. This strategy of combining genetic and clinical data, as observed by de Haan et al,19 is likely to further improve the performance of TiC, especially its sensitivity, specificity, and the area under the receiver operating characteristic curve.
Given the lack of universally acceptable risk categories for VTE, in our net reclassification improvement analysis, we established categories based on risk tertiles, which could be an overly demanding process. Nevertheless, this limitation was resolved using the IDI index, which measures risk as a continuum and thus should be considered a more powerful indicator for reclassification in our study.
Although we have described the differences in the design of the 2 populations, the differences in the predictive capacities of the TiC GRS in those 2 populations may raise concerns about the replicability of TiC in other populations. As mentioned, MARTHA has a special design to put the new markers under additional stress to evaluate whether they add value to FVL+PT. It is a case–control study enriched in carriers of FVL and PT mutations, for which >50% of participants are carriers of 1 of these genetic variants. It is expected that the area under the receiver operating characteristic curve and the IDI and net reclassification improvement results could be modest when comparing TiC with FVL+PT because these 2 mutations account for an important part of the risk, especially if half of the participants are carriers. Despite this big effect of these 2 mutations (due to the study design), it is important to emphasize that TiC also significantly improved the predictive capacity of FVL+PT in MARTHA population. Moreover, we must consider the sensitivity and specificity of TiC because those classification functions, contrary to other statistical functions, are more strongly associated with the capacity of the test than with the specific characteristic of the population in which it is tested.41 The sensitivity and sensibility of TiC were the same in the SANTPAU and MARTHA populations (sensitivity 0.85 for both; specificity 0.25 versus 0.26 in SANTPAU and MARTHA, respectively). This supports the idea that TiC will have similar performance in other populations. Considering the number, weight, and pathological relevance of the genetic variants included in TiC, the significant improvement in area under the received operating character curve and IDI in MARTHA, and the similar sensitivity and specificity in both populations, it is reasonable to believe that TiC will be replicable.
Finally, replication of the present results in other populations, evaluation of the clinical utility (measured as reduction in VTE events by the use of TiC GRS), and analysis of the cost‐effectiveness of TiC remain to be accomplished.
In summary, to the best of our knowledge, this study is the first to assess the efficiency of new genetic markers of VTE, such as TiC, according to the recommendations of experts in the field of cardiovascular disease.37 By determining the discrimination and reclassification capacity of these markers and their clinical utility (measured as sensitivity) in comparison to conventional recognized risk models (FVL+PT or family history), we were able to conclude that by using new genetic markers, especially TiC, conventional VTE risk assessment algorithms are substantially improved. New risk scores such as the TiC GRS, proposed in this paper, should allow for more tailored thromboprophylaxis strategies and improve estimates of a patient's risk of VTE.
Supplementary Material
Appendix Supplementary Tables S1-S3.
Sources of Funding
Supported by Spanish grants RD12/0042/0032, RD12/0042/0061, PI 11/0184, PI 12/0612, and FI12/00322.
Disclosures
Moyano and Salas are employees of the company Gendiag.exe, S.L. assigned of the patent covering TiC GRS. Soria is a member of the Thrombo inCode Scientific Committee, advising Ferrer in Code.
References
- 1.Cohen AT, Agnelli G, Anderson FA, Arcelus JI, Berqvist D, Brech JG, Greer IA, Heit JA, Hutchinson JL, Kakkar AK, Mottier D, Oger E, Samana MM, Spannagl Mfor the VTE Impact Assessment Group in Europe (VITAE). Venous thromboembolism (VTE) in Europe. Thromb Haemost. 2007; 98:756-764. [DOI] [PubMed] [Google Scholar]
- 2.Anderson FA, Jr, Zayaruany M, Heit JA, Fidan D, Cohen AT. Estimated annual numbers of US acute‐care hospital patients at risk for venous thromboembolism. Am J Hematol. 2007; 82:777-782. [DOI] [PubMed] [Google Scholar]
- 3.Stein PD, Matta F. Epidemiology and incidence: the scope of the problem and risk factors for development of venous thromboembolism. Crit Care Clin. 2011; 27:907-932. [DOI] [PubMed] [Google Scholar]
- 4.Dobesh PP. Economic burden of venous thromboembolism in hospitalized patients. Pharmacotherapy. 2009; 29:943-953. [DOI] [PubMed] [Google Scholar]
- 5.Souto JC, Almasy L, Borrell M, Blanco‐Vaca F, Mateo J, Soria JM, Coll I, Felices R, Stone W, Fontcuberta J, Blangero J. Genetic susceptibility to thrombosis and its relationship to physiological risk factors: the GAIT study. Genetic analysis of idiopathic thrombophilia. Am J Human Genet. 2000; 67:1452-1459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Reitsma PH. Genetic heterogeneity in hereditary thrombophilia. Haemostasis. 2000; 30suppl 2:1-10. [DOI] [PubMed] [Google Scholar]
- 7.Kanaji T, Okamura T, Osaki K, Kuroiwa M, Shimoda K, Hamasaki N, Niho Y. A common genetic polymorphism (46 C to T substitution) in the 5′‐untranslated region of the coagulation factor XII gene is associated with low translation efficiency and decrease in plasma factor XII level. Blood. 1998; 91:2010-2014. [PubMed] [Google Scholar]
- 8.Van HylckamaVlieg A, Komansin N, Ariëns RAS, Poort SR, Grant PJ, Bertina RM, Rossendaal FR. Factor XIII Val34Leu polymorphism, factor XIII antigen levels and activity and the risk of deep venous thrombosis. BJH. 2002; 119:169-175. [DOI] [PubMed] [Google Scholar]
- 9.Williamson D., Brown K, Luddington R, Baglin C, Baglin T. Factor V Cambridge: a new mutation (Arg306Thr) associated with resistance to activated protein C. Blood. 1998; 91:1140-1144. [PubMed] [Google Scholar]
- 10.Norstrom E, Thorelli E, Dahlbäck B. Functional characterization of recombinant FV Hong Kong and FV Cambridge. Blood. 2002; 100:524-530. [DOI] [PubMed] [Google Scholar]
- 11.Bertina R, Koeleman BPC, Koster T, Rosendaal FR, Driven RJ, De Ronde H, Van der Velden PA, Reitsma PH. Mutation in blood coagulation factor V associated with resistance to activated protein C. Nature. 1994; 369:64-67. [DOI] [PubMed] [Google Scholar]
- 12.Poort SR, Rosendaal FR, Reitsma PH, Bertina RM. A common genetic variation in the 3′‐untranslated region of the prothrombin gene is associated with elevated plasma prothrombin levels and an increase in venous thrombosis. Blood. 1996; 88:3698-3703. [PubMed] [Google Scholar]
- 13.Corral J, Gonzalez‐Conejero R, Soria JM, Gonzalez‐Porras JR, Pérez‐Ceballos E, Lecumberri R, Roldán V, Souto JC, Miñano A, Hernández‐Espinosa D, Alberca I, Fontcuberta J, Vicente V. A nonsense polymorphism in the protein Z‐dependent protease inhibitor increases the risk for venous thrombosis. Blood. 2006; 108:177-183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Corral J, Hernandez‐Espinosa D, Soria JM, González‐Conejero R, Ordonez A, González‐Porras JR, Pérez‐Ceballos E, Lecumberri R, Sánchez I, Roldan V, Mateo J, Minano A, González M, Alberca I, Fontcuberta J, Vicente V. Antithrombin Cambridge II (A384S): an nderstimated genetic risk factor for venous thrombosis. Blood. 2007; 109:4258-4263. [DOI] [PubMed] [Google Scholar]
- 15.O'Donnell J, Boulton FE, Manning RA, Laffan MA. Amount of H antigen expressed on circulating von Willebrand factor is modified by ABO blood group genotype and is a major determinant of plasma von Willebrand factor antigen levels. Arterioscler Thromb Vasc Biol. 2002; 22:335-341. [DOI] [PubMed] [Google Scholar]
- 16.Morange PE, Tregouet DA. Lessons from genome‐wide association studies in venous thrombosis. J Thromb Haemost. 2011; 9suppl 1:258-264. [DOI] [PubMed] [Google Scholar]
- 17.Tang W, Teichert M, Chasman DI, Heit JA, Morange PE, Li G, Pankratz N, Leebeek FW, Paré G, de Andrade M, Tzourio C, Psaty BM, Basu S, Ruiter R, Rose L, Armasu SM, Lmley T, Heckbert SR, Uitterlinde AG, Lathrop M, Rice KM, Cushman M, Hofman A, Lambert JC, Glazer NL, Pankow JS, Witterman JC, Amouyel P, Bis JC, Bovill EG, Kong X, Tracy RP, Boerwinkle E, Rotter JI, Trégouët DA, Loth DW, Stricker BHC, Ridker PM, Folsom AR, Smith NL. A genome‐wide association study for venous thromboembolism: the extended cohorts for heart ad aging research in genomic epidemiology (CHARGE) consortium. Genet Epidemiol. 2013; 37:512-521. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Sabater‐Lleal M, Martinez‐Perez A, Buil A, Folkersen L, Souto JC, Bruzelius M, Borrell M, Odeberg J, Silveira A, Eriksson P, Almasy L, Hamsten A, Soria JM. A genome‐wide association study identifies KNG1 as a genetic determinant of plasma factor XI level and activated partial thromboplastin time. Arterioscler Thromb Vasc Biol. 2012; 32:2008-2016. [DOI] [PubMed] [Google Scholar]
- 19.De Haan HG, Bezemer ID, Doggen CJM, Le Cessie S, Reitsma PH, Arellano AR, Tong CH, Devlin JJ, Bare LA, Rosendaal FR, Vossen CY. Multiple SNP testing improves risk prediction of first venous thrombosis. Blood. 2012; 120:656-663. [DOI] [PubMed] [Google Scholar]
- 20.Tirado I, Soria JM, Mateo J, Oliver A, Souto JC, Santamaria A, Felices R, Borrell M, Fontcuberta J. Association after linkage analysis indicates that homozygosity for the 46C>T polymorphism in the F12 gene is a genetic risk factor for venous thrombosis. Thromb Haemost. 2004; 91:899-904. [DOI] [PubMed] [Google Scholar]
- 21.Trégouët DA, Heath S, Saut N, Biron‐Andreani C, Schved JF, Pernod G, Galan P, Drouet L, Zelenika D, Juhan‐Vague I, Alessi MC, Tiret L, Lathrop M, Emmerich J, Morange PE. Common susceptibility alleles are unlikely to contribute as strongly as the FV and AB0 loci to VTE risk: results from a GWAS approach. Blood. 2009; 113:5298-5303. [DOI] [PubMed] [Google Scholar]
- 22.Yip SP. Single‐tube multiplex PCR‐SSCP analysis distinguishes 7 common ABO alleles and readily identifies new alleles. Blood. 2000; 95:1487-1492. [PubMed] [Google Scholar]
- 23.Juul K, Tybjaerg‐Hansen A, Schnohr P, Nordestgaard BG. Factor V Leiden and the risk for venous thromboembolism in the adult Danish population. Ann Intern Med. 2004; 140:330-337. [DOI] [PubMed] [Google Scholar]
- 24.Wu O, Robertson L, Twaddle S, Lowe G, Clark P, Walker I, Brenkel I, Greaves M, Langhorne P, Regan L, Greer Ifor The Thrombosis: Risk and Economic Assessment of Thrombophilia Screening—TREATS‐Study. Screening for thrombophilia in high‐risk situations: a meta‐analysis and cost‐effectiveness analysis. Br J Haematol. 2005; 131:80-90. [DOI] [PubMed] [Google Scholar]
- 25.de Van Water N, Tan T, Ashton F, O'Grady A, Day T, Browett P, Ockelford P, Harper P. Mutations within the protein Z‐dependent protease inhibitor gene are associated with venous thromboembolic disease: a new form of thrombophilia. Br J Haematol. 2004; 127:190-194. [DOI] [PubMed] [Google Scholar]
- 26.Reuner KH, Jenetzky E, Aleu A, Litfin F, Mellado P, Kloss M, Jüttler E, Grau AJ, Rickmann H, Patscheke H, Lichy C. Factor XII c46T gene polymorphism and the risk of cerebral venous thrombosis. Neurology. 2008; 70:129-132. [DOI] [PubMed] [Google Scholar]
- 27.Cochery‐Nouvellon E, Mercier E, Lissalde‐Lavigne G, Daurès J‐P, Quéré I, Dauzat M, Marès P, Gris C. Homozygosity for the C46T polymorphism of the F12 gene is a risk factor for venous thrombosis during the first pregnancy. J Thromb Haemost. 2007; 5:700-707. [DOI] [PubMed] [Google Scholar]
- 28.Catto AJ, Kohler HP, Coore J, Mansfield MW, Stickland MH, Grant PJ. Association of a common polymorphism in the factor XIII gene with venous thrombosis. Blood. 1999; 93:906-908. [PubMed] [Google Scholar]
- 29.Wells PS, Anderson JL, Scarvelis DK, Doucette SP, Gagnon F. Factor XIII Val34Leu variant is protective against venous thromboembolism: a HuGE review and meta‐analysis. Am J Epidemiol. 2006; 164:101-109. [DOI] [PubMed] [Google Scholar]
- 30.Arellano AR, Bezemer ID, Tong CH, Catanese JJ, Devlin JJ, Reitsma PH, Bare LA, Rosendaal FR. Gene variants associated with venous thrombosis: confirmation in the MEGA study. J Thromb Haemost. 2010; 8:1132-1134. [DOI] [PubMed] [Google Scholar]
- 31.Austin H, De Staercke C, Lally C, Bezemer ID, Rosendaal FR, Hooper WC. New gene variants associated with venous thrombosis: a replication study in White and black Americans. J Thromb Haemost. 2011; 9:489-495. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Bezemer ID, Bare LA, Doggen CJ, Arellano AR, Tong C, Rowland CM, Catanese J, Young BA, Reitsma PH, Devlin JJ, Rosendaal FR. Gene variants associated with deep vein thrombosis. JAMA. 2008; 299:1306-1314. [DOI] [PubMed] [Google Scholar]
- 33.Uitte de Willige S, de Visser MC, Houwing‐Duistermaat JJ, Rosendaal FR, Vos HL, Bertina RM. Genetic variation in the fibrinogen gamma gene increases the risk of deep venous thrombosis by reducing plasma fibrinogen gamma levels. Blood. 2005; 106:4176-7183. [DOI] [PubMed] [Google Scholar]
- 34.Grünbacher G, Weger W, Max‐Neuhold E, Pilger E, Köppel H, Wascher T, März W, Renner W. The fibrinogen gamma (FGG) 10034C>T polymorphism is associated with venous thrombosis. Thromb Res. 2007; 121:33-36. [DOI] [PubMed] [Google Scholar]
- 35.Tirado I, Mateo J, Soria JM, Oliver A, Martínez‐Sánchez E, Vallvé C, Borrell M, Urrutia T, Fontcuberta J. The ABO blood group genotype and factor VIII levels as independent risk factors for venous thromboembolism. Thromb Haemost. 2005; 93:468-474. [DOI] [PubMed] [Google Scholar]
- 36.Morange PE, Oudot‐Mellakh T, Cohen W, Germain M, Saut N, Antoni G, Alessi MC, Bertrand M, Dupuy AM, Letenneur L, Lathrop M, Lopez LM, Lambert JC, Emmerich J, Amouyel P, trégoüet DA. KNG1 ile581Thr and susceptibility to venous thrombosis. Blood. 2011; 117:3692-3694. [DOI] [PubMed] [Google Scholar]
- 37.Hlatky MA, Greenland P, Arnett DK, Ballantyne CM, Criqui MH, Elkind MSV, Go AS, Harrell FE, Jr, Hong Y, Howard BV, Howard VJ, Hsue PY, Kramer CM, McConnell JP, Normand SLT, O′Donnell CJ, Smith SC, Jr, Wilson PWFon behalf of the American Heart Association Expert Panel on Subclinical Atherosclerotic Disease and Emerging risk factors and the Stroke Council. Criteria for evaluation of novel cardiovascular risk: a scientific statement from the American Heart Association. Circulation. 2009; 119:2408-2416. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Hanley JA, Haijian‐Tilaki KO. Sampling variability of nonparametric estimates of the areas under receiver operating characteristic curves: an update. Acad Radiol. 1997; 4:49-58. [DOI] [PubMed] [Google Scholar]
- 39.Pencina MJ, D'Agostino RB, Sr, D'Agostino RB, Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008; 27:157-172. [DOI] [PubMed] [Google Scholar]
- 40.Pencina MJ, D'Agostino RB, Pencina KM, Janssens AC, Greeland P. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol. 2012; 176:473-481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Lalkhen AG, McCluskey A. Clinical tests: sensitivity and specificity. Cont Educ Anaesth Crit Care Pain. 2008; 8:221-223. [Google Scholar]
- 42.R Development Core Team. R: a Language and Environment for Statistical Computing. 2010Vienna, Austria: R Foundation for Statistical Computing; http://www.R-roject.org/ [Google Scholar]
- 43.Varga EA, Kujovich JL. Management of inherited thrombophilia: guide for genetics professionals. Clin Genet. 2012; 81:7-17. [DOI] [PubMed] [Google Scholar]
- 44.Manolio TA, Collins FS, Cox NJ, Goldstein DB, Hindorff LA, Hunter DJ, McCarthy MI, Ramos EM, Cardon LR, Chakravarti A, Cho JH, Guttmacher AE, Kong A, Kruglyak L, Mardis E, Rotimi CN, Slatkin M, Valle D, Whittemore AS, Boehnke M, Clark AG, Eichler EE, Gibson G, Haines JL, Mackay TF, McCarroll SA, Visscher PM. Finding the missing heritability of complex diseases. Nature. 2009; 461:747-753. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Antoni G, Morange PE, Luo Y, Saut N, Burgos G, Heath S, Germain M, Biron‐Andreani C, Schved JF, Pernod G, Galan P, Zelanika D, Alessi MA, Drouet L, Visvikis‐Siest S, Wells PS, Lathrop M, Emmerich J, Tregouet DA, Gagnon F. A multi‐stage multi‐design strategy provides strong evidence that the BAI3 locus is associated with early‐onset venous thromboembolism. J Thromb Hemost. 2010; 8:2671-2679. [DOI] [PubMed] [Google Scholar]
- 46.Wu C, Dwivedi DJ, Pepler L, Lysov Z, Waye J, Julian J, Desch K, Ginsburg D, Weitz JI, Kearon C, Liaw P. Targeted gene sequencing identifies variants in the protein C and endothelial protein C receptor genes in patients with unprovoked venous thromboembolism. Arterioscler Thromb Vasc Biol. 2013; 33:2674-2681. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Zöller B, Li X, Sundquist J, Sundquist K. Age‐ and gender‐specific familial risks for venous thromboembolism: a nationwide epidemiological study base don hospitalizations in Sweden. Circulation. 2011; 124:1012-1020. [DOI] [PubMed] [Google Scholar]
- 48. Antithrombotic therapy and prevention of thrombosis, 9th ed: American College of Chest Physicians Evidence‐Based Clinical Practice Guidelines. Chest. 2012; 142 suppl:e1S-e801S. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix Supplementary Tables S1-S3.