Abstract
Objectives
Coronavirus disease 2019 (COVID-19) reportedly causes thromboembolic complications due to coagulopathy with hypercoagulability and a hypofibrinolytic state. We evaluated the time-course of coagulopathy in patients with severe COVID-19 from admission to discharge from our intensive-care unit (ICU).
Methods
We conducted a retrospective study of adults with severe COVID-19 admitted to our ICU between January 20, 2021, and March 31, 2022. We obtained clinical information, laboratory data, and rotational thromboelastometry (ROTEM) parameters at admission and discharge.
Results
Fifteen patients were included. Fibrinogen and D-dimer values did not change significantly but were above the normal ranges at admission and discharge. Regarding ROTEM parameters, the maximum clot firmness in fibrinogen function (FIBTEM), a marker of hypercoagulability, did not change significantly but was above the normal range at admission and discharge [median (interquartile range), admission vs. discharge: 31 (25-34) mm vs. 31 (27-32) mm, p=0.589]. The maximum lysis at 60 minutes in the extrinsic coagulation pathway (EXTEM) and intrinsic coagulation pathway (INTEM), as markers of the fibrinolytic function, were both significantly lower at discharge than at admission [median (interquartile range), admission vs. discharge: EXTEM, 3 (2-4) vs. 1 (0-2), p=0.011; INTEM, 3 (1-6) vs. 1 (0-2), p=0.008].
Conclusion
This study revealed a persistent hypercoagulable state at ICU discharge and a worse hypofibrinolytic state at discharge than at admission. These results may contribute to a better understanding of coagulopathies in the acute to subacute phases of severe COVID-19.
Keywords: coagulopathy, COVID-19, hypercoagulability, hypofibrinolysis, thromboelastometry
Introduction
Coronavirus disease 2019 (COVID-19) is characterized by a markedly increased inflammatory response accompanied by coagulopathy, and severe COVID-19 causes acute respiratory failure and thromboembolic events, which are associated with a coagulopathic state (1-3). Hypercoagulability and a hypofibrinolytic state are characteristic findings in patients with COVID-19 (4-12). Although a number of reports have described the coagulation parameters or patterns of rotational thromboelastometry (ROTEM) at intensive-care unit (ICU) admission, there are few reports describing coagulopathies in the subacute phase of severe COVID-19, such as at discharge from the ICU or hospital. It is important for clinicians to recognize how long coagulopathies persist in patients with severe COVID-19 in order to determine how long anticoagulant therapy should be continued.
In this study, we evaluated and compared the coagulation parameters and patterns of ROTEM at ICU admission and discharge.
Materials and Methods
Study design
This was a single-center, retrospective, observational study of adults with severe COVID-19 pneumonia admitted to our critical-care center between January 20, 2021, and March 31, 2022. Consecutive patients ≥18 years old who required mechanical ventilation for severe COVID-19 were included in this study. We retrospectively collected the clinical and laboratory data recorded during each patient’s ICU stay.
This study followed the principles of the Declaration of Helsinki. This study was approved by IRB of Yamaguchi University Hospital (No. 2021-205).
Data collection
We collected the following data: age, sex, body mass index (BMI), pre-existing comorbidities, Acute Physiology and Chronic Health Evaluation (APACHE)-II score on admission, laboratory data, ROTEM parameters, therapeutic interventions in our hospital, thromboembolic events, major bleeding complications, total length of ICU stay, and ICU and hospital mortality. These data were collected from the patients’ medical records.
ROTEM parameters
A ROTEM sigma (Tem International, Munich, Germany) was used for the thromboelastric analyses to assess the intrinsic coagulation pathway (INTEM), extrinsic coagulation pathway (EXTEM), and fibrinogen function (FIBTEM). The following ROTEM variables were analyzed: clotting time (CT) was defined as the time in seconds until the clot strength reached 2 mm and reflected the kinetics of clot formation; clot formation time (CFT) was defined as the time in seconds until the clot strength reached 20 mm and reflected the kinetics of clot formation; maximum clot firmness (MCF) was defined as the maximum amplitude of clot firmness; lysis index 60 (LI60) was defined as the clot firmness at 60 minutes as a percentage of the MCF; and maximum lysis (ML) 60 was defined as the difference between the MCF and the lowest clot amplitude after MCF at 60 minutes and reflected the fibrinolytic activity. In our institution, ROTEM was routinely performed in severe COVID-19 patients on admission to the ICU and at ICU discharge starting in January 2021.
Statistical analyses
We compared the laboratory data and the ROTEM parameters between ICU admission and ICU discharge. The results are presented as the median [interquartile range (IQR)] or number (percentage) of patients, as appropriate. Continuous variables were compared using Wilcoxon’s signed-rank test, and categorical variables were compared using McNemar’s test. The threshold of significance was set at p<0.05.
All analyses were performed using Statistical Package for Social Sciences version 19.0 (SPSS, Chicago, USA).
Results
Fifteen patients with severe COVID-19 pneumonia requiring mechanical ventilation who underwent ROTEM upon ICU admission and at ICU discharge were included in this study. The baseline patient characteristics are shown in Table 1. The median (IQR) age was 59 (54-71) years old, 80.0% were men, and the BMI was 27.1 (25.8-28.3) kg/m2. There were no patients with a history of VTE or anticoagulant usage before admission. The arterial partial pressure of oxygen/inspired oxygen concentration ratio just after intubation following ICU admission and the APACHE-II score were 125 (98-153) and 17 (13-30), respectively. All patients were administered remdesivir, dexamethasone, and a prophylactic dose of unfractionated heparin. One patient (6.7%) had thrombotic complications (deep venous thrombosis and pulmonary embolism), but none had bleeding complications. The duration of mechanical ventilation and ICU stay were 9 (8-12) days and 15 (12-18) days, respectively.
Table 1.
Baseline Characteristics of Patients with Severe COVID-19 Pneumonia.
| All patients (n=15) | ||
|---|---|---|
| Age, median (IQR) | 59 (54-71) | |
| Sex, n (%) | ||
| Male | 12 (80.0) | |
| BMI, kg/m2 | 27.1 (25.8-28.3) | |
| Comorbidities, n (%) | ||
| Hypertension | 12 (80.0) | |
| Diabetes | 9 (60.0) | |
| Hyperlipidemia | 6 (40.0) | |
| Chronic kidney disease | 2 (13.3) | |
| Previous VTE | 0 (0) | |
| Anticoagulant therapy before admission, n (%) | 0 (0) | |
| COVID-19 vaccination, n (%) | 1 (6.7) | |
| Time of onset | ||
| The 3rd wave in Japan (2020.10-2021.2), n (%) | 1 (6.7) | |
| The 4th wave in Japan (2021.4-2021.5), n (%) | 9 (60.0) | |
| The 5th wave in Japan (2021.7-2021.9), n (%) | 3 (20.0) | |
| The 6th wave in Japan (2022.1-2022.3), n (%) | 2 (13.3) | |
| Intubation, n (%) | 15 (100) | |
| ECMO, n (%) | 1 (6.7) | |
| P/F ratio after intubation, median (IQR) | 125 (98-153) | |
| APACHE II score, median (IQR) | 17 (13-30) | |
| Pharmacological treatment for COVID-19 | ||
| Remdesivir, n (%) | 15 (100) | |
| Dexamethasone, n (%) | 15 (100) | |
| Tocilizumab, n (%) | 1 (6.7) | |
| Baricitinib, n (%) | 3 (20.0) | |
| Unfractionated heparin, n (%) | 15 (100) | |
| At ICU discharge | ||
| Oxygen administration, n (%) | 14 (93.3) | |
| Steroid administration, n (%) | 7 (46.7) | |
| Antiplatelet administration, n (%) | 3 (20.0) | |
| Duration of mechanical ventilation (days), median (IQR) | 9 (8-12) | |
| ICU stay (days), median (IQR) | 15 (12-18) | |
| ICU mortality, n (%) | 0 (0) | |
| Hospital mortality, n (%) | 0 (0) | |
| Deep venous thrombosis, n (%) | 1 (6.7) | |
| Pulmonary embolism, n (%) | 1 (6.7) | |
| Other thromboembolic events, n (%) | 0 (0) | |
| Major bleeding, n (%) | 0 (0) |
COVID-19: coronavirus disease 2019, IQR: interquartile range, BMI: body mass index, VTE: venous thromboembolism, ECMO: extracorporeal membrane oxygenation, P/F: arterial partial pressure of oxygen/inspired oxygen concentration, APACHE-II: Acute Physiology and Chronic Health Evaluation-II, ICU: intensive care unit
Laboratory parameters at ICU admission and ICU discharge are shown in Table 2. The C-reactive protein (CRP) level was higher at ICU admission than at ICU discharge [admission vs. discharge: 10.77 (4.22-14.22) mg/dL vs. 1.42 (0.92-2.70) mg/dL, p=0.005]. Regarding standard coagulation parameters, the platelet count was within the normal range at admission and discharge [18.6 (14.7-29.1) ×1010/L vs. 22.8 (18.3-32.6) ×1010/L, p=0.427]. The activated partial thromboplastin time (APTT) was within the normal range at both admission and discharge but was significantly lower at discharge than at admission [APTT: 35.8 (30.3-41.1) seconds vs. 30.3 (27.5-34.3) seconds, p=0.041]. Fibrinogen and D-dimer values did not change significantly between admission and discharge, but the values were above the normal ranges at both admission and discharge [fibrinogen: 507 (458-658) mg/dL vs. 562 (486-611) mg/dL, p=0.609; D-dimer: 1.3 (1.1-2.5) mg/L vs. 2.0 (1.6-2.8) mg/L, p=0.410].
Table 2.
Laboratory Parameters at ICU Admission and Discharge.
| Normal range | ICU admission | ICU discharge | p value | |||||
|---|---|---|---|---|---|---|---|---|
| WBC (×106/L), median (IQR) | 3,300-8,600 | 8,590 (6,830-9,700) | 9,670 (7,420-11,930) | 0.182 | ||||
| CRP (mg/dL), median (IQR) | 0.00-0.14 | 10.77 (4.22-14.22) | 1.42 (0.92-2.70) | 0.005 | ||||
| LDH (U/L), median (IQR) | 124-222 | 423 (383-560) | 288 (268-316) | 0.011 | ||||
| Plt (×1010/L), median (IQR) | 15.8-34.8 | 18.6 (14.7-29.1) | 22.8 (18.3-32.6) | 0.427 | ||||
| PT (%), median (IQR) | 80.0-120.0 | 78.8 (73.6-94.9) | 95.7 (86.1-102.0) | 0.394 | ||||
| PT-INR, median (IQR) | - | 1.12 (1.03-1.17) | 1.02 (0.99-1.09) | 0.255 | ||||
| APTT (s), median (IQR) | 23.9-39.7 | 35.8 (30.3-41.1) | 30.3 (27.5-34.3) | 0.041 | ||||
| FIB (mg/dL), median (IQR) | 200-400 | 507 (458-658) | 562 (486-611) | 0.609 | ||||
| D-dimer (mg/L), median (IQR) | 0.0-1.0 | 1.3 (1.1-2.5) | 2.0 (1.6-2.8) | 0.410 |
ICU: intensive care unit, WBC: white blood cell count, IQR: interquartile range, CRP: C-reactive protein, LDH: lactate dehydrogenase, Plt: platelet count, PT: prothrombin time, INR: international normalized ratio, APTT: activated partial thromboplastin time, FIB: fibrinogen
The ROTEM parameters measured at ICU admission and discharge are shown in Table 3 and typical temograms of patients with severe COVID-19 are shown in Figure. The MCF in FIBTEM was above the reference range at both admission and ICU discharge [admission vs. discharge: 31 (25-34) mm vs. 31 (27-32) mm, p=0.589]. The CT in EXTEM was significantly lower at discharge than at admission [74 (65-78) seconds vs. 60 (58-65) seconds, p=0.012]. The MCF values in EXTEM and INTEM were within the normal ranges but close to the upper limit of the normal ranges at both admission and discharge [EXTEM-MCF: 70 (68-77) mm vs. 72 (70-76) mm, p=0.690; INTEM-MCF: 66 (65-74) s vs. 70 (68-72) mm, p=0.425]. Although the LI60 values in EXTEM and INTEM were within the normal ranges at both admission and discharge, they were significantly higher at discharge than at admission [EXTEM-LI60: 97 (96-99) vs. 99 (98-100), p=0.011; INTEM-LI60: 97 (96-99) vs. 99 (98-100), p=0.008]. The ML60 values in EXTEM and INTEM were significantly lower at ICU discharge than at ICU admission [EXTEM-ML60: 3 (2-4) vs. 1 (0-2), p=0.011; INTEM-ML60: 3 (1-6) vs. 1 (0-2), p=0.008].
Table 3.
ROTEM Parameters at ICU Admission and Discharge.
| Normal range | ICU admission | ICU discharge | p value | |||||
|---|---|---|---|---|---|---|---|---|
| FIBTEM | ||||||||
| MCF, median (IQR) | 6-21 | 31 (25-34) | 31 (27-32) | 0.589 | ||||
| EXTEM | ||||||||
| CT (s), median (IQR) | 50-80 | 74 (65-78) | 60 (58-65) | 0.012 | ||||
| CFT (s), median (IQR) | 46-149 | 51 (45-57) | 45 (37-53) | 0.319 | ||||
| MCF (mm), median (IQR) | 55-72 | 70 (68-77) | 72 (70-76) | 0.690 | ||||
| LI60 (%), median (IQR) | 94-100 | 97 (96-99) | 99 (98-100) | 0.011 | ||||
| ML60 (%), median (IQR) | - | 3 (2-4) | 1 (0-2) | 0.011 | ||||
| INTEM | ||||||||
| CT (s), median (IQR) | 161-204 | 205 (179-222) | 171 (155-205) | 0.105 | ||||
| CFT (s), median (IQR) | 62-130 | 59 (49-72) | 57 (45-64) | 0.378 | ||||
| MCF (mm), median (IQR) | 51-69 | 66 (65-74) | 70 (68-72) | 0.425 | ||||
| LI60 (%), median (IQR) | 87-100 | 97 (96-99) | 99 (98-100) | 0.008 | ||||
| ML60 (%), median (IQR) | - | 3 (1-6) | 1 (0-2) | 0.008 |
ICU: intensive care unit, FIBTEM: fibrinogen function, MCF: maximum clot firmness, IQR: interquartile range, CFT: clot formation time, CT: clotting time, LI60: lysis index at 60 minutes, ML60: maximum lysis at 60 minutes
Figure.
Typical rotational thromboelastometry temograms in a healthy volunteer (a) as well as at ICU admission (b) and discharge (c) in a patient with severe COVID-19. A, clot amplitude at 5, 10, 20, or 30 minutes; CFT: clot formation time, COVID-19: coronavirus disease 2019, CT: coagulation time, EXTEM: extrinsic coagulation pathway, FIBTEM: fibrinogen function, ICU: intensive care unit, INTEM: intrinsic coagulation pathway, LI: lysis index at 30, 45, or 60 minutes as a percentage of the MCF, MCF: maximum clot firmness
Discussion
In this study, we evaluated the coagulation parameters and ROTEM parameters at ICU admission and discharge in patients with severe COVID-19 requiring mechanical ventilation. We revealed that a hypercoagulable and hypofibrinolytic state is present not only at ICU admission but also at ICU discharge in patients with severe COVID-19.
Patients with severe COVID-19 were previously reported to exhibit coagulopathies characterized by increased fibrinogen and D-dimer in conventional coagulation tests, and COVID-19-related coagulopathies were found to increase the risk of arterial and venous thrombosis (1-11). Our data demonstrate the existence of a hypercoagulable state with increased fibrinogen at ICU admission that persisted until ICU discharge. The coagulation parameter APTT was significantly lower at ICU discharge than at admission. These results suggest that the hypercoagulable state persists through to ICU discharge in patients with severe COVID-19.
Although coagulation parameters like PT-INR, APTT, and fibrinogen are readily available, they may not fully reveal the coagulation state, as they only reflect a partial component of the coagulation cascade. Thromboelastometry modalities, such as ROTEM, may be more useful than conventional coagulation test for evaluating coagulopathies, as these methods evaluate the overall coagulation process based on a whole blood sample. Hincker et al. reported that patients who underwent major non-cardiac surgery and experienced postoperative thromboembolic complications had significantly shorter EXTEM-CFT and INTEM-CFT values and higher FIBTEM-MCF, EXTEM-MCF, and INTEM-MCF values at preoperative ROTEM examinations than patients without such complications (12). Furthermore, they found that the ROTEM results might predict the risk of postoperative thromboembolic complications. Similarly, EXTEM-CFT and INTEM-CFT values were shorter and FIBTEM-MCF, EXTEM-MCF, and INTEM-MCF values were greater than the reference range in patients with COVID-19 (5-7,11,13,14). Our ROTEM results at ICU admission and discharge were generally consistent with these reports (5,6,8,11,13-19).
Another advantage of thromboelastometry is its ability to assess the fibrinolysis state. In a previous study, many patients with severe COVID-19 exhibited a hypofibrinolytic state, especially when assessed with a viscoelastic device (8,9,11). Furthermore, this hypofibrinolytic state was potentially associated with thromboembolic complications with severe COVID-19 patients (8,9,11). Kruse et al. reported that critically ill patients with COVID-19 and thromboembolic events had significantly lower EXTEM-ML and INTEM-ML values than patients without thromboembolic events, and lower EXTEM-ML values indicated a hypofibrinolytic state (8). Similarly, Creel-Bulos et al. reported that critically ill patients COVID-19 with a hypofibrinolytic state had a significantly higher rate of thromboembolic events than those without a hypofibrinolytic state (11). These reports suggest that a hypofibrinolytic state plays an important role in thromboembolic complications in patients with severe COVID-19. In addition, we found that the hypofibrinolytic state in patients with severe COVID-19 did not resolve and was actually worse at ICU discharge than at admission. These results suggest that patients with severe COVID-19 have a high risk of venous thromboembolism after ICU discharge.
In patients with COVID-19, a previous study showed that EXTEM-CFT and INTEM-CFT values were shorter and FIBTEM-MCF, EXTEM-MCF, and INTEM-MCF values were greater than reference ranges on admission, and these results were consistent with our own findings (5-7,11,13). However, there have been few studies reporting the changes in these ROTEM parameters after treatment. Pavoni et al. reported ROTEM results in COVID-19 patients admitted to the ICU measured at Days 1, 5, and 10 of the ICU stay. In their study, the hypercoagulable state (indicated by a shortened CFT or increased MCF on ROTEM) persisted to Day 5 but was ameliorated by Day 10 (14). These results were inconsistent with our data, which indicated a persistent hypercoagulable and hypofibrinolytic state at ICU discharge (median length of ICU stay: 14 days). The reason for this difference might be differences in the severity of COVID-19 between the two studies, as our patients were in a worse condition upon admission to the ICU and had a higher ratio of requiring mechanical ventilation than Pavoni’s patients. In that sense, this was the first report evaluating coagulopathy using ROTEM at ICU admission and discharge in severe COVID-19 patients requiring mechanical ventilation.
Several limitations associated with the present study warrant mention. First, this was a single-center retrospective study with a small number of patients. Second, the therapeutic courses of the patients differed before admission to our ICU; some patients were treated for several days in the general ward, while others were directly transferred from home to the ICU by ambulance. Third, the patients were not followed up after discharge from our hospital. Therefore, the longer-term outcomes, including conventional parameters, ROTEM parameters, or thromboembolic events, were not available. Fourth, whether or not our findings are specific to COVID-19 is unclear, as we did not compare the data with those of patients without COVID-19.
Conclusion
Among patients admitted to the ICU with severe COVID-19 pneumonia, coagulopathies characterized by a hypercoagulable and hypofibrinolytic state persisted until ICU discharge. These results may support our understanding of the associated coagulopathies and risk of thromboembolic events during the acute and subacute phases of severe COVID-19.
The authors state that they have no Conflict of Interest (COI).
References
- 1. Ribes A, Vardon-Bounes F, Memier V, et al. Thromboembolic events and COVID-19. Adv Biol Regul 77: 100742, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Nahum J, Morichau-Beauchant T, Daviaud F, et al. Venous thrombosis among critically ill patients with coronavirus disease 2019 (COVID-19). JAMA Netw 3: e2010478, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Klok FA, Kruip MJHA, van der Meer NJM, et al. Incidence of thrombotic complications in critically ill ICU patients with COVID-19. Thromb Res 191: 145-147, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Iba T, Levy JH, Levi M, Thacil J. Coagulopathy in COVID-19. J Thromb Haemost 18: 2103-2109, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Chaudhary R, Kreutz RP, Bliden KP, Tantry US, Gurbel PA. Personalizing antithrombotic therapy in COVID-19: role of thromboelastography and thromboelastometry. Thromb Haemost 120: 1594-1596, 2020. [DOI] [PubMed] [Google Scholar]
- 6. Nougier C, Benoit R, Simon M, et al. Hypofibrinolytic state and high thrombin generation may play a major role in SARS-CoV2 associated thrombosis. J Thromb Haemost 18: 2215-2219, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Coccheri S. COVID-19: the crucial role of blood coagulation and fibrinolysis. Intern Emerg Med 15: 1369-1373, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Kruse JM, Magomedov A, Kurreck A, et al. Thromboembolic complications in critically ill COVID-19 patients are associated with impaired fibrinolysis. Crit Care 24: 676, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Wright FL, Vogler TO, Moore EE, et al. Fibrinolysis shutdown correlation with thromboembolic events in severe COVID-19 infection. Am Coll Surg 231: 193-203, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Helms J, Tacquard C, Severac F, et al. High risk of thrombosis in patients with severe SARS-CoV-2 infection: a multicenter prospective cohort study. Intensive Care Med 46: 1089-1098, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Creel-Bulos C, Auls SA, Caridi-Scheible M. Fibrinolysis shutdown and thrombosis in a COVID-19 ICU. Shock 55: 845-846, 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Hincker A, Feit J, Sladen RN, Wagener G. Rotational thromboelastometry predicts thromboembolic complications after major non-cardiac surgery. Crit Care 18: 549, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Almskog LM, Wikman A, Svensson J, et al. Rotational thromboelastometry results are associated with care level in COVID-19. Thromb J Thrombolysis 51: 437-445, 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Pavoni V, Gianesello L, Pazzi M, Stera C, Meconi T, Frigieri FC. Evaluation of coagulation function by rotation thromboelastometry in critically ill patients with severe COVID-19 pneumonia. J Thromb Thrombolysis 50: 281-286, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Tsantes AE, Tsantes AG, Kokoris SI, et al. COVID-19 infection-related coagulopathy and viscoelastic methods: a paradigm for their clinical utility in critical illness. Diagnostics 10: 817, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Ibanez C, Perdomo J, Calvo A, et al. High D dimers and low global fibrinolysis coexist in COVID-19 patients: what is going on in there. J Thromb Thrombolysis 51: 308-312, 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Mazzeffi MA, Chow JH, Tanaka K. COVID-19 associated hypercoagulability: manifestations, mechanism, and management. Shock 55: 465-471, 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Collett LW, Gluck S, Strickland RM, Reddi BJ. Evaluation of coagulation status using viscoelastic testing in intensive care patients with coronavirus disease 2019 (COVID-19): an observational point prevalence cohort study. Aust Crit Care 34: 155-159, 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Meizoso JP, Moore HB, Moore EE. Fibrinolysis shutdown in COVID-19: clinical manifestations, molecular mechanisms, and therapeutic implications. Am Coll Surg 232: 995-1003, 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]