Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2026 Apr 14.
Published in final edited form as: Crit Care Med. 2013 Oct;41(10):2344–2353. doi: 10.1097/CCM.0b013e31828e9a49

RBC Transfusion Practices Among Critically Ill Patients: Has Evidence Changed Practice?

David J Murphy 1, Dale M Needham 2, Giora Netzer 3, Scott L Zeger 4, Elizabeth Colantuoni 4, Paul Ness 5, Peter J Pronovost 6, Sean M Berenholtz 6
PMCID: PMC13075646  NIHMSID: NIHMS2163602  PMID: 23939350

Abstract

Objective:

Increasing evidence, including publication of the Transfusion Requirements in Critical Care trial in 1999, supports a lower hemoglobin threshold for RBC transfusion in ICU patients. However, little is known regarding the influence of this evidence on clinical practice over time in a large population-based cohort.

Design:

Retrospective population-based cohort study.

Setting:

Thirty-five Maryland hospitals.

Patients:

Seventy-three thousand three hundred eighty-five non-surgical adults with an ICU stay greater than 1 day between 1994 and 2007.

Interventions:

None.

Measurements and Main Results:

The unadjusted odds of patients receiving an RBC transfusion increased from 7.9% during the pre-Transfusion Requirements in Critical Care baseline period (1994–1998) to 14.7% during the post-Transfusion Requirements in Critical Care period (1999–2007). A logistic regression model, including 40 relevant patient and hospital characteristics, compared the annual trend in the adjusted odds of RBC transfusion during the pre- versus post-Transfusion Requirements in Critical Care periods. During the pre-Transfusion Requirements in Critical Care period, the trend in the adjusted odds of RBC transfusion did not differ between hospitals averaging > 200 annual ICU discharges and hospitals averaging ≤ 200 annual ICU discharges (odds ratio, 1.07 [95% CI, 1.01–1.13] annually and 1.03 [95% CI, 0.99–1.07] annually, respectively; p = 0.401). However, during the post-Transfusion Requirements in Critical Care period, the adjusted odds of RBC transfusion decreased over time in higher ICU volume hospitals (odds ratio, 0.96 [95% CI, 0.93–0.98] annually) but continued to increase in lower ICU volume hospitals (odds ratio, 1.10 [95% CI, 1.08–1.13] annually), p < 0.001.

Conclusions:

In this population-based cohort of ICU patients, the unadjusted odds of RBC transfusion increased in both higher and lower ICU volume hospitals both before and after Transfusion Requirements in Critical Care publication. After adjusting for relevant characteristics, the odds continued to increase in lower ICU volume hospitals in the post-Transfusion Requirements in Critical Care period, but it decreased in higher ICU volume hospitals. This suggests that evidence supporting restrictive RBC transfusion thresholds may not be uniformly translated into practice in different hospital settings.

Keywords: blood transfusion, health services research, intensive care units, outcome and process assessment (healthcare), quality of healthcare


RBC transfusions are common and potentially lifesaving in selected critically ill patients (13). Physicians historically maintained hemoglobin (Hgb) concentrations greater than 10 g/dL based on pathophysiologic reasoning that this level was necessary for adequate tissue oxygenation (4, 5). However, more recent empirical research has demonstrated that RBC transfusion can increase patients’ risk for acute lung injury, healthcare-associated infections, and hospital mortality (68).

The landmark Transfusion Requirements in Critical Care (TRICC) trial demonstrated the safety of a restrictive RBC transfusion threshold (i.e., Hgb ≤ 7 g/dL) versus a more liberal threshold (i.e., Hgb ≤ 10 g/dL) (9). This multicenter trial concluded that patients randomized to the restrictive transfusion strategy had a trend toward improved 30-day mortality rates compared with the patients in the liberal strategy group, with better 30-day mortality rates in younger patients, and those with lower severity of illness. A single-center study suggested that ICU transfusion strategies may have become more restrictive over time (10); however, overall limited evidence exists regarding the impact of this evidence on transfusion practices over time (11).

Our objective was to assess changes in RBC transfusion practices over a 14-year period in a statewide population-based sample of hospitals. We describe the odds of ICU patients receiving an RBC transfusion during their hospital stay, estimate the average annual change in RBC transfusions before and after the TRICC publication, and evaluate factors influencing transfusions.

MATERIALS AND METHODS

Study Design and Data Source

We performed a retrospective longitudinal cohort study of patients admitted to an ICU in the state of Maryland between 1994 and 2007. Data for this study were obtained from the Maryland Health Services Cost Review Commission (HSCRC) database, which contains discharge and billing information on all patients treated in nonfederal Maryland hospitals. This database has been used extensively for research purposes including studies evaluating transfusion practices (1215). The 14-year study duration includes 5 years prior to the 1999 publication of the TRICC trial (pre-TRICC period, 1994–1998) and the subsequent 9 years for which complete data were available at the time of this study (post-TRICC period, 1999–2007) (9).

Study Population

Data were extracted on all nonsurgical patients 18 years old or older admitted to an acute care hospital with an ICU length of stay (LOS) greater than 1 day. This ICU cohort excluded records with an HSCRC variable indicating a hospital discharge within the prior 30 days to minimize nonindependence of the observations. For each patient, we extracted patient demographic, clinical, and treatment data including up to 15 International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9–CM) diagnosis and procedure codes (16).

Surgical patients were not included in the study because the dataset limited our ability to evaluate RBC transfusions due to intraoperative blood loss. To maintain consistency with the TRICC eligibility criteria, our cohort was further refined by excluding patients with active hemorrhage, active myocardial ischemia, chronic anemia, or a pregnancy-related hospitalization (9, 10, 1719). Patients with active hemorrhage or myocardial ischemia were identified using validated ICD-9–CM codes used in published studies (10, 17, 18). Patients with chronic anemia were excluded based on ICD-9–CM codes corresponding to the Elixhauser comorbidity measures of “iron deficiency anemia” and “other deficiency anemia” (19) and those with a pregnancy-related hospitalization were identified using the relevant major diagnostic category (20). In addition, if no RBC transfusions were reported by a hospital for an entire year, all discharges from that hospital during that year were excluded to minimize reporting bias.

Outcome Variable

The primary outcome, RBC transfusion, was based on the presence of an ICD-9–CM procedure code for allogeneic RBC transfusion (99.04) in the discharge record (16). The ability of the HSCRC database to identify patients receiving a RBC transfusion has been previously validated and used in other population-based studies examining transfusion practices (14, 15, 21). This code yields a dichotomous variable (i.e., transfused vs not transfused) during the entire hospital stay and does not measure the indication for transfusion (e.g., pretransfusion hemoglobin), timing of transfusion (i.e., before, during, or after ICU admission), number of units transfused, or the provider deciding to transfuse.

Primary Exposure and Covariates

The primary exposure variable was the calendar year of discharge. We also included multiple patient and hospital covariates that may have influenced the association between the primary exposure and outcome variables. Patient covariates were related to demographic characteristics (i.e., age, sex, and race), comorbid illnesses (i.e., 28 individual Elixhauser comorbidity measures) (19), and measures of a patient’s hospital course. Hospital course was characterized using the following five variables: admission via the emergency department, principal discharge diagnostic category, any diagnosis of sepsis during hospitalization (22, 23), ICU LOS, and non-ICU LOS (i.e., total hospital LOS minus ICU LOS).

Hospital covariates included ICU volume and teaching status. Prior literature suggests that transfusion practices may differ based on ICU patient volume (2, 14, 24, 25). Hospitals were dichotomized as higher ICU volume (> 200 ICU admissions/yr) and lower ICU volume (≤ 200 ICU admissions/yr) based on the mean annual number of ICU admissions during the baseline study period (26). Teaching status was determined by reviewing the list of institutions accredited by the Accreditation Council for Graduate Medical Education (27). To account for potential differences in overall diagnosis and procedure coding practices between hospitals and temporal trends over the 14-year study period, the average number of diagnoses and procedures coded per eligible patient for each hospital year were calculated and included as a covariate (15).

Statistical Analysis

Descriptive analyses of the patient and hospital characteristics associated with the pre-TRICC (1994–1998) and post-TRICC (1999–2007) periods were performed. Covariates were modeled based on the published literature (14, 15, 19, 22, 23). When such information was not available, we examined a scatterplot of the covariate and outcome variable using a locally weighted regression to determine appropriate modeling (28, 29).

We developed a multivariable logistic regression model that included the previously described patient and hospital covariates to evaluate the independent association of time (i.e., calendar year) with the odds of RBC transfusion. In this model, calendar year was modeled continuously with a linear spline to evaluate for a difference in the average annual change in the odds of transfusion during the pre-TRICC baseline period (1994–1998) versus post-TRICC period (1999–2007). The linear spline allowed for statistical comparison of the average annual change in transfusions between the two periods. A second, otherwise identical, regression model using 14 indicator variables for each admission year was also created to visually assess the appropriateness of modeling discharge year as a continuous variable with spline term in the primary model.

We accounted for potential statistical clustering within each hospital using a hospital-level random effect with robust variance estimation (29, 30). An interaction between hospital ICU volume and calendar year was identified; thus, the model included separate admission year spline terms for higher and lower ICU volume hospitals. Variable inflation factors were used to check for multicollinearity in the model (29).

All analyses were performed using STATA v10.1 (StataCorp, College Station, TX). Descriptive statistics were reported using mean and sd for continuous data and proportions for categorical data. Results from the logistic regression models were reported as odds ratios (OR) (95% CI). A two-sided p < 0.05 was used to determine statistical significance. The Institutional Review Board at the Johns Hopkins School of Medicine approved the study.

RESULTS

From 1994 to 2007, 155,420 nonsurgical adult admissions with an ICU LOS greater than 1 day occurred (Fig. 1). Of these, 73,385 admissions met inclusion criteria. Within this cohort, 21,958 admissions (30%) from 29 hospitals occurred during the pre-TRICC period and 51,427 admissions (70%) from 35 hospitals were during the post-TRICC period. The mean number of eligible patients admitted annually to each hospital was 123 for lower ICU volume hospitals and 403 for higher volume hospitals. Teaching hospitals comprised 30% of lower ICU volume hospitals and 42% of higher volume hospitals.

Figure 1.

Figure 1.

Study flow diagram.

Patient Characteristics by Study Period

Patient characteristics in the pre-TRICC period and the post-TRICC period are reported in Table 1. The prevalence of most comorbid illnesses increased from the pre- to post-TRICC period, including renal failure (5.7–13.3%), liver disease (1.7–3.4%), coagulopathy (4.2–9.2%), and sepsis (10.6–20.8%). Both the mean ICU and non-ICU hospital LOSs remained relatively constant between the study periods (4.2 d vs 4.3 d and 3.8 d vs 3.6 d, respectively).

TABLE 1.

Cohort Characteristics by Study Period and Average Annual ICU Patient Volume Category

All Hospitals Lower Volume (< 200) Higher Volume (≥ 200)
Variable 1994–1998 1999–2007 1994–1998 1999–2007 1994–1998 1999–2007
Patients, n 21,958 51,427 7,791 18,410 14,167 33,017
Hospitals, n 29 35 19 23 10 12
Patient characteristics
 Age (yr), mean (sd) 62.3 (17.9) 62.6 (17.8) 60.2 (18.4)b 61.8 (18.0)b 63.5 (17.6)b 63.0 (17.6)b
 Female (%) 51.0b 50.1b 50.0 49.3 51.5b 50.5b
 Caucasian (%) 78.5b 66.7b 70.9b 61.6b 82.8b 69.5b
 Comorbidities (%)
  Congestive heart failure 16.7b 20.4b 20.2b 23.9b 14.8b 18.4b
  Cardiac arrhythmias 14.9b 17.4b 18.0b 20.4b 13.2b 15.8b
  Valvular heart disease 6.5b 7.6b 8.3b 9.4b 5.6b 6.5b
  Pulmonary circulation disorders 2.5b 4.1b 3.1b 4.8b 2.2b 3.7b
  Peripheral vascular disorders 5.8 6.1 5.9b 6.7b 5.7 5.8
  Hypertension 22.1b 28.8b 21.1b 27.3b 22.6b 29.7b
  Paralysis 2.3b 2.0b 2.8 2.5 1.9 1.7
  Other neurologic disorders 6.9b 5.3b 8.8b 6.8b 5.9b 4.4b
  Chronic pulmonary disorders 25.3b 27.7b 27.8b 30.2b 23.9b 26.3b
  Diabetes (uncomplicated) 17.3b 20.3b 16.9b 21.1b 17.6b 19.8b
  Diabetes (complicated) 4.1b 5.1b 5.0b 5.9b 3.6b 4.7b
  Hypothyroidism 5.6b 7.8b 5.0b 7.6b 5.9b 7.9b
  Renal failure 5.7b 13.3b 6.1b 16.1b 5.4b 11.8b
  Liver disease 1.7b 3.4b 1.8b 3.2b 1.6b 3.5b
  Peptic ulcer disease (nonbleeding) 1.6b 1.1b 1.5b 1.0b 1.6b 1.1b
  Acquired immunodeficiency syndrome 0.3b 1.0b 0.3b 1.0b 0.3b 1.0b
  Lymphoma 0.6b 1.2b 0.7b 1.4b 0.6b 1.0b
  Metastatic cancer 1.7b 2.5b 2.1b 2.9b 1.5b 2.3b
  Solid tumor without metastasis 6.6 7.0 5.7b 7.3b 7.1 6.9
  Collagen vascular disease 1.9 2.1 2.1 2.1 1.8b 2.2b
  Coagulopathy 4.2b 9.2b 4.7b 11.1b 4.0b 8.2b
  Obesity 6.1b 8.5b 4.7b 8.7b 6.8b 8.4b
  Weight loss 4.1b 6.5b 3.8b 8.0b 4.2b 5.7b
  Fluid and electrolyte disorders 34.6b 43.4b 44.3b 52.4b 29.3b 38.4b
  Alcohol abuse 6.3b 7.7b 6.9b 8.5b 5.9b 7.3b
  Drug abuse 4.1b 7.0b 5.2b 7.5b 3.5b 6.8b
  Psychoses 4.1b 5.9b 5.3b 7.6b 3.5b 5.0b
  Depression 5.5b 7.7b 4.7b 7.7b 5.9b 7.7b
 Admission via emergency department (%) 78.8b 90.1b 87.3b 92.4b 74.1 88.8
 Principal discharge diagnosis (%)
  Pulmonary 30.0b 28.6b 34.1b 30.1b 27.7b 27.8b
  Circulatory 24.9b 18.5b 20.0b 15.6b 27.6b 20.1b
  Neurological 13.9b 12.3b 15.2b 11.8b 13.2b 12.6b
  Infectious disease 5.9b 11.9b 6.8b 15.6b 5.3b 9.9b
  Gastroenterological 6.4b 7.2b 4.2b 4.7b 7.6b 8.5b
  Other 18.9b 21.5b 19.7b 22.1b 18.5b 21.1b
 In-hospital sepsis diagnosis (%) 10.6b 20.8b 14.5b 27.7b 8.4b 16.9b
 ICU LOS (d), mean (sd) 4.2 (3.9)b 4.3 (3.8)b 4.0 (3.5)b 4.2 (3.6)b 4.4 (4.0) 4.4 (3.9)
 Non-ICU LOS (d), mean (sd) 3.8 (6.2)b 3.6 (5.6)b 5.4 (6.7)b 4.8 (6.1)b 2.9 (5.7) 2.9 (5.3)
Hospital characteristics
 Annual ICU volume, mean (sd) 200 (207)b 221 (224)b 116 (41)b 125 (62)b 360 (293)b 407 (300)b
 Teaching (%) 41.6b 39.9b 55.8b 45.7b 33.7b 36.7b
 Diagnoses per patient,a mean (sd) 7.3 (1.4)b 10.2 (2.4)b 8.0 (1.5)b 11.4 (1.9)b 7.0 (1.3)b 9.6 (2.4)b
 Procedures per patient,a mean (sd) 1.9 (1.4)b 2.4 (1.3)b 2.6 (1.0)b 2.9 (1.0)b 1.6 (1.5)b 2.1 (1.3)b

LOS = length of stay.

a

Mean number of International Classification of Diseases, 9th Revision, Clinical Modification diagnosis or procedure codes documented per patient by hospital for the year each patient was discharged.

b

p < 0.05 comparing characteristics between study periods.

On average, the annual number of eligible ICU patients discharged by the hospitals remained relatively stable between the study periods (200 and 221 patients per hospital annually, respectively). The mean (sd) number of diagnoses coded by hospitals increased from 7.3 (1.4) diagnoses per patient in the pre-TRICC period to 10.2 (2.4) diagnoses per patient in the post-TRICC period. Similarly, the number of procedures coded increased from the pre-TRICC period to the post-TRICC period (1.9 [1.4] to 2.4 [1.3] procedures per patient, respectively).

Patient Characteristics by Transfusion Status

Patient and hospital characteristics associated with the odds of RBC transfusion are described in Table 2. In the multivariable logistic regression model, a higher adjusted odds of RBC transfusion was associated with patient age (OR, 1.11 [95% CI, 1.09–1.13] per 10 yr) and female sex (OR, 1.21 [95% CI, 1.15–1.29]), whereas Caucasians were less likely to receive a RBC transfusion (OR, 0.82 [95% CI, 0.76–0.87]). The adjusted odds of RBC transfusion was increased in patients with metastatic cancer (OR, 2.73 [95% CI, 2.40–3.11]), lymphoma (OR, 2.55 [95% CI, 2.11–3.09]), coagulopathy (OR, 1.95 [95% CI, 1.81–2.10]), liver disease (OR, 1.81 [95% CI, 1.60–2.05]), and renal failure (OR, 1.60 [95% CI, 1.49–1.73]).

TABLE 2.

Cohort Characteristics by RBC Transfusion Status and Odds Ratio for RBC Transfusion

Variable Transfused (n = 7,532) Not Transfused (n = 65,853) Unadjusted OR (95% CI) Adjusted OR (95% CI)a
Patient characteristics
 Age (yr) 64.1 (16.5) 62.3 (17.9) 1.06 (1.04–1.07)b 1.11 (1.09–1.13)b
 Female 51.7 50.2 1.06 (1.01–1.11) 1.21 (1.15–1.29)
 Caucasian 60.9 71.3 0.63 (0.60–0.66) 0.82 (0.76–0.87)
 Comorbid diagnoses
  Congestive heart failure 29.1 18.2 1.84 (1.75–1.95) 1.14 (1.06–1.22)
  Cardiac arrhythmias 24.1 15.8 1.69 (1.60–1.79) 1.03 (0.95–1.10)
  Valvular heart disease 11.3 6.8 1.75 (1.62–1.89) 1.19 (1.08–1.31)
  Pulmonary circulation disorders 4.1 3.5 1.17 (1.04–1.32) 0.81 (0.71–0.93)
  Peripheral vascular disorders 6.4 5.9 1.08 (0.98–1.19) 1.23 (1.11–1.38)
  Hypertension 19.5 27.6 0.63 (0.60–0.67) 0.84 (0.78–0.90)
  Paralysis 2.7 2.0 1.35 (1.17–1.57) 1.09 (0.92–1.29)
  Other neurologic disorders 7.7 5.6 1.41 (1.29–1.54) 1.04 (0.94–1.15)
  Chronic pulmonary disorders 25.2 27.2 0.90 (0.85–0.95) 0.89 (0.83–0.95)
  Diabetes (uncomplicated) 17.3 19.6 0.86 (0.80–0.91) 0.99 (0.92–1.06)
  Diabetes (complicated) 7.1 4.6 1.61 (1.46–1.77) 1.39 (1.24–1.56)
  Hypothyroidism 5.8 7.3 0.78 (0.71–0.86) 0.90 (0.80–1.01)
  Renal failure 24.0 9.6 2.99 (2.81–3.17) 1.60 (1.49–1.73)
  Liver disease 7.4 2.3 3.32 (3.00–3.67) 1.81 (1.60–2.05)
  Peptic ulcer disease (nonbleeding) 1.2 1.2 0.97 (0.77–1.21) 1.23 (0.96–1.57)
  Acquired immunodeficiency syndrome 1.8 0.7 2.68 (2.21–3.26) 1.46 (1.16–1.84)
  Lymphoma 2.8 0.8 3.61 (3.07–4.24) 2.55 (2.11–3.09)
  Metastatic cancer 5.9 1.8 3.39 (3.03–3.79) 2.73 (2.40–3.11)
  Solid tumor without metastasis 7.8 6.8 1.17 (1.07–1.27) 1.30 (1.17–1.44)
  Collagen vascular disease 2.9 2.0 1.46 (1.26–1.69) 1.30 (1.10–1.54)
  Coagulopathy 22.8 6.0 4.62 (4.34–4.92) 1.95 (1.81–2.10)
  Obesity 3.9 8.2 0.45 (0.40–0.51) 0.62 (0.55–0.71)
  Weight loss 16.0 4.6 3.91 (3.64–4.20) 1.59 (1.45–1.73)
  Fluid and electrolyte disorders 62.7 38.3 2.70 (2.57–2.84) 1.34 (1.26–1.42)
  Alcohol abuse 8.7 7.1 1.24 (1.14–1.35) 1.12 (1.01–1.24)
  Drug abuse 5.8 6.2 0.93 (0.84–1.03) 0.85 (0.75–0.97)
  Psychoses 3.4 5.6 0.59 (0.52–0.67) 0.64 (0.55–0.74)
  Depression 4.5 7.3 0.60 (0.53–0.67) 0.81 (0.72–0.91)
 Admission via emergency department 83.2 87.1 0.73 (0.69–0.78) 0.77 (0.71–0.84)
 Principal discharge diagnosis
  Pulmonary 25.2 29.5 Reference Reference
  Circulatory 9.3 21.7 0.50 (0.46–0.55) 0.83 (0.74–0.92)
  Neurological 4.3 13.8 0.36 (0.32–0.41) 0.48 (0.42–0.55)
  Infectious disease 26.0 8.3 3.65 (3.41–3.92) 1.57 (1.43–1.72)
  Gastroenterological 13.2 6.2 2.48 (2.28–2.69) 2.99 (2.70–3.31)
  Other 22.0 20.5 1.25 (1.17–1.34) 1.75 (1.61–1.90)
 In-hospital sepsis diagnosis 48.4 14.2 5.65 (5.38–5.95) 1.93 (1.80–2.07)
 ICU length of stay (d) 6.9 (6.3) 4.0 (3.3) 1.14 (1.13–1.14) 1.09 (1.09–1.10)
 Non-ICU length of stay (d) 7.4 (9.1) 3.2 (5.1) 1.08 (1.08–1.09) 1.05 (1.04–1.05)
Hospital characteristics
 Teaching hospital 58.2 38.4 2.24 (2.13–2.38) 1.21 (0.86–1.71)
 Diagnoses per patientc 10.8 (2.5) 9.2 (2.5) 1.39 (1.37–1.41) 1.05 (1.01–1.09)
 Procedures per patientc 3.0 (1.2) 2.2 (1.3) 1.54 (1.51–1.57) 1.09 (1.03–1.15)
Calendar time by mean ICU volumed
 1994–1998 (lower ICU volume hospitals) 8.1 10.9 1.02 (0.99–1.05)e 1.03 (0.99–1.07)e
 1994–1998 (higher ICU volume hospitals) 10.7 20.3 0.99 (0.96–1.02)e 1.07 (1.01–1.13)e
 1999–2007 (lower ICU volume hospitals) 35.9 23.9 1.15 (1.13–1.17)e 1.10 (1.08–1.13)e
 1999–2007 (higher ICU volume hospitals) 45.3 45.0 1.14 (1.12–1.15)e 0.96 (0.93–0.98)e

OR = odds ratio, ICD-9–CM = International Classification of Diseases, 9th Revision, Clinical Modification.

a

Adjusted odds of transfusion estimated using a logistic regression model with a hospital-level random effect and robust variance estimation to account for within-hospital clustering and includes all listed patient and hospital variables with calendar time modeled as a continuous variable.

b

Per additional 10 yr.

c

Mean number of ICD-9–CM diagnosis or procedure codes documented per patient by hospital for the year each patient was discharged.

d

Higher ICU volume hospitals (mean > 200 annual ICU discharges); lower ICU volume hospitals (mean ≤ 200 annual ICU discharges).

e

Per additional year.

Hospital admission via the emergency department was associated with a decreased adjusted odds of RBC transfusion (OR, 0.77 [95% CI, 0.71–0.84]), whereas the in-hospital diagnosis of sepsis increased the adjusted odds (OR, 1.93 [95% CI, 1.80–2.07]). Each additional ICU day and non-ICU hospital day also independently increased the odds of RBC transfusion (OR, 1.09 [95% CI, 1.09–1.10] and OR, 1.05 [95% CI, 1.04–1.05] per day, respectively). Hospital teaching status was not independently associated with the odds of RBC transfusion (OR, 1.21 [95% CI, 0.86–1.71]).

In the pre-TRICC period, the odds of RBC transfusion did not increase significantly in lower ICU volume hospitals (OR, 1.03 [95% CI, 0.99–1.07] per yr). Although not statistically different from the lower ICU volume hospitals (p = 0.401), the odds of RBC transfusion did increase in higher ICU volume hospitals (OR, 1.07 [95% CI, 1.01–1.13]).

In the post-TRICC period, the adjusted odds of RBC transfusion increased annually (OR, 1.10 [95% CI, 1.08–1.13]) in lower ICU volume hospitals, representing a 7% marginal increase compared with the baseline trend (p = 0.015). However, in higher ICU volume hospitals, the odds of RBC transfusion decreased annually during the same period (OR, 0.96 [95% CI, 0.93–0.98]), reflecting an 11% marginal decrease (p = 0.002). The 13% relative difference between the post-TRICC transfusion trends for lower and higher ICU volume hospitals was statistically significant (p < 0.001). No multicollinearity was identified in the multivariable model. Sensitivity analyses evaluating for interactions between calendar year and hospital teaching status and between calendar year and sepsis diagnosis did not alter our findings.

Odds of Receiving a Transfusion Over Study Period

The unadjusted odds of patients in lower ICU volume hospitals receiving a transfusion increased from 7.9% in the pre-TRICC period to 14.7% in the post-TRICC period (Fig. 2). In higher ICU volume hospitals, patients receiving a RBC transfusion also increased between the periods from 5.7% to 10.3%. For most of the study years, the unadjusted odds of patients receiving a RBC transfusion was higher in the lower ICU volume hospitals (Fig. 2A). After accounting for patient and hospital characteristics, the adjusted odds of receiving a transfusion in lower ICU volume hospitals increased from 3% in 1994 to 7% in 2007 (Fig. 2B). In contrast, the adjusted odds of receiving a transfusion in higher ICU volume hospitals, although initially increasing, have decreased back to 3% during the later study years.

Figure 2.

Figure 2.

Unadjusted (A) and adjusted (B) odds of RBC transfusion for ICU patients in Maryland over 14 yr by mean annual ICU patient volume category (1994–2007). Adjusted odds estimated using a logistic regression model with a hospital-level random effect and robust variance estimation to account for within-hospital clustering and includes adjustment for average annual ICU volume category, age, sex, race, individual Elixhauser comorbidity measures, hospital admission via emergency department, principal discharge diagnosis, sepsis, ICU length of stay, non-ICU hospital length of stay, teaching status, and average number of diagnoses and procedures per patient coded by each hospital year. Squares = lower volume; circle = higher volume; dotted line = lower volume linear model; continuous line = higher volume linear model.

DISCUSSION

Our population-based cohort study of nonsurgical adult ICU patients yielded two particularly notable findings. First, the odds of receiving a RBC transfusion increased over a 14-year period in this statewide population. Second, after accounting for patient and hospital factors, the annual trend in the odds of RBC transfusion differed between higher ICU volume hospitals (i.e., > 200 admissions annually) and lower ICU volume hospitals (i.e., ≤ 200 discharges annually) following publication of the TRICC trial in 1999 despite similar RBC transfusion trends during the baseline period. Specifically, during the 9 years after publication of this landmark article supporting a more restrictive transfusion strategy, the odds of RBC transfusion decreased by 11% in higher ICU volume hospitals but increased by 7% in lower ICU volume hospitals. This suggests that clinical research evidence supporting restrictive RBC transfusion thresholds may not be uniformly translated into practice in different hospital settings and supports other evidence linking the volume of procedures to patient outcomes.

This is the first study to evaluate RBC transfusion practices in a large population of diverse hospitals over time. Prior multicenter studies assessed the frequency of RBC transfusions in ICU patients have frequently examined practices in samples of ICUs in a cross-sectional fashion over relatively brief study periods (2, 3, 3134). For example, the Anemia and Blood Transfusion in Critical Care (ABC) study found that 37% of 3,534 European ICU patients enrolled in November 1999 were transfused (2). The subsequent Sepsis Occurrence in Acutely Ill Patients (SOAP) study, reported that 33% of 3,147 European ICU patients were transfused in May 2002 (31).

In the United States, the CRIT study enrolled almost 5,000 patients and observed that 44% were transfused at least one RBC unit (3). Within their retrospective analysis of discharge data, Dasta et al (34) reported that 23% of 46,672 ICU patients admitted in 2004 received any RBC transfusion. The heterogeneity of these cross-sectional results (23–44%) from these European and American studies makes interpretation of trends in RBC transfusion practice over time challenging (35).

Our observation of increasing frequency RBC transfusions in a longitudinal analysis of a large population is consistent with other studies examining changes in RBC transfusion practices in other patient populations (15, 24, 36). For example, an analysis by Pham et al (15) identified increasing RBC transfusions in a population-based cohort of surgical patients during a similar time period. Additionally, our finding of decreasing adjusted odds of RBC transfusion after TRICC publication in higher ICU volume hospitals is consistent with a recent analysis of clinical data by Netzer et al (10), who observed an improvement in RBC transfusion practices in a single large academic medical ICU between 1997 and 2007. By examining transfusions in this statewide population-based study, our study suggest that such improvements in higher ICU volume hospitals may not be generalizable to lower ICU volume hospitals.

Our finding that the adjusted trends in RBC transfusion decreased in higher ICU volume hospitals after publication of the TRICC trial but increased in lower volume hospitals suggests that evidence-based practices are not uniformly translated into different hospital settings. This is congruent with other studies that have identified relationships between hospital or provider volume and adherence to evidence-based practices and patient outcomes in the ICU and other settings (3744). Furthermore, this observation also suggests that while the translation of evidence into practice tends to be slow overall (45, 46), higher volume hospitals may adopt new evidence-based practices faster than lower volume hospitals.

Although our study was not designed to evaluate the specific reasons for the different trends based on hospital volume, literature examining barriers to evidence-based practices can provide a framework for starting to understand these differences in performance (4749). These frameworks have been applied to understanding barriers to evidence-based practice in the ICU including the use of lung protective ventilation for patients with acute respiratory distress syndrome (5052), the prevention of ventilator-associated pneumonia (5355), and management of sepsis (56, 57). Efforts to close the gap between RBC transfusion evidence and clinical practice in different ICU volume hospitals will likely benefit from a similarly detailed assessment of provider and organizational characteristics, which may influence this gap including provider factors (e.g., awareness and agreement with evidence) and organizational factors (e.g., policies, presence of decision support aids, provider staffing, and interprofessional communication) (51, 5759).

Although a population-based administrative dataset provided an opportunity to assess large-scale changes in RBC transfusion practice over a 14-year period, we recognize several limitations. First, our study did not evaluate other potentially important measures of transfusion practice, such as the indication for transfusion (e.g., pretransfusion hemoglobin) or number of RBC units transfused, which are not available in this administrative dataset. In addition, our study was not able to identify the subset of medical patients who may have required a surgical procedure during their hospitalization. Nevertheless, the proportion of patients receiving any transfusion is a meaningful and commonly reported measure (2, 3) which allows for the evaluation of changes in transfusion practices over time in a large population.

Second, we could not evaluate several potentially important provider and organizational characteristics that may influence transfusion practices. For example, the dataset did not include data regarding hospital type (e.g., tertiary, community, and rural), hospital bed size, or ICU staffing model, which may influence transfusion practices. However, we did assess ICU volume and teaching status, which can be a reasonable surrogate as higher ICU volume and teaching centers are more likely at larger tertiary care versus smaller rural hospitals in the state of Maryland.

Third, we may have misclassified the RBC transfusion status of patients due to variations in medical record documentation, medical coding errors, or changes in coding practices over time. However, RBC transfusion status in the HSCRC database has been previously validated (14, 21). Furthermore, we minimized the risk of changes in coding practices between hospitals and over the study period by adjusting for the average number of diagnoses and procedures coded by hospitals each year in our regression model.

Fourth, our study may not have completely accounted for changes in patient severity of illness over the study period resulting in residual confounding. Similar to other studies, we minimized this risk by adjusting for comorbid illness (i.e., Elixhauser comorbidity measures) (19, 58) and other patient characteristics which may also influence transfusion practices (e.g., emergent admissions and major admitting diagnosis) (1315, 59). Finally, our study was not able to account for other factors in addition to publication of the TRICC trial that may have influenced transfusion behaviors, including changes in testing methods for infectious diseases (e.g., HIV and hepatitis C virus), availability and cost of blood transfusions, and sepsis treatment recommendations (60).

Despite these potential limitations, this study has several strengths. First, this study represents the largest ICU patient sample and covers a longer study period than previously published studies examining RBC transfusion practices. Second, in describing the association between ICU volume and transfusion practices over time, this study contributes to the growing literature regarding the relationship between patient volume and patient outcomes (3844). Third, despite the potential limitations of administrative data (e.g., coding errors), these data may be less subject to other threats to internal validity (e.g., recall bias, selection, and nonresponse bias) and external validity (e.g., Hawthorne effect) than other study methodologies such as surveys and observational cohorts studies.

Based on our findings, future studies are necessary to identify the provider and organizational factors that may influence the translation of RBC transfusion evidence into practice. Given the observed difference in RBC transfusion trends between higher and lower ICU volume hospitals, studies evaluating provider and organizational factors in these settings may be of particular benefit. Such research could accelerate efforts to identify effective strategies to enhance the translation of RBC clinical research evidence into practice across the healthcare system.

CONCLUSIONS

In summary, we found that RBC transfusions are increasingly common over a 14-year period in this large population-based cohort of critically ill, nonsurgical patients. After accounting for patient and hospital characteristics, the trend of RBC transfusion decreased in higher ICU volume hospitals after TRICC trial publication but continued to increase in lower ICU volume hospitals. This finding suggests that clinical research evidence supporting restrictive RBC transfusion thresholds may not be uniformly translated into practice in different hospital settings. Further research is necessary to identify the provider and organizational characteristics that influence the translation of RBC transfusion clinical research evidence into practice.

Acknowledgments

Dr. Murphy was supported by an institutional training grant from the NIH (T32 HL007534). Dr. Pronovost was supported by a K24 award from the NIH. Dr. Berenholtz was supported by the National Institutes of Health, Agency for Healthcare Research and Quality, Health Research and Education Trust, The Commonwealth Fund, Centers for Disease Control and Prevention, and Michigan Health & Hospital Association Keystone Center for Patient Safety & Quality for unrelated research; has equity ownership in Docusys; and receives honoraria and travel expenses from various hospitals and hospital associations for consulting. Dr. Netzer received funding from the National Institutes of Health. Dr. Ness consulted for TerumoBCT and Fenwal Labs. The remaining authors have disclosed that they do not have any potential conflicts of interest.

REFERENCES

  • 1.von Ahsen N, Müller C, Serke S, et al. : Important role of nondiagnostic blood loss and blunted erythropoietic response in the anemia of medical intensive care patients. Crit Care Med 1999; 27:2630–2639 [DOI] [PubMed] [Google Scholar]
  • 2.Vincent JL, Baron JF, Reinhart K, et al. ; ABC (Anemia and Blood Transfusion in Critical Care) Investigators: Anemia and blood transfusion in critically ill patients. JAMA 2002; 288:1499–1507 [DOI] [PubMed] [Google Scholar]
  • 3.Corwin HL, Gettinger A, Pearl RG, et al. : The CRIT study: Anemia and blood transfusion in the critically ill–current clinical practice in the United States. Crit Care Med 2004; 32:39–52 [DOI] [PubMed] [Google Scholar]
  • 4.Allen JB, Allen FB: The minimum acceptable level of hemoglobin. Int Anesthesiol Clin 1982; 20:1–22 [PubMed] [Google Scholar]
  • 5.Welch HG, Meehan KR, Goodnough LT: Prudent strategies for elective red blood cell transfusion. Ann Intern Med 1992; 116:393–402 [DOI] [PubMed] [Google Scholar]
  • 6.Marik PE, Corwin HL: Efficacy of red blood cell transfusion in the critically ill: A systematic review of the literature. Crit Care Med 2008; 36:2667–2674 [DOI] [PubMed] [Google Scholar]
  • 7.Netzer G, Shah CV, Iwashyna TJ, et al. : Association of RBC transfusion with mortality in patients with acute lung injury. Chest 2007; 132:1116–1123 [DOI] [PubMed] [Google Scholar]
  • 8.Gong MN, Thompson BT, Williams P, et al. : Clinical predictors of and mortality in acute respiratory distress syndrome: Potential role of red cell transfusion. Crit Care Med 2005; 33:1191–1198 [DOI] [PubMed] [Google Scholar]
  • 9.Hébert PC, Wells G, Blajchman MA, et al. : A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion Requirements in Critical Care Investigators, Canadian Critical Care Trials Group. N Engl J Med 1999; 340:409–417 [DOI] [PubMed] [Google Scholar]
  • 10.Netzer G, Liu X, Harris AD, et al. : Transfusion practice in the intensive care unit: A 10-year analysis. Transfusion 2010; 50:2125–2134 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Hillyer CD, Blumberg N, Glynn SA, et al. ; NHLBI Working Group in Transfusion Recipient Epidemiology and Outcomes Research: Transfusion recipient epidemiology and outcomes research: Possibilities for the future. Transfusion 2008; 48:1530–1537 [DOI] [PubMed] [Google Scholar]
  • 12.Reynolds HN, McCunn M, Borg U, et al. : Acute respiratory distress syndrome: Estimated incidence and mortality rate in a 5 million-person population base. Crit Care 1998; 2:29–34 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Pronovost PJ, Jenckes MW, Dorman T, et al. : Organizational characteristics of intensive care units related to outcomes of abdominal aortic surgery. JAMA 1999; 281:1310–1317 [DOI] [PubMed] [Google Scholar]
  • 14.Berenholtz SM, Pronovost PJ, Mullany D, et al. : Predictors of transfusion for spinal surgery in Maryland, 1997 to 2000. Transfusion 2002; 42:183–189 [DOI] [PubMed] [Google Scholar]
  • 15.Pham JC, Catlett CL, Berenholtz SM, et al. : Change in use of allogeneic red blood cell transfusions among surgical patients. J Am Coll Surg 2008; 207:352–359 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.United States Health Care Financing Administration: The International Classification of Diseases, 9th Revision, Clinical Modification: ICD-9–CM. Fourth Edition. Washington, DC, U.S. Department of Health and Human Services, Public Health Service, 1991 [Google Scholar]
  • 17.Kiyota Y, Schneeweiss S, Glynn RJ, et al. : Accuracy of Medicare claims-based diagnosis of acute myocardial infarction: Estimating positive predictive value on the basis of review of hospital records. Am Heart J 2004; 148:99–104 [DOI] [PubMed] [Google Scholar]
  • 18.Arnason T, Wells PS, van Walraven C, et al. : Accuracy of coding for possible warfarin complications in hospital discharge abstracts. Thromb Res 2006; 118:253–262 [DOI] [PubMed] [Google Scholar]
  • 19.Elixhauser A, Steiner C, Harris DR, et al. : Comorbidity measures for use with administrative data. Med Care 1998; 36:8–27 [DOI] [PubMed] [Google Scholar]
  • 20.All Patient Refined Diagnosis Related Groups Methodology Overview, Version 20.0. Wallingford, CT, 3M Health Information Systems, 2003 [Google Scholar]
  • 21.Segal JB, Ness PM, Powe NR: Validating billing data for RBC transfusions: A brief report. Transfusion 2001; 41:530–533 [DOI] [PubMed] [Google Scholar]
  • 22.Martin GS, Mannino DM, Eaton S, et al. : The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med 2003; 348:1546–1554 [DOI] [PubMed] [Google Scholar]
  • 23.Dombrovskiy VY, Martin AA, Sunderram J, et al. : Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: A trend analysis from 1993 to 2003. Crit Care Med 2007; 35:1244–1250 [DOI] [PubMed] [Google Scholar]
  • 24.Nilsson KR, Berenholtz SM, Dorman T, et al. : Preoperative predictors of blood transfusion in colorectal cancer surgery. J Gastrointest Surg 2002; 6:753–762 [DOI] [PubMed] [Google Scholar]
  • 25.Goodnough LT, Johnston MF, Toy PT: The variability of transfusion practice in coronary artery bypass surgery. Transfusion Medicine Academic Award Group. JAMA 1991; 265:86–90 [PubMed] [Google Scholar]
  • 26.Kahn JM, Ten Have TR, Iwashyna TJ: The relationship between hospital volume and mortality in mechanical ventilation: An instrumental variable analysis. Health Serv Res 2009; 44:862–879 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.American Medical Association: FRIEDA Online. Available at: http://www.ama-assn.org/ama/pub/education-careers/graduate-medical-education/freida-online.shtml. Accessed April 8, 2010
  • 28.Diggle P, Heagerty P, Liang K-Y, et al. : Analysis of Longitudinal Data. Second Edition. Oxford, NY, Oxford University Press, 2002 [Google Scholar]
  • 29.Stata Corp: Stata Reference Manual: Release 8. College Station, TX, Stata Corp., 2003 [Google Scholar]
  • 30.Williams RL: A note on robust variance estimation for cluster-correlated data. Biometrics 2000; 56:645–646 [DOI] [PubMed] [Google Scholar]
  • 31.Vincent JL, Sakr Y, Sprung CL, et al. ; Sepsis Occurrence in Acutely Ill Patients Investigators: Sepsis in European intensive care units: Results of the SOAP study. Crit Care Med 2006; 34:344–353 [DOI] [PubMed] [Google Scholar]
  • 32.French CJ, Bellomo R, Finfer SR, et al. : Appropriateness of red blood cell transfusion in Australasian intensive care practice. Med J Aust 2002; 177:548–551 [DOI] [PubMed] [Google Scholar]
  • 33.Murphy DJ, Howard D, Muriithi A, et al. : Red blood cell transfusion practices in acute lung injury: What do patient factors contribute? Crit Care Med 2009; 37:1935–1940 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Dasta J, Mody SH, McLaughlin T, et al. : Current management of anemia in critically ill patients: Analysis of a database of 139 hospitals. Am J Ther 2008; 15:423–430 [DOI] [PubMed] [Google Scholar]
  • 35.Walsh TS, Saleh EE: Anaemia during critical illness. Br J Anaesth 2006; 97:278–291 [DOI] [PubMed] [Google Scholar]
  • 36.Maddux FW, Dickinson TA, Rilla D, et al. : Institutional variability of intraoperative red blood cell utilization in coronary artery bypass graft surgery. Am J Med Qual 2009; 24:403–411 [DOI] [PubMed] [Google Scholar]
  • 37.Birkmeyer JD, Siewers AE, Finlayson EV, et al. : Hospital volume and surgical mortality in the United States. N Engl J Med 2002; 346:1128–1137 [DOI] [PubMed] [Google Scholar]
  • 38.Kahn JM, Goss CH, Heagerty PJ, et al. : Hospital volume and the outcomes of mechanical ventilation. N Engl J Med 2006; 355:41–50 [DOI] [PubMed] [Google Scholar]
  • 39.Thiemann DR, Coresh J, Oetgen WJ, et al. : The association between hospital volume and survival after acute myocardial infarction in elderly patients. N Engl J Med 1999; 340:1640–1648 [DOI] [PubMed] [Google Scholar]
  • 40.Nathens AB, Jurkovich GJ, Maier RV, et al. : Relationship between trauma center volume and outcomes. JAMA 2001; 285:1164–1171 [DOI] [PubMed] [Google Scholar]
  • 41.Ross JS, Normand SL, Wang Y, et al. : Hospital volume and 30-day mortality for three common medical conditions. N Engl J Med 2010; 362:1110–1118 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Gandjour A, Bannenberg A, Lauterbach KW: Threshold volumes associated with higher survival in health care: A systematic review. Med Care 2003; 41:1129–1141 [DOI] [PubMed] [Google Scholar]
  • 43.Williams SC, Koss RG, Morton DJ, et al. : Case volume and hospital compliance with evidence-based processes of care. Int J Qual Health Care 2008; 20:79–87 [DOI] [PubMed] [Google Scholar]
  • 44.O’Brien SM, Delong ER, Peterson ED: Impact of case volume on hospital performance assessment. Arch Intern Med 2008; 168:1277–1284 [DOI] [PubMed] [Google Scholar]
  • 45.Balas EA, Boren SA: Managing clinical knowledge for health care improvement. Yearb Med Inform 2000; 65–70 [PubMed] [Google Scholar]
  • 46.Lenfant C: Shattuck lecture–clinical research to clinical practice–lost in translation? N Engl J Med 2003; 349:868–874 [DOI] [PubMed] [Google Scholar]
  • 47.Cabana MD, Rand CS, Powe NR, et al. : Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282:1458–1465 [DOI] [PubMed] [Google Scholar]
  • 48.Gurses AP, Murphy DJ, Martinez EA, et al. : A practical tool to identify and eliminate barriers to compliance with evidence-based guidelines. Jt Comm J Qual Patient Saf 2009; 35:526–532 [DOI] [PubMed] [Google Scholar]
  • 49.Berwick DM: Disseminating innovations in health care. JAMA 2003; 289:1969–1975 [DOI] [PubMed] [Google Scholar]
  • 50.Umoh NJ, Fan E, Mendez-Tellez PA, et al. : Patient and intensive care unit organizational factors associated with low tidal volume ventilation in acute lung injury. Crit Care Med 2008; 36:1463–1468 [DOI] [PubMed] [Google Scholar]
  • 51.Cooke CR, Watkins TR, Kahn JM, et al. : The effect of an intensive care unit staffing model on tidal volume in patients with acute lung injury. Crit Care 2008; 12:R134. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Kalhan R, Mikkelsen M, Dedhiya P, et al. : Underuse of lung protective ventilation: Analysis of potential factors to explain physician behavior. Crit Care Med 2006; 34:300–306 [DOI] [PubMed] [Google Scholar]
  • 53.Cook DJ, Meade MO, Hand LE, et al. : Toward understanding evidence uptake: Semirecumbency for pneumonia prevention. Crit Care Med 2002; 30:1472–1477 [DOI] [PubMed] [Google Scholar]
  • 54.Rello J, Lorente C, Bodí M, et al. : Why do physicians not follow evidence-based guidelines for preventing ventilator-associated pneumonia? A survey based on the opinions of an international panel of intensivists. Chest 2002; 122:656–661 [DOI] [PubMed] [Google Scholar]
  • 55.Berenholtz SM, Pham JC, Thompson DA, et al. : Collaborative cohort study of an intervention to reduce ventilator-associated pneumonia in the intensive care unit. Infect Control Hosp Epidemiol 2011; 32:305–314 [DOI] [PubMed] [Google Scholar]
  • 56.Carlbom DJ, Rubenfeld GD: Barriers to implementing protocol-based sepsis resuscitation in the emergency department–results of a national survey. Crit Care Med 2007; 35:2525–2532 [DOI] [PubMed] [Google Scholar]
  • 57.Mikkelsen ME, Gaieski DF, Goyal M, et al. : Factors associated with nonadherence to early goal-directed therapy in the ED. Chest 2010; 138:551–558 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Quan H, Sundararajan V, Halfon P, et al. : Coding algorithms for defining comorbidities in ICD-9–CM and ICD-10 administrative data. Med Care 2005; 43:1130–1139 [DOI] [PubMed] [Google Scholar]
  • 59.Rubenfeld GD, Angus DC, Pinsky MR, et al. : Outcomes research in critical care: Results of the American Thoracic Society Critical Care Assembly Workshop on Outcomes Research. The Members of the Outcomes Research Workshop. Am J Respir Crit Care Med 1999; 160:358–367 [DOI] [PubMed] [Google Scholar]
  • 60.Dellinger RP, Carlet JM, Masur H, et al. ; Surviving Sepsis Campaign Management Guidelines Committee: Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock. Crit Care Med 2004; 32:858–873 [DOI] [PubMed] [Google Scholar]

RESOURCES