Predictive Validity of the Braden Scale for Patients in Intensive Care Units

Abstract

Background

Patients in intensive care units are at higher risk for development of pressure ulcers than other patients. In order to prevent pressure ulcers from developing in intensive care patients, risk for development of pressure ulcers must be assessed accurately.

Objectives

To evaluate the predictive validity of the Braden scale for assessing risk for development of pressure ulcers in intensive care patients by using 4 years of data from electronic health records.

Methods

Data from the electronic health records of patients admitted to intensive care units between January 1, 2007, and December 31, 2010, were extracted from the data warehouse of an academic medical center. Predictive validity was measured by using sensitivity, specificity, positive predictive value, and negative predictive value. The receiver operating characteristic curve was generated, and the area under the curve was reported.

Results

A total of 7790 intensive care patients were included in the analysis. A cutoff score of 16 on the Braden scale had a sensitivity of 0.954, specificity of 0.207, positive predictive value of 0.114, and negative predictive value of 0.977. The area under the curve was 0.672 (95% CI, 0.663–0.683). The optimal cutoff for intensive care patients, determined from the receiver operating characteristic curve, was 13.

Conclusions

The Braden scale shows insufficient predictive validity and poor accuracy in discriminating intensive care patients at risk of pressure ulcers developing. The Braden scale may not sufficiently reflect characteristics of intensive care patients. Further research is needed to determine which possibly predictive factors are specific to intensive care units in order to increase the usefulness of the Braden scale for predicting pressure ulcers in intensive care patients.

Patients admitted to the intensive care unit (ICU) have a higher incidence of pressure ulcers than general hospital patients.¹ In the United States, the prevalence of pressure ulcers in ICUs was from 16.6% to 20.7% in 2009² and the incidence in acute care settings can be as high as 38%.³ Defined as an injury of an area of skin and underlying tissue related to prolonged pressure against the skin, a pressure ulcer may be acquired or may substantially worsen during hospitalization.⁴ As a result, the cost and length of both ICU and hospital stays can increase, and patients’ quality of life can be diminished by pain and infection.³ An estimated 2.5 million patients are treated annually in acute care settings in the United States, and the estimated annual expenditure for treating pressure ulcers is $11 billion; however, pressure ulcers are largely preventable.³ Therefore, accurate risk assessment is critical, particularly in high-risk populations such as ICU patients, in order to encourage effective implementation of targeted preventive measures.

Review of the Literature

The Braden scale⁵ is one of the most widely used risk assessment scales in the United States.^6–11 It measures the risk for development of a pressure ulcer by using 6 subscales, each denoting a factor that has been found to contribute to pressure ulcer formation: mobility, activity, sensory perception, skin moisture, nutritional state, and friction/shear. Each of the subscales is scored from 1 to 4 (1–3 for friction/shear), with 1 representing the highest risk. The total Braden score ranges from 6 to 23. A lower total Braden score means a greater risk of pressure ulcers developing. Eighteen is the cutoff score that is generally accepted in practice across clinical settings in the United States for predicting risk of pressure ulcers; however, a score of 16 has been recommended for ICU patients.¹²

The Braden scale has been tested in various settings, such as acute care settings, nursing homes, and tertiary care hospitals^6,10,12,13; however, only a few validity evaluations were conducted on patients in the ICU, where the challenges to prevention of pressure ulcers are the greatest. In addition, results of validation studies conducted in ICUs were limited or inconclusive because of inadequate sample sizes^14–16 and high measurement errors.¹⁷ Furthermore, only 4 of the subscales (skin moisture, mobility, friction/shear, and sensory perception) were significantly associated with development of pressure ulcers in ICU patients.^18–20 Therefore, it is uncertain to what extent the Braden scale should be the risk assessment instrument of choice in ICUs. Published reports address the need for further evaluation to determine the validity of the Braden scale for identifying patients at high risk for development of pressure ulcers in ICUs.^21–24

Electronic health records (EHRs) have been implemented at the various health care settings. Despite the technical advances in health care, little research has used EHR data to support risk assessment and prevention of pressure ulcers. The Information Warehouse at The Ohio State University Wexner Medical Center (OSUWMC) is an enterprise data warehouse. The Information Warehouse compiles EHR data from the various electronic records systems throughout OSUWMC (eg, administrative system, laboratory system, computerized provider order entry, and ICU documentation system). Four years of Braden score records were available in the Information Warehouse. The purpose of this study was to use the ICU EHR data to evaluate the validity of the Braden scale for predicting pressure ulcers in ICU patients.

Methods

Setting

The setting for this study was 3 adult ICUs in the OSUWMC. OSUWMC serves as a major referral center for patients from Ohio and throughout the Midwest. OSUWMC has 2 medical ICUs and 1 surgical ICU. The medical ICUs each have 39 beds, and the surgical ICU has 44 beds. In the ICUs, Braden scores are recorded for each patient on admission and periodically thereafter, with patients scoring at risk for pressure ulcers reassessed more often than patients with lower risk scores.

Data Extraction

For data extraction from the Information Ware-house, the following eligibility criteria were documented and applied: adult patients (age ≥18 years) admitted to ICUs between January 1, 2007, and December 31, 2010, comprised the sample, with 2 exceptions. Patients whose ICU stay was shorter than 3 days were excluded, because it is reported that pressure ulcers usually develop more than 72 hours after admission.^8,19 Additionally, patients who had a pressure ulcer at the time of admission were excluded, thereby enabling inclusion only of patients who acquired a pressure ulcer during the hospital stay.

Patients who had a pressure ulcer develop were identified by reviewing discharge diagnoses represented by using codes from the International Classification of Diseases, Ninth Revision (ICD-9). For instance, if a patient had an ICD-9 code of 707.05 (Pressure ulcer, Buttock), the patient was considered as a case and included in the pressure ulcer group. On the other hand, if a patient did not have any of the ICD-9 codes representing pressure ulcers, the patient was included in the non–pressure ulcer group (comparison).

Approval for the data extraction was obtained from the institutional review board. Data were de-identified by the staff in the Information Warehouse as the honest broker. Data elements included demographics (age, sex, and race/ethnicity), length of ICU stay, admission and discharge diagnoses, and Braden score at admission. All data included in the present analysis were coded (structured) data; free text data (unstructured) were not included.

Data Analysis

Python scripts²⁵ (Python Software Foundation) and MySQL database were used for data cleaning and preparation for analysis. For instance, if a patient had more than 1 ICU admission record during the study period, only the first admission record was included in the analysis. Patients’ demographics and the incidence of pressure ulcers were summarized by using descriptive statistics. Pressure ulcer and non–pressure ulcer groups were compared by using the χ² test for categorical variables and an independent 2-sample t test for continuous variables. Predictive validity was measured by using sensitivity, specificity, positive predictive value, and negative predictive value. The receiver operating characteristic curve was generated, and the area under the curve (AUC) was calculated.

Sensitivity is the probability that the scale classifies a patient as at risk of pressure ulcer development when given the patient has a pressure ulcer, that is, the true-positive rate. A false-positive result occurs when a patient does not have a pressure ulcer, but the scale classifies the patient as at risk of pressure ulcer development. Specificity is the probability that the scale does not classify a patient as at risk of pressure ulcer development given that the patient does not have a pressure ulcer, that is, the true-negative rate. A false-negative result occurs when a patient has a pressure ulcer but the scale does not classify the patient as at risk of pressure ulcer development; thus, true instances are missed by the scale. Positive predictive value is the proportion of patients classified as at risk who actually have pressure ulcers develop, whereas the negative predictive value is the proportion of patients classified as not at risk who do not have pressure ulcers develop.²⁶ It is ideal that all 4 indicators have high values, but in reality, when the sensitivity goes up, the specificity goes down.

A cutoff point for classification of a patient as at risk is generally determined by considering costs of a false-positive result, the significance of missing a case, and the prevalence of the disease.²⁷ When it is important to identify patients who are likely to have a pressure ulcer develop and should receive intensive preventive interventions, weight should be given to sensitivity and negative predictive value. Such an emphasis will ensure that most patients at high risk for pressure ulcers will not be missed; however, it may result in overuse of preventive resources on patients who may not have a pressure ulcer develop because of the potential for a high false-negative rate.

The receiver operating characteristic curve displays the trade-off between sensitivity and specificity for a range of test scores. True-positive rate (sensitivity) is plotted on the vertical axis against the false-positive rate (specificity) on the horizontal axis over a range of potential cutoff scores.²⁸ The AUC is a measure of how well a scale can discriminate between 2 groups, for example, pressure ulcer group vs non–pressure ulcer group. A higher AUC value means a higher discriminating power. An AUC of 1 indicates a perfect accuracy whereas an AUC of 0.5 means no better than random chance. All statistical analyses were performed with SPSS version 19.0 for Windows (SPSS Inc).

Results

A total of 7790 ICU patients were included in the analysis. Patients’ demographic data are summarized in Table 1. Approximately 57% of the patients were male and 82% were white. The mean age of the patients was 57.7 years and the mean length of ICU stay was 10 days.

Table 1.

Demographic characteristics of patients in the intensive care unit (N = 7790)

Characteristic	No. (%)
Sex
Male	4473 (57.4)
Female	3317 (42.6)
Race/ethnicity
White	6399 (82.1)
Black	1077 (13.8)
Hispanic	57 (0.7)
Asian	49 (0.6)
American Indian	14 (0.2)
Native Hawaiian	2 (0.03)
Other	82 (1.1)
Unknown	110 (1.4)
Age, mean (SD), y	57.7 (15.95)
Days in intensive care unit, mean (SD)	10 (9.97)

Among the study subjects, the top 10 most frequent admission diagnoses were acute respiratory failure (8.1%), septicemia (3.6%), shortness of breath (3.2%), multiple myeloma (2.8%), altered mental status (2.2%), subarachnoid hemorrhage (2.1%), unspecified congestive heart failure (1.6%), pneumonia—organism not specified (1.4%), coronary atherosclerosis of native coronary artery (1.4%), and intracerebral hemorrhage (1.4%).

During the study period, the incidence of pressure ulcers ranged from 8.1% to 10.5% among the ICU study patients (Table 2). Thus, each year, between 154 and 207 patients had pressure ulcers develop.

Table 2.

Pressure ulcer incidence in intensive care units: 2007 to 2010 (N = 7790)

	2007	2008	2009	2010	Total
No. of patients with pressure ulcer at discharge	190	202	207	154	753
Total No. of patients	1924	1979	1979	1908	7790
Incidence, %	9.9	10.2	10.5	8.1	9.7

Patients in the pressure ulcer group were more likely to be male and white, and older than patients in the non–pressure ulcer group. In terms of length of ICU stay, the pressure ulcer group had a significantly longer mean stay than the non–pressure ulcer group, with the pressure ulcer group having a mean stay of 12.8 days compared with a mean stay of 9.7 days for the non–pressure ulcer group. ICU patients who had pressure ulcers develop had lower Braden scores (mean, 12.1; SD, 2.5) than patients who did not have pressure ulcers develop (mean, 14.2; SD, 3.7; Table 3).

Table 3.

Comparison between pressure ulcer and non–pressure ulcer groups (N = 7790)

Characteristics	Pressure ulcer (N= 753)	No pressure ulcer (N= 7037)	Statistic	P
Sex, No. (%) of patients			χ2 = 12.52	<.001
Male	478 (63.5)	3995 (56.8)
Female	275 (36.5)	3042 (43.2)
Race/ethnicity, No. (%) of patients			χ2 = 6.04	.01
White	594 (78.9)	5805 (82.5)
Nonwhite	159 (21.1)	1232 (17.5)
Age	58.7 (15.2)	57.6 (16.0)	t = 1.99	.05
Days in intensive care unit	12.8 (13.8)	9.7 (9.4)	t = 6.09	.001
Total Braden score	12.1 (2.5)	14.2 (3.7)	t = −20.77	.001

From the receiver operating characteristic curve, a cutoff score of 13 showed best balance among sensitivity, specificity, positive predictive value, and negative predictive value. The AUC was 0.672 (95% CI, 0.663–0.683). Table 4 shows the predictive validity of the Braden scale for cutoff scores of 13, 16, and 18.

Table 4.

Predictive validity of the Braden scale (N = 7790)

Cutoff score	Sensitivity (95% CI)	Specificity (95% CI)	Positive predictive value (95% CI)	Negative predictive value (95% CI)
13	0.781 (0.750–0.810)	0.469 (0.458–0.481)	0.136 (0.126–0.147)	0.952 (0.945–0.959)
16	0.954 (0.936–0.967)	0.207 (0.197–0.216)	0.114 (0.106–0.122)	0.977 (0.967–0.984)
18	0.976 (0.962–0.986)	0.146 (0.138–0.154)	0.109 (0.102–0.117)	0.983 (0.973–0.990)

Discussion

Our study used a large set of longitudinal EHR data from the Information Warehouse to evaluate the predictive validity of the Braden scale. For the 4-year study period, incidence rates for pressure ulcers ranged from 8.1% to 10.5%. These ICU pressure ulcer rates were lower than those reported in some other studies that were conducted in a similar time period, for example, 18.3% by Kim et al¹⁵ and 28.4% by Suriadi et al.²⁹ A possible reason that our incidence rates were lower may be that we used discharge ICD-9 codes to identify patients with a pressure ulcer. It is possible that the number of pressure ulcer cases may have been underestimated if the ICD-9 codes were missing in the discharge documentation. In addition, the diagnoses of pressure ulcers could not be validated in this secondary analysis.

During the study period, the incidence rate for pressure ulcers decreased in 2010. This decrease may be explained by an increase in institutional nursing initiatives aimed at decreasing pressure ulcer rates and an increasing awareness of the difference between pressure- and moisture-related wounds. The initiatives included several strategies such as use of specialty beds in high-risk patients, weekly skin rounds in the surgical ICU, education for the ICU staff regarding pressure ulcers and moisture-related wounds, making information available on the institution’s internal website, and adding a chronic wound nurse practitioner who sees patients at the wound clinic with the wound ostomy continence nurses and the clinical nurse specialist.

A cutoff score of 18 is generally accepted across all settings.¹² However, using the same cutoff score may not be appropriate for ICU patients, who are at higher risk for a pressure ulcer developing. In our study, the Braden scale was highly sensitive for a cutoff score of 18; however, it showed low specificity and positive predictive value, which suggested a high false-positive rate. This result indicates that preventive intervention may be provided to patients not at risk of development of pressure ulcers. In our data, a cutoff score of 13 showed the best balance among sensitivity, specificity, positive predictive value, and negative predictive value. This finding is consistent with a prior study by Kim et al.¹⁵ In the study with surgical ICU patients, 14 was identified as the optimal cutoff score. Overprediction may require unnecessary and costly interventions, which may not be a cost-effective use of health care resources. Therefore, a more accurate way of predicting which patients will have pressure ulcers develop in the ICU is needed.

In terms of AUC, it is suggested that an AUC of 0.9 or greater represents an outstanding discrimination between the pressure ulcer and non–pressure ulcer groups; an AUC between 0.8 and 0.9 indicates an excellent discrimination; an AUC less than 0.8 but equal to 0.7 or more represents an acceptable discrimination.³⁰ The AUC of the Braden scale was 0.672, which indicates poor accuracy at predicting pressure ulcers. The AUC shown in a retrospective analysis that used data from an ICU was 0.62.³¹ On the other hand, prospective studies with ICU patients showed a higher AUC: Seongsook et al¹⁶ reported 0.707, Suriadi et al³² reported 0.79, and Kim et al¹⁵ reported 0.881. Nevertheless, the results of the studies were limited by their small sample sizes, and the authors of these earlier studies addressed the need for further evaluation.

The Braden scale appeared to have best validity and reliability among a number of widely used scales for assessing risk of pressure ulcers in a systematic review³³ that analyzed studies conducted in a variety of settings, including ICUs. However, when predictive validity of the Braden scale was compared with that of other scales in ICU patients, the Braden scale showed poorer results than the Cubbin and Jackson scale and the Douglas scale.^7,16 ICU patients differ from general hospital patients in health conditions. For instance, ICU patients are more likely to have several comorbid conditions, be hemodynamically unstable, receiving vasoactive medications, ventilator-dependent, and often sedated. Thus, the Braden scale may not sufficiently reflect the characteristics of ICU patients.

In previous studies, numerous factors appeared to be associated with pressure ulcers in patients admitted to the ICU: history of vascular disease, mechanical ventilation, dopamine treatment,³⁴ severity of illness score (Acute Physiology and Chronic Health Evaluation II), hypotension,³⁵ cardiovascular instability,⁷ age,^1,18 length of ICU stay,¹⁸ and bowel incontinence.¹ The risk factors vary across the studies, and it is still not clear which factors are associated with ICU pressure ulcers. Consequently, it is necessary to carry out replication research with a large data set in order to examine which factors are ICU-specific, so that ICU-specific risk factors can be taken into consideration in addition to the Braden scale when risk of pressure ulcer development is predicted in ICU patients.

Some Braden subscales may be less sensitive than others for predicting a pressure ulcer in ICU patients. Several studies^18–20 demonstrated that only 4 (ie, skin moisture, mobility, friction/shear, and sensory perception) of the Braden subscales are significantly associated with development of pressure ulcers in ICU patients. Therefore, further evaluation is needed to determine whether improved predictive validity in ICU patients could be achieved by revising the instrument to include only these 4 subscales and/or which subscale(s) of the Braden scale are most pertinent to ICU patients.

The increasing adoption and use of EHR in health care settings facilitates researchers’, practitioners’, and administrators’ reuse of EHR data for additional purposes, such as secondary analysis for patient outcomes research. When EHR data are used, data collection requires less time and effort than retrospective chart reviews. This study pilot tested EHR data retrieval and reuse in the area of pressure ulcer prediction, in order to obtain insight for developing strategies to transform EHR data into usable knowledge that is available in real time to assist in the prevention and management of pressure ulcers in ICU patients.

We found challenges in using EHR data. It was not possible to extract several data items pertinent to skin assessment data such as skin color and the stage, location, and depth of the pressure ulcer. One of the issues encountered related to inconsistent data presentation in the database. Multiple pressure ulcer documentation forms/screens have been used over time, and the data have been stored in different data fields and/or separate data clusters. Thus, it was sometimes impossible to locate the right data fields. Another challenge was related to data capture issues of the EHR system. For instance, we were unable to separate medical and surgical ICU patients in our data. The units in this research setting take “overflow” patients, meaning patients may be placed in another ICU if bed capacity in an ICU is reached. In addition, patients are often transferred from one ICU to another; for example, a medical patient may transfer to the surgical ICU postoperatively, or a surgical patient with difficulty being weaned off of mechanical ventilation may be transferred to the medical ICU for ventilator management and weaning. This workflow was not appropriately reflected in the EHR system, and there was no data element that indicated whether a patient was in a surgical or medical ICU. This specific information was not captured.

The results of this study have limited generalizability, as the data were from only 1 institution. Interrater reliability was not examined in this study because it was not the primary interest. In a prior study¹⁷ conducted on 2 independent samples of ICU patients, interrater reliability of the Braden scale was 0.72 (95% CI, 0.52–0.87) for one sample and 0.84 (95% CI, 0.53–.085) for the other, which may show measurement error. In general, at least 0.9 is recommended for reliability estimates.³⁶ It may be possible that there was inconsistency among nurses in measuring pressure ulcer risk by using the Braden scale. All of the ICU patients received preventive care according to the ICU standard prevention protocol. We were unable to examine whether the admission Braden score had made a difference in care provision in the ICUs.

Conclusion

Predictive validity of the Braden scale for intensive care patients was examined by using 4 years of electronic health records. The Braden scale shows insufficient predictive validity and poor accuracy in discriminating ICU patients at risk of pressure ulcers developing. The Braden scale may not sufficiently reflect the characteristics of ICU patients. Further research is needed to determine which possibly predictive factors are specific to the ICU in order to increase the usefulness of the Braden scale for predicting pressure ulcers in ICU patients.