Abstract
Our study sought to estimate the association between race, gender, comorbidity and body mass index (BMI) on the incidence of hospital‐acquired pressure ulcer (PU) from a population‐based retrospective cohort comprising 242 745 unique patient hospital discharges in two fiscal years from July 2009 to June 2010 from 15 general and tertiary care hospitals. Cases were patients with a single inpatient encounter that led to an incident PU. Controls were patients without a PU at any encounter during the two fiscal years with the earliest admission retained for analysis. Logistic regression models quantified the association of potential risk factors for PU incidence. Spline functions captured the non‐linear effects of age and comorbidity. Overall 2·68% of patients experienced an incident PU during their inpatient stay. Unadjusted analyses revealed statistically significant associations by age, gender, race, comorbidity, BMI, admitted for a surgical procedure, source of admission and fiscal year, but differences by gender and race did not persist in adjusted analyses. Interactions between age, comorbidity and BMI contributed significantly to the likelihood of PU incidence. Patients who were older, with multiple comorbidities and admitted for a surgical diagnosis‐related groups (DRG) were at greater risk of experiencing a PU during their stay.
Keywords: Electronic health record, Epidemiology, Hospital‐acquired pressure ulcer
Introduction
Pressure ulcers (PU) are one of the leading hospital‐acquired conditions (HAC) that have a severe impact on health care quality, patient satisfaction and cost of hospital care. In 2006, the estimate of the annual cost of adult hospital stays with a diagnosis of PUs was $11 billion 1. Treating hospital‐acquired PUs extends the hospital length of stay considerably and increases the psychological and physical pain and suffering of the patient 1, 2, 3, 4.
The elderly are particularly vulnerable 5, 6. Many studies of incidence or prevalence occur in nursing homes, skilled nursing facilities, rehabilitation and long‐term care settings 6, 7, 8, 9, 10. In acute care settings, a few studies reveal that incidence is lower in 400–499 bed facilities and higher in 100–199 bed facilities 5. In addition to bed size, increased staffing levels and training contribute to reductions in incidence rates and individual patient risk 11, 12, 13.
Sustained force on areas of the body causes PUs. They occur where minimal tissue covers bony prominences like the sacrum and heels 14. Individual skin type and bed linen‐changing protocols can jointly affect patient risk of PUs, owing to incontinence, friction and shear 15, 16, 17. Barker et al. 18 demonstrated that the consistent use of a well‐integrated risk assessment protocol reduced the risk of PUs in hospital settings, and Frankel, Sperry, and Kaplan documented an increased risk of PU development in patients undergoing surgery associated with age and comorbidities 19. Multiple studies have examined PU development during surgeries of 3 or more hours duration 7, 20.
Age, incontinence and body mass index (BMI) are well‐established risk factors, with older age and BMI of <19 kg/m2 particularly important risk factors for PU development 21, 22, 23. Male sex and poor nutritional status have also been identified as risk factors, but the evidence is limited 16, 21, 24, 25. The incidence is very high among those who have had a prior PU 9, 21, 22, 26, 27. Incidence is lower among the overweight (BMI 25–<30 kg/m2) and obese (BMI ≥30 kg/m2) persons 23, 28. Although higher incidence is seen with increasing age and comorbidity, the dependence is curvilinear and might level off or even decline towards the tails of the age/comorbidity distributions.
The Braden scale assesses a patient's risk of developing a PU by examining criteria related to activity, mobility, skin moisture, nutritional status, friction and shear, and their ability to sense pain and discomfort related to pressure on parts of their body 29, 30. Braden scores could not be used because at the time of this study scores had not been uniformly incorporated with the patient clinical data.
The Charlson Comorbidity Index (CCI) is a summary measure of several comorbid conditions associated with risk of mortality 31, and has been demonstrated to be a useful predictor of major morbidity following paraesophageal hernia repair and organ transplant 32, 33. The use of the CCI to determine mortality outcomes in patients with PUs has shown a moderately elevated risk of mortality when comorbidities include vascular disease, such as congestive heart failure; diabetes; and metastatic cancer 34. The relationship is complicated by the interplay between comorbidities, demographic factors and BMI. Further, hospital operational factors can increase the risk.
In some studies, African American patients present with higher risk; but this finding is also inconsistent 35, 36. Fogerty et al. 6 using the 2003 Nationwide Inpatient Sample (NIS) of nearly 8 million discharges reported that risk factors for PU included age, gender (being male) and African American origin. NIS is the largest all‐payer inpatient health care database built from all discharges from a random selection of community‐based hospitals in the USA. The NIS contains primary and secondary diagnoses and procedures, admission/discharge status, patient demographic characteristics and hospital characteristics. However, NIS cannot link individual patients with their multiple encounters nor does it contain BMI.
The epidemiology of PUs currently lacks evidence of the interplay between hospital organisation characteristics and individual factors in the occurrence of PUs. The studies presented in the literature have used a few very large hospitals or units within a given facility 8, 9, 10, 22, 37. These studies focus on individual characteristics of patients within the facilities and cannot assess the hospital operational differences.
The goals of our study were to examine the joint effects of demographic variables, with the primary focus of age, race, gender, comorbidity and BMI on the incidence of PUs within a large hospital system. We conducted a retrospective population‐based cohort study in adult patients in 15 general and tertiary care hospitals in Maryland, Michigan, Idaho, Indiana and Iowa owned, managed and operated by Trinity Health. These long‐established facilities were in continuous operation during the two fiscal years of our study, July 2009–June 2011, and had a common electronic health record platform for patient clinical data. Our study was approved by the Institutional Review Board at Michigan State University.
Materials and methods
Databases
At the time of this analysis, Trinity Health was the tenth largest health care system and the third largest Catholic health care system in the USA. In 2012, Trinity Health managed 12 and owned 35 hospitals or systems across 10 states. These 47 facilities were organised into 20 ministry organisations (MOs). In 2002, Information Technology Services (ITS), a unit of Trinity Health, began a multiyear investment to deploy Cerner© (Kansas City, MO) as an electronic health record across all MOs, beginning with the smaller MOs. By 2011, 19 MOs used the Cerner© platform for all inpatient and ambulatory health care delivery. Eighteen of these MOs used Cerner© consistently for two prior fiscal years. In parallel with the Cerner© deployment, Trinity Health complemented an existing financial data warehouse with clinical data into a Unified Data Warehouse (UDW). The UDW follows the flow of a patient encounter through admission, care delivery, discharge, billing and reimbursement. The financial and health information management data within UDW are complemented with clinical data from the Cerner© Enterprise Data Warehouse (EDW). Using these data, Trinity Health and Michigan State University (MSU) jointly initiated an effort to leverage the UDW in the analysis of PUs, falls and deep vein thrombosis (DVT) as defects in care. Herein, we present the analysis of PUs.
Patients
A flowchart describing the patient inclusion process is presented in Figure 1. From a comprehensive database of 429 493 hospital encounters with discharge dates between 1 July 2009 and 30 June 2011 from 18 Trinity Health hospitals, 3 hospitals that switched to emergent care or were sold during the study period were excluded. Each hospital was assigned with an anonymising identification number. From these encounters, we removed discharges of (i) patients less than 15 years of age, almost all of which were newborn discharges (MSDRG code from 789 through 795), (ii) less than 24 hours stay which were primarily outpatient encounters for treatments such as dialysis, alcohol and substance abuse rehabilitation, outpatient surgery, and others. Patients other than African Americans, White or other origins were excluded. This categorisation dropped records where race specification was missing, unknown or where the patient did not self‐identify his or her race. Other reported races were of small numbers of Asians, Hispanics, Native Americans, Alaskan Americans, Multiracial groups, and patients who classified themselves as ‘other race’. The resultant file covers 358 318 hospital encounters.
Definition of PU cases and non‐PU controls
ITS used a combination of ICD‐9CM codes (International Classification of Diseases, 9th Edition, Clinical modification) and nursing notes from the patients' medical chart to designate whether or not a hospital episode was associated with an incident PU. Incident PUs are defined as those that were absent on admission but occurred during the acute care encounter. There were 8899 hospital encounters during which the patient acquired a PU.
A PU diagnosis is recorded with the ICD‐9CM codes between 707.x and 707.xy where the digits x and y are used for additional description of the chronicity and anatomical location of the PU. Each discharge record in our database contained inter alia up to 30 diagnosis codes. We found a 96·6% correlation between the ITS supplied incident PU flag and the presence of discharge diagnosis codes of 707.xy. The ITS supplied incident PU flag was generated from nursing notes and other clinical (EDW) data. The discharge diagnosis codes are generated from billing data (UDW). Given that EDW is a more comprehensive source of clinical data and has high correlation with discharge diagnosis, our analyses relied on the ITS supplied flag solely for case definition. Retaining only unique patients resulted in 6505 patients with an incident PU.
Unique patients without an incident PU were the pool of potential controls. These patients were obtained from the analytic file of 358 318 hospital encounters from which patients who ever had a PU were excluded. For patients with multiple encounters, the record with the earliest discharge date as the index encounter was chosen. The analytic file contained 236 280 controls. The resultant analytic file was comprised of approximately 242 785 unique patients (6505 cases and 236 280 controls) divided over the two fiscal periods, 51·4% in FY2010 and 48·6% in FY2011.
Comorbidity Index
The CCI for each patient was calculated from the ICD‐9CM diagnosis codes 31, 38. The CCI is a weighted sum of the presence of the following diseases and medical conditions: myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, dementia, chronic pulmonary disease, connective tissue disease, peptic ulcer disease, liver disease (mild, moderate or severe), diabetes (with or without end organ damage), paraplegia/hemiplegia, renal disease, malignancy (any tumour, metastatic tumour), and AIDS/HIV. The CCI was initially categorised into the following five subgroups: 0, 1, 2, 3, 4 and ≥5, and subsequently, a flexible spline function of the continuous CCI was constructed to model its effect on the incidence of PU.
In general, a spline function 39, 40 of a continuous variable x is a smooth function composed of polynomial pieces connected at interior points, called knots on the domain of x. The spline for CCI had knots at 1 and 5, and three linear segments, <1, between 1 and 5, and ≥5.
Medical versus surgical diagnosis‐related groups
We used the CMS (Centers for Medicare & Medicaid Services) categorisation of diagnosis‐related groups (DRG) as an indicator for receipt of medical or surgical procedure during a hospital encounter 41.
Admission type
Admission is recorded by hospital registration clerks and indicates where a patient was immediately prior to being admitted. Several categories of admission source were condensed into the following types: Home, Emergency, Transfer, or Other. Transfers were largely comprised of patients from other hospitals, skilled nursing facilities, and other health care facilities. An admission from home was designated for any non‐health care facility as the point of origin. ‘Other’ type captured primarily admissions from a clinic or physician's office.
Statistical analysis
Statistical analyses were conducted using SAS software, version 9·3 (SAS Institute Inc, Cary, NC). Characteristics of patients were summarised as frequencies and proportions for categorical variables and by means, standard deviations and percentiles for continuous variables. Comparisons between PU cases and non‐PU controls were assessed using χ 2 tests for categorical variables and using t‐tests for continuous variables. Statistical significance was declared for a P‐value <0·05.
Analysis of incidence of PUs
Associations of potential risk factors with PU incidence were assessed from a series of logistic regression models. Initially each risk factor was assessed individually in univariable models. Multiple logistic regression was used to obtain the independent effects of a risk factor adjusted for the presence of all other confounding effects. Graphical displays and residual statistics were used to discern an appropriate functional form of the continuous risk factors age and CCI. The models also evaluated all two‐way interactions for statistical significance. Stepwise selection process was used to retain all main effects irrespective of their significance but only significant two‐way interactions (P‐value <0·05). The final model was subjected to rigorous evaluation for detecting potential outliers and influential observations, and was assessed for overall goodness‐of‐fit and predictive power. A model's predictive ability was assessed using the c‐statistic, and goodness‐of‐fit using the Hosmer–Lemeshow χ 2 test 42.
Constructed effects for age and CCI
Spline functions were examined to evaluate the curvilinear effect of age on PU incidence on the log odds (logit) scale. Although several functions were equally compelling, a linear spline was selected for age with four interior knots positioned at the quintiles (age: 32, 50, 64, 77 years), whereas for CCI, two knots at 1 and 5 were adequate.
Body mass index
BMI defined as weight in kilograms divided by the squared height in metres was extracted from the medical chart. It was recorded as a whole number. Unfortunately in 12·8% of the cohort, BMI was missing. Two strategies were adopted to address this issue to maintain the full sample size: (i) BMI was categorised into four subgroups: <19, 19–<25, 25–<30 and ≥30, and an additional missing group was used; (ii) a multiple imputation scheme was used to estimate the missing BMI values. The imputation scheme relied on demographic variables and the 17 comorbidity components of the CCI. A regression‐based method [using PROC MI in SAS software 43] generated 5 separate data sets. The imputations maintained the original range of known BMI values (10–40 kg/m2).
Hierarchical analyses
Because our study cohort was assembled within each of the 15 hospitals, a limited hierarchical analysis in which patients within a hospital were deemed to share characteristics of the hospitals was carried out 44. A random intercept for each hospital served to cluster patients. Generalised linear mixed effects models with fixed effects for gender, race, age, CCI, BMI, surgical DRG, admission type and fiscal year were estimated using PROC GLIMMIX in SAS software 43.
Receiver operating characteristic analysis of models
The subject‐specific predicted probability of PU incidence π from the logistic model was used to calculate the model's discriminative power as measured by the c‐statistic 45. For a pair of patients, one with an incident PU (case) and the other without PU (control), the c‐statistic is the probability that the model estimates a higher probability of PU incidence in the case than in the control. The c‐statistic is equivalent to the area under the receiver operating characteristic (ROC) curve. For a cut‐off a, sensitivity is the proportion among cases where π ≥ a, and specificity is the proportion among controls where π < a. The ROC maps the points (sensitivity, 1 − specificity) as the cut‐point a varies between 0 and 1. A c‐statistic above 0·75 is considered excellent. Submodels with fewer covariates may be compared with respect to their c‐statistics 46.
Results
The overall incidence of PUs was 2·68% [95% confidence interval (CI): 2·62−2·74%]. Incidence was also approximately constant over the two fiscal year periods, that is, 2·48% (95% CI: 2·39−2·56%) in FY2009 and 2·89% (95% CI: 2·80−2·99%). Table 1 describes characteristics of patients in the two fiscal periods. Owing to large sample size, there were statistical differences in patient characteristics between the two periods, but qualitatively these differences were inconsequential. No significant differences in patient characteristics were found in the two reporting periods within PU cases and within non‐PU controls. Beginning in January 2010, the Trinity UCO implemented a PU screening standard for all inpatient admissions. We attribute the slightly higher incidence in 2010 to increased PU awareness and surveillance across the organisation. Analyses were conducted with both fiscal years combined.
Table 1.
Period | July 2009–June 2010: No. (%) | July 2010–June 2011: No. (%) | Combined period July 2009–June 2011: No. (%) | |||
---|---|---|---|---|---|---|
Characteristic | PU cases (n = 3087) | Non‐PU controls (n = 121 582) | PU cases (n = 3418) | Non‐PU controls (114 698) | PU cases (n = 6505) | Non‐PU controls (236 280) |
Gender | ||||||
Female | 1665 (53·9) | 77 174 (63·5) | 1821 (53·3) | 71 652 (62·5) | 3486 (53·6) | 148 826 (63·0) |
Male | 1422 (46·1) | 44 408 (36·5) | 1597 (46·7) | 43 046 (37·5) | 3019 (46·4) | 87 454 (37·0) |
Race | ||||||
Black | 413 (13·4) | 17 403 (14·3) | 443 (13·0) | 14 901 (13·0) | 856 (13·2) | 32 304 (13·7) |
White | 2543 (82·4) | 95 587 (78·6) | 2881 (84·3) | 92 258 (80·4) | 5424 (83·4) | 187 845 (79·5) |
Other | 131 (4·2) | 8592 (7·1) | 94 (2·8) | 7539 (6·6) | 225 (3·5) | 16 131 (6·8) |
Age (years) | ||||||
15–25 | 15 (<0·5) | 10 523 (8·7) | 17 (0·5) | 9993 (8·7) | 32 (0·5) | 20 516 (8·7) |
25–35 | 32 (1·0) | 17 921 (14·7) | 32 (0·9) | 17 922 (15·6) | 64 (<1·0) | 35 843 (15·2) |
35–45 | 89 (2·9) | 12 756 (10·5) | 81 (2·4) | 12 486 (10·9) | 170 (2·6) | 25 242 (10·7) |
45–55 | 251 (8·1) | 15 214 (12·5) | 271 (7·9) | 14 534 (12·7) | 522 (8·0) | 29 748 (12·6) |
55–65 | 441 (14·3) | 17 869 (14·7) | 527 (15·4) | 17 585 (15·3) | 968 (14·9) | 35 454 (15·0) |
65–75 | 567 (18·4) | 17 824 (14·7) | 681 (19·9) | 16 835(14·7) | 1248 (19·2) | 34 659 (14·7) |
75–85 | 907 (29·4) | 18 410 (15·1) | 977 (28·6) | 15 659 (13·7) | 1884 (29·0) | 34 069 (14·4) |
≥85 | 785 (25·4) | 11 065 (9·1) | 832 (24·3) | 9684 (8·4) | 1617 (24·9) | 20 749 (8·8) |
Comorbidity | ||||||
0 | 277 (9·0) | 59 837 (49·2) | 324 (9·5) | 59 003 (51·4) | 601 (9·4) | 118 840 (50·3) |
1 | 431 (14·0) | 24 834 (20·4) | 486 (14·2) | 23 490 (20·5) | 917 (14·1) | 48 324 (20·5) |
2 | 533 (17·3) | 15 360 (12·6) | 599 (17·5) | 13 849 (12·1) | 1132 (17·4) | 29 209 (12·4) |
3 | 534 (17·3) | 8844 (7·3) | 554 (16·2) | 7581 (6·6) | 1088 (16·7) | 16 425 (7·0) |
4 | 435 (14·1) | 5269 (4·3) | 478 (14·0) | 4427 (3·9) | 913 (14·0) | 9696 (4·1) |
≥5 | 877 (28·4) | 7438 (6·1) | 977 (28·6) | 6348 (5·5) | 1854 (28·5) | 13 786 (5·8) |
PU, pressure ulcer; CCI, Charlson Comorbidity Index.
Denominators for percentages are the total number of patients within each characteristic. Percentages might not sum to 100 due to rounding.
Approximately 63% of the cohort were female, almost 80% were White and 39% were 65 years of age or older (Table 2). Forty‐nine percent had no presenting comorbidity, whereas 6·4% had six or more comorbidities. Because BMI was not available for 12·8% of the cohort, the strategy of multiple imputation was used to estimate the missing data. After imputation, the overall percentage of subjects in the four BMI groups (in kg/m2): <19, 19–<25, 25–<30, 30+ were 3·8%, 24·8% 33·7% and 37·7%, respectively. Subsequent analyses used the imputed BMI.
Table 2.
Characteristic | % of total | No. of cases (%)* | Odds ratio (95% CI) | C‐statistic |
---|---|---|---|---|
Sex | 0·547 | |||
Female | 62·7 | 3486 (2·29) | Reference | |
Male | 37·3 | 3019 (3·34) | 1·47 (1·40, 1·55) | |
Race | 0·522 | |||
White | 79·6 | 5424 (2·81) | Reference | |
Black | 13·7 | 856 (2·58) | 0·92 (0·85, 0·99) | |
Other | 6·7 | 225 (1·38) | 0·48 (0·42, 0·55) | |
Age (years) | 0·738 | |||
15–25 | 8·5 | 32 (0·16) | 0·09 (0·06, 0·13) | |
25–35 | 14·8 | 64 (0·18) | 0·10 (0·08, 0·13) | |
35–45 | 10·5 | 170 (0·67) | 0·38 (0·32, 0·46) | |
45–55 | 12·5 | 522 (1·72) | Reference | |
55–65 | 15·0 | 968 (2·66) | 1·56 (1·40, 1·73) | |
65–75 | 14·8 | 1248 (3·48) | 2·05 (1·85, 2·28) | |
75–85 | 14·8 | 1884 (5·24) | 3·15 (2·86, 3·48) | |
≥85 | 9·2 | 1617 (7·23) | 4·44 (4·02, 4·91) | |
CCI | 0·793 | |||
0 | 49·2 | 601 (0·50) | 0·13 (0·12, 0·14) | |
1 | 20·3 | 917 (1·86) | 0·49 (0·45, 0·54) | |
2 | 12·5 | 1132 (3·73) | Reference | |
3 | 7·2 | 1088 (6·21) | 1·71 (1·57, 1·86) | |
4 | 4·4 | 913 (8·61) | 2·43 (2·22, 2·66) | |
≥5 | 6·4 | 1854 (11·85) | 3·47 (3·21, 3·75) | |
Body mass index (kg/m2) | 0·624 | |||
Missing | 12·8 | 605 (1·94) | 0·48 (0·43, 0·52) | |
<19 | 3·3 | 796 (10·05) | 2·69 (2·47, 2·93) | |
19–25 | 22·0 | 2128 (3·99) | Reference | |
25–30 | 29·2 | 1603 (2·26) | 0·56 (0·52, 0·59) | |
≥30 | 32·8 | 1373 (1·73) | 0·42 (0·40, 0·45) | |
Fiscal period | 0·520 | |||
FY2009 | 51·4 | 3087 (2·48) | Reference | |
FY2010 | 48·7 | 3481 (2·89) | 1·17 (1·12, 1·23) | |
Surgical DRG | 0·568 | |||
No | 1376 (1·65) | Reference | ||
Yes | 5129 (3·22) | 1·99 (1·87, 2·11) | ||
Admission type | 0·622 | |||
Home | 61·5 | 2796 (1·87) | Reference | |
Emergency | 27·2 | 2516 (3·81) | 2·07 (1·96, 2·19) | |
Transfer | 6·8 | 1072 (6·48) | 3·63 (3·37, 3·90) | |
Other | 4·5 | 121 (1·10) | 0·58 (0·49, 0·70) |
CCI, Charlson Comorbidity Index; CI, confidence interval.
Denominators for percentages are the total number of patients with each characteristic.
Correlates of PU incidence
There were significant differences in characteristics between PU cases and non‐PU controls. Overall, the mean age in cases was 73·1 years (SD = 14·7 years) compared to a mean age of 55·1 years (SD = 21·5 years) among controls (P < 0·0001). There was a difference of 20 years in the median ages (76 years in cases versus 56 years in controls). The age range in the two groups was approximately the same: 15–109 years in controls and 17–107 years in cases. Each risk factor shown in Table 2 was associated with PU incidence (P < 0·0001, χ 2 tests) with the comorbidity index exhibiting the strongest association. Each of the 17 components of the CCI was associated with incidence (Table 3). Generally, PU incidence increased with increasing age and higher comorbidity. Being male was associated with a higher risk of incidence. Patients reporting African American or other race were at lower risk of PU incidence compared with those reporting White race. Patients with BMI (in kg/m2) between 19 and <25 reported a higher risk of PU incidence than those with BMI of <19 (OR = 2·47; 95% CI: 2·27, 2·69), but a lower risk in the BMI 25 to <30 group (OR = 0·58; 95% CI: 0·54, 0·62), and in the BMI ≥30 group (OR = 0·45; 95% CI: 0·42, 0·49).
Table 3.
Comorbidity | No. PU cases (%) | Odds ratio | 95% lower CL | 95 % upper CL |
---|---|---|---|---|
Myocardial infarction | 1032 (5·95) | 2·54 | 2·38 | 2·72 |
Congestive heart failure | 2540 (9·60) | 5·69 | 5·40 | 5·99 |
Peripheral vascular disease | 992 (8·45) | 3·78 | 3·52 | 4·05 |
Cerebrovascular disease | 904 (6·36) | 2·70 | 2·52 | 2·91 |
Dementia | 573 (9·08) | 3·88 | 3·55 | 4·25 |
Chronic pulmonary disease | 2273 (5·30) | 2·58 | 2·45 | 2·72 |
Connective tissue disease | 263 (5·04) | 1·97 | 1·73 | 2·23 |
Peptic ulcer disease | 195 (6·11) | 2·41 | 2·08 | 2·79 |
Mild liver disease | 416 (6·75) | 2·74 | 2·47 | 3·04 |
Diabetes without complications | 2097 (5·04) | 2·37 | 2·25 | 2·50 |
Diabetes with complications | 556 (8·92) | 3·80 | 3·47 | 4·16 |
Paraplegia and hemiplegia | 451 (12·59) | 5·55 | 5·01 | 6·14 |
Renal disease | 2227 (9·40) | 5·21 | 4·94 | 5·49 |
Cancer | 814 (5·61) | 2·32 | 2·15 | 2·50 |
Moderate or severe liver disease | 143 (10·32) | 4·25 | 3·57 | 5·07 |
Metastatic carcinoma | 414 (7·89) | 3·26 | 2·94 | 3·61 |
AIDS | 20 (5·13) | 1·97 | 1·25 | 3·09 |
CL, confidence limit.
Denominator for percentages is the number of patients with the comorbidity.
Adjusted analyses
Initially, our multiple logistic regression model for PU incidence considered all main effects: sex, race, age, CCI, BMI, a surgical DRG, admission type and fiscal year. Our focus was primarily on the first five patient characteristics. Sex was not significant (P = 0·427) and race exhibited significance for the comparison of other race with White race. Patients admitted for a surgical DRG were at higher risk of acquiring a PU (OR = 2·98, 95% CI: 2·80, 3·17). Selected odds ratios and 95% confidence intervals are shown in Table 4. Logistic regression analysis for PU incidence is based on five imputed data sets which produce overall (within imputation) regression parameter estimates and their standard errors (within—between imputations). Odds ratios and confidence intervals, and P‐values for Wald χ 2 tests of significance were computed from these estimates.
Table 4.
Characteristic | Group | Odds ratio (95% CI) | P‐value |
---|---|---|---|
Sex (ref Female) | Male | 1·02 (0·97, 1·08) | 0·427 |
Race (ref White) | Black | 1·03 (0·95, 1·12) | |
Other | 0·84 (0·73, 0·97) | 0·034 | |
BMI, kg/m2 (ref 19–25) | <19 | 2·21 (2·02, 2·41) | |
25– 30 | 0·69 (0·65, 0·74) | <0·0001 | |
≥30 | 0·62 (0·58, 0·67) | ||
Surgical DRG (ref No) | Yes | 2·98 (2·80, 3·17) | <0·0001 |
Admission type (ref Home) | Emergency | 1·78 (1·66, 1·91) | |
Other | 0·49 (0·41, 0·60) | <0·0001 | |
Transfer | 2·33 (2·16, 2·52) | ||
Fiscal year (ref FY2009) | FY2010 | 1·63 (1·54, 1·74) | <0·0001 |
CI, confidence interval.
Main effects model adjusted for age (P < 0·0001) and comorbidity (P < 0·0001) as linear splines.
Interaction effects
All two‐way interactions for inclusion in the main effects models were evaluated, restricting to age, sex, race, CCI and BMI. All effects were allowed to compete for inclusion, but only effects and their interactions that were significant at P < 0·05 were retained. Three significant interactions were found between age, CCI and BMI. Both sex and race were no longer significant. The final parsimonious model found age, CCI and BMI and their two‐way interactions, surgical DRG, admission type and fiscal year as significant. There was a marginal improvement in the c‐statistic from 0·856 to 0·860 in the main effects model. The Hosmer–Lemeshow goodness‐of‐fit test was not significant (8 df χ2 test, P = 0·606). The effects of age and CCI on the likelihood of PU incidence are shown in Figures 2 and 3 for the four BMI groups at specified profiles.
The contour plot (Figure 4) depicts the joint effect of age and CCI on the probability of PU incidence, displayed for the four categories of BMI. Probability levels are marked on the display and indicate a higher incidence of PU with a combination of increasing age and comorbidity. The relationship is, however, complex as the interaction model has 56 estimated parameters. The effects of a surgical DRG, admission type and fiscal year were also significant, and were quantitatively about the same as those in the main effects model of Table 4.
Hierarchical model
Inclusion of a random effect for patients from the hospital served to cluster patients within hospital. The effect was significant. However, with respect to the fixed effects evaluated in this study, the estimated odds ratios (Table 5) were substantively the same as those in Table 4. We used random effects logistic regression analysis for PU incidence based on five imputed data sets which produced overall (within imputation) regression parameter estimates and their standard errors (within–between imputations). Odds ratios and confidence intervals were computed from these estimates.
Table 5.
Characteristic | Group | Odds ratio (95% CI) |
---|---|---|
BMI, kg/m2 (ref 19–25) | <19 | 3·07 (2·41, 3·91) |
25–30 | 0·65 (0·53, 0·80) | |
≥30 | 0·52 (0·42, 0·64) | |
Surgical DRG (ref No) | Yes | 2·92 (2·74, 3·11) |
Admission type (ref Home) | Emergency | 1·73 (1·60, 1·86) |
Other | 0·53 (0·44, 0·64) | |
Transfer | 2·43 (2·25, 2·64) | |
Fiscal year (ref FY2009) | FY2010 | 1·60 (1·50, 1·71) |
BMI, body mass index; CI, confidence interval; DRG, diagnosis‐related groups.
Adjusted for age (P < 0·0001) and comorbidity (P < 0·0001) as linear splines.
Model contains one random intercept for clustering patients within hospital and interactions terms for age with BMI, and comorbidity with BMI.
Age and comorbidity are fixed at mean values, 55·6 years and 1·27, respectively, in calculation of BMI odds ratios.
Predicted PU incidence probabilities
From our model, the predicted PU incidence probability was calculated for each patient. There is an one‐to‐one correspondence between the predicted probability and the weighted sum of covariates in the model. The weights are the estimated regression coefficients (log‐odds ratios) and the weighted sum is called the risk score. We formed four risk groups using the quartiles of the risk scores. Figures 5 and 6 show the distributions of BMI and CCI in the four risk groups of PU incidence. In the lowest risk group, comorbidities are virtually absent; in the highest risk group, nearly 61% have three or more comorbidities.
ROC analysis of models
We compared the c‐statistic across four models: (i) the full model contained age (spline), BMI, CCI (spline) and their two‐way interactions, surgical DRG, admission type and fiscal year, (ii) a submodel of (i) with only age, BMI, CCI and their two‐way interactions, (iii) a submodel of (ii) with only the main effects of age, BMI and CCI, (iv) the hierarchical model with a random effect for clustering patients by hospital, and all effects in (i). ROC curves are shown in Figure 7. Although significant differences are seen in the three comparison of models (i), (ii), (iii) with model (iv), the quantitative differences are small. Nevertheless, model (iv) is the best.
Discussion
Multiple studies have demonstrated the association between demographic risk factors and development of hospital‐acquired PUs, particularly in the demographic categories of age, sex, and comorbidity; findings related to BMI and race are inconsistent. Our study focused on race, age, sex, BMI, number of comorbidities and the type of comorbidities present. Our analysis of PU incidence using electronic health records found that patients with the highest risk are older, have five or more comorbidities, were transferred from another health care facility, and discharged with a surgical DRG. Our evidence of admission type and DRG offer the potential of new intervention modalities. No difference in PU risk based on race and gender after controlling for age, comorbidity and BMI was found.
Regular performance of a well‐integrated risk assessment for PU has been shown to reduce the risk of incident PU in health care settings by providing an easy framework for identifying high‐risk patients 47. Research has also demonstrated that the most common risk assessments used have limited success in allocation of resources for PU prevention, and recommendations for further research on prediction models have been recommended 20. Kim and Lang developed a predictive model based on Braden Scale scores and personal risk factors 48. At the initiation of our study in 2009, the 15 hospitals in our database had not completed the inclusion of the Braden assessment in the Cerner© Enterprise Data Warehouse. Therefore, we limited our study to patient variables that were substantially complete, uniformly collected and entered, along with other clinical data. As of late 2011, Trinity Health uniform clinical care mandates that a Braden assessment be given to each admitted patient.
Transfer from another health care facility carried higher risk of PU development (OR = 3·63) than admission for emergent care or admission from home. Although why this relationship exists could not be discerned from the data available, increased risk of PU development that is related to the age and health condition of the patients transferred from other health care facilities was suspected.
Age has a positive linear relationship with PU development. As age increased, the odds of developing a PU increased. The highest risk group was patients of age ≥85 years (OR = 4·44); however, significant risk increase started in the 65‐ to <75‐year group (OR = 2·05) and was also elevated in the 75‐ to <85‐year group (OR = 3·15). In our study, the relationship between age and PU incidence was more effectively captured using a spline function.
Surgeries with an operating time of 3 or more hours put patients at a greater risk of PU development during surgery or the days after the surgery when the patient is dependent on hospital staff for mobility 7, 20. Our study demonstrates that admission for surgery increases the risk of developing a PU (OR = 1·99). Frankel et al. 19 demonstrated the increased risk of surgery patients with independent risk factors for age greater than 60 years, and comorbidities such as diabetes, spinal cord injury, vascular disease and renal disease. These factors had a stronger association than the Apache II risk score and ICU length of stay. In the surgical ICU, PUs developed in patients with these risk factors in spite of the use of early nutritional intervention and specialty beds.
The CCI was originally developed and validated for hospital mortality prediction. However, it has been used extensively as a summary measure of presenting comorbidity in relation to other adverse outcomes. Our study also shows that the CCI has a strong association with PU development. Specific components of the index that carried the highest risk were vascular diseases, such as congestive heart failure, diabetes, renal disease, paraplegia or hemiplegia, and severe liver disease. Each of these conditions carried a risk of at least threefold for development of a PU during hospitalisation. Although the odds of PU incidence generally increased with higher CCI: for example, OR = 3·47 for CCI ≥5 versus CCI = 2, the relationship was not linear over the whole range and a spline function of CCI was used to capture its effect on PU. As demonstrated by Kim and Lang 48, comorbidities may function best as a predictor when used in conjunction with a comprehensive assessment to determine the risk of PU.
Our study shows that a combination of risk factors (age, BMI, CCI) has a strong association with PU incidence. In our ROC analysis, these variables and their interactions conferred an impressive c‐statistic of 0·838.
Patients admitted for surgery or transfer patients or patients with many comorbidities would benefit from a comprehensive risk assessment and close monitoring for PU development. It may prove useful to flag these patients for comprehensive risk assessment beyond Braden or closer observation despite Braden Scale results.
Strengths
A principal strength of this study is the large inpatient population from which our sample was drawn which included 15 individual hospitals from five states. Further, we were able to identify patients with multiple hospital admissions during the time period of the study. This permitted us to limit the representation of each patient in our data to a single admission (the first admission), thereby assuring that no individual could contribute multiple PU incidents to the analysis. This eliminated some opportunities for confounding which could account for differences in results reported here, as compared with results reported in other large scale studies. For example, Fogerty et al. 6 reported African American race as a risk factor for PU, while we did not observe an association between African American race and risk of PU. We suspected that African Americans may have been at greater risk of multiple admissions with PU. A post hoc examination of the data on all patients who had at least one incident PU over possibly multiple admissions showed that incidence of PU on multiple hospital stays was higher in African Americans (18·9%) than in Whites (13·0%), or in ‘Other’ race (12·1%). In addition to the 6 505 patients studied here with a single PU incidence, there were 1043 patients who had at least two PU incidences. This highlights the importance of avoiding inappropriate interpretation of findings that are based on data not permitting the identification of multiple admissions of the same patient. Our data will now permit the future search of factors accounting for the observed differential rates of occurrence of PU and other target outcomes among patient groups with multiple admissions.
Limitations
The primary limitation of our study is similar to that faced when traditional medical records are abstracted: there is a possibility that PUs counted as incident were in reality present at the time of admission. Omission of PUs present on admission from the medical records could be the result of not recognising the condition or simply not recording it present at admission. Therein also lies the possibility that a control patient had developed an incident PU that was not discerned during the hospital stay. Retrospective cohort studies are limited by the quality of data available, and determining whether electronic health records improve data quality is an important avenue for future research.
We also noted a slightly higher incidence of PU in FY2010 compared with that in FY2009. This may be an artefact of ascertainment. FY2010 was the second year of full EHR implementation in Trinity Health Systems, and it is possible that reporting of PU had improved. The fiscal crisis might have led to lower rates of health‐seeking behaviours in FY2009, and FY2010 might have shown an increase based on improvements in the health‐seeking behaviours. To investigate this difference further, we would need to examine PU incidence data for fiscal years prior to 2009 and after 2010.
This study was limited in terms of data available. As such, we were unable to analyse the hospital‐level data on PU incidence. There was limited information on hospital size and staffing ratios, but staff training and education requirements were not available for analysis. We were able to discern differences between hospitals and PU development; three hospitals demonstrated a protective effect against PU development, while four hospitals had an increased risk of PU development, but specific hospital‐level characteristics that might inform these differences were not studied. As noted previously, Braden scale scores were not available at the time this research was conducted, and a deeper examination of the relationship between these risk factors and the Braden scale should be examined to determine the usefulness of our model in our health system.
The utilisation of one hospital system may limit the generalisability of our results. None of the major metropolitan regions of the USA were represented in the sample. However, four regions of the USA are represented: the East Coast, the Midwest, the Central Plains and the Northwest, which does provide insight into a diverse range of population characteristics. The sample size of this study gives it a greater advantage, and its findings are consistent with those of other studies examining PU incidence and risk. We have also contributed two possible categories for informing PU risk through comprehensive assessment administration: a surgery DRG and transfer from another health care facility. It is hoped that these results will help to generate a staff and EHR‐friendly model of PU prediction that provides improved care for patients across health care systems.
Acknowledgements
The authors thank the editor and the reviewers for their comments and remarks that helped improve the initial version of our manuscript. This research was supported in part by the Trinity Institute for Health and Community Benefit. The authors thank Katrina Wright, Joan Hall, Tina Maxbauer, Joyce Ellies, Hank Groot, Jana Inwood, Kyle Johnson, Tom Centlivre, Charles Bowling and Robert Sloan for assembling the patient database. They also express their gratitude to Paul Conlon for his advice and guidance in understanding the clinical operations of CHE/Trinity Health hospitals.
References
- 1. Russo CA, Steiner C, Spector W. Hospitalizations related to pressure ulcers among adults 18 years and older, 2006. Statistical Brief #64. Washington, DC: Agency for Healthcare Research and Quality, 2008. [PubMed] [Google Scholar]
- 2. Theisen S, Drabik A, Stock S. Pressure ulcers in older hospitalised patients and its impact on length of stay: a retrospective observational study. J Clin Nurs 2012;21:380–7. [DOI] [PubMed] [Google Scholar]
- 3. Vetrano DL, Landi F, De Buyser SL, Carfi A, Zuccala G, Petrovic M, Volpato S, Cherubini A, Corsonello A, Bernabei R, Onder G. Predictors of length of hospital stay among older adults admitted to acute care wards: a multicentre observational study. Eur J Intern Med 2014;25:56–62. [DOI] [PubMed] [Google Scholar]
- 4. Graves N, Birrell F, Whitby M. Effect of pressure ulcers on length of hospital stay. Infect Control Hosp Epidemiol 2005;26:293–7. [DOI] [PubMed] [Google Scholar]
- 5. Whittington K, Patrick M, Roberts JL. A national study of pressure ulcer prevalence and incidence in acute care hospitals. J Wound Ostomy Continence Nurs 2000;27:209–15. [DOI] [PubMed] [Google Scholar]
- 6. Fogerty MD, Abumrad NN, Nanney L, Arbogast PG, Poulose B, Barbul A. Risk factors for pressure ulcers in acute care hospitals. Wound Repair Regen 2008;16:11–8. [DOI] [PubMed] [Google Scholar]
- 7. Beckrich K, Aronovitch SA. Hospital‐acquired pressure ulcers: a comparison of costs in medical vs. surgical patients. Nurs Econ 1999;17:263–71. [PubMed] [Google Scholar]
- 8. Terekeci H, Kucukardali Y, Top C, Onem Y, Celik S, Oktenli C. Risk assessment study of the pressure ulcers in intensive care unit patients. Eur J Intern Med 2009;20:394–7. [DOI] [PubMed] [Google Scholar]
- 9. Nijs N, Toppets A, Defloor T, Bernaerts K, Milisen K, Van Den Berghe G. Incidence and risk factors for pressure ulcers in the intensive care unit. J Clin Nurs 2009;18:1258–66. [DOI] [PubMed] [Google Scholar]
- 10. Shannon RJ, Brown L, Chakravarthy D. Pressure Ulcer Prevention Program Study: a randomized, controlled prospective comparative value evaluation of 2 pressure ulcer prevention strategies in nursing and rehabilitation centers. Adv Skin Wound Care 2012;25:450–64. [DOI] [PubMed] [Google Scholar]
- 11. Pearson A, Pallas LO, Thomson D, Doucette E, Tucker D, Wiechula R, Long L, Porritt K, Jordan Z. Systematic review of evidence on the impact of nursing workload and staffing on establishing healthy work environments. Int J Evid Based Healthc 2006;4:337–84. [DOI] [PubMed] [Google Scholar]
- 12. Lahmann NA, Kottner J, Dassen T, Tannen A. Higher pressure ulcer risk on intensive care? – Comparison between general wards and intensive care units. J Clin Nurs 2012;21:354–61. [DOI] [PubMed] [Google Scholar]
- 13. Baharestani MM, Black JM, Carville K, Clark M, Cuddigan JE, Dealey C, Defloor T, Harding KG, Lahmann NA, Lubbers MJ, Lyder CH, Ohura T, Orsted HL, Reger SI, Romanelli M, Sanada H. Dilemmas in measuring and using pressure ulcer prevalence and incidence: an international consensus. Int Wound J 2009;6:97–104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Bluestein D, Javaheri A. Pressure ulcers: prevention, evaluation, and management. Am Fam Physician 2008;78:1186–94. [PubMed] [Google Scholar]
- 15. Smith A, McNichol LL, Amos MA, Mueller G, Griffin T, Davis J, McPhail L, Montgomery TG. A retrospective, nonrandomized, beforeand‐ after study of the effect of linens constructed of synthetic silk‐like fabric on pressure ulcer incidence. Ostomy Wound Manage 2013;59:28–30 2–4. [PubMed] [Google Scholar]
- 16. Keller BP, Wille J, van Ramshorst B, van der Werken C. Pressure ulcers in intensive care patients: a review of risks and prevention. Intensive Care Med 2002;28:1379–88. [DOI] [PubMed] [Google Scholar]
- 17. Mathus‐Vliegen EM. Old age, malnutrition, and pressure sores: an ill‐fated alliance. J Gerontol A Biol Sci Med Sci 2004;59:355–60. [DOI] [PubMed] [Google Scholar]
- 18. Barker AL, Kamar J, Tyndall TJ, White L, Hutchinson A, Klopfer N, Weller C. Implementation of pressure ulcer prevention best practice recommendations in acute care: an observational study. Int Wound J 2013;10:313–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Frankel H, Sperry J, Kaplan L. Risk factors for pressure ulcer development in a best practice surgical intensive care unit. Am Surg 2007;73:1215–7. [PubMed] [Google Scholar]
- 20. Schoonhoven L, Defloor T, Grypdonck MH. Incidence of pressure ulcers due to surgery. J Clin Nurs 2002;11:479–87. [DOI] [PubMed] [Google Scholar]
- 21. Baumgarten M, Margolis DJ, Localio AR, Kagan SH, Lowe RA, Kinosian B, Holmes JH, Abbuhl SB, Kavesh W, Ruffin A. Pressure ulcers among elderly patients early in the hospital stay. J Gerontol A Biol Sci Med Sci 2006;61:749–54. [DOI] [PubMed] [Google Scholar]
- 22. Baumgarten M, Margolis DJ, Localio AR, Kagan SH, Lowe RA, Kinosian B, Abbuhl SB, Kavesh W, Holmes JH, Ruffin A, Mehari T. Extrinsic risk factors for pressure ulcers early in the hospital stay: a nested case–control study. J Gerontol A Biol Sci Med Sci 2008;63:408–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. VanGilder C, MacFarlane G, Meyer S, Lachenbruch C. Body mass index, weight, and pressure ulcer prevalence: an analysis of the 2006–2007 International Pressure Ulcer Prevalence Surveys. J Nurs Care Qual 2009;24:127–35. [DOI] [PubMed] [Google Scholar]
- 24. Banks M, Bauer J, Graves N, Ash S. Malnutrition and pressure ulcer risk in adults in Australian health care facilities. Nutrition 2010;26:896–901. [DOI] [PubMed] [Google Scholar]
- 25. Coleman S, Gorecki C, Nelson EA, Closs SJ, Defloor T, Halfens R, Farrin A, Brown J, Schoonhoven L, Nixon J. Patient risk factors for pressure ulcer development: systematic review. Int J Nurs Stud 2013;50:974–1003. [DOI] [PubMed] [Google Scholar]
- 26. Tescher AN, Branda ME, Byrne TJ, Naessens JM. All at‐risk patients are not created equal: analysis of Braden pressure ulcer risk scores to identify specific risks. J Wound Ostomy Continence Nurs 2012;39:282–91. [DOI] [PubMed] [Google Scholar]
- 27. Theaker C, Mannan M, Ives N, Soni N. Risk factors for pressure sores in the critically ill. Anaesthesia 2000;55:221–4. [DOI] [PubMed] [Google Scholar]
- 28. Drake DJ, Swanson M, Baker G, Pokorny M, Rose MA, Clark‐Reed L, Waters W, Watkins FR Jr, Engelke MK. The association of BMI and Braden total score on the occurrence of pressure ulcers. J Wound Ostomy Continence Nurs 2010;37:367–71. [DOI] [PubMed] [Google Scholar]
- 29. Bergstrom N, Braden B, Kemp M, Champagne M, Ruby E. Multi‐site study of incidence of pressure ulcers and the relationship between risk level, demographic characteristics, diagnoses, and prescription of preventive interventions. J Am Geriatr Soc 1996;44:22–30. [DOI] [PubMed] [Google Scholar]
- 30. Bergstrom N, Braden B, Kemp M, Champagne M, Ruby E. Predicting pressure ulcer risk: a multisite study of the predictive validity of the Braden Scale. Nurs Res 1998;47:261–9. [DOI] [PubMed] [Google Scholar]
- 31. Charlson ME, Pompei P, Ales KL, Mackenzie CR. A new method of classifying prognostic co‐morbidity in longitudinal‐studies – development and validation. J Chronic Dis 1987;40:373–83. [DOI] [PubMed] [Google Scholar]
- 32. Ballian N, Luketich JD, Levy RM, Awais O, Winger D, Weksler B, Landreneau RJ, Nason KS. A clinical prediction rule for perioperative mortality and major morbidity after laparoscopic giant paraesophageal hernia repair. J Thorac Cardiovasc Surg 2013;145:721–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Grosso G, di Francesco F, Vizzini G, Mistretta A, Pagano D, Echeverri GJ, Spada M, Basile F, Gridelli B, Gruttadauria S. The Charlson Comorbidity Index as a predictor of outcomes in liver transplantation: single‐center experience. Transplant Proc 2012;44:1298–302. [DOI] [PubMed] [Google Scholar]
- 34. Flattau A, Blank AE. Risk factors for 90‐day and 180‐day mortality in hospitalised patients with pressure ulcers. Int Wound J 2014;11:14–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Bergstrom N, Horn SD. Racial disparities in rates of pressure ulcers in nursing homes and site of care. JAMA 2011;306:211–2. [DOI] [PubMed] [Google Scholar]
- 36. Metersky ML, Hunt DR, Kliman R, Wang Y, Curry M, Verzier N, Lyder CH, Moy E. Racial disparities in the frequency of patient safety events: results from the National Medicare Patient Safety Monitoring System. Med Care 2011;49:504–10. [DOI] [PubMed] [Google Scholar]
- 37. Black J, Berke C, Urzendowski G. Pressure ulcer incidence and progression in critically ill subjects: influence of low air loss mattress versus a powered air pressure redistribution mattress. J Wound Ostomy Continence Nurs 2012;39:267–73. [DOI] [PubMed] [Google Scholar]
- 38. Pompei P, Charlson ME, Ales K, Mackenzie CR, Norton M. Relating patient characteristics at the time of admission to outcomes of hospitalization. J Clin Epidemiol 1991;44:1063–9. [DOI] [PubMed] [Google Scholar]
- 39. Harrell FE, Lee KL, Pollock BG. Regression‐models in clinical‐studies – determining relationships between predictors and response. J Natl Cancer Inst 1988;80:1198–202. [DOI] [PubMed] [Google Scholar]
- 40. Gurrin LC, Scurrah KJ, Hazelton ML. Tutorial in biostatistics: spline smoothing with linear mixed models. Stat Med 2005;24:3361–81. [DOI] [PubMed] [Google Scholar]
- 41. Averill RF, Goldfield N, Hughes JS, Bonazelli J, Steinbeck BA, Mullin R, Tang AM. Development of the all patient refined DRGs (APR‐DRGs) Research Report. Wallingford, CT: 3M Health Information Systems, 2003. [Google Scholar]
- 42. Hosmer DW, Hosmer T, le Cessie S, Lemeshow S. A comparison of goodness‐of‐fit tests for the logistic regression model. Stat Med 1997;16:965–80. [DOI] [PubMed] [Google Scholar]
- 43. SAS Institute Inc . SAS/STAT User's guide, 9.3 edn. Cary, NC: SAS Institute Inc, 2011. [Google Scholar]
- 44. Gardiner JC, Luo ZH, Roman LA. Fixed effects, random effects and GEE: what are the differences? Stat Med 2009;28:221–39. [DOI] [PubMed] [Google Scholar]
- 45. Pepe MS. The statistical evaluation of medical tests for classification and prediction. New York, NY: Oxford University Press, 2003. [Google Scholar]
- 46. Delong ER, Delong DM, Clarke‐Pearson DI. Comparing the areas under 2 or more correlated receiver operating characteristic curves – a nonparametric approach. Biometrics 1988;44:837–45. [PubMed] [Google Scholar]
- 47. Hendrichova I, Castelli M, Mastroianni C, Piredda M, Mirabella F, Surdo L, De Marinis MG, Heath T, Casale G. Pressure ulcers in cancer palliative care patients. Palliat Med 2010;24:669–73. [DOI] [PubMed] [Google Scholar]
- 48. Kim TY, Lang N. Predictive modeling for the prevention of hospital‐acquired pressure ulcers. AMIA Annu Symp Proc 2006;434–8. [PMC free article] [PubMed] [Google Scholar]