Abstract
Objective
To assess the value added to the National Early Warning Score (NEWS) by mid-regional pro-adrenomedullin (MR-proADM) blood level in predicting deterioration in mild to moderately ill people.
Design
Prospective observational study.
Setting
The Medical Admissions Suite of the Royal Victoria Infirmary, Newcastle.
Participants
300 adults with NEWS between 2 and 5 on admission. Exclusion criteria included receiving palliative care, or admitted for social reasons or self-harming. Patients were enrolled between September and December 2015, and followed up for 30 days after discharge.
Outcome measure
The primary outcome measure was the proportion of patients who, within 72 hours, had an acuity increase, defined as any combination of an increase of at least 2 in the NEWS; transfer to a higher-dependency bed or monitored area; death; or for those discharged from hospital, readmission for medical reasons.
Results
NEWS and MR-proADM together predicted acuity increase more accurately than NEWS alone, increasing the area under the curve (AUC) to 0.61 (95% CI 0.54 to 0.69) from 0.55 (95% CI 0.48 to 0.62). When the confounding effects of presence of chronic obstructive pulmonary disease or heart failure and interaction with MR-proADM were included, the prognostic accuracy further increased the AUC to 0.69 (95% CI 0.63 to 0.76).
Conclusions
MR-proADM is potentially a clinically useful biomarker for deterioration in patients admitted to hospital with a mild to moderately severe acute illness, that is, with NEWS between 2 and 5. As a growing number of National Health Service hospitals are routinely recording the NEWS on their clinical information systems, further research should assess the practicality and use of developing a decision aid based on admission NEWS, MR-proADM level, and possibly other clinical data and other biomarkers that could further improve prognostic accuracy.
Keywords: internal medicine
Strengths and limitations of this study.
This is the first study to use rigorous statistical methods to assess the value added by mid-regional pro-adrenomedullin (MR-proADM) to the admission National Early Warning Score for predicting clinically important deterioration in mild to moderately ill patients.
Overall prognostic accuracy might have been greater had more severely ill patients been included, but the aim of this study was to predict deterioration in less severely ill patients who could benefit from closer observation.
This was an observational study and thus could not directly assess the use of more accurate prediction of deterioration.
Initial evidence for MR-proADM as a biomarker for deterioration appears promising, but requires further validation for clinical use.
Introduction
The National Early Warning Score (NEWS) is recommended for assessing severity of illness in patients presenting in primary or secondary National Health Service (NHS) care and for surveillance of patients in hospital.1 2 Six physiological parameters (which can be measured at the bedside) are scored: respiratory rate, oxygen saturation, temperature, systolic blood pressure, pulse rate and level of consciousness. The scores are aggregated, and, if the patient requires oxygen, the total is increased. NEWS predicts death, cardiac arrest and unplanned intensive care unit (ICU) admission within 24 hours.3–5 However, NEWS does not identify all patients who turn out to be seriously ill,6–8 and there are also patients whose NEWS is usually elevated and who do not require the level of observation that the NEWS tool would suggest. For example, people with chronic obstructive pulmonary disease (COPD) or chronic heart failure (HF) have higher baseline NEWSs than those without these comorbidities. The prognostic accuracy of NEWS for patients presenting to the emergency department (ED) has been confirmed in a wide range of severity of illness,9 10 as has its reduced accuracy in people with COPD.11 However, no previous studies of the prognostic accuracy of NEWS in the ED/medical admissions unit (MAU, also called Assessment Suite (AS)) have focused on patients admitted with mild to moderately severe illness. Since a clinically important proportion of these patients do deteriorate unexpectedly, improved risk stratification would be useful.
Mid-regional pro-adrenomedullin (MR-proADM) is one of several promising biomarkers for severe illness and deterioration.12–16
MR-proADM is a precursor of adrenomedullin (ADM), a member of the calcitonin peptide family. ADM is widely expressed and has roles in vasodilation, immune modulation and metabolic regulation. It is upregulated in severe infections, inflammation, vasodilation, stimulation of diuresis, increased cardiac output and stroke.17–19 ADM has a short half-life, but MR-proADM is more stable and directly reflects ADM concentrations in blood. Both ADM and MR-proADM levels are strongly associated with risk of mortality, regardless of aetiology.20–26 In people presenting with acute chest pain, MR-proADM has been reported to improve the Global Registry of Acute Coronary Events risk classification by 41%.27 As with the NEWS, people with COPD or chronic HF have higher baseline levels of MR-proADM.
The aim of this study was to assess whether the MR-proADM level used alongside the NEWS would improve prediction of deterioration over NEWS alone in patients admitted to the MAU with mild to moderately severe illness.
Methods
Patient and public involvement
Patients and the public were not specifically involved in the planning and execution of this study. However, the National Institute for Health Research now requires that the research it supports includes active involvement and engagement with patients and the public.
Study participants and study design
This was a prospective observational cohort study. Patients were enrolled between September and December 2015 at the Royal Victoria Infirmary, Newcastle, and followed up for 30 days after discharge. If the patient died within 30 days of follow-up, this and the cause of death were recorded. Adults admitted to the MAU were recruited for the study between 9:00 and 16:00 on weekdays.
Sample size was based on a pragmatic recruitment target for a 3-month observational study. A recent unpublished audit conducted in the MAU at the Royal Victoria Infirmary found a deterioration rate of 20%. With 300 patients and complete data collection, 60 events would be anticipated. With this number of events, a multivariable prediction model could include up to six independent predictors. This is based on a widely accepted rule of thumb that models with fewer than 10 events per predictor tend to be overfitted.28 However, recent research suggests that the ‘ten events per variable’ rule of thumb may be optimistic.29 Because the aim of this study was to assess if further research would be indicated, 60 is considered an acceptable number of events, even if the rule of thumb is optimistic.
Patients were considered eligible for inclusion in the study if their NEWS on admission was at least 2 and not greater than 5, and all NEWS parameters were recorded. Patients were excluded from the study if they were receiving palliative care, were admitted for social reasons only, or were self-harming, or overdosing with drugs or other substances.
Recorded data
Demographic and admission data included gender, year of birth, reason for admissions, diagnosis on discharge and the presence of comorbidities in which baseline MR-proADM levels are chronically raised: COPD with hypoxia (PaO2 <10 kPa)7; HF30; acute brain injury6; acute coronary syndrome27; acute venous thromboembolism21; high international normalised ratio (>2); acute kidney injury; electrolyte disturbances (Na+ <130 or >150 mmol/L; K+ <3.0 or >5.5 mmol/L); hyperglycaemia in type 1 diabetes (random glucose >10 mmol/L).
The NEWS was assessed at admission and over the next 72 hours, and the scores and assessment times were recorded. The seven clinical parameters used to determine the NEWS were recorded for the baseline (admission) assessment only. Baseline NEWSs were used to determine eligibility for this study. Subsequent NEWSs were used in the analyses to identify deterioration.
Blood samples were taken at hospital admission for assessment of MR-proADM, C reactive protein (CRP) and white blood cell (WBC) count.
Laboratory tests
Plasma was obtained from blood samples (collected in EDTA) that were no longer clinically required. Plasma was stored in aliquots at –80°C.
MR-proADM was assayed in the on-site Blood Sciences Laboratory using the B·R·A·H·M·S Kryptor system according to the manufacturer’s instructions.
Blood samples were analysed in batches by personnel blinded with regard to the condition and NEWS of the patient. Nurses who assessed the NEWS and healthcare professionals managing patients in the MAU were blinded to MR-proADM results.
Outcome measures
Outcome measure 1: acuity increase
A patient was classified as having an acuity increase if one or more of the following occurred within 72 hours from admission:
Transfer to a higher level of care (ICU or high-dependency unit).
Readmission to hospital for reasons related to the initial admission.
Death for reasons related to the initial admission.
NEWS increased by at least 2 compared with the admission score.
Outcome measure 2: deterioration event
For most of the observed acuity increase cases, the reason for classification was an increase in the NEWS (table 1). An increase in NEWS reflects both measurement variation and physiological variation, so additional exploratory analyses were carried out to assess the performance of MR-proADM in predicting deterioration. Deterioration events were classified as the occurrence of one or more of the following:
Table 1.
Characteristics of the study population, classified by outcome 1 (acuity increase), outcome 2 (deterioration event) and all patients
| Outcome 1: acuity increase | Outcome 2: deterioration event | All patients (n=292) |
|||
| Present (e=84) | Absent | Present (e2=32) | Absent | ||
| Age (mean years, SD) | 65 (17) | 62 (21) | 63 (14) | 63 (20) | 63 (20) |
| Gender (no of females, %) | 41 (49%) | 107 (51%) | 15 (47%) | 133 (51%) | 148 (51%) |
| NEWS 2 (n, %) | 34 (40%) | 82 (40%) | 12 (38%) | 104 (40%) | 116 (40%) |
| NEWS 3 (n, %) | 26 (31%) | 59 (28%) | 9 (28%) | 76 (29%) | 85 (29%) |
| NEWS 4 (n, %) | 11 (13%) | 43 (21%) | 4 (13%) | 50 (19%) | 54 (18%) |
| NEWS 5 (n, %) | 13 (15%) | 24 (12%) | 7 (22%) | 30 (12%) | 37 (13%) |
| MR-proADM (mean nmol/L, SD) | 1.50 (1.4) (0.72, 1.12, 1.79) |
1.19 (0.9) (0.68, 0.93, 1.28) |
1.89 (2.0) (0.93, 1.13, 1.95) |
1.20 (0.9) (0.68, 0.93, 1.39) |
1.28 (1.1) (0.68, 0.97, 1.48) |
| CRP (mg/L) | 59 (79) (5, 22, 80) |
42 (70) (4, 13, 41) |
61 (90) (7, 23, 67) |
45 (71) (4, 16, 51) |
47 (73) (4, 17, 54) |
| WBC (×109/L) | 12 (5) (9, 10, 14) |
11 (5) (8, 10, 14) |
12 (4) (9, 12, 15) |
11 (5) (8, 10, 14) |
11 (5) (8, 10, 14) |
| COPD/HF (n, %)* | 33 (39%) | 46 (22%) | 12 (38%) | 67 (26%) | 79 (28%) |
| Other comorbidities (n, %) | 17 (20%) | 55 (26%) | 15 (47%) | 57 (22%) | 72 (25%) |
| Length of stay (hours) | 168 (196) (63, 110, 194) |
137 (176) (26, 68, 176) |
173 (172) (59, 106, 259) |
143 (172) (33, 72, 176) |
146 (182) (35, 77, 182) |
| Length of stay in MAU (hours) | 31 (19) (17, 25, 43) |
24 (16) (13, 21, 30) |
27 (17) (18, 23, 35) |
26 (17) (15, 22, 31) |
26 (17) (15, 22, 31) |
| Monitored beds (n, %) | 31 (37%) | 58 (27%) | 11 (34%) | 78 (30%) | 89 (30%) |
| Deterioration time (hours) | 15 (13) (5, 9, 21) |
NA | 170 (226) (19, 33, 301) |
NA | |
Data are presented as number (n) and percentages (%) for counts, or mean and SD for continuous normally distributed data, or (25th, 50th, 75th percentile) for continuous non-normally distributed data.
*For COPD: e=number with acuity increase, 82; e2=number with deterioration event, 29; n=total number of patients, 282.
COPD, chronic obstructive pulmonary disease; CRP, C reactive protein; HF, heart failure; MAU, medical admissions unit; MR-proADM, mid-regional pro-adrenomedullin; NA, not applicable; NEWS, National Early Warning Score; WBC, white blood cell.
Transfer to higher level of care within 72 hours from admission;
Death (for reasons related to the admission) within 30 days;
Readmission to hospital (for the same reason as the previous admission) within 30 days from first admission.
Classification based on this definition is unlikely to be subject to clinically important measurement variation. This analysis, therefore, should optimise the prognostic accuracy for events which are both clinically and economically important.
Outcome measure 3: length of stay
Length of stay was defined as the duration (in days) from admission to discharge or death.
Statistical analysis
All data analyses were performed using the R language, V.3.2.0,31 with the support of RStudio, V.0.99.896 (RStudio). The following R packages were used: ggplot2, pROC, psych, PredictABEL, Hmisc and rms.
Logistic regression models were compared for their accuracy in predicting deterioration outcome measures as prespecified in an analysis plan. Analyses are presented as unadjusted parameter estimates of risk (OR, with CIs) and estimates adjusted for identified clinical confounding factors. The aims of the multivariable analyses were twofold: first, to estimate the effect size and significance adjusted for other identified influential predictors and interactions; and second, to investigate whether the addition of other predictors improved the goodness of fit and accuracy of prediction.
Only complete cases were analysed since missingness was minimal: 10 records without data on comorbidities (details in footnote in table 1).
For each measure of deterioration (acuity increase, deterioration event and length of stay), logistic regression models were compared for the following sets of predictor variables:
Predictor set a. Comparator (base case): NEWS on admission.
Predictor set b. Primary analysis: NEWS, MR-proADM.
Predictor set c. Secondary analyses: NEWS and MR-proADM always included. Age, gender, CRP, WBC, presence of COPD or HF, presence of other comorbidities and interactions between predictors when appropriate.
Predictors (and the underlying assumption of linearity of their relationship with the outcome of interest) were initially investigated through univariate analyses based on simple log and quadratic functions. Transformations were applied if they improved the goodness of fit as assessed by the Akaike information criterion (AIC) and were retained in the multivariable setting. NEWS was treated as an ordinal variable. We assessed interactions through visual data exploration without significance testing as the study was not powered for this. For the multivariable regression models, the set of predictors was assessed for independence through backward elimination, based on changes in AIC.
The analysis plan for the secondary outcome of length of stay was similar: using multiple linear regressions based on transformed outcomes to address non-normality. Dependent and exploratory variables were log-transformed if not normally distributed. Normality was assessed by visualising the data. More details on the methods used are reported in online supplementary material 1.
bmjopen-2017-020337supp001.pdf (703.1KB, pdf)
Goodness of fit of logistic regression models was assessed with the C-statistic (which is the area under the receiver operating characteristic curve and is used as a measure of discrimination) presented with 95% CIs. To assess the value added by including the MR-proADM level with the NEWS in predicting deterioration, continuous net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were calculated.32 33
For internal validation of the statistical models, the C-statistic was evaluated after correcting for optimistic predictions through bootstrapping with 10 000 resamples.
Results
Study enrolment
The process of recruitment and enrolment of patients for the study is shown in figure 1. The study recruited 300 patients, and 292 were included in the analysis. Five patients were excluded because the blood samples for MR-proADM were taken more than 12 hours from baseline NEWS assessment; three patients were excluded from the primary outcome due to missing follow-up NEWSs.
Figure 1.
Patient recruitment process. AS, Assessment Suite, also called as Medical Admissions Unit (MAU); GP, general practitioner; MR-proADM, mid-regional pro-adrenomedullin; NEWS, National Early Warning Score.
Patient characteristics
Patient demographics and mean biomarker levels for each covariate are reported in table 1. The cohort was evenly divided in gender and had a mean age of 63 years and mean NEWS on admission of 3, with the majority of patients having NEWS of 2. COPD or HF was present in 28%, and 25% had other comorbidities.
The study population was homogeneous across acuity increase and no acuity increase outcomes in terms of gender, age and NEWS on admission. Table 2 shows the frequencies of criteria determining acuity increase and deterioration event. Notably, around 95% of acuity increases were the result of an increase in NEWS, while readmission was the reason for around 70% of deterioration events.
Table 2.
Criteria met by patients classified with an acuity increase or deterioration event
| Criterion for deterioration |
Acuity increase
(e=84) |
Deterioration event
(e2=32) |
| NEWS (n, %) | 81 (96.4%) | NA |
| ICU transfer (n, %) | 1 (1.2%) | 4 (12.5%) |
| Death (n, %) | 0 (0%) | 6 (18.8%) |
| Readmission (n, %) | 2 (2.4%) | 22 (68.7%) |
ICU, intensive care unit; NA, not applicable; NEWS, National Early Warning Score.
Patients who experienced acuity increase had higher MR-proADM and CRP levels at admission, and longer length of stay in the hospital and in the MAU.
The prevalence of acuity increase was 29% (somewhat higher than the anticipated 20%). The prevalence of deterioration events was 11%. The numbers of events provided sufficient statistical power to assess statistical significance for the primary outcome, acuity increase, but not for the secondary outcome, deterioration event.
Accuracy of MR-proADM for predicting acuity increase
Potentially useful predictors with univariate analysis of acuity increase were MR-proADM (OR 1.27, 95% CI 1.02 to 1.62; p=0.037), age2 (OR 1.00, 95% CI 0.99 to 1.00; p=0.023) and the presence of COPD or HF (OR 2.25, 95% CI 1.30 to 3.91; p=0.004; online supplementary figure S1). The prognostic accuracy of CRP, WBC and NEWS did not reach the threshold of significance (p=0.88, p=0.090, table 3, and p=0.416, table 4, respectively).
Table 3.
Univariate regression analyses for predicting the three outcomes of interest: acuity increase, deterioration event and length of stay
| Beta | CI | OR (CI) | P values | |
| Acuity increase: univariate logistic regressions (n=292, e=84) | ||||
| MR-proADM | 0.24 | −0.02 to 0.48 | 1.27 (1.02 to 1.62) | 0.037 |
| CRP | 0.003 | −0.0005 to 0.0063 | 1.00 (1.00 to 1.01) | 0.088 |
| WBC | 0.04 | −0.008 to 0.094 | 1.05 (1.00 to 1.10) | 0.09 |
| Gender | 0.14 | −0.38 to 0.65 | 1.15 (0.69 to 1.92) | 0.684 |
| Age | 0.1 | 0.019 to 0.1925 | 1.11 (1.02 to 1.21) | 0.023 |
| Age2 | −0.0008 | −0.0016 to 0.0001 | 1.00 (0.99 to 1.00) | |
| Other comorbidities | −0.32 | −0.96 to 0.28 | 0.72 (0.38 to 1.32) | 0.267 |
| COPD/HF* | 0.81 | 0.26 to 1.36 | 2.25 (1.30 to 3.91) | 0.004 |
| Deterioration event: univariate logistic regressions (n=292, e2=32) | ||||
| MR-proADM | 0.37 | 0.11 to 0.64 | 1.44 (1.12 to 1.90) | 0.006 |
| CRP | 0.003 | −0.002 to 0.01 | 1.00 (1.00 to 1.01) | 0.255 |
| WBC | 0.02 | −0.05 to 0.09 | 1.02 (0.95 to 1.10) | 0.506 |
| Gender | 0.17 | −0.57 to 0.92 | 1.19 (0.57 to 2.50) | 0.648 |
| Age | 0.21 | 0.06 to 0.40 | 1.23 (1.06 to 1.49) | 0.013 |
| Age2 | −0.002 | −0.003 to 0.001 | 1.00 (1.00 to 1.00) | |
| Other comorbidities | 1.14 | 0.38 to 1.90 | 3.14 (1.47 to 6.69) | 0.003 |
| COPD/HF* | 0.67 | −0.14 to 1.46 | 1.96 (0.87 to 4.29) | 0.095 |
| Length of stay: simple linear regressions (n=292, e=84, e2=32) | ||||
| MR-proADM | 0.7 | 0.49 to 0.92 | NA | <0.0001 |
| CRP | 0.05 | −0.05 to 0.15 | NA | 0.368 |
| WBC | −0.06 | −0.38 to 0.27 | NA | 0.73 |
| Gender | 0.08 | −0.04 to 0.20 | NA | 0.18 |
| Age | 0.007 | 0.004 to 0.010 | NA | <0.0001 |
| Other comorbidities | 0.18 | 0.05 to 0.32 | NA | 0.009 |
| COPD/HF* | 0.07 | −0.07 to 0.21 | NA | 0.318 |
The p values are for the statistical significance of the corresponding covariate in the related model. Analyses for the NEWS as a predictor are shown in Table 4.
*n=282, e=82, e2=29.
COPD, chronic obstructive pulmonary disease; CRP, C reactive protein; e, number of acuity increases; e2, number of deterioration events; HF, heart failure; MR-proADM, mid-regional pro-adrenomedullin; n, total number of cases; NA, not applicable; NEWS, National Early Warning Score; WBC, white blood cell.
Table 4.
Multivariable regression analyses for the outcomes of interest: acuity increase, deterioration event and length of stay (outcomes 1, 2 and 3, respectively) with NEWS comparator group
| Beta | CI | OR (CI) | P values | ||
| Acuity increase: multivariate logistic regressions | |||||
| Predictor set a
n=292 e=84 e2=32 |
NEWS 3 | 0.06 | −0.55 to 0.67 | 1.06 (0.57 to 1.95) | 0.416 |
| NEWS 4 | −0.48 | −1.29 to 0.27 | 0.62 (0.27 to 1.31) | ||
| NEWS 5 | 0.27 | −0.54 to 1.04 | 1.31 (0.58 to 2.84) | ||
| Predictor set b
n=292 e=84 e2=32 |
NEWS 3 | 0.03 | −0.59 to 0.65 | 1.03 (0.56 to 1.91) | 0.247 |
| NEWS 4 | −0.53 | −1.35 to 0.23 | 0.59 (0.26 to 1.26) | ||
| NEWS 5 | 0.18 | −0.63 to 0.97 | 1.20 (0.53 to 2.64) | ||
| MR-proADM | 0.24 | 0.02 to 0.49 | 1.28 (1.02 to 1.63) | 0.039 | |
| Predictor set c
n=282 e=82 e2=29 |
NEWS 3 | −0.11 | −0.76 to 0.54 | 0.90 (0.47 to 1.71) | 0.221 |
| NEWS 4 | −0.89 | −1.77 to 0.08 | 0.41 (0.17 to 0.93) | ||
| NEWS 5 | 0.09 | −0.77 to 0.91 | 1.09 (0.46 to 2.50) | ||
| MR-proADM | 0.41 | 0.13 to 0.76 | 1.51 (1.14 to 2.14) | 0.01 | |
| COPD/HF | 1.81 | 0.80 to 2.85 | 6.08 (2.23 to 17.35) | 0.001 | |
| MR-proADM×COPD/HF | −0.71 | −1.40 to 0.10 | 0.49 (0.25 to 0.91) | 0.03 | |
| Deterioration event: multivariate logistic regressions | |||||
| Predictor set a
n=292 e=84 e2=32 |
NEWS 3 | 0.03 | −0.92 to 0.94 | 1.03 (0.40 to 2.55) | 0.512 |
| NEWS 4 | −0.37 | −1.68 to 0.74 | 0.69 (0.19 to 2.10) | ||
| NEWS 5 | 0.7 | −0.36 to 1.70 | 2.02 (0.70 to 5.50) | ||
| Predictor set b
n=292 e=84 e2=32 |
NEWS 3 | −0.01 | −0.97 to 0.92 | 0.99 (0.38 to 2.51) | 0.564 |
| NEWS 4 | −0.43 | −1.76 to 0.70 | 0.65 (0.17 to 2.02) | ||
| NEWS 5 | 0.6 | −0.49 to 1.62 | 1.81 (0.61 to 5.05) | ||
| MR-proADM | 0.36 | 0.10 to 0.64 | 1.43 (1.11 to 1.89) | 0.007 | |
| Predictor set c
n=282 e=82 e2=29 |
NEWS 3 | 0.16 | −0.83 to 1.12 | 1.17 (0.44 to 3.07) | 0.389 |
| NEWS 4 | −0.49 | −1.86 to 0.69 | 0.62 (0.16 to 2.00) | ||
| NEWS 5 | 0.69 | −0.44 to 1.76 | 1.99 (0.64 to 5.81) | ||
| MR-proADM | 0.32 | 0.02 to 0.64 | 1.37 (1.02 to 1.89) | 0.044 | |
| Other comorbidities | 0.94 | 0.10 to 1.77 | 2.56 (1.10 to 5.85) | 0.026 | |
| Age | 0.21 | 0.06 to 0.41 | 1.23 (1.06 to 1.50) | 0.011 | |
| Age2 | −0.002 | −0.003 to 0.001 | 1.00 (1.00 to 1.00) | ||
| Length of stay: multiple linear regressions | |||||
| Predictor set a
n=292 e=84 e2=32 |
NEWS 3 | −0.07 | −0.21 to 0.08 | NA | 0.052 |
| NEWS 4 | 0.07 | −0.10 to 0.24 | NA | ||
| NEWS 5 | 0.21 | 0.01 to 0.40 | NA | ||
| Predictor set b
n=292 e=84 e2=32 |
NEWS 3 | −0.1 | −0.24 to 0.04 | NA | 0.033 |
| NEWS 4 | 0.05 | −0.11 to 0.21 | NA | ||
| NEWS 5 | 0.14 | −0.04 to 0.32 | NA | ||
| MR-proADM | 0.69 | 0.48 to 0.91 | NA | <0.0001 | |
| Predictor set c
n=282 e=82 e2=29 |
NEWS 3 | −0.12 | −0.25 to 0.02 | NA | 0.031 |
| NEWS 4 | 0.04 | −0.11 to 0.20 | NA | ||
| NEWS 5 | 0.14 | −0.04 to 0.32 | NA | ||
| MR-proADM | 0.55 | 0.31 to 0.80 | NA | <0.0001 | |
| Age | 0.004 | 0 to 0.007 | NA | 0.027 | |
Predictor set a includes only the NEWS as a predictor. Predictor set b includes MR-proADM and NEWSs. Predictor set c includes MR-proADM, NEWSs and other significant predictors and interactions. The p values are for the statistical significance of the corresponding covariate in the related model.
COPD, chronic obstructive pulmonary disease; HF, heart failure; MR-proADM, mid-regional pro-adrenomedullin; NA, not applicable; NEWS, National Early Warning Score.
The prognostic accuracy for acuity increase of NEWS on its own was limited and not significant (area under the curve (AUC) 0.55, 95% CI 0.48 to 0.62), but when MR-proADM was included as an additional predictor, the accuracy of the model increased (AUC 0.61, 95% CI 0.54 to 0.69; OR 1.28, 95% CI 1.02 to 1.63; p=0.039) (tables 4 and 5, figure 2A). When including MR-proADM with NEWS, the reclassification of patients was also significant, especially for the NRI (NRI 0.3, SE 0.1, p=0.007, IDI 0.017; table 4).
Table 5.
Model comparisons
| AIC | Deviance | AUC (CI) or R2 for linear regression |
LR (df), p value | NRI (SE), p value | IDI (SE), p value | |
| Acuity increase: logistic regressions | ||||||
| Outcome 1—predictor set a | 348 | 356 | 0.55 (0.48 to 0.62) | |||
| Outcome 1—predictor set b | 343 | 353 | 0.61 (0.54 to 0.69) | 5 (1), 0.033 | 0.3 (0.1), 0.007 | 0.017 (0.009), 0.058 |
| Outcome 1—predictor set c | 317 | 331 | 0.69 (0.63 to 0.76) | 14 (2), 0.001* | 0.4 (0.1), 0.0004* | 0.05 (0.01), 0.0009* |
| Deterioration event: logistic regressions | ||||||
| Outcome 2—predictor set a | 199 | 207 | 0.57 (0.47 to 0.68) | |||
| Outcome 2—predictor set b | 192 | 202 | 0.65 (0.54 to 0.76) | 7 (1), 0.007 | 0.4 (0.2), 0.003 | 0.04 (0.02), 0.10 |
| Outcome 2—predictor set c | 177 | 193 | 0.73 (0.63 to 0.84) | 15 (3), 0.0019* | 0.5 (0.2), 0.012* | 0.06 (0.02), 0.0004* |
| Length of stay: linear regressions (LR) | ||||||
| Outcome 3—predictor set a | 77 | −381 | 0.03 | |||
| Outcome 3—predictor set b | 68 | −417 | 0.14 | 9 (1), <0.001 | ||
| Outcome 3—predictor set c | 67 | −420 | 0.16 | 1 (1), 0.026 | ||
Outcomes 1, 2 and 3 refer to acuity increase, deterioration event and length of stay, respectively. The predictors are set a, NEWS alone; set b, NEWS and MR-proADM; and set c, NEWS, MR-proADM and other significant predictors and interactions detailed in Table 3.
*Comparison is between predictor set b and c. Since there was a mismatch between the cases for predictor set a and b (10 missing values in chronic obstructive pulmonary disease/heart failure), in the model with predictors set b, the 10 cases missing in predictor set c were dropped to allow the comparison.
AIC, Akaike information criterion; AUC, area under the receiver operating characteristic curve; IDI, integrated discrimination improvement; LR, likelihood ratio; MR-proADM, mid-regional pro-adrenomedullin; NEWS, National Early Warning Score; NRI, net reclassification improvement.
Figure 2.
(A) Prognostic accuracy for acuity increase. Predictor set a: NEWS; predictor set b: NEWS, MR-proADM; predictor set c: NEWS, MR-proADM, COPD/HF, interaction between MR-proADM and COPD/HF. (B) Comparisons as for panel (A) but for predicting a deterioration event. Predictor set c: NEWS, MR-proADM level, age2, other comorbidities. (C) Length of stay predicted by MR-proADM level. COPD, chronic obstructive pulmonary disease; HF, heart failure; MR-proADM, mid-regional pro-adrenomedullin; NEWS, National Early Warning Score.
The prognostic accuracy of MR-proADM and the additional value it provides to the NEWS was confirmed for deterioration events and length of stay (tables 4 and 5, and figure 2B,C).
For MR-proADM alone, the AUCs were for acuity increase 0.58 (0.51 to 0.66) and deterioration event 0.64 (0.54 to 0.74). For length of stay, the R2 was 0.12.
Effect on prognostic accuracy when clinical information is added to the set of predictors
Secondary multivariable modelling evaluated the prognostic accuracy of MR-proADM when adjusted for the clinical factors in predictive set c: age, gender, CRP, WBC, presence of COPD or HF, and presence of other comorbidities,
For acuity increase, COPD or HF comorbidity status and its interaction with MR-proADM level improved the prognostic accuracy of the model: AUC increased from 0.61 (95% CI 0.54 to 0.69) to 0.69 (95% CI 0.63 to 0.76) and net reclassification index from 0.3 to 0.4 (table 5).
For deterioration events, the presence of other comorbidities (excluding COPD and HF) and age2 increased the prognostic accuracy of MR-proADM (tables 4 and 5). The prognostic accuracy of length of stay (outcome 3) of MR-proADM is also increased by including age in the model (tables 4 and 5, online supplementary figure S2).
Potential confounding effects
Shorter-term outcomes
NEWS and MR-proADM were less accurate in predicting acuity increase within 24 and 12 hours from admission than in predicting acuity increase within 72 hours (online supplementary table S1 and S2).
Interval between admission NEWS scoring and blood collection
Because ward processes did not allow the times of scoring NEWS and collecting blood to be specified for research, we assessed for a confounding effect from variation in the timings, but found no evidence for it (online supplementary table S3).
Correlations among biomarkers
Diagnostic plots, shown in online supplementary figure S1 and S3, show no multicollinearity in the data, no autocorrelation, no heteroscedasticity and no data points that stood out in terms of their influence on results.
Sensitivity and specificity
As overall measures of accuracy, sensitivity and specificity were calculated (where appropriate) for each model using Youden’s index. The results are shown in online supplementary table S4. In practice, the trade-off between sensitivity and specificity would depend on the type of clinical decision to be made on the result (ie, ‘rule-in’ or ‘rule out’) and this would differ from the approach in Youden’s Index, which gives equal weight to false-positive and false-negative results.
Internal validation
C-statistic values after correcting for optimistic predictions (ie, bootstrapped average of the AUC for each model) were as follows: for acuity increase—predictor set a, C-stat 0.53; predictor set b, C-stat 0.59; predictor set c, C-stat 0.66; for deterioration events—predictor set a, C-stat 0.52; predictor set b, C-stat 0.61; predictor set c, C-stat 0.68; for length of stay— predictor set a, R2=0.003; predictor set b, R2=0.12; predictor set c, R2=0.13. AUCs decreased slightly with the bootstrapped averages, but the differences between the AUCs for predictor sets a, b and c were constant. These results are an internal validation, and further validation on an external dataset is required.
Discussion
Accuracy of prediction of deterioration by MR-proADM
This study shows that MR-proADM may be a clinically useful biomarker for predicting deterioration (ie, acuity increase) within 72 hours from admission to hospital in mild to moderately ill patients with admission NEWS between 2 and 5. By design, NEWSs in this range imply a low risk of deterioration, and our data are consistent with this. Previous evaluations of the NEWS assessed on admission have found that it predicts deterioration,3–5 34 which may seem inconsistent. But these studies included all patients admitted to ED, whatever their NEWS.
For most of the observed acuity increase events, the reason for classification was an increase in the NEWS. Because an increase in NEWS reflects both measurement variation and physiological variation, additional exploratory analyses were carried out to assess the performance of MR-proADM, using an operational definition of deterioration, deterioration event, designed to minimise measurement variation. NEWS on its own had low prognostic accuracy for deterioration events. However, MR-proADM level and NEWS together predicted deterioration events with an AUC of 0.65. Considering baseline patient characteristics further increased the accuracy of the model (AUC=0.73).
Comorbidities and interactions with MR-proADM levels
MR-proADM levels in people with COPD and/or HF are chronically raised and are not predictive of deterioration. However, in other people whose MR-proADM levels are not chronically raised, high levels are predictive of acuity increase (online supplementary figure S1). Including these comorbidities and their interaction with MR-proADM level increased the prognostic accuracy of the logistic regression model.
Limitations
This study included only patients who were admitted with a NEWS between 2 and 5. The prognostic accuracy of the MR-proADM would perhaps have been greater if more extreme cases had been included. However, patients with NEWSs more than 5 are already known to be severely ill and to require close monitoring and/or management at higher levels of care.
Internal validation found that the uncorrected C-statistics are optimistic, which implies that external validation in an independent study would be useful. However, after correction for optimistic predictions, the study’s conclusions remain unchanged.
Interpretations and implications
The contributions of MR-proADM to the accuracy of the prognostic models suggests that it could provide additional prognostic information over and above NEWS.
Secondary analyses suggest that a potentially useful clinical decision aid could be based on the NEWS, MR-proADM level and clinical features.
Future research and development
As a growing number of NHS hospitals are implementing the NEWS on their clinical information systems, it should be practical to develop a decision aid based on admission NEWS, MR-proADM level and clinical features. Other biomarkers may further improve prognostic accuracy for deterioration, for example, lactate,3 peroxiredoxin-4 and copeptin,22 35 36 and soluble urokinase plasminogen activator receptor.37 The feasibility, cost-effectiveness and acceptability of such decision aids needs to be evaluated in further research.
A rapid point-of-care test for MR-proADM could facilitate the assessment process and reduce delays in arranging optimal levels of care and intensity of monitoring. Future research could identify the threshold MR-proADM level corresponding to the optimal combination of sensitivity and specificity for a binary test (eg, ‘present’ or ‘absent’) for deterioration.
Supplementary Material
Acknowledgments
The authors would like to thank the patients who accepted to be part of the study and the team of research nurses for the data collection: Laura Shewan, Louise Taylor, Janine Gradwell, Karen Martin, Katherine Cullen, Gerry Jones, Graham Soulsby, Carmen Scott and Helen Reed. We also thank the reviewers for their helpful suggestions and comments.
Footnotes
Contributors: AJS, DAP, MP and DDS devised the study. RAO, SG and AJA managed the project. SG performed the statistical analyses with advice from DS. All authors contributed to the final manuscript.
Funding: The study was funded by B·R·A·H·M·S GmbH. The NIHR funds the NIHR Newcastle In Vitro Diagnostics Co-operative.
Disclaimer: The views expressed are those of the authors and not necessarily those of B·R·A·H·M·S GmbH, the NHS, the NIHR or the Department of Health.
Competing interests: None declared.
Patient consent: Obtained.
Ethics approval: The study was approved by the Newcastle and North Tyneside Research Ethics Committee (15/NE/0120), and by the R&D Committee of the Newcastle upon Tyne Hospitals NHS Foundation Trust (reference no. 7495).
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: No additional data are available.
References
- 1. Royal College of Physicians of London. National Early Warning Score (NEWS): standardising the assessment of acute-illness severity in the NHS—report of a working party. London: Royal College of Physicians of London, 2012. [Google Scholar]
- 2. McGinley A, Pearse RM. A national early warning score for acutely ill patients. BMJ 2012;345:e5310 10.1136/bmj.e5310 [DOI] [PubMed] [Google Scholar]
- 3. Jo S, Yoon J, Lee JB, et al. . Predictive value of the National Early Warning Score-Lactate for mortality and the need for critical care among general emergency department patients. J Crit Care 2016;36:60–8. 10.1016/j.jcrc.2016.06.016 [DOI] [PubMed] [Google Scholar]
- 4. Smith GB, Prytherch DR, Meredith P, et al. . The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation 2013;84:465–70. 10.1016/j.resuscitation.2012.12.016 [DOI] [PubMed] [Google Scholar]
- 5. Smith ME, Chiovaro JC, O’Neil M, et al. . Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Ann Am Thorac Soc 2014;11:1454–65. 10.1513/AnnalsATS.201403-102OC [DOI] [PubMed] [Google Scholar]
- 6. Teasdale GM. National Early Warning Score (NEWS) is not suitable for all patients. BMJ 2012;345:e587:e5875 10.1136/bmj.e5875 [DOI] [PubMed] [Google Scholar]
- 7. Eccles SR, Subbe C, Hancock D, et al. . CREWS: improving specificity whilst maintaining sensitivity of the National Early Warning Score in patients with chronic hypoxaemia. Resuscitation 2014;85:109–11. 10.1016/j.resuscitation.2013.08.277 [DOI] [PubMed] [Google Scholar]
- 8. Badriyah T, Briggs JS, Meredith P, et al. . Decision-tree early warning score (DTEWS) validates the design of the National Early Warning Score (NEWS). Resuscitation 2014;85:418–23. 10.1016/j.resuscitation.2013.12.011 [DOI] [PubMed] [Google Scholar]
- 9. Abbott TE, Vaid N, Ip D, et al. . A single-centre observational cohort study of admission National Early Warning Score (NEWS). Resuscitation 2015;92:89–93. 10.1016/j.resuscitation.2015.04.020 [DOI] [PubMed] [Google Scholar]
- 10. Corfield AR, Lees F, Zealley I, et al. . Utility of a single early warning score in patients with sepsis in the emergency department. Emerg Med J 2014;31:482–7. 10.1136/emermed-2012-202186 [DOI] [PubMed] [Google Scholar]
- 11. Bilben B, Grandal L, Søvik S. National Early Warning Score (NEWS) as an emergency department predictor of disease severity and 90-day survival in the acutely dyspneic patient—a prospective observational study. Scand J Trauma Resusc Emerg Med 2016;24:80 10.1186/s13049-016-0273-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Valenzuela-Sánchez F, Valenzuela-Méndez B, Rodríguez-Gutiérrez JF, et al. . New role of biomarkers: mid-regional pro-adrenomedullin, the biomarker of organ failure. Ann Transl Med 2016;4:329 10.21037/atm.2016.08.65 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Stokes NR, Dietz BW, Liang JJ. Cardiopulmonary laboratory biomarkers in the evaluation of acute dyspnea. Open Access Emerg Med 2016;8:35–45. 10.2147/OAEM.S71446 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Sinning C, Zengin E, Zeller T, et al. . Candidate biomarkers in heart failure with reduced and preserved ejection fraction. Biomarkers 2015;20:258–65. 10.3109/1354750X.2015.1068856 [DOI] [PubMed] [Google Scholar]
- 15. Kutz A, Hausfater P, Amin D, et al. . The TRIAGE-ProADM Score for an early risk stratification of medical patients in the emergency department—development based on a multi-national, prospective, observational study. PLoS One 2016;11:e0168076 10.1371/journal.pone.0168076 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Albrich WC, Dusemund F, Rüegger K, et al. . Enhancement of CURB65 score with proadrenomedullin (CURB65-A) for outcome prediction in lower respiratory tract infections: derivation of a clinical algorithm. BMC Infect Dis 2011;11:112 10.1186/1471-2334-11-112 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. López J, Martínez A. Cell and molecular biology of the multifunctional peptide, adrenomedullin. Int Rev Cytol 2002;221:1–92. [DOI] [PubMed] [Google Scholar]
- 18. Serrano-Ponz M, Rodrigo-Gasqué C, Siles E, et al. . Temporal profiles of blood pressure, circulating nitric oxide, and adrenomedullin as predictors of clinical outcome in acute ischemic stroke patients. Mol Med Rep 2016;13:3724–34. 10.3892/mmr.2016.5001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Zudaire E, Portal-Núñez S, Cuttitta F. The central role of adrenomedullin in host defense. J Leukoc Biol 2006;80:237–44. 10.1189/jlb.0206123 [DOI] [PubMed] [Google Scholar]
- 20. Zuur-Telgen MC, Brusse-Keizer MG, VanderValk PD, et al. . Stable-state midrange-proadrenomedullin level is a strong predictor of mortality in patients with COPD. Chest 2014;145:534–41. 10.1378/chest.13-1063 [DOI] [PubMed] [Google Scholar]
- 21. Pedowska-Włoszek J, Kostrubiec M, Kurnicka K, et al. . Midregional proadrenomedullin (MR-proADM) in the risk stratification of patients with acute pulmonary embolism. Thromb Res 2013;132:506–10. 10.1016/j.thromres.2013.08.008 [DOI] [PubMed] [Google Scholar]
- 22. Nickel CH, Messmer AS, Geigy N, et al. . Stress markers predict mortality in patients with nonspecific complaints presenting to the emergency department and may be a useful risk stratification tool to support disposition planning. Acad Emerg Med 2013;20:670–9. 10.1111/acem.12172 [DOI] [PubMed] [Google Scholar]
- 23. Landman GW, van Dijk PR, Drion I, et al. . Midregional fragment of proadrenomedullin, new-onset albuminuria, and cardiovascular and all-cause mortality in patients with type 2 diabetes (ZODIAC-30). Diabetes Care 2014;37:839–45. 10.2337/dc13-1852 [DOI] [PubMed] [Google Scholar]
- 24. Funke-Kaiser A, Mann K, Colquhoun D, et al. . Midregional proadrenomedullin and its change predicts recurrent major coronary events and heart failure in stable coronary heart disease patients: the LIPID study. Int J Cardiol 2014;172:411–8. 10.1016/j.ijcard.2014.01.064 [DOI] [PubMed] [Google Scholar]
- 25. Gaggin HK, Januzzi JL. Biomarkers and diagnostics in heart failure. Biochim Biophys Acta 2013;1832:2442–50. 10.1016/j.bbadis.2012.12.014 [DOI] [PubMed] [Google Scholar]
- 26. Vigué B, Leblanc PE, Moati F, et al. . Mid-regional pro-adrenomedullin (MR-proADM), a marker of positive fluid balance in critically ill patients: results of the ENVOL study. Crit Care 2016;20:363 10.1186/s13054-016-1540-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Tzikas S, Keller T, Ojeda FM, et al. . MR-proANP and MR-proADM for risk stratification of patients with acute chest pain. Heart 2013;99:388–95. 10.1136/heartjnl-2012-302956 [DOI] [PubMed] [Google Scholar]
- 28. Peduzzi P, Concato J, Kemper E, et al. . A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996;49:1373–9. 10.1016/S0895-4356(96)00236-3 [DOI] [PubMed] [Google Scholar]
- 29. van Smeden M, de Groot JA, Moons KG, et al. . No rationale for 1 variable per 10 events criterion for binary logistic regression analysis. BMC Med Res Methodol 2016;16:163 10.1186/s12874-016-0267-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Travaglino F, Russo V, De Berardinis B, et al. . Thirty and ninety days mortality predictive value of admission and in-hospital procalcitonin and mid-regional pro-adrenomedullin testing in patients with dyspnea. Results from the VERyfing DYspnea trial. Am J Emerg Med 2014;32:334–41. 10.1016/j.ajem.2013.12.045 [DOI] [PubMed] [Google Scholar]
- 31. R: A language and environment for statistical computing. R development core team. Vienna, Austria.: R Foundation for Statistical Computing, 2011. [Google Scholar]
- 32. Pencina MJ, D’Agostino RB, D’Agostino RB, et al. . Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27:157–72. 10.1002/sim.2929 [DOI] [PubMed] [Google Scholar]
- 33. Steyerberg EW, Vedder MM, Leening MJ, et al. . Graphical assessment of incremental value of novel markers in prediction models: from statistical to decision analytical perspectives. Biom J 2015;57:556–70. 10.1002/bimj.201300260 [DOI] [PubMed] [Google Scholar]
- 34. Alam N, Vegting IL, Houben E, et al. . Exploring the performance of the National Early Warning Score (NEWS) in a European emergency department. Resuscitation 2015;90:111–5. 10.1016/j.resuscitation.2015.02.011 [DOI] [PubMed] [Google Scholar]
- 35. Nickel CH, Ruedinger J, Misch F, et al. . Copeptin and peroxiredoxin-4 independently predict mortality in patients with nonspecific complaints presenting to the emergency department. Acad Emerg Med 2011;18:851–9. 10.1111/j.1553-2712.2011.01126.x [DOI] [PubMed] [Google Scholar]
- 36. Iversen K, Gøtze JP, Dalsgaard M, et al. . Risk stratification in emergency patients by copeptin. BMC Med 2014;12:80 10.1186/1741-7015-12-80 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Rasmussen LJ, Ladelund S, Haupt TH, et al. . Soluble urokinase plasminogen activator receptor (suPAR) in acute care: a strong marker of disease presence and severity, readmission and mortality. A retrospective cohort study. Emerg Med J 2016;33:769–75. 10.1136/emermed-2015-205444 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2017-020337supp001.pdf (703.1KB, pdf)