Skip to main content
NCBI home page
PLOS One logoLink to PLOS One
. 2022 Nov 18;17(11):e0277850. doi: 10.1371/journal.pone.0277850

Evaluation of the added value of Brain Natriuretic Peptide to a validated mortality risk-prediction model in older people using a standardised international clinical assessment tool

John W Pickering 1,2, Richard Scrase 3, Richard Troughton 2, Hamish A Jamieson 1,*
Editor: Alonso Soto4
PMCID: PMC9674136  PMID: 36399481

Abstract

The ability to accurately predict the one-year survival of older adults is challenging for clinicians as they endeavour to provide the most appropriate care. Standardised clinical needs assessments are routine in many countries and some enable application of mortality prediction models. The added value of blood biomarkers to these models is largely unknown. We undertook a proof of concept study to assess if adding biomarkers to needs assessments is of value. Assessment of the incremental value of a blood biomarker, Brain Naturetic Peptide (BNP), to a one year mortality risk prediction model, RiskOP, previously developed from data from the international interRAI-HomeCare (interRAI-HC) needs assessment. Participants were aged ≥65 years and had completed an interRAI-HC assessment between 1 January 2013 and 21 August 2021 in Canterbury, New Zealand. Inclusion criteria was a BNP test within 90 days of the date of interRAI-HC assessment. The primary outcome was one-year mortality. Incremental value was assessed by change in Area Under the Receiver Operating Characteristic Curve (AUC) and Brier Skill, and the calibration of the final model. Of 14,713 individuals with an interRAI-HC assessment 1,537 had a BNP within 90 days preceding the assessment and all data necessary for RiskOP. 553 (36.0%) died within 1-year. The mean age was 82.6 years. Adding BNP improved the overall AUC by 0.015 (95% CI:0.004 to 0.028) and improved predictability by 1.9% (0.26% to 3.4%). In those with no Congestive Heart Failure the improvements were 0.029 (0.004 to 0.057) and 4.0% (0.68% to 7.6%). Adding a biomarker to a risk model based on standardised needs assessment of older people improved prediction of 1-year mortality. BNP added value to a risk prediction model based on the interRAI-HC assessment in those patients without a diagnosis of congestive heart failure.

Introduction

At a global level the population is ageing and western countries in particular are doing so with increasing levels of comorbidities and complex health conditions [1]. To be able to provide timely and appropriate care for older individuals with complex conditions, clinicians need reliable mortality prediction tools to aid both clinical decision making and inform conversations with patients and family. In this way, wherever possible, common understanding can be reached about appropriate interventions and treatment options including making comprehensive future health care plans [2].

Reliable and accurate mortality prediction can be of significant value to clinicians when considering the risks and benefits of specific medications or when formulating individualised care plans for older adults with considerable variation in health status and disability [3–5]. The development of mortality tools which are more easily applied in the real-world environment and which can provide clinically meaningful information to guide mortality assessment have been facilitated by the evolution of standardised assessment tools and the accompanying big data [6]. The international Residential Assessment Instrument (interRAI) is one such suite of standardised assessment tools containing questions around demographics, function, comorbidities, and living conditions, and which is now used in over 35 countries and in a multitude of clinical settings [7,8]. Each tool is specific to a setting, e.g. long-term care facilities, or discipline e.g mental health, although a large proportion of the questions are identical. This enables monitoring of changes of status between assessments as the patient moves through their health journey and life course [9]. Although the primary purpose of any interRAI assessment is to provide a standardised assessment to create individualised care plans, they also provide a comprehensive dataset at a population level which can be valuable to aid research aimed at improving outcomes for our most vulnerable older population [10].

The interRAI-Home Care (interRAI-HC) is mandated for use throughout all of New Zealand with older adults requiring assessment for Government funded support or who are being considered for possible entry into aged residential care [8]. This means the individuals being assessed tend to have complex co-morbidities and at a population level are frailer and more vulnerable than their peers. This clinically vulnerable group is, therefore, the exact older cohort clinicians would most want to have a greater understanding of their morbidity trajectory, so that they can more appropriately support with care better tailored to the individual.

The authors recently developed and validated a mortality risk model (RiskOP) for those people aged 65 plus using data from the interRAI-HC only [11]. The RiskOp model predicts one-year mortality with good discrimination and excellent calibration.

The broad context for this research is the question of whether individual biomarkers, not currently part of most needs assessments, can improve risk prediction models that are based on standardised assessments in older adults. To answer this question, we undertook a proof of concept using a validated prediction model with good discrimination and calibration (therefore with less room for improvement than a poor one), RiskOp and with a biomarker associated with mortality across a variety of clinical settings, Brain Natriuretic Peptide (BNP) [12,13]. In western countries in particular heart failure is a leading cause of death among the older population [14]. BNP is a good predictor of mortality in adults both with and without a prior diagnosis of heart failure [13,15,16]. We also chose BNP because it is in frequent clinical use, it is inexpensive and it is clinically accessible. We hypothesised that BNP may add useful prognostic information to the RiskOP one-year mortality risk prediction model for older adults.

Methods

Design

We assessed the added value of BNP to the RiskOP one-year mortality prediction model which uses the interRAI-HC instrument [11]. The interRAI-HC is administered to older people to assess their need for home support or for entry to a long-term care facility. The inteRAI -HC assessment tool is administered by a trained health professional who is usually either a Registered Nurse or a Social Worker. These health professionals are required to attend a training course and are then need to complete yearly updates to ensure that the quality of their assessments is maintained. These assessments consist of over 200 questions across 20 domains asked of the patient and sometimes their care givers. Data quality and completeness has been previously demonstrated [8]. RiskOP uses the answer to 16 questions concerning Age, Sex, Body Mass Index, Cancer, Congestive Heart Failure (CHF), Previous Stroke, Parkinsons, Dyspnea, Weight loss, Fatigue, Oxygen therapy, Skin ulcer, frequency of going out, exercise, time since last hospital stay and change in activities of daily living.

Participants

Participants included all people aged ≥65 years who completed an interRAI-HC assessment between 1 January 2013 and 21 August 2020 in Canterbury, New Zealand, and who had given written consent for their data to be used for research. Previous research has demonstrated 93% of all those who are assessed give this consent [8]. The RiskOP model had been developed in a subset of these patients (up to March 2018) [11]. Ethical approval was provided by the New Zealand Ministry of Health and Disability Ethics Committee (HDEC) (14/STH/140/AM08).

Inclusion criteria was a single BNP test within 90 days of the date of interRAI-HC assessment. Exclusion criteria was those identified as being on a palliative pathway. This was identified by a question which asked if they had been told that in the best clinical judgment of the physician the individual has end stage disease with approximately 6 or fewer months to live.

The interRAI-HC assessment data and BNP test results was provided by the Canterbury District Health Board. The National Mortality Collection Register administered by the New Zealand Ministry of Health provided the dates of deaths. Linkage between the data sets was made by encrypted national health index numbers as every person who has ever interacted with the public health system in New Zealand has a unique number. All participants were followed for a minimum of 12 months or until death.

Subgroups

Participants with and without a diagnosis of Congestive Heart Failure separately.

Outcomes

The primary outcome was one-year mortality.

Statistical analysis

Data are presented as n (%) for categorical variables, mean and standard deviation for normally distributed quantitative variables and median (lower quartile and upper quartile) for non-normally distributed variables. All confidence intervals are 95% calculated using bootstrapping.

The RiskOP model was applied to each patient to obtain a prediction of one year mortality and recalibrated for the local data. This is referred to as the baseline model. A logistic regression model containing the predictions from the Baseline model, the log-base-2 transformed BNP concentrations (as a continuous variable), and the time from BNP measurement to interRAI-HC assessment was constructed and one-year mortality predictions calculated. This was called the new model.

The added value of BNP to RiskOP was assessed by the relative change in Brier score (Brier skill), difference in area under the receiver operator characteristic curve (δAUC), Integrated Discrimination Improvement (IDI) for those who died (IDIevent) and those who did not die separately (IDInon-event), risk assessment plots (RAP), calibration plots and decision curves. The Brier score is a measure of the variation in risk prediction from the actual outcomes and the Brier skill measures the improvement in this measurement with the addition of a biomarker [17]. The AUC is a measure of discrimination and is the probability that if we were to draw at random a person who died and a person who survived that the risk prediction for the person who died would be greater than for the person who survived. An AUC of 0.5 suggests no discrimination. An AUC of 1 means perfect discrimination. The IDIevent represents the mean increase in predicted risk for those who had the event (those who died) whereas the IDInon-event represents the mean decrease in predicted risk for those who did not have the event [18]. RAPs plot Sensitivity verse risk prediction and 1-Specificity verse risk prediction [19]. Improved performance with the addition of a biomarker would be observed by increased separation of the curves. Calibration plots divide the data into quantiles and plot the mean predicted risk verses the actual risk (nevent/n) within each quantile. Decision curves enable the assessment of the additional net benefit of BNP at specific risk thresholds of relevance to the clinician and patient taking into account both benefits and harms of a proposed treatment [20]. In this case the proposed treatment is simply telling the person that they are at risk of dying in one year, and the benefit of being correct is weighted the same as the harm of being incorrect. The prediction threshold on the x-axis is the threshold of predicted risk above which each person is classified as high risk (in this case high risk of dying in one year) and below which they are classified as low risk. The net benefit is true positives. It is maximal and equal to the prevalence at a prediction threshold of zero. A net benefit of say, 0.1, in this case is equivalent to a strategy that correctly identifies as dying in one year 10 out of 100 at risk people. The higher a decision curve is, the better it performs at identifying true positives.

Results

There were 18,720 interRAI-HC assessments from 14,713 individuals in the data set. Of these individuals 6,899 (46.9%) had a BNP test within the period of which 1,690 (24.5%) were within 90 days preceding the assessment and had at least one year follow-up, (Fig 1). 1537 individuals had all the required data to calculate one-year mortality predictions with the RiskOP model. The mean age was 82.6 years and 54.2% were female (Table 1). Of these 785 had CHF of whom 324 (41.3%) died within a year. Within those with CHF the median (IQR) BNP was 139 (58–297) pg/mL. Of the 752 without CHF 229 (30.5%) died within one year. Within those without CHF the median (IQR) BNP was 57 (23–130) pg/mL.

Fig 1. Consort diagram.

Fig 1

Open in a new tab

Table 1. Demographics.

Variable Survived
(N = 984)		 Died 1
(N = 553)		 Total
(N = 1537)
Age years 82.0 (8.3) 83.5 (8.2) 82.6 (8.3)
Sex Female 578 (58.4%) 258 (46.7%) 833 (54.2%)
Ethnicity Māori 16 (1.6%) 21 (3.8%) 37 (2.4%)
Pacific Peoples 10 (1.0%) 8 (1.4%) 18 (1.2%)
Asian 4 (0.4%) 2 (0.4%) 6 (0.4%)
European 773 (78.6%) 428 (77.4%) 1201 (78.1%)
Other/Unknown/refused to answer/Not Stated 181 (18.4%) 94 (17.0%) 275 (17.9%)
Scale BMI Normal 304 (30.9%) 209 (37.8%) 513 (33.4%)
Underweight 66 (6.7%) 63 (11.4%) 129 (8.4%)
Overweight 212 (21.5%) 108 (19.5%) 320 (20.8%)
Obese 200 (20.3%) 66 (11.9%) 266 (17.3%)
Unknown 202 (20.5%) 107 (19.3%) 309 (20.1%)
Cancer Diagnosed 129 (13.1%) 105 (19.0%) 234 (15.2%)
Congestive Heart Failure Diagnosed 461 (46.8%) 324 (58.6%) 785 (51.1%)
Previous stroke Diagnosed 196 (19.9%) 101 (18.3%) 297 (19.3%)
Parkinsons Diagnosed 36 (3.7%) 12 (2.2%) 48 (3.1%)
Dyspnaea No 292 (29.7%) 107 (19.3%) 399 (26.0%)
  Yes-moderate 277 (28.2%) 131 (23.7%) 408 (26.5%)
  Yes-normally 279 (28.4%) 176 (31.8%) 455 (29.6%)
  Yes-at rest 136 (13.8%) 139 (25.1%) 275 (17.9%)
Weight loss 149 (15.1%) 138 (25.0%) 287 (18.7%)
Fatigue None 122 (12.4%) 34 (6.1%) 156 (10.1%)
  Minimal 365 (37.1%) 128 (23.1%) 493 (32.1%)
  Moderate 348 (35.4%) 217 (39.2%) 565 (36.8%)
  Severe+ 149 (15.1%) 174 (31.5%) 323 (21.0%)
Days went out in last 3 days None 276 (28.0%) 255 (46.1%) 531 (34.5%)
Usually 73 (7.4%) 45 (8.1%) 118 (7.7%)
From 1 to 2d 334 (33.9%) 157 (28.4%) 491 (31.9%)
Three 301 (30.6%) 96 (17.4%) 397 (25.8%)
Hours exercise daily None 189 (19.2%) 131 (23.7%) 320 (20.8%)
  Less than 1h 394 (40.0%) 241 (43.6%) 635 (41.3%)
  From 1 to 2h 315 (32.0%) 144 (26.0%) 459 (29.9%)
  More than 2h 86 (8.7%) 37 (6.7%) 123 (8.0%)
On Oxygen therapy Yes 35 (3.6%) 59 (10.7%) 94 (6.1%)
Days since last hospital stay None in last 90d 153 (14.2%) 52 (8.5%) 205 (12.1%)
  From 31 to 90d 318 (29.5%) 128 (20.9%) 446 (26.4%)
  From 8 to 30d 285 (26.4%) 158 (25.8%) 443 (26.2%)
  From 0 to 7d 322 (29.9%) 274 (44.8%) 596 (35.3%)
Change in ADL status in last 6 months Improved or no change 455 (46.2%) 150 (27.1%) 605 (39.4%)
Declined 484 (49.2%) 371 (67.1%) 855 (55.6%)
Uncertain 45 (4.6%) 32 (5.8%) 77 (5.0%)
Open in a new tab

Overall there was a modest improvement in prediction with the addition of BNP, (Table 2).

Table 2. Discrimination metrics for prediction of mortality.

Metric ALL CHF No CHF
n 1537 785 752
n died 554 (519 to 588) 324 (299 to 352) 229 (205 to 255)
IDIevent 0.009 (0.003 to 0.015) -0.006 (-0.012 to -0.001) 0.041 (0.028 to 0.054)
IDInon-event 0.006 (0.002 to 0.01) 0.003 (-0.002 to 0.009) 0.014 (0.007 to 0.021)
Brier: Baseline 0.199 (0.19 to 0.208) 0.216 (0.204 to 0.229) 0.18 (0.166 to 0.195)
Brier: New 0.195 (0.185 to 0.204) 0.213 (0.201 to 0.224) 0.172 (0.158 to 0.188)
Brier skill (%) 1.9 (0.26 to 3.4) 1.6 (-0.03 to 3.4) 4.0 (0.68 to 7.6)
AUC: Baseline 0.716 (0.69 to 0.741) 0.689 (0.651 to 0.726) 0.725 (0.682 to 0.766)
AUC: New 0.731 (0.705 to 0.757) 0.699 (0.661 to 0.736) 0.754 (0.713 to 0.792)
δAUC 0.015 (0.004 to 0.028) 0.01 (-0.004 to 0.024) 0.029 (0.004 to 0.057)
Open in a new tab

All confidence intervals are 95%.

The AUC increased marginally (δAUC = 0.015) from 0.716 to 0.731, and the Brier skill of 1.9% suggests a reduction in overall prediction error. The mean prediction for those who died (IDIevent) increased marginally (0.9%) and the mean prediction for those who did not die (IDInonevent) decreased marginally (0.6%). In the cohort with CHF, BNP had minimal impact on mortality prediction (Figs 2 and 3. There was no apparent improvement in risk for those who did or who did not die in one year, (Fig 3B). The decision curve suggests a net benefit of the addition of BNP in the mid-prediction threshold range of 0.4 to 0.7, (Fig 4B). In contrast, in those without CHF BNP improved discrimination (δAUC = 0.029 (0.004 to 0.057)) with a final AUC of 0.75. There was no apparent improvement in risk for those who did or did not die in one year, (Fig 3C). The improvement was greatest for those who died with probabilities above about 0.3. The decision curve shows that the addition of BNP improves the application of the risk model amongst those with baseline risk up to 0.75, (Fig 4C). The new model remained well calibrated overall in each of the CHF and no CHF cohorts, (Fig 5).

Fig 2. ROC curves.

Fig 2

Open in a new tab

Sensitivity verse 1-Specificity. Dotted lines are baseline model RiskOP model, solid lines are the new model with addition of BNP.

Fig 3. Risk assessment plots.

Fig 3

Open in a new tab

Plots illustrating the change in predicted probabilities for participants who died (teal Sensitivity Metric curves) and did not die (red 1-Specificity Metric curves). The dotted line is the predicted probability curve of the baseline model, RiskOP. The solid line is the predicted probability curve with the addition of BNP to RiskOP (New model). Improved prediction is where the teal Sensitivity curve moves towards the upper right corner (higher average risk) with the addition of BNP and/or where the red 1-Specificity curve moves towards the bottom left corner (lower average risk) with the addition of BNP. All patients included in panel A, with separate models for Congestive Heart Failure participants in panel B, and non-Congestive Heart Failure participants in panel C.

Fig 4. Decision curves.

Fig 4

Open in a new tab

At specific prediction of mortality thresholds the new model (teal curve) with the addition of BNP has higher net benefit to the baseline model (red curve). All patients included in panel A, with separate models for Congestive Heart Failure participants in panel B, and non-Congestive Heart Failure participants in panel C.

Fig 5. Calibration curves.

Fig 5

Open in a new tab

Actual percentage of people with outcomes (and 95% CI) verse predicted percentage (and 95%CI) for all patients split into 10 groups. Perfect calibration is the dashed line. The new model (teal curve) has similar or better calibration than the baseline model (red curve). All patients are included in panel A. Panel B is for participants with Congestive Heart Failure, and Panel C for participants without Congestive Heart Failure.

Discussion

The addition of BNP measurements to a one-year mortality risk prediction model designed for use with a standardised needs assessment for older people marginally added value overall. The additional value was in people without a diagnosis of CHF. More specifically, the study indicated it had additional prognostic information for clinicians when treating those patients without a diagnosis of CHF and with a risk prediction of below 0.75 (<75% probability of dying within 1-year).

Several previous studies have developed mortality risk prediction models utilising existing health data. These include a large UK study which included a patient questionnaire and biochemical tests [21]. In 2020 Canadian research which utilises an earlier version of the interRAI-HC, developed the RESPECT tool [22]. Similar to RiskOP, this model was developed in a large group of older adults (mean age 79.6) and had similar discriminatory performance. Given that the interRAI suite of tools are standardised and well established internationally, RESPECT or RiskOP could be expected to be transferrable to other countries using the interRAI-HC.

The large UK based QRISK3 study [6] focussed on future risk of cardiovascular disease. This study utilised the QResearch database at over 1300 General Practices throughout the UK. It included a blood test for high density lipoprotein cholesterol. Adjusted hazard ratios for cardiovascular disease in men was 1.19 (1.18 to 1.19; 95% CI) per unit increase of total cholesterol: HDL cholesterol ratio. It is a rare example of the value of standardised and connected data bases that support not just the individual but population health.

The utilisation of specific blood tests, such as BNP have long been identified as predicting mortality in older adults, including those without any known cardiovascular issues [13]. Indeed, the value of BNP as a predictor of mortality continues to be recognized [13,15,23], including most recently in relation to BNP mortality and its association with Covid-19 [24]. Furthermore, frailty, which is in itself a risk factor for mortality [25], was associated with an increased risk of elevated BNP in a recent study of over 1300 community dwelling elders with no previous cardiovascular history [26].

Given the value of BNP in terms of identifying both frailty and risk of mortality, the addition of this relatively low cost and easy to acquire biomarker to the RiskOP mortality model adds a modest effect to the accuracy of a comprehensive risk prediction, although its value is greater for those without a prior diagnosis of heart failure. That there is little additional prognostic value of BNP in those with heart failure may seem surprising. This could simply be that as the people with CHF are more likely to die the covariates in the RiskOP model better correlate with BNP meaning it has less affect on overall mortality prediction. While intriguing, further investigation is beyond the scope of this manuscript.

Like all assessments, and clinical outcome measures, the prediction models here are designed to be used as a valuable aid to clinical decision making but not as a replacement to skilled clinical judgement and of course individual patient choice which is an important part of any decision-making discussion. Added to this is the important cultural context when any health professional is preparing to discuss a specific patients clinical trajectory and the options available [27]. This group had low numbers of Māori (indigenous New Zealanders) and specific ethnic analysis was not viable. Further research with appropriate numbers of Māori is vital in creating a culturally appropriate model. Additionally, incorporating Haoura Māori values would be evaluated and included before considering the roll-out of any prediction model.

A limitation of this study is that the model has been developed in a national New Zealand based cohort of older adults and the results need to be validated in an international context. However, the advantage of using the interRAI-HC to develop the model is that interRAI-HC is used internationally with similarly trained assessors require ongoing review. This increases the likelihood that this model would validate externally. We also acknowledge that for practical reasons, the interRAI-HC assessments and the BNP blood tests were not usually completed at the same time in the patient journey. Ideally blood draws for biomarker testing would be cotemporaneous with the interRAI assessment. Given many of the interRAI assessors are trained nurses, this is practical. How this cotemporaneous measurement affects the accuracy of the new model would need further investigation. Additionally, while the RiskOP model incorporates Age, Sex, and Body Mass Index, other factors which may be associated with BNP such as inflammation, kidney function, and atrial fibrillation were not measured or included as covariates. If this was able to be done these may well improve the accuracy of the risk prediction. Further investigation would also need to assess performance in sex, age, and ethnicity sub-groups. Finally, the net benefit and decision curve analysis is merely a tool for understanding if BNP could be of added value in decision making. For it to be seen as suitable for clinical practice, a more comprehensive net-benefit analysis would be needed weighing both benefits and harms at each decision point, e.g. for risk stratifying patients to low, intermediate, or high risk.

The addition of a BNP blood test to the RiskOP mortality model improves its accuracy, particularly for those that do not have a diagnosis of congestive heart failure. This is a proof of concept that the addition of a biomarker to risk prediction from a standardised assessment can improve mortality prediction. Further research would need to be undertaken in order to ascertain the value of including additional commonly used blood tests for the older adult population such as sodium, creatinine, albumin, and C-reactive protein. This research would not only need to ascertain the statistical improvement in risk prediction, but also look at a cost-benefit analysis, and ideally would include calibration of models for specific groups including in New Zealand Māori and Pacific Peoples. Improved mortality prediction gives clinicians, patients and family more confidence that they have received the necessary information to make informed decisions about appropriate interventions and treatment options for an individual.

Data Availability

Because of restrictions of use of clinical data we cannot make the data set freely available. However, we can make it available if contacted subject to ethics approval from a New Zealand ethics board and a formal use agreement between ourselves and those who wish to use the data. For New Zealand data this is particularly important because some of the data is from Māori, the indigenous people of New Zealand, who are acknowledged to have sovereignty of their data and its use. The restrictions are imposed by the New Zealand Ministry of Health and Disability Ethics Committee (HDEC). Data access queries may be sent to the Department of Medicine, University of Otago Christchurch, Christchurch, New Zealand (contact via christchurch@otago.ac.nz).

Funding Statement

Health Research Council of New Zealand grant 17/363 The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Rechel B, Doyle Y, Grundy E, McKee M. How can health systems respond to population ageing? World Health Organization, on behalf of the European Observatory on Health Systems and Policies; 2009. p. 43. Report No.: 10. [Google Scholar]
  • 2.Kim P, Daly JM, Berry-Stoelzle MA, Schmidt ME, Michaels LC, Dorr DA, et al. Prognostic Indices for Advance Care Planning in Primary Care: A Scoping Review. J Am Board Fam Med. 2020;33: 322–338. doi: 10.3122/jabfm.2020.02.190173 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Kim NH, Cho HJ, Kim S, Seo JH, Lee HJ, Yu JH, et al. Predictive Mortality Index for Community-Dwelling Elderly Koreans. Medicine (Baltimore). 2016;95: e2696. doi: 10.1097/MD.0000000000002696 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Campbell AJ, Diep C, Reinken J, McCosh L. Factors predicting mortality in a total population sample of the elderly. J Epidemiol Community Health. 1985;39: 337–342. doi: 10.1136/jech.39.4.337 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64: 749–759. doi: 10.1016/j.jclinepi.2010.10.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Hippisley-Cox J, Coupland C, Brindle P. Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ. 2017; j2099. doi: 10.1136/bmj.j2099 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Hirdes JP, Ljunggren G, Morris JN, Frijters DH, Finne Soveri H, Gray L, et al. Reliability of the interRAI suite of assessment instruments: a 12-country study of an integrated health information system. BMC Health Serv Res. 2008;8: 434. doi: 10.1186/1472-6963-8-277 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Schluter PJ, Ahuriri-Driscoll A, Anderson TJ, Beere P, Brown J, Dalrymple-Alford J, et al. Comprehensive clinical assessment of home-based older persons within New Zealand: an epidemiological profile of a national cross-section. Aust N Z J Public Health. 2016;40: 349–355. doi: 10.1111/1753-6405.12525 [DOI] [PubMed] [Google Scholar]
  • 9.de Stampa M, Cerase V, Bagaragaza E, Lys E, Alitta Q, Gammelin C, et al. Implementation of a Standardized Comprehensive Assessment Tool in France: A Case Using the InterRAI Instruments. Int J Integr Care. 2018;18: 5. doi: 10.5334/ijic.3297 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Jamieson HA, Schluter PJ, Pyun J, Arnold T, Scrase R, Nisbet-Abey R, et al. Fecal Incontinence Is Associated With Mortality Among Older Adults With Complex Needs: An Observational Cohort Study. 2017;112: 1431–1437. doi: 10.1038/ajg.2017.200 [DOI] [PubMed] [Google Scholar]
  • 11.Pickering JW, Abey-Nesbit R, Allore H, Jamieson H. Development and validation of multivariable mortality risk-prediction models in older people undergoing an interRAI home-care assessment (RiskOP). EClinicalMedicine. 2020; 100614. doi: 10.1016/j.eclinm.2020.100614 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Wallen T, Landahl S, Hedner T, Nakao K, Saito Y. Brain natriuretic peptide predicts mortality in the elderly. Heart. 1997;77: 264–267. doi: 10.1136/hrt.77.3.264 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.York MK, Gupta DK, Reynolds CF, Farber-Eger E, Wells QS, Bachmann KN, et al. B-Type Natriuretic Peptide Levels and Mortality in Patients With and Without Heart Failure. J Am Coll Cardiol. 2018;71: 2079–2088. doi: 10.1016/j.jacc.2018.02.071 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Reilev M, Lundby C, Jensen J, Larsen SP, Hoffmann H, Pottegård A. Morbidity and mortality among older people admitted to nursing home. Age Ageing. 2020;49: 67–73. doi: 10.1093/ageing/afz136 [DOI] [PubMed] [Google Scholar]
  • 15.Doust J, Lehman R, Glasziou P. The Role of BNP Testing in Heart Failure. 2006;74: 6. [PubMed] [Google Scholar]
  • 16.Khanam SS, Son J-W, Lee J-W, Youn YJ, Yoon J, Lee S-H, et al. Prognostic value of short-term follow-up BNP in hospitalized patients with heart failure. BMC Cardiovasc Disord. 2017;17: 215. doi: 10.1186/s12872-017-0632-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiol Camb Mass. 2010;21: 128–138. doi: 10.1097/EDE.0b013e3181c30fb2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Pencina MJ, D’Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Stat Med. 2008;27: 157–172. doi: 10.1002/sim.2929 [DOI] [PubMed] [Google Scholar]
  • 19.Pickering JW, Endre ZH. New Metrics for Assessing Diagnostic Potential of Candidate Biomarkers. Clin J Am Soc Nephrol. 2012;7: 1355–1364. doi: 10.2215/CJN.09590911 [DOI] [PubMed] [Google Scholar]
  • 20.Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak Int J Soc Med Decis Mak. 2006;26: 565–574. doi: 10.1177/0272989X06295361 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Ganna A, Ingelsson E. 5 year mortality predictors in 498 103 UK Biobank participants: a prospective population-based study. The Lancet. 2015;386: 533–540. doi: 10.1016/S0140-6736(15)60175-1 [DOI] [PubMed] [Google Scholar]
  • 22.Hsu AT, Manuel DG, Spruin S, Bennett C, Taljaard M, Beach S, et al. Predicting death in home care users: derivation and validation of the Risk Evaluation for Support: Predictions for Elder-Life in the Community Tool (RESPECT). Can Med Assoc J. 2021;193: E997–E1005. doi: 10.1503/cmaj.200022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Voors AA, Ouwerkerk W, Zannad F, van Veldhuisen DJ, Samani NJ, Ponikowski P, et al. Development and validation of multivariable models to predict mortality and hospitalization in patients with heart failure: Mortality and hospitalization models in heart failure. Eur J Heart Fail. 2017;19: 627–634. doi: 10.1002/ejhf.785 [DOI] [PubMed] [Google Scholar]
  • 24.Qin J-J, Cheng X, Zhou F, Lei F, Akolkar G, Cai J, et al. Redefining Cardiac Biomarkers in Predicting Mortality of Inpatients With COVID-19. Hypertension. 2020;76: 1104–1112. doi: 10.1161/HYPERTENSIONAHA.120.15528 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Cornman JC, Glei DA, Goldman N, Weinstein M. Physiological Dysregulation, Frailty, and Risk of Mortality Among Older Adults. Res Aging. 2017;39: 911–933. doi: 10.1177/0164027516630794 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Wang Z-D, Yao S, Shi G-P, Wang Y, Shi J-M, Guo J-H, et al. Frailty index is associated with increased risk of elevated BNP in an elderly population: the Rugao Longevity and Ageing Study. Aging Clin Exp Res. 2020;32: 305–311. doi: 10.1007/s40520-019-01189-4 [DOI] [PubMed] [Google Scholar]
  • 27.DeSouza R. Wellness for all: the possibilities of cultural safety and cultural competence in New Zealand. J Res Nurs. 2008;13: 125–135. doi: 10.1177/1744987108088637 [DOI] [Google Scholar]

Decision Letter 0

Juan A López-Rodríguez

9 Jun 2022

PONE-D-22-10241Evaluation of the added value of Brain Natriuretic Peptide to a validated mortality risk-prediction model in older people using a standardised international clinical assessment toolPLOS ONE

Dear Dr. Pickering,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jul 24 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Juan A López-Rodríguez

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research.

3. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Evaluation of the added value of Brain Natriuretic Peptide to a validated mortality risk prediction model in older people using a standardised international clinical assessment tool

The authors explore the question if addition of the cardiac biomarker BNP adds prognostic value to a established mortality prediction score in elderly adults in NZ.

This is a well-designed study with a well-phenotyped cohort and fair sample size. Improving risk prediction in the elderly (mean age >80 in this cohort) is of crucial clinical importance for the patient, patient’s family and physician. The main findings of this study are (1) adding BNP to the RiskOP model significantly improved the accuracy of one-year mortality prediction, (2) especially in patients without a diagnosis of CHF. The study utilizes advanced statistical models.

Major

BNP or NT-proBNP is a valuable prognostic biomarker in a wide range of cardio-vascular diseases. BNP is influenced by age, BMI, inflammation, gender, atrial fibrillation and kidney function. The authors should adjust for those metrics before performing survival analyses.

The authors should explain how BNP levels were incorporated in the RiskOP Model. Were age appropriate thresholds used?

It would be helpful for the reader to report the actual BNP values of the cohorts.

The authors should explain why they chose BNP as a biomarker, why not serum sodium, albumin or CRP?

The mean age was >80 years old, individuals at this age have a very high change for diastolic dysfunction. Do the authors have access to echocardiograms in the CHF cohort? Is there a way to report the prevalence of Afib in both cohorts?

The authors should clarify when how the RiskOP model was derived, is this part of the interRAI assessment?

Are the authors suggesting that BNP should be part of a routine assessment in the elderly? Could they estimate a benefit-cost ratio?

Image quality of the figures needs to be improved

Minor

page 9, 69, sentence needs improved syntax

page 11, 102, doe the authors mean “informed” instead of “inform”?

page 12, 146, what do the authors mean by “cost effective”?

page 17, 236, this is confusing. So how many BNP measurements were included in the analysis?

Page 17, table 1 the column Died should have the figure 1 deleted

Page 19, 258 should read “died” not “die”

Page 19, 268, Fig2, should read “died” not “dies”

Page 20, 283 and 284, should read congestive heart failure

Page 20, 290, sentence needs improved syntax

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Nils Patrick Nickel

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Nov 18;17(11):e0277850. doi: 10.1371/journal.pone.0277850.r002

Author response to Decision Letter 0

8 Jul 2022

PONE-D-22-10241 Response to Reviewers

Evaluation of the added value of Brain Natriuretic Peptide to a validated mortality risk-prediction model in older people using a standardised international clinical assessment tool

Thank you Dr Nickel for your considered review. We have responded under each point below.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Response: Thank you.

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Response: As the first author, and an experienced biostatistican, I appreciate the honesty of this response.

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Response: Because of restrictions of use of clinical data we cannot make it freely available. However, we can make it available if contacted subject to ethics approval from a New Zealand ethics board and a formal use agreement between ourselves and those who wish to use the data. For New Zealand data this is particularly important because some of the data is from Māori, the indigenous people of New Zealand, who are acknowledged to have sovereignty of their data and its use.

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Response: Thank you.

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Evaluation of the added value of Brain Natriuretic Peptide to a validated mortality risk prediction model in older people using a standardised international clinical assessment tool

The authors explore the question if addition of the cardiac biomarker BNP adds prognostic value to a established mortality prediction score in elderly adults in NZ.

This is a well-designed study with a well-phenotyped cohort and fair sample size. Improving risk prediction in the elderly (mean age >80 in this cohort) is of crucial clinical importance for the patient, patient’s family and physician. The main findings of this study are (1) adding BNP to the RiskOP model significantly improved the accuracy of one-year mortality prediction, (2) especially in patients without a diagnosis of CHF. The study utilizes advanced statistical models.

Major

BNP or NT-proBNP is a valuable prognostic biomarker in a wide range of cardio-vascular diseases. BNP is influenced by age, BMI, inflammation, gender, atrial fibrillation and kidney function. The authors should adjust for those metrics before performing survival analyses.

The authors should explain how BNP levels were incorporated in the RiskOP Model. Were age appropriate thresholds used?

Response: Age, Sex, and BMI are part of the RiskOP model (see end of the 1st paragraph of the methods section) , therefore from a statistical analysis perspective it is not appropriate to account for them a second time (i.e. their influence, through RiskOp, is already accounted for). Unfortunately, we do not have information on AF and kidney function. We have acknowledged this limitation in the discussion:

“Additionally, which the RiskOP model incorporates Age, Sex, and Body Mass Index, other factors which may be associated with BNP such as inflammation, kidney function, and atrial fibrillation were not measured or included as covariates. If this was able to be done these may well improve the accuracy of the risk prediction.“

BNP concentrations were included as a continuous variable. Dichotomising at any threshold would effectively throw away information and make the models poorer. We have added the phrase “(as a continuous variable)” in the description of the model at the end of the second paragraph of the statistical analysis section.

It would be helpful for the reader to report the actual BNP values of the cohorts.

The authors should explain why they chose BNP as a biomarker, why not serum sodium, albumin or CRP?

Response: Thank you for the suggestion. We have added the information on BNP concentrations to the first paragraph of the Results. We chose BNP simply because it is one of the most frequently measured biomarkers in Heart Failure patients in New Zealand. We thought this adequate to assess the proof of concept that the addition of a biomarker to risk prediction from a standardised assessment can improve mortality prediction (line 347). Future studies may look at multi-biomarker response and we have now introduced this possibility into the discussion, we had already indicated that sodium or albumin may be included, but have now added CRP and creatinine.

The mean age was >80 years old, individuals at this age have a very high change for diastolic dysfunction. Do the authors have access to echocardiograms in the CHF cohort? Is there a way to report the prevalence of Afib in both cohorts?

Response: Unfortunately, we do not have access to echocardiograms or AF information. We note that in this cohort of people with high-needs, in New Zealand echo-cardiograms are often not done. We have acknowledged the limitation in the limitations section (as in the addition we made in response to an earlier point raised by the referee).

The authors should clarify when how the RiskOP model was derived, is this part of the interRAI assessment?

Response: The development of the RiskOP model was published separately with detail on how in Lancet eClinical Medicine (reference 11). It would be too much detail to include here. We have clarified in the text that it was derived on people with a Home Care interRAI assessment (interRAI-HC) – i.e. the same as all people in this present study. The sentence in the introduction now reads:

“The authors recently developed and validated a mortality risk model (RiskOP) for those people aged 65 plus using data from the interRAI-HC only[11].”

And we added a sentence in the methods:

“The RiskOP model had been developed in a subset of these patients (up to March 2018)[11].”

Are the authors suggesting that BNP should be part of a routine assessment in the elderly? Could they estimate a benefit-cost ratio?

Response: No, we are not suggesting it should be part of a routine assessment in all elderly. Any conclusion we draw must be limited to the high-needs cohort (in this case identified by needing an interRAI Home Care assessment). What we are demonstrating in this proof-of-concept is that a biomarker, in this case BNP, could make risk prediction for mortality better. It is too early to suggest routine adoption, the consequences and benefits of better risk prediction need to be discussed by the medical community. Additionally, we need to assess the added value of other biomarkers, such as creatinine and CRP, which could be measured at the same time. We believe only then a cost-benefit analysis would be applicable (we cannot estimate with on the current data). We have expanded the discussion around future directions:

“Further research would need to be undertaken in order to ascertain the value of including additional commonly used blood tests for the older adult population such as sodium, creatinine, albumin, and C-reactive protein. This research would not only need to ascertain the statistical improvement in risk prediction, but also look at a cost-benefit analysis, and ideally would include calibration of models for specific groups including in New Zealand Māori and Pacific Peoples.”

Image quality of the figures needs to be improved

Response: Thank you – probably a product of the pdf generation, we will work with the editors to ensure good quality.

Minor

Response: Thank you. Unfortunately, these line and page numbers do not correspond with the ones on our documents, nevertheless, we have tracked down these as best we can.

page 9, 69, sentence needs improved syntax

Response: We were uncertain on the exact sentence referred to, but revised the entire page and found a sentence with a syntax issue which we fixed.

page 11, 102, doe the authors mean “informed” instead of “inform”?

Response: the sentence is “...to make informed decisions...” and is therefore correct.

page 12, 146, what do the authors mean by “cost effective”?

Response: Simply that its costs are reasonable compared to other costs (ie it is inexpensive). We have rephrased this to avoid confusion.

page 17, 236, this is confusing. So how many BNP measurements were included in the analysis?

Response: Just one per participant, we have added the word “single” in the methods to clarify and changed the wording of the sentence in question.

Page 17, table 1 the column Died should have the figure 1 deleted

Response: Thank you, deleted.

Page 19, 258 should read “died” not “die”

Response: Thank you, corrected.

Page 19, 268, Fig2, should read “died” not “dies”

Response: Thank you, corrected.

Page 20, 283 and 284, should read congestive heart failure

Response: Oops – thank you (I must now check the work I’m doing at the same time with CAD that I’ve not made the opposite error there!).

Page 20, 290, sentence needs improved syntax

Response: We have reworded.

Attachment

Submitted filename: PONE-D-22-10241 Response to Reviewers.docx

Click here for additional data file. (23KB, docx)

Decision Letter 1

Alonso Soto

4 Sep 2022

PONE-D-22-10241R1Evaluation of the added value of Brain Natriuretic Peptide to a validated mortality risk-prediction model in older people using a standardised international clinical assessment toolPLOS ONE

Dear Dr. Pickering,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The article is very interesting and certainly adds to scientific knowledge.  It is a well-designed study regarding the addition of a biomarker to existing prediction tool. Methods and results are very well written. However, the discussion does not address appropriately the limitations of the modified tool and the explanation for the lack of utility in those with CHF. Although the improvement is statistically significant, it seems not to be very relevant from a clinical point of view. The conclusions are a bit misleading overeating the value of the modified tool and should be modified stating that the improvement in performance is just marginal.

It should be discussed that the addition of a laboratory examination would be a major change in a score that is intended to be solely based on clinical data. Although the authors mentioned that is an inexpensive marker, this is not the same for most countries. In fact, BNP is unavailable even in reference hospitals in less developed countries.

As mentioned, there is no explanation about the reason why to perform a subgroup analysis on those patients with and without CHF. Moreover, there is a lack of discussion of the possible reason of the poor performance of the modified tool in the CHF subgroup. I think that BNP probably does not add prognostic information if the patient has already a diagnosis of CHF. In patients without CHF, BNP probably identify some undiagnosed or subclinical cases. This or other possible explanations should be included in a separate paragraph in the discussion.

Please submit your revised manuscript by Oct 19 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Alonso Soto, PhD

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: (No Response)

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Excellent paper! Very interesting for the readership. The authors did an excellent job in answering the reviewers questions.

Reviewer #2: 1) If the authors have adequately addressed your comments raised in a previous round of review …

I had the opportunity to read the questions of the first reviewer and the answers of the authors. To my knowledge, they answered correctly to the remarks. The manuscript was adapted correctly.

2) Is the manuscript technically sound, and do the data support the conclusions?

Absolutely.

3) Has the statistical analysis been performed appropriately and rigorously?

Appropriately: this article might be interesting for clinicians. But no single clinician understands more than 10% of the statistics applied. The question is, if so many analyses are necessary for this relatively simple question. "μηδέν άγαν" (meden agan)! Never too much, said Socrates. So why all these analyses, while a simple ROC says enough in this case. Moreover, the promised basic ROC curve (graph) is missing.

Rigorously: decision curve analysis: crucial is here the balance between giving erroneously a bad message, or erroneously a good message. The authors boldly put this balance at 1/1, which is against the philosophy of medical decision making. The key task in decision analysis is always the estimation of this balance. Without a thorough even qualitative research concerning this balance, decision curve analysis makes no sense.

4) Have the authors made all data underlying the findings in their manuscript fully available?

This question has been answered in the first review.

5) Is the manuscript presented in an intelligible fashion and written in standard English

The English is perfect. But I think even some statisticians will have difficulty understanding all analyses done. I had to go back to the difficult decision curve analysis, to understand fully what the authors mean.

As the authors state that this article might be interesting for clinicians, I propose to show the basic ROC curves with AUC statistics. And to state clearly the difference between significance and effect size. The latter is very small in this research, even in non-cardiac patients. The authors use the word ‘marginally’ in the first paragraph of the discussion, but go further with ‘clear added value’.

6) Abstract

Methods: the authors state: “Incremental value was assessed by change in Area Under the Receiver Operating Characteristic Curve (AUC)”

In the results section we (clinicians) miss a classical ROC curve. Instead, we see graphs that take time to understand.

Conclusions: I did not understand immediately the conclusions. The authors conclude that the BNP improves the prediction for non-cardiac patients. I think I’m not the only reader to expect BNP to influence prediction especially in cardiac patients. I would state this as an unexpected paradox: BNP has an added value only in non-CHF.

Suggestion:

After revision by the first reviewer, I should state this is an excellent article, but not for clinicians or Plos One. In the present writing, I would submit it to Medical Decision Making, given the thorough statistical analyses (except for the decision curve analysis, which should be revised). If it is intended for Plos One, I would revise it along the suggestions I made hereabove.

Reviewer #3: In this manuscript, Pickering et al. performed a proof-of-concept study to evaluate the potential of adding BNP to the one-year mortality prediction model. The study included >1500 individuals with completed interrail-HC assessment and BNP measurement. It was found that the addition of BNP would increase AUC by 0.015 (95% CI 0.004-0.028). The improvement mostly came from individuals without CHF. Overall, the manuscript is well-written.

Following are some minor comments.

1. In the abstract, please define CHF. Given that more than half of individuals had CHF, such information could be also included in the abstract.

2. Please also include the number of mortalities in the abstract

3. The sample size used for the calculation is unclear. In Table 2, it is 1537, but in the Results section, it is indicated that 1572 individuals had all the required data. The total number of individuals with CHF (n=785) and without CHF (n=752) would be 1537. This number should be also used in the abstract, but not 1690 individuals.

4. It would be useful to comment on why the addition of BNP only improved the prediction performance for individuals without CHF but not for those with CHF.

5. Is there any sex-specific difference in the prediction performance?

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Nils Patrick Nickel

Reviewer #2: Yes: Jef Van den Ende

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Decision Letter 2

Alonso Soto

4 Nov 2022

Evaluation of the added value of Brain Natriuretic Peptide to a validated mortality risk-prediction model in older people using a standardised international clinical assessment tool

PONE-D-22-10241R2

Dear Dr. Pickering,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Alonso Soto, PhD

Academic Editor

PLOS ONE

Acceptance letter

Alonso Soto

10 Nov 2022

PONE-D-22-10241R2

Evaluation of the added value of Brain Natriuretic Peptide to a validated mortality risk-prediction model in older people using a standardised international clinical assessment tool.

Dear Dr. Pickering:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Alonso Soto

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Attachment

    Submitted filename: PONE-D-22-10241 Response to Reviewers.docx

    Click here for additional data file. (23KB, docx)
    Attachment

    Submitted filename: Response to Reviewers R2 220927.docx

    Click here for additional data file. (20.8KB, docx)

    Data Availability Statement

    Because of restrictions of use of clinical data we cannot make the data set freely available. However, we can make it available if contacted subject to ethics approval from a New Zealand ethics board and a formal use agreement between ourselves and those who wish to use the data. For New Zealand data this is particularly important because some of the data is from Māori, the indigenous people of New Zealand, who are acknowledged to have sovereignty of their data and its use. The restrictions are imposed by the New Zealand Ministry of Health and Disability Ethics Committee (HDEC). Data access queries may be sent to the Department of Medicine, University of Otago Christchurch, Christchurch, New Zealand (contact via christchurch@otago.ac.nz).

    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES