The diagnostic accuracy and utility of a B-type natriuretic peptide test in a community population of patients with suspected heart failure

Abstract

Background

National guidelines suggest the use of natriuretic peptides in suspected heart failure but there have been no studies comparing assays in primary care.

Aim

To test and compare the diagnostic accuracy and utility of B-type natriuretic peptide (BNP) and N-terminal B-type natriuretic peptide (NT proBNP) in diagnosing heart failure due to left ventricular systolic dysfunction in patients with suspected heart failure referred by GPs to one-stop diagnostic clinics.

Design of study

Community cohort, prospective, diagnostic accuracy study.

Setting

One-stop diagnostic clinics in Darlington Memorial and Bishop Auckland General Hospitals and general practices in South Durham.

Subjects

Two hundred and ninety-seven consecutive patients with symptoms and signs suggestive of heart failure referred from general practice.

Method

The study measured sensitivity, specificity, positive and negative predictive values (PPV, NPV), and area under receiver operating characteristic curve for BNP (near patient assay) and NT proBNP (laboratory assay) in diagnosis of heart failure due to left ventricular systolic dysfunction. The NPV of both assays was determined as a potential method of reducing the number of referrals for echocardiography.

Results

One hundred and fourteen of the 297 patients had left ventricular systolic dysfunction (38%). At the manufacturer's recommended cut-off of 100 pg/ml BNP gave a NPV of 82%. BNP performed better at a cut-off of 40 pg/ml with a NPV of 88%. At a cut-off of 150 pg/ml, NT proBNP gave a NPV of 92%. Using cut-offs of 40 pg/ml and 150 pg/ml for BNP and NT pro-BNP, respectively, could have prevented 24% and 25% of referrals to the clinic, respectively.

Conclusions

In this setting, NT pro-BNP performed marginally better than BNP, and would be easier to use practically in primary care. A satisfactory cut-off has been identified, which needs validating in general practice. NT pro-BNP could be used to select referrals to a heart failure clinic or for echocardiography. This process needs testing in real-life general practice.

INTRODUCTION

Heart failure is an increasing public health problem that is largely managed in primary care.¹ Diagnosis of heart failure in patients presenting to general practice is difficult, and may be incorrect in up to 70% of cases.^1–3 Echocardiography is currently considered to be the investigation of choice for confirming left ventricular systolic dysfunction. It is, however, not uniformly available to all GPs, and may be an expensive option for a first-line investigation.^4–6 Even if open access echocardiography is available its use is variable, and many GPs have difficulties with interpretation of the results.⁴ Furthermore, the capacity for performing the test is limited by lack of availability of suitably trained technicians and cardiologists to give a clinical interpretation of results. Observational studies of open access echocardiography services have shown that only 14–23% of patients referred have left ventricular systolic dysfunction.³

How this fits in

Small, single-centre studies have suggested that B-type natriuretic peptide (BNP) or N-terminal BNP (NT proBNP) has reproducible value as a test to rule out heart failure due to left ventricular systolic dysfunction. Despite inclusion in national guidelines, uptake of BNP or NT proBNP use has been slow in the NHS. This is the first study, using consecutive patients referred from primary care to one-stop diagnostic clinics that has compared a point-of-care assay of BNP with a laboratory NT proBNP assay and demonstrated high negative predictive value for both methods. We have identified suitable cut-offs for BNP and NT proBNP in a patient group representative of patients suspected by GPs of having heart failure, that could be used in primary care.

B-type natriuretic peptide (BNP) is one of a family of structurally similar peptide hormones. The major site of BNP production is the left ventricle.⁷ Cleavage of the precursor protein (proBNP) produces BNP, which causes diuresis, natriuresis, vasodilatation and smooth muscle relaxation, and the biologically inactive peptide N-terminal BNP (NT proBNP).⁸ Both are readily detectable in plasma and rise with increased ventricular and atrial stretch and pressure overload.⁷ Plasma levels are raised in heart failure, rising in line with severity and New York Heart Association functional class.^9,10

It has been proposed that BNP or NT proBNP, tests that can be performed using venous blood, can be used by GPs to identify patients with heart failure.^11–13 Small, single-centre studies have suggested that BNP or NT proBNP has reproducible value as a test to rule out heart failure due to left ventricular systolic dysfunction and to potentially preselect patients for referral for echocardiography.^14–16 However, other studies have questioned the accuracy of BNP in excluding heart failure.^17–19 Most studies used ‘in house’ assays and echocardiography, radionuclide ventriculography or cardiac catheterisation as the gold standard comparison, and examined selected groups undergoing these investigations, which were not representative of ‘all comers’ presenting to general practice.

In late 2001 two assays became commercially available, a point-of-care BNP fluorescence immunoassay using the Biosite Triage System (Biosite Diagnostics, Velizy, France) and an automated laboratory ECLIA assay NT proBNP system developed from a standard microtitre plate system from Roche Diagnostics and run on the Elecys analyser.

Both companies presented assay reference ranges based on 97.5 percentiles in healthy volunteers up to the age of 65 years. As the average age of patients with heart failure is around 75 years, this presented difficulties in use of the assays in our population of patients with suspected heart failure. There have been no comparative studies of the two assays in patients suspected by their GPs of having heart failure. Our primary study aim was to test and compare the diagnostic accuracy and utility of BNP and NT proBNP in diagnosing heart failure due to left ventricular systolic dysfunction in patients with suspected heart failure, referred by GPs to one-stop diagnostic clinics.

METHOD

All 109 local GPs from 23 Darlington and Dales practices were invited to refer patients with symptoms and signs suggestive of heart failure to a one-stop diagnostic clinic within their local hospital.²⁰ All practices covering a population of 190 000 patients agreed to participate, and 94 GPs referred at least one patient to the clinics. The 15 GPs who did not refer did not differ from the 94 who did refer on the basis of age, sex, geographical location, ethnicity, practice partnership size, or length of time in practice. Practices received an educational session on current diagnosis and management of heart failure from study clinicians and were given a locally produced guideline on the diagnosis and management of heart failure due to left ventricular systolic dysfunction. A referral template was issued to all GPs and their secretaries. The study ran over a 12-month period.

All patients referred were clinically assessed by clinicians with routine biochemistry, haematology, chest X-ray and 12-lead electrocardiogram (ECG) results available. Spirometry was conducted where considered appropriate.

Sampling for BNP and NT proBNP

Venous blood samples were drawn by clinicians under standard clinic conditions. For the triage BNP assay, whole blood was drawn into an EDTA tube and the sample assayed within 1 hour. For the Roche Diagnostics NT proBNP assay, the sample was taken into vacutainer tubes containing a sample-separating gel and placed on ice. The sample was transferred to the lab where it was spun, separated and frozen at −20°C. Samples were then assayed in batches using the Elecys analyser with laboratory staff blinded to the clinical assessments. The results were not used by clinicians in diagnostic or management decision making. Quality controls were carried out for both BNP and NT proBNP assays before each run of assays.

Analysis

Concentrations of BNP and NT proBNP both exhibited skewed distributions and were log transformed before analysis. The diagnostic performance of the assays were assessed using receiver operating characteristic (ROC) curves, formed by plotting sensitivity on the y axis and 1-specificity on the x axis for all possible cut-off values of each diagnostic test.²¹

In addition to the area under the curve (AUC), we identified the cut-off value that maximised sensitivity without unacceptable loss of specificity, ensuring high negative predictive values.

The McNemar test was used to assess the significance of differences between sensitivity and specificity. Differences between AUCs were tested using the method developed by Hanley and McNeil.^21,22

Echocardiography and electrocardiography

Full standard echocardiography was performed and reported by British Society of Echocardiography-accredited clinical physiologists blinded to the clinical details, clinical assessment and BNP/NT proBNP results. Siemens Sequoia C256 and GE Vivid 7 echocardiogram machines were used at Bishop Auckland and Darlington Memorial hospitals respectively. Left ventricular function was assessed by ‘eyeball’ assessment, by left ventricular ejection fraction calculated by Simpson's rule, and by wall motion index using the American Society of Echocardiography 16 segment model.²³ Doppler studies were also carried out and other cardiac abnormalities that may have led to breathlessness or a raised BNP/NT proBNP result were documented.

Fifteen per cent of echocardiograms were independently assessed by a cardiologist as a measure of quality control. Left ventricular systolic dysfunction was defined as mild, moderate or severe by ‘eyeball’ assessment (n = 114).²⁴ Ejection fraction was measured in patients in sinus rhythm if adequate images were obtained and left ventricular ejection fraction <0.40 was considered to represent left ventricular systolic dysfunction (n = 67).²⁵ A wall motion index of >1.2 was taken to be abnormal (n = 66). As it was not possible to measure ejection fraction or wall motion index in all subjects, clinicians diagnosed heart failure due to left ventricular systolic dysfunction if one or more parameter was abnormal.

ECGs were reported by two doctors as being either normal or abnormal using the Minnesota criteria.²⁶

RESULTS

Three hundred and five consecutive patients referred by their GPs to one-stop diagnostic clinics at two hospital sites (Darlington Memorial and Bishop Auckland General) were invited to participate.²⁰ Two hundred and ninety-seven patients gave informed consent to be included, and eight patients either declined study entry or were considered incapable of giving informed consent.

One hundred and fourteen patients (38%) had left ventricular systolic dysfunction. Table 1 lists the differences in baseline characteristics/demographics of the left ventricular systolic dysfunction and non-left ventricular systolic dysfunction populations. Significantly more patients in the left ventricular systolic dysfunction group were male, were receiving angiotensin-converting enzyme (ACE) inhibitors, had previous myocardial infarction, or had atrial fibrillation, reflecting individuals with high cardiovascular risk factors for development of left ventricular systolic dysfunction. Significantly more patients with hypertension did not have left ventricular systolic dysfunction. However, many of these had left ventricular hypertrophy and diastolic dysfunction, and would have been classified in other studies as having heart failure with preserved systolic function.²⁷ These conditions have been shown to raise natriuretic peptide levels.^28,29

Table 1.

Baseline patient characteristics.

	Left ventricular systolic dysfunction patients (n = 114)a	Non-left ventricular systolic dysfunction patients (n = 183)	P values
Mean age (years)	73.5	74.0	nsb
Age range	34–94	43–94	ns
Sex, % male (n)	47 (54)	30 (55)	<0.005
Diuretics, % (n)	70 (80)	63 (115)	ns
ACE inhibitor or angiotensin II receptor blocker, % (n)	46 (53)	33 (60)	<0.05
β-blocker, % (n)	18 (20)	21 (39)	ns
Hypertension, % (n)	27 (31)	39 (71)	<0.05
Ischaemic heart disease, % (n)	33 (38)	27 (49)	ns
Previous myocardial infarction, % (n)	24 (27)	7 (12)	<0.0001
Atrial fibrillation, % (n)	25 (29)	15 (27)	<0.05
Diabetes, % (n)	8 (9)	10 (18)	ns
Chronic obstructive pulmonary disease, % (n)	19 (22)	26 (47)	ns

^aNumber of patients.

^bLeft ventricular systolic dysfunction (LVSD) and non-LVSD patients not significantly different at 5% level.

All 297 patients had a natriuretic assay, but due to technical reasons only 263 BNP and 273 NT proBNP assays were completed. These reasons included malfunction of the triage machine and supply problems with diagnostic kits. This was random and was not related to the presence or severity of heart failure. Figure 1 shows the ROC curves for both assays. Table 2 lists the areas under the ROC curves for both assays by sex. Differences in area between BNP and NT proBNP were not statistically significant.

Figure 1.

ROC curve: ‘eyeball’ (all patients)

Table 2.

Area under ROC curve.

Peptide, sex	Left ventricular systolic dysfunction (n = 114)
BNP
Alla	0.79
Male	0.79
Female	0.80
NT proBNP
Alla	0.81
Male	0.79
Female	0.82

^aDifferences between BNP and NT proBNP were not significant at the 5% level. ROC = receiver operator characteristic.

We selected a range of cut-off points to optimise the test performances. For BNP we found the best test performance and highest negative predictive value (NPV) was at 40 pg/ml. For NT proBNP the optimum value was 150 pg/ml. During the period of study the manufacturers made cut-off recommendations: Biosite (BNP) recommend 100 pg/ml and Roche diagnostics (NT proBNP) recommend 125 pg/ml for both sexes in the US, and 100 pg/ml and 150 pg/ml for males and females respectively in Europe and the UK. Table 3 compares the NPV for both BNP and NT proBNP using the recommended cut-offs with our own results.

Table 3.

Negative predictive values at different cut-off points.

Peptide	Cut-off point (pg/ml)	NPV (95% CI)
BNP all	40	0.88 (0.80 to 0.96)
BNP All	100	0.82 (0.76 to 0.89)
NT proBNP all	125	0.92 (0.85 to 0.99)
NT proBNP all	150	0.92 (0.86 to 0.98)
NT proBNP male	100	0.89 (0.74 to 1.00)
NT proBNP female	150	0.94 (0.88 to 1.00)

NPV = negative predictive value.

Table 4 shows the overall performance characteristics of the two assays at our chosen cut-off. The differences between BNP and NT proBNP were not statistically significant. The cut-off points of 40 pg/ml for BNP and 150 pg/ml for NT proBNP both offer a high NPV of (0.88 and 0.92 respectively) but specificity is relatively poor (0.38 and 0.40 respectively). However, 61% of patients with false-positive results (38% of total patients) had other significant cardiac or related abnormalities that could have raised natriuretic peptide levels. These included left ventricular hypertrophy (n = 37),²⁹ atrial fibrillation (n = 17),³⁰ mitral regurgitation (n = 17), pulmonary hypertension (n = 17), diastolic dysfunction (n = 14),²⁸ aortic stenosis (n = 3),³¹ aortic regurgitation (n = 2), atrial flutter (n = 2), right ventricular hypertrophy (n = 2), cardiac amyloidosis (n = 1), lupus and paraproteinaemia, paraproteinaemia (n = 1), lung carcinoma (n = 1), and cor pulmonale (n = 1).³² There were some patients who had more than one of these pathologies co-existing.

Table 4.

Diagnostic utility of BNP and NT proBNP at optimal cut-off values.

Peptide	Cut-off point (pg/ml)	Sensitivitya (95% CI)	Specificitya (95% CI)	PPVa (95% CI)	NPVa (95% CI)
BNP	40	0.92 (0.87 to 0.97)	0.38 (0.30 to 0.45)	0.49 (0.42 to 0.57)	0.88 (0.80 to 0.96)
NT proBNP	150	0.94 (0.90 to 0.99)	0.40 (0.33 to 0.47)	0.48 (0.41 to 0.55)	0.92 (0.86 to 0.98)

^aDifferences between BNP and NT proBNP were not significant at the 5% level. PPV = positive predictive value. NPV = negative predictive value.

DISCUSSION

Summary of main findings

This is the first study, using consecutive patients referred from primary care to one-stop diagnostic clinics that has compared a point-of-care assay of BNP with a laboratory NT proBNP assay and demonstrated high NPV for both methods. The patient group was representative of patients suspected by their GPs of having heart failure.

Comparison with existing literature

The definition of heart failure due to left ventricular systolic dysfunction varies between mortality studies, and between specialist guideline bodies.^12,13,25,33 All guidelines suggest echocardiography as the gold standard for confirmation of left ventricular systolic dysfunction.^12,13 However, it is not always possible to measure left ventricular ejection fraction, especially in patients with obesity or chronic obstructive pulmonary disease. In this situation, ‘eyeball’ assessment of left ventricular function has been shown to be an accurate measure.²⁴ This is often a ‘real-life’ assessment technique employed by experienced clinical physiologists and we felt it important to conduct this study using this measure. This is the first study to compare BNP and NT proBNP against this standard, and demonstrate high negative predictive values for ruling out left ventricular systolic dysfunction.

Nine patients were false negatives, where BNP or NT proBNP were below the cut-off points. All were on cardio-active therapy (nine on diuretics, three on ACE inhibitors and one on a β-blocker) which could have lowered natriuretic peptide levels.^34–36 If natriuretic peptide assays were readily available such patients would have been tested before starting treatment. We are not yet at this stage in the NHS, largely because there are unanswered questions and funding issues around the use of natriuretic peptides in primary care. It would not have been ethical to stop treatment before assaying patients, due to the risk of inducing decompensated heart failure. However, we should be reassured that individuals with low natriuretic peptide levels have a good prognosis and most were on appropriate therapy already.^37,38

Since our study was initiated, Biosite has suggested a cut-off point of 100 pg/ml for ruling out heart failure. Our results show that this gave a NPV of 82% compared to 88% at 40 pg/ml. Roche diagnostics suggest cut-offs of 125 pg/ml for both sexes in the US, but 100 pg/ml for males and 150 pg/ml for females in Europe and the UK. Our results showed a NPV of 92% at 125 pg/ml, 89% for males at 100 pg/ml, 94% for females at 150 pg/ml, and 92% for both sexes at a 150 pg/ml cut-off (Table 3). Although Biosite suggests a cut-off of 100pg/ml our figures suggest that 40 pg/ml is a better cut-off and Val-HeFT data suggest that a BNP >40 pg/ml is associated with increased all-cause mortality.³⁹ In this study 13 patients with BNP levels between 40 and 100 pg/ml who had left ventricular systolic dysfunction would have been denied evidence-based therapy.

Although point-of-care BNP may be useful when a rapid result is needed, for example in the emergency department or medical assessment unit, it is likely to be impractical in general practice.⁴⁰ Furthermore, it would not be practical for individual or even groups of practices to own a Biosite Triage machine costing around £2000; each test is costly at around £15 each, kit shelf lives are short, and performing quality controls is not a familiar task for GPs. Laboratory assays are simple and familiar to GPs, samples are stable at room temperature for up to 72 hours, and quality controls are stringent.⁴¹ This makes the laboratory assay an ideal test for use in general practice.

Using cut-offs of 40 pg/ml and 150 pg/ml for BNP and NT pro-BNP, respectively, could have prevented 24% and 25% of referrals to the clinic, respectively. This could potentially free up valuable echocardiography capacity and clinician time in a healthcare system in which the availability of both is limited.

Guidelines and National Institute for Health and Clinical Excellence guidance for the diagnosis and management of heart failure due to left ventricular systolic dysfunction suggest that if an ECG is normal then left ventricular systolic dysfunction is very unlikely, with a NPV of 97% in some studies.⁴² However, a NPV of 83% in this study suggests that significant left ventricular systolic dysfunction can be present in the presence of a normal ECG. If current guidance had been adhered to in referral of patients for echocardiography 18% (n = 21) of patients with left ventricular systolic dysfunction would have been missed. Previous studies that suggested a normal ECG effectively ruled out left ventricular systolic dysfunction may not have been representative of the type of patients referred by GPs with suspected heart failure.⁴² In this study BNP and NT proBNP both performed better than ECG in selection of patients for further assessment of left ventricular function.

Despite inclusion in guidelines, uptake of BNP or NT proBNP use has been slow in the NHS.¹² Clinicians and primary care trusts still harbour concerns about appropriate cut-offs, the extra cost of BNP assays, lack of expedient referral pathways for patients with a raised BNP level, and absence of cost–benefit/effectiveness data.

Wright and colleagues demonstrated in a small randomised controlled trial that NT proBNP measurement significantly improves the diagnostic accuracy of heart failure by GPs over and above customary clinical review.⁴³ However, this was a trial situation, and is not necessarily representative of UK GPs using NT proBNP in patients with suspected heart failure and then deciding whether or not to refer for echocardiography.

Strengths and limitations of the study

Consecutive patients referred to the clinics from primary care were studied, hence reducing selection bias. However, we did not study practices to check whether there were any patients not being referred to clinics. Since most GPs know that echocardiography is needed for patients with suspected heart failure and that the clinics are well advertised, we feel it is likely that we captured all the patients GPs worried about. Measurement bias was reduced by utilising the same high-quality echocardiography equipment operated by British Society of Echocardiography-accredited cardiac physiologists, reported to a uniform standard and quality checked by one cardiologist. Disease progression bias was reduced by all tests being taken at a one-stop diagnostic clinic.

Thirty-eight per cent is an unexpectedly high prevalence rate for left ventricular systolic dysfunction in this population, and it is possible that selection of patients was influenced by the education sessions given to practices. This may have an impact on the generalisability of this study.

These diagnostic values have been derived in one cohort, but require validation in a second cohort with different subjects and clinicians.

Implications for clinical practice and future research

Further research studying the use of BNP or NT proBNP, with or without electrocardiography, as screening tests in patients with suspected heart failure in an everyday NHS primary care environment is urgently needed. A cut-off point of 150 pg/ml for NT proBNP could be used to select referrals to a one-stop diagnostic clinic or for echocardiography, but this process needs testing in ‘real-life’ general practice. GPs would have NT proBNP available to use in triage of patients with symptoms and signs suggestive of heart failure. An assay result equal to or more than 150 pg/ml would prompt GPs to refer for echocardiography. An assay result of less than 150 pg/ml would effectively rule out heart failure and prompt the GP to seek an alternative cause for the patient's symptoms and signs. This would test the validity of this cut-off and provide cost–benefit data to inform further use of NT proBNP in primary care.

Such a future validation study could also recruit GPs with and without the educational programme to see whether this has an effect on diagnostic pick up rates and appropriate referral for echocardiography.

Table 5.

Utility of the ECG in diagnosing left ventricular systolic dysfunction by ‘eyeball’ assessment.

Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
0.82 (0.74 to 0.89)	0.58 (0.51 to 0.65)	0.55 (0.47 to 0.62)	0.83 (0.77 to 0.90)

PPV = positive predictive value. NPV = negative predictive value.

Funding body

Ahmet Fuat was supported by a Research Training Fellowship funded by the NHS Executive, Northern and Yorkshire. The Triage BNP analyser and test kits were funded by a grant from Darlington Primary Care Trust. NT proBNP tests were provided free of charge by Roche Diagnostics and analysed without charge by the Department of Biochemistry, Darlington Memorial Hospital

Ethics approval

The study was given ethics approval by County Durham and Darlington Local Research Ethics Committee (01-02/76/Dec01) and South Durham Health Care NHS Trust Research Governance Approval (JCL-AL-019)

Competing interests

Ahmet Fuat and Ali Mehrzad have attended and spoken at conferences organised and sponsored by Roche Diagnostics