Lung ultrasound in heart failure: Lessons from re‐analysis of Lung Ultrasound 2011 database

Abstract

Introduction: In the setting of patients presenting with shortness of breath to an Emergency Department a simple lung ultrasound protocol aimed at detecting pulmonary oedema has been shown to have diagnostic accuracy of 85%. This article reviews data from the original study, in an attempt to determine whether adjusting the protocol and/or interpretive criteria would improve results.

Method: A large lung ultrasound project provided the dataset. Inter‐rater and intertest discrepancies were reviewed. Then original stored images and comments were retrospectively analysed using alternate interpretive criteria. Specific variations included changing the number of B‐lines required to define ‘wet lung’ and assessing other pleural line abnormalities. Where they had been acquired cardiac loops were reviewed in addition to the lung images.

Results: The 204 original studies available were reviewed. Some disagreement could be attributed to inexperience and unclear definitions. Adjusting the number of B‐lines did not improve diagnostic accuracy. All positive scans, with numerous B‐lines were reviewed using more advanced diagnostic criteria (pleural line abnormalities) and the number of false positives was decreased. In cases where cardiac views were available, their inclusion was beneficial.

Conclusion: A simple lung ultrasound protocol to assess for ‘wet lung’ in patients presenting to Emergency Departments provides diagnostic accuracy of around 85% in the hands of relative novices. More advanced interpretation of the same_ultrasound images, and the addition of cardiac views, is likely to further improve diagnostic accuracy.

Introduction

The ‘B line’ artefact is now established as a sensitive sonographic sign of pulmonary oedema, a great improvement over auscultation and at times better than CXR and BNP. ^{1 – 5}

Lung ultrasound (LUS) is cheap, portable, feasible and safe. Clinicians are striving to understand the best way to use this new tool, but the tool has limitations. When predicting pulmonary oedema, the B line appears to have reached a ceiling ^{5 – 7} trading off between sensitivity and specificity. New_research seeks to improve B line test characteristics by altering sample selection, protocols or reference definitions. In this article we speculate how the new research might refine our own practice. ^{8 – 10}

‘LUS2011’ is a data set of 204 elderly patients who presented to our emergency department in 2011–2012 complaining of some degree of breathlessness. ^{8 – 10} They were examined with an 8 view LUS protocol (adapted from Volpicelli ¹¹ )) and bedside and then blinded interpretation of still images were recorded and compared against chart review. Our overall diagnostic accuracy for identifying pulmonary oedema was 85% – an improvement on conventional practice but seemingly mediocre when compared with other sonographic studies examined in a recent meta analysis. ¹

The purpose of this paper is to re‐evaluate our data set in the light of recent literature, to see how we can improve our protocol. This paper compares the methods used in hallmark LUS papers against the LUS2011 dataset to highlight the factors that interact to produce sensitivity and specificity estimates. From this we derive a simple flow chart with recommendations for optimal use of LUS in the ED investigation for pulmonary oedema.

Method

The original protocol has been described in detail. ^{8 – 10}

The study analysed 204 patient data sets comprised of demographic data, a set of lung scans, bedside sonologist image interpretation, retrospective blinded sonologist image interpretation, formally reported chest x‐ray, and a subsequent specialist chart review (single auditor), blinded to LUS data. The post hoc study data sets also took into account the triage category, interpretation of extra views acquired (cardiac and IVC where performed), and the comments recorded independently by the bedside sonologist and the auditor.

All scan sets with either bedside or blinded interpretation differing from the gold standard (auditor) diagnosis were re‐examined to try and identify the source of error.

Where the bedside interpretations disagreed with the blinded interpretation, these scan sets were reviewed with reference to bedside comments and auditor diagnosis.

The literature was searched to identify potential improvements in the study protocol, with particular reference to papers referenced in the recent meta‐analysis. ^{5 – 7 , 11 – 17} Alternative protocols were applied retrospectively to the data set.

These were:

• 1Lowering the threshold to diagnose heart failure: – from 12 B lines to include those studies with nine or more B lines or bilateral effusions.
• 2Removing positive studies where alternate or dual pathology is likely: Extracting all those with pleural line abnormalities, significant asymmetry, spared areas or comments from the bedside regarding the same.
• 3Review of additional views, specifically cardiac and inferior vena cava: These were not included routinely in the interpretation of our simple LUS protocol, but sonologists were encouraged to attempt the views for the purpose of this planned post hoc study. A minority of our cases included these views, but to investigate feasibility, the 30 clearest cardiac views were sent to the chart auditor after the trial had closed. The chart auditor was a cardiologist with a special interest in echocardiography who assessed and rated the views as to the likelihood of cardiogenic pulmonary oedema. The auditor gave one of four ratings; unlikely, equivocal/non contributory, likely and very likely. These were compared against the stored bedside comments; ‘normal heart’ or ‘good LV’ or ‘isolated right sided dysfunction’ scored ‘unlikely, the absence of comment scored ‘equivocal’, any note of poor LV function or gross valvular dysfunction scored a ‘likely’ and ‘very poor LVEF’ scored ‘very likely’.

Statistical Methods: No attempt was made to statistically analyse the smaller numbers. Raw results are presented. Weighted kappa (VassarStat online calculator ¹⁸ ) was used to assess the agreement of the subset of cardiac view interpretation.

Results

One scan set was missing from the post hoc data set, giving 203 scan sets available for review. There were 35 discrepancies in a study of 204 cases. In 15 there was disagreement in the ultrasound interpretation between the bedside sonologist and the blinded ultrasound reviewer. In 20 studies the agreed LUS diagnosis differed from the final diagnosis defined by the retrospective chart audit.

Inter‐rater disagreement (between bedside sonologist and blinded ultrasound expert)

There was inter‐rater disagreement in 15/35 cases. Of these disagreements, two were unexplained, possibly due to suboptimal capture of B lines on the still images recorded for retrospective review. ⁸ .

Alternatively, the blinded reviewer may have interpreted overgain in the far field as confluent B lines.

In seven studies, the expert reviewer outperformed the sonologists, generally in the more subtle scans.

In six cases, extra lung pathologies also associated with B lines confounded interpretation of the protocol. We had no prospective strategy to count confluent B lines, or B lines that radiated from pleural line abnormalities, collapsed lung, or under effusions. Figures 1–6 demonstrate the lung signs that complicated interpretation of the protocol.

Figure 1.

This demonstrates the ultrasound appearance of normal lung. The horizontal echogenic line 2 cm below the skin is the pleural surface. In dynamic scanning this can be seen to ‘slide’ with respiration and ‘pulse’ with mediastinal movement. The vertical acoustic shadow artefacts are rib shadows (broad arrows). Occasional vertical, bright echogenic lines originating from the pleural surface are called comet tails if short (short arrow) and B‐lines if they continue to the deepest part of the image (long arrow).

Figure 6.

Pleural effusions are a common manifestation of a broad range of pathologies. Pleural fluid usually collects in dependent areas. It may compress adjacent lung, which can be seen to expand with respiration, or it may be part of an inflammatory response to an adjacent area of hepatisation. In either case, B lines may be seen deep to the effusion or consolidation, and the clinician must consider possible differentials before assuming the effusion is secondary to pulmonary oedema.

Figure 2.

B‐lines are bright echogenic lines originating at the pleural surface and passing to the deepest portions of the image. They move with pleural movement and are thought to be caused by reverberation where there is an interface between interstitial / alveolar fluid or fibrosis and air. In pulmonary oedema these become numerous. They have been described as spot lights, torch beams and even lung rockets shining from the pleural surface deep into the image. Counting the number of B‐lines is used to determine the presence of pulmonary oedema; however in the presence of confounding pathology such as fibrosis, contusion, or lymphangitis, with similar appearance the diagnosis must correlate clinically.

Figure 3.

In severe pulmonary oedema B‐lines may become confluent. Differentiating confluent B‐lines from an over gained image may be difficult. While it may seem obvious that where there are confluent B‐lines there is wet lung, it has been suggested that to give them a numerical value one can estimate the proportion of the visualised pleural line (long line) taken up by the confluence (short line), as a percentage, and divide by ten. In this example the confluence represents 50% of the distance between the two ribs – so 5 B‐lines.

Figure 4.

This case demonstrates pleural line abnormalities (PLA). B‐lines here can be seen originating more deeply within lung substance, behind small sub pleural consolidations. These are unlikely to be present where the pathology is acute pulmonary oedema alone. Lichtenstein calls these ‘sub B lines’. They are non‐specific and may be widespread or localised and occur with pneumonia, inflammation, metastases, pulmonary emboli and ALI/ARDS or where dual pathology exists.

Figure 5.

The appearance of pneumonia and lung consolidation, known as hepatisation, is shown here. Consolidated lung has the appearance of liver with a homogenous grey appearance. Air bronchograms may be seen with bright linear, air related artifact (arrows). Where the irregular interface between consolidated lung and aerated lung occurs an irregular echogenic line is seen. This has been called the “shredded lung” sign (curve). Altering the gain settings can help differentiate consolidated lung from effusion. Echogenic material within an effusion will tend to swirl with respiratory movement. Air bronchograms remain more static within solid lung.

Ultrasound diagnosis – retrospective chart auditor disagreement

There was discrepancy between the agreed ultrasound diagnosis and auditor in 20/35 cases. There were 10 false positives and 10 false negatives.

Of the false positives, six had extra pathology and four had indistinct or borderline numbers of B lines. Comments indicated that most with extra pathology were recognised at the bedside but sonologists were advised to interpret strictly as per protocol in the initial study. Two of the ‘false positives’ had cardiac clips strongly supporting the ultrasound diagnosis against the original auditor diagnosis.

Of the false negatives, two had bilateral effusions only, and eight were unexplained. The unexplained moiety was comprised of the missing scan set, two morbidly obese patients with clear scan sets, one with lung plus cardiac signs suggestive of pulmonary embolism and four scan sets with subthreshold changes.

Effect of changing definitions

In studies included in the meta‐analysis, Vitturi ¹⁶ , Cibinel ⁶ and Liteplo ⁵ varied the number of B lines to identify acute pulmonary oedema, or counted pleural effusions as indicating a positive scan. We re‐examined our scans using bilateral effusions and/or a lower threshold of nine B lines as indicative of heart failure. The diagnosis changed to the correct one in six cases of 203, and to the incorrect one in eight others.

Effect of complex interpretation

Copetti ²⁰ describes signs that may differentiate acute lung injury (ALI) and acute respiratory distress syndrome (ARDS) from cardiogenic pulmonary oedema. Markers of primary lung pathology are pleural line abnormalities, subpleural consolidations and spared areas – where B lines appear to skip contiguous regions of lung tissue.

Fifty–eight scans, identified as wet (54) or indeterminate (4) by either or both bedside sonologist and expert reviewer were re‐examined looking for signs associated with ALI/ARDS. Of18 false positives (10 false positives and eight inter‐rater disagreements), 10 were recognised as likely lung pathology at the bedside, as indicated by sonologist comments. On re‐examination three more demonstrated signs associated with ALI/ARDS. These 13 scans sets showed six consolidations, five pleural line abnormalities, one spare area, one loss of sliding. Of the 40 unanimously true positive scans, 11 demonstrated one or more of these signs.

The remaining five false positive scans (four agreed ultrasound diagnosis, one inter‐rater disagreement) had indistinct or predominantly laterobasal distribution of B lines.

Potential for extending acquisition

In the subset of reviewed cardiac loops, the agreement between bedside sonologist and cardiologist auditor was moderate (kappa = 0.57, 95% CI 0.369 −0.778). Two loops scored by the auditor as ‘very likely’ to be associated with heart failure had been previously assessed as ‘dry’ on chart review but designated ‘wet’ by LUS. Seven of the eight heart failure ‘unlikely’ were consistent between auditor and LUS diagnosis, the eighth had a written comment ‘significant mitral regurgitation’ but this was not demonstrated on the cine loop sent to the auditor.

Discussion

To improve the precision of lung ultrasound in heart failure diagnosis, clinicians need strategies to deal with the weakly positive scan, and the scan with too many signs.

The ‘weak positive’ issue is not fixed by changing B line thresholds or including effusions. The ‘overly positive’ scan can often be clarified using more advanced interpretation of LUS images, or by the additional information provided by bedside echocardiography.

A literature review highlighted ways that other groups have used to improve diagnostic accuracy using LUS to detect pulmonary oedema.

• 1Scanning by expert sonologists ^{6 , 12 – 14 , 19}
• 2Recruiting only acute or severe dyspnoea (higher pre‐test probability) ^{6 , 9 , 12 , 14}
• 3Exclusion of patients with possible lung pathology other than pulmonary oedema. ¹⁴

While these changes would result in better test characteristics, not all are practical in the clinical situation.

Combining our own experience with lessons from recent literature, we would aim for the following improvements.

1 Increasing sonologist expertise through education and experience

From our previous paper, the blinded expert reviewer outperformed the novice at the bedside. ⁹ This suggests a pleasing objectivity of the test, and also that experience may improve accuracy. Novices must avoid over‐gain in the far field, and in borderline cases cineloops may be saved. The inter‐rater disagreement also suggests that our protocol of 4 hours training with limited proctoring was barely enough for the relatively ultrasound naïve, though it may suffice for more experienced practitioners.

2 Standardising LUS protocols and interpretive criteria

Tightening of protocol definitions would improve agreement. As we progressed we discovered that not all B lines were created equal. Our simple teaching program did not include a way of interpreting B lines in the presence of confounding pathologies, nor how to quantify areas of confluent B lines. We became aware of a method of ‘counting’ confluent areas just after we closed our recruitment. ²⁰ This led to discrepancies between bedside interpretation and blinded review.

Another source of uncertainty was whether to count B lines from the shred sign of collapsed lung edge adjacent to effusion. The effusion is excess fluid, but not necessarily due to heart failure. This needs to be defined prior to any new project.

3 Design LUS protocols to match acuity

LUS scanning was pioneered in the ICU, ¹² among the critically ill. We feel that it is in the undifferentiated breathless patient that a LUS protocol aimed at detecting pulmonary oedema has its greatest utility. Supporting this we found that patients with heart failure presented with a higher triage category. ⁹ As interest spreads, attempts are made to apply the LUS criteria to more stable groups of patients, such as cardiac outpatient clinics, and this is where complexities will arise. ²

To deal with the less acute patient, some groups have redefined the ‘wet’ scan, decreasing the number of required B lines, or by counting pleural effusions. Similarly radiologists use cardiomegaly and effusions to diagnose heart failure. ^{3 , 22} Studies counting effusions and fewer B lines have met with mixed success, usually improving sensitivity but losing specificity. ^{2 , 5 , 6 , 16 , 23} The same occurred in this retrospective analysis.

4 Recognition of patients with other lung pathology

The B line is a sign common to several pathologies. Protocols lose accuracy when they assume B lines identify a particular condition. This lack of specificity can be minimised by selecting a test population from which the confounding pathologies have been excluded. ¹⁴

This approach is feasible when the patient can give a good history, and alternative imaging is available. If alternative imaging is not available, the practitioner may choose – for example – to disregard a positive LUS in a patient with known fibrosing lung disease, but regard a negative LUS as excluding pulmonary oedema.

Alternatively, advanced acquisition and interpretation skills can be added to scans that are strongly positive. Researchers have examined patterns of B lines in ALI/ARDS ^{19 , 21 – 23} and identified pleural line abnormalities in pleuritis, ²⁴ pneumonia ^{24 – 28} and pulmonary emboli. ²⁹ These signs reduced the false positive rate among the more sick patients although some were present in the true positives.

It is more accurate to say that PLA rules in extra lung pathology, it does not rule out pulmonary oedema.

Cardiac or IVC views may help to differentiate pulmonary oedema related B‐lines from other causes of B‐lines. The skilled sonologist can add cardiac and IVC views to the lung scan. Expert operator trials ¹⁹ and small groups have been successful in this practice. ^{7 , 16 , 17 , 30 , 31} The addition of further cardiac or IVC views appears feasible for inexpert sonologists, even if the clips are read at a later date by an experienced practitioner.

This study was not designed to compare cardiac views, so the moderate agreement is encouraging. In our study, out of 30 selected cardiac views, two would have reversed the auditor's diagnosis in the direction of the LUS finding, removing two of the false positives.

Limitations

Post hoc subgroup analyses are by nature weakened by unblinding, selection bias and small numbers. We had prepared for post hoc subgroup analysis by saving all scans and bedside comments. Re‐analysis of scan sets was performed en masse and without reference to other results.

Another limitation lies in the loss of information from the moving image gleaned at the bedside, to a still picture seen by a blinded reviewer. We included this limit, as we wanted to see if the chosen LUS protocol remained robust in the face of economical storage and audit.

Further research

It will be interesting to continue B line research with information that has since been published. Quantification of confluent B lines and the effect of supine posture were not part of the original teaching package and may improve accuracy of LUS for pulmonary oedema.

A recent paper implies a manoeuvre that may improve precision. ³¹ This group demonstrated in heart failure patients, that B lines increase in the supine position. Patients with a borderline number of B lines could be re‐examined with a second scan in the supine position, the equivalent of a small fluid challenge. If the number of B lines increases, cardiac decompensation might be inferred. No change in B lines would suggest fixed lung pathology.

Studies documenting the prevalence of pleural line abnormalities in the well population would also be very useful.

What our research adds

Figure 7 is a decision tree with suggestions for dealing with the issues of the weakly positive scan, and the scan with extra lung pathology. This is not a replacement for Lichtenstein's BLUE protocol, which was designed for experienced sonologists treating the critically ill. This is a simplification that introduces new practitioners to the lung scanning techniques of the BLUE protocol.

Figure 7.

Flow chart for 8 view Lung Scan Protocol in breathless older patients.

Conclusion

There is a diagnostic accuracy of 85% for sonologists looking for lung changes consistent with pulmonary oedema in breathless older patients. We believe that this figure will improve with ongoing education and experience of the sonologists, with better patient selection and consideration of alternate diagnoses, and with a more advanced but standardised scanning protocol.