Accuracy of Non-Invasively Determined Pulmonary Artery Systolic Pressure

Abstract

The non-invasive estimation of pulmonary artery systolic pressure (PASP) has become a standard component of echocardiographic examination. Our aim was to evaluate the accuracy of this modality in a large series of unselected studies obtained in clinical practice. All right heart catheterizations (RHC) over a 4 year period were reviewed. Studies with echocardiography available within 48 hours were evaluated for agreement of PASP. In an effort to mirror clinical practice, RHC was used as the gold standard and values for PASP were taken directly from their respective clinical reports. Overall, 792 RHC-echocardiogram pairs were identified. Echocardiographic PASP could not be estimated in 174 of these studies (22.0%). Correlation between modalities was moderate, but agreement was poor (r=0.52, p<0.001, bias 9.0%, 95% limits of agreement −53.2% to 71.2%). Misclassification of clinical PASP categories occurred more often than not (54.4%). Multivariate analysis utilizing multiple potential sources of error could only account for 3.2% of the total variation in discrepancy between study modalities (p=0.003). In conclusion, non-invasively estimated pulmonary artery systolic pressure had limited agreement with invasively determined pressure and misclassification of PASP clinical categories occurred in the majority of cases. Given the widespread use of echocardiographically determined PASP these data are in need of replication in a large prospective study.

Introduction

Several small studies have recently raised concerns about the agreement between echocardiographically estimated PASP (PASP_ECHO) and RHC derived PASP (PASP_RHC). To date PASP_ECHO has yet to be validated in a large series of unselected patients. Our aim was to evaluate the accuracy of PASP_ECHO in a “real world” population performed during standard patient care conditions and to investigate potential sources of error. We hypothesized that favorable results of earlier studies, performed under research conditions, would not necessarily be reproducible in actual clinical practice.

Methods

All consecutive RHCs performed in the cardiac catheterization laboratory of our institution from January 1, 2004 to January 1, 2008 were reviewed. Patients with echocardiographic examinations within 48 hours of the RHC were eligible for study inclusion. Additional inclusion criteria required mean right atrial pressure and pulmonary artery systolic pressure to be listed in the RHC procedure report. As recommended by the recent Expert Consensus Document on Pulmonary Hypertension, PASP_RHC was used as the “gold standard” (1). In an effort to mirror actual clinical practice, invasively determined variables were obtained directly from the cardiac catheterization procedure report generated at the time of clinical study by experienced interventional cardiologists.

Values for echocardiographically estimated right atrial pressure and PASP_ECHO were estimated at the time of clinical interpretation by level III, board certified echocardiographers. Briefly, the tricuspid regurgitation velocity signal was recorded in multiple views (tricuspid inflow, parasternal short axis, and apical 4 chamber) with continuous wave spectral Doppler. Using the highest obtainable maximum velocity from these Doppler profiles, peak tricuspid regurgitation gradient was determined using the modified Bernoulli equation [4*(tricuspid regurgitation velocity) ²] (2). The inferior vena cava (IVC) was imaged from the subcostal view while patients were asked to rapidly inhale or “sniff” during imaging. Estimated right atrial pressure was determined from the inspiratory degree of IVC collapse as follows: no inspiratory collapse = 20 mmHg, < 50% inspiratory collapse = 15 mmHg, > 50% collapse = 10 mmHg, and full collapse = 5 mmHg) (3). The estimated tricuspid regurgitation gradient and right atrial pressure were then added to derive PASP_ECHO. If the IVC was not adequately visualized, right atrial pressure was assumed 14 mmHg as suggested by Kircher et. al. (3). If the tricuspid regurgitation gradient was unavailable or deemed unreliable by the interpreting physician, PASP_ECHO was not reported. PASP clinical categories were defined as: normal (≤ 30 mmHg), mildly elevated (31–45 mmHg), moderately elevated (46–60 mmHg), and severely elevated (> 60 mmHg). Institutional review board approval was obtained for the study.

Values are reported as median with interquartile range (IQR), mean +/− standard deviation, and percentile value for non-parametric, parametric, and categorical variables respectively. Mean difference and mean percentage difference between modalities were calculated by taking the mean of the absolute value of echo-derived parameter subtracted from the RHC-derived parameter for each individual study pair. Independent Student’s t-test was used to compare means of independent continuous variables. Pearson’s Chi Square was used to evaluate categorical variables. Correlation values reported are Pearson. Bland-Altman analysis was used to describe agreement between studies, reported as bias and 95% limits of agreement. To describe the effect of univariate sources of error, the cohort was dichotomized based on various candidate sources of error and odds ratios for PASP clinical category misclassification were reported base on comparison of the two dichotomized groups. Multivariate regression analysis with forced entry was used to determine the overall contribution of identified sources of error and reported as an r² value. Bland-Altman analysis was done using GraphPad Prism version 5.01 (GraphPad Software Inc, La Jolla, California) all other statistical analysis was performed with SPSS version 17.0 (SPSS Inc, Chicago, Illinois). Significance was defined as 2-tailed p<0.05.

Results

Overall 792 RHC-echocardiogram pairs were identified. PASP_ECHO could not be estimated in 174 of these studies (22.0%). Inability to estimate PASP_ECHO was highly associated with mild or absent tricuspid regurgitation (OR=9.1, p=0.008). Only one patient with moderate or greater tricuspid regurgitation did not have PASP_ECHO reported. In total, 618 study pairs were available for analysis. Of these study pairs, 46.3% were performed on the same day and 82.8% within 1 day. Of the patients that had non-same day studies, 30.1% had echocardiograms prior to RHC and 69.9% had their echocardiograms subsequent to RHC. Summary hemodynamic and echocardiographic data are presented in Table 1.

Table 1.

Summary cohort hemodynamic and echocardiographic data (n=618)

Variable
Systolic blood pressure (mmHg)	129.2 ± 22.2
Dystolic blood pressure (mmHg)	80.3 ± 14.3
RHC derived parameters
Systolic pulmonary artery pressure (mmHg)	38.1 ± 13.4
Diastolic pulmonary artery pressure (mmHg)	16.4 ± 7.8
Mean pulmonary artery pressure (mmHg)	24.9 ± 9.2
Tricuspid regurgitation gradient (mmHg)	29.2 ± 11.3
Pulmonary capillary wedge pressure (mmHg)	15.9 ± 8.0
Pulmonary vascular resistance (wood units)	1.6 (1.0,2.4)
Mean right atrial pressure (mmHg)	8.9 ± 6.5
Cardiac output (L/min)	5.6 ± 1.8
Cardiac index (L/min/m^2)	2.8 ± 0.8
Pulmonary artery systolic pressure < 30 mmHg	177 (28.6%)
Pulmonary artery systolic pressure 30–49.9 mmHg	281 (45.5%)
Pulmonary artery systolic pressure 45–59.9 mmHg	106 (17.2%)
Pulmonary artery systolic pressure >60 mmHg	54 (8.7%)
Echocardiographically derived parameters
Pulmonary artery systolic pressure (mmHg)	41.0 ± 11.8
Mean right atrial pressure (mmHg)	11.6 ± 4.2
Ejection fraction (%)	62 (40,70)
Moderate to severe tricuspid regurgitation	160 (25.9 %)
Tricuspid regurgitation gradient (mmHg)	29.3 ± 10.7
Pulmonary artery systolic pressure < 30 mmHg	85(13.8%)
Pulmonary artery systolic pressure 30–49.9 mmHg	332 (53.7%)
Pulmonary artery systolic pressure 45–59.9 mmHg	150 (24.3%)
Pulmonary artery systolic pressure >60 mmHg	51 (8.3%)

Cardiac output and index were determined by the Fick method. Values represent mean ± standard deviation, median with interquartile range, or n (percent).

Despite a moderate strength correlation, there was relatively poor agreement between the two modalities (Figure 1, Table 2). Using each individual patient as their own reference, mean difference between echo and RHC PASP was 10.0 ± 7.9 mmHg and mean percentage difference was 30.2 ± 29.6%. The proportion of PASP_ECHO estimates discordant with PASP_RHC at different percentage discrepancy cutoffs is illustrated in Figure 2 and Table 3. Echocardiography tended to overestimate PASP with 62.6% of values overestimated and 37.4% underestimated (p<0.001). The tendency to overestimate increased as the percentage discrepancy cutoff increased (Table 3).

Figure 1.

Proportion of non-invasive pulmonary artery systolic pressure estimates discordant to invasively determined pulmonary artery systolic pressure by various different error cutoffs.

Table 2.

Various statistics describing the agreement between invasive and non-invasively determined pulmonary artery systolic pressure

Sub-analysis Category	n	Mean Error	r	p Value	Discordant PASP Clinical Category	Odds Ratio	p Value	Bias	95% Limits of Agreement
Total cohort	618	30.20%	0.52	<0.001*	54.40%	-	-	9.0%	−53.2% to 71.2%
Tricuspid regurgitation gradient	618	30.20%	0.55	<0.001*	-	-	-	1.1%	−70.5% to 72.1%
Right atrial pressure estimatable	428	26.30%	0.57	<0.001*	51.60%	0.7	0.04*	4.5%	−56.3% to 65.3%
Same day studies	286	30.10%	0.59	<0.001*	50.70%	0.8	0.09	9.6%	−51.9% to 71.0%
Outpatients	206	31.50%	0.59	<0.001*	53.70%	1.0	0.82	11.3%	−50.4% to 73.0%
Echocardiography after right heart catheterization	432	30.50%	0.53	<0.001*	53.00%	1.2	0.30	9.6%	−53.9% to 73.1%
Body surface area ≤ 2	325	32.70%	0.53	<0.001*	51.40%	0.8	0.12	12.5%	−50.6% to 75.6%
Moderate and severe tricuspid regurgitation	160	28.60%	0.59	<0.001*	55.00%	0.9	0.58	10.9%	−46.6% to 68.4%
Moderate tricuspid regurgitation	130	30.10%	0.58	<0.001*	55.40%	1.1	0.79	12.3%	45.7% to 70.3%
Severe tricuspid regurgitation	30	22.70%	0.63	<0.001*	40.00%	0.5	0.10	4.7%	−50.1% to 59.6%
Combined variable	98	26.70%	0.61	<0.001*	55.30%	1.0	0.90	8.0%	−49.3% to 65.4%

PASP: Pulmonary artery systolic pressure. PASP clinical categories were defined as: normal (≤ 30 mmHg), mildly elevated (31–45 mmHg), moderately elevated (46–60 mmHg), and severely elevated (>60 mmHg). Odds ratio describes the odds of clinical category misclassification between the subgroup and the remainder of the cohort excluded from the subgroup. Bias and 95% limits of agreement calculated with Bland-Altman method and r value represents Pearson correlation between invasively and non-invasively determined PASP within the subgroup.

^*represents significant p Value.

Figure 2.

Bland-Altman plot of echocardiographically estimated and invasively measured pulmonary artery systolic pressures. Dashed lines represent 95% limits of agreement.

Table 3.

Degree of over and under estimation at various percentage discrepancy values

Percentage Discrepancy	Pulmonary Artery Systolic Pressure			Tricuspid Regurgitation Gradient
	Total with Discrepancy	Overestimate	Underestimate	Total with Discrepancy	Overestimate	Underestimate
5%	533 (86.2%)	335 (62.9%)	198 (37.1%)	540 (87.4%)	284 (52.6%)	256 (47.4%)
10%	466 (75.4%)	303 (65.0%)	163 (35.0%)	464 (75.1%)	243 (52.4%)	221 (47.6%)
20%	336 (54.4%)	241 (71.7%)	95 (28.3%)	335 (54.2%)	188 (56.1%)	147 (43.9%)
30%	233 (37.7%)	184 (79.0%)	49 (21.0%)	232 (37.5%)	147 (63.4%)	85 (36.6%)
40%	165 (26.7%)	141 (85.5%)	24 (14.5%)	158 (25.6%)	109 (69.0%)	49 (31.0%)
50%	119 (19.3%)	111 (93.3%)	8 (6.7%)	102 (16.5%)	79 (77.5%)	23 (22.5%)

Values represent n (percent).

To evaluate the relevance of these discrepancies to clinical practice, the propensity for PASP_ECHO to misclassify PASP clinical categories was examined. More often than not, patients were misclassified by at least one clinical category and 8.7% were misclassified by ≥ 2 categories (Figure 3). While there was significant misclassification of patients both above and below the invasively determined PASP clinical category, the overall trend was misclassification into a higher rather than lower PASP clinical category (p<0.001) (Figure 3). The inability of echocardiography to correctly identify patients with normal PASP was particularly evident with a positive predictive value of 75.4% and a negative predictive value of 54.1%. Only 26.0% of patients with a normal PASP_RHC were correctly classified as having a normal PASP_ECHO. In this group, the mean PASP_ECHO was 38.3 +/− 6.3 mmHg vs. 24.18 +/− 3.7 mmHg for RHC. Of these patients, 83.2% were classified as mildly elevated and 16.0% as moderately elevated. Raising the definition of an abnormal PASP_ECHO to 40 mmHg improved the positive predictive value to 85.5% but decreased the negative predictive value to 42.2%. Additionally, 45.9% of patients classified as having a normal PASP_ECHO had elevated PASP_RHC. Given that the recommended hemodynamic definition for pulmonary artery hypertension employs mean pulmonary artery pressure rather than systolic pressure, we undertook an analysis based on invasively determined mean pulmonary artery pressure (1). Of the patients with a normal PASP_ECHO, 23.8% had a mean pulmonary artery pressure ≥ 25 mmHg by RHC.

Figure 3.

Figure 3A/B. Pie chart representation of PASP clinical category misclassification. Figure 3A represents number of PASP clinical categories misclassified in the total cohort. Figure 3B represents a breakdown of studies with overestimated and underestimated PASP clinical categories. PASP: Pulmonary artery systolic pressure.

There was a net overestimation of right atrial pressure by echocardiography (RHC = 8.9 +/− 6.5 mmHg vs. echo = 11.6 +/− 4.2 mmHg) (bias = 2.7 mmHg, 95% limits of agreement = −15.4 to 10.0 mmHg). Strength of correlation was small to moderate (r=0.32, p<0.001). Removal of studies in which right atrial pressure could not be estimated (and thus had been assumed 14 mmHg, n=190) improved the correlation between right atrial pressure values somewhat (r=0.40, p<0.001), however, this had limited effect on the agreement between PASP values or the degree of PASP clinical category misclassification (Table 2). To remove the influence of right atrial pressure estimation completely, invasive right atrial pressure was subtracted from invasive PASP to and compared to the tricuspid regurgitation gradient calculated by echocardiography. The overall level of agreement, however, remained similar (Table 2). The primary influence of removal of echocardiographically-estimated right atrial pressure was to decrease the bias toward overestimation, particularly at smaller error cutoffs (Table 3).

To address the issue that the time lapse between studies may have caused the disagreement between modalities (due to interval changes in volume status or drug therapy), subgroup analysis of same day studies were performed (n=286). Excluding non-same day studies provided only marginal improvement to the agreement between modalities (Table 2).

To explore the concept that the rapidly changing physiology of a hospitalized patient may have contributed to the discrepancy, subgroup analysis of outpatients was done, again with limited improvement in agreement (Table 2). If echocardiography was performed before RHC, diuresis or addition of drug therapy during the time between studies would be expected to lead to overestimation by echocardiography. To address this concern, subgroup analysis of patients with echocardiography performed subsequent to RHC was done, again with minimal improvement in agreement between studies (Table 2).

Larger degrees of tricuspid regurgitation have been reported to correlate with tricuspid regurgitation spectral envelope quality (4). Additionally, given the strong association between ≤ mild tricuspid regurgitation and an inability to estimate PASP in this cohort, subgroup analysis of patients with various degrees of tricuspid regurgitation severity was performed, however, this did not substantially improve the agreement (Table 2). Analyzing only patients with severe tricuspid regurgitation decreased the clinical category misclassification rate by 14.4%, however 95.2% of the population needed to be excluded to achieve this improvement (Table 2). Additionally, we investigated the hypothesis that patients with a large body habitus may have limited ultrasound windows and thus be more likely to have poor Doppler signal quality and/or poor IVC visualization leading to inaccurate estimates. Excluding patients with a body surface area (BSA) ≥ 2.0 m² did not improve agreement (Table 2). Interestingly, the mean error was actually reduced in the group with a BSA ≥ 2.0 m².

Given that the source of error is likely multifactorial and no single analysis could substantially improve the agreement, we employed a combined approach limiting analysis to variables that had the largest trends toward improvement (severe tricuspid regurgitation was not included secondary to the small number of patients). Similar to the univariate exclusions, limiting analysis to outpatients with ≥ moderate tricuspid regurgitation with echocardiography performed on the same day as RHC and a right atrial pressure that could be estimated, produced only a small improvement in agreement (Table 2). Using multivariate regression analysis including inpatient vs. outpatient status, severity of tricuspid regurgitation, time interval between studies, body surface area, temporal relationship between studies, and echocardiographic estimation of right atrial pressure only explained 3.2% of the total variability in the model (r²=0.32, p=0.003). The only independently significant component of the model was body surface area. Removal of this variable from the model reduced r² to 0.8% (p=0.08).

Discussion

The principal finding in this large series of consecutive RHC-echocardiogram study pairs, obtained during routine patient care conditions, is the frequent occurrence of clinically significant discrepancies between echocardiographically and invasively determined PASP. Using RHC as the gold standard, echocardiography misclassified PASP clinical categories in the majority of patients. The ability of echocardiography to correctly classify normal vs. elevated PASP was particularly poor, falsely categorizing over two thirds of patients with normal PASP as elevated. A predominant source of discrepancy could not be identified and correction for several potential sources of error failed to meaningfully improve the agreement between modalities.

Initial enthusiasm for PASP_ECHO was largely driven by several small, single center studies (5–7). These studies yielded excellent operating characteristics for PASP_ECHO, however, these results were obtained in a research setting, conditions likely difficult to replicate in a busy clinical environment. Additionally, although excellent correlation coefficients were reported in the aforementioned studies, correlation is not an ideal method for the comparison of tests given that a pair of tests can have excellent correlation but poor agreement (8). Overall, reported correlation coefficients in this literature have been highly variable ranging from r=0.31to r=0.99(5,6,9–18). Recently several smaller reports, in selected groups of patients, have demonstrated reasonably good correlation but frequent misclassification of patient groups by PASP_ECHO (4,19–21). The current work adds to this literature as it is the first large series compiled from consecutive unselected patients referred for RHC. As such, it may more accurately reflect general clinical practice.

PASP_ECHO has been advocated as the initial test to identify patients with elevated pulmonary pressure (1,22). It has also been recommended that elevated PASP be confirmed via RHC (1). This strategy relies on the assumption of a high positive predictive value for PASP_ECHO to prevent unnecessary invasive procedures, however, the positive predictive value in this population was only 75.4%. In addition to the potential morbidity and financial burden associated with unnecessary right heart catheterization, a large number of unwarranted subspecialty consultations are likely obtained since the majority of echocardiograms are ordered by generalists (23). Additionally, a high negative predictive value is necessary to prevent missed diagnosis. In our cohort the negative predictive value was only 54.1%, similar to results obtainable by chance. Approximately one quarter of patients with normal PASP_ECHO actually had an invasively measured mean PA pressure ≥ 25 mmHg, a threshold employed in most definitions of pulmonary artery hypertension (1). Given the poor prognosis of untreated pulmonary artery hypertension, timely diagnosis is critical (24). We additionally investigated the operating characteristics of defining an abnormal PASP_ECHO as ≥ 40 mmHg. This successfully increased the positive predictive value to 85.5% but decreased the negative predictive value even further to 42.2%.

Despite efforts to control multiple factors, we were unable to find a factor that substantially reduced the disagreement between PASP_ECHO and PASP_RHC. Univariate adjustment to account for right atrial pressure estimation, non-simultaneous nature of the studies, inpatient vs. outpatient status, body habitus, or degree of tricuspid regurgitation (a surrogate for Doppler quality) did not make a meaningful improvement in the agreement (4). Multivariate analysis using the above variables was only able to account for 3.2% of the variability in the discrepancy between modalities. Multiple other sources of error such as non-coaxial interrogation of the tricuspid regurgitation jet, measurement of the tricuspid valve closure artifact, RR irregularity, potential inaccuracy of the modified Bernoulli equation, contamination of the Doppler signal by aortic stenosis, and measurement of post-extrasystolic potentiation beats are likely operative but not amenable to investigation in this dataset. The painstaking attention to detail, frequently employed in the setting of a research study, can likely overcome many of the above sources of error. However, in a busy clinical setting these errors likely accumulate since PASP_ECHO estimation is only one item on a long list of echocardiographic objectives that the sonogropher and echocardiographer are charged to complete.

This study is subject to the limitations inherent to retrospective data collection, however, the large sample size and sequential nature of study selection offsets some of these limitations. Additionally the intention of the study was to avoid the research environment and study what is actually achieved in clinical practice. The non-simultaneous nature of the studies is not ideal, however adjusting for time elapsed between studies did not improve the agreement significantly. Additionally, a recent report with less than 1 hour between studies had very similar results (21). Incorporation of clinical information other than echocardiographic and RHC-derived hemodynamic variables may provide additional insight into the potential sources of error. Due to technical limitations in 22% of studies PASP was not reported, however, this frequency is consistent with that reported in other series (11,12,25). Inclusion of these studies would have likely led to larger discrepancies between studies. The single center nature of this study limits generalization. However, given that this study represents the experience of a high volume center with >20,000 echocardiographic studies interpreted yearly, exclusively by level III trained board certified echocardiographers, it is unlikely that our results represent a substantial outlier among echocardiography laboratories. Use of echocardiographic contrast has been reported to increase the accuracy of this modality, however, difficult to implement into the routine imaging protocol (26). Additionally, patients with normal PASP_ECHO would be less likely to be referred to RHC for further evaluation of pulmonary pressures. The phenomenon of regression toward the mean would predict that these patients would have a lower PASP on subsequent measurements, regardless of the modality employed. Finally, it is possible part of the discrepancy between modalities is secondary to error in RHC measurements and these errors have been well described (27). However, RHC is the modality recommended by experts and used in clinical practice as the gold standard for the measurement of pulmonary artery pressures (1,22).