Reliability of Echocardiographic Indicators of Pulmonary Vascular Disease in Preterm Infants at Risk for Bronchopulmonary Dysplasia

Abstract

Objectives

To determine the assessment and inter-rater reliability of echocardiographic evaluations of pulmonary vascular disease (PVD) in preterm infants at risk for bronchopulmonary dysplasia.

Study design

We prospectively studied echocardiograms from preterm infants (birthweights 500 – 1250g) at 7 days of age and 36 weeks postmenstrual age (PMA). Echocardiograms were assessed by both a cardiologist on clinical service and a single research cardiologist. Interpretations were reviewed for inclusion of determinants of PVD and assessed for inter-rater reliability using the Prevalence Adjusted Bias Adjusted Kappa Score.

Results

180 and 188 matching research and clinical echocardiogram reports were available for the 7 day and 36 week PMA studies. At least one of the specific qualitative measures of PVD was missing from 54% of the clinical reports. PVD was diagnosed at 7 days in 31% and 20% of research and clinical interpretations, respectively (kappa score of 0.54). At 36 weeks, PH was diagnosed in 15.6% and 17.8% of research and clinical interpretations, respectively (kappa score of 0.80).

Conclusions

Although all qualitative variables of PVD are not consistently provided in echocardiogram reports, the inter-rater reliability of cardiologists evaluating measures of PVD revealed strong agreement, especially at 36 weeks PMA. We speculate that establishment of a protocol for echocardiographic evaluation may improve the identification of PVD in preterm infants.

Preterm infants are at high risk for late respiratory morbidity and mortality due to the development of bronchopulmonary dysplasia (BPD), the chronic lung disease of preterm infants (1–3). Animal models reveal that early disruption of angiogenesis impairs alveolarization and causes sustained abnormalities of lung function, suggesting that pulmonary vascular disease (PVD) may contribute to the pathogenesis of BPD (4). Pulmonary hypertension (PH) is a clinical manifestation of PVD that is a significant complication in preterm infants who develop BPD, and is associated with significant morbidity and mortality, as evidenced by a 2-year mortality rate of 33–48% after PH diagnosis (5, 6). Recent data suggest the incidence of PH in extremely preterm infants is 17–34%, indicating that routine screening may be useful for identifying infants at increased risk (5, 7–11). Moreover, signs of early PVD have been associated with increased risk for BPD and late PH (12). Despite its impact on the clinical course of BPD, making the diagnosis of early PVD or PH is difficult because signs and symptoms are often subtle and similar to those of lung disease alone. The early identification of PVD or PH in preterm infants may provide an opportunity for implementation of preventative or treatment strategies to improve long term outcomes.

Although cardiac catheterization remains the gold standard for the diagnosis of PH, transthoracic echocardiography offers non-invasive, readily available testing (13). A commonly utilized objective echocardiographic measure of PH is the estimated right ventricular systolic pressure (RVSP), which is derived from the tricuspid regurgitant jet velocity (TRJV) and modified Bernoulli equation (TRJV² X4) (14). The pattern of shunting through a ventricular septal defect (VSD) or patent ductus arteriosus (PDA) also provides a semi-quantitative assessment of the resistance between the pulmonary and systemic vascular beds. Qualitative findings include septal wall flattening, right ventricular hypertrophy (RVH), right atrial dilation, and RV dilation, but these are not as reliable for determining PH severity, especially in the absence of a measurable TRJV (7). In a prior prospective study, RVSP could be estimated from TRJV in only 8% of preterm infants (12). Therefore, evaluation of PVD in this population routinely requires the application of qualitative echocardiographic measures.

The physiological decrease in pulmonary vascular resistance (PVR) after birth also complicates assessment for PVD. The optimal timing for this transition and the clinical impact of “delayed” or “incomplete” transition is not well understood. Findings of increased PVR, right ventricular hypertrophy, or right atrial dilation, may be physiologically normal and not evidence for PVD or PH. Moreover, qualitative measures of PVD may not be routinely included in the clinical echocardiogram report during the early postnatal period. Additionally, echocardiograms are often performed to evaluate for anatomic abnormalities and cardiac function, which may not direct the interpreting cardiologist to comment on each sign of PVD.

We hypothesized that subjective variables of PVD are not consistently communicated in clinical echocardiogram reports, but when reported, show good agreement between their interpretations in preterm infants at early (day 7) and late (36 weeks PMA) time points. As part of a prospective observational study to determine the incidence and risk factors for BPD in preterm infants (12), we evaluated the consistency of interpretation of measures of PVD in echocardiograms performed at 7 days of age and 36 weeks postmenstrual age (PMA) between a research cardiologist blinded to clinical course and the clinical echocardiography service.

Methods

All echocardiograms and data were prospectively obtained as part of an observational research study that enrolled subjects between July 2006 and March 2012 at 3 hospitals associated with The University of Colorado School of Medicine and 5 hospitals affiliated with Indiana University School of Medicine (12). The protocol was approved by the Institutional Review Boards at each site and written informed consent was obtained from a parent or guardian of all participants.

Enrollment criteria consisted of birth weight 500–1250 grams, gestational age less than 34 weeks PMA, and enrollment by seven days of age. Exclusion criteria included clinical evidence of congenital heart disease (except PDA, atrial septal defect (ASD) less than 1cm or ventricular septal defect (VSD) less than 2mm), lethal congenital abnormality and/or anticipated death.

Echocardiograms were performed at 7 days of age and at 36 weeks PMA using a standardized image acquisition protocol for each patient. All sonographers were trained on the protocol by the lead sonographer via teleconference with a PowerPoint presentation. As studies were reviewed centrally, feedback was provided to individual sonographers to improve image quality. Research interpretation for all echocardiograms was performed by a single cardiologist at Children’s Hospital Colorado blinded to the subjects’ clinical status. Measurements included TRJV, measurable dimensions of heart chambers, any detectable shunt lesions (patent foramen ovale (PFO), ASD, VSD, PDA) and direction of shunt flow. Estimates of RVSP were calculated using the modified Bernoulli equation with no allowance for right atrial pressure. If the spectral Doppler pattern of the tricuspid regurgitant jet did not provide a clearly defined and reproducible ascending limb, apex and descending limb, assessments of TRJV were not considered as suitable for determining RVSP for these studies.

Qualitative echocardiogram measures of PVD and increased right ventricular pressure were evaluated including right atrial enlargement, right ventricular dilation, RVH, ventricular septal wall flattening, and pulmonary artery dilation. Abnormalities in these measures were assessed by degree of severity: mild, moderate or severe. Septal wall flattening was defined as decreased septal curvature into the right ventricle at end systole. Septal wall flattening was considered moderate if the septum was completely flat (“D shaped”) and severe if there was reverse curvature into the left ventricle.

The echocardiograms were first interpreted in real-time by one of 18 local clinical cardiologists to provide results in a timely fashion to caregivers. Clinical echocardiogram reports were obtained retrospectively by review of medical records. We did not have access to the medical records from some of the affiliated hospitals to collect all of the clinical echocardiogram reports. The presence of any of the following variables were sufficient for the diagnosis of PH in our analyses at 36 weeks PMA: any degree of right ventricle hypertrophy, right ventricle dilation or septal wall flattening. In the absence of specific documentation of subjective PH variables, the following statements were accepted as negative findings: “normal right heart chamber size and right systolic function” and “no 2-dimensional evidence of right ventricle pressure or volume overload”.

Statistical Analyses

All data were managed in a REDCap (Research Electronic Data Capture) database hosted at the University of Colorado Denver Development and Informatics Service Center (15). The prevalence adjusted bias adjusted kappa score was used to assess agreement between cardiologists’ evaluation of echocardiogram evidence of PVD or PH (16). This measure accounts for the bias that occurs with the high or low prevalence of a given response and was calculated on each echocardiogram indicator. Kappa scores were interpreted using established methods (16)., and were interpreted as the following: <0 less than chance agreement; 0.01–0.20 slight agreement; 0.21–0.40 fair agreement; 0.41–0.60 moderate agreement; 0.61–0.80 substantial agreement; 0.81–0.99 almost perfect agreement (17).

Results

Three hundred sixteen infants were enrolled in the parent study between July 2006 and March 2012. Of the enrollees, 277 survived and had an echocardiogram performed at 36 weeks PMA (12). Clinical echocardiogram reports were not available for 66 infants who had research echocardiograms performed, leaving 211 infants with at least one set of matching research and clinical echocardiogram interpretations available for analysis. These included 188 paired interpretations for the 7 day echocardiograms and 180 paired interpretations for the 36 week echocardiograms. The discrepancy between infants enrolled and the available matching echocardiogram reports was due to an inability to access all clinical reports from affiliated institutes. Clinical characteristics of study patients are shown in Table I.

Table 1.

Subject Characteristics (n=277)

	Matched Clinical ECHO (n=211*)	No Matched Clinical ECHO (n=66)	p-value
	N (%) Median (IQR)	N (%) Median (IQR)
Birth Weight (g)	900 (737–1075)	939 (770 – 1070)	0.46
Birth Weight Strata (g)
500–749g (n=67)	669 (610–695)	640 (600 – 670)	0.14
750–999g (n=116)	870 (810–930)	885 (810 – 943)	0.54
1000–1250g (n=95)	1130(1078–1185)	1125 (1054 – 1210)	0.76
Gestational Age	27 (25–28)	27 (26 – 28)	0.23
Sex (Male)	103 (48.8)	32 (48.5)	0.96
Small for Gestational Age	21 (10)	8 (12.1)	0.62
Antenatal Corticosteroids	169 (80.1)	48 (72.7)	0.20
Cesarean Delivery	156 (73.9)	50 (65.8)	0.99
Maternal Complications
Pre-Existing Hypertension	27 (12.8)	4 (6.1)	0.18
Prolonged Rupture of Membranes	40 (19)	8 (12.1)	0.20
Chorioamnionitis	48(22.8)	8 (12.1)	0.06
Preeclampsia	57(27)	17 (25.8)	0.82
PDA-Medical Treatment	95(45)	23 (34.9)	1.00
PDA-Surgical Ligation	37(17.5)	7 (10.6)	0.07
Days of CPAP	12(5–24)	14 (7 – 29)	0.34
Required CPAP at 36 weeks PMA	13(6.2)	2 (3)	0.53
Days of MV	16 (5–41)	11 (4 – 37)	0.48
Required MV at 36 weeks PMA	14 (6.6)	6 (9.1)	0.50
Total Oxygen Days	81(46–106)	78 (47 – 98)	0.42
Discharged on Oxygen	128(60.7)	38 (57.6)	0.69
Length of stay (NICU)	92(74–116)	82 (72 – 108)	0.15
Mortality	4(1.9)	3 (4.6)	0.36

PDA: patent ductus arteriosis; CPAP: continuous positive airway pressure; MV: mechanical ventilation; NICU: neonatal intensive care unit

^*Total sample size for both time points equals 211. Sample size for each time point was determined independently basedon matching research and clinical echocardiograms.

In the clinical reports, at least one of the specific qualitative indices of PVD including RVH, RV dilation, and septal wall flattening was missing in 54% of 7 day reports and 53% of 36 week PMA reports. When missing variables from the clinical reports were assigned a negative value (i.e. “normal”), there was near perfect agreement between clinical and research interpretation of right atrial enlargement, RVH and RV dilation (Table II). Evaluation of septal wall flattening revealed kappa scores of 0.55 and 0.86, respectively, demonstrating moderate agreement on the 7-day echocardiograms and almost perfect agreement on the 36 week PMA echocardiograms. Of the clinical echocardiogram reports that did not specifically comment on septal flattening, 35% of 7 day echocardiograms and 3% of 36 week PMA echocardiograms were assessed to have septal wall flattening per the research cardiologist interpretation. TRJV was consistently reported in both clinical and research interpretations and was observed in 10% of 7-day echocardiograms and 7.8% of 36 week PMA echocardiograms, however there was a discrepancy in which patients were reported to have a TRJV.

Table 2.

Echocardiogram findings associated with pulmonary hypertension at 7 days of age and 36 weeks PMA.

	7 days of age (n=188)				36 weeks Post-Menstrual Age (n=180)
	Research Echo Incidence n (%)	Clinical Echo Incidence n (%)	Unreported in Clinical Echo n (%)	Kappa*	Research Echo Incidence n (%)	Clinical Echo Incidence n (%)	Unreported in Clinical Echo n (%)	Kappa*
Cardiac Level or Ductal Shunting	160 (85.1)	168 (89.4)	1(0.5)	0.79	159 (88.3)	152 (84.4)	2 (1.1)	0.74
PDA	68 (36.2)	71 (37.8)	6 (3.2)	0.95	22 (12.2)	27 (15)	15 (8.3)	0.86
ASD/PFO	154 (81.9)	157 (83.5)	6 (3.2)	0.73	140 (77.8)	140 (77.8)	9 (5)	0.80
VSD	5 (2.7)	6 (3.9)	10 (5.3)	0.99	6 (3.3)	7 (3.9)	13 (7.2)	0.99
RA Enlargement	0	2 (1.1)	2 (1.1)	0.98	9 (5)	5 (2.8)	3 (1.7)	0.91
RV Hypertrophy	3 (1.6)	4 (2.1)	48 (25.5)	0.97	5 (2.8)	6 (3.3)	46 (25.6)	0.92
RV Dilation	3 (1.6)	4 (2.1)	3 (1.6)	0.95	16 (8.9)	12 (6.7)	2 (1.1)	0.91
Septal Wall Flattening	59 (31.4)	35 (18.6)	102 (54.3)	0.55	21 (11.7)	24 (13.3)	96 (53.3)	0.86

^*All PABAK values were calculated assuming missing echocardiogram interpretations to be negative or “no”.

A PDA was observed in 68 (36.2%) research reports and 71 (37.8%) clinical reports of infants at 7 days of age, resulting in a kappa score of 0.95. Three (4.41%) infants with a PDA were found to have a bidirectional shunt, all of which had septal wall flattening. All other infants with PDA had left to right shunting; no infants had right to left shunting. Of infants evaluated at 7 days of age, 26/68 (38.2%) with a PDA had septal wall flattening, while 33/120 (27.5%) of infants without a PDA were found to have septal wall flattening (p=0.128).

At 36 weeks PMA, a PDA was reported in 22 (12.2%) of research studies and 27 (15%) of clinical studies, resulting in a kappa score of 0.86. One infant with PDA was observed to have bidirectional shunting with septal flattening. Of 21 infants with a PDA with left to right shunting, one also had septal flattening, while 20 did not (p=0.09). No PDA identified at 36 weeks was associated with right to left shunting. The majority of PDAs at 7 days (60%, 41/68) and at 36 weeks (77%, 17/22), were very small such that we were not able to obtain a reliable Doppler envelope. Therefore, velocity information was not available to improve quantification of pulmonary artery pressure.

At 36 weeks PMA, PH was diagnosed in 28 (15.6%) of research and 32 (17.8%) of clinical interpretations, resulting in a kappa score of 0.80, which indicates substantial agreement (Table III). Research and clinical cardiologists agreed on the diagnosis of PH based on subjective measurements in 27 patients at 36 weeks PMA. However, the research interpretation identified PH in 11 patients at 36 weeks PMA in which the clinical interpretations did not, whereas clinical echocardiogram interpretation identified PH in 7 patients in which the research interpretation did not. Differences in the assessment of septal wall flattening accounted for nearly all of this discrepancy.

Table 3.

Incidence of Pulmonary Hypertension and Inter-Rater Reliability

	Research Evaluation Incidence n(%)	Clinical Evaluation Incidence n(%)	Kappa*
7 Days of Age	59 (31.4)	38 (20.2)	0.54
36 Weeks PMA	28 (15.6)	32 (17.8)	0.80

^*Research and Clinical PH were determined to exist if Right Ventricular Hypertrophy,

Right Ventricular Dilation or Septal Wall Flattening were present. All PABAK values were calculated comparing Research PH and Clinical PH incidence values.

Discussion

To determine if echocardiographic findings of PVD are evaluated and reported in a consistent, systematic manner, we compared inter-observer variability of echocardiogram interpretation and reporting for clinical and research cardiologists. We found substantial agreement between cardiologists for the diagnosis of PH at 36 weeks PMA, with almost perfect agreement for many of the individual qualitative echocardiographic variables of PVD. Although the strength of agreement for individual qualitative echocardiographic variables of PVD on postnatal day 7 was not as strong, our findings suggest that early echocardiogram findings of PVD may provide a consistent metric for research studies, and perhaps clinical care. Additionally, we showed that clinical reports often lacked specific comment on a subset of qualitative PVD-related measurements. Assigning these missing variables to a “normal” value resulted in substantial agreement between echocardiogram interpretations for qualitative signs of PVD based on kappa scores. However, for septal wall flattening at 7 days of age, only a moderate agreement was observed with a kappa score of 0.55.

Our findings support the hypothesis that although each variable of PVD may not be included on clinical echocardiogram reports, when present, there is strong agreement between clinical and research interpretations. The reason for lack of inclusion in clinical reports is likely multifactorial. Because RVH, RV dilation and degree of septal wall flattening can be considered normal physiological findings during the transition to ex-utero circulation, specific comment on these indices may not have been thought necessary by the clinical cardiologists at 7 days of age. However, by protocol, the research cardiologist commented on all variables without regard to whether the finding could be explained as part of normal physiology. Similarly, the presence of a PDA, especially in the early perinatal period, may impact the cardiologist interpretation of septal wall flattening. As infants age, these factors have less impact on the interpretation of the degree of flattening. Indeed, in older children, septal wall flattening has been proven to be a sensitive marker for RV systolic hypertension (19). However, this measure is vulnerable to inaccuracy from slightly off-axis images, which may result in an artificially flattened septum and an inaccurate identification of the end-systole when the measurement is most sensitive. In a previous report of older of BPD infants with suspected or confirmed PH who underwent cardiac catheterization, septal wall flattening was reported to be 88% sensitive but only 33% specific for the diagnosis of PH compared with cardiac catheterization (7).

In the presence of echocardiographic findings that are strongly suggestive of PVD, such as severe end-systolic ventricular septal flattening, cataloging the many subjective indicators of PVD, such as mild RV dilation, may not be viewed as clinically warranted. Additionally, the lack of comment on specific findings of PVD in some clinical reports may be in part due to the ordered indication for the study. If the indication for the study was not provided or recorded, systematic assessment of all subjective indicators may not have been performed or reported.

A standard practice for the evaluation of PVD may alleviate inconsistencies in interpretation; however, specific guidelines for the use of echocardiography in screening and follow-up of preterm infants do not exist (5, 7, 10). Additionally, a clear statement of the indication for the echocardiogram is key to achieving accurate and complete communication. Qualitative signs of PVD are often used alone or in conjunction with TRJV for diagnosis of PH. However, TRJV alone should be used with caution, as its accuracy in evaluating RVSP decreases as the pressure increases, when compared directly with catheterization. This highlights the importance of using TRJV in conjunction with subjective variables of elevated right heart pressures when making clinical judgments (18). When these subtle subjective findings are the only indicators of possible PVD, their absence from the clinical echocardiogram report may lead to inaccurate or incomplete interpretation. In our study, this occurred in 24 cases at 36 weeks PMA. Therefore, we recommend the development and implementation of a standard protocol for the evaluation of PVD including quantitative values such as TRJV and PDA velocity when available, and subjective measures including septal wall flattening, right atrial dilation, RV dilation and RVH. Such tools will allow for focused, efficient and complete evaluation of PVD.

Evidence of septal wall flattening at 7 days of age is associated with increased risk for development and severity of BPD and development of late PH at 36 weeks PMA, and RV dilation at 7 days of age is associated with increased risk for PH at 36 weeks PMA (12). These findings suggest that echocardiographic signs of PVD at 7 days of age could be important prognostic signs for BPD and late PH. Based on these findings, we suggest these subjective variables of PVD should be completely evaluated in all early postnatal echocardiograms of preterm infants <1250 grams.

Several limitations to this study exist. First, we assumed that the absence of specific comments on findings such as septal wall flattening in clinical echocardiogram reports indicated a negative finding. As such, it is impossible to know whether the interpreting cardiologist acknowledged a negative finding but did not comment on it, or whether he/she did not evaluate the specific indices. Treating absent comments as negative findings may falsely increase level of agreement between cardiologists. Second, the presence of any subjective characteristic associated with PH was deemed sufficient for the diagnosis of PH for the purpose of our study. Because the findings may be independent of PH, the incidence of PH may have been overestimated. Additionally, we did not use PDA velocity, an objective measure of RVSP, in our evaluation of PVD. Additionally, the subjective evaluation of the right ventricle, including size, thickness and function, is challenging and a limitation of this study as both reviewers may be incorrect in their assessments. The right ventricle has a very complex geometry, thus the most reliable objective measure of its size and function is through 3D methodology such as cardiac MRI. Recent studies and advances in technology suggest that 3D echocardiographic evaluation of RV diastolic dimension and ejection fraction correlate with MRI findings. However, the appropriate 3D technology to perform this study was not available at all sites at the initiation of this study. Lastly, cardiac catheterization was not performed in the vast majority of patients, therefore it is not possible to determine with certainty if PH was indeed present or absent.

In conclusion, echocardiography is an often used, non-invasive measure of PVD in preterm infants with BPD. We found that agreement between clinical and research cardiologists’ evaluation of the subjective findings of PVD at 36 weeks PMA is strong, demonstrating substantial to near perfect agreement for most variables. However, qualitative indicators of PVD are not reported consistently, especially in early postnatal echocardiograms. The establishment of a protocol for echocardiogram interpretation of PVD may improve the consistency of reporting, and has the potential to provide a consistent metric for research studies in this population as well as to improve interdisciplinary communication and the clinical care of these high-risk infants.