Appropriate Use Criteria for Paediatric Echocardiography in an Outpatient Practice: A Validation Study

Abstract

Background:

Although transthoracic echocardiography is the dominant imaging modality in congenital heart disease, optimal utilization is unclear. We assessed if adherence to the paediatric appropriate use criteria for outpatient transthoracic echocardiography could reduce inappropriate use without missing significant cardiac disease.

Methods:

Using the appropriate use criteria, we determined the indication and appropriateness rating for each initial echocardiogram performed at our institution during calendar year 2014 (N=1383). Chart review documented ordering provider training, patient demographics, and study result, classified as normal, abnormal, or abnormal motivating treatment within a two year follow up period. We tested whether provider training level or patient age correlated with echocardiographic findings or appropriateness rating.

Results:

We found that 83.9% of echocardiograms were normal and that 66.7% had an appropriate indication. Nearly all abnormal results and all results motivating treatment were in appropriate studies, giving an odds ratio for an abnormal result if an appropriate indication was present of 2.73 (95% confidence interval 1.92 to 3.89, p<.001). None of the remaining initial abnormal results with less than appropriate indications became significant resulting in treatment over two years. Results suggest a potential reduction in imaging volume of as much as 33% with application of the criteria. Cardiologists ordered nearly all studies resulting in treatment but also more echocardiograms with less appropriate indications. Most examinations were in older patients; however, most abnormal results were in patients younger than one year.

Conclusions:

The appropriate use criteria can be utilized to safely reduce echocardiography volume while still detecting significant heart disease.

Introduction

Paediatric transthoracic echocardiography volume has increased as paediatric echocardiography has become more available and image acquisition standardized.^1,2 The improved availability, perceived negligible risk, and practicality that make echocardiography a popular diagnostic tool also promote misuse and represent an opportunity for reducing unnecessary and expensive diagnostic studies. ^3,4,5 There was no official evidence-based practice guideline for the ordering of paediatric transthoracic echocardiograms until the recent publication of the appropriate use criteria for initial outpatient paediatric transthoracic echocardiography.^4,5 This research assesses the impact of applying appropriate use criteria to the ordering of echocardiograms in an outpatient practice.

Appropriate use criteria for adult transthoracic echocardiography first appeared ten years ago and have improved test ordering behaviour.^3,6,7 Validation of the adult criteria has provided critical insight into how appropriate use criteria impact ordering behaviour, influence detection of clinically relevant findings, change clinical care, and reduce cost.^3,6-9 The recently published paediatric appropriate use criteria are undergoing the first steps of validation as implementation projects and educational interventions have already occurred.^10-14 As paediatric providers increasingly look to the appropriate use criteria to guide their ordering behaviour, our validation study evaluates how adherence to the criteria could be used to reduce inappropriate transthoracic echocardiography volume, detect clinically relevant heart disease, and alter clinical practice.

Materials and Methods

We performed an Institutional Review Board approved retrospective chart review of the institutional electronic medical record to identify all patients less than or equal to 18 years having an initial transthoracic echocardiogram during calendar year 2014. We chose this period to allow two years of follow up from the time of the initial transthoracic echocardiogram to ascertain whether clinically significant cardiac disease resulting in treatment was diagnosed or missed. Extracted data included date of birth, age at time of study, sex, study indication as specified by the provider in the electronic order and office visit note, outpatient study location, and date of study. International classification of disease coding was not used to determine study indication. Patients were not included in the study if another institution had already performed a transthoracic echocardiogram.

We used ordering provider identity to classify providers as either general paediatrician or specialist and if specialist, whether trained in paediatric cardiology. Using the appropriate use criteria document, we matched each echocardiographic indication to the corresponding appropriate use criteria rating. When multiple indications were present for the same study, we used the indication with the highest appropriate use criteria rating. We added the physical finding of systolic click as an appropriate indication since the authors of the original appropriate use criteria document have subsequently suggested this modification.⁴

We then reviewed the echocardiographic report findings and classified them as normal or abnormal based on a previously published objective grading system⁴. Normal results required no intervention, cardiology follow up, or change in management. Normal results included incidental findings such as patent foramen ovale and aortic arch branching variants not associated with vascular ring. Abnormal transthoracic echocardiographic findings were further categorized as abnormal motivating treatment if they were followed by interventional cardiac catheterization, cardiac surgery, ablation, device implantation, or treatment with a cardiac medication for ventricular dysfunction or pulmonary over-circulation within two years after the initial echocardiogram.

The primary outcome assessed was the proportion of abnormal and abnormal motivating treatment studies with regards to appropriate use criteria rating and ordering provider training. Additional outcomes were the diagnostic yield based on ordering provider type and appropriate use criteria rating consistent with prior literature.^3,10-14 Results were stratified based on patient age: less than 3 months, from 3 months to 11 months, 1 to 4 years, 5 to 9 years, and greater than 9 years.

Descriptive statistics were calculated for all variables of interest and included count, median, standard deviations, and interquartile range when appropriate. We calculated the odds ratios and their 95% confidence intervals to compare the proportion of abnormal transthoracic echocardiogram findings at varying appropriateness level and used the Chi-square test to compare proportions when analysing categorical values (considered statistically significant when p value ≤ 0.05).

Results

There were 1383 patients (median age 11 years, interquartile range 7-17 years) of whom 44.6% were female. Test results were normal in 1161 patients (83.9%), abnormal in 222 patients (16.1%), and abnormal motivating treatment in 27 patients (1.9%) (Figure 1). Appropriateness rating was appropriate in 927 studies (67.0%), may be appropriate in 229 (16.6%), and rarely appropriate in 227 exams (16.4%). We could not assign an appropriate use criteria indication to only 24 patients due to insufficient clinical information and did not include them. All these studies were ordered by general paediatricians and had normal results.

Figure 1.

Proportion of both abnormal and normal echocardiograms in each appropriateness category out of 1383 total studies.

The most common appropriate use criteria categories were systemic disorders, murmur, chest pain, and family history (Table 1). Overall, the most common indications were suspected connective tissue disorder (n=100), pathologic murmur (n=97), exertional chest pain (n=93), innocent murmur (n=86), and systemic hypertension (n=75). For studies deemed appropriate, suspected connective tissue disease (n=100), pathologic murmur (n=97), and exertional chest pain (n=93) were the most common indications. For may be appropriate indications, chest wall deformities and scoliosis pre-operatively (n=57), chromosomal abnormality with undefined risk of cardiovascular disease (n=37), and family history of congenital left-sided lesion (n=27) were most common. Among rarely appropriate indications, innocent murmur (n=86), palpitations with benign history, electrocardiogram, and no other signs or symptoms (n=24), and syncope with benign family history, normal electrocardiogram, and no other signs or symptoms (n=20) were most common. We found that 38 of the appropriate use criteria were not used once in our population.

Table 1:

Echocardiographic results in each Appropriate Use Criteria category.

Appropriate Use Criteria Category	Total (No.)	Abnormal, No (% total)	Abnormal Motivating Treatment No. (% total)
All echocardiograms	1383	222 (16.1%)	27 (1.9%)
Palpitations	73	1 (1%)	0
Syncope	90	15 (16%)	4 (4%)
Chest Pain	177	21 (11.9%)	2 (1.1%)
Murmur	229	73 (31.9%)	10 (4.4%)
Other Signs and Symptoms	101	9 (8.9%)	0
Prior Test Results	139	20 (14.4%)	3 (2.2%)
Systemic Disorders	392	66 (16.8%)	6 (1.5%)
Family History	173	15 (8.7%)	1 (0.6%)
Outpatient Neonates	9	2 (22%)	1 (11%)

Upon application of the appropriate use criteria to one year of outpatient initial transthoracic echocardiograms, we found that a large majority of echocardiographic results were normal (83.9%) and that a large majority of examinations with abnormal echocardiographic results were in studies classified as appropriate (81.1%). The odds ratio for a transthoracic echocardiogram with an appropriate indication to have an abnormal result compared to less appropriate indications was 2.73 (95% confidence interval 1.92 to 3.89, p<.001). Among the echocardiograms with appropriate indications, 80.6% had normal findings (Figure 1). The proportion of abnormal echocardiographic results was not statistically different between rarely appropriate or may be appropriate indications (p=.971); however, the proportion of abnormal studies was lower for may be appropriate and rarely appropriate indications as compared to appropriate indications (p<.001 for both). All 27 abnormal motivating treatment results were in appropriate studies. In this group, eleven patients had surgery, eight had interventional catheterizations, five were started on afterload reducing agents for ventricular dysfunction, and three patients died from non-cardiac causes prior to anticipated interventions. There were an additional six patients under the age of two years with at least moderate atrial septal defects seen on their initial study. All had echocardiograms with appropriate indications, but none had an intervention as it is routine practice to delay device or surgical closure of such defects to an age that would be missed by our follow up period.

The most common abnormal echocardiographic findings were left ventricular inflow/outflow anomalies, left ventricular functional or structural anomalies, and aortic dilation (Table 2). When further stratified into abnormal motiving treatment findings, a different pattern emerged with patent ductus arteriosus, left ventricular functional or structural anomalies, and coronary artery anomalies being the most common. Only 42 abnormal echocardiograms, 18.9% of our abnormal results and none of our abnormal motivating treatment results, were in indications less than appropriate (Table 3). After two years of follow up within this group, 29 had unchanged findings on repeat transthoracic echocardiography and the remainder were discharged from cardiac care. The most common findings in this group of 42 patients were mild aortic root dilation (n=10), bicuspid aortic valve without stenosis or regurgitation (n=6), mild pulmonary stenosis (n=6), mild mitral valve prolapse (n=6), tiny patent ductus arteriosus without left-sided chamber dilation (n=3), and moderate secundum atrial septal defect (n=3). In eight of the 29 patients, the echocardiographic indication was a chest wall deformity or scoliosis preoperatively. Their echocardiogram showed either mild aortic dilation, mitral valve prolapse, or both in all eight. Furthermore, there were only 21 abnormal echocardiograms, 9.5% of our abnormal results, and no abnormal motivating treatment results in studies with rarely appropriate indications. Ten of these patients have been subsequently discharged from cardiac care, and the remainder have findings relatively common in the general population such as bicommissural aortic valve.

Table 2:

Abnormal echocardiographic findings.

Echocardiographic Finding	Abnormal (n=222)	Abnormal Motivating Treatment (n=27)
Ventricular septal defect	22	1
Atrial septal defect	14	2
Pulmonary venous anomalies	5	2
Patent ductus arteriosus	12	7
Aortic arch hypoplasia/coarctation	3	1
Aortic dilation/aortopathy	29	0
Coronary artery anomalies	13	5
Right ventricular inflow/outflow anomalies	19	2
Left ventricular inflow/outflow anomalies	60	0
Left ventricular functional anomalies	39	7
Other	6	0

Table 3:

Echocardiographic result and appropriateness rating according to patient age.

	Total	<3 mo.	3-11 mo.	1-4 yrs	5-9 yrs	>9 yrs
Patient No. (%)	1383	74 (5.3%)	62 (4.5%)	238 (17.2%)	296 (21.4%)	713 (51.6%)
Appropriate	927	40	38	146	222	481
• Normal	747	17	29	115	187	399
• Abnormal	180	23	9	31	35	82
• Abnormal with Treatment	27	4	3	4	2	14
May be Appropriate	229	5	10	44	38	132
• Normal	208	5	9	42	32	120
• Abnormal	21	0	1	2	6	12
• Abnormal with Treatment	0	0	0	0	0	0
Rarely Appropriate	227	29	14	48	36	100
• Normal	206	24	11	45	32	94
• Abnormal	21	5	3	3	4	6
• Abnormal with Treatment	0	0	0	0	0	0

Most patients were greater than 9 years of age (51.6%) with a similar age distribution seen in normal studies (Table 3). In abnormal studies, there was a higher proportion of patients less than 3 months of age. Because of these divergent trends, the odds ratio of having an abnormal transthoracic echocardiogram result in a patient younger than one-year relative to over one-year was 2.08 (p<.001, confidence interval 1.56 to 2.77). Studies deemed appropriate or may be appropriate closely followed the age distribution in our overall study population (Table 3). Rarely appropriate studies were more common in patients less than 3 months of age (p<.001) and less common in patients greater than 10 years (p=.026).

We also stratified our data based on provider type (Figure 2). There were no differences in patient age or sex between cardiologists and non-cardiologists. Cardiologists ordered proportionally less may be appropriate studies (p<.001) but more rarely appropriate studies (p<.001) than their non-cardiologist counterparts. In terms of echocardiographic results, there was a significantly higher proportion of abnormal studies resulting in treatment that were ordered by cardiologists (p=.006), but the proportion of abnormal studies was nearly identical. Similar trends persist upon further stratification of the non-cardiologist group into general paediatrician and paediatric specialist. Paediatric specialists ordered significantly more may be appropriate studies (p=.013) and less rarely appropriate studies (p=.013) than their general paediatrician colleagues.

Figure 2.

Appropriate Use Criteria rating and echocardiographic result according to ordering provider type.

Discussion

Modelling application of appropriate use criteria to a population of outpatients undergoing initial transthoracic echocardiography, we found that 83.9% of echocardiograms were normal and that 81.1% of abnormal findings were in studies classified as appropriate. Along with the odds ratio of 2.73, this pattern suggests potential for a targeted reduction in imaging volume of as much as 33% by using the appropriate use criteria document. To better understand this potential, we further defined which categories were high yield as well as what abnormal findings were seen in less appropriate indications.

The highest yield appropriate use criteria categories for abnormal echocardiographic results were syncope, murmur, systemic disorders, and outpatient neonates (Table 1). Our overall yield of abnormal echocardiograms was very similar to that seen in several recent studies.^{10, 11, 14} The indications of exertional syncope, exertional chest pain, and pathologic murmur alone accounted for 55.6% of our abnormal results motivating treatment. When combined with the indication of muscular dystrophy which accounted for another 18.5% of abnormal motivating treatment findings, these four criteria comprised nearly three-quarters of our patients with an echocardiographic finding that resulted in treatment in our two year follow up window.

For indications related to chest pain and syncope, the presence of exertional symptoms accounted for nearly all patients who had treatment due to an abnormal finding on echocardiography. This pattern and diagnostic yield for abnormal results motiving treatment correlates with that seen in recent literature discussing echocardiograms for syncope and chest pain.^11,16,17 Our yield for the indication of murmur was also similar to other studies; however, our yield for any abnormal result for the indications of chest pain and syncope was higher.¹¹ Because the assigned severity of an abnormal echocardiographic result in our study was related to whether an intervention occurred within two years, our reported yields are not entirely comparable to other studies that used different or no follow up periods.^11,14,16,18 Still, our yield for indications related to chest pain or syncope are similar to other reported yields when considering that only those findings thought to be causing the presenting symptoms resulted in an intervention in our study (Table 1). The 4% yield for syncope and 1.1% for chest pain that resulted in treatment were related to findings such as anomalous origin of the right coronary artery and severe non-compaction. The remainder of the abnormal findings found on studies with indications related to chest pain or syncope were incidental in nature and not related to symptoms.

Results suggest minimal risk of missing clinically significant disease with application of the appropriate use criteria. Only 42 patients had an abnormal transthoracic echocardiogram with an indication less than appropriate, and 13 were discharged from ongoing cardiology care after a subsequent normal transthoracic echocardiogram. Further, only 21 patients with a rarely appropriate indication had an abnormal result with 10 being discharged on subsequent follow up. In none of these patients did treatment follow diagnosis suggesting that the findings were clinically less significant. Moreover, none of the echocardiographic findings in these patients progressed in severity over two years. Additionally, the most common less than appropriate indication in patients with abnormal echocardiographic results still requiring cardiology care was a chest wall deformity or scoliosis preoperatively. This highlights the importance of considering connective tissue disease evaluation in patients with pectus excavatum and scoliosis, but does not necessarily suggest that these indications should be ranked as appropriate instead of may be appropriate.

All 27 patients with abnormal results motivating treatment had appropriate indications with 22 having indications with the highest rating of nine. Therefore, with strict application of the criteria, no abnormal findings resulting in treatment over a reasonable two year follow up period would be missed by only performing echocardiograms on patients with appropriate indications. In this scenario, transthoracic echocardiography volume would be reduced by 33.0% or 456 studies. Using the payment for a congenital transthoracic echocardiogram of $594 from the 2014 Medicare hospital ambulatory payment classification system, such a reduction would decrease payments by $270,864 over the study period¹⁹. This conservative total does not include professional physician reimbursement. Even with a suggested reduction of echocardiography volume by one third, the remaining patients with appropriate transthoracic echocardiograms would have 80.6% normal results. Our results suggest that within this group a priority should be placed on those patients with exertional symptoms, pathologic findings on physical exam, significant cardiac family history, and systemic disorders with a high risk of cardiac involvement.

About 90% of our transthoracic echocardiography volume was in patients over one-year old; however, those patients under one year of age were statistically more likely to have an abnormal result. This may be explained by symptomatic complaints driving echocardiography volume in older patients, rather than concerning physical exam findings as in neonates and infants.^20-22 Other than symptoms associated with exertion, symptom driven echocardiograms within all age groups in our population had low diagnostic yield. Concerning findings on physical exam, on the other hand, were high yield in all age groups. Taken together, our results show the importance of pathologic findings on physical exam at any age and that most of the reduction in unnecessary transthoracic echocardiography volume would occur in patients greater than one.

As in other studies, cardiologists in our study were more likely to be the ordering provider for those patients with an echocardiographic finding resulting in treatment.²⁴ Cardiologists in our study though were also more likely to order echocardiograms for rarely appropriate indications. This trend may be related to referral pressure for obtaining a transthoracic echocardiogram or simply the relative ease of obtaining an echocardiogram as a cardiologist.^25-27 Paediatric subspecialists other than cardiologists also tend to order transthoracic echocardiograms for diagnoses with a known risk of cardiac involvement, all indications rated as appropriate by the appropriate use criteria. This may explain why ordering behaviour was so similar. As nearly 70% of our study population had transthoracic echocardiograms ordered by a cardiologist with many of the remaining studies ordered by a paediatric subspecialist other than a cardiologist, we are limited in our ability to make strong conclusions about transthoracic echocardiograms ordered by general paediatricians. Our results support the great need for continuing educational interventions both for cardiologists and general pediatricians.²⁸ Additionally, our study supports the conclusion that when a non-cardiologist orders a transthoracic echocardiogram for an appropriate indication that concurrent referral to a cardiologist is reasonable due to the higher likelihood of having an abnormal result so that care may be streamlined.

Our study is limited to a single centre but likely reflects the experience of other paediatric cardiac centres with the caveat that our institution attracts patients with connective tissue disease. Still, transthoracic echocardiograms performed for any connective tissue related indication comprised less than 10% of our study volume. Our study period also overlaps by one month with the publication of the appropriate use criteria; however, there was unlikely a change in ordering provider practice within that brief time. Because not every outpatient has a transthoracic echocardiogram performed, we also do not know the true number of children with cardiac pathology that would have an abnormal echocardiogram. As such, any retrospective study such as ours is unable to determine the sensitivity and specificity of the appropriate use criteria or rates of abnormal findings within the general population. Our study is also unable to assess the benefit of a normal echocardiographic result regardless of appropriateness.

This study shows that the appropriate use criteria, with only a single modification for click heard on exam, has practical application to a typical outpatient population with an open access format paediatric echocardiography lab. This study further demonstrates that by following the rubric created by the appropriate use criteria, unnecessary transthoracic echocardiographic volume can indeed be substantially reduced without impacting the detection of clinically relevant cardiac disease. Further refinements of the appropriate use criteria to potentially remove low yielding criteria and increase the scoring of higher yielding criteria will be necessary in the future. Importantly, our findings highlight the critical nature of a thorough cardiovascular exam and history when evaluating patients with potential cardiac involvement. In an era of downward pressure on both medical expenditure and reimbursement with paradoxical increases in imaging volume and complexity, clinicians must rely not only on the appropriate use criteria but also on the foundations of clinical medicine to best serve their patients and responsibly use constrained medical resources.