A Multi-Institutional Analysis From Standardized Clinical Assessment and Management Plans (SCAMPs®), the Pediatric Health Information Systems Database, and the National Ambulatory Medical Care Survey

Abstract

We conducted a study to assess test characteristics of red-flag criteria for identifying cardiac-disease-causing-chest-pain and technical-charges of low-probability referrals. Accuracy of red-flag criteria was ascertained through study of chest pain SCAMPs® data. Patients were divided into: Group1 (concerning clinical elements) and Group2 (without). We compared incidence of cardiac-disease-causing-chest-pain between these two-groups. Technical-charges of Group2 were analyzed using PHIS Database. Potential savings for the US population was estimated using NAMSC. 52% of subjects formed Group1. Cardiac-disease-causing-chest-pain was identified in 8/1,656 (0.48%). No heart disease was identified in patients in Group2 (p=0.03). Applying red-flags in determining need for referral identified patients with cardiac-disease-causing-chest-pain with 100% sensitivity. Median technical-charges for Group2, over a 4-year-period, were US2014$775,559. Eliminating cardiac testing of low-probability referrals would save US2014$3,775,182 in technical charges annually. Red-flag criteria were an effective screen for children with chest pain. Eliminating cardiac testing in children without red-flags for referral has significant technical-charge savings.

Background

In the medical home model, primary pediatricians are the first contact for triaging complaints and for identifying which patients need to be seen by medical specialists ^1,2. Appropriately triaging emotion-laden complaints, such as chest pain, can be problematic. Chest pain is a common complaint, but it is very infrequently caused by a cardiac etiology ^3,4. To date there have not been effective empirical criteria for identifying which children should be referred for cardiac evaluation. Therefore, referral is at the discretion of the primary care physician, and thus is prone to practice variation. Previous studies have demonstrated that pediatricians feel under-educated about the issue of subspecialty referral ⁵, and some pediatricians feel pressured to make unwarranted subspecialty referrals related solely on parent concerns ^5,6.

We conducted a retrospective study to determine if clinical indicators (red-flags) identified a sub-population of children with chest pain, in whom cardiac disease explained their chest pain. We also used data from a national outpatient survey to measure the incidence of low-probability referrals, and combined this with administrative data to measure the magnitude of technical charges from resultant cardiac testing of these referrals.

Methods:

Data Sources:

Standardized Clinical Assessment and Management Plans (SCAMPs®) is a quality improvement (QI) tool, the goals of which are to reduce practice variation, reduce unnecessary utilization of resources and improve patient care and outcomes ⁷. A SCAMPs® for the evaluation of chest pain in a pediatric cardiology clinic was designed and first implemented by Boston Children’s Hospital and later expanded to include New England Congenital Cardiology Association (NECCA) (11-hospitals) as well as Children’s National Health System (CNHS) (1 hospital in metropolitan Washington, DC area). For each patient a SCAMPs® data form (SDF) was completed by the cardiologist at the time of the first outpatient encounter. Data collected on the SDF included details of the chest pain, relationship to exercise, associated cardiac symptoms, past medical history and family history including sudden unexplained death and cardiomyopathy. Physical examination characteristics were documented as were details of the electrocardiogram (ECG). The SDF highlighted red-flags for the cardiologist based on the patient data with the suggestion that patients meeting specific criteria undergo an echocardiogram. (Box 1) Indications for echocardiography were based upon the SCAMPs® methodology previously described ⁸. Cardiologists were permitted to diverge from suggested next steps and were encouraged to provide the reason. Both the findings of and reasons for ordering the echocardiogram (as well as other testing) were documented. The final diagnosis as determined by the cardiologist was captured in the SDF. The accuracy of this has been demonstrated in previous studies ⁹. The data were submitted to and analyzed by the Institute for Relevant Clinical Data Analytics in Boston, MA. The institutional review board (IRB) of participating sites reviewed the SCAMPs® and determined that it did not represent human subjects research rather a quality improvement. Subsequently, the current project was reviewed by the IRB at CNHS for purpose of publication and was determined to be exempt from review.

Box 1. Items representing red-flag for referrals.

Patient History

• Chest pain with exertion

• Exertional syncope

• Chest pain that radiates to back, jaw, left arm, or left shoulder

• Chest pain that increases with supine position

• Chest pain temporally associated with fever (>38.4oC)

Past Medical History^*

• Hypercoagulable state

• Arthritis/Vasculitis

• Immobilization

Family History

• Sudden unexplained death

• Cardiomyopathy

• Hypercoagulable state

Physical Examination

• RR> 40

• Temperature > 38.4° C

• Ill-appearing

• Painful/swollen extremities

• Non-innocent murmur

• Distant heart sounds

• Gallop

• Pulmonic component of S2

• Pericardial friction rub

• Peripheral edema

* Chest pain SCAMPs® excluded patients with known cardiac disease which otherwise would have represented a red-flag for referral.

The Pediatric Health Information System (PHIS) database is an administrative pediatric database that includes clinical and resource utilization data for 45 not-for-profit, tertiary care pediatric hospitals in the United State ¹⁰. These hospitals are affiliated with the Children’s Hospital Association (CHA), Overland Park, KS. The data warehouse function for the PHIS database is managed by Truven Health Analytics (Ann Arbor, MI) ¹⁰. Each participating hospital submits their charge master with the accompanying charges. The individual charge codes are mapped to a common system (Clinical Transaction Classification [CTC]) to allow comparative analysis. For example, each hospital’s unique charge code for ECG is mapped to CTC code 511101 that enables comparing the charges for ECGs among all participating hospitals. The population examined included patients seen from January 1, 2014 to December 31, 2014. We arbitrarily choose this interval as representative. Therefore charges are expressed in United States year 2014 dollars (US2014$). We generated PHIS reports for selected charge codes (CTC code), at the de-identified patient level to calculate total charges and charge/day per charge code. Volume of cases per item ranged from 143 (Exercise Stress Test – CTC 511115) to 72,782 (ECG – CTC 511101).

The National Ambulatory Medical Care Survey (NAMCS) is a national probability sample survey of visits to office-based physicians conducted by the National Center for Health Statistics, Centers for Disease Control and Prevention ¹¹. To quantify the magnitude of referrals in children presenting with chest pain, the NAMCS organization provided the dataset for the year 2012 (most recent data reported) for following diagnosis codes which were recorded as one of the three reasons for the visit: 1050.0 Chest pain and related symptoms; 1050.1 Chest pain, soreness; 1050.2 Chest discomfort, pressure, tightness, heaviness; 1050.3 Burning sensation in the chest; 1265.0 Heart pain. From that larger dataset, we extracted a subset of data with the following criteria; the total number of patients between 7 and 21 years of age and presenting with chest pain. Though not coincident with the SCAMPs® data, we do not anticipate significant changes in referral patterns for chest pain between 2012 and 2014.

Study population:

The study population and their data were extracted from the third iteration analysis of the Chest Pain SCAMP. All children and adolescents from 7 to 21 years of age with chest pain as the primary complaint or a significant part of the presenting complaint at the participating sites were included. Patients with previously diagnosed heart disease were excluded.

Data Analysis:

Analysis was divided into two parts. In the first analysis, a retrospective cohort study was performed to evaluate the test characteristics of red-flag criteria for chest pain referral. The second part of the analysis combined data from the first analysis with charge data from the PHIS and incidence data from NAMCS to estimate the magnitude of economic impact from these low probability referrals.

For the first analysis, criteria to identify higher risk referrals were identified. These were concerning elements in the patients’ history, past medical history, family history and/or examination based on the SCAMPs® methodology ⁸. These red-flag criteria were used to divide the study population between those with concerning elements (positive red-flag criteria (or higher risk)) (Group1) and those without (Group 2). The risk of cardiac disease as the etiology of chest pain and number of incidental cardiac findings were then compared between the two groups. Test characteristics (sensitivity, specificity, positive predictive value, and negative predictive value) of these criteria were calculated.

Given that these red-flag criteria correctly identified a sub-population with no cardiac disease, we sought to estimate the economic impact of these low probability referrals. This is the second component of the analysis. Using the original cohort of chest pain SCAMPs® cohort, we measured the proportion of patients undergoing ECG, echocardiograms, cardiopulmonary exercise tests, Holter monitor, event monitor, loop monitors, and cardiac magnetic resonance imaging in the low probability SCAMPs® group (negative red-flag criteria; Group 2) (between 2010–2014). Dividing the total number for each diagnostic test by the total number of patients in the cohort generates the probability of each diagnostic test per visit. Next, we used data from PHIS database to calculate the median technical charge for each commonly performed cardiac diagnostic test, i.e. charge per test. Finally, we estimated the annual number of low probability chest pain visits in the United States. Using the annual estimated incidence of referral for chest pain from NAMCS and assuming that the proportion of low probability referrals was identical to that seen in the SCAMPs® cohort we generated an annual volume of low probability visits. Multiplying these three factors together generated a total charge burden of tests in patients with a very low probability of having heart disease (probability of diagnostic test per visits x charge per test x annual volume = estimated technical charge savings).

There were several limitations to this methodology. Professional charges could not be and were not measured. Cost to families both in terms of direct cost and lost productivity were not included in this analysis, but both of these represent additional savings. No attempt was made to convert charges to costs. Hospital specific cost to charge ratios for individual services are not available in PHIS. However, by using median charges potential bias from variation in pricing is reduced, and though the final result is expressed as a charge (which is potentially inflated from the true cost) it does reflect the scale of the potential savings.

Statistical analysis

Descriptive statistics were reported with mean, standard deviation, frequency, and percentage, as appropriate. Fisher’s exact tests were used to compare percentages between groups. Statistical significance was achieved with a two-sided p-value <0.05 and analyses were performed with SAS v9.4 (SAS Institute, Cary, NC).

Results:

Between June 2010 and April 2014, 3167 patients were evaluated at outpatient cardiology clinics across 13 academic and community sites. The mean age at initial visit was 13.3±3.4 years. A cardiac cause of chest pain was identified in 8/3167 (0.25%, [95% CI: 0.11%- 0.50%]) patients. Within this cohort, (52.3% (n=1656) had concerning clinical data elements (Group 1) and 47.7% (1511) had no red-flags (Group 2). (Figure 1)

Figure 1.

Patient distribution

Resource utilization differed between the two groups (Table 1). Echocardiograms were obtained in 1283/1656 (77.5%) patients in Group 1 and 361/1511 (23.4%) patients in Group 2 (p<0.001). Among echocardiograms with results available, an incidental finding was noted in 86/1202 (7.2%) patients in Group 1 and 35/329 (10.6%) patients in Group 2 (p=0.05) but none of the findings were thought to be related to chest pain. A cardiac cause for chest pain was identified in 8/1,656 (0.48%, [0.21%, 0.95%]) patients in Group 1 and 0/1511 (0%, [0%, 0.32%]) patients in Group 2 (p=0.02). Therefore, in this study sample, the presence of any red-flag identified subjects with a cardiac cause of chest pain with 100% sensitivity, 48% specificity, 0.5% positive predictive value and 100% negative predictive value.

Table 1: Diagnostic test frequency by each group.

CT computerized tomography, HDL high density lipoprotein

	Group 1 (n=1656)	Group 2 (n=1511)	p
Echocardiograms	1283 (77.5%)	361 (23.4%)	<0.001
Stress echocardiogram	5 (0.3%)	0 (0.0%)	0.06
Sedated echocardiogram	1 (0.1%)	0 (0.0%)	0.99
Chest roentgenograms	148 (8.9%)	132 (8.7%)	0.85
Electrocardiogram	1562 (94.3%)	1379 (91.3%)	0.75
Exercise stress test	115 (6.9%)	18 (1.2%)	<0.0001
Holter monitor	70 (4.2%)	46 (3.0%)	0.09
Event monitor	38 (2.3%)	49 (3.2%)	0.13
Loop monitor	0 (0.0%)	5 (0.3%)	0.02
Fasting lipid profile	13 (0.8%)	4 (0.3%)	0.05
Non-fasting total cholesterol & HDL	2 (0.1%)	1 (0.1%)	0.99
Cardiac magnetic resonance imaging	5 (0.3%)	4 (0.3%)	0.99
Thyroid stimulating hormone	3 (0.2%)	3 (0.2%)	0.99
Other blood test (complete blood count, liver function tests, etc.)	5 (0.3%)	0 (0.0%)	0.06
Urinalysis	1 (0.1%)	1 (0.1%)	0.99
Renal ultrasound	1 (0.1%)	2 (0.1%)	0.61
Glucose	0 (0.0%)	2 (0.1%)	0.23
Electrolytes	2 (0.1%)	1 (0.1%)	0.99
C-Reactive protein	3 (0.2%)	0 (0.0%)	0.25
Metabolic cart	2 (0.1%)	0 (0.0%)	0.50
Ambulatory blood pressure monitoring	2 (0.1%)	0 (0.0%)	0.50
Nuclear Stress Test (Myocardial Perfusion (MIBI))	2 (0.1%)	0 (0.0%)	0.50
Cardiac CT Angiography	1 (0.1%)	0 (0.0%)	0.99

The distribution of technical charges for each diagnostic test performed in Group 2 is summarized in Table 2. To extrapolate technical charge for low probability referrals in the SCAMPs® cohort, we measured the usage of each cardiac test in Group 2 visits and combined it with the median technical charge for that service. For example a non-congenital echocardiogram was obtained in 294 (19.5%) patients with a median charge of US2014$665. Over the 1,511 low probability referral visits in the 4 years of SCAMPs® data, this would result in technical charges of US2014$195,610 for non-congenital echocardiograms. The cumulative technical charges for these low risk referrals are plotted in Figure 2.Over the almost four years of the implementation of the SCAMPs®, the cumulative technical charges for these visits are US2014$775,560.

Table 2: Technical charges for diagnostic tests for hospitals in PHIS database (2014).

MRI: Magnetic resonance imaging

	Charge per Test (US2014$)
	Minimal	1st Quartile	Median	3rd Quartile	Maximum
Electrocardiogram	59	213	260	332	722
Echocardiogram (congenital)	927	1,731	2,765	3,525	5,318
Echocardiogram (non-congenital)	148	411	665	873	2,507
Exercise stress test	309	521	1,147	1,480	2,298
Holter monitor	214	612	795	1,129	2,001
Event monitor	241	615	890	1,238	2,102
Loop monitor	218	934	1,484	2,962	23,332
Cardiac MRI with contrast	1,391	3,230	4,281	5,366	9,281

Figure 2.

Testing rate and total technical charges in the chest pain patients lacking referral criteria

To measure the technical charge burden of this referral pattern across the country, we used the NAMCS database to estimate the total number of office visits for chest pain. In 2012, NAMCS identified 621,785 office based pediatric visits for chest pain, representing 1.33% of all encounters in the United States for children aged 7 to 21 years. Primary pediatricians refer an average of 1 patient of every 40 consecutive visits to subspecialists ⁶, which represents approximately 15,545 referrals to cardiology per year. Extrapolating from the data from SCAMPs® we estimate that 7415 (47.7%) would not meet any medical red-flag criteria for referrals yearly and would have extremely low probability of cardiac disease. Assuming that cardiologists participating in SCAMPs® QI initiative are representative, the cumulative charges for low probability visits are US2014$3,775,182 annually. Based on the 3rd quartile, the annual technical charge savings would be US2014$4,968,974. (Figure 2)

Discussion

In this study, we demonstrate that red-flag criteria successfully identify a portion of the chest pain referrals that have very low probability of cardiac disease. However, despite low clinical risk of cardiac disease, these referrals result in a significant charge burden. Implementation of red-flag criteria to identify children who have very low probability of cardiac disease, would result in reduction in unnecessary testing and referrals, with a reduction in charges of US2014$ 3,775,182 per year. These red flag criteria are exceedingly sensitive, and may also serve to help alleviate anxiety in both families and primary care practitioners, since they are extremely effective at “ruling out” cardiac chest pain.

The SCAMPs® methodology gives primary care providers evidence-based guidelines to better assess the child who presents with chest pain rather than immediately sending the child to a cardiology consultations. Based on the results of our analysis, the potential healthcare charge savings would be substantial when there is a more focused and data-driven model to identify those patients who may require a cardiology consult for their chest pain.

Previous publications have addressed the problems surrounding early consultation and subspecialty referral. This is a global issue and has recently been recognized by the American Board of Pediatrics as an essential “entrustable professional activity”, one in which residents must demonstrate proficiency by the end of their training ^5,12. Thus, new residency curriculum and continuing medical education activities should focus on empowering primary care pediatricians to recognize red-flags for referrals as well as effectively communicate and negotiate with families surrounding emotionally laden complaints and requests for subspecialty referral ^5,13. The authors of this paper support the development and implementation of quality improvement initiatives at the primary care level. The goals of the quality improvement initiative are to help reduce unnecessary testing and referrals, improve communication skills and ultimately improve patient care. Similar initiatives have been successful in the past ¹⁴.

Chest pain in children can be anxiety-provoking for families. A disconnect between family perception of underlying cause and actual medical diagnosis in this population is not uncommon ³. When handled appropriately in the pediatric office, the influence of parental anxiety on decision-making could be reduced with parental education and reassurance. Parents can be assured knowing that while chest pain is not an uncommon finding in children, it is very rare to have a cardiac origin. Forrest et al. reported that approximately 1 in 6 pediatric referrals involved parental request for specialty care ⁶. More importantly, Lee and colleagues noted anxiety and depression were often evident in children presenting with non-cardiac chest pain and somatization in their parents ¹⁵. Studies have found that 20.5 % to 47% of children with non-cardiac chest pain have associated panic disorder ^16,17. Thus, the challenge for the primary pediatrician is to manage the underlying driving force that contributes to referrals in patients who do not have underlying cardiac pathology.

The benefit of red-flag criteria is not restricted to primary care pediatricians. In this study, cardiologists pursued additional testing in the cohort without red-flags. Though resource utilization was less intense than in the group with red-flags, cardiac testing was used frequently. For example, 23% of subjects with low risk for cardiac chest pain underwent echocardiography. None of these studies yielded a cardiac cause for chest pain and while incidental cardiac findings were identified in a small number, none led to immediate intervention. Parental anxiety/preference was the reason for ordering this test in 14% of cases, suggesting that some communication gap in reassurance exists between the cardiologist and the family. Although cardiologists utilized more testing in the high risk group (positive red-flag group), cardiac testing was still utilized in the low risk group, reflecting the significant anxiety that this chief complaint induces. This practice not only represents unnecessary expenditures but also sends the wrong message to the referring pediatricians and individual families. The importance of successful family counseling on the part of cardiologists and pediatricians is critical to avoid inappropriate testing and referral.

Limitations:

The SCAMPs® data form, which was filled out by the cardiologist, relied on information provided by parents and children and did not include direct information from primary pediatricians. The lack of communication between primary pediatricians and subspecialists has been noted previously ¹⁸. However, our SCAMPs® data form was robust and validated in a large pediatric outpatient population ⁸; therefore, it is reasonable to conclude that all aspects of care related to complaints of chest pain were captured. In this report, we did not perform long-term follow up on those patients with no discernible cardiology problem which is consistent with current clinical practice. Saleeb et al. found that no patient diagnosed with non-cardiac chest pain discharged from the cardiology clinic died as a result of a cardiac condition (nearly 18,000 patient years) ⁹. The cardiologist is expected to make the final decision whether the patient has cardiac disease or needs exercise restrictions. SCAMPs® captured the current practice and cardiologists were encouraged to use their clinical judgment and diverge from the algorithm if they felt it was necessary. Since our report intercepted patients at the level of the cardiologist, the charge savings might have been underestimated as we did not account for any urgent care/emergency department visits that might have happened prior to the cardiology visit. In this report, we used the reported average referral pattern of primary pediatricians. That might have underestimated the potential charge savings as complaints like chest pain can be anxiety provoking and referrals might occur at a higher ratio. Since the PHIS database only includes technical charges, our overall estimate of charge savings omitted charges related to professional billing for visits and test interpretation. One might suggest that referring patients with chest pain who lack red-flags to cardiologists might be less costly than having the pediatricians order the cardiac testing, i,e, ECG, echocardiogram, etc. In this report, in the population with no red-flags, none of the cardiac tests identified a cardiac cause of chest pain and no patient required immediate cardiac intervention. Therefore, in the absence of red-flags, patients need not be referred to cardiology nor do they need additional cardiac testing.

As mentioned, no measurement of professional costs was made. Also, charging practices by hospital vary significantly. We did not have the ability to convert charges to costs, but used median charge as an estimate (admittedly inflated) of the true cost of services. PHIS is a representative sample of primary pediatric hospitals but charging practices may not be representative of other types of institutions. Further research in more generalizable samples would be interesting. There will still be costs for managing the care of the patient in the pediatric office, but these are likely the same since the cardiology visit is completely unnecessary.

Conclusions:

As is demonstrated in this study, cardiac disease causing chest pain in children is very rare. Furthermore, it is safe to conclude that pediatric patients with chest pain without red-flags for referral do not have cardiac disease as an explanation for their symptoms. Therefore, these low risk patients can be managed successfully by their pediatric providers. Pediatricians and cardiologists need to recognize these red-flags, while at the same time being prepared to allay parental anxiety which will result in reduction of unnecessary referrals and testing.

Abbreviations:

CHA

Children’s Hospital Association

NAMCS

National Ambulatory Medical Care Survey

NECCA

New England Congenital Cardiology Association

PHIS

Pediatric Health Information System

SCAMPs®

Standardized Clinical Assessment and Management Plans

SDF

SCAMPs® data form